Using MIST as a Self-Service Platform
The original version of this guide is available in the MIST documentation repository. Access to the repository is required.
Steps
- Prerequisites
- Configure CI/CD variables
- Request access to the MIST repository group
- Create project repositories
- Set up the Terraform repository
- Create virtual machines and optional DNS zones
- Run the Terraform pipeline
- Set up the Ansible repository
- Run the Ansible pipeline
Introduction
MIST is a self-service platform for managing Linux virtual machines in NREC. Infrastructure is defined in Git repositories and deployed through pipelines, using standardised modules. The platform currently supports AlmaLinux 10 virtual machines.
MIST is available to project teams as a self-service offering. This guide helps you use MIST independently within your project, from meeting the prerequisites and initial setup to day-to-day use. For more information about the platform, see mist.uib.no. Available standardised modules are listed in the MIST module catalogue.
Its goal is to enable you to provision and manage the MIST resources your project needs without having to open a separate ticket in hjelp.uib.no for every step. Clear responsibilities, applicable security requirements, and transparent operational procedures remain in place.
The following sections explain which access and information you need, how to configure MIST for your project, and whom to contact if you have questions or encounter issues.
Prerequisites
Before you begin, make sure that you have the following:
- An NREC project. You can request one through request.nrec.no.
- An NREC API key (OS_PASSWORD). If you have forgotten your API key, you can reset it by following the instructions in the NREC documentation.
- Access to the MIST repositories. Request access by sending an email to nrec-vb@uib.no.
- A Gitlab personal access token for your user with read and write permissions.
- An SSH key pair.
Configure CI/CD variables
Configure the required CI/CD variables in your project repository group in git.app.uib.no. Set these as group variables only, so that they are available to both the Ansible and Terraform repositories. Mark every variable indicated as masked and hidden accordingly.
| Variable | Required | Description | Configuration |
|---|---|---|---|
GITLAB_ACCESS_TOKEN | Yes | Personal access token for GitLab. | Masked and hidden |
OS_PASSWORD | Yes | Password for your NREC user. | Masked and hidden |
OS_USERNAME | Yes | NREC username, for example username@uib.no. | Masked and hidden |
OS_PROJECT_NAME | Yes | Name of your OpenStack project. | — |
SSH_PUB_KEY | Yes | Base64-encoded public SSH key. This key is added to the default user on the virtual machines. | — |
SSH_KEY | Yes | Base64-encoded private SSH key. Ansible uses this key to access the virtual machines. | Masked and hidden |
VAULT_PASSWORD | No | Password for group_vars/all/vault.yml. | Masked and hidden |
ANSIBLE_DIFF | No | Runs Ansible with --diff. | Set to true; unset or false runs without diff |
FORCE_DEPLOY | No | Starts deployment automatically on the main branch. | Set to true; otherwise deployment is manual |
You can create the base64 values for the SSH keys with the following commands:
base64 < sshkey.pub
base64 < sshkey
Copy the output of the first command to SSH_PUB_KEY and the output of the second command to SSH_KEY.
For the current CI/CD variable documentation, see the MIST pipeline documentation.
Request access to the MIST repository group
Request access to the Infrastructure as Code repository group. This group contains the MIST repositories and standardised modules needed to configure your project repositories and pipelines.
Create project repositories
In git.app.uib.no, create a repository group for your project. We recommend using the name of your NREC project for the group, but this is not required.
Within this group, create two repositories:
- An Ansible repository for configuration management and software deployment.
- A Terraform repository for defining and managing your NREC infrastructure.
We also recommend creating a third repository for project documentation. Keeping it separate from the infrastructure and configuration repositories makes project-specific operational information easier to maintain and share.
Set up the Terraform repository
You can use the central MIST Terraform pipeline in your Terraform repository. Create a .gitlab-ci.yml file with the following configuration:
include:
- project: sc/infrastructure-as-code/mist-pipelines/terraform
ref: 1.0.0
file: /nrec.yml
default:
tags: []
The central pipeline includes a runner that is not available through self-service access. Setting default.tags to an empty list removes this runner tag, allowing the pipeline to run with the runner available to your project.
Create a main.tf file in the Terraform repository with the following initial configuration:
terraform {
backend "http" {}
}
terraform {
required_version = ">= 1.0"
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
version = ">= 3.0.0"
}
}
}
resource "openstack_compute_keypair_v2" "ci_key_uib-it-vb-admin" {
name = "pipelinekey-uib-it-vb-admin"
public_key = var.ssh_pub_key
}
provider "openstack" {
region = var.nrec_region
}
Create a security.tf file to define the security group for your virtual machines. The SSH rules below allow connections from the UiB network and from the GitLab runners. The ssh_scrunner rule is required: it allows the GitLab runner that runs Ansible to connect to the virtual machines over SSH.
resource "openstack_networking_secgroup_v2" "defaultMNREC" {
name = "defaultMNREC"
description = "Allow ICMP and SSH from UiB"
}
resource "openstack_networking_secgroup_rule_v2" "icmp_rule" {
direction = "ingress"
ethertype = "IPv4"
protocol = "icmp"
remote_ip_prefix = "0.0.0.0/0"
security_group_id = openstack_networking_secgroup_v2.defaultMNREC.id
}
resource "openstack_networking_secgroup_rule_v2" "ssh_ipv4" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "129.177.0.0/16"
security_group_id = openstack_networking_secgroup_v2.defaultMNREC.id
}
resource "openstack_networking_secgroup_rule_v2" "ssh_git1" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "158.39.74.247/32"
security_group_id = openstack_networking_secgroup_v2.defaultMNREC.id
}
resource "openstack_networking_secgroup_rule_v2" "ssh_git2" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "158.37.64.190/32"
security_group_id = openstack_networking_secgroup_v2.defaultMNREC.id
}
resource "openstack_networking_secgroup_rule_v2" "ssh_scrunner" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "158.37.65.6/32"
security_group_id = openstack_networking_secgroup_v2.defaultMNREC.id
}
resource "openstack_networking_secgroup_rule_v2" "ssh_ipv6" {
direction = "ingress"
ethertype = "IPv6"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "2001:700:200::/48"
security_group_id = openstack_networking_secgroup_v2.defaultMNREC.id
}
Create a variables.tf file with the variables used by the Terraform configuration:
variable "nrec_region" {
description = "OpenStack Region"
type = string
}
variable "ssh_pub_key" {
description = "SSH public key for OpenStack instances"
type = string
}
Example virtual machine
The following example creates a virtual machine, a persistent volume, and public and internal DNS records. It is a template and must be adapted for your project. In particular, update the DNS zones and names, image ID, flavor, key pair, security groups, network, and volume size to match your requirements.
resource "openstack_dns_recordset_v2" "prjman_record" {
zone_id = openstack_dns_zone_v2.mist-uib-no_zone.id
name = "projects.mist.uib.no."
type = "A"
records = [openstack_compute_instance_v2.prjman.access_ip_v4]
ttl = 3600
description = "DNS entry prjman"
}
resource "openstack_dns_recordset_v2" "internalprjman_record" {
zone_id = openstack_dns_zone_v2.uib-it-vb-admin_zone.id
name = "projects.uib-it-vb-admin.uib.no."
type = "A"
records = [openstack_compute_instance_v2.prjman.access_ip_v4]
ttl = 3600
description = "DNS entry internal project"
}
resource "openstack_compute_instance_v2" "prjman" {
name = "prjman"
image_id = "36ab8e92-a967-42ef-b618-bc6abdf0b98d"
flavor_name = "m1.xlarge"
key_pair = openstack_compute_keypair_v2.ci_key_uib-it-vb-admin.name
security_groups = ["default", "defaultMNREC"]
network {
name = "dualStack"
}
depends_on = [
openstack_blockstorage_volume_v3.prjman_vol
]
}
resource "openstack_blockstorage_volume_v3" "prjman_vol" {
name = "prjman_vol"
size = "20"
description = "volume for prjman"
lifecycle {
prevent_destroy = true
}
}
resource "openstack_compute_volume_attach_v2" "attach_prjman" {
instance_id = openstack_compute_instance_v2.prjman.id
volume_id = openstack_blockstorage_volume_v3.prjman_vol.id
}
Optional: manage DNS zones with Terraform
If you want to manage DNS zones with Terraform, use the following as a starting point. Adapt the zone names, contact email, and descriptions to your project. The DNS record example above must reference the corresponding zone resources.
resource "openstack_dns_zone_v2" "uib-it-vb-admin_zone" {
name = "uib-it-vb-admin.uib.no."
email = "nrec-vb@uib.no"
ttl = 3600
description = "DNS-Zone uib-it-vb-admin"
type = "PRIMARY"
}
resource "openstack_dns_zone_v2" "mist-uib-no_zone" {
name = "mist.uib.no."
email = "nrec-vb@uib.no"
ttl = 3600
description = "DNS-Zone mist.uib.no"
type = "PRIMARY"
}
Run the Terraform pipeline
After you have completed the configuration and set the CI/CD variables described above, run the pipeline in your Terraform repository. The pipeline applies the Terraform configuration and creates the NREC resources defined in your repository.
Set up the Ansible repository
The following example shows a typical directory structure for an Ansible project repository. Adapt it to the roles and services used by your project.
.
├── ansible.cfg
├── files
│ ├── elabftw
│ │ ├── metadata.xml
│ │ └── metadata.xml.bkp
│ ├── proxy
│ │ ├── favicon.svg
│ │ └── index.html
│ └── rspace
│ └── opt
│ └── rspace
│ ├── configs
│ │ ├── apache.setup
│ │ ├── db-config.cnf
│ │ ├── server.xml
│ │ └── tomcat-locate.sh
│ └── templates
│ └── db-template.sql
├── group_vars
│ ├── all
│ │ ├── main.yml
│ │ └── vault.yml
│ ├── uib_ita_fd_eln_dev_app
│ │ └── main.yml
│ └── uib_ita_fd_eln_dev_proxy
│ └── main.yml
├── host_vars
│ ├── chemotion
│ │ └── main.yml
│ ├── elabftw
│ │ └── main.yml
│ ├── main.yml
│ └── rspace
│ └── main.yml
├── inventory
├── postchecks
│ ├── chemotion.gitlab-ci.yml
│ ├── elabftw.gitlab-ci.yml
│ ├── proxy.gitlab-ci.yml
│ └── rspace.gitlab-ci.yml
├── precheck.yml
├── README.md
├── requirements.yml
└── site.yml
Use the central MIST Ansible pipeline in your Ansible repository. Create a .gitlab-ci.yml file with the following configuration:
---
include:
- project: sc/infrastructure-as-code/mist-pipelines/ansible
ref: 1.0.0
file: /ansible.yml
default:
tags: []
The central pipeline defines a runner tag that is not available through self-service access. Setting default.tags to an empty list unsets this tag, so that the pipeline uses the shared runner available to your project.
Create an ansible.cfg file in the root of the Ansible repository:
[defaults]
inventory = inventory
host_key_checking = False
retry_files_enabled = False
stdout_callback = yaml
deprecation_warnings = False
roles_path = ./roles:./roles/galaxy:/etc/ansible/roles
pipelining = True
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible_facts
fact_caching_timeout = 86400
Create an inventory file in the root of the Ansible repository. The following is an example only: replace the inventory group names, virtual machine names, DNS names, and Ansible users with the values for your own virtual machines.
[uib_it_vb_admin_prod]
prometheus ansible_host=prometheus.uib-it-vb-admin.uib.no
content ansible_host=content.uib-it-vb-admin.uib.no
[uib_it_vb_admin_prod:vars]
ansible_user=rocky
ansible_python_interpreter=/usr/libexec/platform-python
[uib_it_vb_admin_alma10_prod]
mist ansible_host=mist.uib-it-vb-admin.uib.no
loki ansible_host=loki.uib-it-vb-admin.uib.no
grafana ansible_host=grafana.uib-it-vb-admin.uib.no
[uib_it_vb_admin_alma10_prod:vars]
ansible_user=almalinux
ansible_python_interpreter=/usr/bin/python3
[uib_it_vb_admin_alma10_test]
prjman ansible_host=projects.uib-it-vb-admin.uib.no
[uib_it_vb_admin_alma10_test:vars]
ansible_user=almalinux
Create a precheck.yml file in the root of the Ansible repository. This file is required by the central pipeline and runs the prechecks role against all hosts.
---
- name: Run Prechecks only
hosts: all
become: true
roles:
- role: prechecks
If your virtual machines do not use Puppet images, set the following variable in inventory variables instead of in precheck.yml. You can set it for all hosts in group_vars/all/main.yml, or set it in the relevant group_vars or host_vars file:
---
prechecks_puppet_image: false
Create a requirements.yml file to declare the Ansible collections and roles used by your project. Add only the roles your project needs. The following is an example:
---
collections:
- name: community.general
version: ">=8.0.0"
- name: ansible.posix
roles:
# standard roles
- name: hostfile
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-hostfile.git"
scm: git
version: "${ROLE_HOSTFILE_VERSION}"
- name: lvm
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-lvm.git"
scm: git
version: "${ROLE_LVM_VERSION}"
- name: prechecks
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-prechecks.git"
scm: git
version: "${ROLE_PRECHECKS_VERSION}"
- name: prometheus_node_exporter
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-prometheus_node_exporter.git"
scm: git
version: "${ROLE_PROMETHEUS_NODE_EXPORTER_VERSION}"
- name: promtail
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-promtail.git"
scm: git
version: "${ROLE_PROMTAIL_VERSION}"
- name: sudoadd
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-sudoadd.git"
scm: git
version: "${ROLE_SUDOADD_VERSION}"
- name: sysprep
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-sysprep.git"
scm: git
version: "${ROLE_SYSPREP_VERSION}"
- name: userlogins
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-userlogins.git"
scm: git
version: "${ROLE_USERLOGINS_VERSION}"
# NFS NETAPP
- name: msdefender
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-msdefender.git"
scm: git
version: "${ROLE_MSDEFENDER_VERSION}"
- name: nfsmount
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-nfsmount.git"
scm: git
version: "${ROLE_NFSMOUNT_VERSION}"
- name: nginx
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-nginx.git"
scm: git
version: "${ROLE_NGINX_VERSION}"
- name: docker
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-docker.git"
scm: git
version: "${ROLE_DOCKER_VERSION}"
- name: local_users_and_groups
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-local_users_and_groups.git"
scm: git
version: "${ROLE_LOCUSERSGROUPS_VERSION}"
- name: iptables
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-iptables.git"
scm: git
version: "${ROLE_IPTABLES_VERSION}"
- name: files_and_pkgs
src: "git+https://oauth2:${GITLAB_ACCESS_TOKEN}@git.app.uib.no/sc/infrastructure-as-code/shared-ansible-roles/ansible-role-files_and_pkgs.git"
scm: git
version: "${ROLE_FILESANDPKGS_VERSION}"
All available roles can be found in the shared Ansible roles repository.
Create a site.yml file to define which roles are applied to each inventory group. The following is a simple example; adapt the host groups and roles to your project, and make sure that every referenced role is listed in requirements.yml.
---
- name: Configure entire project
hosts: uib_clarino_repo
become: true
roles:
- role: sysprep
- role: hostfile
- role: prometheus_node_exporter
- role: promtail
- role: userlogins
- role: sudoadd
- name: Configure proxy server
hosts: proxy
become: true
roles:
- role: iptables
- role: lvm
- role: apache
- name: Configure backup server
hosts: backup
become: true
roles:
- role: lvm
- role: local_users_and_groups
- role: nfsmount
- role: files_and_pkgs
- role: sourcebuild
- role: cronjobs
Create group_vars files for settings shared by an inventory group, and host_vars files for settings that apply to individual hosts. Before configuring a role, read the README.md in that role's repository to understand the available options and their expected values. The MIST roles are documented in the shared Ansible roles repository.
Run the Ansible pipeline
After you have added and committed the Ansible configuration, run the pipeline in the Ansible repository. The pipeline installs the declared roles, validates the configuration, and runs the prechecks. After the checks have completed successfully, start the deployment job to configure your virtual machines. Set FORCE_DEPLOY to true if the deployment should start automatically on the main branch.
Support & Contact
- MIST Platform & Access Requests: Email
nrec-vb@uib.nofor access to MIST repositories or platform-specific questions. - NREC Cloud Infrastructure Support: Email
support@nrec.nofor underlying NREC OpenStack cloud platform issues. - Platform Info & Module Catalogue: mist.uib.no
- NREC Project Portal: request.nrec.no
- General UiB IT Support: Submit a ticket at hjelp.uib.no