Scyld Cloud Controller

The Scyld Cloud Controller is directly responsible for the creation and manipulation of VMs through OpenStack. This means defining VMs via Nova, creating their boot volumes in Cinder, and attaching public IP addresses as necessary for external access. The Cloud Controller maintains a mapping of users to VMs, and makes sure that only allowed users are able to access each VM.

The Cloud Controller also has APIs to manage users, groups, and storage volumes, though these are not always used depending on the deployment specifics. Each API resource is described in detail below.

server-instance

The server-instance resource is for manipulating VMs. In SCM, a server-instance is a VM, and a server-instance directly maps to a VM in OpenStack. Each VM defined in SCM receives a universally unique identifier (UUID) that uniquely identifies the VM.

The UUID used by SCM is not the same as the UUID used by OpenStack. This lets SCM present a persistent VM UUID to a user across multiple power state changes. When a VM is powered off via SCM, the OpenStack VM is actually deleted. All of the VM’s configuration information is stored on its boot disk, which is persistent. When the VM is powered back on via SCM, a brand new OpenStack VM is created, but booted from the pre-existing boot disk. In this way the VM appears persistent, but without SCM creating its own UUID the OpenStack UUID would change every time the VM is powered off.

server-flavor

The server-flavor resource retrieves the list of available flavors defined in OpenStack Nova. These flavors are listed in the Cloud Portal as instance-types, and represent the combination of RAM, CPU cores, and boot disk size that the VM will be allocated. This resource is read-only as presented by SCM. New flavors are not defined via the SCM API, but by OpenStack command-line interface (CLI) commands such as nova flavor-create.

server-image

The server-image resource retrieves a list of available images defined in OpenStack Glance. More accurately, the API returns the list of images that are available for use by the querying user. It is possible to make images in Glance private to a specific user. When this is the case, the superuser sees all images all the time, but a regular user only sees the images that they are allowed access to. This is also a read-only API. New images are defined/uploaded directly using OpenStack CLI commands such as glance image-create.

user

The user resource is used to establish a user account with the Cloud Controller. This is necessary in order to create VMs using the Cloud Controller because SCM uses OpenStack’s Keystone identity management system. When a user is created in the Cloud Controller, the Cloud Controller creates a user with the same name in Keystone.

Unless a user is using the APIs directly, user creation is a step that is automated by the Cloud Portal. The ultimate result of user creation within the Cloud Controller depends on the back-end user driver configured, which is chosen based on whether LDAP/AD integration is enabled, whether NIS is to be used to make users available to a cluster, or whether local user entries are to be made.

In the case of LDAP/AD integration, creating a user in the Cloud Controller only results in the creation of a user in Keystone. Additionally, the user is associated with the corresponding Cloud Auth user.

When local user creation is enabled, including NIS, creating a user in the Cloud Controller results in a call to useradd, such that the user is created in the appropriate /etc/passwd file. Using NIS, this entry can then be made globally available to other nodes, such as NFS storage servers, and ClusterWare head nodes and compute nodes.

group

The group resource is used to create groups in the local /etc/group file. This only makes sense when using the local user and NIS backends, and the group modification APIs are disabled when LDAP/AD integration is enabled.

The Cloud Controller only allows users in the same account owner domain to be in the same group. For example, if Account Owner A has Managed Users B and C, then only users A, B, and C can be added to the group. Account Owner D would not be allowed to be added to the group.

storage-volume

The storage-volume resource is designed to be used in conjunction with HPC clusters, such that a created storage-volume maps directly to a user’s $HOME directory within the cluster. If user $HOME directories do not need to be created or managed by SCM, this API can be disabled.

When enabled, having an active storage-volume is a prerequisite for VM creation. The rationale behind this is that a user cannot do any meaningful work without a $HOME directory. Storage-volume creation is often the most custom part of an SCM deployment. Many storage environments exist, and the steps needed to create, mount, and export a storage volume can vary greatly. Custom routines can be added to address these steps, and to add in support for things like setting quotas.

Scyld Cloud Controller Setup

Configuration settings for the Scyld Cloud Controller are in the cloudcontroller.ini file. Important configuration settings are:

  • virtualization.boot_timeout - Time in seconds a VM has to boot into ‘ACTIVE’ state from a powered off state or creation before it is declared ‘ERROR’ and configuration is aborted.
  • virtualization.reboot_timeout - Time in seconds a VM has to boot into ‘ACTIVE’ state from a powered off state or creation before it is declared ‘ERROR’ and configuration is aborted. Reboots are generally quicker than a first boot.
  • virtualization.image_config_routines - A map of image IDs to Python classes to perform additional VM configuration after booting.
  • openstack.autodelete_bootvol - This boolean flag controls whether the Cinder volume used to boot a VM is automatically deleted when a VM is deleted.
  • storage.nfs_servers - For storage drivers that reach out to NFS servers, this is a comma separated list of their IP addresses.
  • nfs_export.default_options - For NFS exports created by the storage driver, this is the default set of export options.
  • nfs_export.auto_exports - A dictionary of CIDR:exportoption pairs that serve as a list of exports that should be created for every storage-volume. For example, exporting a newly created $HOME to cluster compute nodes.
  • nfs_export.auto_mounts - A dictionary of mountpoint:mountoption pairs that should be mounted automatically in every VM created by SCM.
  • nfs_export.auto_clients - A list of IP addresses that identify nodes that should automatically mount newly created exports. This is typically used for nodes that are not login nodes.
  • userauth.manage_sshd_acls - This boolean flag (default True) causes AllowUsers lines to be written to the VM’s /etc/ssh/sshd_config file when the VM starts. If the instance image’s property is_linux is True (the default), then one line is written for each user account that’s permitted to access the instance, and the VM’s ssh server is restarted. This allows those users to log in with passwords rather than ssh keys.