OpenStack Ephemeral Instances

What is Ephemeral?

Ephemeral is the opposite of persistent. In the OpenStack context this is a VM that is created from a fixed base image, when the instance is terminated all the local changes to configuration, packages, or data since it was booted are lost.

Specific system changes can be captured in a new image by taking a “snapshot”. This is usually done to persist package upgrades and/or configuration changes.

Ephemeral systems can store persistent data on AFS or NFS network storage. OpenStack also provides a volume service for persistent storage. These volumes exist independently of any particular VM and can be attached to a single VM at a time. TIG provides standard backup services for AFS and NFS but not for OpenStack Volumes.

What are they good for

At first it may seem that servers that essentially revert all their state on termination are not a good idea, but they do provide several main benefits from which a number of others derive:

• It is easy to start many copies of the identical system. This can either be for scale out or to provide test and development copies that are identical to the production systems(s)

• Restarts are always in a known good state. This provides excellent stability and restartability. It is also useful in the case of a security breach you can restart in a known clean state and patch the vulnerability where as on a traditional persistent type system (physical or virtual) the only sure way to recover is a complete re-install.

• Images are very space efficient. Obviously one image that serves many instances is less resource intensive than having unique storage for each instance. Instances that share the same base image running on the same physical hardware use copy on write overlays to hold their unique runtime state (rather than multiple full copies of the image), further increasing space efficiency.

What are they NOT good for

Ephemeral instances are not good for singular systems whose main purpose is to gather data, for example serving a wiki to a small group. Though even in this case if you were to use network storage and or TIG provided MySQL or PostgreSQL databases it would be possible to architect a solution using an ephemeral system.

More information is about Persistent VMs is available on their own page.

Basic web based launch process

This describes the minimal steps needed to boot. Optional steps such as booting multiple instances at once or setting a fixed IP address are not covered here.

Pre-requisites:

• have an OpenStackAccount
• have an ssh key uploaded

Steps, it is much faster to do than to explain, honest:

1. Login to https://horizon.csail.mit.edu

2. If you belong to more than one project select the correct project from the drop-down menu in the left column.

3. Select Instances from the left column menu

4. Select + Launch Instance button at top right

5. Fill in an arbitrary Instance Name

6. Select a Flavor this is a preset selection of CPU,RAM, and Disk resources. When in doubt “m1.1core” is a good option.

7. Select Instance Boot Source

   • Boot from image to boot from a public image, pick this if you don’t know which to choose.
   • Boot from snapshot to boot from one of your project’s snapshots

8. Select Image Name, a good default is CSAIL Ubuntu 18.04LTS which is toward the bottom of the listing

9. Select the Access & Security tab, check that your key is listed in Keypair, it should be selected by default.

10. Select the networking tab

11. Select the + in the block labeled inet under Available Networks, this will then move the inet block to Selected Networks and be labled nic: 1 (note for most projects where this is the only option you can skip this step)

12. Finally select the Launch button.

You should then be directed back to the Instances page where you will see your new instance in the spawning state. After some time (a few seconds up to about 2min) the system will be updated to Active. This means the VM has been successfully built and launch, the operating system is active but still booting at this point. It may be another minute for the system to boot up to the point where ssh is running and you can connect.

To connect to the system note the IP your system was assigned, this is shown on the instances page. Then connect as the ‘ubuntu’ user using your ssh private key. If you launched an “CSAIL” image you could also connect with your CSAIL username and password but the ‘ubuntu’ user has passwordless sudo access where your normal user does not.

ssh -i </path/to/your/private_key> ubuntu@<IP>

Customization and snapshots

Runtime Customization

All of our Ubuntu based images include the cloudinit package which provides a rich set of runtime customization via “usrdata”. This is a text file that can be pasted in at boot time. The web interface calls this Customization Script which can be found on the last, Post-Creation, tab of the Launch Instance dialog.

This is very useful for creating similar, but slightly different, instances from the same base image.

The simplest form is to paste in a script, anything starting with #!/path/to/interpreter such as #!/bin/sh, #!/usr/bin/env ruby. This will be run as root toward the end of the boot process.

For longer or frequently used scripts you can also pass a URL to the script rather than the script itself by starting the text with #include, it is also possible to combine scripts by including multiple URLs:

#include
http://people.csail.mit.edu/jon/cloudinit/echos.sh

Cloud-init is capable of many more complex tricks, more or less fully documented at http://cloudinit.readthedocs.org/en/latest/index.html

A good all (well most) in one example with comments http://www.knowceantech.com/2014/03/amazon-cloud-bootstrap-with-userdata-cloudinit-github-puppet/

Persisting changes via Snapshots

If you need to update one of your images or create a new image based on an existing public image, snapshots provide you a way to capture the current state of a running VM into a bootable base image.

When creating a new image it is best to land one of the “s1” type VMs probably and s1.2core or s1.4core. Snapshots are a compressed image of the root filesystem at the block level and include all the empty space. If you base your snap shot on an “xl” instance with a 64G root it will only boot “xl” type instances. If you create the snapshot from an “s1” instance with 10G of space you can boot any instance type and the root filesystem will be expanded to fit the instance root volume on boot.

The actual process of creating a snapshot is very easy:

1 Boot the base image you want to start with 1 Make the changes you want 1 Select Create Snapshot from the Actions column (far right) of the Instances page, note this may make the instance unresponsive for a minute or two.

Snapshot can be renamed easily, for example if you want to rename “myimage” to “myimage.old” before taking a new snapshot:

1 Go to the Images & Snapshots page in the Web UI 1 Select the Project button at the top (this is where you are by default if your project has snapshots) 1 Find the snapshot you wish to rename and select edit from the more drop down menu in the Actions column

User accessible fault recovery

For ephemeral instance the easiest and fastest path to recovery for stuck instances is to terminate and relaunch them. If you are afraid to loose state you should be using [Persistant VMs].

That said we recognize the world is an imperfect place, Persistent VMs page has more details on fault recovery. You can also contact help@csail.mit.edu if you need to recover a failed ephemeral VM and we’ll help as much as we can.