Easy Steps to Start Up Amazon EC2 Images

Posted by: Wesley Willard on

Amazon Elastic Compute Cloud (EC2) has generated enormous amounts of buzz in the last couple of months. EC2 allows scalable deployment of applications by providing a way for customers to create server “instances” which can run any number of operating systems, such as Windows XP and Vista, and any number of Linux distributions. The customer can load any software of their choice on to the machines, and customize them at will. A customer can create, launch, and terminate server instances as needed, paying only for usage.

In addition to providing a Amazon Management Console to manage these instances, Amazon also provides command line tools which allow for the automation of instance administration. By combining these tools with some basic shell commands, you can customize the startup of your EC2 instances. I will provide an example that shows how this can be accomplished.

This example assumes you have the following available:


Okay, let's get started. First of all, we'll define some basic shell variables:

export EC2_HOME=/opt/ec2/tools/ec2-api-tools-1.3-30349/
export EC2_PRIVATE_KEY=~/ec2/pk-HGHEBCKY657B2ZUH7YIVXZKT5QVNBWFM.pem
export EC2_CERT=~/ec2/cert-HGHEBCKY657B2ZUH7YIVXZKT5QVNBWFM.pem
export amiid="ami-cccb2ca5"
export key="mykey"
export zone="us-east-1c"
export group="server"
export id_file="mykey.pem"
export vol_name="vol-8abe59e3"
export mount_point="/mnt/vol"
export device_name="/dev/sdf"
export ip="174.129.232.50"

The variables that start with the name “EC2” are used by the Tools API. They are the directory where you downloaded your Tools installation and the key and cert files provided to you by Amazon when you created your instance. The other variables are used by the shell commands in this example, and include:
  • Amazon Image ID
  • Key name associated with this instance
  • The zone in which this instance was created
  • The group associated with this instance
  • The name of the user's EC2 identity file
  • The ESB volume ID
  • The mount directory within the EC2 instance
  • The device name to associate with the ESB volume when attached
  • The Elastic IP address to associate with the instance

Now it's time to start the instance, and then loop until it is running.

#
# Start the instance
# Capture the output so that
# we can grab the INSTANCE ID field
# and use it to determine when
# the instance is running 
#
echo Launching AMI ${amiid}
${EC2_HOME}/bin/ec2-run-instances ${amiid} -z ${zone} -k ${key} --group ${group} 
--group default > /tmp/a if [ $? != 0 ]; then echo Error starting instance for image ${amiid} exit 1 fi export iid=`cat /tmp/a | grep INSTANCE | cut -f2` # # Loop until the status changes to “running” # sleep 30 echo Starting instance ${iid} export RUNNING="running" export done="false" while [ $done == "false" ] do export status=`${EC2_HOME}/bin/ec2-describe-instances ${iid} | grep INSTANCE | cut -f6` if [ $status == ${RUNNING} ]; then export done="true" else echo Waiting... sleep 10 fi done echo Instance ${iid} is running

Now we have the running instance ID, which we will use going forward. We next attach the ESB volume to the running instance, associating a device name. After we attach the volume, we wait until its status indicates that it is attached.

#
# Attach the volume to the running instance
#
echo Attaching volume ${vol_name}
${EC2_HOME}/bin/ec2-attach-volume ${vol_name} -i ${iid} -d ${device_name} 
sleep 15

#
# Loop until the volume status changes
# to "attached" 
#
export ATTACHED="attached"
export done="false"
while [ $done == "false" ]
do
   export status=`${EC2_HOME}/bin/ec2-describe-volumes | grep ATTACHMENT | grep ${iid} | cut -f5`
   if [ $status == ${ATTACHED} ]; then
      export done="true"
   else 
      echo Waiting...
      sleep 10
   fi
done
echo Volume ${vol_name} is attached

Now we associate the Elastic IP address with the running instance. This capability is important in an environment where instances are being started and stopped at various points for scalability reasons, so these operations will happen with no interruption for the user.

#
# Associate the Elastic IP with the instance
# After this operation we just sleep a bit
#
echo Associating elastic IP address ${ip}
${EC2_HOME}/bin/ec2-associate-address 174.129.232.50 -i ${iid}
sleep 30

Our final step for starting our instance is to copy and execute some additional commands within the running instance. These operations will create a mount point and mount the volume. Our commands assume that any partitioning and file system type creation has already been setup in the Amazon image’s /etc/fstab file. We use SSH to copy and execute these commands. Because EC2 does not allow username/password authentication, we must provide our identity file to SSH.

#
# Start the operations within the instance
# Copy over the mount script and execute it
# The script setup_vol does:
#     mkdir /mnt/vol
#     mount /mnt./vol
# 
scp -i ${id_file} setup_vol root@${ip}:/root
ssh -i ${id_file} root@${ip} . ./setup_vol

Finally, because this instance was created with an Ubuntu Desktop image, we can set up an NX Server to allow us to access our running instance with a remote desktop:

ssh -i ${id_file} root@${ip} /usr/NX/bin/nxserver --stop
ssh -i ${id_file} root@${ip} /usr/NX/bin/nxserver --start

Last, but not least, tell the user that we are ready to start using the running instance.

echo Image ${amiid} instance ${iid} is ready to go!
echo ${iid} > current_instance

I have attempted to demonstrate how you can use automation to facilitate the easy startup of your Amazon EC2 instance. This example can serve as a starting point for further customizations specific to your environment.

In Amazon Web ServicesAmazon EC2

Comments (6)Add Comment
Thanks!
written by Channing Walton, July 08, 2009
Thanks for this, its very helpful. smilies/cheesy.gif
report abuse
vote down
vote up
Votes: +0
...
written by Dimitris, July 11, 2009
You can also add /opt/ec2/tools/ec2-api-tools-1.3-30349/bin to your path so that you can easily run the commands from bash.

in your .bash_profile:

export PATH=$PATH:/opt/ec2/tools/ec2-api-tools-1.3-30349/bin
report abuse
vote down
vote up
Votes: +0
How to use script if have HTTP Proxy tunneling
written by caftech, July 21, 2009
Hi,

I tried to use your script but since my company requires all internet access to go thru HTTP Proxy tunneling, how should I customize the script to do that? Currently, when I run the script it fails with:

Launching AMI ami-cecb2fa7
Unexpected error:
java.net.NoRouteToHostException: No route to host
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:550)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.(SSLSocketImpl.java:394)
at com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:123)
at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:81)
at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:126)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
at org.apache.commons.httpclient. MultiThreadedHttpConnectionManager$HttpConnectionAdapte r.open(MultiThreadedHttpConnectionManager.java:1321)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.codehaus.xfire.transport.http.CommonsHttpMessageSender.send(CommonsHttpMessageSender.java:369)
at org.codehaus.xfire.transport.http.HttpChannel.sendViaClient(HttpChannel.java:123)
at org.codehaus.xfire.transport.http.HttpChannel.send(HttpChannel.java:4smilies/cool.gif
at org.codehaus.xfire.handler.OutMessageSender.invoke(OutMessageSender.java:26)
at org.codehaus.xfire.handler.HandlerPipeline.invoke(HandlerPipeline.java:131)
at org.codehaus.xfire.client.Invocation.invoke(Invocation.java:79)
at org.codehaus.xfire.client.Invocation.invoke(Invocation.java:114)
at org.codehaus.xfire.client.Client.invoke(Client.java:336)
at org.codehaus.xfire.client.XFireProxy.handleRequest(XFireProxy.java:77)
at org.codehaus.xfire.client.XFireProxy.invoke(XFireProxy.java:57)
at $Proxy12.runInstances(Unknown Source)
at com.amazon.aes.webservices.client.Jec2.runInstances(Jec2.java:76smilies/cool.gif
at com.amazon.aes.webservices.client.cmd.RunInstances.invokeOnline(RunInstances.java:142)
at com.amazon.aes.webservices.client.cmd.BaseCmd.invoke(BaseCmd.java:631)
at com.amazon.aes.webservices.client.cmd.RunInstances.main(RunInstances.java:19smilies/cool.gif
Error starting instance for image ami-cecb2fa7
report abuse
vote down
vote up
Votes: +0
...
written by caftech, July 21, 2009
Okay, got it to work now, have to pass the following to the JVM. Do so by adding the following to ec2-cmd:

EC2_JVM_ARGS="-Dhttps.proxyHost=mycompany.domain -Dhttps.proxyPort=80"
export EC2_JVM_ARGS
report abuse
vote down
vote up
Votes: +0
Really useful, thanks!
written by Kieran, September 05, 2009
Good work. Was a really useful basis for my own script. Cheers
report abuse
vote down
vote up
Votes: +0
Thanks!
written by Jeff, September 07, 2009
Just been using EC2 for a few weeks now and this is exactly what I was looking for!!! Hooray.
report abuse
vote down
vote up
Votes: +0

Write comment
quote
bold
italicize
underline
strike
url
image
quote
quote
smile
wink
laugh
grin
angry
sad
shocked
cool
tongue
kiss
cry
smaller | bigger

busy