The first thing needed is a client computer running any kind of the supported OSs by the Amazon API tools and of course the tools themselves. After you’ve installed those and configured all credentials the right way, we can create a new pair of SSH keys for our new project. Due to the location I’m currently in, I’ve chosen to use Amazon Instances in the western EU. Execute the following line on your command-line:

ec2-add-keypair –region eu-west-1 test-keypair

The result should look something like this:

[Deprecated] Xalan: org.apache.xml.res.XMLErrorResources_en_US
KEYPAIR test-keypair e1:1a:d1:a1:a1:1c:10:a1:b1:d1:cb:11:11:1a:11:11:f1:11:ae:fe
—–BEGIN RSA PRIVATE KEY—–
XXX
—–END RSA PRIVATE KEY—–

You can now copy the lines from BEGIN… until END… (including those two) into a file which you’ll be using as SSH-key for connecting to your instance. Don’t forget to chmod 600 it!

Next, let’s see what base-images for creating our instance we could you – first, provided by Amazon themselves:

ec2-describe-images –region eu-west-1 -o ‘amazon’

The list is contained of several different OSs and versions, although in our current project we can’t make use of any of those. Therefor, we’re now searching for a perfectly fitting, really good operating system:

ec2-describe-images –region eu-west-1 -a | grep -i debian

And yet, we receive another list with several different versions of the Debian Linux distribution. After we’ve picked the one we’d like to run, we should check what instances are currently up and running:

ec2-describe-instances –region eu-west-1

If you’re using Amazon’s EC2 for the first time, there shouldn’t be any items listed. We can now start our very first instance, by copying the instance’s identifier (in the second column, a string starting with ami-) and pasting it into our command:

ec2-run-instances –region eu-west-1 -k test-keypair -g ‘http/s’ -g ‘ssh’ ami-b8446fcc

In this command, we tell Amazon to start up a new instance that’s built on top of the ami-b8446fcc-image, using the key-pair we just created before and using some custom built firewall-rules named “http/s” (which allows us to connect to port 80 and 443) and “ssh” (port 22).

We need to wait a few seconds, until the instance comes up. We can use the describe-instances command from above to check the instance’s status:

ec2-describe-instances –region eu-west-1

As soon as it’s up and running, the “pending” column should have been replaced by a dynamically allocated hostname and the status “running”. Keep in mind, that this hostname/ip is dynamically allocated! If you want a fixed IP, you need to allocate and assign an Elastic IP – I’ll show you later how to do so.
If our firewall rules worked out, we can now connect using SSH:

ssh -i ~/Library/EC2/id_rsa-test-keypair root@ec2-11-111-11-111.eu-west-1.compute.amazonaws.com

By default, Amazon sets up a Small Instance, that provides around 10 GB of hard drive, an Dual-Core AMD Opteron 2218 with 2600 MHz and around 1.7 GB of RAM. Small instances, in comparison to the bigger ones, also still provide a real swap-partition which is limited to 895 MB. Here, you could run into problems when installing some Oracle for example, since the DB would like to have 1 GB or more Swap-space. If 895 MB should not be enough, the only way to enlarge it seems to be to use a file within your file-system for that. Either, by placing it somewhere into / or by using /mnt for that. /mnt is a special mount in Amazon instances which provides you 147 GB of additional volatile storage. Amazon doesn’t guarantee in any way the storage to be stable/solid or even backed up – and usually /mnt is only used for bundling your instance. You might now think “so, where could I place my data, if / is only providing me 10 GB of space and /mnt should not be used for storing sensitive data?” – well, there’s a third possibility named Elastic Block Storage. An EBS is being displayed to your instance as regular block-device (/dev/sdb, …) that’s format- and mountable. There, sensitive data could be stored, by bind-mounting the directories you’d like to have your data in. I’m not going to explain how this works, else I’ll never finish writing this brief documentation.

However, since we’re connected to the instance now, we can set it up the way we want it, with whatever software we need on it. I’ve assumed, that most people would set it up as regular LAMP-instance, therefor I’ve also added the firewall-rule for HTTP/HTTPS. After we finished installing the software we need, there’s one more package that should be plugged into the system: The Amazon AMI Tools. Simply wget them from http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip and unzip the package. Those tools make use of Ruby, so ensure having it installed on your instance. Also, you need to have the private key (pk-.pem) and the certificate (cert-.pem) you created the instance with somewhere within your instance’s filesystem, for later use.

Before we bundle up our system now, let’s come back to the topic we had before: The hostname/IP. Amazon allocates some dynamic address, unless you tell them to give you a fixed (called Elastic) IP. You can do that by simply executing:

ec2-allocate-address –region eu-west-1

… within your client’s command-line (where the Amazon API Tools have been installed – not on the instance!). As result you’ll get some IP address that has now being allocated by Amazon for you. The IP isn’t bound to any instance yet – it’s just allocated for you to be able to use it. ATTENTION: You pay for Elastic IPs as long as you do not assign them to an instance. Yes, that’s right. Amazon charges you for every allocated but unused IP hourly. By that, they want to prevent people “collecting” addresses, I guess. To assign the address you just received to your instance, simply run:

ec2-associate-address –region eu-west-1 -i i-11e11b1e 79.125.11.11

The i- is your actual instance’s ID, where the last, dot-separated number is the IP you’ve received. And yes, those are fake numbers – so don’t even try.

Now it could take a bit for Amazon to reconfigure the instance, but as soon as it finished, you should be able to re-connect to the instance using the IP you just assigned to it.

As last task for today, let’s bundle up the image the way we have it now. Bundling an image allows you more than just eating up your credit card’s limit by dumping your S3 buckets. On the one hand, with bundles you can recover machines that crashed or lost data within a few blinks and on the other, you can created new instances out of a bundle (talking about “scalability”).

For bundling, we use the AMI tools we installed. First of all, let’s create a directory for the bundle:

mkdir /mnt/myimage

After that, run the bundle-vol-tool:

ec2-bundle-vol -k pk-.pem -c cert-.pem -s -u -d /mnt/myimage/

This command takes several parameters for the private key, the certificate, the size of the resulting bundle in MB and your User-ID (without dashes). The User-ID can be found within your Account Information on Amazon’s EC2 site. The command should ask you, what architecture you’d like to bundle the system for – i386 should work out perfectly for what we’re doing. The following procedure could take some time, since the tool collections every peace of the system and builds a bundle into the directory we specified. As soon as the tool finished, we can upload our bundle to our S3-bucket:

ec2-upload-bundle –location EU -b -m /mnt/myimage/image.manifest.xml -a -s

Again, we need to specify some credentials (our access-key and the secret-key) for the upload to work. Also, we need to pick a globally unique bucket-name for uploading the bundle, what shouldn’t be that hard as long as you don’t try stuff like “linux” or other common words. The bundle will then be uploaded to your (private) bucket, so you have it for later use.

From within the web-interface you could now simply create new instances out of the uploaded bundle, without even knowing how the actual system was set up or having the Amazon API tools installed on your client.

Cool stuff, enjoy!

Who's watching you?

Some time ago, I got interested in these tiny little cams with direct Ethernet connection, that contain a lightweight webserver that streams the actual picture to the network the cam’s connected to. I was holding such a cam in the hands at work, some while ago, and recently I was wondering myself, how expensive these would actually be. Not that I would really want to purchase one, just for informing myself. So, I opened Google and entered the manufactures name followed by some other keywords. The first links Google showed me were the links to the actual products, but while flying over Google’s first page of results I found some link directed to an IP address and ending with an .shtml file. The structure of the URL looked similar like the one I’ve seen from the cams we have at work… call it boredom, but I followed the link and – I was surprised – I landed on such a cam’s video server which streamed me the picture of the traffic on some highway. I got the idea to enter the site’s title into Google and search for more links similar to this one. And what should I say… after around one hour of search I’ve already found seven sites containing the live-pictures of webcams which in my opinion seem not to be intended for public-viewing. Or at least, I don’t know why someone should want the internet to watch his bar’s guests or his co-workers to be visible on the world wide web.

I’m really astonished about some administrator’s sloppiness on such things. Especially, because it’s not them who are visible on the internet, but the people around them. On some webcams I really got the impression, that the administrators of these webcams thought, that it would be good for them to be able to access the webcams from anywhere, and to make this possible just use “security by obscurity”. The webcams where all only accessible directly by entering the server’s (or gateway’s, dunno) IP-address and had no security at all, for accessing the live-stream.

However, I think, I kinda found some new hobby for long nights. And you should think twice when you see a camera connected to the network somewhere – you might be visible to the whole internet. Oh, this reminds me of the local Subway, where three cameras are hanging in the corners, but no sign indicates that the place is getting filmed. Hm…

conn nat-t
ikelifetime=60m
keylife=20m
rekeymargin=3m
keyingtries=1
keyexchange=ikev2
authby=secret
left=192.168.10.193
leftsubnet=10.1.0.0/24
leftfirewall=yes
right=%defaultroute
rightid=client@example.com
rightsourceip=10.100.0.2
auto=add

We added a new virtual IP (rightsourceip) for the client. The network of this IP will be defined on the server’s configuration. We need this for the whole scenario to work out, even if the client/Roadwarrior is behind a NAT. As rightid (client-id) we use the client’s e-mail address.
Server config:

config setup
plutostart=no

conn nat-t
ikelifetime=60m
keylife=20m
rekeymargin=3m
keyingtries=1
keyexchange=ikev2
authby=secret
left=10.1.0.4
leftsubnet=10.1.0.0/24
leftid=192.168.10.193
leftfirewall=yes
right=%any
rightid=client@example.com
rightsubnetwithin=10.100.0.0/24
auto=add


Here, we also defined the client’s e-mail address as rightid, defined that the right side could be anything (“%any”) and told the server to serve the virtual network 10.100.0.0/24 for the right side. By that, the configuration can be applied to different clients and the actual IP configuration is provided on the client’s side. Yet, I did not find out whether there’s a possibility to set up some DHCP server and provide connecting clients a dynamic address automatically.
However, this setup now also works with Roadwarriors that are behind NATs, what means that the actual setup could look like this:

[roadwarrior]—-[nat]—internet—[nat]—[server]—network

Isn’t this cool?
Enjoy.