The first thing needed is a client computer running any kind of the supported OSs by the Amazon API tools and of course the tools themselves. After you’ve installed those and configured all credentials the right way, we can create a new pair of SSH keys for our new project. Due to the location I’m currently in, I’ve chosen to use Amazon Instances in the western EU. Execute the following line on your command-line:

ec2-add-keypair –region eu-west-1 test-keypair

The result should look something like this:

[Deprecated] Xalan: org.apache.xml.res.XMLErrorResources_en_US
KEYPAIR test-keypair e1:1a:d1:a1:a1:1c:10:a1:b1:d1:cb:11:11:1a:11:11:f1:11:ae:fe
—–BEGIN RSA PRIVATE KEY—–
XXX
—–END RSA PRIVATE KEY—–

You can now copy the lines from BEGIN… until END… (including those two) into a file which you’ll be using as SSH-key for connecting to your instance. Don’t forget to chmod 600 it!

Next, let’s see what base-images for creating our instance we could you – first, provided by Amazon themselves:

ec2-describe-images –region eu-west-1 -o ‘amazon’

The list is contained of several different OSs and versions, although in our current project we can’t make use of any of those. Therefor, we’re now searching for a perfectly fitting, really good operating system:

ec2-describe-images –region eu-west-1 -a | grep -i debian

And yet, we receive another list with several different versions of the Debian Linux distribution. After we’ve picked the one we’d like to run, we should check what instances are currently up and running:

ec2-describe-instances –region eu-west-1

If you’re using Amazon’s EC2 for the first time, there shouldn’t be any items listed. We can now start our very first instance, by copying the instance’s identifier (in the second column, a string starting with ami-) and pasting it into our command:

ec2-run-instances –region eu-west-1 -k test-keypair -g ‘http/s’ -g ‘ssh’ ami-b8446fcc

In this command, we tell Amazon to start up a new instance that’s built on top of the ami-b8446fcc-image, using the key-pair we just created before and using some custom built firewall-rules named “http/s” (which allows us to connect to port 80 and 443) and “ssh” (port 22).

We need to wait a few seconds, until the instance comes up. We can use the describe-instances command from above to check the instance’s status:

ec2-describe-instances –region eu-west-1

As soon as it’s up and running, the “pending” column should have been replaced by a dynamically allocated hostname and the status “running”. Keep in mind, that this hostname/ip is dynamically allocated! If you want a fixed IP, you need to allocate and assign an Elastic IP – I’ll show you later how to do so.
If our firewall rules worked out, we can now connect using SSH:

ssh -i ~/Library/EC2/id_rsa-test-keypair root@ec2-11-111-11-111.eu-west-1.compute.amazonaws.com

By default, Amazon sets up a Small Instance, that provides around 10 GB of hard drive, an Dual-Core AMD Opteron 2218 with 2600 MHz and around 1.7 GB of RAM. Small instances, in comparison to the bigger ones, also still provide a real swap-partition which is limited to 895 MB. Here, you could run into problems when installing some Oracle for example, since the DB would like to have 1 GB or more Swap-space. If 895 MB should not be enough, the only way to enlarge it seems to be to use a file within your file-system for that. Either, by placing it somewhere into / or by using /mnt for that. /mnt is a special mount in Amazon instances which provides you 147 GB of additional volatile storage. Amazon doesn’t guarantee in any way the storage to be stable/solid or even backed up – and usually /mnt is only used for bundling your instance. You might now think “so, where could I place my data, if / is only providing me 10 GB of space and /mnt should not be used for storing sensitive data?” – well, there’s a third possibility named Elastic Block Storage. An EBS is being displayed to your instance as regular block-device (/dev/sdb, …) that’s format- and mountable. There, sensitive data could be stored, by bind-mounting the directories you’d like to have your data in. I’m not going to explain how this works, else I’ll never finish writing this brief documentation.

However, since we’re connected to the instance now, we can set it up the way we want it, with whatever software we need on it. I’ve assumed, that most people would set it up as regular LAMP-instance, therefor I’ve also added the firewall-rule for HTTP/HTTPS. After we finished installing the software we need, there’s one more package that should be plugged into the system: The Amazon AMI Tools. Simply wget them from http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip and unzip the package. Those tools make use of Ruby, so ensure having it installed on your instance. Also, you need to have the private key (pk-.pem) and the certificate (cert-.pem) you created the instance with somewhere within your instance’s filesystem, for later use.

Before we bundle up our system now, let’s come back to the topic we had before: The hostname/IP. Amazon allocates some dynamic address, unless you tell them to give you a fixed (called Elastic) IP. You can do that by simply executing:

ec2-allocate-address –region eu-west-1

… within your client’s command-line (where the Amazon API Tools have been installed – not on the instance!). As result you’ll get some IP address that has now being allocated by Amazon for you. The IP isn’t bound to any instance yet – it’s just allocated for you to be able to use it. ATTENTION: You pay for Elastic IPs as long as you do not assign them to an instance. Yes, that’s right. Amazon charges you for every allocated but unused IP hourly. By that, they want to prevent people “collecting” addresses, I guess. To assign the address you just received to your instance, simply run:

ec2-associate-address –region eu-west-1 -i i-11e11b1e 79.125.11.11

The i- is your actual instance’s ID, where the last, dot-separated number is the IP you’ve received. And yes, those are fake numbers – so don’t even try.

Now it could take a bit for Amazon to reconfigure the instance, but as soon as it finished, you should be able to re-connect to the instance using the IP you just assigned to it.

As last task for today, let’s bundle up the image the way we have it now. Bundling an image allows you more than just eating up your credit card’s limit by dumping your S3 buckets. On the one hand, with bundles you can recover machines that crashed or lost data within a few blinks and on the other, you can created new instances out of a bundle (talking about “scalability”).

For bundling, we use the AMI tools we installed. First of all, let’s create a directory for the bundle:

mkdir /mnt/myimage

After that, run the bundle-vol-tool:

ec2-bundle-vol -k pk-.pem -c cert-.pem -s -u -d /mnt/myimage/

This command takes several parameters for the private key, the certificate, the size of the resulting bundle in MB and your User-ID (without dashes). The User-ID can be found within your Account Information on Amazon’s EC2 site. The command should ask you, what architecture you’d like to bundle the system for – i386 should work out perfectly for what we’re doing. The following procedure could take some time, since the tool collections every peace of the system and builds a bundle into the directory we specified. As soon as the tool finished, we can upload our bundle to our S3-bucket:

ec2-upload-bundle –location EU -b -m /mnt/myimage/image.manifest.xml -a -s

Again, we need to specify some credentials (our access-key and the secret-key) for the upload to work. Also, we need to pick a globally unique bucket-name for uploading the bundle, what shouldn’t be that hard as long as you don’t try stuff like “linux” or other common words. The bundle will then be uploaded to your (private) bucket, so you have it for later use.

From within the web-interface you could now simply create new instances out of the uploaded bundle, without even knowing how the actual system was set up or having the Amazon API tools installed on your client.

Cool stuff, enjoy!

The actual reason, why I upgraded was KVM. Lenny seems to be getting more and more stable and by that usable as server system. Of course, it’s not etch yet, but since I’m trying to migrate from VMware (-Server) to KVM I’m actually forced to use Lenny. There are no KVM packages available for Etch, besides the ones from Backports.org. Unfortunatelly, even those are only available at version 28-4, what’s not that usable on a productive system.

However, so like I said, today I s/etch/lenny/g on my APT sources.list and ran an aptitude dist-upgrade. I was pretty surprised how fluent the upgrade worked out, the only thing it broke was the Roundcubemail I have installed on the machine for checking the Maildir via HTTP(S). Apache still works fine, Courier also runs, Fetchmail, and all the other stuff I had configured upgraded with nearly no problems.

Unfortunately, the upgrade still caused problems regarding my everyday-work. The most annoying thing is the fact, that I’m forced to use Icedove as mail client from now on, at least until some strange bug regarding the SSL authentication has been fixed in Evolution. The curious thing is, that with the previous version of Courier, Evolution worked just fine. I don’t know, what exactly the Courier developers changed, but it caused evolution to not be able anymore to authenticate against it. Some might say, it sounds like an Courier bug, but to be honest, I know how things are being implemented into Evolution and I don’t believe it’s Courier’s fault.

It’s pretty much a bummer, because I actually liked working with Evolution. Not because it’s fast – it’s not. And not because it’s light – it’s neither. And not even because it works perfectly in the infrastructure I’m sitting in – it definitely did not. But Evolution unfortunatelly is the only mail reader for the GNOME desktop environment, which really integrates into the desktop. I could always take a quick look at the meetings/appointments I have for a specific test, by just clicking the the clock on my menubar. Also, it was integrated into GNOME’s keyring, so that my mail-account password was just getting unlocked after the login.

However, this tiny, integrative features enhanced the usability of Evolution a lot, even if everything else might just suck. Now I’m trying to find out which bugreport documented the problem I’m experiencing, to put myself on CC. I’m really wondering when this will be fixed. If it takes as long as fixing problems regarding shared folders or caldav, then I guess I’ll just try to say Hello! to my future e-mail client Icedove.

Meh.

First of all: You cannot tunnel RDP directly through a proxy. RDP doesn’t speak any HTTP(S) to make the proxy connect to the RDP-Server or anything else. So you’ll need an application, that surrounds this RDP datachannel with HTTP, prefferable HTTPS. I found prtunnel for my Mac on DarwinPorts. This software allows you to tunnel anything you want through an http/socks proxy by connecting to the proxy, making it connect (by sending HTTP commands) to the preffered host and open a local port for the application (e.g. rdesktop) to connect. Good, so let’s connect using prtunnel to myrdpmachine.com:3389 and be happy! – NAH. As soon as you’ll try that you’ll see that it’s not that simple. Most http-proxies do not allow CONNECTs to other ports than 80/443. So you can either set up your RDP daemon to use that port – never found that option in Windoze – or you can use an SSH jumphost, since it’s pretty simple to change the SSH port to 443. So, you connect with prtunnel to your SSH machine on port 443, where the SSH daemon runs, open an SSH tunnel through that machine to the myrdpmachine.com port 3389 and connect with your RDP client on localhost:. Okay, let’s stop the theory and begin with the practice:

Open three terminals and execute the following command on the first one:

prtunnel -V -t http -H 'proxy address' -P 'proxy port' \
'port on local machine' 'remote host to connect to over proxy' \
'remote port, put SSHd on 443'

Then, terminal #2 gets the following command: ssh -L’local tunneling port’:'destination host’:'destination port’ -p ‘local port to connect to, the same given at prtunnel’ ‘user’@localhost
After that you can hapily run your rdesktop localhost:’local tunneling port’ and start RDPing. To make the stuff even more clear, here a concrete example:

prtunnel -V -t http -H 192.168.111.2 -P 3128 13337 192.168.111.3 443
ssh -L13338:192.168.111.24:3389 -p 13337 root@localhost
rdesktop localhost:13338

That’s all the magic. Though, you need to pay attention when selecting your ports, because of course only free ports will work and you really should try to keep them higher than 1024 unless you want to become root. Also you need to remember that running an RDP session over HTTP(S) might get the attention of a firewall or whatever monitoring application is available in that network. “Abnormal behaviour” – you’d never get such an 50:50 up- and downtraffic unless you run some peer-2-peer application or remote desktops.

But of course you can modify the commands and use it to be able to connect to let’s say Jabber from a network where only 80/443-outgoing is available – all you need is a jumphost.