The first thing needed is a client computer running any kind of the supported OSs by the Amazon API tools and of course the tools themselves. After you’ve installed those and configured all credentials the right way, we can create a new pair of SSH keys for our new project. Due to the location I’m currently in, I’ve chosen to use Amazon Instances in the western EU. Execute the following line on your command-line:

ec2-add-keypair –region eu-west-1 test-keypair

The result should look something like this:

[Deprecated] Xalan: org.apache.xml.res.XMLErrorResources_en_US
KEYPAIR test-keypair e1:1a:d1:a1:a1:1c:10:a1:b1:d1:cb:11:11:1a:11:11:f1:11:ae:fe
—–BEGIN RSA PRIVATE KEY—–
XXX
—–END RSA PRIVATE KEY—–

You can now copy the lines from BEGIN… until END… (including those two) into a file which you’ll be using as SSH-key for connecting to your instance. Don’t forget to chmod 600 it!

Next, let’s see what base-images for creating our instance we could you – first, provided by Amazon themselves:

ec2-describe-images –region eu-west-1 -o ‘amazon’

The list is contained of several different OSs and versions, although in our current project we can’t make use of any of those. Therefor, we’re now searching for a perfectly fitting, really good operating system:

ec2-describe-images –region eu-west-1 -a | grep -i debian

And yet, we receive another list with several different versions of the Debian Linux distribution. After we’ve picked the one we’d like to run, we should check what instances are currently up and running:

ec2-describe-instances –region eu-west-1

If you’re using Amazon’s EC2 for the first time, there shouldn’t be any items listed. We can now start our very first instance, by copying the instance’s identifier (in the second column, a string starting with ami-) and pasting it into our command:

ec2-run-instances –region eu-west-1 -k test-keypair -g ‘http/s’ -g ‘ssh’ ami-b8446fcc

In this command, we tell Amazon to start up a new instance that’s built on top of the ami-b8446fcc-image, using the key-pair we just created before and using some custom built firewall-rules named “http/s” (which allows us to connect to port 80 and 443) and “ssh” (port 22).

We need to wait a few seconds, until the instance comes up. We can use the describe-instances command from above to check the instance’s status:

ec2-describe-instances –region eu-west-1

As soon as it’s up and running, the “pending” column should have been replaced by a dynamically allocated hostname and the status “running”. Keep in mind, that this hostname/ip is dynamically allocated! If you want a fixed IP, you need to allocate and assign an Elastic IP – I’ll show you later how to do so.
If our firewall rules worked out, we can now connect using SSH:

ssh -i ~/Library/EC2/id_rsa-test-keypair root@ec2-11-111-11-111.eu-west-1.compute.amazonaws.com

By default, Amazon sets up a Small Instance, that provides around 10 GB of hard drive, an Dual-Core AMD Opteron 2218 with 2600 MHz and around 1.7 GB of RAM. Small instances, in comparison to the bigger ones, also still provide a real swap-partition which is limited to 895 MB. Here, you could run into problems when installing some Oracle for example, since the DB would like to have 1 GB or more Swap-space. If 895 MB should not be enough, the only way to enlarge it seems to be to use a file within your file-system for that. Either, by placing it somewhere into / or by using /mnt for that. /mnt is a special mount in Amazon instances which provides you 147 GB of additional volatile storage. Amazon doesn’t guarantee in any way the storage to be stable/solid or even backed up – and usually /mnt is only used for bundling your instance. You might now think “so, where could I place my data, if / is only providing me 10 GB of space and /mnt should not be used for storing sensitive data?” – well, there’s a third possibility named Elastic Block Storage. An EBS is being displayed to your instance as regular block-device (/dev/sdb, …) that’s format- and mountable. There, sensitive data could be stored, by bind-mounting the directories you’d like to have your data in. I’m not going to explain how this works, else I’ll never finish writing this brief documentation.

However, since we’re connected to the instance now, we can set it up the way we want it, with whatever software we need on it. I’ve assumed, that most people would set it up as regular LAMP-instance, therefor I’ve also added the firewall-rule for HTTP/HTTPS. After we finished installing the software we need, there’s one more package that should be plugged into the system: The Amazon AMI Tools. Simply wget them from http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip and unzip the package. Those tools make use of Ruby, so ensure having it installed on your instance. Also, you need to have the private key (pk-.pem) and the certificate (cert-.pem) you created the instance with somewhere within your instance’s filesystem, for later use.

Before we bundle up our system now, let’s come back to the topic we had before: The hostname/IP. Amazon allocates some dynamic address, unless you tell them to give you a fixed (called Elastic) IP. You can do that by simply executing:

ec2-allocate-address –region eu-west-1

… within your client’s command-line (where the Amazon API Tools have been installed – not on the instance!). As result you’ll get some IP address that has now being allocated by Amazon for you. The IP isn’t bound to any instance yet – it’s just allocated for you to be able to use it. ATTENTION: You pay for Elastic IPs as long as you do not assign them to an instance. Yes, that’s right. Amazon charges you for every allocated but unused IP hourly. By that, they want to prevent people “collecting” addresses, I guess. To assign the address you just received to your instance, simply run:

ec2-associate-address –region eu-west-1 -i i-11e11b1e 79.125.11.11

The i- is your actual instance’s ID, where the last, dot-separated number is the IP you’ve received. And yes, those are fake numbers – so don’t even try.

Now it could take a bit for Amazon to reconfigure the instance, but as soon as it finished, you should be able to re-connect to the instance using the IP you just assigned to it.

As last task for today, let’s bundle up the image the way we have it now. Bundling an image allows you more than just eating up your credit card’s limit by dumping your S3 buckets. On the one hand, with bundles you can recover machines that crashed or lost data within a few blinks and on the other, you can created new instances out of a bundle (talking about “scalability”).

For bundling, we use the AMI tools we installed. First of all, let’s create a directory for the bundle:

mkdir /mnt/myimage

After that, run the bundle-vol-tool:

ec2-bundle-vol -k pk-.pem -c cert-.pem -s -u -d /mnt/myimage/

This command takes several parameters for the private key, the certificate, the size of the resulting bundle in MB and your User-ID (without dashes). The User-ID can be found within your Account Information on Amazon’s EC2 site. The command should ask you, what architecture you’d like to bundle the system for – i386 should work out perfectly for what we’re doing. The following procedure could take some time, since the tool collections every peace of the system and builds a bundle into the directory we specified. As soon as the tool finished, we can upload our bundle to our S3-bucket:

ec2-upload-bundle –location EU -b -m /mnt/myimage/image.manifest.xml -a -s

Again, we need to specify some credentials (our access-key and the secret-key) for the upload to work. Also, we need to pick a globally unique bucket-name for uploading the bundle, what shouldn’t be that hard as long as you don’t try stuff like “linux” or other common words. The bundle will then be uploaded to your (private) bucket, so you have it for later use.

From within the web-interface you could now simply create new instances out of the uploaded bundle, without even knowing how the actual system was set up or having the Amazon API tools installed on your client.

Cool stuff, enjoy!

For my tests, I was using a vanilla Ubuntu 9.04 with latest package versions/updates installed. The installation of the PPA was pretty simple, thanks to the DEB provided on the Ubuntu One site, although I could not install the Ubuntu One GNOME-Client using the one-click feature implemented on the official Install-site. This probably could be the first thing a regular user could trap on.

So, after installing the packages manually by using a terminal and aptitude, the client appeared in Ubuntu’s “Internet”-submenu of the panel-main-menu. When a user installs Ubuntu One on a new computer and clicks the client’s icon within the menu to start the service, he will get a browser-window containing an Ubuntu One authorization-page. There, the user needs to click the “Authorize this Computer” button, so the computer will be able to sync with the Ubuntu One cloud. I took a quick look behind the scenes and as far as I’ve seen, Ubuntu One uses regular OAuth to authenticate computers to the cloud. The OAuth-token therefor will be saved within the “Passwords & Keys” thingy, that’s found somewhere in the main-menu.

However, the sync service itself creates a “Ubuntu One” folder within your home-directory, which syncs up into the cloud. Within this folder, there’s also a sym-linked-folder for Public content. The client-daemon doesn’t seem to be using some mechanism like iNotify to sync the folder contents up to the cloud, or if it does, it’s simply broken. At least on my test-machine, the client synchronized periodically – and by periodically I mean cycles with noticeable pauses in between. On one way that’s good, because not every tiny-whiny change that will be made to the folder’s contents will be pushed up to the cloud, what would literally kill your bandwidth, but on the other hand, the actual duration from one sync to another is just too big to be used with multiple devices simultaneously (e.g. an Android implementation or even another client-computer).

Talking about the simultaneousness, I don’t think that Ubuntu One is (yet) planned to be used on multiple computers simultaneous. On my test-setup I’ve created a sub-folder named “Home”, where I moved some dot-starting-folders from within my actual $HOME to and soft-linked them back to their actual location. My setup then looked something like:


devilx@vm-ubuntu:~$ ls -la | grep ^lrw*
lrwxrwxrwx 1 devilx devilx 38 2009-09-12 20:32 .evolution -> /home/devilx/Ubuntu One/Home/evolution
lrwxrwxrwx 1 devilx devilx 36 2009-09-12 20:31 .mozilla -> /home/devilx/Ubuntu One/Home/mozilla
lrwxrwxrwx 1 devilx devilx 35 2009-09-12 20:33 .purple -> /home/devilx/Ubuntu One/Home/purple
devilx@vm-ubuntu:~$


By this, I’ve pushed the preferences and information of my Firefox, my Evolution and my Pidgin into the cloud. A quick test showed me, that none of those three applications complained about their new “home” being a symlink to some directory within the cloud – and neither about any insufficient permissions that could have happened, if the cloud was set-up crappy (+1P for Ubuntu One). Still, this setup seems not that perfect for me, at the moment.

The synchronization of a modified .mozilla and especially of a modified .evolution folder seems to take for years. While I’m writing this text, the client-daemon is still synchronizing the files (it started nearly at the same time I’ve started writing this entry) and says “Updating 6 of 270 files…” – this could become a looooong night. Though, I don’t want to complain about Ubuntu One’s performance, since yet, it’s still in Beta (even if I don’t know, if Beta isn’t just an upcoming trend every company has to stick with -> e.g. Google).

I’ll try to clone this Ubuntu-installation and run them both with Ubuntu One being active – first, one after another (to see, if the applications could be synced without them complaining about parameters, that are incorrect for the secondary host they get synced to -> e.g. hostname information) and then I’ll try to run them simultaneously and hope for the best. If it should work out (what I don’t expect, just from what I’ve seen in similar projects), it would be really cool, because then, most applications could be synced this way without much hassle. I’d be really surprised, if this should be working, because then, the cloud seems to implement some kind of “merge” service, which allows you to upload data from two (or more) clients and seamlessly merge it up in the cloud together to one, consistent state.

Eh, I will see. Altogether, Ubuntu One is yet already working pretty cool, though I’m not quite sure, what a regular user should do with it, if he’s already aware of service like Dropbox or Amazon’s S3 – because at the moment, Ubuntu One unfortunately isn’t any better than those services. It just integrates more seamless into the GNOME-desktop. Hm… oh well.

The current service-site describes the services mainly as synchronization option for your Ubuntu workstations, so that everything is kept up to date. My first thought on this was, that they’re using webdav (just like Apple does for their iDisk) for accomplishing this task, but as it seems, there’s no webdav involved. Maybe it’s built-up on Amazon’s S3?

I don’t know yet, since up to now I haven’t received my invitation to join and use this service. I was told that invitations are being sent amongst others depending on the service’s usage, so I really hope (since the ubuntuone.com isn’t that popular, yet) to receive mine soon.

Depending of how good this service works already, it could be a really good competitor to Apple’s MobileMe – especially if it should get an own “Exchange for the Rest of Us”, heh. I don’t know what’s planned to be implemented and as it seems there’s not much talking about that, but I’d really welcome it to see a working solution on that area.

As soon as I’ll receive the invitation, I will take Ubuntu One on a test-drive and try to see whether it would be possible to use this service on other distributions as well. I really can’t wait to get my hands on the service’s software, heh…

I’m using the service for the first time on a Mac, since at work I only tested out their functionality under linux and had quite some problems with S3. Amazon’s Simple Storage Service is pretty cool regarding their pricing and provides a very acceptable up- and download speed – other than for example my Strato FTP does. The only complicated thing for somebody who’s new in that domain might be the “How do I start?“-point. S3 uses a “web-interface” which communicates using a documented REST-API. Unfortunately, under Linux there aren’t many clients which actually can communicate with this interface. Actually, I’ve only found the S3-Firefox-Plugin, which provided a way to maintain the different buckets and their content. Under OS X, there are some more (mostly Java-based) clients, including the native Cyberduck more-than-FTP client, what surprised my pretty much. This was actually the point where I’ve switched from Panic’s “Transmit” to Cyberduck, by the way.

However, I created my own buckets and uploaded some content to use it as web-share and test out a bit more the speed it provides. One con I’ve found up to now is the bucket-naming. Your personal bucket subjects a global namespace, what can lead to period of time needed for finding a name which is not occupied yet. This makes it for example pretty hard to automize bucket creations, for using them in an own Cloud-Service.

Besides, the EC2 is pretty cool, too. It provides a very flexible and (in comparison to S3) very automatable solution for cloud computing in general. It’s very fast to set up a new instance of whatever linux or windows on EC2 and run the stuff needed on it. The only con is the pricing, I guess – In my opinion it’s way too much for a Xen-I-think-it-is machine. For the price you would pay within one frequent month you could get a real root server at some hosting provider. Of course, you would lose the flexibility, but surely it depends on what you’re looking for.

In total, I’m really stunned of the infrastructure Amazon provides with their EC2/S3/etc. Services and I like the smooth way everything works out on them. I could really think of some cool projects to realize on top of these services… hm.