Image management in Eucalyptus

Managing images in Eucalyptus is generally done by Euca2ools distributed by Eucalyptus team.

All users may upload and register images (depending on access granted to them by the Eucalyptus administrator), but only the admin user may ever upload/register kernels or ramdisks.

Now source the ‘eucarc’ from you ~/.euca directory if you haven’t.

Registering kernel image Execute the following commands to bundle and register the kernel image (vmlinuz-2.6.35-22-server)

shaon@client:~$ euca-bundle-image -i vmlinuz-2.6.32-28-generic --kernel true
shoan@client:~$ euca-upload-bundle -b mybucket -m /tmp/vmlinuz-2.6.32-28-generic.manifest.xml
shaon@client:~$ euca-register mybucket/vmlinuz-2.6.32-28 generic.manifest.xml

Save the output produced by the last command above (eki-XXXXXXXX), which will be needed while registering the disk image.

Registering ramdisk image Execute the following commands to bundle and register the ramdisk image (initrd.img-2.6.35-22-server)

shaon@client:~$ euca-bundle-image -i initrd.img-2.6.32-28-generic --ramdisk true
shaon@client:~$ euca-upload-bundle -b mybucket -m /tmp/initrd.img-2.6.32-28-generic.manifest.xml
shaon@client:~$ euca−register mybucket/initrd.img−2.6.35−22−server.manifest.xml

Save the output produced by the last command above (eri-XXXXXXXX), which will be needed while registering the disk image.

Registering disk image Execute the following commands to bundle and register the ramdisk image ( imagefinal .img)

shaon@client:~$ euca-bundle-image -i imagefinal.img --kernel eki-XXXXXXXX --ramdisk eri-XXXXXXXX
shaon@client:~$ euca-upload-bundle -b mybucket -m /tmp/imagefinal.img.manifest.xml
shaon@client:~$ euca-register mybucket/imagefinal.img.manifest.xml

Replace eki-XXXXXXXX and eri-XXXXXXXX with the exact values you have saved earlier.

To see the the uploaded images

shaon@client:~$ euca-describe-images
IMAGE    eki-7A8D1340    mybucket/vmlinuz-2.6.32-28-generic.manifest.xml    admin    available    public   	 x86_64kernel              instance-store
IMAGE    eri-B603142C    mybucket/initrd.img-2.6.32-28-generic.manifest.xml    admin    available    public   	 x86_64    ramdisk              instance-store
IMAGE    emi-CF6C10B0    mybucket/imagefinal.img.manifest.xml    admin    available    public   	 x86_64    machine    eki-7A8D1340    eri-B603142C    instance-store

Running a custom image

Add a new key-pair to run a new instance with the key.

shaon@client:~$ cd ~/.euca/
shaon@client:~/.euca$ euca-add-keypair jssecacerts > jssecacerts.priv
shaon@client:~/.euca$ chmod 600 jssecacerts.priv
shaon@client:~/.euca$ euca-describe-keypairs

Now run instance with the following command

shaon@client:~/.euca$ euca-run-instances emi-CF6C10B0 -k jssecacerts -t c1.medium

Instance Security

# Check group detail
shaon@client:~/.euca$ euca-describe-groups

# Add new group
shaon@client:~/.euca$ euca-add-group -d "Web Servers" webservers

# Add tcp and icmp permission to the webservers group
shaon@client:~/.euca$ euca-authorize -P tcp -s 0.0.0.0/0 webservers
shaon@client:~/.euca$ euca-authorize -P icmp -s 0.0.0.0/0 webservers
shaon@client:~/.euca$ euca-authorize -P tcp -s 0.0.0.0/0 default
shaon@client:~/.euca$ euca-authorize -P icmp -s 0.0.0.0/0 default
shaon@client:~/.euca$ euca-authorize -P tcp -p 80 default

Tadaa!!

Creating image for Eucalyptus

To create image we will need a raw HDD for the virtual machine. So, set as much space as you need.

kvm-img create -f raw server.img 5G

Get Ubuntu 11.04 Server downloaded from here,

wget http://releases.ubuntu.com/natty/ubuntu-11.04-server-amd64.iso 

To start the installation process, boot a KVM instance with the OS installer ISO in the virtual CD-ROM and set up a VNC port at 0

sudo kvm -m 256 -cdrom ubuntu-11.04-server-amd64.iso -drive file=server.img,if=scsi,index=0 -boot d -net nic -net user -nographic  -vnc :0

Connect to the VM using the Client PC’s IP with VNC port 0 and finish the installation i.e. 10.10.10.4 :0

vncviewer 10.10.10.4 :0

Create a single ext4 partition mounted on ‘/’ during the installation of Ubuntu. Do not create a swap partition.

After finishing the installation relaunch the VM by executing the following command,

sudo kvm -m 256 -drive file=server.img,if=scsi,index=0,boot=on -boot c -net nic -net user -nographic -vnc :0

Now you can add any packages you want. So for our test purpose we will just update and upgrade the system.

sudo apt-get update
sudo apt-get upgrade

Install the following packages as well

sudo apt-get install openssh-server cloud-init

Remove the network persistent rules to make sure that the new network interface eth0 without creating any problem.

$ sudo rm -rf /etc/udev/rules.d/70-persistent-net.rules

Shutdown the VM.

To upload the image on Eucalyptus, it needs to be an ext4 filesystem image. To obtain an ext4 filesystem image, do the following

sudo losetup  -f  server.img
sudo losetup -a
/dev/loop0: [0801]:16908388 ($filepath)

Observe the name of the loop device ( /dev/loop0 in our setup) when $filepath is the path to the mounted .raw file.

Now we need to find out the starting sector of the partition

sudo fdisk -cul /dev/loop0

You should see an output like this

Disk /dev/loop0: 5368 MB, 5368709120 bytes
149 heads, 8 sectors/track, 8796 cylinders, total 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00072bd4
Device Boot Start End Blocks Id System
/dev/loop0p1 * 2048 10483711 5240832 83 Linux

Make a note of the starting sector of the /dev/loop0 partition i.e the partition whose ID is 83. This number should be multiplied by 512 to obtain the correct value. In this case: 2048 x 512 = 1048576

Unmount the loop0 device:

sudo losetup -d /dev/loop0

Now mount only the partition(/dev/loop0) of server.img which we had previously noted down, by adding the -o parameter with value previously calculated value

sudo losetup -f -o 1048576 server.img
sudo losetup -a

You’ll see a message like this

/dev/loop0: [0801]:16908388 ($filepath) offset 1048576

Make a note of the mount point of our device(/dev/loop0 in our setup) when $filepath is the path to the mounted .raw file.

Copy the entire partition to a new .raw file

sudo dd if=/dev/loop0 of=serverfinal.img

Now we have our ext4 filesystem image i.e serverfinal.img.

Unmount the loop0 device

sudo losetup -d /dev/loop0

Tweaking /etc/fstab

You will need to tweak /etc/fstab to make it suitable for a cloud instance.

Loop mount the serverfinal.img

sudo mount -o loop serverfinal.img /mnt

Edit /mnt/etc/fstab and modify the line for mounting root partition(which may look like the following)

UUID=e7f5af8d-5d96-45cc-a0fc-d0d1bde8f31c  /           ext4    errors=remount-ro  0       1

to

LABEL=uec-rootfs              /          ext4           defaults     0    0

Kernel and Initrd

Copy the kernel and the initrd image from /mnt/boot to user home directory. These will be used later for creating and uploading a complete virtual image to Eucalyptus.

sudo cp /mnt/boot/vmlinuz-2.6.38-7-server /home/shaon
sudo cp /mnt/boot/initrd.img-2.6.38-7-server /home/shaon

Unmount the Loop partition

sudo umount  /mnt

Change the filesystem label of serverfinal.img to ‘uec-rootfs’

sudo tune2fs -L uec-rootfs serverfinal.img

Now, we have all the components of the image ready to be uploaded to Eucalyptus server.

Eucalyptus installation

For installation three two PC (Server1 and Server2) is needed for the cloud purpose and one PC is for client which will also be serving for creating KVM images. Server2 and the client machine should have VT enabled as we will be running all our VMs on Server2 and client PC will be using to create the necessary KVM images.

Required configuration

Required Setup

Server1 setup

• Boot the Ubuntu 11.04 64 bit Server CD/pen drive and from the graphical menu, select Ubuntu Enterprise Cloud and follow the installation menu.
• If you are using DHCP for the public network, then just select eth0 and let the network to be setup automatically. Otherwise set your ethernet as mentioned above.
• When the installation will be asking for the Cloud Controller address just put it blank.
• For Server1 it’ll install the ‘Cluster Controller’, ‘Walrus Storage Service’, ‘Cluster Controller’ and ‘Storage Controller’.
• Select eth1 for communication with nodes
• Eucalyptus cluster name – Cluster1 (on anything)
• Select an IP range to be used for the nodes, i.e. 192.168.1.10-192.168.1.99

Post installation setup

Set up static IP for eth1, edit /etc/network/interfaces and add the following to it,

auto eth1
iface eth1 inet static
address 192.168.20.1
netmask 255.255.255.0
network 192.168.20.0
broadcast 192.168.20.255

Run the following command to restart the networking,

localadmin@server1:~$ sudo /etc/init.d/networking restart

Update and upgrade the Eucalyptus to get the latest version of it,

localadmin@server1:~$ sudo apt-get update
localadmin@server1:~$ sudo apt-get upgrade eucalyptus

Install NTP package. Server1 is going to act as an NTP server for the nodes.

localadmin@server1:~$ sudo apt-get install ntp

Open and edit /etc/ntp.conf to make sure that the server serves time even when it’s connectivity to the internet is down. Add the following line to the file so that NTP server uses it’s own clock source.

server 127.127.1.0
fudge 127.127.1.0 stratum 10

Restart NTP server to make the changes active

localadmin@server1:~$ sudo /etc/init.d/ntp restart

Restart the Cluster Controller

localadmin@server1:~$ sudo restart eucalyptus-cc CLEAN=1

[NTP stands for Network Time Protocol, and it is an Internet protocol used to synchronize the clocks of computers to some time reference]

Server2 setup

• Boot Ubuntu 11.04 64 bit Server and select ‘Install Ubuntu Enterprise Cloud’ and continue the basic installation process.
• For network setup select eth0 and configure it manually. Set the private IP to 192.168.20.2 and the gateway as 192.168.20.1.
• For UEC setup it’ll ask certain configuration option. If it doesn’t select the Cluster Controller by itself, put the the Cluster Controller address 192.168.20.1
• In cloud installation mode select ‘Node Controller’

Post installation setup

Set up static IP for eth1 by adding the few lines to /etc/network/interfaces so that it looks like following

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
    address 192.168.20.2
    netmask 255.255.255.0
    network 192.168.20.0
    broadcast 192.168.20.255
    # gateway 192.168.20.1
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 192.168.1.1
    bridge_ports eth0
    bridge_fd 9
    bridge_hello 2
    bridge_maxage 12
    bridge_stp off

auto eth1
iface eth1 inet static
address 192.168.1.103
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

Run the following command to restart the networking,

localadmin@server1:~$ sudo /etc/init.d/networking restart

Update and upgrade the Eucalyptus to get the latest version of it,

localadmin@server1:~$ sudo apt-get update
localadmin@server1:~$ sudo apt-get upgrade eucalyptus

Install NTP package.

localadmin@server1:~$ sudo apt-get install ntp

Open the file /etc/ntp.conf and add the following line

server 192.168.20.1

Restart NTP server to make the changes active

localadmin@server1:~$ sudo /etc/init.d/ntp restart

Open the file /etc/eucalyptus/eucalyptus.conf and make the following changes,

VNET_PUBINTERFACE=”br0”
VNET_PRIVINTERFACE=”br0”
VNET_BRIDGE=”br0”
VNET_DHCPDAEMON=”/usr/sbin/dhcpd3”
VNET_DHCPUSER=”dhcpd”
VNET_MODE=”MANAGED−NOVLAN”

Now run the following command to restart the Node Controller to make all the changes active,

localadmin@server2:~$ sudo restart eucalyptus-nc

Setup CC’s SSH public key to NC

On the Node Controller, temporarily set a password for the “eucalyptus” user,

localadmin@server2:~$ sudo passwd eucalyptus

On the Cluster Controller:

localadmin@server1:~$ sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub eucalyptus@192.168.20.2

Remove the password of the “eucalyptus” account from the Node,

localadmin@server1:~$ sudo passwd -d eucalyptus

Client setup

• Boot the 11.04 32/64 bit Desktop and install it.

Install KVM on the client machine.

shaon@client:~$ sudo apt-get install qemu-kvm

Post installation setup

To administrate the cloud we need to install euca2ools

shaon@client:~$ sudo apt-get install euca2ools

Monitoring

• Login to the web interface from https://192.168.10.121:8443, default username is ‘admin’ and password is ‘admin’.
• Download the user credentials from credential tab and save it to ~/.euca directory (if .euca is not there just create and save the credentials there)
• Extract the credentials and source the eucarc script so that euca2ools can used this as environmental variables.

$ cd .euca
$ unzip xxxxxxxxx.zip
$ source eucarc

Verify that euca2ools can communicate with the UEC properly and all the services are running correctly run the following command,

$ euca-describe-availability-zones verbose

It’ll give output something like this,

AVAILABILITYZONE	cluster1	192.168.1.102
AVAILABILITYZONE	|- vm types	free / max   cpu   ram  disk
AVAILABILITYZONE	|- m1.small	0001 / 0002   1    192     2
AVAILABILITYZONE	|- c1.medium	0001 / 0002   1    256     5
AVAILABILITYZONE	|- m1.large	0000 / 0001   2    512    10
AVAILABILITYZONE	|- m1.xlarge	0000 / 0001   2   1024    20
AVAILABILITYZONE	|- c1.xlarge	0000 / 0000   4   2048    20

If VCPUs are found 0000, then use the following command on the Server1 to make sure that it finds the Node Controller and approve when prompts 192.168.20.2

localadmin@server1:~$ sudo euca_conf --discover-nodes

tadaa!!!

Eucalyptus and it’s components

Eucalyptus

Eucalyptus is an open source Linux based software architecture which provides an EC2-compatible cloud computing platform and S3-compatible cloud storage platform. It implements scalable, efficient-enhancing and private and hybrid clouds within and organization’s IT infrastructure. It gives an Infrastructure as a Service (IaaS) solution. Users can use commodity hardware.

Eucalyptus was developed to support the high performance computing (HPC). Eucalyptus can be deployed without modification on all major Linux OS distributions, including Ubuntu, RHEL/CentOS, openSUSE, and Debian.
Eucalyptus Features

For implementing, managing and maintaining the virtual machines, network and storage Eucalyptus has variety of features.

• SSH Key Management
• Image Management
• Linux-based VM Management
• IP Address Management
• Security Group Management
• Volume and Snapshot Management

Eucalyptus Fundamental Architecture

Components of Eucalyptus:

1. Cluster Controller (CC) Cluster Controller manages the one or more Node controller and responsible for deploying and managing instances on them. It communicates with Node Controller and Cloud Controller simultaneously. CC also manages the networking for the running instances under certain types of networking modes available in Eucalyptus.

2. Cloud Controller (CLC) Cloud Controller is front end for the entire ecosystem. CLC provides an Amazon EC2/S3 compliant web services interface to the client tools on one side and interacts with the rest of the components of the Eucalyptus infrastructure on the other side.

3. Node Controller (NC) It is the basic component for Nodes. Node controller maintains the life cycle of the instances running on each nodes. Node Controller interacts with the OS, hypervisor and the Cluster Controller simultaneously.

4. Walrus Storage Controller (WS3) Walrus Storage Controller is a simple file storage system. WS3 stores the the machine images and snapshots. It also stores and serves files using S3 APIs.

5. Storage Controller (SC) Allows the creation of snapshots of volumes. It provides persistent block storage over AoE or iSCSI to the instances.

Eucalyptus Architecture

Google Easter Eggs or Nerd Jokes

A new Easter Egg of Google has revealed lately. Just go to Google and type in “do a barrel roll” and search. Enjoy the spinning!

This came from a Nintendo 64 game “Star Fox”, where one of the game character Peppy asks the user to do a barrel roll. It also gives the user another option “z or r twice”, guess what? this also does the trick. Type in “z or r twice” and search.

So now you do a roll on google but that’s not all. Type in “tilt” or “askew“, see what happens!

Type “anagram” and search, google will show you “Did you mean: nag a ram”, which is a anagram of the word anagram.

Well, do you know the spelling of “recursion”, search with “recursion”, looks like you are searching recursively as you will always find “Did you mean: recursion”.

Do you know what is the loneliest number? Type in “the loneliest number”, google calculator will return 1.

Let’s find the answer to the ultimate question of life, the universe, and everything. Just type in “the answer to the ultimate question of life, the universe, and everything” or “the answer to life, the universe, and everything”, google calculator will return 42.

This was first found in the novel The Hitchhiker’s Guide to the Galaxy by Douglas Adams. The writer was asked so many times why he chose 42, and finally he replied “I sat at my desk, stared into the garden and thought ’42 will do’ I typed it out. End of story.” Anyway, it  is now a modern internet phenomena. Another funny thing happened with this number.

Searching “a bakers dozen” will result 13. It’s history of 13th century during the reign of Henry III, if a a baker was found to have shortchanged customers could be subject to loose their hand to an axe. So to avoid that bakers would give 13 for the price of 12 as an insurance to be certain of not being known as a cheat.

“once in a blue moon” returns 1.16699016 × 10^-8 hertz.

Type in “where is chuck norris” and click I’m feeling lucky and you are there.

Also type in “epic google” and press I’m feeling lucky and wait!

Similar searches: (press I’m feeling lucky)
Google sphere
Google gravity
Google reverse/mirror
weenie google
Rainbow google
Google loco
Annoying Google
Google pacman

Feel free to add if you know any other