Run Appscale on Eucalyptus

Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet). – Wikipedia

According to Wikipedia currently there are few popular service models exist.

1. Infrastructure as a service (IaaS)
2. Platform as a service (PaaS)
3. Software as a service (SaaS)

So, I have an Eucalyptus cloud, which is great, serves as AWS-like IaaS platform. But now I want PaaS. And right here Appscale comes into play with full compatibility of Google App Engine (GAE) applications. In this post, we will install the popular open source PaaS framework Appscale on Eucalyptus, the AWS compatible open source IaaS platform.

Agenda
0. Introduction
1. Resize Lucid image
2. Install Appscale from source
3. Install Appscale Tool
4. Bundle Appscale image
5. Run Appscale
6. Run an application on Appscale

Eucalyptus
Eucalyptus Cloud platform is open source software for building AWS-compatible private and hybrid clouds. It pools together existing virtualized infrastructure to create cloud resources for compute, network and storage.

Appscale
AppScale is an open-source framework for running Google App Engine applications. It is an implementation of a cloud computing platform (Platform-as-a-Service), supporting Xen, KVM, Amazon EC2 and Eucalyptus. It has been developed and is maintained by AppScale Systems.

Resize Lucid image

Download the Lucid image,
wget http://cloud-images.ubuntu.com/lucid/current/lucid-server-cloudimg-amd64.tar.gz

To install Appscale we need a bigger space than it comes by default with the image. Let’s increase the root size of the image,

dd if=/dev/zero of=lucid-server20g.img bs=1M count=20480

mkfs.ext3 -F lucid-server20g.img

mkdir new
mkdir old

mount -o loop lucid-server-cloudimg-amd64.img old/
mount -o loop lucid-server20g.img new/

(cd old; tar czf - .) | (cd new; tar xzf -)

umount old
umount new

[detail from Eucalyptus knowledge base article]

So, the problem I was facing, the modified image wasn’t working properly with the kernel/ramdisk those came with tar file. My colleague Deependra figured out the problem gets resolved with the latest kernel. So, you can either start up an instance upgrade with the 3.0.0-31 kernel or download the updated kernel and ramdisk from the following link as long as they are there,

http://dl.dropbox.com/u/5458574/initrd.img-3.0.0-31-server
http://dl.dropbox.com/u/5458574/vmlinuz-3.0.0-31-server

Now follow Eucalyptus Administration Guide to add the image.

Run an instance to build the Appscale image,

euca-run-instances -k sshlogin -t m1.large emi-7A453BC1

Install Appscale from source

apt-get install -y git-core
git clone git://github.com/AppScale/appscale.git --branch master
cd appscale/debian
bash appscale_build.sh

Install Appscale Tool

git clone git://github.com/AppScale/appscale-tools.git --branch master
cd appscale-tools/debian
bash appscale_build.sh

export PATH=$PATH:/usr/local/appscale-tools/bin

add-apt-repository "deb http://downloads.eucalyptus.com/software/euca2ools/2.1/ubuntu lucid main"

Bundle Appscale image

Now bundle the instance and register it,

euca-bundle-vol -p appscale-1-6-7-8g -d ./ -e /var/lib/dhcp3 -s 8192
euca-upload-bundle -b demo -m ./appscale-1-6-7-8g.manifest.xml
euca-register demo/appscale-1-6-7-8g.manifest.xml

Run Appscale

euca-run-instances -k sshlogin -t m1.large emi-D4A33E62

Now login to the Appscale instance. Run the following to create the configuration file,

appscale init cloud

Now modify the AppScalefile file to be used by Appscale.

So for this demo, our minimal configuration file looks like following,

infrastructure : 'euca'
machine : 'emi-D4A33E62'
instance_type : 'c1.xlarge'
table : 'cassandra'
keyname : 'appscalekey'
group : 'appscalegroup'
verbose : True
min : 3
max : 3

The machine would be the emi which has Appscale installed on it, instance_type we prefer to have the biggest possible one, for this demo we will be using default table, we turned on the verbose flag on, since this is a test environment, we also interested in the power of cloud service, so we set min, max to 3, this will spawn three instances and replicate data base 3 times.

So, now we are all set to start the Appscale platform. Run the following to start up Appscale,

appscale up

So after all those verbose output, now we see the following,

AppScale successfully started!
View status information about your AppScale deployment at http://10.104.3.129/status


And if we check http://10.104.3.129/status, we should see the healthy status

Appscale API status

Run an application on Appscale

We will be running a simple chat application that is available on the google app engine samples. So we save the following app and unzip it,
https://google-app-engine-samples.googlecode.com/files/simpleajaxchat_10312008.zip

Run the following command to deploy the first application,

appscale deploy ~/samples/simpleajaxchat/

And the happy line would look like the following,
Your app can be reached at the following URL: http://10.104.3.129:8080

Sample chat application

And now we have our first application hosted on Appscale PaaS platform.

For more monitoring, login the the Appscale console and select monitoring data.

Appstale status from command line,

appscale status

For troubleshooting Appscale comes with a nice tool. Run the following to gather all the logs at one place,

appscale logs /path/filename

Login to the master node,

appscale ssh

Shutdown Appscale,

appscale down

There is a lot more available on github,
https://github.com/AppScale/

Also you can submit your issues here,
https://github.com/AppScale/appscale/issues

Try out the latest Eucalyptus code from github,
https://github.com/eucalyptus/eucalyptus

We are always open to your comments and feedback,
https://eucalyptus.atlassian.net/
https://engage.eucalyptus.com/

Eucalyptus HA and lock files

Eucalyptus comes with its own High Availability (HA) feature from version 3.0 and with the passage of time, it is becoming more and more stable with every release.

With recent release (3.2.0 and onwards), it has got few behavioral changes, which of course makes it more robust. To describe it in one sentence, “Lock files are to prevent data loss during HA failover”. So, lock files are to ensure that Cloud Controller (CLC) database synchronization happens properly and in any case it does not end-up in split-brain situation.

In the recent versions, Eucalyptus introduces two kinds of lock files, disabled.lock and partitioned.lock.

disabled.lock
This file is generated to enforce the correct order of starting CLC services. So, if you stop the disabled CLC disabled.lock file is generated, then you stop the enabled CLC and try to start the disabled CLC, it won’t start, unless the cloud administrator enforce by deleting the disabled.lock file manually.

partitioned.lock
On the other hand this file is generated basically to protect split brain situation. It is to prevent two CLCs to go into enabled state. When ever there is a network outage between CLC <-> CLC, the partitioned.lock file is generated. If we see more closely, when the database becomes inaccessible, the partitioned.lock file is generated.

When the CLC fails to start because of disabled.lock or partitioned.lock Eucalyptus populates logs in cloud-fault.log. This basically represents that Eucalyptus cannot proceed by itself any further and it requires manual involvement of cloud administrator.

These are the basic use cases of both lock files. partitioned.lock is little more aggressive than disabled.lock. It actually has few more use cases, which I am yet to figure out though. In brief, when the database is killed on the enabled CLC a partitioned.lock can be generated or when the public interface is down on the enabled CLC it generates partitioned.lock file, it happens even if it’s a non-HA setup. Another case would be, if the enabled CLC cannot talk to it’s database and it finds another database on a different host Eucalyptus generates partitioned.lock.

This may not be all, the code is being improved everyday, but this is the last known behavior of the lock files.

Comments and suggestions are most welcome.

Eucalyptus and Nagios

Reblogged from nurmiblog:

Production deployments of Eucalyptus, like production deployments of any infrastructure software running in a data center, require some amount of health and status monitoring be happening in order to both allow the Eucalyptus/data-center administrator the ability to stay on top of evolving resource situations and to provide invaluable diagnostic information when something is going sideways within the resource pool.  Fortunately for all of us, there exists a wide variety of health/status monitoring system out there, and several of them are of extremely high quality, tried and tested, and are available as part of major Linux distributions as pre-packaged open-source solutions.  

Read more… 1,429 more words

Monitoring with Nagios

With the growing number of servers, monitoring often becomes harder day by day. There are various solutions for this. Two very popular I have seen around are Zenoss and Nagios.

Well, here comes the very common question. Which one to use? The best answer would be, the one with less learning and more features Most often the combination isn’t so easy to achieve.

There are so many comparisons around monitoring tools. I have found these two relevant to this post, one from Linux.com and another from Zenoss.org Help yourself with the links, I am not going to make any comparison between these two.

At Eucalyptus, we have customers who run a lot of servers and monitoring tool is not another fancy product for them, rather it’s a necessity.

But deploying a quick Nagios monitoring system on CentOS is only few steps away. It should not be any different on other distros as well.

Install Nagios packages,

yum install -y nagios nagios-plugins nagios-plugins-nrpe nagios-plugins-all

Set password for Nagios admin,

htpasswd /etc/nagios/passwd nagiosadmin

Start the services and update runlevel for each of them,

chkconfig nagios on
chkconfig httpd on
service nagios start
service httpd start

Define host and services

vim /etc/nagios/objects/host1.cfg
# Define a host

define host{
        use                     linux-server
        host_name               host1
        address                 10.104.3.128
        }

# Define a service

define service{
        use                             generic-service
        host_name                       host1
        service_description             PING
        check_command                   check_ping!100.0,20%!500.0,60%
        }

Add the following line to nagios.cfg,

cfg_file=/etc/nagios/objects/host1.cfg

Reload Nagios service,

service nagios reload

Check Nagios admin panel at http://<ip_address>/nagios

Tada!!

And of course, it is released under the GPL license.

Contributing to Eucalyptus made easy

The power of open source is incredible. Don’t believe me? Click here or here Contribution is something that makes open source projects most powerful. It has been proven with Linux a long time ago. Also, giant companies have started to believe in open source. Open source is not just about making a product free to use, what I believe is, it’s about giving users the freedom to do whatever they want to do with a product, with the source code.

But sometimes, I feel, is it always that easy to contribute? Well, that’s certainly a question. Being an user, it often needs some guidance to get started with something. Why do I believe that? Well, you will know, if you are in such a place, where you are the only person doing something alone

Eucalyptus comes with such a license, that allows its users to modify the codes as they want and if someone wants to contribute, s/he is always welcome to do so.

Detail on Contributing Code to Eucalyptus

So, how do we start?

Here is the few thing that we will be using,
1. A machines, with CentOS 6.3 installed
2. src.rpm of Eucalyptus 3.1.2

Set up build dependency repository,

cat <<EOF >/etc/yum.repos.d/builddeps.repo
[builddeps]
name=builddeps
baseurl=http://downloads.eucalyptus.com/software/eucalyptus/build-deps/3.1/centos/6/x86_64/
gpgcheck=0
enabled=1
EOF

Configure the EPEL package repository,

yum install http://downloads.eucalyptus.com/software/eucalyptus/3.1/centos/6/x86_64/epel-release-6.noarch.rpm -y 

Download the source RPM,

wget -nd http://downloads.eucalyptus.com/software/eucalyptus/3.1/centos/6/source/eucalyptus-3.1.2-0.210.el6.src.rpm

Download and install build-dependencies,

yum install -y rpm-build gcc make

Install the build dependencies for the source RPM,

yum-builddep --nogpgcheck -y eucalyptus-3.1.2-0.210.el6.src.rpm

Install the source RPM,

rpm -ivh eucalyptus-3.1.2-0.210.el6.src.rpm

At this point we have eucalyptus.spec file ready. Run the following commands to get the bits in the ~/rpmbuild/BUILD directory.

cd ~/rpmbuild/SPECS
rpmbuild -bp eucalyptus.spec

Make a patch from the original bits,

cp -r eucalyptus-3.1.2 eucalyptus-3.1.2.orig
vim eucalyptus-3.1.2/<file_that_needs_aid>
diff -Npru eucalyptus-3.1.2.orig eucalyptus-3.1.2 > <name_the_patch>.patch
cp <name_the_patch>.patch ~/rpmbuild/SOURCES/

Now, modify the spec file from the ~/rpmbuild/SPECS/ and add the patch into it.

Build SRPMs and RPMs,

rpmbuild -ba eucalyptus.spec

Test your newly created RPMs and contribute to the community.

Happy patching!

Contribution is not only limited to source code.

File bug reports: eucalyptus.atlassian.net

Send a pull request on Github: https://github.com/eucalyptus/

Also, tell us about your problem with Eucalyptus on Engage: engage.eucalyptus.com

Eucalyptus Source Installation

Eucalyptus 3.2 is becoming ready. It is going to be out soon. But if you think, it’s too hard for you to wait till GA, there are two solutions for you to try out. You can check out nightly builds or go for source installation, which is *fairly* easy.

For this installation I am using two CentOS 6.3 boxes, one as Frontend (CLC/Walrus/CC/SC) and another for backend (NC).

Download the source code from Github,

 # git clone git://github.com/eucalyptus/eucalyptus --recursive 

Configure repositories,

 # yum --nogpg install http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm -y 

[walrus host only]

 # yum --nogpg install http://elrepo.org/linux/elrepo/el6/x86_64/RPMS/elrepo-release-6-4.el6.elrepo.noarch.rpm -y 

Create a eucadeps.repo file for eucalyptus dependencies,

[euca-deps]name=euca-deps
baseurl= http://downloads.eucalyptus.com/software/eucalyptus/build-deps/3.1/rhel/$releasever/$basearch
gpgcheck=0
enabled=1

[euca-runtime]
name=euca-deps
baseurl= http://downloads.eucalyptus.com/software/eucalyptus/runtime-deps/3.1/rhel/$releasever/$basearch
gpgcheck=0
enabled=1

[euca2ools]
name=euca2ools
baseurl= http://downloads.eucalyptus.com/software/euca2ools/2.1/centos/$releasever/$basearch
gpgcheck=0
enabled=1

Install the required dependencies,

 # yum install gcc make ant ant-nodeps axis2-adb-codegen axis2-codegen axis2c-devel jpackage-utils libvirt-devel libxml2-devel libxslt-devel python-devel python-setuptools rampartc-devel swig velocity xalan-j2-xsltc gawk java-devel java-devel openssl-devel curl-devel axis2c dhcp41 dhcp41-common postgresql91 postgresql91-server python-boto rampartc iscsi-initiator-utils perl-Crypt-OpenSSL-RSA perl-Crypt-OpenSSL-Random sudo which jpackage-utils java lvm2 velocity drbd83 drbd83-kmod drbd83-utils drbd euca2ools bitstream-vera-fonts dejavu-serif-fonts bridge-utils iptables vtun device-mapper coreutils e2fsprogs file parted util-linux rsync PyGreSQL libcurl curl httpd libvirt scsi-target-utils vconfig wget kvm patch 

Download euca-WSDL2C.sh and save it to /opt on both machines,

 # wget https://raw.github.com/eucalyptus/eucalyptus-rpmspec/master/euca-WSDL2C.sh -O /opt/euca-WSDL2C.sh 

Download and save eucalyptus-nc-libvirt.pkla to /var/lib/polkit-1/localauthority/10-vendor.d/

 # wget https://raw.github.com/eucalyptus/eucalyptus/master/tools/eucalyptus-nc-libvirt.pkla -O /var/lib/polkit-1/localauthority/10-vendor.d/eucalyptus-nc-libvirt.pkla 

Add user “eucalyptus” on frontend,

 # adduser eucalyptus 

On NC,

 # useradd -G kvm eucalyptus 

Lets build Eucalyptus source now.

Set the following paths on both machines,

# export EUCALYPTUS="/"

Now start compiling Eucalyptus on both machines. Go to the eucalyptus directory and run the following,

 # ./configure '--with-axis2=/usr/share/axis2-*' --with-axis2c=/usr/lib64/axis2c --prefix=$EUCALYPTUS --with-apache2-module-dir=/usr/lib64/httpd/modules --with-db-home=/usr/pgsql-9.1 --with-wsdl2c-sh=/opt/euca-WSDL2C.sh 

 # make clean ; make ; make install 

Add the following in /etc/sysconfig/network,

 NOZEROCONF=yes 

On NC run the following,

# modprobe kvm_intel
# modprobe vhost_net

Configure eucalyptus.conf on both machines.

Run the following on both machines to set permission properly,

 # su -c "euca_conf --setup" 

On Frontend, initialize database and start eucalyptus-cloud service, which will start Cloud Controller, Walrus, Storage Controller, as we have put all in one box. And then start the Cluster Controller. Finally, start Node controller.

# euca_conf --initialize
# service eucalyptus-cloud start

# service eucalyptus-cc start

# service eucalyptus-nc start

Register Eucalyptus components,

# euca_conf --register-walrus --partition walrus --host 10.104.1.179 --component W00
# euca_conf --register-cluster --partition cluster01 --host 10.104.1.179 --component CC00
# euca_conf --register-sc --partition cluster01 --host 10.104.1.179 --component SC00

From 3.2.0, it is a must to set storage manager,

 # euca-modify-property -p cluster01.storage.blockstoragemanager=overlay 

Register node,

 # euca_conf --register-nodes 10.105.1.180 

Good Luck!

Amadeyr Cloud Ltd. partners with Eucalyptus System Inc.

Amadeyr Cloud Ltd. a social enterprise aimed at bridging the literacy and digital divide in Bangladesh and maximizing information efficiency by the use of information technology, has signed a partnership contract with Eucalyptus Systems, creators of the most widely deployed open source on-premise infrastructure as a service (IaaS) cloud platform.

For more than 2 years Amadeyr Cloud Ltd. is building it’s own infrastructure on open source technologies. Amadeyr Cloud (ACL) has introduced Amadeyr Tablet, android hand held devices that have been specifically designed and customized by ACL for use by semi literate, and illiterate users. Amadeyr Cloud has successfully run multiple pilot projects in rural areas demonstrating access to information through audio and visual cues.

About Amadeyr Cloud
Amadeyr Cloud Ltd. (ACL) was founded by a group of professionals whose expertise focuses on infrastructure projects, project financing, public policy, legal and regulatory frameworks, and enabling creative solutions to complex problems.  ACL has over the past 2 years focused on solutions to the problem of bridging the digital and literacy divide in Bangladesh and maximizing information efficiency.
Upon the request of the Government of Bangladesh (GOB), ACL has conceptualized and developed a computerized model for the distribution of farm subsidies to the farmers of Bangladesh.  The “Digital Krishi Subsidy Distribution System” or “Agri Input Card/Krishi Card” developed by ACL has been delivered to the GOB for implementation.

About Eucalyptus Systems
Eucalyptus Systems provides IT organizations in enterprises, government agencies and Web and mobile businesses with the most widely deployed cloud software platform for on-premise Infrastructure-as-a-Service (IaaS). To date, over 25,000 Eucalyptus clouds have been started all over the world, including more than 20 percent of Fortune 100 companies. Eucalyptus is specifically designed for enterprise cloud use, and the software platform is uniquely suited for private cloud or hybrid cloud computing. Built as an open source cloud cloud product, Eucalyptus supports the industry-standard Amazon Web Services (AWS) cloud APIs, as well as all major virtualization platforms including Xen, KVM and VMware vSphere, ESX and ESXi. The company has an active and growing ecosystem of customers, partners, developers and researchers that benefit from Eucalyptus’ open, fast and standards-compliant path to cloud computing. For more information about Eucalyptus, please visit www.eucalyptus.com.