Pursuit of Quality at Eucalyptus

At Eucalyptus, if one thing we care about it is the quality of the product. Our goal is to deliver an AWS compatible software that just works. To ensure the highest level of quality we try to follow the optimum strategy possible in both Development and in QA.

We have adopted Agile Software Development model a while back and we love it. As a part of our strategy, there are several type of projects we open in Jira. After getting the green signal from Product Management, we get Epic and then after architectural discussion Story tickets are created with Sub-tasks for developers to work on. We also have Jira tickets for New Features which are basically similar to Story but comparatively smaller tasks and generally have no Sub-tasks. At this point one scrum master from each team works closely with the developers to keep track of the progress. Then comes Bugs, everyone in the company or outside the company can report bugs, of course they can open new features as well. Typically, the workflow is very simple, all the bugs becomes confirmed either by the Tier3 members or in the bug scrub meetings. Then a developer gets assigned to a bug and the bug status gets from In Progress to In QA and after successful verification it goes into Release Pending state by a Quality Engineer. At this point the bits are ready to be merged in the master branch. And YES! Everything we do is publicly visible, we love Open Source!

As Quality Engineers, our job is to be in the process where a Story or New Feature or Bug’s status becomes In QA.

Eucalyptus is a complex piece of software and like most other system application out there, it is necessary to check not only the bug or any specific feature but also the entire product and with continuous development the testing is also a continuous process. Here, I will quote from one of colleagues, Kyo Lee, “Q: When do you stop testing? A: You never stop testing.” Achieving this goal would never be possible if we would plan this to do manually (or multiply the number of employees by 4 times?). Here comes Eutester, Eutester4J and se34euca, test frameworks built for Eucalyptus, but not limited to Eucalyptus.

Eutester is a functional testing framework for Eucalyptus written in Python. It uses Boto, a python interface to Amazon Web Services to ensure the AWS compatibility. Most of the Eucalyptus services are covered with various test cases which are written on top of Eutester. On the other hand, Eutester is definitely not limited to Eucalyptus functional testing, there are scripts to help with installation of Eucalyptus, Riak CS etc. It is also used to collect logs and various configuration, artifacts from the Eucalyptus hosts to debug issues. There are test cases for both administrators and users. Also, many test cases are built to reproduce and verify bugs. All the test cases are found on Eutester github repository.

For Java developers we have Eutester4J, another test framework built with AWS Java SDK, yet another product to test AWS Java SDK compatibility a.k.a AWS compatibility. It uses TestNG to build various test suites for Eucalyptus. For dependency management, it uses Ivy to make life easier for all.

Now to test the user console we have this fantastic project that we call se34euca, which uses Selenium and automates the entire process of testing the user console.

Well, we have all these projects, but how do we achieve automation at it’s highest level? The answer is Jenkins! We use Jenkins to automate all the test cases. We took the advantage of continuous integration to automate the test frameworks. Now the answer leads us to more questions, “how/where do I install Jenkins? how do I integrate those frameworks?” Yes, to answer all those questions, we have MicroQA. It’s the home of all those frameworks. MicroQA uses Vagrant (yet another cool tool, eh?) to speed up the installation. Yes, it is even possible to test Eucalyptus cloud from your laptop.

With all these, what we never forget is Manual Testing. We do test manually, mainly when a new feature becomes available, sometimes when we verify certain bugs, manual testing is unavoidable. We use open source test management software Testlink to manage manual test cases.

Last but not least, unlike typical tech organizations, we work very closely with developers, we all make sure we understand our job which is not to count the number of bugs fixed/raised/verified, nether the number of features/test cases, rather it is to deliver a quality product.

live-long-and-prosper-tee-shirt-cbs114b

jquery AJAX with Django

For my current project I am working on Django. But implementing ajax was little uncomfortable for me in the first place.

I wanted to use POST method but in Django there is a Cross Site Request Forgery protection(csrf) which don’t allow developers to implement it directly. The csrf protection also protect the PUT and DELETE.

So to enable the protection and work with those methods we need to go through certain steps. For forms the steps is pretty much simple, just add this {% csrf_token %} after <form>.

But to use it with jquery and ajax it’s little different. At first I tried to implement $.post() as it wasn’t working because of csrf error, I added a .js file and I was happy with it (btw, I was using Firefox until then). but then when I tried to view the application on Google Chrome it was showing our very known HTTP 403 error. And a little tracing lead me to some more detail.

jQuery.ajaxTransport.sendjquery.js:8088
jQuery.extend.ajaxjquery.js:7566
jQuery.each.jQuery.(anonymous function)jquery.js:7092
(anonymous function):8000/:198
jQuery.Callbacks.firejquery.js:1032
jQuery.Callbacks.self.fireWithjquery.js:1150
jQuery.extend.readyjquery.js:421
DOMContentLoaded

Little more reading lead me to some real fix. For ajax call, instead of using $.post() I just had to use $.ajax() and set an additional parameter.

$.ajax({
    url: "/test/test_method",
    success: function(data) {
        ...do something..
    },
    crossDomain: false
});

So basically the above code sends a GET request by default, but I was trying to use POST method. So I used the ajax csrf script from Django doc. This script sets a custom X-CSRFToken header to the value of the CSRF token on each XMLHttpRequest.

jQuery(document).ajaxSend(function(event, xhr, settings) {
    function getCookie(name) {
        var cookieValue = null;
        if (document.cookie && document.cookie != '') {
            var cookies = document.cookie.split(';');
            for (var i = 0; i < cookies.length; i++) {
                var cookie = jQuery.trim(cookies[i]);
                // Does this cookie string begin with the name we want?
                if (cookie.substring(0, name.length + 1) == (name + '=')) {
                    cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                    break;
                }
            }
        }
        return cookieValue;
    }
    function sameOrigin(url) {
        // url could be relative or scheme relative or absolute
        var host = document.location.host; // host + port
        var protocol = document.location.protocol;
        var sr_origin = '//' + host;
        var origin = protocol + sr_origin;
        // Allow absolute or scheme relative URLs to same origin
        return (url == origin || url.slice(0, origin.length + 1) == origin + '/') ||
            (url == sr_origin || url.slice(0, sr_origin.length + 1) == sr_origin + '/') ||
            // or any other URL that isn't scheme relative or absolute i.e relative.
            !(/^(\/\/|http:|https:).*/.test(url));
    }
    function safeMethod(method) {
        return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
    }

    if (!safeMethod(settings.type) && sameOrigin(settings.url)) {
        xhr.setRequestHeader("X-CSRFToken", getCookie('csrftoken'));
    }
});

add these three MIDDLEWARE_CLASSES

'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.csrf.CsrfResponseMiddleware',

and tada!!! it’s working on all the browsers.

Setting up django using virtualenv

Still I remember, once I was crying(!) to people to teach me virtualenv 😛 It’s actually I didn’t know that what it does, how it does and what are the benefits of using virtualenv.

Anyway, there are few and very important benefits of using virtualenv. This fantastic tool gives theh power of creating isolated Python environment. Keeps the headache of permission, version of the applications, dependencies etc.

The funnies thing about virtualenv is that, it is enormously easy to setup. YES, I mean it 🙂

sudo easy_install virtualenv

Now go to the directory where you want to create the project and type

virtualenv --no-site-packages vdjango

now install django

./vdjango/bin/easy_install django

Start a django project

./vdjango/bin/django-admin.py startproject myapp

now to the project directory and run the project.

../vdjango/bin/python manage.py runserver 0.0.0.0:8000

tada!!!!

virtualenv – EnvironmentError: mysql_config not found

So I was setting up virtualenv, during the setup process I was having this very common(!) error when I was trying to install MySQL-python. The solution is bit tricky. Though I found solution like, setting PATH on .profile, but actually that might work for normal installation.

I am on Mac OS 10.6.8 and for me the mysql_config file was in /usr/local/mysql/bin/ directory.

Edit the /bin/activate file from the virtualenv directory and add the following lines,

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
PATH="$PATH:/usr/local/mysql/bin/"
export PATH

then run the following command

source virtualenvDirectory/bin/activate

this will change your path to virtualenv’s bin directory.

now try to install MySQL-python again.

and to undo the changes to $PATH

deactivate virtualenvDirectory/bin/activate

and tada!!!