lettuce 0.1 official release

27 days after its first release candidate, lettuce is now much mature and also got new features.

Lettuce is a BDD tool written in python, 100% based on cucumber.

There are two reasons for the name:

  • Lettuce is a green vegetable, just like cucumber the idea is that your tests must be always green.
  • Letuce is the name of a brazillian band that I pretty much like

Hands on!

There are links to lettuce’s official documentation at the end of this post, but there is a sneak peak:

Features are described in files with .feature extension and must be, by defaul within your project, in a folder called “features“.

Features are described like this:

Feature: Introduce lettuce to my friends
  In order to show it working
  As lettuce author
  I want to create a scenario that passes

  Scenario: Concatenate names
    Given I have the following names:
      | name | surname |
      | john | doe     |
      | ian  | murdock |
    When I join them
    Then I see the data:
      | joined       |
      | Doe, John    |
      | Murdock, Ian |

In order to define steps all you need to do is write a new python file wihin features folder, Lettuce will import them all, recursively.

Example:

# -*- coding: utf-8 -*-
from lettuce import step, world, before
from some_module import NameJoiner

# setup
@before.each_scenario
def set_joined(scenario):
  world.joined = []

# steps
@step('I have the following names')
def set_names(step):
  world.people = step.hashes

@step('join them')
def join_names(step):
  for person in world.people:
    joiner = NameJoiner(
      name=person["name"],
      surname=person["surname"]
    )
  world.joined.append(joiner.join())

@step('I see the data')
def check_names(step):
  for name, data in zip(world.joined, step.hashes):
    assert name == data['joined']

(the full example is available here aqui)

Motivation

Since I started with TDD in Python I had many experiences, I’ve tested many mock mock libraries and test techniques.
In web projects I’ve used Pyccuracy, so that I could both describe behaviour of the application and make automated acceptance tests.

Althrough, I had a chance to try Cucumber with Django here at globo.com, and the result was: I fell in love with it, and had incredibly productive features, such as step tables which allows drawing a ascii table and map it into a list of hashes, and the scenario outlines.

From there I had THE inspiration: to implement a tool that works exactly like Cucumber, but in python.

Why ?

Python programmers it’s easier and frictionless to use libraries written in Python. I personally avoid mixing programming languages in a single project, some technologies can suit perfectly in the project, no matter in what language, but if there is a Python option, I will give it a go.

Cucumber IMHO one lib to make ruby even more “sexy” :)
Build a robust application, well tested required patience, what about turn it into fun ! ?
It’s easier to package python modules for debian/ubuntu.

Besides, even considering the fact that  Cucumber already supports, would be more interesting to user pure python within step definitions.
Supposing that a web application will be written with Django and Cucumber, would be very useful to manipulate models within step definitions. However turning it possible, would need efforts to run rubypython, which the latest release dates october 2009.

Examples of similar libraries between python and ruby

Web framework:

ruby: rails
python: django

Micro web framework:

ruby: sinatra
python: cherrypy

Automated deploy:

ruby: capistrano
python: fabric

Behaviour-driven development:

ruby: cucumber
python: lettuce ?! (maybe ?! Who knows ?! :P )

More on lettuce

Documentation

Available at http://lettuce.it covers all supported features until now.

Contribute!

I’ll appreciate all kinds of feedback, to do so we have some channels:

Lettuce is under GNU GPL 3+ and have a long way until get robust, and any kind of help is welcome: patches, documentation, language support and so on!

Grab it, and happy hacking :)

Apresentando Lettuce, BDD em python

Depois de muitas horas de hacking intenso, venho apresentar a primeira release candidate do Lettuce.

Trata-se de uma ferramenta de BDD 100% baseada no Cucumber.

O nome teve duas motivações:

  • Lettuce (Alface) é um vegetal verde, assim como Cucumber (Pepino), a idéia é que os testes sempre estejam verdes :)
  • Letuce é também o nome de uma banda à qual tenho muita admiração.

Mãos ao código

Deixei links para a documentação no fim deste post, mas aqui vai uma prévia:

Funcionalidades são descritas em arquivos com a extensão .feature, e devem ficar por padrão, dentro do seu projeto, chamada features

Por sua vez, features são descritas assim:

Feature: Introduce lettuce to my friends
  In order to show it working
  As lettuce author
  I want to create a scenario that passes

  Scenario: Concatenate names
    Given I have the following names:
      | name | surname |
      | john | doe     |
      | ian  | murdock |
    When I join them
    Then I see the data:
      | joined       |
      | Doe, John    |
      | Murdock, Ian |

Para implementar as steps, basta criar um arquivo python qualquer dentro da pasta features o Lettuce importará todos eles recursivamente.
Exemplo:

# -*- coding: utf-8 -*-
from lettuce import step, world, before
from some_module import NameJoiner

# setup
@before.each_scenario
def set_joined(scenario):
    world.joined = []

# steps
@step('I have the following names')
def set_names(step):
    world.people = step.hashes

@step('join them')
def join_names(step):
    for person in world.people:
        joiner = NameJoiner(
            name=person["name"],
            surname=person["surname"]
        )
        world.joined.append(joiner.join())

@step('I see the data')
def check_names(step):
    for name, data in zip(world.joined, step.hashes):
        assert name == data['joined']

(baixe o exemplo completo aqui)

Motivação

Desde que comecei a praticar TDD em Python tive várias experiências, testei várias bibliotecas de mock e técnicas de teste.
Em projetos de aplicações web usei o Pyccuracy, assim podia descrever o comportamento da aplicação ao mesmo tempo que fazia testes automatizados da interface.

No entanto, tive a chance de testar o  Cucumber num projeto Django aqui na globo.com, me apaixonei por funcionalidades incrivelmente produtivas, como step tables que permite desenhar uma tabela com caracteres, e mapear como lista de hashes, bem como scenario outlines.

Daí veio a grande inspiração: implementar uma ferramenta como o Cucumber, mas em python puro.

Por que?
Para programadores python, é mais cômodo usar bibliotecas em python puro. Eu pessoalmente evito misturar linguagens, a não ser que a melhor solução seja realmente em outra linguagem.

Cucumber é, na minha opinião, uma lib que torna ruby ainda mais sexy :)
Criar uma aplicação robusta, bem testada requer paciência, que tal tornar isso mais divertido ?
É mais fácil empacotar módulos python para debian/ubuntu :P

Além disso, apesar de Cucumber dar suporte a Python, seria interessante usar python puro dentro das step definitions.
Supondo o desenvolvimento de uma aplicação Django com cucumber, seria útil manipular os models dentro de step definitions. Entretanto, para isso acontecer, é necessário usar algo como rubypython, cuja última release data outubro de 2009.

Exemplos de bibliotecas similares/equivalentes entre python e ruby

Framework web:

ruby: rails
python: django

Micro-framework web:

ruby: sinatra
python: cherrypy

Deploy automatizado:

ruby: capistrano
python: fabric

Behaviour-driven development:

ruby: cucumber
python: lettuce ?! (quem sabe :P )

Direto ao assunto!

Hoje estou liberando oficialmente a primeira release candidate do Lettuce, que atualmente possui as seguintes features:

  • Hooks pre/pós execução de:
    • Toda a suíte de testes
    • Cada feature
    • Cada scenario
    • Cada step
  • Contexto global: world
  • Scenario outlines
  • Step tables
  • Nomes e formato de arquivo de features compatível com cucumber
  • Output colorido
  • Output sem cor

Mas… O que falta para fechar a primeira release oficial? Além de ter feedback sobre possíveis bugs das features já implementadas para melhorá-las, a intenção é que a primeira release também tenha as seguintes funcionalidades:

Documentação

Não poderia faltar, mas ainda está sendo escrita.
De qualquer forma, já possui o básico para colocar o um novo projeto funcionando. E se você já conhece o Cucumber, não terá dificuldades.

Acesse a documentação aqui

Ou melhor ainda, vá direto ao “quick start”

Contribua!

Estou sedento por feedbacks (negativos e positivos), sendo assim:

Por fim, o lettuce está sob GNU GPL 3+ e ainda tem um longo caminho pela frente. Toda forma de ajuda é bem vinda, seja com patches, documentação, suporte para outros idiomas, etc.

Happy hacking :)

Playing with ANSI colors and python with Couleur

Last night I’ve just released the first version of Couleur, which is (or at least try to be) a simple and awesome tool for using ANSI colors with python.

Couleur indeed turns very easy to handle shell colors in python.

In action

import time
import couleur

sh = couleur.Shell(linebreak=True)
for mood in ["bad", "awful", "normal", "nice", "awesome"]:
  sh.red_on_black(mood, replace=True)
  time.sleep(1)

Couleur creates colored output dinamically, so that you can mix colors and modifiers at will:

import couleur
sh = couleur.Shell(linebreak=True)
sh.bold_black_and_bold_yellow_on_red("Gray | Yellow")

And even use “static” output, let’s simulate the output of a “git clone”:

import time
import couleur

sh = couleur.Shell(linebreak=True)
for x in range(101):
  if x is 0:
    print
  sh.normal_and_bold_green(
    "Counting objects: |%d%%" % x,
    replace=True
  )
  time.sleep(0.08)

Installation

user@machine:~$ sudo pip install couleur

Full Documentation

Available here

Contributing

Fork me on github, write tests, write code, make it pass, send a pull request :)

Avoid python pitfalls and be happy

This morning I was just reading the feed, when I stumbled on this post about one of the weird behaviors of lists in Python.

In a few words, lists and dictionaries behave like C pointers, thus they are mutable objects.

So, if you are beginning on python, you probably will like those articles:

Hope it helps :)

Solving “connection refused” issues on Debian Sid

Some weeks ago I started to experiment some network issues on my debian sid:

  • The selenium-remote-control was not running:

I could even try to do a: “telnet localhost 4444″, it just did not work.

  • My Apache-Solr was not capable to subscribe to apache-activemq

When reading the log, I saw:

Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused
Suddenly I noticed something weird in my netstat:

tcp6       0      0 :::8983                 :::*                    LISTEN      3865/solr-globocom
tcp6       0      0 :::8161                 :::*                    LISTEN      3901/activemq-globo
tcp6       0      0 :::61616                :::*                    LISTEN      3901/activemq-globo
tcp6       0      0 127.0.0.1:41879         127.0.0.1:61616         ESTABLISHED 3865/solr-globocom
tcp6       0      0 127.0.0.1:61616         127.0.0.1:41879         ESTABLISHED 3901/activemq-globo

Everything is being bound as IPv6… humm, that doesn’t have a good smell….

Looking for solutions I’ve found this file:

/etc/sysctl.d/bindv6only.conf

interesting, huh?!?

Openning up that file, I saw a full description of it, then I just needed to change its property from this:

net.ipv6.bindv6only = 1

to this:

net.ipv6.bindv6only = 0

Well, if you have the same problem, jsut change that, restart your PC, and be happy!

Introducing Sponge, a python web framework aimed on testing

Sponge first prage

Some time ago, a friend of mine created a cherrypy-based continuous integration application, we both work with Django, but we think that its stack is too big for simple web projects, specially when we do not need to access SQL databases.

More on Django

In adition, I think that Django-powered applications are hard to make unit tests, mostly related to the fact that models, templates and other django helpers need the DJANGO_SETTINGS_MODULE environment variable set, which turn its tests wired.

We have created a few patterns to make Django projects unit-testable, but it is for another post :)

Are you trying to say that Django is not a good framework ?

Nope! I work with Django, and I just LOVE Django, that is a great framework. Its models, forms, and everything else are awesome, and help me to write web applications without much effort, thus keeping high level of quality.

So, why did you create another framework ?

As I’ve just said some posts ago, I really enjoy agile, and I use to write simple web applications that does not demand the whole Django stack, and I need to have agility to write my tests.

CherryPy is a good solution for me, I don’t want another huge-ish framework, even if its based on cherrypy. I just need a few helpers functions, and a pattern to follow.

Idealizing sponge

Bernardo and me wrote a few web applications with cherrypy, and always focusing in TDD, we needed to create some helper classes to cherrypy. Sometime after, we figured out that those helpers could be grouped in a simple library, or a kind of tiny framework.

I started some mashups, and Sponge has been developed.

The name comes from the fact that a sponge can be used to dry stuff. We like the concept of DRY, hence the name :)

Sponge has been developed with TDD, but that is not all.

I’m trying to keep 100% of code coverage in both unit + functional tests, at minimum. It’s because 100% of coverage does not assure that the code works. Lines of code can be covered with some tests, but the same lines can have one or more different behaviours that won’t be found with 100% of coverage.

Principles

  • Help and incentivate developer to write unit and functional tests
  • Less friction as possible
  • Be simple to configure
  • Be simple to deploy
  • Be 100% testable, specially with unit tests
  • Provide pretty URLS through a rails-like syntax
  • Be as most compliant as possible, with package-systems, such as APT
  • Free software (LGPL-3)
  • Comes with jQuery javascript library and 960 grid system
  • Fun to use :)

Hands on!

Installing

If you are using Debian Unstable, just run:

sudo aptitude install python-sponge

If not, you will need, unfortunately, to use setuptools:

In this case, just grab the bleeding edge version through git:

git clone git://github.com/gabrielfalcao/sponge.git

Or just the latest stable relase:

Sponge 0.3.1

Extract, if needed, and run:

sudo python setup.py install

Tutorial

Sponge comes with a simple tutorial within its documentation, but you can access it online

Emacs + Snippets

People use to joke on me about the fact that I use EMACS, some say that I even need to use my toes to acomplish EMACS commands, and so on…

But I keep being even more productive, and every day I figure out new ways to improve it.

I really use EMACS the whole day, I keep editing anything with this awesome editor.

This weekend I was starting a new project, and as I use to, I was needing to repeat  the same GPL headers at every file.

So I found myself questioning if it could be more productive than inserting rough templates on my files.

Then I found out this yasnippet, which is totally awesome, and made me be even more productive.

For those who also like emacs, I share my emacs configuration on github.

There is a screencast that I made. Is a EMACS snippet demo.

Bolacha, httplib2 wrapper with cookie handling and file upload.

I really like httplib2, but unfortunately it does not oficially support sending multipart/form-data.

So last Monday I was at work, and realized that I was needing exactly to make a http upload from within a python program, and also needed to make a session login right before.

I remember that I needed that before, but I could not find something simple and concise, yet pythonic and well tested.

So I decided to work on a proof of concept, 2 days after that, at last wednesday, I got really sick. I thought was just a simple flu, but I got even worst. Going to hospital, doctors found out I have sinusitis.

So since wednesday I am at home, having lots of damn medicines.

But the good thing is that I got time enough to hack :)

So I would like to introduce you Bolacha.

You can see de docs here

There is a debian package available here

The code is available at my github page: http://github.com/gabrielfalcao/bolacha/tree/master

New projects + agile = releases

When I lived in Belo Horizonte/MG, I used to make hack parties with some friends of mine (nothing to say about all those pizzas and beer).

The hack parties had no specific goal, but a main idea: Hack some free software, and share the knowledge.

Many of us, but specifically Lincoln and I, used to create new projects in every hack party,but only a handful of those were actually released. It was often matter for jokes like: “Oh, you are up to raise the “too many projects” exception”.

I have two words for that fact: “not agile”.

Our projects were often POCs, and we had no culture of TDD, short releases and so on.

But since I started to work at Globo.com, I learned so much about agile, specially Scrum. And some months ago I’ve just accomplished what I call as “agile feeling”.

I mean, it’s a feeling that I can deliver software with quality in a short period of time.

But how ?

The idea is quite simple, you just have to follow some principles:

Think simple

This basically means focusing at the problem at hand, never trying to solve future problems you don’t have. Most of the time the problems your foresaw will never happen, yet you spent a lot of time preparing for them.

Test before programming

Write tests for small parts of your software, they will be very modular, and you will get a trust-able software, and I can assure you that you will be 100% able to embrace the changes.

Release fully functional versions, with simple features

As human beings, we want to embrace all possibilities and solve all possible problems at once.

But this is not agile at all. Talk to your client, be pragmatic, he will like to listen the truth: “You can not ship a full-featured final version of software within the term. But you can totally ship a primary version, fully functional, with all the basic needs of the client. Keep improving the software in next releases”

I tried to summarize those thoughts, but I need blog posts for each one.

Applying those thoughts to personal projects

As I said, I have unfinished projects, and they are still unfinished, mostly because were started with a non-agile approach, and I can’t get interested to finish them.

But there are a few projects I worked in, and others I am the creator. For instance:

  • Dead Parrot, a pythonic RESTFul framework django-based. Is on release 0.1-hellopolly, being used within some globo.com projects.
  • Ma-Chérie, a simple filesystem-based web application to navigate through pictures. Current release: 0.1
  • Sponge, a tiny web framework built on top of CherryPy and used Genshi as default template language. Sponge is the base of Ma-Chérie. Current release: 0.1
  • Pyccuracy, s BDD acceptance testing tool. Heynemann started that project, as a comitter, I worked in the biggest refactoring. Current release 1.0.3-viagra
  • Sleepy, is a crude, slow and rough template language, 100% based in regular expressions (which mades sleepy so slow). Actually I do not plan release it by now,  I am just having some proofs of concept of how possible is to write a template language 100% regex based, just with TDD.

All projects above, except Sleepy, were released as well, all them were developed with TDD and have  a good code coverage.

But the big deal is that, actually they will never get finished, for a simple reason. They will be in continuous development and continuous released as well.

By the way, if you want to contribute, just go to my github profile and check them out.

Eclipse + Java + UML + RESTful, the real silver bullet

Some days ago I was just thinking about all these agile bullshit.

I mean, no matter if you do TDD or not, if you do care about continuous integration, this is all a real lose of time.

When thinking about shipping good software, the best combination is, indeed, quite simple:

  1. Use a powerful IDE. You as developer don’t even need to now how the things work, let the IDE do everything for you. Eclipse is the one you really need.
  2. Use a well known, certified and reliable language. Why using dynamic languages ? You don’t need to be agile, you need to ship something with trust value, so build it with java.
  3. Don’t code, use wizards and boilerplate generators. Once again, what is better, easy money or thousands of hours hacking things from scratch ? You can use UML-based code generators, and super-duper eclipse plugins to generate your code. Forget all stress and make your project manager happy.
  4. Prepare yourself for possible future needs. To build a real scallable application you must ALWAYS use a complex SOA approach, so you can separate business-rules in each module and distribute in many servers. A RESTful approach is even better. Forget everything about agile software development, your customer doesn’t need a term, he needs a full-featured project.

To but it bluntly, Fred Brooks was totally wrong in his book “The Mythical Man-Month”, about “no silver bullet”, the steps below are, indeed, a silver bullet and you can see proofs of this sentence in the “Martin Flawler’s” article.