Category Archives: BravoDelta

Want CI Consul Love? OK! Run Consul in Travis-CI [example scripts]

Posted on by
1

If you are designing an application that uses microservice registration AND continuous integration then this post is for you!  If not, get with the program, you are a fossil.

Inside The EngineSunday night, I posted about the Erlang Consul client I wrote for our Behavior Driven Development (BDD) testing infrastructure.  That exposed a need to run a Consul service in the OpenCrowbar Travis-CI build automation that validates all of our pull requests.  Basically, Travis spins up the full OpenCrowbar API and workers (we call it the annealer) which in turn registers services in Consul.

NOTE: This is pseudo instructions.  In the actual code (here too), I created a script to install consul but this is more illustrative of the changes you need to make in your .travis.yml file.

In the first snippet, we download and unzip consul.  It’s in GO so that’s about all we need for an install.  I added a version check for logging validation.

before_script:
  - wget 'https://dl.bintray.com/mitchellh/consul/0.4.1_linux_amd64.zip'
  - unzip "0.4.1_linux_amd64.zip"
  - ./consul --version

In the second step, we setup the consul service and register it to itself in the background.  That allows the other services to access it.

script: 
  - ../consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul &

After that, the BDD infrastructure can register the fake services that we expect (I created an erlang consul:reg_serv(“name”) routine that makes this super easy).  Once the services are registered, OpenCrowbar will check for the services and continue without trying to instantiate them (which it cannot do in Travis).

Here’s the pull request with the changes.

Erlang Consul Client

Posted on by
1

OpenCrowbar has been using Consul more and more deeply.  We’ve reached the point where we must register services on Consul to pass automated tests.

Consequently, I had to write a little Consul client in Erlang.

The client is very basic, but it seems to perform all of the required functions.  It relies on some other libraries in OpenCrowbar’s BDD but they are relatively self-contained.  Pulls welcome if you’d like to help build this out.

Here’s the code and the API reference.   Check in OpenCrowbar/BDD for latest updates and dependencies.

Behavior Driven Development (BDD) and Crowbar

Posted on by
5

Test Test TestI’m a huge advocate of both behavior and test driven development (BDD & TDD). For the Crowbar 2 refactor, I’ve insisted (with community support) that new code has test coverage to the largest extent possible. While this inflicted some technical debt, it dramatically reduces the risk and effort for new developers to contribute.

For open source projects, they are even more important because they allow the community to contribute to the project with confidence.

A core part of this effort has been the Erlang BDD (“bravo delta”) tool that I had started before my team began Crowbar (code link).

I’m a big fan of BDD domain specific languages (DSL) because I think that they are descriptive. Ideally, everyone on the team including marketing and documentation authors should be able to understand and contribute to these tests.

I’ve been training our QA team on how the BDD system works and they are surprised at the clarity of the DSL. By reading the DSL for a feature, then can figure out what the developer had in mind for the system to do. Even better, they can see which use-cases the developer is already testing. Yet the real excitement comes from the potential to collaborate on the feature definitions before the code is written. My blue-sky-with-rainbows hope is that developers and testers will sit down together and review the BDD feature descriptions before code is written (perhaps even during planning). Even short of that nirvana, the BDD feature descriptions provide something that everyone can review and discuss where code (even with verbose documentation) falls short.

Ok, so you already know the benefits of BDD. Why didn’t I do this in the Cucumber? It’s the leading tool and a logical fit for a Rails project like Crowbar. Frankly, I have a love-hate relationship with Cucumber.

  1. It’s slow. And that does not scale for testing. I’m of the belief that slows tests destroy developer productivity because they encourage distractions.  Our BDD is fast and is not yet optimized.
  2. Too coupled to app framework – you can bypass the UI/API for testing if needed. If I’m doing behavior testing then I want to make sure that everything I test is accessible to the user too.
  3. While Cucumber has a lot of good “webrat” steps to validate basic web pages, I found that these were very basic and I quickly had to write my own.
  4. Erlang pattern matching made it much easier to define steps in a logical way with much less RegEx than Cucumber
  5. Erlang is designed to let us parallelize the tests.
  6. I like programming in Erlang (and I had started BDD before I started Crowbar)

And it goes beyond just testing our code…

We ultimately want to use the BDD infrastructure to gate Crowbar deployments not just code check-ins. That means that when Crowbar orchestrates an application deployment, the BDD infrastructure will execute tests to verify that the deployment is exhibiting the expected behaviors. If the installation does not pass then Crowbar would roll-back or hold the deployment.

This objective is not new or unique – it’s modus operandi at advanced companies who practice continuous deployment. Our position is that this should be an integral part of the orchestration framework.

One side benefit of the BDD system as designed is that it is also a simulator. We are able to take the same core infrastructure and turn it into a load generator and database populator. Unlike more coupled tools, you can run these from anywhere.

Post Script: Here’s the topic that I’m submitting for presentation at OSCON…

Continue reading →

Erlang HTTP client RESTful API Post (example code)

Posted on by
6

Sometimes I just need to feed the SEO monster for Erlang.  In this case, I could not find a good post that described doing an Erlang http:request that posts with a json payload.  This is to support the BDD (“Cucumber but really fast”) testing framework that I’ve included in Crowbar.

Method = post,
URL = "http://192,168.124.10:3000/node/2.0/new",
Header = [],
Type = "application/json",
Body = "{\"name\":\"foo.example.com\"}",
HTTPOptions = [],
Options = [],
R = httpc:request(Method, {URL, Header, Type, Body}, HTTPOptions, Options),
{ok, {{"HTTP/1.1",ReturnCode, State}, Head, Body}} = R.

{"id", Key} = lists:keyfind("id",1,json:parse(Body)),

Note: See the full source: https://github.com/crowbar/barclamp-crowbar/blob/master/BDD/digest_auth.erl

The Body will contain your JSON response and I used our Erlang json lib to parse that to find the ID.

In the actual BDD code, it’s a little more complex because I also use our Erlang digest auth.  I hope this helps you!

Why I love erlang – a mini recursive JSON parser

Posted on by
1

As a mental break and to support my erlang version of Cucumber (“BravoDelta”), I spent a little time building out a JSON parser.

Some notes before the code:

  • I could have done it without the case statements (using pattern matching in the functions) but I felt the code was not as readable and there were some cases where I needed the RAW input.
  • I used records because it was important to return BOTH the list and the remaining text.  It also improves the readability if you follow know the syntax (#json = new record, JSON#json = existing)
  • Has minimal error checking – fails = good in a BDD tool
  • Assumed that keys are “safe” words (don’t really need quotes)

Here’s the code.  Enjoy!

Note 2013-11-15: Here’s the active source for this on github.

-export([json/1]).
-record(json, {list=[], raw=[]}).
-record(jsonkv, {value=[], raw=[]}).
% handles values that are quoted (this one ends the quote)
json_value_quoted(Value, [$" | T]) ->
  #jsonkv{value=Value, raw=T};

json_value_quoted(Value, [Next | T]) ->
  json_value_quoted(Value ++ [Next], T).

% returns JSON Key Values with remaining JSON
json_value(Value, RawJSON) ->
  [Next | T] = RawJSON, 
  case Next of
    $: -> throw('unexpected token');
    ${ -> J = json(RawJSON),                                  % recurse to get list
            #jsonkv{value=J#json.list, raw=J#json.raw};  
    $, -> #jsonkv{value=string:strip(Value), raw=RawJSON};    % terminator, return
    $} -> #jsonkv{value=string:strip(Value), raw=RawJSON};    % terminator, return
    $" -> json_value_quoted(Value, T);                        % run to next quote,exit
    _ -> json_value(Value ++ [Next], T)                       % recurse
  end.
% parses the Key Value pairs (KVPs) based on , & } delimiters
json(JSON, Key) ->
  [Next | T] = JSON#json.raw,
  case {Next, T} of
    {$", _} -> json(JSON#json{raw=T}, Key);        % ignore
    {${, _} -> json(#json{raw=T}, []);             % start new hash
    {$,, _} -> json(JSON#json{raw=T}, []);         % add new value
    {$:, _} -> KV = json_value([], T),  % get value for key
            List = lists:merge(JSON#json.list, [{string:strip(Key), KV#jsonkv.value}]),
            json(#json{list=List, raw=KV#jsonkv.raw}, []);  % add new KVP
    {$}, []} -> JSON#json.list;                    %DONE!
    {$}, _} -> JSON#json{raw=T};                   %List parse, but more remains!
    {_, _} -> json(JSON#json{raw=T}, Key ++ [Next])  % add to key
  end.
% entry point
json(RawJSON) ->
  json(#json{raw=RawJSON}, []).

Introducing BravoDelta: Erlang BDD based on Cucumber

Posted on by
7

I highly recommend Armstrong's Programming Erlang

I ❤ Erlang.  I learned about Erlang while simultaneously playing w/ CouchDB (written in Erlang) and reading Siebel’s excellent Coders At Work interview of Erlang creator Joe Armstrong.  Erlang takes me back to my lisp and prolog days – it’s interesting, powerful and elegant.  Even better, it’s performant, time tested and proven.

To whet my Erlang skills, I decided to port of the most essential development tools I’ve used: Cucumber BDD.  I think that using BDD is one of the most critical success criteria for a project that wants to move quickly and respond to customers.  If you’d like to see Cucumber in action, check out my WhatTheBus project.  A Cucumber test is written in “simple English” and looks like this:

Scenario: Web Page1
    When I go to the home page.
    Then I should see "Districts".

To run Bravo Delta, you’ll need Erlang installed on your system.  You may also want to setup the WhatTheBus project because the initial drop uses that RoR web site as it’s target.  I’ve uploaded the code onto GitHub project BravoDelta (code contributions welcome).

NOTE: This is a functional core – it is not expected to be a complete Cuke replacement at this point!

The code base consists of the following files:

  • bdd.erl (main code file, start using bdd:test(“scenario”).)
  • bdd.config (config file)
  • bdd_webrat.erl (standard steps that are used by many web page tests)
  • bravodelta.erl (same custom steps, must match feature file name)
  • bravodelta.feature (BDD test file)
  • bdd_utils.erl (utilities called by bdd & webrat)
  • bdd_selftest.erl (unit tests for utils – interesting pattern for selftest in this file!)
  • bdd_selftest.data (data for unit tests)

Erlang makes parsing the feature file very easy.  Unlike Cucumber, there is no RegEx craziness because Erlang has groovy pattern matching.  Basically, each step decomposes into a single line starting with Given, When, or Then.  The code is designed so that developers can easily add custom steps and there are pre-built steps for common web tasks in the “webrat” step file.  A step processor looks like this in Erlang:

step(_Config, _Given, {step_when, _N, ["I go to the home page"]}) ->
	bdd_utils:http_get(_Config, []);
step(_Config, _Result, {step_then, _N, ["I should see", Text]}) ->
	bdd_utils:html_search(Text,_Result).

The steps are called by an Erlang recursive routine in BDD for each scenario in the feature file.  Explaining that code will have to wait for a future post.

The objective for Bravo Delta is to demonstrate simple Erlang concepts.  I wanted to make sure that the framework was easy to extend and could grow overtime.  My experience with Erlang is that my code generally gets smaller and more powerful as I iterate.  For example, moving from three types of steps (given_, when_, then_) to a single step type with atoms for type resulted in a 20% code reduction.

I hope to use it for future BDD projects and grow its capability because it is fast and simple to extend – I found Cucumber to be very slow.  It should also be possible to execute features in parallel with minimal changes to this code base.  That makes Bravo Delta very well suited to large projects with lots of tests and automated build systems.