Erlang HTTP client RESTful API Post (example code)

Posted on August 15, 2012 by Rob H

Sometimes I just need to feed the SEO monster for Erlang. In this case, I could not find a good post that described doing an Erlang http:request that posts with a json payload. This is to support the BDD (“Cucumber but really fast”) testing framework that I’ve included in Crowbar.

Method = post,
URL = "http://192,168.124.10:3000/node/2.0/new",
Header = [],
Type = "application/json",
Body = "{\"name\":\"foo.example.com\"}",
HTTPOptions = [],
Options = [],
R = httpc:request(Method, {URL, Header, Type, Body}, HTTPOptions, Options),
{ok, {{"HTTP/1.1",ReturnCode, State}, Head, Body}} = R.

{"id", Key} = lists:keyfind("id",1,json:parse(Body)),

Note: See the full source: https://github.com/crowbar/barclamp-crowbar/blob/master/BDD/digest_auth.erl

The Body will contain your JSON response and I used our Erlang json lib to parse that to find the ID.

In the actual BDD code, it’s a little more complex because I also use our Erlang digest auth. I hope this helps you!

WhatTheBus Map seeds planted, looks like Cucumbers

Posted on February 2, 2010 by Rob H

Today’s WhatTheBus progress (see tag “MapSeeds”) created the foundation for live maps. These maps will only show a single bus at a time because our initial use case is that a parent wants to monitor their child’s bus.

To accomplish this use case, we need to provide a list of all the buses in the system (grouped by district) and then allow users to select a bus and see the map.

This foundation started with a series of Cucumber tests that verified the navigation structure and the base map page. These pages used the existing page tests (yawn).

In fact, most of this work is pretty dull Rails web navigation stuff. The interesting parts were:

DRY the bus location from cache by moving it into the model
Add jRails so that we can use jQuery goodness
Optimization: added index for Bus xref
Optimization: added ?cache flag for the bus json requests to bypass database calls on location only requests

In our next pass, we’ll add support for Google Maps (v3). Until then, we just have some simple script that pulls the current bus position using our existing bus JSON request. I am using the simulator (“rake sim:move”) to verify this; unfortunately, Cucumber does have native AJAX support.

Here’s the <script>


<script type="text/javascript">
  var xref = '<%= @bus.xref %>';
  var name = '<%= @bus.name %>';

  function initMap() {
    window.setInterval('updateMap();', <%= MAP_UPDATE_SECS %>000);
    updateMap();
  }

  function updateMap() {
    jQuery("#ll").text("?,?");

    // get the data for the map
    jQuery.getJSON("/bus/index/"+ xref +".json?cache", {}, function(data){
    jQuery("#ll").text(data.buses[xref].lat + ',' + data.buses[xref].lng);
    });
  }
</script>

WhatTheBus fun with Cucumber and MemCacheD

Posted on January 26, 2010 by Rob H

Sometimes a problem has to kick you upside the head so you can learn an important lesson. Tonight’s head slapper was an interaction between Cucumber and MemCacheD.

If you are using CUCUMBER AND MEMCACHE read this post carefully so you don’t get burned. If you’re using MemCache and not writing tests then return to Jail, do not collect $200.

It’s important to note that Cucumber has the handy side effect of running each scenario in a transaction. The impact is that the data from each scenario does not impact the next scenario. (note: you can pre-load data into cucumber using fixtures).

However, Cucumber does not do any rollback for Cache keys added into MemCache. In fact, your MemCache entries will happily persist between your development and test systems.

WhatTheBus has a simple check to reduce database writes – it only writes to the database if there is no cache hit for the bus. My thinking is that we only need to add a new bus if there is no key as shown in this partial snippet:

  cache = Rails.cache.read params[:id]
  if cache.nil?
     bus = Bus.find_or_create_by_xref :name => params[:name], :xref => params[:id]
  end

This works great for live testing, but fails in technicolor for Cucumber because tests with the same ID will not make it to the find_or_create.

To solve the problem, I had to add a pre-condition (‘given’ in Cucumber speak) to each scenario to make sure the cache was cleared. It looks like this in the scenario feature:

  Given no cache for "1234"

And that’s translated as code in the steps like so:

  Given /^no cache for "([^\"]*)"$/ do |id|
   Rails.cache.delete id
  end

WhatTheBus, Day1: MemCacheD roundtrip

Posted on January 18, 2010 by Rob H

Today I got the very basic bus data collection working using Cucumber TDD. That means that I wrote the basic test I wanted to prove BEFORE I wrote the code that operates the test.

The Cucumber feature test looks like this:

Feature: Mobile Access
In order to ensure that location updates are captured
School Bus Location providers
want to have data they send stored on the site

Scenario: Update Location
When bus named “lion” in the “eanes” district with a id of “1234” goes to “32,-97”
When I go to the bus “1234” page
Then json has an object called “buses”
And json has a record “1234” in “buses” with “lat” value “32”
And json has a record “1234” in “buses” with “lng” value “-97”

There’s is some code behind this feature that calls the web page and gets the JSON response back. The code that actually does the work in the bus controller is even simpler:

The at routine takes location updates just parses the parameters and stuffs it into our cache. For now, we’ll ignore names and district data.

def at

Rails.cache.write params[:id], “#{params[:lat]},#{params[:lng]},#{params[:name]},#{params[:district]}”, :raw=>:true, :unless_exist => false, :expires_in => 5.minutes
render :nothing => true

end

The code that returns the location (index) pulls the string out of the cache and returns the value as simple JSON.

def index

data = Rails.cache.read(params[:id], :raw => true).split(‘,’)
if data.nil?
render :nothing => true
else
render :json => {:buses => { params[:id].to_sym => { :lat => data[0], :lng => data[1] } } }
end

end

Not much to it! It’s handy that Rails has memcache support baked right in! I just had to add a line to the environment.rb file and start my memcached server.

Rob Hirschfeld

On Computing, Containers, Cloud & Tech Culture

Tag Archives: cucumber

Erlang HTTP client RESTful API Post (example code)

WhatTheBus Map seeds planted, looks like Cucumbers

WhatTheBus fun with Cucumber and MemCacheD

WhatTheBus, Day1: MemCacheD roundtrip