OpenStack Swift Retriever Demo Online (with JavaScript xmlhttprequest image retrieval)

This is a follow-up to my earlier post with the addition of WORKING CODE and an ONLINE DEMO. Before you go all demo happy, you need to have your own credentials to either a local OpenStack Swift (object storage) system or RackSpace CloudFiles.

The demo is written entirely using client side JavaScript. That is really important because it allows you to test Swift WITHOUT A WEB SERVER. All the other Swift/Rackspace libraries (there are several) are intended for your server application to connect and then pass the file back to the client. In addition, the API uses meta tags that are not settable from the browser so you can’t just browse into your Swift repos.

Here’s what the demo does:

  1. Login to your CloudFiles site – returns the URL & Token for further requests.
  2. Get a list of your containers
  3. See the files in each container (click on the container)
  4. Retrieve the file (click on the file) to see a preview if it is an image file

The purpose of this demo is to be functional, not esthetic. Little hacks like pumping the config JSON data to the bottom of the page are helpful for debugging and make the action more obvious. Comments and suggestions are welcome.

The demo code is 4 files:

  1. demo.html has all the component UI and javascript to update the UI
  2. demo.js has the Swift interfacing code (I’ll show a snippet below) to interact with Swift in a generic way
  3. demo.css is my lame attempt to make the page readable
  4. jQuery.js is some first class code that I’m using to make my code shorter and more functional.

1-17 update: in testing, we are working out differences with Swift and RackSpace. Please expect updates.

HACK NOTE: This code does something unusual and interesting. It uses the JavaScript XmlHttpRequest object to retrieve and render a BINARY IMAGE file. Doing this required pulling together information from several sources. I have not seen anyone pull together a document for the whole process onto a single page! The key to making this work is overrideMimeType (line G), truncating the 32 bit string to 16 bit ints ( & 0xFF in encode routine), using Base64 encoding (line 8 and encode routine), and then “src=’data:image/jpg;base64,[DATA GOES HERE]'” in the tag (see demo.html file).

Here’s a snippet of the core JavaScript code (full code) to interact with Swift. Fundamentally, the API is very simple: inject the token into the meta data (line E-F), request the /container/file that you want (line D), wait for the results (line H & 2). I made it a little more complex because the same function does EITHER binary or JSON returns. Enjoy!

retrieve : function(config, path, status, binary, results) {

1   xmlhttp = new XMLHttpRequest();

2   xmlhttp.onreadystatechange=function()  //callback

3      {

4         if (xmlhttp.readyState==4 && xmlhttp.status==200) {

5            var out = xmlhttp.responseText;

6            var type = xmlhttp.getResponseHeader("content-type");

7            if (binary)

8               results(Swift.encode(out), type);

9            else

A               results(JSON.parse(out));

B         }

C      }

D   xmlhttp.open('GET',config.site+'/'+path+'?format=json', true)

E   xmlhttp.setRequestHeader('Host', config.host);

F   xmlhttp.setRequestHeader('X-Auth-Token', config.token);

G   if (binary) xmlhttp.overrideMimeType('text/plain; charset=x-user-defined');

H   xmlhttp.send();
}

OpenStack Swift Demo (in a browser)

I’m working on mini-demo project for OpenStack Swift.  To keep things very simple and easy to understand, I decided that the whole demo would work in JavaScript in the browser.  I also choose to use RackSpace’s CloudFiles as a Swift target for testing since they have the same API are are universally available (unlike my lab systems).

One advantage of this approach is that FireBug makes it very nice to debug and check the activity of the code.  Unfortunately, FireBug also seems to eat the headers that I need.  *Let me phrase that in a google friend way so that someone else will not loose the 2 hours I just lost*

“XmlHttpRequest setRequestHeader FireFox Not Respected when using FireBug”

It works great in Safari. So onward and upward.  So far, I’ve got step #1 ready – getting the authorization token back from the cloud site.

Here’s the HTML page (you need jQuery too).  Basically, it uses the username and key from the inputs to set “x-auth-user” and “x-auth-key” header attributes.  These attributes will allow Swift to return a token that you can use on future requests when you want to do useful work.

<!DOCTYPE html>

<html>

<head>

<title>Dell Swift Demo [0.0]</title>

<script src=”jquery.js” type=”text/javascript”></script>

<script type=”text/javascript” charset=”utf-8″>

var xmlhttp = null;

function swiftLogin() {

var usr = $(‘input:text[name=usr]’).val();

var key = $(‘input:text[name=key]’).val();

// code for IE7+, Firefox, Chrome, Opera, Safari (UR SOL IE<7)

xmlhttp = new XMLHttpRequest();

xmlhttp.onreadystatechange=function() //callback

{

if (xmlhttp.readyState==2)

{

$(‘#status’).replaceWith(xmlhttp.getResponseHeader(“X-Auth-Token”));

}

}

xmlhttp.open(‘GET’,’https://auth.api.rackspacecloud.com/v1.0&#8242;, true);

xmlhttp.setRequestHeader(‘Host’, ‘auth.api.rackspacecloud.com’);

xmlhttp.setRequestHeader(‘X-Auth-User’, usr);

xmlhttp.setRequestHeader(‘X-Auth-Key’, key);

xmlhttp.send();

}

</script>

</head>

<body>

<div id=”credentials”>

<fieldset id=”credentials” class=””>

<legend>Swift Login</legend>

<label for=”user”>User: </label><input type=”text” name=”usr” value=”user” id=”user”>

<label for=”key”>Key: </label><input type=”text” name=”key” value=”key” id=”key”>

<input type=”button” name=”Login” value=”login” id=”Login” onclick=”swiftLogin();”>

</fieldset>

</div>

<div id=”status”>[pending]</div>

<div id=”footer”>Time?</div>

<script type=”text/javascript”>

$(‘#footer’).replaceWith((new Date).toString());

swiftLogin();

</script>

</body>

</html>

WhatTheBus gets its move on: live maps arrive

The “livemap” tag for WhatTheBus completes the work that I seeded in the last commit.

In the last update, we had finished a page that used jQuery AJAX updates to get the latest bus location from the cache.  Using the simulator to provide updates, the AJAX updates shows that we could get location updates as latitude and longitudes.  By watching the dev web server, I also saw that these requests were super light (0ms DB, 0ms view).

In this update, I took the same basic code and added Google maps (v3) interaction.  Using Google and jQuery together maps is refreshingly easy.

The first step was to render the map when the page loads.  This required adding the Google javascript library and pointing the page’s onload event  to initMap.  The initMap function creates a new map object centered on the bus’ location, creates a bus marker at the center, and then replaces the page’s live_map div with the map.

<script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=true&amp;key=<%= MAP_KEY || "not_set_in_#{RAILS_ENV}_config" %>"> </script>
<script type="text/javascript">
var map;
var busMarker;
var xref = '<%= @bus.xref %>';
var name = '<%= @bus.name %>';
var tstamp = new Date();
function initMap() {
  var centerCoord = new google.maps.LatLng(<%= @pos %>);
  var mapOptions = {
    zoom: 16,
    center: centerCoord,
    mapTypeId: google.maps.MapTypeId.ROADMAP
  };
  map = new google.maps.Map(document.getElementById("live_map"), mapOptions);
  busMarker = new google.maps.Marker({
    position: centerCoord,
    map: map,
    title: name,
    icon: "/images/bus.png"
  });
  window.setInterval('updateMap();', <%= MAP_UPDATE_SECS %>000);
}

The second step was to add the AJAX call on a timer.  After adding the update function timer registration, the updates simply extended the existing AJAX request.  This existing request already had the bus’ position so the work centered on interacting with the map.

To keep things friendly, we turn the last marker into a bullet point and change the bus name to the last time.  Then we create a new marker based on that latest position and center the map on that position too.  To prevent reduce server load for non-reporting buses, we move off the map page if there is no position data.

function updateMap() {
  busMarker.setIcon("/images/track.png");
  busMarker.setTitle(tstamp.getHours() + ":" + tstamp.getMinutes() + ":" + tstamp.getSeconds());
  tstamp = new Date();
  // get the data for the map
  jQuery.getJSON("/bus/index/"+ xref +".json?cache", {}, function(data){
    if (data.length==0) window.location.href("/bus/index/<%= @bus.xref %>");
      var newCoord = new google.maps.LatLng(parseFloat(data.buses[xref].lat), parseFloat(data.buses[xref].lng));
      busMarker = new google.maps.Marker({
        position: newCoord,
        map: map,
        title: name,
        icon: "/images/bus.png"
      });
      map.setCenter(newCoord);
  });
}

After a few tweaks to add navigation between the existing pages, the first use-case of WhatTheBus is in the bag!

The next step is to setup the code online and get a district to send updates!