Tag Archives: javascript

Parsing LLAP packets using NodeRED on the Raspberry Pi

It’s taken a few weeks but here is the next instalment in my promised series of articles. Part of the scenario requires me to detect a particular scenario using a remote sensor and relay it on to a variety of other systems via my Raspberry Pi. The sensor itself is assembled using the rather brilliant Ciseco Wireless Inventor’s Kit which provides all you need for prototyping remote sensors and transmitting data over a short range radio link back to a waiting Pi.

It’s a pretty simple yet powerful set of kit. You build your sensor using the breadboard and XinoRF module supplied which communicates with a Slice of Radio transceiver module you attach to the Pi’s GPIO pins. Your applications then send and receive data to and from the remote sensor via the Pi’s serial port and you’re away. The protocol used for the serial communications is designed by Ciseco and is called Lightweight Logical Application Protocol (LLAP). It takes the form of a simple fixed-field text datagram and as such can be parsed by pretty well any programming language that can interrogate the serial port.

In my case I’m using NodeRED to parse the LLAP packets so I can invoke other services in response to an event triggered via a sensor. The event is detected by a switch (a pressure mat in fact) that sets the D03 pin on the XinoRF module to be “high”. The nice thing about how the XinoRF works is that it operates in an event-driven way — i.e. you don’t need to poll to check the state of the pin, it transmits when there is a change in the state from “low” to “high” or vice versa. The other XinoRF pins perform different functions in different ways, but I thought I’d show how to at least send and receive so others can tailor to their scenario.

I’m assuming here that folks are familiar with the NodeRED concepts, so will get straight into the meat of how it works.

Configuring A serial input node to Receive llap packets into a flow

The first thing to do is configure the serial input node in NodeRED. The settings for my node are shown below.

Screen Shot 2014-10-14 at 17.56.37

You can confirm the serial port name using the RasWIK documentation and tools. A particular point to note is the Split input settings which determines at which point the input node offers up data to the next node in the flow. This can be set to a delimiter, or set to a timeout, wherein if no data is received within a certain period, the transmission is considered complete. I discovered that the latter timeout approach worked with a setting of 250 milliseconds, though it does introduce the possibility of multiple LLAP packets being received in one go, which we now need to address within the next node in our flow.

Creating a function node to segment multiple llap packets into individual messages

For greatest efficiency want all subsequent nodes in the flow to be designed to ingest a single LLAP packet in a single message. To ensure this downstream behaviour is honoured we create a function node after our serial input node to check for multiple packets and segment them into individual messages if so. The way NodeRED is designed to work, we can return an array of messages and it will cycle through each message in turn, invoking the remainder of the flow as if triggered by a single message.

The first thing we do in our function node is test for the simple case of a single packet which we can test by the length of the payload, LLAP packets being 12 characters in length.

var instring = msg.payload;
if (instring.length<=12) {
   console.log("Single packet: "+instring);
   return msg;

If the length of the payload is longer, it means we have multiple packets so we need to iterate over the payload, extracting each packet into an array of new messages to return at the end of the function.

else {
   var packets = [];
   console.log("Multiple packets: "+instring);
   while (instring.length>=12) {
      var packet = instring.substr(0, 12);
      console.log("Unspliced packet: "+packet);
      packets[packets.length] = {payload:packet};
      instring = instring.substr(12);
      console.log("instring is now length "+instring.length);
   return [packets]; 
return null;

An important point to note (highlighted above) is that although we have created an array of payload objects, NodeRED requires that we in turn encase that array in another array when we return it for processing downstream.

Parsing the LLAP packet

In our subsequent function node, we can now parse the single LLAP packet to determine whether we have a “high” event for the D03 pin we’re interested in. The format of an LLAP packet in this case would be:


Our code simply needs to check the packet that is received, and parse it see if it corresponds to the above event. Note that in a simple scenario we could just do a direct string comparison between what we’ve received and a template packet such as that above, but if we have multiple stations (denoted by characters 1 and 2, “–” in the example) then we might have more complex flow control logic. My function node simply emits the word “HIGH” if we’ve got a D03 “high” event from the XinoRF.

var thePayload = msg.payload;
var pin = thePayload.substring(3,6);
if (pin == "D03") {
   // Now test if it was HIGH
   if (thePayload.substring(6,10) == "HIGH") {
      console.log("HIGH on D03");

Now in my scenario, the switch (pressure mat) could toggle on and off in quick succession which would technically send a number of LLAP packets back to the Pi. Within a certain time window, I’m only interested in a single “high” event, so have added some logic using NodeRED’s context to test when I last received a “high” event and only taking action if it was more than a minute ago.

      var timeNow = new Date();
      console.log("timeNow is "+timeNow);
      if (context.lastHigh) {
         console.log("context.lastHigh is "+context.lastHigh);
         var lastTimeMillis = context.lastHigh.getTime();
         var currentTimeMillis = timeNow.getTime();
         var delta = currentTimeMillis - lastTimeMillis;
         if (delta < 60000) { 
            console.log("Less than a minute since last HIGH: "+
               delta+" ms.");
            return null;
         console.log("Over a minute, actually "+delta+" ms.");
      } else {
      context.lastHigh = timeNow;
      return {payload: "HIGH"}
return null;

So, if we get a “high” event and we haven’t seen one for at least a minute, the subsequent nodes in the flow will receive a packet with a payload of “HIGH”.

And with that we’re free to continue our processing, which could be as simple as a debug node. Here’s a snapshot of my NodeRED workspace — in this case I’m using the “high” event to trigger a Tweet.

Screen Shot 2014-10-14 at 18.33.17



Simple database operations via HTTP with Node JS, MongoDB and Bluemix

In my previous posting I mentioned how I’d planned to harvest some of the useful fragments from a home project that might be useful to others. This time I thought I’d capture the simple mechanism I created to keep an audit database for my application using MongoDB running on Bluemix.

I opted for Mongo due to its simplicity and close integration with JavaScript — a perfect option for creating something quickly and easily to run in a Node JS environment. Mongo is a NoSQL database, meaning that you don’t have to define a specific schema for data, you simply store what you need as a JSON object in the form of key/value pairs. This means you can store and retrieve a wide variety of different data objects in the same database, and aren’t constrained by decisions made early on in the project if your needs change. Whilst it wasn’t a design point of mine, Mongo also is designed to scale too.

As described previously, I’m using the Node JS Web Starter boilerplate as my starting point. I’ve previously added the Twilio service, now to add Mongo, I simply select the MongoLab service from the Data Management set of services on Bluemix console, and add it to my application.

Screen Shot 2014-08-06 at 15.55.50When you create the MongoLab service for the app, Bluemix provides a link to the MongoLab admin page. The nice thing about MongoLab as a service provider is that it gives you nice user friendly tools for creating Collections, reviewing documents etc. I created a collection in there called easyFiddle using the web console.

Screen Shot 2014-08-06 at 16.01.55

Having configured the Mongo Collection, the next step is to make sure that the Mongo libraries are available to the Node JS environment. As with Twilio before, we simply make sure we have an entry in the package.json file.

   "name": "NodejsStarterApp",
   "version": "0.0.1",
   "description": "A sample nodejs app for BlueMix",
   "dependencies": {
      "mongodb": "1.4.6",
      "express": "3.4.7",
      "twilio": "^1.6.0",
      "jade": "1.1.4"
   "engines": {
      "node": "0.10.26"
   "repository": {}

Just as before, Bluemix will handle the installation of the packages for us when we push the updates to the server.

Within our code, we now need to instantiate the Mongo driver objects with the credentials generated by Bluemix for the MongoDB instance running in MongoLabs. Bluemix supplies the credentials for connection via the VCAP_SERVICES environment variable.

var services = JSON.parse(process.env.VCAP_SERVICES || "{}");
var mongo = services['mongolab'][0].credentials;

We will reference the mongo object to retrieve the credentials when we connect to the database.

As I did with Twilio, I am using a simple HTTP-based service that will in this case create an audit record in a database. I’m using Express again (as described previously), together with the same basic authentication scheme. My service works on HTTP GET requests to /audit with two query parameters device and event.

// Leave out the auth parameter if you're not using an 
// authentication scheme
app.get('/audit', auth, function(req, res) {
   var theDevice = req.param("device");
   var theEvent = req.param("event");

Now it’s a case of connecting to Mongo, and inserting a JSON object as a document to contain the two parameters.

   mongodb.MongoClient.connect(mongo.uri, function(err,db) {
      // We'd put error handling in here -- simply check if 
      // err is set to something
      var collection = db.collection('easyFiddle');
      var doc = {
         "device": theDevice, 
         "event": theEvent, 
         "date": new Date()
      collection.insert(doc, {w:1}, function(err, result) {
         // Again, we'd put error handling in here

And that’s it. We can now create an audit entry using our browser if we choose, with a URL that looks like:


I’ve added other services using the same method to variously query and delete all the records in the Collection. Whilst I’ll not include them all here, note that the syntax for deleting all the records in a collection is a bit non-obvious — the examples show you how to delete a record matching a given key/value pair, but are less clear on how to delete them all. You do so simply by supplying a null instead of a name/value pair the method call:

collection.remove(null, {w:1}, function(err, result) {
   // Error handling
   // ...

Note that the result variable will contain the number of records deleted.

Hopefully this posting has helped get you going anyway. A great resource to help you navigate your way around the Node JS API for Mongo can be found in the MongoDB documentation.

Sending SMS messages using Twilio and Bluemix

I’ve been tinkering with an Internet of Things project at home for a while which I’ll write up in due course, but in the course of doing so have knocked up a few useful fragments of function that I thought I’d share in case other people need them. The first of these is a simple Node.js app to send an SMS message via Twilio using IBM Bluemix.

There’s lots of material on Twilio and Bluemix but by way of a very rapid summary, Twilio provide nice, friendly APIs over telephony-type services (such as sending SMS messages), and Bluemix is IBM’s Cloud Foundry-based Platform-as-a-Service offering to enable developers to build applications rapidly in the cloud. Twilio have created a service within Bluemix that developers can pick up and use to enable their applications with the Twilio services. One of the things I wanted for my application was a way of notifying me that something had happened, and a text message suited my needs nicely. Twilio provide a free trial service with a few restrictions which you can upgrade once you want to do something serious.

To begin with, I created myself a Node application on Bluemix using the Node JS Web Starter application boilerplate provided:

My approach was to create a simple HTTP-based service that I could invoke with the destination phone number, and the message itself as parameters. To make the Twilio service available to my Node application, it was simply case of adding the service to my application in Bluemix. Twilio is listed as one of the Mobile services:

Once you have added the Twilio service, you configure it in Bluemix by providing the Account SID and Auth Token values that you find on the account details page once you have registered and logged in to Twilio.

The Node JS Web Starter boilerplate creates a simple template for a web server that serves up pages using the Express framework on top of Node. Express is handy, in that it provides a useful framework for handling HTTP requests, so I decided to stick with it for my HTTP service. The first change I needed to make to the boilerplate was to add a reference to Twilio in the package.json file so that the modules would be available to my code.

   "name": "NodejsStarterApp",
   "version": "0.0.1",
   "description": "A sample nodejs app for BlueMix",
   "dependencies": {
      "mongodb": "1.4.6",
      "express": "3.4.7",
      "twilio": "^1.6.0",
      "jade": "1.1.4"
   "engines": {
      "node": "0.10.26"
   "repository": {}

When you push your updated code to Bluemix, Bluemix automatically does the npm install to go and fetch the modules based on the package.json.

Within the app, you then need to set up the Twilio package ready for sending messages. First, we need to require the Twilio package so we can access the service from our code, and then retrieve the Account SID and Auth Token values configured in Bluemix from the VCAP_SERVICES environment variable that Bluemix provides to the Node runtime.

var twilio = require('twilio'); // Twilio API
var services = JSON.parse(process.env.VCAP_SERVICES || "{}");
var twilioSid, twilioToken;
services['user-provided'].forEach(function(service) {
   if (service.name == 'Twilio-ph') { 
      twilioSid = service.credentials.accountSID;
      twilioToken = service.credentials.authToken;

Note that Twilio-ph is the name I gave to the Twilio service when I added it to my application in Bluemix, yours may vary so remember to change it if different.

The environment is now set up, so now we need to create our HTTP handler using Express to form the basis of our service.  I’ve added Basic Authentication to my handler to prevent spam to my Twilio account, this is nice and easy to do using Express.

// Configured environment variables to protect Twilio requests
var USER = process.env.USER;
console.log("USER = "+USER);
var PASSWORD = process.env.PASSWORD;
console.log("PASSWORD = "+PASSWORD);

// Basic authentication to restrict access to my services.
var auth = express.basicAuth(USER, PASSWORD);

I’ve used environment variables that I’ve set in the Bluemix environment, clearly in a production environment one would use a proper directory. You can set your own environment variables within your application by going to the Runtime view of your application and selecting the USER-DEFINED button on the main panel.

The HTTP handler simply looks for the URI pattern /twilio as a GET request and reads the destination telephone number and the message content as query parameters. The auth object passed in applies the Basic Authentication rule defined previously.

app.get('/twilio', auth, function(req, res) {
   var toNum = req.param("number");
   var twilioMessage = req.param("message");
   var fromNum = 'your Twilio number';

Your Twilio number can be found on the Twilio account details page.

Twilio make it really easy to send a message using their twilio.RestClient API wrapper class. You simply instantiate an instance of the twilio.RestClient class, and invoke the sendMessage method with a JSON object containing parameters describing who the message is to, the number it is sent from and the message to be included in the SMS. You provide a callback function that is invoked when the request is completed.

   var client = new twilio.RestClient(twilioSid, twilioToken);
         to: toNum,
         from: fromNum, 
         body: twilioMessage
      function(err, message) {
         if (err) {
            console.error("Problem: "+err+": "+message);
            res.send("Error! "+err+": "+message);
         } else {
            console.log("Twilio message sent from "+fromNum+
            " to "+toNum+": "+twilioMessage);

Once deployed, the service can be invoked with a URL in the form described below:


If invoked programmatically the authentication credentials will be required in the HTTP header. If tried from a browser, the browser will prompt for a username and password combination. Ultimately you’ll receive an SMS on your phone:
twilio sample
And there it is. I can now trigger SMS messages either from my NodeRED flows, browser or any other apps I might write.

Fixing jQuery Mobile headers in a Worklight app on iOS 7

One of the fun bits (depending on your inclination) of working cross-platform is discovering and mitigating the nuanced differences as you try your app on different devices. One such difference in iOS 7 is the transparency of the iOS status bar that contains the wifi strength, battery life and so on.

iOS header area

If you’re not including a header in your app then this won’t make a whole lot of difference to you, but if you are, you’ll find the iOS status bar overlays your header which can mess up your carefully placed buttons, iconography and header text.

I’ve come up with a simple workaround for jQuery Mobile running in a Worklight environment that I’ve posted here for ease of reuse and in case anybody else is looking for similar. The same principle should apply equally in a vanilla Cordova app too.

My example uses a simple jQuery Mobile header on a page.

<div data-role="page" id="mypage">
   <div data-role="header" data-position="fixed" 
      <a href="#home" 
class="ui-btn ui-icon-back ui-btn-icon-notext ui-corner-all"
      <h1>My heading</h1>
   <div data-role="main" class="ui-content">
      Blah blah

The overlap of the status bar is 20 points, so when the app renders we need to first detect whether we’re on a version of iOS that needs adjusting, then fix the elements contained in the header to allow for the status bar height.
For the purposes of demonstration I’ve simplified the below just to test for the iOS version assuming an Apple device, but of course you can add further tests for other platforms.

//... head contents
function onDeviceReady() {
   if (parseFloat(window.device.version) >= 7.0) {
      $('H1').each(function() {
         // `this` is the h1, the padding goes on the
         // containing header div.
         $(this).parent().css("padding-top", "20px");
         // sort any buttons/icons rendered from A tags too
         $(this).siblings('A').css("margin-top", "20px");
// Fire when Cordova is up and ready
                           onDeviceReady, false);
//... rest of head contents

The logic of script searches for h1 tags on the assumption that they will be used for header text. If your interface is styled differently, you might want to come up with a different “eye-catcher” tag or attribute so jQuery can select all the nodes for the headers in the app. Having found the h1 tags, it then adjusts the containing div nodes to pad out from the top by the required amount. My application has a tags for buttons in the header, which a bit of experimentation showed were not being adjusted along with the containing div, so I’ve adjusted them directly.

Notice that I’ve used CSS padding for the h1 – this means that the additional offsetting will be filled with the natural background scheme of the header, rather than a blank white oblong which would occur if margin were used. The jQuery icons for my back link get distorted by tinkering with the padding, so I’ve used margin which works just fine as they layer over the top of the heading colour scheme.

Mobile web frameworks, and other religious debates

It is an interesting litmus test of the maturity of any given technology trend as to when it starts to develop its own set of heated points of debate and argument. We had the “browser wars” of the late ‘90s, and then proprietary plug-ins versus AJAX/open web and more recently which AJAX framework is “best”.

The rise of mobile apps as a party that everyone wants to be at has further amplified this frameworks debate, as the focus has evolved from AJAX on the desktop to the mobile platforms. A quick Google and you’ll find any number of fora debating the merits of jQuery Mobile vs Dojo Mobile vs Sencha Touch and so on.

So there are several, which one is best then?

In fact participation in such debate in isolation is ultimately futile. That a particular topic becomes the subject of almost religious fervour in itself betrays that absolute truth either is either very hard or impossible to prove. They key to finding an answer is understanding the context. What is best for one situation may not be best for another, and to suggest otherwise would do the asker of the question a disservice, assuming they are asking for help.

There are a number of considerations though that can help navigate to what best might be.

You’re at the mercy of consumers

Technical debate is all fine and good, but in the mobile world, we know that consumers will decide the success or failure of the app. A poor experience by the end user will ultimately be its undoing. The framework must be able to meet the experience expected by the users. This is of course a key factor in determining whether a native or mobile web/hybrid approach is applicable in the first place, but that is another discussion entirely.

Don’t forget also that user experience and aesthetics are two different things – nice transitions or shading will never rectify a fundamentally flawed user experience. Rejecting a framework purely because it apparently contains less bundled eye candy than alternatives still may not mean you’ve chosen wisely.

A green field is increasingly rare

Even in the evolving world of mobile, it is increasingly likely that there will be some existing apps with which the new apps will have to live happily. A few things to consider might be:

  1. Is there already an incumbent framework?
  2. Is the existing framework capable of building what is required to the right quality in the given timescales?
  3. Are the developers effective using it?

If the answer to the above is a clean sweep of “Yes”, then unless there is a non-technical reason why the existing framework should be abandoned,then that probably suggests that sticking with what is there is the best option.

A hygiene factor for any technology decision, but an important consideration is the current position in the “marketplace” of a given framework. Is the framework under consideration acknowledged by other developers (and vendors) as strategic, or are references thin on the ground?

Skills matter

The accelerated lifecycle of the mobile world means that development time is at a premium. Adopting a framework or approach that is a closer match to the skills available within the organisation means greater opportunity for reuse both of assets and skills, and shortens the time required for developers to get up to speed. Related to the previous consideration, if there is an incumbent framework and the decision is made to replace it then selecting a replacement with some similar characteristics would make sense – e.g. script-centric vs markup centric.

It’s still a browser

The growth of AJAX as a technique in general has placed far greater expectation on the browser environment in terms of its criticality to the delivery of the application. It is easy to forget that for all the enhancement and development since the birth of the internet, fundamentally a browser renders documents, and Javascript is there to augment that core purpose. I’ve always been fairly sceptical of attempting to layer on more tiers of engineering into the browser than are absolutely necessary.

So when looking at the various frameworks,  it should be borne in mind that it’s not necessarily the same as a package selection exercise with enterprise software products. Looking at one framework for the UI, another for handling MVC, another for service invocation and so on may well be overcomplicating things unless that specific combination is absolutely the only way to deliver the experience. It is relatively straightforward, for example, to create a simple MVC framework within most mobile frameworks without introducing the complexity and bloat of yet another framework.

Horses for courses

And finally a variation on the consultant’s answer of “it depends”, but it is certainly true that choosing the right framework depends on what you want to do with it.

For example, I like prototyping using jQuery for its lightweight CSS/HTML-centric approach, whereas for construction of reusable components in an enterprise app I can see where the Dojo Toolkit with its Java-like packaging and widget framework has its strengths. That’s not to say you can’t prototype that way in Dojo or indeed create widgets in jQuery, just they each have different strengths depending on the use for me personally. So a key consideration here when evaluating a framework is determining what its core use is going to be – for example, do you need to make a strategic decision for a new service or are you looking to put something disposable together quickly? In the latter case, depending on skill levels some may choose not to use a framework at all.

Sample Dojo Widget

Without a sample it can take a fair amount of debug effort to work out all the moving parts to create a Dojo Widget programmatically. This posting builds on the guidance on the Dojo site based on my own experiences and frequently-asked-questions of others.

Key things to remember:

  • Careful which widgets you subclass – some of them require inside knowledge and can cause a lot of pain. Try and use dijit._Widget if you can.
  • You need a constructor to get hold of the HTML attributes of your widget from the markup and create the fields on your widget class so they get found by the template. That stumped me for a while, the attributes aren’t mixed into the widget by default.
  • Be sure to set templateString to an empty string.
  • All Dojo attachPoint does is create a variable with the name you specify pointing to the particular node in the template. Just means you don’t have to lookup by ID within the widget, you can just use the variable direct. This is described in the Dojo book but I found that it made things seem a lot more complicated than they actually are.
  • attachPoints to widgets give you the widget handle, not the DOM node.

The JavaScript

Here’s the JavaScript for the Widget. It periodically emits a publication to a topic to which things to be refreshed on the UI subscribe.



	[dijit._Widget, dijit._Templated],
		templatePath: dojo.moduleUrl("com.ibm.sample.ui","templates/RefresherWidget.html"),
		templateString: '',
		widgetsInTemplate: true,
		replaceVars: true,

		id: "",
		minimum: "2",
		maximum: "30",
		value: "10",

		timeout: null,

		constructor: function(args) {
			this.id = args.id;
			this.minimum = args.minimum;
			this.maximum = args.maximum;
			this.value = args.value;

		postCreate: function() {
		   this.checkbox.slider = this.slider;
		   this.checkbox.widget = this;
		   this.checkbox.onChange = function () {

		   this.link.href = 'javascript: dojo.publish("ibm/sample/widgets/refresher/'+this.id+'", []);';

		   this.inherited("postCreate", arguments);

		notifyChange: function(autoFlag) {
			if (autoFlag) {
				if (this.timeout == null) {
			} else {
				if (this.timeout != null) {
					this.timeout = null;

		automaticRefresh: function() {
			var funcString = "dojo.publish('ibm/sample/widgets/refresher/"+this.id+"', []); dijit.byId('"+this.id+"').automaticRefresh();";
			this.timeout = window.setTimeout(funcString, this.slider.getValue()*1000);


Here is the HTML:

<div dojoAttachPoint="containerNode">
	<table border="0" cellpadding="0" cellspacing="0">
				<input dojoAttachPoint="checkbox" dojotype="dijit.form.CheckBox"
		    <td width="5"></td>
		    <td><span class="lotusInlinelist">Automatic refresh</span></td>
			<td colspan="3" height="5px;"></td>
	<div dojoAttachPoint="slider" dojoType="dijit.form.HorizontalSlider"
		minimum="${minimum}" maximum="${maximum}" value="${value}" intermediateChanges="true"
		style="padding-top: 5px; height: 20px; width: 150px;">
	<ul class="lotusInlinelist">
		<li class="lotusFirst"><a onClick=";" dojoAttachPoint="link" class="lotusAction" href="javascript:;">Refresh now</a></li>