Tag Archives: node

Parsing LLAP packets using NodeRED on the Raspberry Pi

It’s taken a few weeks but here is the next instalment in my promised series of articles. Part of the scenario requires me to detect a particular scenario using a remote sensor and relay it on to a variety of other systems via my Raspberry Pi. The sensor itself is assembled using the rather brilliant Ciseco Wireless Inventor’s Kit which provides all you need for prototyping remote sensors and transmitting data over a short range radio link back to a waiting Pi.

It’s a pretty simple yet powerful set of kit. You build your sensor using the breadboard and XinoRF module supplied which communicates with a Slice of Radio transceiver module you attach to the Pi’s GPIO pins. Your applications then send and receive data to and from the remote sensor via the Pi’s serial port and you’re away. The protocol used for the serial communications is designed by Ciseco and is called Lightweight Logical Application Protocol (LLAP). It takes the form of a simple fixed-field text datagram and as such can be parsed by pretty well any programming language that can interrogate the serial port.

In my case I’m using NodeRED to parse the LLAP packets so I can invoke other services in response to an event triggered via a sensor. The event is detected by a switch (a pressure mat in fact) that sets the D03 pin on the XinoRF module to be “high”. The nice thing about how the XinoRF works is that it operates in an event-driven way — i.e. you don’t need to poll to check the state of the pin, it transmits when there is a change in the state from “low” to “high” or vice versa. The other XinoRF pins perform different functions in different ways, but I thought I’d show how to at least send and receive so others can tailor to their scenario.

I’m assuming here that folks are familiar with the NodeRED concepts, so will get straight into the meat of how it works.

Configuring A serial input node to Receive llap packets into a flow

The first thing to do is configure the serial input node in NodeRED. The settings for my node are shown below.

Screen Shot 2014-10-14 at 17.56.37

You can confirm the serial port name using the RasWIK documentation and tools. A particular point to note is the Split input settings which determines at which point the input node offers up data to the next node in the flow. This can be set to a delimiter, or set to a timeout, wherein if no data is received within a certain period, the transmission is considered complete. I discovered that the latter timeout approach worked with a setting of 250 milliseconds, though it does introduce the possibility of multiple LLAP packets being received in one go, which we now need to address within the next node in our flow.

Creating a function node to segment multiple llap packets into individual messages

For greatest efficiency want all subsequent nodes in the flow to be designed to ingest a single LLAP packet in a single message. To ensure this downstream behaviour is honoured we create a function node after our serial input node to check for multiple packets and segment them into individual messages if so. The way NodeRED is designed to work, we can return an array of messages and it will cycle through each message in turn, invoking the remainder of the flow as if triggered by a single message.

The first thing we do in our function node is test for the simple case of a single packet which we can test by the length of the payload, LLAP packets being 12 characters in length.

var instring = msg.payload;
if (instring.length<=12) {
   console.log("Single packet: "+instring);
   return msg;
} 

If the length of the payload is longer, it means we have multiple packets so we need to iterate over the payload, extracting each packet into an array of new messages to return at the end of the function.

else {
   var packets = [];
   console.log("Multiple packets: "+instring);
   while (instring.length>=12) {
      var packet = instring.substr(0, 12);
      console.log("Unspliced packet: "+packet);
      packets[packets.length] = {payload:packet};
      instring = instring.substr(12);
      console.log("instring is now length "+instring.length);
   }
   return [packets]; 
}
return null;

An important point to note (highlighted above) is that although we have created an array of payload objects, NodeRED requires that we in turn encase that array in another array when we return it for processing downstream.

Parsing the LLAP packet

In our subsequent function node, we can now parse the single LLAP packet to determine whether we have a “high” event for the D03 pin we’re interested in. The format of an LLAP packet in this case would be:

a--D03HIGH--

Our code simply needs to check the packet that is received, and parse it see if it corresponds to the above event. Note that in a simple scenario we could just do a direct string comparison between what we’ve received and a template packet such as that above, but if we have multiple stations (denoted by characters 1 and 2, “–” in the example) then we might have more complex flow control logic. My function node simply emits the word “HIGH” if we’ve got a D03 “high” event from the XinoRF.

var thePayload = msg.payload;
var pin = thePayload.substring(3,6);
if (pin == "D03") {
   // Now test if it was HIGH
   if (thePayload.substring(6,10) == "HIGH") {
      console.log("HIGH on D03");

Now in my scenario, the switch (pressure mat) could toggle on and off in quick succession which would technically send a number of LLAP packets back to the Pi. Within a certain time window, I’m only interested in a single “high” event, so have added some logic using NodeRED’s context to test when I last received a “high” event and only taking action if it was more than a minute ago.

      var timeNow = new Date();
      console.log("timeNow is "+timeNow);
		
      if (context.lastHigh) {
         console.log("context.lastHigh is "+context.lastHigh);
         var lastTimeMillis = context.lastHigh.getTime();
         var currentTimeMillis = timeNow.getTime();
         var delta = currentTimeMillis - lastTimeMillis;
         if (delta < 60000) { 
            console.log("Less than a minute since last HIGH: "+
               delta+" ms.");
            return null;
         }
         console.log("Over a minute, actually "+delta+" ms.");
      } else {
         console.log("Continuing.");
      }
      context.lastHigh = timeNow;
      return {payload: "HIGH"}
   }
}
return null;

So, if we get a “high” event and we haven’t seen one for at least a minute, the subsequent nodes in the flow will receive a packet with a payload of “HIGH”.

And with that we’re free to continue our processing, which could be as simple as a debug node. Here’s a snapshot of my NodeRED workspace — in this case I’m using the “high” event to trigger a Tweet.

Screen Shot 2014-10-14 at 18.33.17

 

 

Simple database operations via HTTP with Node JS, MongoDB and Bluemix

In my previous posting I mentioned how I’d planned to harvest some of the useful fragments from a home project that might be useful to others. This time I thought I’d capture the simple mechanism I created to keep an audit database for my application using MongoDB running on Bluemix.

I opted for Mongo due to its simplicity and close integration with JavaScript — a perfect option for creating something quickly and easily to run in a Node JS environment. Mongo is a NoSQL database, meaning that you don’t have to define a specific schema for data, you simply store what you need as a JSON object in the form of key/value pairs. This means you can store and retrieve a wide variety of different data objects in the same database, and aren’t constrained by decisions made early on in the project if your needs change. Whilst it wasn’t a design point of mine, Mongo also is designed to scale too.

As described previously, I’m using the Node JS Web Starter boilerplate as my starting point. I’ve previously added the Twilio service, now to add Mongo, I simply select the MongoLab service from the Data Management set of services on Bluemix console, and add it to my application.

Screen Shot 2014-08-06 at 15.55.50When you create the MongoLab service for the app, Bluemix provides a link to the MongoLab admin page. The nice thing about MongoLab as a service provider is that it gives you nice user friendly tools for creating Collections, reviewing documents etc. I created a collection in there called easyFiddle using the web console.

Screen Shot 2014-08-06 at 16.01.55

Having configured the Mongo Collection, the next step is to make sure that the Mongo libraries are available to the Node JS environment. As with Twilio before, we simply make sure we have an entry in the package.json file.

{
   "name": "NodejsStarterApp",
   "version": "0.0.1",
   "description": "A sample nodejs app for BlueMix",
   "dependencies": {
      "mongodb": "1.4.6",
      "express": "3.4.7",
      "twilio": "^1.6.0",
      "jade": "1.1.4"
   },
   "engines": {
      "node": "0.10.26"
   },
   "repository": {}
}

Just as before, Bluemix will handle the installation of the packages for us when we push the updates to the server.

Within our code, we now need to instantiate the Mongo driver objects with the credentials generated by Bluemix for the MongoDB instance running in MongoLabs. Bluemix supplies the credentials for connection via the VCAP_SERVICES environment variable.

var services = JSON.parse(process.env.VCAP_SERVICES || "{}");
...
var mongo = services['mongolab'][0].credentials;

We will reference the mongo object to retrieve the credentials when we connect to the database.

As I did with Twilio, I am using a simple HTTP-based service that will in this case create an audit record in a database. I’m using Express again (as described previously), together with the same basic authentication scheme. My service works on HTTP GET requests to /audit with two query parameters device and event.

// Leave out the auth parameter if you're not using an 
// authentication scheme
app.get('/audit', auth, function(req, res) {
   var theDevice = req.param("device");
   var theEvent = req.param("event");

Now it’s a case of connecting to Mongo, and inserting a JSON object as a document to contain the two parameters.

   mongodb.MongoClient.connect(mongo.uri, function(err,db) {
      // We'd put error handling in here -- simply check if 
      // err is set to something
      var collection = db.collection('easyFiddle');
		
      var doc = {
         "device": theDevice, 
         "event": theEvent, 
         "date": new Date()
      };
		
      collection.insert(doc, {w:1}, function(err, result) {
         // Again, we'd put error handling in here
         res.json(doc);
      });			
   });
});

And that’s it. We can now create an audit entry using our browser if we choose, with a URL that looks like:

http://appname.mybluemix.net/audits?device=TEST&event=TESTING

I’ve added other services using the same method to variously query and delete all the records in the Collection. Whilst I’ll not include them all here, note that the syntax for deleting all the records in a collection is a bit non-obvious — the examples show you how to delete a record matching a given key/value pair, but are less clear on how to delete them all. You do so simply by supplying a null instead of a name/value pair the method call:

collection.remove(null, {w:1}, function(err, result) {
   // Error handling
   // ...
});

Note that the result variable will contain the number of records deleted.

Hopefully this posting has helped get you going anyway. A great resource to help you navigate your way around the Node JS API for Mongo can be found in the MongoDB documentation.

Sending SMS messages using Twilio and Bluemix

I’ve been tinkering with an Internet of Things project at home for a while which I’ll write up in due course, but in the course of doing so have knocked up a few useful fragments of function that I thought I’d share in case other people need them. The first of these is a simple Node.js app to send an SMS message via Twilio using IBM Bluemix.

There’s lots of material on Twilio and Bluemix but by way of a very rapid summary, Twilio provide nice, friendly APIs over telephony-type services (such as sending SMS messages), and Bluemix is IBM’s Cloud Foundry-based Platform-as-a-Service offering to enable developers to build applications rapidly in the cloud. Twilio have created a service within Bluemix that developers can pick up and use to enable their applications with the Twilio services. One of the things I wanted for my application was a way of notifying me that something had happened, and a text message suited my needs nicely. Twilio provide a free trial service with a few restrictions which you can upgrade once you want to do something serious.

To begin with, I created myself a Node application on Bluemix using the Node JS Web Starter application boilerplate provided:

My approach was to create a simple HTTP-based service that I could invoke with the destination phone number, and the message itself as parameters. To make the Twilio service available to my Node application, it was simply case of adding the service to my application in Bluemix. Twilio is listed as one of the Mobile services:

Once you have added the Twilio service, you configure it in Bluemix by providing the Account SID and Auth Token values that you find on the account details page once you have registered and logged in to Twilio.

The Node JS Web Starter boilerplate creates a simple template for a web server that serves up pages using the Express framework on top of Node. Express is handy, in that it provides a useful framework for handling HTTP requests, so I decided to stick with it for my HTTP service. The first change I needed to make to the boilerplate was to add a reference to Twilio in the package.json file so that the modules would be available to my code.

 
{
   "name": "NodejsStarterApp",
   "version": "0.0.1",
   "description": "A sample nodejs app for BlueMix",
   "dependencies": {
      "mongodb": "1.4.6",
      "express": "3.4.7",
      "twilio": "^1.6.0",
      "jade": "1.1.4"
   },
   "engines": {
      "node": "0.10.26"
   },
   "repository": {}
}

When you push your updated code to Bluemix, Bluemix automatically does the npm install to go and fetch the modules based on the package.json.

Within the app, you then need to set up the Twilio package ready for sending messages. First, we need to require the Twilio package so we can access the service from our code, and then retrieve the Account SID and Auth Token values configured in Bluemix from the VCAP_SERVICES environment variable that Bluemix provides to the Node runtime.

var twilio = require('twilio'); // Twilio API
...
var services = JSON.parse(process.env.VCAP_SERVICES || "{}");
...
var twilioSid, twilioToken;
services['user-provided'].forEach(function(service) {
   if (service.name == 'Twilio-ph') { 
      twilioSid = service.credentials.accountSID;
      twilioToken = service.credentials.authToken;
   }
});

Note that Twilio-ph is the name I gave to the Twilio service when I added it to my application in Bluemix, yours may vary so remember to change it if different.

The environment is now set up, so now we need to create our HTTP handler using Express to form the basis of our service.  I’ve added Basic Authentication to my handler to prevent spam to my Twilio account, this is nice and easy to do using Express.

// Configured environment variables to protect Twilio requests
var USER = process.env.USER;
console.log("USER = "+USER);
var PASSWORD = process.env.PASSWORD;
console.log("PASSWORD = "+PASSWORD);

// Basic authentication to restrict access to my services.
var auth = express.basicAuth(USER, PASSWORD);

I’ve used environment variables that I’ve set in the Bluemix environment, clearly in a production environment one would use a proper directory. You can set your own environment variables within your application by going to the Runtime view of your application and selecting the USER-DEFINED button on the main panel.

The HTTP handler simply looks for the URI pattern /twilio as a GET request and reads the destination telephone number and the message content as query parameters. The auth object passed in applies the Basic Authentication rule defined previously.

app.get('/twilio', auth, function(req, res) {
   var toNum = req.param("number");
   var twilioMessage = req.param("message");
   var fromNum = 'your Twilio number';

Your Twilio number can be found on the Twilio account details page.

Twilio make it really easy to send a message using their twilio.RestClient API wrapper class. You simply instantiate an instance of the twilio.RestClient class, and invoke the sendMessage method with a JSON object containing parameters describing who the message is to, the number it is sent from and the message to be included in the SMS. You provide a callback function that is invoked when the request is completed.

   var client = new twilio.RestClient(twilioSid, twilioToken);
   client.sendMessage(
     {
         to: toNum,
         from: fromNum, 
         body: twilioMessage
      }, 
      function(err, message) {
         if (err) {
            console.error("Problem: "+err+": "+message);
            res.send("Error! "+err+": "+message);
            return;
         } else {
            res.send("Done!");
            console.log("Twilio message sent from "+fromNum+
            " to "+toNum+": "+twilioMessage);
         }
      }
   );
});

Once deployed, the service can be invoked with a URL in the form described below:

http://appname.mybluemix.net/twilio?number=number&message=mess

If invoked programmatically the authentication credentials will be required in the HTTP header. If tried from a browser, the browser will prompt for a username and password combination. Ultimately you’ll receive an SMS on your phone:
twilio sample
And there it is. I can now trigger SMS messages either from my NodeRED flows, browser or any other apps I might write.