Quick guide to using SSL/TLS support in MQTT and the micro broker

I have recently been using the new TLS/SSL support for the micro broker and MQTT v5 client introduced in Expeditor 6.2. I have recorded the following instructions as a quick-start guide to create a simple, server-authentication SSL connection using a self-signed certificate. Further information on configuring SSL for micro broker can be found on the Redbooks Wiki.

1. Create key pair and keystore for the micro broker

Find the binaries directory for the JRE you are using. For the Expeditor DRE (J2SE) runtime, for example, they can be found in a folder named similar to (may vary slightly with build levels)  C:\Program Files\IBM\Lotus\Expeditor\rcp\eclipse\plugins\com.ibm.rcp.j2se.win32.x86_1.6.0.20081029a-200811140851\jre\bin.

When in the JRE binaries directory, issue the following command to create a certificate:

keytool -genkey -alias sampleCert -keystore c:\sample\sampleStore.jks -keypass default -storepass default -dname “CN=Sample, OU=PVC, O=IBM, C=UK” -keyalg RSA

Make sure you are using the same JRE to create the keystore as you use to run the micro broker as the keystores aren’t compatible across different JREs.

2. Write the logic in a plugin to create an SSL listener on the broker

This is achieved using the micro broker administrative API. The following will add an SSL listener alongside the default non-encrypted listener.

public static final String BROKER_INSTANCE_NAME = “Sample_Broker”;
private static final int  BROKER_SECURE_PORT = 8883;
private static final String BROKER_KEYSTORE = “C:/sample/sampleStore.jks”;
// Password as per the keytool sample above
private static final char[] BROKER_KEYSTORE_PASSWORD ={‘d’,'e’,'f’,'a’,'u’,'l’,'t’};

…

BrokerDefinition def = brokerFactory.createBrokerDefinition(BROKER_INSTANCE_NAME);
def.setDataDirectory(Platform.getInstanceLocation().getURL().getFile()+”/microbroker”);
def.setAutoStartEnabled(false);
broker = brokerFactory.create(def);

// Now start up
broker.start();
// Now add SSL support — we want both SSL and non-SSL support
// otherwise we could have specified an SSL-only approach with
// def.setDefaultListenerSecure(true);
// and then set the SSL definition as per the following snippet.
// Note that the broker must be started to add the SSL listener
// if we want both secured and non-secured listeners. The broker
// can only be configured with a single listener initially.
Communications brokerComms = broker.getCommunications();

MQTTTCPListenerDefinition mqttld =
brokerComms.createMQTTTCPListenerDefinition(BROKER_SECURE_PORT);
mqttld.setSecure(true);
ServerSSLDefinition ssld = mqttld.createSSLDefinition();
ssld.setKeyStore(BROKER_KEYSTORE);
ssld.setKeyStorePassword(BROKER_KEYSTORE_PASSWORD);
mqttld.setSSLDefinition(ssld);
brokerComms.addTCPListener(mqttld);

Once your application has been started up (and hopefully your broker configured and running successfully) you
can check it is running by using a command such as (on Windows):

netstat -aN | find “LISTEN”

and see something listening on port 8883 (and 1883 for that matter if you use the above snippet).

3. Set the connection URI of the MQTT v5 client to use SSL

In the simple (server authentication) case, an MQTT v5 client can be connected using SSL simply by modifying the MQTT URI used at connection time from something like

tcp://localhost:1883

to

ssl://localhost:8883

The MQTT client will then know to attempt the connection using SSL.

A note on using self-signed certificates

By default, Expeditor running in GUI mode will offer a dialogue to the user prompting them as to whether they wish to accept the server certificate or reject it since the provider is not a recognised certification authority (i.e. it’s a certificate we’ve created ourselves for testing).

There are a couple of things to watch out for here. If the GUI is not yet available (i.e. the MQTT client application has loaded and attempted to connect before the GUI has loaded) or is not available (i.e. Expeditor is running “headless”) by default the Expeditor SSL support will automatically reject certificates from non-trusted certification authorities.

In order to prevent this issue, we must modify the plugin_customization.ini file
which by default is installed on Windows in

C:\Program Files\IBM\Lotus\Expeditor\rcp

Add the following line to always allow certificates from unknown certification authorities:

com.ibm.rcp.security.jceproxy/ssl.unknowncert.action=ALLOW

Note that an addition command line parameter is required in any Run Configuration profiles using the Expeditor test environment to point at the plugin_customization.ini file:

-plugincustomization “C:/Program Files/IBM/Lotus/Expeditor/rcp/plugin_customization.ini”

WARNING: this will cause Expeditor to always accepted untrusted certificates — this tip is intended for development purposes and should NOT be used in production deployments.

Full details of Expeditor’s SSL configuration options can be found on the Expeditor Info Center.

Using the new MQTT v5 client in Expeditor 6.2

With the launch of Expeditor 6.2 comes the next generation of the MQTT client. MQTT is IBM’s specialist messaging protocol designed for use in fragile or expensive networks (e.g. mobile or satellite links) and in constrained devices such as sensors and mobile devices. This latest version (version 5 of the protocol) adds a number of features including the following:

• Support for the point-to-point (i.e. queues) as well as the traditional pub/sub messaging paradigm.
• Different payload types such as a textual string in addition to the basic byte array of the previous client.
• The ability to specify whether an individual subscription is durable (i.e. survives a disconnection and continues to collect messages) or not on a per-subscription basis. In the previous incarnation, the durability of a client’s subscriptions was specified for the whole client. This could mean resources within the broker were consumed by unnecessarily durable subscriptions.
• The ability to connect to a messaging server over SSL and using authentication credentials.
• A variety of additional header information for messages including expiry and priority as in JMS.
• The ability to start and stop message delivery without having to disconnect and reconnect again. This allows the application to control the flow of messages without the overhead of a full connection handshake each time.

Full details can be found in the Javadoc but I’ve included a simple sample here to get you started.

Install the MQTT v5 client into the Expeditor client runtime

In Expeditor 6.2, the MQTT v5 client is not installed by default with the base client installer. You will need to install it as an additional feature from the update site contained in desktop/updates/platform folder in the install media, even for use in the toolkit. A screen shot showing which feature is required is shown below.

Sample MQTT application

The following examples show a class that connects to a broker and subscribes durably for messages and a class that publishes a message. The subscriber makes use of the message delivery start/stop feature to enable message delivery only when both subscriptions are in place. You will see that I’ve included some tests that determine the type of payload of a given message as well.

Notice the different package/plug-in name for the v5 MQTT client.

import com.ibm.micro.client.MqttCallback;
import com.ibm.micro.client.MqttClient;
import com.ibm.micro.client.MqttConnectOptions;
import com.ibm.micro.client.MqttDeliveryToken;
import com.ibm.micro.client.MqttDestination;
import com.ibm.micro.client.MqttException;
import com.ibm.micro.client.MqttMessage;
import com.ibm.micro.client.MqttSubscriptionOptions;

public class MqttSubscriber implements MqttCallback {

private MqttClient mqttClient = null;
private String TOPIC_SUFFIX_ALERTS = “alerts”;
private String TOPIC_SUFFIX_DATA = “data”;
public MqttSubscriber() {
super();
}
/* MQTT Client API */
public void connectionLost(Throwable arg0) {
//
System.out.println(“Connection to the micro broker lost”);
}

public void deliveryComplete(MqttDeliveryToken arg0) {
//
}

public void deliveryFailed(MqttDeliveryToken arg0, MqttException arg1) {
//
}

public void messageArrived(MqttDestination destination, MqttMessage message) throws Exception {
System.out.println(“Message has arrived over MQTT.”);
String sourceTopic = destination.getName();
// In MQTT v5 we have different payload types.
if (message.getPayloadType() == MqttMessage.PAYLOAD_TEXT) {
String payload = message.getStringPayload();
System.out.println(“Message payload: “+payload);
} else {
// Enforce a “text only” policy.
System.err.println(“Message is not a text string.”);
}
}

public void start(String mqttUri, String name) throws MqttException {
mqttClient = new MqttClient(mqttUri, “Sub_Client”);
mqttClient.setCallback(this);
MqttConnectOptions options = new MqttConnectOptions();
// Wait until we’re ready to receive messages (overrides default)
options.setAutoStart(false);
// We don’t want our state cleaned up, we need to keep
// Durable subscriptions.
options.setPurge(false);
// Connect
mqttClient.connect(options);

MqttSubscriptionOptions subOpts = new MqttSubscriptionOptions();
subOpts.setDurable(true); // We want durable subscriptions
subOpts.setQos(2); // Once and once only delivery (same as V3 QoS).
mqttClient.subscribe(“acme/sample/”+name+”/”+TOPIC_SUFFIX_ALERTS+”/+”, subOpts);
mqttClient.subscribe(“acme/sample/”+name+”/”+TOPIC_SUFFIX_DATA+”/+”, subOpts);
// Ready to receive.
mqttClient.startListening();

System.out.println(“Connected to “+mqttUri);
}

public void stop() throws MqttException {
//
mqttClient.disconnect();
}

}

The following is the complimentary part of the sample that shows how to publish a message using the v5 client. This class is intended to be run as a simple command line utility.

import com.ibm.micro.client.MqttClient;
import com.ibm.micro.client.MqttException;
import com.ibm.micro.client.MqttMessage;
import com.ibm.micro.client.MqttTopic;

public class MqttPublisher {

public static void main(String[] args) {
try {
// Parameters: <uri> <clientid> <topic> <qos: 0,1 or 2> <string data>
MqttPublisher publisher = new MqttPublisher(args[0], args[1]);
publisher.publish(args[2], Integer.parseInt(args[3]), args[4]);
} catch (Exception ex) {
ex.printStackTrace();
}
}
private String uri = null;
private String clientId = null;
public MqttPublisher(String u, String c) {
uri = u;
clientId = c;
}

public void publish(String topic, int qos, String payload) throws MqttException {
MqttClient client = new MqttClient(“tcp://localhost:1883″, “Pub_Client”);
client.connect();
// Notice the new object model for destinations and messages
MqttTopic t = client.getTopic(topic);
// Create a string payload — v5 discriminates between payload types unlike v3.
MqttMessage message = new MqttMessage(payload);
message.setQos(qos);
t.publish(message);
client.disconnect();
}
}

Enjoy !

Bridging to the WebSphere JMS provider from the micro broker

One of the key value propositions in the micro broker is its ability to connect the integration hub at the edge with an enterprise messaging server. The Bridge within the micro broker provides this capability in three flavours:

• MQTT (v3) direct into another micro broker or Message Broker.
• WebSphere MQ JMS – requires MQ JMS client to be packaged as a plug-in to Expeditor.
• Third-party JMS providers using JNDI – requires the third-party JMS client to be packaged as a plug-in within Lotus Expeditor.

The third flavour above is what we use to connect the micro broker to the WebSphere JMS provider found in WAS, WESB and WPS.

This post describes how one configures the bridge to make use of the WebSphere JMS provider.

Note that Expeditor must be configured to use the Device Runtime Environment (DRE) J2SE JDK since the WebSphere JMS client requires a full J2SE runtime to work properly.

Creating the WebSphere JMS administered objects for JNDI programmatically

For the JNDI flavour of the bridge, a JMS Connection Factory and the target JMS Destinations must be made available using a JNDI provider. The Bridge is then configured with the appropriate JNDI keys to the administered objects. At runtime the objects are retrieved from JNDI by the Bridge using the keys provided. The JMS flavours of the micro broker both also require a “sync-queue” in order to honour once-only persistent qualities of service.

There are two approaches to creating the JMS administered objects. One is to use the WebSphere JNDI provider (JNDI over IIOP). To make use of this capability one needs to package the WebSphere JNDI client as an Expeditor plug-in. This is a good approach when Expeditor will be in close and reliable networked proximity to the WebSphere server since the creation of the JMS administered objects can be done automatically with the Messaging Destinations in WebSphere. Similarly use of a remote JNDI repository fits the JEE administrative model in that the objects can be managed independently of the application. In scenarios where the network may be unreliable (or indeed only available at certain times), an alternative approach is to create the administered objects programmatically and bind them into the built-in JNDI provider within Lotus Expeditor. This means even without a network at start-up time, the bridge can at least start, even if it cannot successfully connect straight away.

Packaging the WebSphere JMS client in Expeditor

To do this, we need to make the WebSphere JMS client available to the Lotus Expeditor runtime in the form of an Expeditor Feature.

A reduced footprint variant of the client is available as a feature pack for WAS. In order for these classes to be found by the Expeditor runtime, you will need to create a plugin in Eclipse containing the WebSphere JMS client JAR files, exporting all the interfaces (apart from javax.jms) contained in the JAR in the MANIFEST.MF for the plug-in.

There are a couple of very important points to note:

1. You will need to make sure that the javax.jms package is NOT exported and any classes in this package are removed from the underlying JAR file. Expeditor has its own version of the JMS interfaces already exported in the platform and two plugins exporting the same interfaces cause JVM errors at runtime.
2. Note that when you package your JMS client, the feature.xml file referencing the plug-in should have the unpack attribute set to true for the plugin containing the JMS client. In order for the interfaces contained within the underlying WebSphere JMS client JAR to be exported correctly when deployed, the JAR cannot be nested inside the plugin JAR. This is due to a limitation of how the OSGi class loader mechanism works.

Creating the administered objects

The following snippet shows how the WebSphere JMS administered objects are created and bound into the Expeditor JNDI repository.

public static final String JNDI_NAME_SIB_CONNECTION_FACTORY = "jms/SIBConnectionFactory";
public static final String JNDI_NAME_SIB_OUTBOUND_QUEUE = "jms/SIBOutQueue";
public static final String JNDI_NAME_SIB_INBOUND_QUEUE = "jms/SIBInQueue";
public static final String JNDI_NAME_SIB_SYNC_QUEUE = "jms/SIBSyncQueue";
public static final String JNDI_NAME_SIB_DEAD_LETTER_QUEUE = "jms/SIBDeadLetterQueue";

...

/**
 * Use the WebSphere JMS provider's Factory APIs to create the necessary objects
 * and bind them in JNDI.
 */
private void createAndBindSibJMSObjects() {
	try {
		// Create the ConnectionFactory
		com.ibm.websphere.sib.api.jms.JmsFactoryFactory jff = com.ibm.websphere.sib.api.jms.JmsFactoryFactory.getInstance();
		com.ibm.websphere.sib.api.jms.JmsConnectionFactory cf = jff.createConnectionFactory();

		cf.setBusName("SCA.APPLICATION.domino8Node01Cell.Bus");
		cf.setProviderEndpoints("localhost:7276:BootstrapBasicMessaging");

		// Create the request queue (messages on the outbound topic will be forwarded here)
		JmsQueue inQueue = com.ibm.websphere.sib.api.jms.JmsFactoryFactory.getInstance().createQueue(
					"PDARemoteSales.MQTT_JSONoverJMS_Export_RECEIVE_D_SIB");
		// Messages expiry after 3 minutes if not consumed.
		inQueue.setTimeToLive((long)180000);

		JmsQueue outQueue = com.ibm.websphere.sib.api.jms.JmsFactoryFactory.getInstance().createQueue(
		"PDARemoteSales.MQTT_JSONoverJMS_Export_SEND_D_SIB");
		// Messages expiry after 3 minutes if not consumed.
		outQueue.setTimeToLive((long)180000);

		JmsQueue deadLetterQueue = com.ibm.websphere.sib.api.jms.JmsFactoryFactory.getInstance().createQueue(
		"PDARemoteSales.Microbroker.DeadLetter");
		// Messages expiry after 3 minutes if not consumed.
		deadLetterQueue.setTimeToLive((long)180000);
		JmsQueue syncQueue = com.ibm.websphere.sib.api.jms.JmsFactoryFactory.getInstance().createQueue(
				"PDARemoteSales.Microbroker.SyncQueue");
		// Messages expiry after 3 minutes if not consumed.
		syncQueue.setTimeToLive((long)180000);

		try {
			InitialContext ctx = new InitialContext();
			ctx.bind(JNDI_NAME_SIB_CONNECTION_FACTORY, cf);
			ctx.bind(JNDI_NAME_SIB_INBOUND_QUEUE, inQueue);
			ctx.bind(JNDI_NAME_SIB_OUTBOUND_QUEUE, outQueue);
			ctx.bind(JNDI_NAME_SIB_SYNC_QUEUE, syncQueue);
			ctx.bind(JNDI_NAME_SIB_DEAD_LETTER_QUEUE, deadLetterQueue);

			System.out.println("All JMS SIB objects bound");
		} catch (NamingException ex) {
			System.err.println("Failed to bind JMS instances.");
			ex.printStackTrace();
		}

	} catch (JMSException e) {
		System.err.println("Failed to create JMS instances");
		e.printStackTrace();
	}

}

The first portion of the code uses the WebSphere JMS client’s Factory classes to instantiate the JMS administered objects with the latter portion binding them into Expeditor’s JNDI provider (note the Expeditor InitialContext will be used by default within an Expeditor environment).

Configuring the micro broker Bridge

The following snippet shows how the Bridge is configured using the micro broker’s administrative API. In this scenario we are bridging into WPS sending messages from a topic in the micro broker into a queue in WPS and from a queue in WPS back to a topic in the micro broker. By this point we have looked up a LocalBroker instance from the micro broker BrokerFactory service.

// Obtain a handle to the broker's bridge
Bridge bridge = broker.getBridge();

// Create a pipe definition -- this is the root of all bridge links
PipeDefinition pipe = bridge.createPipeDefinition("sibPipe");

JNDIConnectionDefinition connectionDefinition = bridge.createJNDIConnectionDefinition(pipe.getName()+"_Connection");

// Set the Initial Context to be used by the bridge to retrieve the administered objects
connectionDefinition.setInitialContext("com.ibm.pvc.jndi.provider.java.InitialContextFactory"); // uses the parameterised initial context factory i.e. XPD
connectionDefinition.setConnectionFactoryKey(JNDI_NAME_SIB_CONNECTION_FACTORY);
connectionDefinition.setDeadLetterKey(JNDI_NAME_SIB_DEAD_LETTER_QUEUE);
connectionDefinition.setSyncQKey(JNDI_NAME_SIB_SYNC_QUEUE);
connectionDefinition.setURL("none"); // Not meaningful for Expeditor JNDI but *is* still required to be set.

pipe.setConnection(connectionDefinition);

// Create an outbound flow that reads from a topic called "localoutbound" and
// puts to the remote queue.
FlowDefinition outbound = bridge.createFlowDefinition("outboundFlow");
// Set the source to be a single topic, "localoutbound"
outbound.setSources(new DestinationDefinition[] { bridge.createTopicDefinition("localoutbound")});
// Set the destination to be a queue on the remote WPS, using the reference bound in JNDI
outbound.setTarget(bridge.createJNDIDefinition(JNDI_NAME_SIB_INBOUND_QUEUE));
// Add the flow to the pipe
pipe.addOutboundFlow(outbound);

// Create an inbound flow that reads from a remote queue called "outbound" and
// puts to a topic called "localinbound"
FlowDefinition inbound = bridge.createFlowDefinition("inboundFlow");
// Set the source to be a queue on the remote WPS, using the reference bound in JNDI
inbound.setSources(new DestinationDefinition[] { bridge.createJNDIDefinition(JNDI_NAME_SIB_OUTBOUND_QUEUE)});
// Set the destination to be a topic called "localinbound" on the local broker
inbound.setTarget(bridge.createTopicDefinition("localinbound"));

// Add the flow to the pipe
pipe.addInboundFlow(inbound);

// Pipe is configured, add it to the bridge
bridge.addPipe(pipe);
// Start the pipe
bridge.startAllPipes();

The pipe is now ready for use.

Gartner Top 10 Strategic Technologies for 2009

…can be found here.

Interesting stuff, particularly as it mentions specifically WOA and enterprise mashups, both of which I’ve discussed earlier and described in my observations on 2008. The Green IT piece also resonates with me, as the focus on low power consumption and many of the power monitoring requirements that are starting to emerge all plays to the strengths of the pervasive messaging and edge SOA technologies I’ve described previously. Cloud is also related, especially in terms of how users interact with the cloud and rapidly build what they need — WOA again becomes important.

What is Extended Integration?

When we talk about SOA and integration software, we typically think of enterprise SOA middleware that lives in a data centre. For example, we think of an Enterprise Service Bus implemented using messaging middleware like IBM WebSphere ESB forming the backbone for the organisation’s IT applications. All of this infrastructure is core to achieving the value proposition of SOA in enterprise terms.

To build on this foundation and put this capability into the hands of users, we need to expand our thinking to consider how we reach out beyond the data centre. This causes us to consider things like:

• Who are the users of the system and what is important to them in business terms?
• How do they need to interact with the system? For example are they in offices on a laptop or desktop PC or mobile?
• How is the organisation structured? Is it a set of large office complexes or a network of branches that interact with a central back-office (such as in retail or banking)?

All of these concerns and more are addressed in the field of what we have come to know as Extended Integration. This whole class of problems relate to extending the reach of SOA out of the data centre and into the hands of the people who need to use it. Extended Integration gives rise to conceptually to a three-tiered architecture as follows:

1. The enterprise SOA platform, such as an ESB, BPM and other enterprise middleware servers.
2. An intermediate integration tier translating the protocols and semantics of the enterprise SOA platform into a form that is easily consumed by the end user applications. This might be in itself an enterprise-class server like WebSphere Application Server (WAS) or a smaller footprint hub like the Lotus Expeditor micro broker. Whilst logically separate from the primary SOA platform, this tier might still be situated within the data centre or remotely, for example deployed in a retail branch.
3. A consumer tier built using appropriate client technologies such as Rich Internet Application (RIA), rich desktop or mobile client or indeed solid state sensors and actuators when the application is consumed by a machine.

The distinction of the middle layer is at the heart of the Extended Integration thought, that of having an intermediate tier to translate between the service orientation of the SOA platform and the required service orientation of the consuming applications.

This way of viewing SOA solutions sharpens the focus particularly on some particular technologies we have available to us today.

Web 2.0, WOA, widgets and mashups

No surprises that any modern IT-related observation includes reference to Web 2.0. This is not a spurious reference I hasten to add, simply an illustration of how a set of technologies and architectures that fall under the Web 2.0 banner appear when viewed through this Extended Integration perspective.

Web Oriented Architecture (WOA) is about recognising that the kind of modern, flexible, heterogeneous web applications we see in the Web 2.0 world are constructed using fine-grained services that take SOA a step further. It is a web-centric view of the world consisting of networked resources and URLs, lightweight contracts between services (i.e. unlike WS-*) and the key to the kind of end-user IT systems that can respond rapidly to business change that will be most needed in these turbulent times. Where we have WS-* underpinning the high qualities of services required for Enterprise SOA, we have REST, URIs, feeds and HTTP underpinning the consumer channels where more often we require a quick, simple and short-term answer in a lightweight device such as a web browser.

Furthermore, we see far better separation of data from presentation in the WOA model, the Achilles Heel of many Web 1.0 systems, with the increased flexibility that brings. We now have an architectural pattern for the web tier constructed of data feeds in well-known formats (e.g. ATOM, JSON), on standard protocols and semantics (HTTP and REST) and to match it, a similarly modular approach for creating the user interface: widgets. Not that widgets are anything new, but the approach of designing the web UI in terms of modular, interoperable components reflects a sea change in recent times. When HTML took hold in earnest, it was considered so simple in comparison with traditional, programming-oriented desktop UI approaches that the UI was a disposable commodity. In many ways this led down the path of sprawling, point-solutions in the web tier on the grounds that they were relatively cheap and easy to extend and replace. Now when we survey the consequences ten or more years on, we realise that in many cases, significant cost savings in terms of development and maintenance can be achieved by applying the same sort of thinking in the web application tier as we apply in enterprise SOA: modularity and designing for reuse.  In IBM our recommended AJAX framework is the Dojo Toolkit, not least because of its comprehensive widget component model that facilitates such reuse and controlled extension of UI functionality.

Mashups have become an increasingly important evolutionary product of this approach. Mashups mix together WOA resources from a variety of sources to produce new systems quickly. Adopting WOA principles across the board for longer-lived systems means that by developing an application on feeds and widgets, it can be effectively “mashup-enabled” at the same time. The key here is that composition is achieved with business-user tools, not programming. From a business perspective that opens the potential for a “just good enough” application without the need for a lengthy and IT-intensive development project. The key focus from an IT perspective for the mashup environment is in providing a sufficiently rich library of widgets and feeds for the business users to consume.

In Web 2.0 terms, Extended Integration is all about mapping SOA and WOA together and translating the semantics of WS-* into the simple REST endpoints that WOA applications and mashups can easily consume. This aspect of Extended Integration closely aligns with the end-user interaction with the SOA system.

Pervasive messaging and edge integration

In a similar vain, the thought of pervasive messaging has been with us now for around ten years. This flavour of messaging-oriented middleware originated from the domain of embedded computing, solid state sensors and telemetry. A particular scenario in point is Supervisory Control and Data Acquisition (SCADA) in the industrial automation field. At a high-level, pervasive messaging couples together minimal-footprint protocols, semantics to deal with expensive and/or fragile networks and small footprint client libraries designed to run within the bounds of a constrained memory and CPU. IBM’s MQTT protocol provides the means for sensors, actuators and other constrained devices to communicate directly with WebSphere enterprise SOA middleware. It should be noted that the protocol specification is published such that third-party software vendors and specialist hardware can be developed talk MQTT into IBM software.

The development of MQTT was an important first step but it soon became clear that having every device in every location transmitting data directly into the enterprise was something of a two-edged sword. For example, one RFID tag passing a reader may generate many events for an individual tag read. Reconciling is technically straightforward within the enterprise SOA tier, however it means that valuable compute resources are spent within the data centre just performing processes on behalf of the remote sites (more generally known as the edge-of-network). This might be a non-trivial amount of processing for a large retail chain, say, consisting of 1000 stores. The development of first the micro broker and more recently the Really Small Message Broker (RSMB) provide the basis for what we term edge integration. Edge integration is about providing the architectural benefits and capabilities of messaging middleware within a sufficiently small footprint that they can be run with relatively low cost alongside the remote assets concerned. Edge integration provides a focal point for communication between the edge and the enterprise systems.

Some typical requirements satisfied by edge integration include:

1. Better separation of concerns between the edge and the enterprise SOA systems as discussed above.
2. Flexibility and interoperability between edge applications and devices. Device and sensor hardware and software often have highly proprietary low-level APIs. The ability to decouple applications and devices using a messaging paradigm facilitates the extension of the system and replacement of hardware and software with minimal disturbance.
3. The need for the remote site to continue operation when the network link to the enterprise is down. Such requirements are commonly found in retail where it is paramount that the store systems can continue processing transactions even if the store itself is offline. The micro broker acts as a focal point for the edge and allows applications to continue to send messages that are persisted on the edge infrastructure and transmitted when the link to the enterprise returns. Furthermore this bi-directional link is achieved using messaging protocols to ensure the data is delivered in a consistent and reliable fashion.

Further discussion of edge integration can be found in the IBM Redwiki Messaging from the Edge in an SOA environment using Lotus Expeditor 6.2.

Edge integration forms an integral part of the Extended Integration thought both in terms of its history as an extended set of protocols for edge-of-network and conceptually within the three-tiered architecture described above. The edge integration hub is the focal point in translating between the enterprise SOA protocols (such as WebSphere MQ) and the specialist protocols of the constrained environment at the edge of the network (such as MQTT).