MQTT over WebSockets

Overview

Extending WebSphere MQ to include support for WebSockets, allowing messaging to web browsers, including mobile browsers, without any additional client software

Background

I’ve talked about MQTT before – a lightweight messaging protocol that I’ve used both on personal projects and my day job.

From mqtt.org:

It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium.

I’ve used it in CurrentCost projects, mobile apps and with small and embedded computers like my Slug.

But in all cases, I’ve needed MQTT client software to talk to the messaging server. Whether writing in C, C#, Java or Python, I’ve needed a client library to get me started, something that knows the sequence of packets that make up the MQTT protocol.

It’d be useful to have a zero-install MQTT client: an MQTT client app delivered over the web, without the user needing to install any additional client libraries, or resort to Java applets.

What this does

One possible way to do this could be WebSockets. Part of HTML5, this is a protocol that describes how to do two-way messaging between web servers and web browsers. And I mean proper two-way communication, including push-notification from the server to the browser, without resorting to hacks or kludges like long polling or hidden iframes.

It’s an emerging protocol, still in draft form, but there are a few implementations around so there are already a few browsers that know how to manage WebSockets.

We’ve been exploring recently how this could work with MQTT – the aim was to build in support for WebSockets into an MQTT messaging server: IBM WebSphere MQ (WMQ).

I’ve mentioned WebSphere MQ before, as back in the dim-and-distant past (well, five or six years ago) I used to be a developer of it.

It’s one of the server implementations that support the MQTT protocol.

By adding support for WebSockets to it, it means that WMQ could send and receive messages to web browsers. It means a web app could be a fully-fledged MQTT client.

How we made it

The first step was server-side – extending the WMQ server to include support for WebSockets. For our proof-of-concept, we wrote our own implementation of the WebSockets protocol from the specification, building it into the WMQ engine.

We wrote a small, lightweight JavaScript client library – something that will manage the WebSockets connection, and provide a web developer with a simple API for publishing and subscribing to MQTT messages without needing to know much about the protocol.

It’s still a work-in-progress, but even now, without any gzipping this library is only 20KB and has no dependencies on any additional libraries or frameworks. So it’s already looking pretty good for use in mobile web apps.

Finally, we wrote a few web apps to demo how the JS client library can be used, and how MQTT can work over the web. This included a few mobile web apps, showing how you could push this to browsers in smartphones on the edge of the network.

What this means

This is pretty cool. I’ve wanted to talk about this since January, but it’s one of the things being demonstrated at our Impact conference in Las Vegas so I’ve had to be patient. But Impact starts today, so I can finally share.

It’s another way web apps could be a part of an overall enterprise messaging ecosystem. (To be fair, it’s not the first way – we’ve had REST HTTP support for years, and I’ve seen more than one web page using Java applets to receive MQTT messages).

But, for many web apps, I think this is nicer. No applets, mobile friendly, and no AJAX polling.

Let me pick one simple, trivial example.

Imagine a web page with the latest score from a Wimbledon tennis match.

To keep it up to date, you either get the user to keep hitting refresh, or you write JavaScript to essentially do that for them – polling automatically and updating the page.

Tennis scores change so frequently and rapidly that you will either show scores that are out of date (bad for the user) or have to poll very frequently (bad for the server).

A push-based approach is much more efficient – each web page tells the server which match it is interested in, and when a new score is available, the server pushes it to the browser where the page can be updated instantly.

Up to the second updates. A mobile web app for the real-time web.

Without needing large numbers of clients to hammer the server with polls. And all backed by server technology with over a decades experience in doing ultra secure, massively scalable, bullet-proof reliable messaging.

Awesome. :-)

Where it works

To be fair, not all mobile web browsers support WebSockets out-of-the-box. But iPhone and BlackBerry do, which was enough for a proof-of-concept.

On the desktop, Chrome and Safari come with WebSockets support, and you can enable WebSockets in Firefox and Opera, too.

Where to find it

Don’t know, just yet. Sorry. It’s an unreleased technology preview. Something that shows you the sort of stuff we’re thinking about and playing with.

Legal types would likely want me to stress that you can’t treat my blog like a product announcement or guaranteed release commitment, but that’s obvious, right? :-)

Who is “we”?

I didn’t do all of this myself, so I should add a few names in here. First and foremost, a massive amount of work was done by Andrew Banks in WMQ Development and James Thomas in Emerging Tech.

And when I ran into problems getting to grips with the nuances of implementing the WebSockets protocol, a bunch of other colleagues also took time to help me, including Rob Smart, Ben Hardill and Patrick Mueller so a grateful thanks to them, too.

#ibmimpact

Tags: , , , , , , ,

6 Responses to “MQTT over WebSockets”

  1. David Illsley says:

    I’ve been wondering who’s been doing this… someone had to be. Does it use HTML5 Local Storage for client persistence, or is it in-memory only?

  2. dale says:

    Yup, localStorage is used for persistence – although this is optional. If a browser doesn’t support it (or the user refuses permission for it) it’ll work without it.

  3. Roger says:

    I presume you’ve heard that some other people have done node.js MQTT work. I’ve not looked at it myself, but there’s a client at https://github.com/yilun/node_mqtt_client and a server at https://github.com/adamvr/MQTT.js

  4. dale says:

    Hi Roger

    I’d seen a couple of tweets about the server bit last week, and instapaper’d them, but I’ve not gotten around to having a proper look yet.

    Sounds great though – useful to have a WebSockets option for smaller or personal projects. I can’t install WebSphere MQ at home :-)

    Kind regards, D

  5. Jim Warner says:

    I have your Android client running (beautifully) to _receive_ and display MQTT messages. I have RSMB running and want to be able to _publish_ to the broker to test the other half of the process.

    I have been through your client many times trying to find where the section is that enables sending. Am I really being dense and missing something in front of my nose. Or is really not there?

  6. dale says:

    Hi Jim

    No – that’s true, I didn’t cover sending messages. This was because there wasn’t much Android-specific to say about it – the publish method in the wmqtt.jar can be used as normal (albeit wrapped in an AsyncTask, as with any network IO operation).

    There is Javadoc in the zip file available at http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24006006&loc=en_US&cs=utf-8&lang=en – but basically you need to call publish() on the MqttClient object, giving it a topic to publish on and a byte array containing your message.

    Hope that helps – give me a shout if you run into problems

    Kind regards, D