{"id":969,"date":"2009-10-10T23:32:22","date_gmt":"2009-10-10T23:32:22","guid":{"rendered":"http:\/\/dalelane.co.uk\/blog\/?p=969"},"modified":"2009-10-10T23:53:02","modified_gmt":"2009-10-10T23:53:02","slug":"writing-an-offline-wiki-client","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=969","title":{"rendered":"Writing an offline wiki client"},"content":{"rendered":"<p>Friday was the seventh IBM Hack Day, and I again got the chance to spend a day playing with some random ideas.<\/p>\n<p>As Hack Days go, I had a surprisingly productive day! I had four ideas on the day: <\/p>\n<ul>\n<li> two mobile hacks (both of which I wrote a chunk of code for),\n<\/li>\n<li> a twitter hack (which never got off the scribbled diagram stage, but it&#8217;s an idea I definitely want to come back to), and\n<\/li>\n<li> a hack to extend an IBM product (which I created an alpha version of) <\/li>\n<\/ul>\n<p>In this post, I&#8217;ll describe what I did for the last of these ideas: writing a client app for the wiki that comes with IBM&#8217;s Lotus Connections. <\/p>\n<p><strong>The idea<\/strong><\/p>\n<p>In the same way that I am writing this post in an offline blogging client, I wanted the same for using wikis: read and make changes to a wiki while offline, with changes uploaded to the online wiki the next time you are online.<\/p>\n<p>This wasn&#8217;t a new idea. In fact, I <a href=\"http:\/\/dalelane.co.uk\/blog\/?p=198\">tried it at IBM HackDay 4 back in 2007<\/a> but the wiki we used at work at the time had no API access for retrieving or updating wiki pages. So I sort of gave up and forgot about the idea.<\/p>\n<p>But now I use <a href=\"http:\/\/www-01.ibm.com\/software\/lotus\/products\/connections\/\" target=\"_blank\">Lotus Connections wikis<\/a> at work. And Lotus Connections does have <a href=\"http:\/\/publib.boulder.ibm.com\/infocenter\/ltscnnct\/v2r0\/topic\/com.ibm.connections.25.help\/c_api_wiki_welcome.html\" target=\"_blank\">an API<\/a> &#8211; an AtomPub API that gives you feeds to know when pages are changed, and a way to publish changes. <\/p>\n<p>So I decided to revisit the idea.<\/p>\n<p><strong>The &#8220;finished&#8221; (ish) hack<\/strong><\/p>\n<p>It&#8217;s still very rough around the edges (this was a HackDay &#8211; I wrote the client code in under a day!) but it already shows the basic idea.<\/p>\n<p><a href=\"http:\/\/www.flickr.com\/photos\/dalelane\/3998628297\/\" title=\"Offline wiki client by dalelane, on Flickr\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/farm3.static.flickr.com\/2522\/3998628297_0a23f76e7f_o.png\" width=\"489\" height=\"318\" alt=\"Offline wiki client\" \/><\/a><\/p>\n<p>The top left view shows the list of your wikis. <\/p>\n<p>Clicking on this fills the list below &#8211; a list of pages in the selected wiki. Clicking on a page in that list opens the contents of the page in the main view on the right. <\/p>\n<p><!--more-->There are a few design decisions perhaps worth mentioning&#8230;<\/p>\n<p><strong>Java &#8211; for cross-platform<\/strong><\/p>\n<p>I thought it&#8217;d be good to write something that (might?) run on Windows, Linux or Mac, so I went with Java. (I did consider basing the hack on my <a href=\"http:\/\/dale.lane.googlepages.com\/\" target=\"_blank\">.NET personal wiki app<\/a>, but I wanted to avoid a Windows-only solution.)<\/p>\n<p><strong>SQLite &#8211; for storing pages offline<\/strong><\/p>\n<p>I did consider using <a href=\"http:\/\/db.apache.org\/derby\/\" target=\"_blank\">Derby<\/a>, as this seems to be <a href=\"http:\/\/developers.sun.com\/javadb\/\" target=\"_blank\">Sun&#8217;s preferred embeddable database of choice<\/a> for Java apps. But in the end, I still went with SQLite because I&#8217;ve used it before and quite like it. <\/p>\n<p>Yeah&#8230; should probably have thought of a more technical rationale. But for the purposes of a Hack Day, I didn&#8217;t really care &#8211; just wanted something I could cache text in!<\/p>\n<p><strong>MQTT &#8211; for being useful while online<\/strong><\/p>\n<p>At a starting point, I could make something which just sucks in a copy of a wiki, so you can read and edit it while you&#8217;re offline.<\/p>\n<p>But I was feeling adventurous on Friday. I wanted more than that. I wanted the client to still be useful while you are online, too.<\/p>\n<p>The main improvement is that the app is (or should be!) quicker to open pages than the web-based version. <\/p>\n<p>But to make it a compelling online client, it needed to be able to keep up-to-date. <\/p>\n<p>If someone else creates a page in a wiki while you are online, it should appear in your list of pages automatically. If a page is edited online, the client app should receive this in the background and download the updated page. <\/p>\n<p>So if you use the client app while online, it should just show you up-to-date (ish?) pages as if you were using the web interface.<\/p>\n<p><object width=\"425\" height=\"344\"><param name=\"movie\" value=\"http:\/\/www.youtube.com\/v\/iEqSHDu9ivs&#038;hl=en&#038;fs=1&#038;rel=0\"><\/param><param name=\"allowFullScreen\" value=\"true\"><\/param><param name=\"allowscriptaccess\" value=\"always\"><\/param><embed src=\"http:\/\/www.youtube.com\/v\/iEqSHDu9ivs&#038;hl=en&#038;fs=1&#038;rel=0\" type=\"application\/x-shockwave-flash\" allowscriptaccess=\"always\" allowfullscreen=\"true\" width=\"425\" height=\"344\"><\/embed><\/object><\/p>\n<p>(<em>Sorry, I suck at video editing, so <a href=\"http:\/\/www.youtube.com\/watch?v=iEqSHDu9ivs\" target=\"_blank\">this video demo<\/a> is longer than it really needs to be!<\/em>)<\/p>\n<p>First idea: polling. <\/p>\n<p><img decoding=\"async\" src=\"http:\/\/i267.photobucket.com\/albums\/ii311\/dale_lane\/091010-polling.jpg\" align=\"left\" hspace=\"10\" vspace=\"10\" style=\"border: thin black solid\"\/><\/p>\n<p>The client app could just poll the Lotus Connections API to know when things change.<\/p>\n<p>Bad idea. <\/p>\n<p>On the (okay, remote!) chance that this app gets used by a large number of people, I&#8217;m guessing the server admins wouldn&#8217;t appreciate thousands of clients hammering their poor server every few minutes. <\/p>\n<p>And infrequent polling (e.g. every 30 minutes) wouldn&#8217;t give me the responsiveness I wanted.<\/p>\n<p>Second idea: compromise. <\/p>\n<p>Write one app that polls the API reasonablly frequently (not too frequently, server admins &#8211; honest!) and put that on a server somewhere. It publishes notifications when there are changes. The wiki client apps can subscribe to receive these notifications. <\/p>\n<p>Multiple (thousands of?) clients get the client GUI benefits of frequently polling, for the cost of only one app polling the Lotus Connections server.<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/i267.photobucket.com\/albums\/ii311\/dale_lane\/091010-push.jpg\" style=\"border: thin black solid\"\/><\/p>\n<p>My server app sucks in the ATOM feed for modified wikis, and uses MQTT to publish any changes. It only publishes the name and modified time for each change &#8211; just enough to tell the wiki client apps that they need to download the page again.<\/p>\n<p>But I didn&#8217;t want all the (thousands of &#8211; <em>am I being optimistic?<\/em> \ud83d\ude42 ) wiki client apps to all instantly then hammer the Lotus Connections server to download the modified page at the same time. <\/p>\n<p>Instead, on receiving the notification MQTT message, the wiki client app marks the page as &#8220;dirty&#8221; in the SQLite database. It will then periodically download an updated copy of all &#8220;dirty&#8221; pages using the Lotus Connections API &#8211; so I&#8217;m sort of rate-limiting myself from within the GUI.<\/p>\n<p><em>The other plus-side to this approach is that I had most of the code to do the server agent piece already to hand from a previous hackday.<\/em><\/p>\n<p><strong>What now?<\/strong><\/p>\n<p>That&#8217;s pretty much what I did on the day. There is more that I want to do on the app, such as:<\/p>\n<ul>\n<li>Add something to the GUI to let you enter your username and password &#8211; at the moment, my username and password is hard-coded in there, so it will submit all edits as me!\n<\/li>\n<li>Add support for more functions from the GUI &#8211; such as adding or deleting pages\n<\/li>\n<li>Do more to handle conflicts &#8211; at the moment, the GUI highlights if a page changes while you have it open if you are online. But there are other conflict scenarios I&#8217;m ignoring &#8211; like what happens if you change a page while offline, but the original page changes before you come back online.\n<\/li>\n<li>Make the wiki page editor more user-friendly &#8211; at the moment, it is a plain text editor for editing HTML. This is mainly because I couldn&#8217;t figure out how to read or write wiki markup using the Lotus Connections API. If I can find a rich SWT widget for reading and editing HTML, that might be good. <\/li>\n<li>Make the push server agent bit optional &#8211; if you have a private wiki, then this approach will only work if you let the user who runs the server polling agent (i.e. me) have read-access to your wiki. So I want to make sure the client will work okay without it. <\/li>\n<\/ul>\n<p>Even so, I think it&#8217;s a decent start \ud83d\ude42 <\/p>\n","protected":false},"excerpt":{"rendered":"<p>Friday was the seventh IBM Hack Day, and I again got the chance to spend a day playing with some random ideas. As Hack Days go, I had a surprisingly productive day! I had four ideas on the day: two mobile hacks (both of which I wrote a chunk of code for), a twitter hack [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[147,421,569,178,422,423,235,114],"class_list":["post-969","post","type-post","status-publish","format-standard","hentry","category-tech","tag-hackday","tag-hackday7","tag-ibm","tag-ibmhackday","tag-lotus","tag-lotusconnections","tag-mqtt","tag-wiki"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/969","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=969"}],"version-history":[{"count":0,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/969\/revisions"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=969"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=969"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=969"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}