{"id":456,"date":"2009-02-07T22:09:00","date_gmt":"2009-02-07T22:09:00","guid":{"rendered":"http:\/\/dalelane.co.uk\/blog\/?p=456"},"modified":"2009-02-18T12:50:59","modified_gmt":"2009-02-18T12:50:59","slug":"getting-data-from-the-currentcost-cc128","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=456","title":{"rendered":"Getting data from the CurrentCost CC128"},"content":{"rendered":"

I really need to stop blogging about CurrentCost<\/a> stuff, this is starting to look like an obsession \ud83d\ude42<\/p>\n

But before I do that, I thought I’d share my experiences in writing a new CurrentCost parser. <\/p>\n

What is this all about?<\/strong>
\nOn the off-chance that anyone following me hasn’t already heard about CurrentCost<\/a> a million times already, CurrentCost meters are domestic electricity monitors that tell you how much electricity your household is using. And they have a connection underneath that lets you grab the data from them.<\/p>\n

As well as getting your current electricity usage, you can also get historical usage totals out of the serial port, which I’ve used in a variety of ways, such as in my Python CurrentCost software<\/a>.<\/p>\n

CurrentCost have now brought out a new model of their monitor – the CC128<\/a>. The data format is different to the current models, so I wanted to try and write a new parser that could handle both the existing model and the new CC128 data.<\/p>\n

Connecting<\/strong>
\nFirst things first, connecting to the meter. <\/p>\n

I used to do this in Python to get data from the meter:<\/p>\n

ser = serial.Serial(port='COM20', timeout=5)\r\nline = ser.readline()<\/code><\/pre>\n
With the CC128, I need to use:<\/p>\n
ser = serial.Serial(port='COM20', \r\n                    baudrate=57600,\r\n                    bytesize=serial.EIGHTBITS,\r\n                    parity=serial.PARITY_NONE,\r\n                    stopbits=serial.STOPBITS_ONE,\r\n                    timeout=3)\r\nline = ser.readline()<\/code><\/pre>\n
Getting history data – how it used to work<\/strong>
\nMy old CurrentCost meter outputs a line of XML<\/a> every six seconds or so. Most of the time it includes history data, but not always. But it was good enough for my app to keep reading updates from the meter until it gets one with history data – at worst, the user waits for about 15 seconds until the app has the data it needs.<\/p>\n
Getting history data – how it works now<\/strong>
\nThe CC128 doesn’t include history data in every (or nearly every) update. The draft doc on the CurrentCost website<\/a> says: <\/p>\n
“History messages are output starting at 1 minute past every odd hour, ie 17:01, 19:01 etc”<\/p><\/blockquote>\n
 so you can potentially have to wait quite a while to get history update (or be connected at the right time!). <\/p>\n
Alternatively, you can force the meter to output the history data by <\/p>\n
“pressing and holding the Down and OK buttons together until the LED flashes”<\/p><\/blockquote>\n
 The LCD display changes to just displaying a single number, flashing 1 to 9 repeatedly until it has finished outputting the history. <\/p>\n
This is a bit of a pain for me with my app<\/a>, as I get the data from the meter remotely so needing to be near enough to the meter to press the buttons is less than ideal. <\/p>\n
How it works now – what it looks like<\/strong>
\nI’ve already blogged about what history data you can get from the CC128<\/a>, and the draft doc for the CC128 XML<\/a> gives a specification of the data. <\/p>\n
What I didn’t realise from reading it was that you don’t get a single line of XML with the whole history data in it. You get a series of XML updates, each of which contains a sliding, overlapping window into the history data.<\/p>\n
First you get two-hourly history, then daily history, then monthly history data. Each set of history data is returned in chronological order – from the oldest data to the most recent.<\/p>\n
For example, my meter has been running for 14 days since I last reset it. So the oldest two-hourly history data is h324<\/code> – the electricity used from 326 hours to 324 hours ago.<\/p>\n
So the first line of two-history data contains the two-hourly history data:
\nh324, h322, h320, h318, ... h230, h228, h226<\/code>
\nThe second line of history data received contains the two-hourly history data:
\nh322, h320, h318, h316, ... h228, h226, h224<\/code>
\nThe third line of history data received contains two-hourly history data:
\nh320, h318, h316, h314, ... h226, h224, h222<\/code><\/p>\n
And so on, until the end of the two-hourly history data:
\nh102, h100, h098, h096, ... h008, h006, h004<\/code><\/p>\n
And then the daily history data:
\nd014, d013, d012, d011, d010, ... d003, d002, d001<\/code><\/p>\n
I would then receive monthly data, except my meter hasn’t been running for a month since last reset, so there isn’t any monthly history data yet. <\/p>\n
Notice that you only receive XML for the data you have – unlike the old meter which receives the full XML in every history update, with zeroes for data not yet available.<\/p>\n
If you want to see an example, here is my history data<\/a> for the last couple of weeks.<\/p>\n
How do you use the CC128 history data<\/strong>
\nThe history data is big. Very big. <\/p>\n
This explains why we don’t get history data every 6 seconds!<\/p>\n
Getting just fourteen days of history data (i.e. 112 updates of two-hourly history, 1 update of daily data, 0 updates of monthly data) takes about a minute<\/strike> ten minutes to be output, and produces about 1MB of XML.<\/p>\n
For my interactive app which displays a progress bar while it’s waiting to receive a complete copy of the history, this means that once the meter history is full, downloading history data could take a few minutes<\/strike> quarter-of-an-hour or longer. <\/p>\n
Still, there are benefits to this approach. The overlap in the history data means that we can handle losing a few updates without losing any data. And in some ways it makes more sense to output a new copy of the history after it has changed – outputting months of history data every 6 seconds is a little redundant.<\/p>\n
Parsing the data<\/strong>
\nI’ve updated my Python parser so that it now can handle receiving XML from either the old or the new CurrentCost meters. It can be downloaded from http:\/\/code.google.com\/p\/currentcost\/<\/a>.<\/p>\n
.
\nUpdate (10th Feb):<\/strong> I massively underestimated the time it takes to download the history. I was going by the timestamps contained in the XML I was downloading – but it turns out that the CurrentCost meter’s clock stops while it is outputting history data<\/a>. What I said took only a minute took more like ten minutes. It also resulted in my CurrentCost clock being ten minutes slow(er) once this was complete – every time I downloaded the history.<\/p>\n
Update (18th Feb):<\/strong> I got a new CC128 last week<\/a> and now it’s been running for a week, I took a look at the XML<\/a>. The new CC128 has a later software version (0.11) than the one I was using when I wrote this post (0.09). The XML produced by the newer version is a little different. There are a couple of new items included in there, for example, there is now a <dsw> (days-since-wipe<\/a>) item in addition to the previous <dsb>. More importantly though, it looks like v0.11<\/a> doesn’t overlap the history data in the way that 0.09<\/a> did. This is great to see. <\/p>\n","protected":false},"excerpt":{"rendered":"
I really need to stop blogging about CurrentCost stuff, this is starting to look like an obsession \ud83d\ude42 But before I do that, I thought I’d share my experiences in writing a new CurrentCost parser. What is this all about? On the off-chance that anyone following me hasn’t already heard about CurrentCost a million times […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[194,212],"class_list":["post-456","post","type-post","status-publish","format-standard","hentry","category-code","tag-currentcost","tag-python"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/456","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=456"}],"version-history":[{"count":0,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/456\/revisions"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=456"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=456"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=456"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}