CurrentCost app … take 2!

a Python CurrentCost appI’ve been talking about it for a while, but I finally got around to spending some time working on a CurrentCost app.

The original code was written in C# using .NET 3.5, and I used WPF (Windows Presentation Foundation) to draw the graphs. For a number of reasons this proved unpopular and I got a ton of emails saying how this was no good for them. So I decided to start again.

The new app is written in Python – using wxPython and matplotlib to create the graphs.

a Python CurrentCost appUsing py2exe, I’ve been able to compile the whole thing (combining my script with a Python interpreter and a copy of all of the third-party libraries I’ve used) into a Windows executable that will (hopefully!) run on any Windows computer, without needing installing or requiring any pre-requisites.

I can also make the Python script itself available, making it something that Linux users could run as well. This means I get the low-overhead Windows experience I wanted, together with the ability to make it cross-platform. Neat!

I’ve been able to produce some more interactive graphs – the graphs in the new app can be zoomed in and out, panned, moved around, printed, and exported to images. As the amount of data in the app builds up, I can see this becoming very useful.

For example, if you’ve got daily CurrentCost readings for a several months, you can either zoom out to see all your data, or zoom in to focus on a particular month of interest. Then click-and-drag on the graph to pan around.

I’ve also tried to improve the way I label the axes, as this was something a number of people found unclear with my first attempt. For example, when looking at hourly data, vertical grid lines are used to mark out days, and the start of each day is marked with the date on the x-axis. (It makes more sense if you see it… honest!)

It’s the product a couple of days’ work, so it’s still very rough around the edges. There’s virtually no error-handling, the graphs are still a little clunky, and I’m convinced that I’m not reading the hourly data quite right (the data is received from the CurrentCost meter in a confusingly-labelled way giving relative times, and I think I’m translating it into actual times incorrectly).

But it’s a start, and I’m already quite pleased with it. Feedback is naturally very welcome… but bear in mind that this should only be considered a very, very early beta!

Download it here

Tags: , , ,

45 Responses to “CurrentCost app … take 2!”

  1. dale says:

    Update: I think I’ve got the hourly data issue fixed!

  2. bee says:

    I’d forgotten how decent python is for cross-platform – neat project. I wrote a similar homebrew app more aimed at background logging which has been graphing currentcost data the last month or so for our household – it certainly highlights good and bad energy habits:
    http://www.flickr.com/photos/barnybug/2736617276/

  3. dale says:

    @bee – thanks for the comment. I’m fairly new to Python, but I’ve been really impressed with how powerful it is, and how nicely it crosses the Windows/Linux divide

    Your app looks interesting – have you considered sharing it on the CurrentCost wiki? (http://currentcost.wetpaint.com/) (something I keep meaning to do with my app!)

  4. Dave says:

    Hi Dale

    I have sent Current Cost a cheque for their data lead which, at present, is a serial lead. However, I had an email from them today saying:

    ………….but we have also now developed a USB cable, although it won?t be available for around another month yet.

    Do you have plans for a USB version?

    Regards
    Dave

  5. dale says:

    @Dave – It depends on how they’re doing the USB cable. If it’s just a straight serial-to-USB cable converter, then this is what I’m using myself already. I’ve written about this before (see ‘Connecting CurrentCost to Windows’), but in short, I wouldn’t expect that it will make much difference.

  6. Dave says:

    Hi Dale

    From what I understand, having just spoken to them, it will be a USB straight to the monitor. I should be getting my serial lead soon.

    Update: They have started to soak testing their software to make sure it is stable and expect to have it ready in a couple of weeks.

  7. Dave says:

    Hi Dale

    I have now got my data lead and I can get Hyper Terminal to see the xtml text but I am affraid your program does nothing. I connect using Com1 but the graphs stay blank. When I click anywhere on the program it say “Not Responding”.

    Is it becuase my monitor is using 2400 – 8 – N- 1 on Com 1?

  8. dale says:

    @Dave – using a baud rate of 2400 suggests that you have one of the first generation CurrentCost meters which don’t output history data

    If you compare the XML output you are getting with the sort of output that is produced by the newer meters, you will likely see that you are missing the “hist” element, and all of the data in it.

    This is the data that this app displays. Without history data, there is nothing for my app to display – so the graphs stay blank as you say.

    The old version of the app naturally had the same problem.

    Sorry about that!

    The older meters do still output the current reading, so your alternative would be to store this. If you store every reading, every 6 seconds, you can build up a history data source of your own. Several of us have done this – I’ve mentioned doing this with MySQL myself, using Perl to query the MySQL store. You can do some quite powerful queries this way.

    The downside is that to build a useful history store, you have to have a computer connected to the CurrentCost meter all the time. Which can potentially negate any benefits of monitoring your electricity usage!

    You can use a very low-powered server (I’ve mentioned using a Slug for this purpose).

    As the newer CurrentCost meters store limited history data on the device themselves, I wrote this Python app to use this – for people who didn’t want the overhead of having a server running all the time!

    Hope this long ramble helped!

  9. Dave says:

    Hi

    It may be because your program is asking for the historical data which the older 2400 units do not have, as these are only dynamic.

    However, Current Cost are selling the newer 9600 units on eBay (just the displays, no transmitters or clamps) for ?15. I have bought one straight away.

  10. dale says:

    @Dave – Ah, I see you worked it out for yourself while I was writing my reply! ๐Ÿ™‚

  11. dale says:

    Update:
    Made a number of minor interface changes.
    Added tab to start giving text-based descriptions of the CurrentCost data
    Added the ability to export data to CSV for external analysis

    New version can be downloaded from the same location

  12. Dave says:

    Hi Dale

    Forgive me for asking but I have now received my new monitor from CC. Howeve, it is not tuned in and the manual says “Locate the tuning push switch ….through a small hole.” Neither of my units have such a hole. Do you now how to start the tuning scan?

  13. Chris says:

    @Dave — the tuning push switch is actually on the transmitter (black box). You need to push it using a pencil or similar, and hold it down.

    Check the video at http://www.currentcost.com/support-1.shtml

    Chris

  14. Dave says:

    Hi

    Thanks for that. I did find it in the end, when all else fails …… read the manual.
    It didn’t seem to make sense to reset the transmitter, as it was the receivers that needed to tune to the transmitter.
    Anyway, I did as the manual asked and now I have two displays working fine. They both read the same except the 2400 unit shows the Temp as being 2 degs higher. The 9600’s temp is correct as per other digital thermometers close by.

    Now why didn’t I see the video?

  15. Dave says:

    Hi Dale

    I am getting this error every time I run the program.

    “D:\WUTemp\Current Cost\currentcost.exe:20: DeprecationWarning: The wxPython compatibility package is no longer automatically generated or actively maintained. Please switch to the wx package as soon as possible.”

    Also I get a popup box with: stdout/stderr when I connect.

    Traceback (most recent call last):
    File “currentcost.py”, line 478, in onConnect
    File “currentcost.py”, line 989, in drawMyGraphs
    File “matplotlib\backends\backend_wxagg.pyc”, line 60, in draw
    File “matplotlib\backends\backend_agg.pyc”, line 261, in draw
    File “matplotlib\figure.pyc”, line 759, in draw
    File “matplotlib\axes.pyc”, line 1523, in draw
    File “matplotlib\axis.pyc”, line 712, in draw
    File “matplotlib\axis.pyc”, line 654, in iter_ticks
    File “matplotlib\dates.pyc”, line 710, in __call__
    File “matplotlib\dates.pyc”, line 436, in viewlim_to_dt
    File “matplotlib\dates.pyc”, line 233, in num2date
    File “matplotlib\dates.pyc”, line 156, in _from_ordinalf
    ValueError: ordinal must be >= 1

  16. dale says:

    Update – v0.8
    Added two new graphs showing a typical day and a typical week
    Added some more descriptions to the ‘trends’ tab

    New version can be downloaded from the same location

  17. dale says:

    [trimmed a few comments from here as Dave and I worked out bugs in the code]

    Current version is 0.8.6 and contains a ton of minor bug fixes

  18. […] My CurrentCost app is a desktop application, written in Python so that it can be used on Windows or Linux (and presumably Mac OS, although I’ve not tried that). It reads data from a CurrrentCost meter, and displays it to the user. […]

  19. Patrick Keane says:

    Hi Dale,

    Being a Linux only user, I would much appreciate a copy of your python script to collect and parse the history data from the currentcost monitor.

    I can use RRDtool to produce the graphs on my Linux box.

    I downloaded your Windows app, and tried it under wine, but it was a leap too far!

    thanks in advance

    regds

    Patrick

  20. dale says:

    Patrick

    I’ve used the app without problems on my Ubuntu desktop. However, without the benefit of py2exe to package it up with all of the third party modules I’ve used (e.g. numpy, pyserial, simplejson, wx, pysqlite, matplotlib, etc.) I’ve avoided sharing the Python scripts to avoid needing to support people installing these libraries themselves.

    It is my plan to make this properly available for Linux soon – either by bundling it up with prereqs as I have with py2exe, or by properly documenting the pre-req requirements.

    If you’d rather not wait and/or you’re happy working out the prereqs for yourself, I’ve uploaded the Python scripts here – you can run the ‘currentcost’ one to start it.

    Kind regards

    D

  21. John Cuthbert says:

    On running currentcost app using com5 starts download data stop after 2 bars.
    using usb cable supplied from current cost and using prolific driver as sugested by currentcost.
    any ideas

  22. dale says:

    @John – I’ll contact you offline to talk about this

  23. […] rely on the history maintained by the meter itself and connect the meter to your PC infrequently (e.g. with a desktop app) […]

  24. Andrew Carr says:

    @dale (or in fact anyone).

    Looking at using this on a Mac – at least for the time being anyhow. Got MacPython installed and it builds currentcost.py, but fails because it can’t find the wx module.

    Anyone successfully managed to get this working on Mac OS x 10.4+

  25. Andrew Carr says:

    @dales – yes had a good look at the pre-requisites… Couldn’t get it working despite spending a few hours on it…. I’ll have another crack at it tomorrow

    Thanks !

  26. Addy says:

    Hi,
    after many late night hours of hacking on my x86 iMac MAC OS X 10.5 (Leopard) I finally managed to get the currentcost.py GUI up and running! I don’t yet have the data cable, so can’t say whether it fully works yet.

    But I tried so many different Python and framework installs, I’m still not sure which one actually did the trick. In the end I stuck with the supplied Mac OS python (2.5.4) and used the Scipy Superpack script from;

    http://macinscience.org/?page_id=6

    I guess this also installed wxpython ?
    Certainly I had no luck with easy_install of wxpython which fails even under Ubuntu Linux. I think the wxpython tarball doesn’t have a top level setup.py script ?

    The final issue I had was with the pysqlite module which used a missing sqlite3 dynamic library symbol. In the end I had to replace the system sqlite3 dynamic library in /usr/lib with one I rebuilt under MacPorts. Yuk!

    So I’ve now ordered the cable and fingers crossed I’ll be able to extract some historic data (RJ45 based unit purchased June 2008).

    Cheers
    Addy

  27. dale says:

    @Addy – Sorry to hear that you had such problems with it! I’ve not tried it on a Mac before, but I don’t remember having these problems when I first set it up on my Ubuntu box.

  28. Addy says:

    Hi Dale, I’m a Linux programmer by day, and so Mac OS X is all new to me too! Pity I can’t just apt-get all the Python modules like on Ubuntu.
    Perhaps I should just have used VMWare Fusion and run Ubuntu as a virtual machine ?

    Anyway it seems to be running now, I reverted the system libsqlite library and exported DYLD_LIBRARY_PATH instead, because my console log was filling up with dyld errors.
    The Mac OS X serial device is /dev/tty.usbserial, but unfortunately it turns out I have the old 2400 baud one and so can only monitor live data (I hacked in baudrate=2400 to the Python serial setup). I just ordered a newer one from the CurrentCost Ebay site. Saving energy is an expensive business ๐Ÿ˜‰

    Thanks for a great app and I hope my experiences will help other Mac people get going.

  29. dale says:

    @Addy – If there is anything else that you think I should add to http://code.google.com/p/currentcostgui/ to improve the documentation for Mac OS peeps, please let me know.

  30. Andy Piper says:

    Addy – I’m hitting the same problem with pysqlite – I have the module installed (with MacPython) but it’s falling over finding _sqlite – would be keen to know what fixed that for you. I also found I needed wxPython, matplotlib, numpy and pyserial (the latter is not listed in Dale’s prereqs)…

  31. Addy says:

    @ Andy Piper
    Hi, I eventually solved the missing symbols in the libsqlite library by rebuilding the MacPorts sqlite3 package. I found some instructions on how to do this by pasting the missing symbol error into Google.
    Then once the MacPorts library was rebuilt I exported DYLD_LIBRARY_PATH to pick up the new MacPorts library under /opt. But I am still using the Mac OS X python binary.

    Sorry, I’m not at home currently so can’t be more precise. Perhaps you can post the error message you are seeing to see if it’s the same problem ?

  32. Addy says:

    @Andy
    I just found the webpage with the ‘fix’ for rebuilding the MacPorts sqlite3 package;

    http://oss.itsystementwicklung.de/trac/pysqlite/ticket/238

    port install sqlite3 +loadable_extensions

    I too was seeing the “_sqlite3_enable_load_extension” symbol not being found error.

  33. dale says:

    @Addy – Thanks for taking the time to share this – I’m gonna try and put it together as instructions at some point, which I will share here.

  34. Andy Piper says:

    Addy, thanks. I installed the sqlite3 package using MacPorts as described but I’m still getting the following error from currentcost.py:

    Traceback (most recent call last):
    File “currentcost.py”, line 50, in
    from currentcostdb import CurrentCostDB
    File “/Users/andyp/Downloads/cc-py/currentcostdb.py”, line 22, in
    from pysqlite2 import dbapi2 as sqlite
    File “build/bdist.macosx-10.4-fat/egg/pysqlite2/dbapi2.py”, line 27, in
    ImportError: No module named _sqlite

    I’ll have a fiddle with DYLD_LIBRARY_PATH to see if that helps. Thanks.

  35. Addy says:

    @ Andy
    That looks like a different problem. I’ve never used Python before so I probably can’t help too much.

    Which version of MacPython did you install ?
    How did you install the prereq packages ?

    Perhaps this article is related ?

    http://objectmix.com/python/304701-import-pysqlite2-import-sqlite3.html

  36. Ian Livesey says:

    Hi Dale,

    Great application works like a charm :o) Thanks for the hard work.
    As I’m a .Net developer, is there any chance of getting my hands on your original C# code/project. I’ve never programmed against a real-time device and have no idea where to start and your code would be a great help?

    Thanks again

    Ian

  37. dale says:

    @Ian – Thanks very much for the comment. The CurrentCost-specific C# code can be downloaded from http://code.google.com/p/currentcost/downloads/list

  38. Psychonix says:

    Hi Dale,

    I’ve been following your work with the Current Cost with great interest for about 8 months now, you’ve done some fantastic stuff here – some things way out of my scope or time to learn. So I thought, what do I want it to do for me? And was there a simpler way of getting the data of it and turning it into something useful ?

    Basically, I wanted to be able to scoop the data into a simple text format that I could then manipulate as I wanted.
    I wanted to have the data displayed in *almost* realtime on a webpage in plain html.
    I wanted to be able to see the data in an easy-to-read format such as a chart / graph.

    All of this without having to install too much software, or do too much coding or to learn a fancy new language that I don’t have the time to use or debug ๐Ÿ™

    I have the following tools to work with :

    Windows Server 2003
    Putty
    Cygwin
    IIS
    FTP
    Google charts (THANK YOU FOR THIS INSIGHT!!)

    It may not be the most elegant of solutions, but I have managed to get something quite simple and nice looking working.

    It works thus :

    Putty opens the com port the CC is attached to and logs all session data to a file
    With Cygwin bash shell running, some awk, sed scripts run to format the output into just the plain data
    The plain data is taken and google chart urls are created from it
    The google urls are dropped into a single html file and ftp’d to my web server

    All the above runs as a cron job in Cygwin.

    You could substitute any webserver, any MS O/S and do anything you want with the raw dat off the com port. You don’t need to use putty, opening the com port and letting the data flood in from cygwin would do equally as well – I ran in in Linux just doing cat /dev/ttyS0 just as well.

    I hope this helps those such as myself, that don’t have the time or knowledge to learn new languages or tools. No compiling need !

    Ta

    P.s. If anyone wants my scrappy unix scripts they can have them ๐Ÿ™‚

  39. Psychonix says:

    Lol, sorry, forgot the important bit, my results. The 2 links below need some refining, but enough to show you what I’ve achieved.

    http://www.psychonix.net/house.html

    http://www.psychonix.net/energy_charts.html

    First link also has data captured from my weather station – ignore the chart at the bottom regarding energy, it’s test data.

  40. Luis says:

    Hi Dale and everyone,

    Sorry for waking up an old post. I know that by now the topic may be off.

    I am trying to achieve something like you kindly provide with your Windows application, but in Linux.

    I’m building a low power PC (a Thin Client BTW) so I can keep it always-on with this task.
    So, ultra low power means low processing power ๐Ÿ˜‰ You know what I mean.
    I would like to maintain a Local database with all my energy and temperature consumption records. The graphical presentation would be made querying this database.

    I saw some comments above that you made available the Python scripts for the Linux’ers. Can I download them? Do you have any update regarding this approach?

    Thank’s and regards,

    Luรƒยญs

  41. dale says:

    Luis – Have you seen http://code.google.com/p/currentcostgui/ ?
    (Please note that I stopped developing/supporting that code some time ago, but it’s still available)

  42. Liviu says:

    Hi,
    I used the CCGui to record live 6s data and then convert to CSV for external analysis. However, after few hours, the live data fails to be retrieved at 6s, but rather at few minutes, without obvious pattern. I’m trying to look into the file were the CCGui stores the data to see if it’s a format I can read (like SQL database) and hopefully recover the 6s samples. My question is, where is CCGui storing the data?
    Thanks,
    Liviu

  43. dale says:

    Hi Liviu- Sorry, as I recall, (and it’s been a few years since I wrote this!) the app doesn’t store the live data. Kind regards, D