Archive for March, 2012

Avoiding my Lucene TooManyClauses exceptions

Tuesday, March 20th, 2012

Before I start, I should point out that I’m not a Lucene expert. This post isn’t a definitive “you should do things this way” commandment from a Lucene mage. Think of it more as “I had this problem, and this seemed to work for me. I’m sharing it in case it helps you, too”.

I’m using Lucene to implement searches. Recently, as my Lucene index has grown (a lot), I was getting a lot of these errors when I tried to do a search:

org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024
    at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:163)
    at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:154)
    at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:63)
    at org.apache.lucene.search.WildcardQuery.rewrite(WildcardQuery.java:54)
    at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383)
    at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383)
    at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:162)
    at org.apache.lucene.search.Query.weight(Query.java:94)
    at org.apache.lucene.search.Searcher.createWeight(Searcher.java:185)
    at org.apache.lucene.search.Searcher.search(Searcher.java:86)

I’m guessing that TooManyClauses is a common problem for people getting going with Lucene.

It’s mentioned in the FAQ, and there are a few StackOverflow threads around about it.

But I couldn’t find a straightforward “you need to follow these steps to fix it” post anywhere, so I’ll add my experience here.

(more…)

Face Movies

Sunday, March 18th, 2012

I’ve been using Picasa to manage my photos of the kids for years.

I noticed in the changelog for the latest version that it now can create chronological “Face Movies”. These are videos of all the photos of a certain person, arranged in order that they were taken, and zoomed, rotated and cropped to line up the face as best as possible between the pictures.

After a little prodding from Tim, I gave it a try.

Here is (nearly) every photo I’ve ever taken of Grace, run through Picasa’s Face Movie.

It takes two-and-a-half minutes, but I think it’s pretty cool. I love how you can watch her face change, and her hair grow over time.

There are settings you can tweak. For example, you can change the frame rate, or get it to filter out photos taken within a specified amount of time of each other. It means you can make shorter versions – I made a video of just some of Grace’s pics that lasts under a minute.

(more…)

A nice memory

Friday, March 16th, 2012

I was reminded of this by my Timehop email this morning. (Incidentally, I’m loving Timehop – thanks to Roo for sharing it).

A year ago today, I had a lousy afternoon. I even tweeted about it.

I didn’t share what happened after, although I possibly should have.

(more…)

Migrating from ActiveMQ to WebSphere MQ

Tuesday, March 13th, 2012

Overview

A side-project I’ve been playing with in the evening: Writing a JMX layer to allow apps written for ActiveMQ to migrate to WebSphere MQ with minimal modifications

Background

This came out of working on something that uses a JMS messaging provider. It uses it internally to allow components to communicate with each other, even when spread across multiple machines.

It uses Apache ActiveMQ – an open-sourced implementation of JMS. I wanted to try and get it working using WebSphere MQ – IBM’s implementation of JMS that I used to work on until five years ago.

As a messaging standard, the fact that both ActiveMQ and WebSphere MQ (WMQ) are JMS providers means that the way it puts and gets messages should just work.

But the JMS standard doesn’t cover administration (how queue managers are created and configured, how they’re started, how queues and topics are created, etc.) or monitoring (getting statistics about how many messages have been put or got, how many messages are on a queue, etc.)

All of this was done in an ActiveMQ-specific ways. This was what needed to be ported if I was going to get this to work with WebSphere MQ.

The project I’m porting is actually a bit of a black box. Rather than make a significant rewrite to get it to go from being ActiveMQ-specific to WMQ-specific, I wanted to see what I could add so that as much of the existing code could just work transparently.

I wanted to write a layer to sit between the ActiveMQ-app and WebSphere MQ, so that the app needn’t realise it’s not talking to the ActiveMQ broker it was written for.

(more…)