Before I start, I should point out that I’m not a Lucene expert. This post isn’t a definitive “you should do things this way” commandment from a Lucene mage. Think of it more as “I had this problem, and this seemed to work for me. I’m sharing it in case it helps you, too”.
I’m using Lucene to implement searches. Recently, as my Lucene index has grown (a lot), I was getting a lot of these errors when I tried to do a search:
org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:163)
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:154)
at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:63)
at org.apache.lucene.search.WildcardQuery.rewrite(WildcardQuery.java:54)
at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383)
at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383)
at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:162)
at org.apache.lucene.search.Query.weight(Query.java:94)
at org.apache.lucene.search.Searcher.createWeight(Searcher.java:185)
at org.apache.lucene.search.Searcher.search(Searcher.java:86)
I’m guessing that TooManyClauses is a common problem for people getting going with Lucene.
It’s mentioned in the FAQ, and there are a few StackOverflow threads around about it.
But I couldn’t find a straightforward “you need to follow these steps to fix it” post anywhere, so I’ll add my experience here.