Before I start, I should point out that I’m not a Lucene expert. This post isn’t a definitive “you should do things this way” commandment from a Lucene mage. Think of it more as “I had this problem, and this seemed to work for me. I’m sharing it in case it helps you, too”.
I’m using Lucene to implement searches. Recently, as my Lucene index has grown (a lot), I was getting a lot of these errors when I tried to do a search:
org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024 at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:163) at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:154) at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:63) at org.apache.lucene.search.WildcardQuery.rewrite(WildcardQuery.java:54) at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383) at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:383) at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:162) at org.apache.lucene.search.Query.weight(Query.java:94) at org.apache.lucene.search.Searcher.createWeight(Searcher.java:185) at org.apache.lucene.search.Searcher.search(Searcher.java:86)
I’m guessing that
TooManyClauses is a common problem for people getting going with Lucene.
But I couldn’t find a straightforward “you need to follow these steps to fix it” post anywhere, so I’ll add my experience here.