java – Lucene Out of Memory

2019年7月28日 257次阅读

我正在使用Lucene v4.10.4.我有相当大的索引,它可能超过几GB.所以我在初始化IndexSearcher时得到OutOfMemoryError：

try (Directory dir = FSDirectory.open(new File(indexPath))) { 

    //Out of Memory here!
    IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(indexDir));

如何告诉Lucene的DirectoryReader不能一次加载到256 MB以上的内存？

日志

Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.lucene.util.fst.BytesStore.<init>(BytesStore.java:68)
    at org.apache.lucene.util.fst.FST.<init>(FST.java:386)
    at org.apache.lucene.util.fst.FST.<init>(FST.java:321)
    at org.apache.lucene.codecs.blocktree.FieldReader.<init>(FieldReader.java:85)
    at org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:192)
    at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:441)
    at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:197)
    at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:254)
    at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:120)
    at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:108)
    at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
    at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:923)
    at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:53)
    at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:67)

最佳答案 首先,您应该检查JVM的当前堆大小.

java -XX:+PrintFlagsFinal -version | grep MaxHeapSize

如果此数字对于您的用例不合理,则应在使用java命令的-Xmx选项运行程序时增加它.分配8GB堆内存的示例命令如下所示：

java -Xmx8g -jar your_jar_file

希望这可以帮助.

点赞