neo4j-import在准备节点索引阶段失败

2019年8月6日 308次阅读

我正在尝试使用neo4-import工具从csv文件导入新的neo4j数据库上的130M节点和200M关系,并且它会卡在流程上.

过程的输出：

Nodes [>:3.35 MB/s---|PROPERTIES(2)======|*NODE:1007.08 MB------|LABEL SCAN-----|v:7.46 MB/s(2)======]131M
Done in 1h 35m 47s 332ms
Prepare node index
[*SORT:1.47 GB---------------------------------------------------------------------------------] 87M

messages.log的内容

2015-06-15 10:38:56.423+0000 INFO  [org.neo4j]: Import starting
2015-06-15 10:38:56.537+0000 INFO  [org.neo4j]: Creating new db @ /var/user/neo/graph22.db/neostore
2015-06-15 10:38:57.264+0000 INFO  [org.neo4j]: Missing counts store, rebuilding it.
2015-06-15 10:38:57.536+0000 INFO  [org.neo4j]: No lucene scan store index found, this might just be first use. Preparing to rebuild.
2015-06-15 10:38:57.581+0000 INFO  [org.neo4j]: No lucene scan store index found, this might just be first use. Preparing to rebuild.
2015-06-15 10:38:57.793+0000 INFO  [org.neo4j]: Rebuilding lucene scan store, this may take a while
2015-06-15 10:38:57.796+0000 INFO  [org.neo4j]: Lucene scan store rebuilt (roughly -1 nodes)
2015-06-15 18:29:33.509+0000 INFO  [org.neo4j]: About to rotate counts store at transaction 1 to [/var/user/neo/graph22.db/neostore.counts.db.b], from [/var/user/neo/graph22.db/neostore.counts.db.a].
2015-06-15 18:29:33.516+0000 INFO  [org.neo4j]: Successfully rotated counts store at transaction 1 to [/var/user/neo/graph22.db/neostore.counts.db.b], from [/var/user/neo/graph22.db/neostore.counts.db.a].
2015-06-15 18:29:33.771+0000 ERROR [org.neo4j]: Error during import null
java.lang.StackOverflowError
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.partition(ParallelSort.java:193)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:243)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.recursiveQsort(ParallelSort.java:246)

是什么导致这个过程失败？有什么办法可以解决这个问题吗？

我正在使用neo4j-community-2.2.2

最佳答案 我有同样的问题,但版本2.2.3.我通过编辑文件bin / neo4j-import并添加更多堆栈空间(以及更多堆,因为我拥有它)来解决它：

EXTRA_JVM_ARGUMENTS="-Dfile.encoding=UTF-8 -Xmx8g -Xss128m"

但现在我有一个不同的问题 – 递归是1023级深度(我只排序约5亿个节点),我不确定排序是否会完成:)

点赞