Java elasticsearch direct buffer exception

1071 查看

在使用elasticsearch 2.2.0版本,使用scroll进行遍历获取数据时,在进行频繁的取数据时,发现内存占用在不断升高,直至抛出下面的异常信息。


[2016-03-23 11:22:30 WARN ] [elasticsearch[Star Thief][transport_client_worker][T#6]{New I/O worker #4416}] [org.elasticsearch.netty.channel.socket.nio.AbstractNioSelector:83] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Direct buffer memory
        at java.nio.Bits.reserveMemory(Bits.java:658)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
        at org.jboss.netty.channel.socket.nio.SocketReceiveBufferAllocator.newBuffer(SocketReceiveBufferAllocator.java:64)
        at org.jboss.netty.channel.socket.nio.SocketReceiveBufferAllocator.get(SocketReceiveBufferAllocator.java:41)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:62)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)

从异常信息来看是direct buffer溢出了,我就尝试加大direct buffer的大小,发现这样虽然可以延后该异常的抛出,但到一定程度还是会抛出该异常。后来查了好久的原因,问题应该还是和这个很像

http://javatar.iteye.com/blog/1138527

如有人遇到,可以参考该文章,尽量找到规避的办法