java – 缓冲区大小如何影响NIO通道性能？

2023年4月20日 267次阅读

我正在阅读Hadoop IPC实现.

https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

/**
 * When the read or write buffer size is larger than this limit, i/o will be 
 * done in chunks of this size. Most RPC requests and responses would be
 * be smaller.
 */
private static int NIO_BUFFER_LIMIT = 8*1024; //should not be more than 64KB.

/**
 * This is a wrapper around {@link WritableByteChannel#write(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * buffer increases. This also minimizes extra copies in NIO layer
 * as a result of multiple write operations required to write a large 
 * buffer.  
 *
 * @see WritableByteChannel#write(ByteBuffer)
 */
private int channelWrite(WritableByteChannel channel, 
                         ByteBuffer buffer) throws IOException {

  int count =  (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
               channel.write(buffer) : channelIO(null, channel, buffer);
  if (count > 0) {
    rpcMetrics.incrSentBytes(count);
  }
  return count;
}


/**
 * This is a wrapper around {@link ReadableByteChannel#read(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * ByteBuffer increases. There should not be any performance degredation.
 * 
 * @see ReadableByteChannel#read(ByteBuffer)
 */
private int channelRead(ReadableByteChannel channel, 
                        ByteBuffer buffer) throws IOException {

  int count = (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
              channel.read(buffer) : channelIO(channel, null, buffer);
  if (count > 0) {
    rpcMetrics.incrReceivedBytes(count);
  }
  return count;
}

逻辑是,
如果缓冲区很小,它将一次读/写通道.如果缓冲区很大,它会多次执行,每次读/写8kb.

我不明白javadocs以及为什么这样做.
为什么“这是为了避免jdk在缓冲区大小增加时创建许多直接缓冲区.”？
大缓冲区大小是否也会影响读取性能？

我理解缓冲区大小如何影响FileInputStream性能(link).但这里是SocketChannel.所以它是无关的.

最佳答案 好问题.在通道中写入时使用sun.nio.ch.IOUtil,并且在其write(..)函数中有以下行

int var7 = var5 <= var6?var6 - var5:0;
ByteBuffer var8 = Util.getTemporaryDirectBuffer(var7);

这是Util.getTemporaryDirectBuffer

static ByteBuffer getTemporaryDirectBuffer(int var0) {
    Util.BufferCache var1 = (Util.BufferCache)bufferCache.get();
    ByteBuffer var2 = var1.get(var0);
    if(var2 != null) {
        return var2;
    } else {
        if(!var1.isEmpty()) {
            var2 = var1.removeFirst();
            free(var2);
        }

        return ByteBuffer.allocateDirect(var0);
    }
}

在负载较重的情况下,当int var0处于较大范围内时,它会创建大量新缓冲区并释放旧的缓冲区(..).因为bufferCache的长度有限(等于系统配置中定义的IOUtil.IOV_MAX.在现代Linux系统上,限制为1024)并且不会存储每个长度的缓冲区.
我认为这意味着这是为了避免jdk在缓冲区大小增加时创建许多直接缓冲区.

点赞