Package org.apache.hadoop.hbase.io

Class ByteBuffAllocator

java.lang.Object

org.apache.hadoop.hbase.io.ByteBuffAllocator

Direct Known Subclasses:

DeallocateRewriteByteBuffAllocator

@Private
public class ByteBuffAllocator
extends Object

ByteBuffAllocator is used for allocating/freeing the ByteBuffers from/to NIO ByteBuffer pool, and it provide high-level interfaces for upstream. when allocating desired memory size, it will return ByteBuff, if we are sure that those ByteBuffers have reached the end of life cycle, we must do the ByteBuff.release() to return back the buffers to the pool, otherwise ByteBuffers leak will happen, and the NIO ByteBuffer pool may be exhausted. there's possible that the desired memory size is large than ByteBufferPool has, we'll downgrade to allocate ByteBuffers from heap which meaning the GC pressure may increase again. Of course, an better way is increasing the ByteBufferPool size if we detected this case.

On the other hand, for better memory utilization, we have set an lower bound named minSizeForReservoirUse in this allocator, and if the desired size is less than minSizeForReservoirUse, the allocator will just allocate the ByteBuffer from heap and let the JVM free its memory, because it's too wasting to allocate a single fixed-size ByteBuffer for some small objects.

We recommend to use this class to allocate/free ByteBuff in the RPC layer or the entire read/write path, because it hide the details of memory management and its APIs are more friendly to the upper layer.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	ByteBuffAllocator.Recycler

Field Summary

Fields

Modifier and Type	Field	Description
static final String	ALLOCATOR_POOL_ENABLED_KEY
static final String	BUFFER_SIZE_KEY
private final Queue<ByteBuffer>	buffers
protected final int	bufSize
static final String	BYTEBUFF_ALLOCATOR_CLASS	Set an alternate bytebuffallocator by setting this config, e.g.
static final int	DEFAULT_BUFFER_SIZE	There're some reasons why better to choose 65KB(rather than 64KB) as the default buffer size:
static final String	DEPRECATED_ALLOCATOR_POOL_ENABLED_KEY	Deprecated. since 2.3.0 and will be removed in 4.0.0.
(package private) static final String	DEPRECATED_BUFFER_SIZE_KEY	Deprecated. since 2.3.0 and will be removed in 4.0.0.
(package private) static final String	DEPRECATED_MAX_BUFFER_COUNT_KEY	Deprecated. since 2.3.0 and will be removed in 4.0.0.
static final ByteBuffAllocator	HEAP
private final LongAdder	heapAllocationBytes
private long	lastHeapAllocationBytes
private long	lastPoolAllocationBytes
private static final org.slf4j.Logger	LOG
static final String	MAX_BUFFER_COUNT_KEY
private final int	maxBufCount
private boolean	maxPoolSizeInfoLevelLogged
static final String	MIN_ALLOCATE_SIZE_KEY
private final int	minSizeForReservoirUse
static final ByteBuffAllocator.Recycler	NONE
private final LongAdder	poolAllocationBytes
protected final boolean	reservoirEnabled
private final AtomicInteger	usedBufCount

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ByteBuffAllocator(boolean reservoirEnabled, int maxBufCount, int bufSize, int minSizeForReservoirUse)

Method Summary

Modifier and Type	Method	Description
ByteBuff	allocate(int size)	Allocate size bytes from the ByteBufAllocator, Note to call the ByteBuff.release() if no need any more, otherwise the memory leak happen in NIO ByteBuffer pool.
SingleByteBuff	allocateOneBuffer()	Allocate an buffer with buffer size from ByteBuffAllocator, Note to call the ByteBuff.release() if no need any more, otherwise the memory leak happen in NIO ByteBuffer pool.
private ByteBuffer	allocateOnHeap(int size)
void	clean()	Free all direct buffers if allocated, mainly used for testing.
static ByteBuffAllocator	create(org.apache.hadoop.conf.Configuration conf, boolean reservoirEnabled)	Initialize an ByteBuffAllocator which will try to allocate ByteBuffers from off-heap if reservoir is enabled and the reservoir has enough buffers, otherwise the allocator will just allocate the insufficient buffers from on-heap to meet the requirement.
private static ByteBuffAllocator	createOnHeap()	Initialize an ByteBuffAllocator which only allocate ByteBuffer from on-heap, it's designed for testing purpose or disabled reservoir case.
private ByteBuffer	getBuffer()
int	getBufferSize()
int	getFreeBufferCount()	The ConcurrentLinkedQueue.size() is O(N) complexity and time-consuming, so DO NOT use the method except in UT.
long	getHeapAllocationBytes()
static long	getHeapAllocationBytes(ByteBuffAllocator... allocators)
static double	getHeapAllocationRatio(ByteBuffAllocator... allocators)
long	getPoolAllocationBytes()
int	getTotalBufferCount()
int	getUsedBufferCount()
boolean	isReservoirEnabled()
protected void	putbackBuffer(ByteBuffer buf)	Return back a ByteBuffer after its use.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

HEAP

public static final ByteBuffAllocator HEAP

ALLOCATOR_POOL_ENABLED_KEY

public static final String ALLOCATOR_POOL_ENABLED_KEY

See Also:

• Constant Field Values

MAX_BUFFER_COUNT_KEY

public static final String MAX_BUFFER_COUNT_KEY

See Also:

• Constant Field Values

BUFFER_SIZE_KEY

public static final String BUFFER_SIZE_KEY

See Also:

• Constant Field Values

MIN_ALLOCATE_SIZE_KEY

public static final String MIN_ALLOCATE_SIZE_KEY

See Also:

• Constant Field Values

BYTEBUFF_ALLOCATOR_CLASS

public static final String BYTEBUFF_ALLOCATOR_CLASS

Set an alternate bytebuffallocator by setting this config, e.g. we can config DeallocateRewriteByteBuffAllocator to find out prematurely release issues

See Also:

• Constant Field Values

DEPRECATED_ALLOCATOR_POOL_ENABLED_KEY

@Deprecated
public static final String DEPRECATED_ALLOCATOR_POOL_ENABLED_KEY

Deprecated.

since 2.3.0 and will be removed in 4.0.0. Use ALLOCATOR_POOL_ENABLED_KEY instead.

See Also:

• Constant Field Values

DEPRECATED_MAX_BUFFER_COUNT_KEY

@Deprecated
static final String DEPRECATED_MAX_BUFFER_COUNT_KEY

Deprecated.

since 2.3.0 and will be removed in 4.0.0. Use MAX_BUFFER_COUNT_KEY instead.

See Also:

• Constant Field Values

DEPRECATED_BUFFER_SIZE_KEY

@Deprecated
static final String DEPRECATED_BUFFER_SIZE_KEY

Deprecated.

since 2.3.0 and will be removed in 4.0.0. Use BUFFER_SIZE_KEY instead.

See Also:

• Constant Field Values

DEFAULT_BUFFER_SIZE

public static final int DEFAULT_BUFFER_SIZE

There're some reasons why better to choose 65KB(rather than 64KB) as the default buffer size:

1. Almost all of the data blocks have the block size: 64KB + delta, whose delta is very small, depends on the size of lastKeyValue. If we set buffer.size=64KB, then each block will be allocated as a MultiByteBuff: one 64KB DirectByteBuffer and delta bytes HeapByteBuffer, the HeapByteBuffer will increase the GC pressure. Ideally, we should let the data block to be allocated as a SingleByteBuff, it has simpler data structure, faster access speed, less heap usage.

2. Since the blocks are MultiByteBuff when using buffer.size=64KB, so we have to calculate the checksum by an temp heap copying (see HBASE-21917), while if it's a SingleByteBuff, we can speed the checksum by calling the hadoop' checksum in native lib, which is more faster.

For performance comparison, please see HBASE-22483.

See Also:

• Constant Field Values

NONE

public static final ByteBuffAllocator.Recycler NONE

reservoirEnabled

protected final boolean reservoirEnabled

bufSize

protected final int bufSize

maxBufCount

private final int maxBufCount

usedBufCount

private final AtomicInteger usedBufCount

maxPoolSizeInfoLevelLogged

private boolean maxPoolSizeInfoLevelLogged

minSizeForReservoirUse

private final int minSizeForReservoirUse

buffers

private final Queue<ByteBuffer> buffers

poolAllocationBytes

private final LongAdder poolAllocationBytes

heapAllocationBytes

private final LongAdder heapAllocationBytes

lastPoolAllocationBytes

private long lastPoolAllocationBytes

lastHeapAllocationBytes

private long lastHeapAllocationBytes

Constructor Details

ByteBuffAllocator

protected ByteBuffAllocator(boolean reservoirEnabled,
 int maxBufCount,
 int bufSize,
 int minSizeForReservoirUse)

Method Details

create

public static ByteBuffAllocator create(org.apache.hadoop.conf.Configuration conf,
 boolean reservoirEnabled)

Initialize an ByteBuffAllocator which will try to allocate ByteBuffers from off-heap if reservoir is enabled and the reservoir has enough buffers, otherwise the allocator will just allocate the insufficient buffers from on-heap to meet the requirement.

Parameters:

conf - which get the arguments to initialize the allocator.

reservoirEnabled - indicate whether the reservoir is enabled or disabled. NOTICE: if reservoir is enabled, then we will use the pool allocator to allocate off-heap ByteBuffers and use the HEAP allocator to allocate heap ByteBuffers. Otherwise if reservoir is disabled then all allocations will happen in HEAP instance.

Returns:

ByteBuffAllocator to manage the byte buffers.

createOnHeap

private static ByteBuffAllocator createOnHeap()

Initialize an ByteBuffAllocator which only allocate ByteBuffer from on-heap, it's designed for testing purpose or disabled reservoir case.

Returns:

allocator to allocate on-heap ByteBuffer.

isReservoirEnabled

public boolean isReservoirEnabled()

getHeapAllocationBytes

public long getHeapAllocationBytes()

getPoolAllocationBytes

public long getPoolAllocationBytes()

getBufferSize

public int getBufferSize()

getUsedBufferCount

public int getUsedBufferCount()

getFreeBufferCount

public int getFreeBufferCount()

The ConcurrentLinkedQueue.size() is O(N) complexity and time-consuming, so DO NOT use the method except in UT.

getTotalBufferCount

public int getTotalBufferCount()

getHeapAllocationBytes

public static long getHeapAllocationBytes(ByteBuffAllocator... allocators)

getHeapAllocationRatio

public static double getHeapAllocationRatio(ByteBuffAllocator... allocators)

allocateOneBuffer

public SingleByteBuff allocateOneBuffer()

Allocate an buffer with buffer size from ByteBuffAllocator, Note to call the ByteBuff.release() if no need any more, otherwise the memory leak happen in NIO ByteBuffer pool.

Returns:

an ByteBuff with the buffer size.

allocateOnHeap

private ByteBuffer allocateOnHeap(int size)

allocate

public ByteBuff allocate(int size)

Allocate size bytes from the ByteBufAllocator, Note to call the ByteBuff.release() if no need any more, otherwise the memory leak happen in NIO ByteBuffer pool.

Parameters:

size - to allocate

Returns:

an ByteBuff with the desired size.

clean

public void clean()

Free all direct buffers if allocated, mainly used for testing.

getBuffer

private ByteBuffer getBuffer()

Returns:

One free DirectByteBuffer from the pool. If no free ByteBuffer and we have not reached the maximum pool size, it will create a new one and return. In case of max pool size also reached, will return null. When pool returned a ByteBuffer, make sure to return it back to pool after use.

putbackBuffer

protected void putbackBuffer(ByteBuffer buf)

Return back a ByteBuffer after its use. Don't read/write the ByteBuffer after the returning.

Parameters:

buf - ByteBuffer to return.