Package org.apache.hadoop.hbase.regionserver

Interface MemStoreLAB

All Known Implementing Classes:

ImmutableMemStoreLAB, MemStoreLABImpl

@Private
public interface MemStoreLAB

A memstore-local allocation buffer.

The MemStoreLAB is basically a bump-the-pointer allocator that allocates big (2MB) chunks from and then doles it out to threads that request slices into the array. These chunks can get pooled as well. See ChunkCreator.

The purpose of this is to combat heap fragmentation in the regionserver. By ensuring that all Cells in a given memstore refer only to large chunks of contiguous memory, we ensure that large blocks get freed up when the memstore is flushed.

Without the MSLAB, the byte array allocated during insertion end up interleaved throughout the heap, and the old generation gets progressively more fragmented until a stop-the-world compacting collection occurs.

This manages the large sized chunks. When Cells are to be added to Memstore, MemStoreLAB's copyCellInto(ExtendedCell) gets called. This allocates enough size in the chunk to hold this cell's data and copies into this area and then recreate a Cell over this copied data.

See Also:

• ChunkCreator

Field Summary

Fields

Modifier and Type	Field	Description
static final String	CHUNK_POOL_INITIALSIZE_KEY
static final String	CHUNK_POOL_MAXSIZE_KEY
static final int	CHUNK_SIZE_DEFAULT
static final String	CHUNK_SIZE_KEY
static final float	INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT
static final String	INDEX_CHUNK_SIZE_PERCENTAGE_KEY
static final int	MAX_ALLOC_DEFAULT
static final String	MAX_ALLOC_KEY
static final String	MSLAB_CLASS_NAME
static final float	POOL_INITIAL_SIZE_DEFAULT
static final float	POOL_MAX_SIZE_DEFAULT
static final boolean	USEMSLAB_DEFAULT
static final String	USEMSLAB_KEY

Method Summary

Modifier and Type	Method	Description
void	close()	Close instance since it won't be used any more, try to put the chunks back to pool
ExtendedCell	copyCellInto(ExtendedCell cell)	Allocates slice in this LAB and copy the passed Cell into this area.
void	decScannerCount()	Called when closing a scanner on the data of this MemStoreLAB
ExtendedCell	forceCopyOfBigCellInto(ExtendedCell cell)	Allocates slice in this LAB and copy the passed Cell into this area.
Chunk	getNewExternalChunk(int size)
Chunk	getNewExternalChunk(ChunkCreator.ChunkType chunkType)
void	incScannerCount()	Called when opening a scanner on the data of this MemStoreLAB
static boolean	isEnabled(org.apache.hadoop.conf.Configuration conf)
boolean	isOffHeap()
boolean	isOnHeap()
static MemStoreLAB	newInstance(org.apache.hadoop.conf.Configuration conf)

Field Details

USEMSLAB_KEY

static final String USEMSLAB_KEY

See Also:

• Constant Field Values

USEMSLAB_DEFAULT

static final boolean USEMSLAB_DEFAULT

See Also:

• Constant Field Values

MSLAB_CLASS_NAME

static final String MSLAB_CLASS_NAME

See Also:

• Constant Field Values

CHUNK_SIZE_KEY

static final String CHUNK_SIZE_KEY

See Also:

• Constant Field Values

CHUNK_SIZE_DEFAULT

static final int CHUNK_SIZE_DEFAULT

See Also:

• Constant Field Values

INDEX_CHUNK_SIZE_PERCENTAGE_KEY

static final String INDEX_CHUNK_SIZE_PERCENTAGE_KEY

See Also:

• Constant Field Values

INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT

static final float INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT

See Also:

• Constant Field Values

MAX_ALLOC_KEY

static final String MAX_ALLOC_KEY

See Also:

• Constant Field Values

MAX_ALLOC_DEFAULT

static final int MAX_ALLOC_DEFAULT

See Also:

• Constant Field Values

CHUNK_POOL_MAXSIZE_KEY

static final String CHUNK_POOL_MAXSIZE_KEY

See Also:

• Constant Field Values

CHUNK_POOL_INITIALSIZE_KEY

static final String CHUNK_POOL_INITIALSIZE_KEY

See Also:

• Constant Field Values

POOL_MAX_SIZE_DEFAULT

static final float POOL_MAX_SIZE_DEFAULT

See Also:

• Constant Field Values

POOL_INITIAL_SIZE_DEFAULT

static final float POOL_INITIAL_SIZE_DEFAULT

See Also:

• Constant Field Values

Method Details

copyCellInto

ExtendedCell copyCellInto(ExtendedCell cell)

Allocates slice in this LAB and copy the passed Cell into this area. Returns new Cell instance over the copied the data. When this MemStoreLAB can not copy this Cell, it returns null.

forceCopyOfBigCellInto

ExtendedCell forceCopyOfBigCellInto(ExtendedCell cell)

Allocates slice in this LAB and copy the passed Cell into this area. Returns new Cell instance over the copied the data. When this MemStoreLAB can not copy this Cell, it returns null. Since the process of flattening to CellChunkMap assumes all cells are allocated on MSLAB, and since copyCellInto does not copy big cells (for whom size > maxAlloc) into MSLAB, this method is called while the process of flattening to CellChunkMap is running, for forcing the allocation of big cells on this MSLAB.

close

void close()

Close instance since it won't be used any more, try to put the chunks back to pool

incScannerCount

void incScannerCount()

Called when opening a scanner on the data of this MemStoreLAB

decScannerCount

void decScannerCount()

Called when closing a scanner on the data of this MemStoreLAB

getNewExternalChunk

Chunk getNewExternalChunk(ChunkCreator.ChunkType chunkType)

getNewExternalChunk

Chunk getNewExternalChunk(int size)

newInstance

static MemStoreLAB newInstance(org.apache.hadoop.conf.Configuration conf)

isEnabled

static boolean isEnabled(org.apache.hadoop.conf.Configuration conf)

isOnHeap

boolean isOnHeap()

isOffHeap

boolean isOffHeap()