Package org.apache.hadoop.hbase.io.hfile

Class HFileReaderImpl.HFileScannerImpl

java.lang.Object

org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.HFileScannerImpl

All Implemented Interfaces:

Closeable, AutoCloseable, HFileScanner, Shipper

Direct Known Subclasses:

HFileReaderImpl.EncodedScanner

Enclosing class:

HFileReaderImpl

public static class HFileReaderImpl.HFileScannerImpl
extends Object
implements HFileScanner

Field Summary

Fields

Modifier and Type	Field	Description
private ByteBuff	blockBuffer
private ByteBufferKeyOnlyKeyValue	bufBackedKeyOnlyKv
protected final boolean	cacheBlocks
protected HFileBlock	curBlock
private int	currKeyLen
private long	currMemstoreTS
private int	currMemstoreTSLen
private int	currTagsLen
private int	currValueLen
protected final boolean	isCompaction
protected ExtendedCell	nextIndexedKey	The next indexed key is to keep track of the indexed key of the next data block.
(package private) final ObjectIntPair<ByteBuffer>	pair
protected final boolean	pread
protected final ArrayList<HFileBlock>	prevBlocks
private boolean	providedCurrentBlockSize
protected final HFile.Reader	reader
private short	rowLen

Constructor Summary

Constructors

Constructor	Description
HFileScannerImpl(HFile.Reader reader, boolean cacheBlocks, boolean pread, boolean isCompaction)

Method Summary

Modifier and Type	Method	Description
private final boolean	_next()
private void	_readMvccVersion(int offsetFromPos)	Actually do the mvcc read.
protected void	assertSeeked()
protected int	blockSeek(Cell key, boolean seekBefore)	Within a loaded block, seek looking for the last key that is smaller than (or equal to?) the key we are interested in.
protected final boolean	checkKeyLen(int v)	Returns True if v <= 0 or v > current block buffer limit.
protected final void	checkKeyValueLen()	Check key and value lengths are wholesome.
protected final boolean	checkLen(int v)	Returns True if v < 0 or v > current block buffer limit.
private final void	checkTagsLen()
void	close()	Close this HFile scanner and do necessary cleanup.
int	compareKey(CellComparator comparator, ExtendedCell key)
ExtendedCell	getCell()	Returns Instance of ExtendedCell.
HFileBlock	getCurBlock()
private int	getCurCellSerializedSize()
DataBlockEncoding	getEffectiveDataBlockEncoding()
protected ExtendedCell	getFirstKeyCellInBlock(HFileBlock curBlock)
ExtendedCell	getKey()	Gets the current key in the form of a cell.
private int	getKVBufSize()
ExtendedCell	getNextIndexedKey()	Returns the next key in the index (the key to seek to the next block)
HFile.Reader	getReader()	Returns Reader that underlies this Scanner instance.
ByteBuffer	getValue()	Gets a buffer view to the current value.
private boolean	isNextBlock()
boolean	isSeeked()
protected int	loadBlockAndSeekToKey(HFileBlock seekToBlock, ExtendedCell nextIndexedKey, boolean rewind, ExtendedCell key, boolean seekBefore)
boolean	next()	Go to the next key/value in the block section.
private boolean	positionForNextBlock()	Set our selves up for the next 'next' invocation, set up next block.
private void	positionThisBlockBuffer()	Set the position on current backing blockBuffer.
protected boolean	processFirstDataBlock()
protected void	readAndUpdateNewBlock(long firstDataBlockOffset)
protected void	readKeyValueLen()
protected void	readMvccVersion(int offsetFromPos)	Read mvcc.
protected HFileBlock	readNextDataBlock()	Scans blocks in the "scanned" section of the HFile until the next data block is found.
void	recordBlockSize(IntConsumer blockSizeConsumer)	Record the size of the current block in bytes, passing as an argument to the blockSizeConsumer.
protected void	releaseIfNotCurBlock(HFileBlock block)	The curBlock will be released by shipping or close method, so only need to consider releasing the block, which was read from HFile before and not referenced by curBlock.
int	reseekTo(ExtendedCell key)	Reseek to or just before the passed cell.
(package private) void	reset()
private void	returnBlocks(boolean returnAll)
boolean	seekBefore(ExtendedCell key)	Consider the cell stream of all the cells in the file, c[0] ..

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

blockBuffer

private ByteBuff blockBuffer

cacheBlocks

protected final boolean cacheBlocks

pread

protected final boolean pread

isCompaction

protected final boolean isCompaction

currKeyLen

private int currKeyLen

currValueLen

private int currValueLen

currMemstoreTSLen

private int currMemstoreTSLen

currMemstoreTS

private long currMemstoreTS

reader

protected final HFile.Reader reader

currTagsLen

private int currTagsLen

rowLen

private short rowLen

bufBackedKeyOnlyKv

private ByteBufferKeyOnlyKeyValue bufBackedKeyOnlyKv

pair

final ObjectIntPair<ByteBuffer> pair

nextIndexedKey

protected ExtendedCell nextIndexedKey

The next indexed key is to keep track of the indexed key of the next data block. If the nextIndexedKey is HConstants.NO_NEXT_INDEXED_KEY, it means that the current data block is the last data block. If the nextIndexedKey is null, it means the nextIndexedKey has not been loaded yet.

curBlock

protected HFileBlock curBlock

providedCurrentBlockSize

private boolean providedCurrentBlockSize

prevBlocks

protected final ArrayList<HFileBlock> prevBlocks

Constructor Details

HFileScannerImpl

public HFileScannerImpl(HFile.Reader reader,
 boolean cacheBlocks,
 boolean pread,
 boolean isCompaction)

Method Details

getCurBlock

public HFileBlock getCurBlock()

updateCurrBlockRef

void updateCurrBlockRef(HFileBlock block)

reset

void reset()

returnBlocks

private void returnBlocks(boolean returnAll)

isSeeked

public boolean isSeeked()

Specified by:

isSeeked in interface HFileScanner

Returns:

True is scanner has had one of the seek calls invoked; i.e. HFileScanner.seekBefore(ExtendedCell) or HFileScanner.seekTo() or HFileScanner.seekTo(ExtendedCell). Otherwise returns false.

toString

public String toString()

Overrides:

toString in class Object

assertSeeked

protected void assertSeeked()

getReader

public HFile.Reader getReader()

Description copied from interface: HFileScanner

Returns Reader that underlies this Scanner instance.

Specified by:

getReader in interface HFileScanner

getKVBufSize

private int getKVBufSize()

close

public void close()

Description copied from interface: HFileScanner

Close this HFile scanner and do necessary cleanup.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface HFileScanner

recordBlockSize

public void recordBlockSize(IntConsumer blockSizeConsumer)

Description copied from interface: HFileScanner

Record the size of the current block in bytes, passing as an argument to the blockSizeConsumer. Implementations should ensure that blockSizeConsumer is only called once per block.

Specified by:

recordBlockSize in interface HFileScanner

Parameters:

blockSizeConsumer - to be called with block size in bytes, once per block.

getCurCellSerializedSize

private int getCurCellSerializedSize()

readKeyValueLen

protected void readKeyValueLen()

checkTagsLen

private final void checkTagsLen()

readMvccVersion

protected void readMvccVersion(int offsetFromPos)

Read mvcc. Does checks to see if we even need to read the mvcc at all.

_readMvccVersion

private void _readMvccVersion(int offsetFromPos)

Actually do the mvcc read. Does no checks.

blockSeek

protected int blockSeek(Cell key,
 boolean seekBefore)

Within a loaded block, seek looking for the last key that is smaller than (or equal to?) the key we are interested in. A note on the seekBefore: if you have seekBefore = true, AND the first key in the block = key, then you'll get thrown exceptions. The caller has to check for that case and load the previous block as appropriate. the key to find find the key before the given key in case of exact match.

Returns:

0 in case of an exact key match, 1 in case of an inexact match, -2 in case of an inexact match and furthermore, the input key less than the first key of current block(e.g. using a faked index key)

getNextIndexedKey

public ExtendedCell getNextIndexedKey()

Description copied from interface: HFileScanner

Returns the next key in the index (the key to seek to the next block)

Specified by:

getNextIndexedKey in interface HFileScanner

seekTo

public int seekTo(ExtendedCell key)
           throws IOException

Description copied from interface: HFileScanner

SeekTo or just before the passed cell. Examine the return code to figure whether we found the cell or not. Consider the cell stream of all the cells in the file, c[0] .. c[n], where there are n cells in the file.

Specified by:

seekTo in interface HFileScanner

Returns:

-1, if cell < c[0], no position; 0, such that c[i] = cell and scanner is left in position i; and 1, such that c[i] < cell, and scanner is left in position i. The scanner will position itself between c[i] and c[i+1] where c[i] < cell <= c[i+1]. If there is no cell c[i+1] greater than or equal to the input cell, then the scanner will position itself at the end of the file and next() will return false when it is called.

Throws:

IOException

reseekTo

public int reseekTo(ExtendedCell key)
             throws IOException

Description copied from interface: HFileScanner

Reseek to or just before the passed cell. Similar to seekTo except that this can be called even if the scanner is not at the beginning of a file. This can be used to seek only to cells which come after the current position of the scanner. Consider the cell stream of all the cells in the file, c[0] .. c[n], where there are n cellc in the file after current position of HFileScanner. The scanner will position itself between c[i] and c[i+1] where c[i] < cell <= c[i+1]. If there is no cell c[i+1] greater than or equal to the input cell, then the scanner will position itself at the end of the file and next() will return false when it is called.

Specified by:

reseekTo in interface HFileScanner

Parameters:

key - Cell to find (should be non-null)

Returns:

-1, if cell < c[0], no position; 0, such that c[i] = cell and scanner is left in position i; and 1, such that c[i] < cell, and scanner is left in position i.

Throws:

IOException

seekTo

public int seekTo(ExtendedCell key,
 boolean rewind)
           throws IOException

An internal API function. Seek to the given key, optionally rewinding to the first key of the block before doing the seek.

Parameters:

key - - a cell representing the key that we need to fetch

rewind - whether to rewind to the first key of the block before doing the seek. If this is false, we are assuming we never go back, otherwise the result is undefined.

Returns:

-1 if the key is earlier than the first key of the file, 0 if we are at the given key, 1 if we are past the given key -2 if the key is earlier than the first key of the file while using a faked index key

Throws:

IOException

seekBefore

public boolean seekBefore(ExtendedCell key)
                   throws IOException

Description copied from interface: HFileScanner

Consider the cell stream of all the cells in the file, c[0] .. c[n], where there are n cells in the file.

Specified by:

seekBefore in interface HFileScanner

Parameters:

key - Cell to find

Returns:

false if cell <= c[0] or true with scanner in position 'i' such that: c[i] < cell. Furthermore: there may be a c[i+1], such that c[i] < cell <= c[i+1] but there may also NOT be a c[i+1], and next() will return false (EOF).

Throws:

IOException

releaseIfNotCurBlock

protected void releaseIfNotCurBlock(HFileBlock block)

The curBlock will be released by shipping or close method, so only need to consider releasing the block, which was read from HFile before and not referenced by curBlock.

readNextDataBlock

protected HFileBlock readNextDataBlock()
                                throws IOException

Scans blocks in the "scanned" section of the HFile until the next data block is found.

Returns:

the next block, or null if there are no more data blocks

Throws:

IOException

getEffectiveDataBlockEncoding

public DataBlockEncoding getEffectiveDataBlockEncoding()

getCell

public ExtendedCell getCell()

Description copied from interface: HFileScanner

Returns Instance of ExtendedCell.

Specified by:

getCell in interface HFileScanner

getKey

public ExtendedCell getKey()

Description copied from interface: HFileScanner

Gets the current key in the form of a cell. You must call HFileScanner.seekTo(ExtendedCell) before this method.

Specified by:

getKey in interface HFileScanner

Returns:

gets the current key as a Cell.

getValue

public ByteBuffer getValue()

Description copied from interface: HFileScanner

Gets a buffer view to the current value. You must call HFileScanner.seekTo(ExtendedCell) before this method.

Specified by:

getValue in interface HFileScanner

Returns:

byte buffer for the value. The limit is set to the value size, and the position is 0, the start of the buffer view.

setNonSeekedState

protected void setNonSeekedState()

positionThisBlockBuffer

private void positionThisBlockBuffer()

Set the position on current backing blockBuffer.

positionForNextBlock

private boolean positionForNextBlock()
                              throws IOException

Set our selves up for the next 'next' invocation, set up next block.

Returns:

True is more to read else false if at the end.

Throws:

IOException

isNextBlock

private boolean isNextBlock()
                     throws IOException

Throws:

IOException

_next

private final boolean _next()
                     throws IOException

Throws:

IOException

next

public boolean next()
             throws IOException

Go to the next key/value in the block section. Loads the next block if necessary. If successful, getKey() and getValue() can be called.

Specified by:

next in interface HFileScanner

Returns:

true if successfully navigated to the next key/value

Throws:

IOException

seekTo

public boolean seekTo()
               throws IOException

Positions this scanner at the start of the file.

Specified by:

seekTo in interface HFileScanner

Returns:

false if empty file; i.e. a call to next would return false and the current key and value are undefined.

Throws:

IOException

processFirstDataBlock

protected boolean processFirstDataBlock()
                                 throws IOException

Throws:

IOException

readAndUpdateNewBlock

protected void readAndUpdateNewBlock(long firstDataBlockOffset)
                              throws IOException

Throws:

IOException

loadBlockAndSeekToKey

protected int loadBlockAndSeekToKey(HFileBlock seekToBlock,
 ExtendedCell nextIndexedKey,
 boolean rewind,
 ExtendedCell key,
 boolean seekBefore)
                             throws IOException

Throws:

IOException

checkKeyLen

protected final boolean checkKeyLen(int v)

Returns True if v <= 0 or v > current block buffer limit.

checkLen

protected final boolean checkLen(int v)

Returns True if v < 0 or v > current block buffer limit.

checkKeyValueLen

protected final void checkKeyValueLen()

Check key and value lengths are wholesome.

updateCurrentBlock

protected void updateCurrentBlock(HFileBlock newBlock)
                           throws IOException

Updates the current block to be the given HFileBlock. Seeks to the the first key/value pair.

Parameters:

newBlock - the block read by HFileReaderImpl.readBlock(long, long, boolean, boolean, boolean, boolean, org.apache.hadoop.hbase.io.hfile.BlockType, org.apache.hadoop.hbase.io.encoding.DataBlockEncoding), it's a totally new block with new allocated ByteBuff, so if no further reference to this block, we should release it carefully.

Throws:

IOException

getFirstKeyCellInBlock

protected ExtendedCell getFirstKeyCellInBlock(HFileBlock curBlock)

compareKey

public int compareKey(CellComparator comparator,
 ExtendedCell key)

shipped

public void shipped()
             throws IOException

Description copied from interface: Shipper

Called after a batch of rows scanned and set to be returned to client. Any in between cleanup can be done here.

Specified by:

shipped in interface Shipper

Throws:

IOException