Package org.apache.hadoop.hbase.regionserver

Class RegionScannerImpl

java.lang.Object

org.apache.hadoop.hbase.regionserver.RegionScannerImpl

All Implemented Interfaces:

Closeable, AutoCloseable, RpcCallback, InternalScanner, RegionScanner, Shipper

Direct Known Subclasses:

ReversedRegionScannerImpl

@Private
class RegionScannerImpl
extends Object
implements RegionScanner, Shipper, RpcCallback

RegionScannerImpl is used to combine scanners from multiple Stores (aka column families).

Field Summary

Fields

Modifier and Type	Field	Description
protected final CellComparator	comparator
private final ScannerContext	defaultScannerContext
private final FilterWrapper	filter
private boolean	filterClosed
protected final boolean	includeStopRow
protected ExtendedCell	joinedContinuationRow	If the joined heap data gathering is interrupted due to scan limits, this will contain the row for which we are populating the values.
(package private) KeyValueHeap	joinedHeap	Heap of key-values that are not essential for the provided filters and are thus read on demand, if on-demand column family loading is enabled.
private static final org.slf4j.Logger	LOG
private final long	maxResultSize
private static final List<Cell>	MOCKED_LIST	A mocked list implementation - discards all updates.
private final String	operationId
private final long	readPt
protected final HRegion	region
private RegionServerServices	rsServices
private final ConcurrentHashMap<RegionScanner,Long>	scannerReadPoints
protected final byte[]	stopRow
(package private) KeyValueHeap	storeHeap

Constructor Summary

Constructors

Constructor	Description
RegionScannerImpl(Scan scan, List<KeyValueScanner> additionalScanners, HRegion region, long nonceGroup, long nonce)

Method Summary

Modifier and Type	Method	Description
private void	checkClientDisconnect(Optional<RpcCall> rpcCall)
void	close()	Closes the scanner and releases any resources it has allocated
private void	closeInternal()
private boolean	filterRow()	This function is to maintain backward compatibility for 0.94 filters.
private boolean	filterRowKey(Cell current)
int	getBatch()
long	getMaxResultSize()
long	getMvccReadPoint()	Returns The Scanner's MVCC readPt see MultiVersionConcurrencyControl
String	getOperationId()
RegionInfo	getRegionInfo()	Returns The RegionInfo for this scanner.
private IOException	handleException(List<KeyValueScanner> instantiatedScanners, Throwable t)
private static boolean	hasNonce(HRegion region, long nonce)
private void	incrementCountOfRowsFilteredMetric(ScannerContext scannerContext)
private void	incrementCountOfRowsScannedMetric(ScannerContext scannerContext)
protected void	initializeKVHeap(List<KeyValueScanner> scanners, List<KeyValueScanner> joinedScanners, HRegion region)
private void	initializeScanners(Scan scan, List<KeyValueScanner> additionalScanners)
boolean	isFilterDone()	Returns True if a filter rules the scanner is over, done.
private boolean	isFilterDoneInternal()
private boolean	joinedHeapMayHaveData(ExtendedCell currentRowCell)	Returns true when the joined heap may have data for the current row
private boolean	moreCellsInRow(Cell nextKv, Cell currentRowCell)	Based on the nextKv in the heap, and the current row, decide whether or not there are more cells to be read in the heap.
boolean	next(List<? super ExtendedCell> outResults)	Grab the next row's worth of values.
boolean	next(List<? super ExtendedCell> outResults, ScannerContext scannerContext)	Grab the next row's worth of values.
private boolean	nextInternal(List<? super ExtendedCell> results, ScannerContext scannerContext)
boolean	nextRaw(List<? super ExtendedCell> outResults)	Grab the next row's worth of values.
boolean	nextRaw(List<? super ExtendedCell> outResults, ScannerContext scannerContext)	Grab the next row's worth of values.
protected boolean	nextRow(ScannerContext scannerContext, Cell curRowCell)
private boolean	populateFromJoinedHeap(List<? super ExtendedCell> results, ScannerContext scannerContext)	Returns true if more cells exist after this batch, false if scanner is done
private boolean	populateResult(List<? super ExtendedCell> results, KeyValueHeap heap, ScannerContext scannerContext, ExtendedCell currentRowCell)	Fetches records with currentRow into results list, until next row, batchLimit (if not -1) is reached, or remainingResultSize (if not -1) is reaced
boolean	reseek(byte[] row)	Do a reseek to the required row.
protected final void	resetFilters()	Reset both the filter and the old filter.
private void	resetProgress(ScannerContext scannerContext, int initialBatchProgress, long initialSizeProgress, long initialHeapSizeProgress)
void	run()	Called at the end of an Rpc Call RpcCallContext
void	shipped()	Called after a batch of rows scanned and set to be returned to client.
protected boolean	shouldStop(Cell currentRowCell)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

storeHeap

KeyValueHeap storeHeap

joinedHeap

KeyValueHeap joinedHeap

Heap of key-values that are not essential for the provided filters and are thus read on demand, if on-demand column family loading is enabled.

joinedContinuationRow

protected ExtendedCell joinedContinuationRow

If the joined heap data gathering is interrupted due to scan limits, this will contain the row for which we are populating the values.

filterClosed

private boolean filterClosed

stopRow

protected final byte[] stopRow

includeStopRow

protected final boolean includeStopRow

region

protected final HRegion region

comparator

protected final CellComparator comparator

scannerReadPoints

private final ConcurrentHashMap<RegionScanner,Long> scannerReadPoints

readPt

private final long readPt

maxResultSize

private final long maxResultSize

defaultScannerContext

private final ScannerContext defaultScannerContext

filter

private final FilterWrapper filter

operationId

private final String operationId

rsServices

private RegionServerServices rsServices

MOCKED_LIST

private static final List<Cell> MOCKED_LIST

A mocked list implementation - discards all updates.

Constructor Details

RegionScannerImpl

RegionScannerImpl(Scan scan,
 List<KeyValueScanner> additionalScanners,
 HRegion region,
 long nonceGroup,
 long nonce)
           throws IOException

Throws:

IOException

Method Details

getRegionInfo

public RegionInfo getRegionInfo()

Description copied from interface: RegionScanner

Returns The RegionInfo for this scanner.

Specified by:

getRegionInfo in interface RegionScanner

hasNonce

private static boolean hasNonce(HRegion region,
 long nonce)

initializeScanners

private void initializeScanners(Scan scan,
 List<KeyValueScanner> additionalScanners)
                         throws IOException

Throws:

IOException

initializeKVHeap

protected void initializeKVHeap(List<KeyValueScanner> scanners,
 List<KeyValueScanner> joinedScanners,
 HRegion region)
                         throws IOException

Throws:

IOException

handleException

private IOException handleException(List<KeyValueScanner> instantiatedScanners,
 Throwable t)

getMaxResultSize

public long getMaxResultSize()

Specified by:

getMaxResultSize in interface RegionScanner

Returns:

The preferred max buffersize. See Scan.setMaxResultSize(long)

getMvccReadPoint

public long getMvccReadPoint()

Description copied from interface: RegionScanner

Returns The Scanner's MVCC readPt see MultiVersionConcurrencyControl

Specified by:

getMvccReadPoint in interface RegionScanner

getBatch

public int getBatch()

Specified by:

getBatch in interface RegionScanner

Returns:

The limit on the number of cells to retrieve on each call to next(). See Scan.setBatch(int)

getOperationId

public String getOperationId()

Specified by:

getOperationId in interface RegionScanner

Returns:

The Scanner's OperationWithAttributes.ID_ATRIBUTE value, or null if not set.

resetFilters

protected final void resetFilters()
                           throws IOException

Reset both the filter and the old filter.

Throws:

IOException - in case a filter raises an I/O exception.

next

public boolean next(List<? super ExtendedCell> outResults)
             throws IOException

Description copied from interface: InternalScanner

Grab the next row's worth of values.

The generic type for the output list result means we will only add ExtendedCell to it. This is useful for the code in HBase as we can pass List<ExtendedCell> here to avoid casting, but may cause some troubles for coprocessors which implement this method. In general, all cells created via the CellBuilder are actually ExtendedCells, so if you want to add something to the result list, you can just cast it to ExtendedCell, although it is marked as IA.Private.

Specified by:

next in interface InternalScanner

Parameters:

outResults - return output array. We will only add ExtendedCell to this list, but for CP users, you'd better just use RawCell as ExtendedCell is IA.Private.

Returns:

true if more rows exist after this one, false if scanner is done

Throws:

IOException

next

public boolean next(List<? super ExtendedCell> outResults,
 ScannerContext scannerContext)
             throws IOException

Description copied from interface: InternalScanner

Grab the next row's worth of values.

The generic type for the output list result means we will only add ExtendedCell to it. This is useful for the code in HBase as we can pass List<ExtendedCell> here to avoid casting, but may cause some troubles for coprocessors which implement this method. In general, all cells created via the CellBuilder are actually ExtendedCells, so if you want to add something to the result list, you can just cast it to ExtendedCell, although it is marked as IA.Private.

Specified by:

next in interface InternalScanner

Parameters:

outResults - return output array. We will only add ExtendedCell to this list, but for CP users, you'd better just use RawCell as ExtendedCell is IA.Private.

Returns:

true if more rows exist after this one, false if scanner is done

Throws:

IOException

nextRaw

public boolean nextRaw(List<? super ExtendedCell> outResults)
                throws IOException

Description copied from interface: RegionScanner

Grab the next row's worth of values. This is a special internal method to be called from coprocessor hooks to avoid expensive setup. Caller must set the thread's readpoint, start and close a region operation, an synchronize on the scanner object. Caller should maintain and update metrics. See RegionScanner.nextRaw(List, ScannerContext)

Specified by:

nextRaw in interface RegionScanner

Parameters:

outResults - return output array

Returns:

true if more rows exist after this one, false if scanner is done

Throws:

IOException - e

nextRaw

public boolean nextRaw(List<? super ExtendedCell> outResults,
 ScannerContext scannerContext)
                throws IOException

Description copied from interface: RegionScanner

Grab the next row's worth of values. The ScannerContext is used to enforce and track any limits associated with this call. Any progress that exists in the ScannerContext prior to calling this method will be LOST if ScannerContext.getKeepProgress() is false. Upon returning from this method, the ScannerContext will contain information about the progress made towards the limits. This is a special internal method to be called from coprocessor hooks to avoid expensive setup. Caller must set the thread's readpoint, start and close a region operation, an synchronize on the scanner object. Example: HRegion region = ...; RegionScanner scanner = ... MultiVersionConcurrencyControl.setThreadReadPoint(scanner.getMvccReadPoint()); region.startRegionOperation(); try { synchronized(scanner) { ... boolean moreRows = scanner.nextRaw(values); ... } } finally { region.closeRegionOperation(); }

Specified by:

nextRaw in interface RegionScanner

Parameters:

outResults - return output array

scannerContext - The ScannerContext instance encapsulating all limits that should be tracked during calls to this method. The progress towards these limits can be tracked within this instance.

Returns:

true if more rows exist after this one, false if scanner is done

Throws:

IOException - e

populateFromJoinedHeap

private boolean populateFromJoinedHeap(List<? super ExtendedCell> results,
 ScannerContext scannerContext)
                                throws IOException

Returns true if more cells exist after this batch, false if scanner is done

Throws:

IOException

populateResult

private boolean populateResult(List<? super ExtendedCell> results,
 KeyValueHeap heap,
 ScannerContext scannerContext,
 ExtendedCell currentRowCell)
                        throws IOException

Fetches records with currentRow into results list, until next row, batchLimit (if not -1) is reached, or remainingResultSize (if not -1) is reaced

Parameters:

heap - KeyValueHeap to fetch data from.It must be positioned on correct row before call.

Returns:

state of last call to KeyValueHeap.next()

Throws:

IOException

moreCellsInRow

private boolean moreCellsInRow(Cell nextKv,
 Cell currentRowCell)

Based on the nextKv in the heap, and the current row, decide whether or not there are more cells to be read in the heap. If the row of the nextKv in the heap matches the current row then there are more cells to be read in the row.

Returns:

true When there are more cells in the row to be read

isFilterDone

public boolean isFilterDone()
                     throws IOException

Returns True if a filter rules the scanner is over, done.

Specified by:

isFilterDone in interface RegionScanner

Returns:

True if a filter indicates that this scanner will return no further rows.

Throws:

IOException - in case of I/O failure on a filter.

isFilterDoneInternal

private boolean isFilterDoneInternal()
                              throws IOException

Throws:

IOException

checkClientDisconnect

private void checkClientDisconnect(Optional<RpcCall> rpcCall)
                            throws CallerDisconnectedException

Throws:

CallerDisconnectedException

resetProgress

private void resetProgress(ScannerContext scannerContext,
 int initialBatchProgress,
 long initialSizeProgress,
 long initialHeapSizeProgress)

nextInternal

private boolean nextInternal(List<? super ExtendedCell> results,
 ScannerContext scannerContext)
                      throws IOException

Throws:

IOException

incrementCountOfRowsFilteredMetric

private void incrementCountOfRowsFilteredMetric(ScannerContext scannerContext)

incrementCountOfRowsScannedMetric

private void incrementCountOfRowsScannedMetric(ScannerContext scannerContext)

joinedHeapMayHaveData

private boolean joinedHeapMayHaveData(ExtendedCell currentRowCell)
                               throws IOException

Returns true when the joined heap may have data for the current row

Throws:

IOException

filterRow

private boolean filterRow()
                   throws IOException

This function is to maintain backward compatibility for 0.94 filters. HBASE-6429 combines both filterRow & filterRow(List<KeyValue> kvs) functions. While 0.94 code or older, it may not implement hasFilterRow as HBase-6429 expects because 0.94 hasFilterRow() only returns true when filterRow(List<KeyValue> kvs) is overridden not the filterRow(). Therefore, the filterRow() will be skipped.

Throws:

IOException

filterRowKey

private boolean filterRowKey(Cell current)
                      throws IOException

Throws:

IOException

nextRow

protected boolean nextRow(ScannerContext scannerContext,
 Cell curRowCell)
                   throws IOException

Throws:

IOException

shouldStop

protected boolean shouldStop(Cell currentRowCell)

closeInternal

private void closeInternal()

close

public void close()

Description copied from interface: InternalScanner

Closes the scanner and releases any resources it has allocated

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface InternalScanner

reseek

public boolean reseek(byte[] row)
               throws IOException

Description copied from interface: RegionScanner

Do a reseek to the required row. Should not be used to seek to a key which may come before the current position. Always seeks to the beginning of a row boundary. if row is null

Specified by:

reseek in interface RegionScanner

Throws:

IOException

shipped

public void shipped()
             throws IOException

Description copied from interface: Shipper

Called after a batch of rows scanned and set to be returned to client. Any in between cleanup can be done here.

Specified by:

shipped in interface Shipper

Throws:

IOException

run

public void run()
         throws IOException

Description copied from interface: RpcCallback

Called at the end of an Rpc Call RpcCallContext

Specified by:

run in interface RpcCallback

Throws:

IOException