Package org.apache.hadoop.hbase.client

Class ScannerCallableWithReplicas

java.lang.Object

org.apache.hadoop.hbase.client.ScannerCallableWithReplicas

All Implemented Interfaces:

RetryingCallable<Result[]>

@Private
class ScannerCallableWithReplicas
extends Object
implements RetryingCallable<Result[]>

This class has the logic for handling scanners for regions with and without replicas. 1. A scan is attempted on the default (primary) region, or a specific region. 2. The scanner sends all the RPCs to the default/specific region until it is done, or, there is a timeout on the default/specific region (a timeout of zero is disallowed). 3. If there is a timeout in (2) above, scanner(s) is opened on the non-default replica(s) only for Consistency.TIMELINE without specific replica id specified. 4. The results from the first successful scanner are taken, and it is stored which server returned the results. 5. The next RPCs are done on the above stored server until it is done or there is a timeout, in which case, the other replicas are queried (as in (3) above).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) class	ScannerCallableWithReplicas.RetryingRPC

Field Summary

Fields

Modifier and Type	Field	Description
private final RpcRetryingCaller<Result[]>	caller
private final ClusterConnection	cConnection
private org.apache.hadoop.conf.Configuration	conf
(package private) ScannerCallable	currentScannerCallable
private Result	lastResult
private static final org.slf4j.Logger	LOG
private Set<ScannerCallable>	outstandingCallables
protected final ExecutorService	pool
private final int	readRpcTimeout
private int	regionReplication
(package private) AtomicBoolean	replicaSwitched
private final int	retries
private final Scan	scan
private final int	scannerTimeout
private boolean	someRPCcancelled
private final TableName	tableName
protected final int	timeBeforeReplicas
private final boolean	useScannerTimeoutForNextCalls

Constructor Summary

Constructors

Constructor	Description
ScannerCallableWithReplicas(TableName tableName, ClusterConnection cConnection, ScannerCallable baseCallable, ExecutorService pool, int timeBeforeReplicas, Scan scan, int retries, int readRpcTimeout, int scannerTimeout, boolean useScannerTimeoutForNextCalls, int caching, org.apache.hadoop.conf.Configuration conf, RpcRetryingCaller<Result[]> caller)

Method Summary

Modifier and Type	Method	Description
private void	addCallsForCurrentReplica(ResultBoundedCompletionService<Pair<Result[],ScannerCallable>> cs, int rpcTimeout)
private void	addCallsForOtherReplicas(ResultBoundedCompletionService<Pair<Result[],ScannerCallable>> cs, int min, int max, int rpcTimeout)
Result[]	call(int timeout)	Computes a result, or throws an exception if unable to do so.
int	getCaching()
Cursor	getCursor()
String	getExceptionMessageAdditionalDetail()	Returns Some details from the implementation that we would like to add to a terminating exception; i.e.
HRegionInfo	getHRegionInfo()
private int	getRpcTimeout()	As we have a call sequence for scan, it is useless to have a different rpc timeout which is less than the scan timeout.
(package private) boolean	isAnyRPCcancelled()
boolean	isHeartbeatMessage()	Returns true when the most recent RPC response indicated that the response was a heartbeat message.
private boolean	isNextCall()
ScannerCallable.MoreResults	moreResultsForScan()
ScannerCallable.MoreResults	moreResultsInRegion()
void	prepare(boolean reload)	Prepare by setting up any connections to servers, etc., ahead of call invocation.
void	setCaching(int caching)
void	setClose()
void	setRenew(boolean val)
private void	setStartRowForReplicaCallable(ScannerCallable callable)	Set the start row for the replica callable based on the state of the last result received.
long	sleep(long pause, int tries)	Sleep and retry.
boolean	switchedToADifferentReplica()	When a scanner switches in the middle of scanning (the 'next' call fails for example), the upper layer ClientScanner needs to know
void	throwable(Throwable t, boolean retrying)	Called when call throws an exception and we are going to retry; take action to make it so we succeed on next call (clear caches, do relookup of locations, etc.).
private void	updateCurrentlyServingReplica(ScannerCallable scanner, Result[] result, AtomicBoolean done, ExecutorService pool)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

currentScannerCallable

volatile ScannerCallable currentScannerCallable

replicaSwitched

AtomicBoolean replicaSwitched

cConnection

private final ClusterConnection cConnection

pool

protected final ExecutorService pool

useScannerTimeoutForNextCalls

private final boolean useScannerTimeoutForNextCalls

timeBeforeReplicas

protected final int timeBeforeReplicas

scan

private final Scan scan

retries

private final int retries

lastResult

private Result lastResult

caller

private final RpcRetryingCaller<Result[]> caller

tableName

private final TableName tableName

conf

private org.apache.hadoop.conf.Configuration conf

scannerTimeout

private final int scannerTimeout

readRpcTimeout

private final int readRpcTimeout

outstandingCallables

private Set<ScannerCallable> outstandingCallables

someRPCcancelled

private boolean someRPCcancelled

regionReplication

private int regionReplication

Constructor Details

ScannerCallableWithReplicas

public ScannerCallableWithReplicas(TableName tableName,
 ClusterConnection cConnection,
 ScannerCallable baseCallable,
 ExecutorService pool,
 int timeBeforeReplicas,
 Scan scan,
 int retries,
 int readRpcTimeout,
 int scannerTimeout,
 boolean useScannerTimeoutForNextCalls,
 int caching,
 org.apache.hadoop.conf.Configuration conf,
 RpcRetryingCaller<Result[]> caller)

Method Details

setClose

public void setClose()

setRenew

public void setRenew(boolean val)

setCaching

public void setCaching(int caching)

getCaching

public int getCaching()

getHRegionInfo

public HRegionInfo getHRegionInfo()

moreResultsInRegion

public ScannerCallable.MoreResults moreResultsInRegion()

moreResultsForScan

public ScannerCallable.MoreResults moreResultsForScan()

call

public Result[] call(int timeout)
              throws IOException

Description copied from interface: RetryingCallable

Computes a result, or throws an exception if unable to do so.

Specified by:

call in interface RetryingCallable<Result[]>

Parameters:

timeout - - the time available for this call. 0 for infinite.

Returns:

computed result

Throws:

IOException

updateCurrentlyServingReplica

private void updateCurrentlyServingReplica(ScannerCallable scanner,
 Result[] result,
 AtomicBoolean done,
 ExecutorService pool)

switchedToADifferentReplica

public boolean switchedToADifferentReplica()

When a scanner switches in the middle of scanning (the 'next' call fails for example), the upper layer ClientScanner needs to know

isHeartbeatMessage

public boolean isHeartbeatMessage()

Returns true when the most recent RPC response indicated that the response was a heartbeat message. Heartbeat messages are sent back from the server when the processing of the scan request exceeds a certain time threshold. Heartbeats allow the server to avoid timeouts during long running scan operations.

getCursor

public Cursor getCursor()

addCallsForCurrentReplica

private void addCallsForCurrentReplica(ResultBoundedCompletionService<Pair<Result[],ScannerCallable>> cs,
 int rpcTimeout)

getRpcTimeout

private int getRpcTimeout()

As we have a call sequence for scan, it is useless to have a different rpc timeout which is less than the scan timeout. If the server does not respond in time(usually this will not happen as we have heartbeat now), we will get an OutOfOrderScannerNextException when resending the next request and the only way to fix this is to close the scanner and open a new one.

The legacy behavior of ScannerCallable has been to use readRpcTimeout despite the above. If using legacy behavior, we always use that.

If new behavior is enabled, we determine the rpc timeout to use based on whether the scanner is open. If scanner is open, use scannerTimeout otherwise use readRpcTimeout.

isNextCall

private boolean isNextCall()

addCallsForOtherReplicas

private void addCallsForOtherReplicas(ResultBoundedCompletionService<Pair<Result[],ScannerCallable>> cs,
 int min,
 int max,
 int rpcTimeout)

setStartRowForReplicaCallable

private void setStartRowForReplicaCallable(ScannerCallable callable)

Set the start row for the replica callable based on the state of the last result received.

Parameters:

callable - The callable to set the start row on

isAnyRPCcancelled

boolean isAnyRPCcancelled()

prepare

public void prepare(boolean reload)
             throws IOException

Description copied from interface: RetryingCallable

Prepare by setting up any connections to servers, etc., ahead of call invocation. TODO: We call prepare before EVERY call. Seems wrong. FIX!!!!

Specified by:

prepare in interface RetryingCallable<Result[]>

Parameters:

reload - Set this to true if need to requery locations

Throws:

IOException - e

throwable

public void throwable(Throwable t,
 boolean retrying)

Description copied from interface: RetryingCallable

Called when call throws an exception and we are going to retry; take action to make it so we succeed on next call (clear caches, do relookup of locations, etc.).

Specified by:

throwable in interface RetryingCallable<Result[]>

Parameters:

t - throwable which was thrown

retrying - True if we are in retrying mode (we are not in retrying mode when max retries == 1; we ARE in retrying mode if retries > 1 even when we are the last attempt)

getExceptionMessageAdditionalDetail

public String getExceptionMessageAdditionalDetail()

Description copied from interface: RetryingCallable

Returns Some details from the implementation that we would like to add to a terminating exception; i.e. a fatal exception is being thrown ending retries and we might like to add more implementation-specific detail on to the exception being thrown.

Specified by:

getExceptionMessageAdditionalDetail in interface RetryingCallable<Result[]>

sleep

public long sleep(long pause,
 int tries)

Description copied from interface: RetryingCallable

Sleep and retry.

Specified by:

sleep in interface RetryingCallable<Result[]>

Parameters:

pause - time to pause

tries - amount of tries until till sleep

Returns:

Suggestion on how much to sleep between retries