Package org.apache.hadoop.hbase.master.procedure

Class SnapshotRegionProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<MasterProcedureEnv>

org.apache.hadoop.hbase.master.procedure.SnapshotRegionProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

@Private
public class SnapshotRegionProcedure
extends Procedure<MasterProcedureEnv>
implements TableProcedureInterface, RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

A remote procedure which is used to send region snapshot request to region server. The basic logic of SnapshotRegionProcedure is similar like ServerRemoteProcedure, only with a little difference, when FailedRemoteDispatchException was thrown, SnapshotRegionProcedure will sleep some time and continue retrying until success.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field	Description
private boolean	dispatched
private ProcedureEvent<?>	event
private static final org.slf4j.Logger	LOG
private RegionInfo	region
private RetryCounter	retryCounter
private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription	snapshot
private boolean	succ

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Constructor Summary

Constructors

Constructor	Description
SnapshotRegionProcedure()
SnapshotRegionProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, RegionInfo region)

Method Summary

Modifier and Type	Method	Description
protected boolean	abort(MasterProcedureEnv env)	The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable.
protected Procedure.LockState	acquireLock(MasterProcedureEnv env)	The user should override this method if they need a lock on an Entity.
private void	complete(MasterProcedureEnv env, Throwable error)
protected void	deserializeStateData(ProcedureStateSerializer serializer)	Called on store load to allow the user to decode the previously serialized state.
protected Procedure<MasterProcedureEnv>[]	execute(MasterProcedureEnv env)	The main code of the procedure.
String	getProcName()
RegionInfo	getRegion()
TableName	getTableName()	Returns the name of the table the procedure is operating on
TableProcedureInterface.TableOperationType	getTableOperationType()	Given an operation type we can take decisions about what to do with pending operations.
protected boolean	holdLock(MasterProcedureEnv env)	Used to keep the procedure lock even when the procedure is yielding or suspended.
(package private) boolean	inRetrying()
protected void	releaseLock(MasterProcedureEnv env)	The user should override this method, and release lock if necessary.
Optional<RemoteProcedureDispatcher.RemoteOperation>	remoteCallBuild(MasterProcedureEnv env, ServerName serverName)	For building the remote operation.
void	remoteCallFailed(MasterProcedureEnv env, ServerName serverName, IOException e)	Called when the executeProcedure call is failed.
void	remoteOperationCompleted(MasterProcedureEnv env)	Called when RS tells the remote procedure is succeeded through the reportProcedureDone method.
void	remoteOperationFailed(MasterProcedureEnv env, RemoteProcedureException e)	Called when RS tells the remote procedure is failed through the reportProcedureDone method.
protected void	rollback(MasterProcedureEnv env)	The code to undo what was done by the execute() code.
protected void	serializeStateData(ProcedureStateSerializer serializer)	The user-level code of the procedure may have some state to persist (e.g.
protected boolean	setTimeoutFailure(MasterProcedureEnv env)	Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.
private void	setTimeoutForSuspend(MasterProcedureEnv env, String reason)
protected void	toStringClassDetails(StringBuilder builder)	Extend the toString() information with the procedure details e.g.
protected boolean	waitInitialized(MasterProcedureEnv env)	The Procedure.doAcquireLock(Object, ProcedureStore) will be split into two steps, first, it will call us to determine whether we need to wait for initialization, second, it will call Procedure.acquireLock(Object) to actually handle the lock for this procedure.

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, bypass, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcedureMetrics, getProcId, getProcIdHashCode, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRollbackSupported, isRunnable, isSuccess, isWaiting, isYieldAfterExecutionStep, removeStackIndex, setAbortFailure, setChildrenLatch, setExecuted, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, shouldWaitClientAck, skipPersistence, suspend, toString, toStringClass, toStringDetails, toStringSimpleSB, toStringState, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.RemoteProcedure

storeInDispatchedQueue

Field Details

LOG

private static final org.slf4j.Logger LOG

snapshot

private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot

event

private ProcedureEvent<?> event

region

private RegionInfo region

dispatched

private boolean dispatched

succ

private boolean succ

retryCounter

private RetryCounter retryCounter

Constructor Details

SnapshotRegionProcedure

public SnapshotRegionProcedure()

SnapshotRegionProcedure

public SnapshotRegionProcedure(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
 RegionInfo region)

Method Details

acquireLock

protected Procedure.LockState acquireLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method if they need a lock on an Entity. A lock can be anything, and it is up to the implementor. The Procedure Framework will call this method just before it invokes Procedure.execute(Object). It calls Procedure.releaseLock(Object) after the call to execute.

If you need to hold the lock for the life of the Procedure -- i.e. you do not want any other Procedure interfering while this Procedure is running, see Procedure.holdLock(Object).

Example: in our Master we can execute request in parallel for different tables. We can create t1 and create t2 and these creates can be executed at the same time. Anything else on t1/t2 is queued waiting that specific table create to happen.

There are 3 LockState:

• LOCK_ACQUIRED should be returned when the proc has the lock and the proc is ready to execute.
• LOCK_YIELD_WAIT should be returned when the proc has not the lock and the framework should take care of readding the procedure back to the runnable set for retry
• LOCK_EVENT_WAIT should be returned when the proc has not the lock and someone will take care of readding the procedure back to the runnable set when the lock is available.

Overrides:

acquireLock in class Procedure<MasterProcedureEnv>

Returns:

the lock state as described above.

releaseLock

protected void releaseLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method, and release lock if necessary.

Overrides:

releaseLock in class Procedure<MasterProcedureEnv>

holdLock

protected boolean holdLock(MasterProcedureEnv env)

Description copied from class: Procedure

Used to keep the procedure lock even when the procedure is yielding or suspended.

Overrides:

holdLock in class Procedure<MasterProcedureEnv>

Returns:

true if the procedure should hold on the lock until completionCleanup()

remoteCallBuild

public Optional<RemoteProcedureDispatcher.RemoteOperation> remoteCallBuild(MasterProcedureEnv env,
 ServerName serverName)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

For building the remote operation. May be empty if no need to send remote call. Usually, this means the RemoteProcedure has been finished already. This is possible, as we may have already sent the procedure to RS but then the rpc connection is broken so the executeProcedures call fails, but the RS does receive the procedure and execute it and then report back, before we retry again.

Specified by:

remoteCallBuild in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteCallFailed

public void remoteCallFailed(MasterProcedureEnv env,
 ServerName serverName,
 IOException e)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Called when the executeProcedure call is failed.

Specified by:

remoteCallFailed in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteOperationCompleted

public void remoteOperationCompleted(MasterProcedureEnv env)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Called when RS tells the remote procedure is succeeded through the reportProcedureDone method.

Specified by:

remoteOperationCompleted in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

remoteOperationFailed

public void remoteOperationFailed(MasterProcedureEnv env,
 RemoteProcedureException e)

Description copied from interface: RemoteProcedureDispatcher.RemoteProcedure

Called when RS tells the remote procedure is failed through the reportProcedureDone method.

Specified by:

remoteOperationFailed in interface RemoteProcedureDispatcher.RemoteProcedure<MasterProcedureEnv,ServerName>

complete

private void complete(MasterProcedureEnv env,
 Throwable error)

getTableName

public TableName getTableName()

Description copied from interface: TableProcedureInterface

Returns the name of the table the procedure is operating on

Specified by:

getTableName in interface TableProcedureInterface

getTableOperationType

public TableProcedureInterface.TableOperationType getTableOperationType()

Description copied from interface: TableProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.

Specified by:

getTableOperationType in interface TableProcedureInterface

Returns:

the operation type that the procedure is executing.

execute

protected Procedure<MasterProcedureEnv>[] execute(MasterProcedureEnv env)
                                           throws ProcedureYieldException,
ProcedureSuspendedException,
InterruptedException

Description copied from class: Procedure

The main code of the procedure. It must be idempotent since execute() may be called multiple times in case of machine failure in the middle of the execution.

Specified by:

execute in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

Returns:

a set of sub-procedures to run or ourselves if there is more work to do or null if the procedure is done.

Throws:

ProcedureYieldException - the procedure will be added back to the queue and retried later.

ProcedureSuspendedException - Signal to the executor that Procedure has suspended itself and has set itself up waiting for an external event to wake it back up again.

InterruptedException - the procedure will be added back to the queue and retried later.

rollback

protected void rollback(MasterProcedureEnv env)

Description copied from class: Procedure

The code to undo what was done by the execute() code. It is called when the procedure or one of the sub-procedures failed or an abort was requested. It should cleanup all the resources created by the execute() call. The implementation must be idempotent since rollback() may be called multiple time in case of machine failure in the middle of the execution.

Specified by:

rollback in class Procedure<MasterProcedureEnv>

Parameters:

env - the environment passed to the ProcedureExecutor

setTimeoutForSuspend

private void setTimeoutForSuspend(MasterProcedureEnv env,
 String reason)

setTimeoutFailure

protected boolean setTimeoutFailure(MasterProcedureEnv env)

Description copied from class: Procedure

Called by the ProcedureExecutor when the timeout set by setTimeout() is expired.

Another usage for this method is to implement retrying. A procedure can set the state to WAITING_TIMEOUT by calling setState method, and throw a ProcedureSuspendedException to halt the execution of the procedure, and do not forget a call Procedure.setTimeout(int) method to set the timeout. And you should also override this method to wake up the procedure, and also return false to tell the ProcedureExecutor that the timeout event has been handled.

Overrides:

setTimeoutFailure in class Procedure<MasterProcedureEnv>

Returns:

true to let the framework handle the timeout as abort, false in case the procedure handled the timeout itself.

abort

protected boolean abort(MasterProcedureEnv env)

Description copied from class: Procedure

The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent.

NOTE: abort() is not like Thread.interrupt(). It is just a notification that allows the procedure implementor abort.

Specified by:

abort in class Procedure<MasterProcedureEnv>

serializeStateData

protected void serializeStateData(ProcedureStateSerializer serializer)
                           throws IOException

Description copied from class: Procedure

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Specified by:

serializeStateData in class Procedure<MasterProcedureEnv>

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected void deserializeStateData(ProcedureStateSerializer serializer)
                             throws IOException

Description copied from class: Procedure

Called on store load to allow the user to decode the previously serialized state.

Specified by:

deserializeStateData in class Procedure<MasterProcedureEnv>

Parameters:

serializer - contains the serialized state

Throws:

IOException

getProcName

public String getProcName()

Overrides:

getProcName in class Procedure<MasterProcedureEnv>

toStringClassDetails

protected void toStringClassDetails(StringBuilder builder)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class Procedure<MasterProcedureEnv>

Parameters:

builder - the string builder to use to append the proc specific information

waitInitialized

protected boolean waitInitialized(MasterProcedureEnv env)

Description copied from class: Procedure

The Procedure.doAcquireLock(Object, ProcedureStore) will be split into two steps, first, it will call us to determine whether we need to wait for initialization, second, it will call Procedure.acquireLock(Object) to actually handle the lock for this procedure.

This is because that when master restarts, we need to restore the lock state for all the procedures to not break the semantic if Procedure.holdLock(Object) is true. But the ProcedureExecutor will be started before the master finish initialization(as it is part of the initialization!), so we need to split the code into two steps, and when restore, we just restore the lock part and ignore the waitInitialized part. Otherwise there will be dead lock.

Overrides:

waitInitialized in class Procedure<MasterProcedureEnv>

Returns:

true means we need to wait until the environment has been initialized, otherwise true.

getRegion

public RegionInfo getRegion()

inRetrying

boolean inRetrying()