Package org.apache.hadoop.hbase.master.assignment

Class SplitTableRegionProcedure

java.lang.Object

org.apache.hadoop.hbase.procedure2.Procedure<TEnvironment>

org.apache.hadoop.hbase.procedure2.StateMachineProcedure<MasterProcedureEnv,TState>

org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure<TState>

org.apache.hadoop.hbase.master.procedure.AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure

All Implemented Interfaces:

Comparable<Procedure<MasterProcedureEnv>>, TableProcedureInterface

@Private
public class SplitTableRegionProcedure
extends AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

The procedure to split a region in a table. Takes lock on the parent region. It holds the lock for the life of the procedure.

Throws exception on construction if determines context hostile to spllt (cluster going down or master is shutting down or table is disabled).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private class	SplitTableRegionProcedure.StoreFileSplitter	Utility class used to do the file splitting / reference writing in parallel instead of sequentially.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

StateMachineProcedure.Flow

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.procedure2.Procedure

Procedure.LockState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

TableProcedureInterface.TableOperationType

Field Summary

Fields

Modifier and Type	Field	Description
private byte[]	bestSplitRow
private RegionInfo	daughterOneRI
private RegionInfo	daughterTwoRI
private static final RegionState.State[]	EXPECTED_SPLIT_STATES
private static final org.slf4j.Logger	LOG
private RegionSplitPolicy	splitPolicy

Fields inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

stateCount

Fields inherited from class org.apache.hadoop.hbase.procedure2.Procedure

NO_PROC_ID, NO_TIMEOUT

Fields inherited from interface org.apache.hadoop.hbase.master.procedure.TableProcedureInterface

DUMMY_NAMESPACE_TABLE_NAME

Constructor Summary

Constructors

Constructor	Description
SplitTableRegionProcedure()
SplitTableRegionProcedure(MasterProcedureEnv env, RegionInfo regionToSplit, byte[] splitRow)

Method Summary

Modifier and Type	Method	Description
protected boolean	abort(MasterProcedureEnv env)	The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable.
protected Procedure.LockState	acquireLock(MasterProcedureEnv env)	The user should override this method if they need a lock on an Entity.
private void	assertSplitResultFilesCount(org.apache.hadoop.fs.FileSystem fs, int expectedSplitResultFileCount, org.apache.hadoop.fs.Path dir)
private void	checkClosedRegions(MasterProcedureEnv env)
private void	checkSplittable(MasterProcedureEnv env, RegionInfo regionToSplit)	Check whether the region is splittable
private TransitRegionStateProcedure[]	createAssignProcedures(MasterProcedureEnv env)
void	createDaughterRegions(MasterProcedureEnv env)	Create daughter regions
private TransitRegionStateProcedure[]	createUnassignProcedures(MasterProcedureEnv env)
private void	deleteDaughterRegions(MasterProcedureEnv env)
protected void	deserializeStateData(ProcedureStateSerializer serializer)	Called on store load to allow the user to decode the previously serialized state.
protected StateMachineProcedure.Flow	executeFromState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)	called to perform a single step of the specified 'state' of the procedure
RegionInfo	getDaughterOneRI()
private static long	getDaughterRegionIdTimestamp(RegionInfo hri)	Calculate daughter regionid to use.
RegionInfo	getDaughterTwoRI()
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState	getInitialState()	Return the initial state object that will be used for the first call to executeFromState().
private RegionInfo	getParentRegion()
private ServerName	getParentRegionServerName(MasterProcedureEnv env)
protected ProcedureMetrics	getProcedureMetrics(MasterProcedureEnv env)	Override this method to provide procedure specific counters for submitted count, failed count and time histogram.
private int	getRegionReplication(MasterProcedureEnv env)
private byte[]	getSplitRow()
protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState	getState(int stateId)	Convert an ordinal (or state id) to an Enum (or more descriptive) state object.
protected int	getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)	Convert the Enum (or more descriptive) state object to an ordinal (or state id).
TableProcedureInterface.TableOperationType	getTableOperationType()	Given an operation type we can take decisions about what to do with pending operations.
private boolean	hasBestSplitRow()
protected boolean	isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)	Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.
private void	openParentRegion(MasterProcedureEnv env)	Rollback close parent region
private void	postRollBackSplitRegion(MasterProcedureEnv env)	Action after rollback a split table region action.
private void	postSplitRegion(MasterProcedureEnv env)	Post split region actions
boolean	prepareSplitRegion(MasterProcedureEnv env)	Prepare to Split region.
private void	preSplitRegion(MasterProcedureEnv env)	Action before splitting region in a table.
private void	preSplitRegionAfterMETA(MasterProcedureEnv env)	Pre split region actions after the Point-of-No-Return step
private void	preSplitRegionBeforeMETA(MasterProcedureEnv env)	Post split region actions before the Point-of-No-Return step
protected void	releaseLock(MasterProcedureEnv env)	The user should override this method, and release lock if necessary.
private void	removeNonDefaultReplicas(MasterProcedureEnv env)
private void	rollbackPrepareSplit(MasterProcedureEnv env)	Rollback prepare split region
protected void	rollbackState(MasterProcedureEnv env, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)	To rollback SplitTableRegionProcedure, an AssignProcedure is asynchronously submitted for parent region to be split (rollback doesn't wait on the completion of the AssignProcedure) .
protected void	serializeStateData(ProcedureStateSerializer serializer)	The user-level code of the procedure may have some state to persist (e.g.
private Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path>	splitStoreFile(HRegionFileSystem regionFs, StoreFileTracker tracker, byte[] family, HStoreFile sf)
private Pair<List<org.apache.hadoop.fs.Path>,List<org.apache.hadoop.fs.Path>>	splitStoreFiles(MasterProcedureEnv env, HRegionFileSystem regionFs)	Create Split directory
void	toStringClassDetails(StringBuilder sb)	Extend the toString() information with the procedure details e.g.
private void	updateMeta(MasterProcedureEnv env)	Add daughter regions to META
private void	writeMaxSequenceIdFile(MasterProcedureEnv env)

Methods inherited from class org.apache.hadoop.hbase.master.procedure.AbstractStateMachineRegionProcedure

getRegion, getTableName, holdLock, setFailure, setRegion

Methods inherited from class org.apache.hadoop.hbase.master.procedure.AbstractStateMachineTableProcedure

checkOnline, checkTableModifiable, getUser, getWALRegionDir, isTableEnabled, preflightChecks, releaseSyncLatch, setUser, waitInitialized

Methods inherited from class org.apache.hadoop.hbase.procedure2.StateMachineProcedure

addChildProcedure, execute, failIfAborted, getCurrentState, getCurrentStateId, getCycles, isEofState, isRollbackSupported, isYieldAfterExecutionStep, isYieldBeforeExecuteFromState, rollback, setNextState, toStringState

Methods inherited from class org.apache.hadoop.hbase.procedure2.Procedure

addStackIndex, afterReplay, beforeReplay, bypass, compareTo, completionCleanup, doExecute, doRollback, elapsedTime, getChildrenLatch, getException, getLastUpdate, getNonceKey, getOwner, getParentProcId, getProcId, getProcIdHashCode, getProcName, getResult, getRootProcedureId, getRootProcId, getStackIndexes, getState, getSubmittedTime, getTimeout, getTimeoutTimestamp, hasChildren, hasException, hasLock, hasOwner, hasParent, hasTimeout, haveSameParent, incChildrenLatch, isBypass, isFailed, isFinished, isInitializing, isLockedWhenLoading, isRunnable, isSuccess, isWaiting, removeStackIndex, setAbortFailure, setChildrenLatch, setExecuted, setFailure, setFailure, setLastUpdate, setNonceKey, setOwner, setOwner, setParentProcId, setProcId, setResult, setRootProcId, setStackIndexes, setState, setSubmittedTime, setTimeout, setTimeoutFailure, shouldWaitClientAck, skipPersistence, suspend, toString, toStringClass, toStringDetails, toStringSimpleSB, updateMetricsOnFinish, updateMetricsOnSubmit, updateTimestamp, wasExecuted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

daughterOneRI

private RegionInfo daughterOneRI

daughterTwoRI

private RegionInfo daughterTwoRI

bestSplitRow

private byte[] bestSplitRow

splitPolicy

private RegionSplitPolicy splitPolicy

EXPECTED_SPLIT_STATES

private static final RegionState.State[] EXPECTED_SPLIT_STATES

Constructor Details

SplitTableRegionProcedure

public SplitTableRegionProcedure()

SplitTableRegionProcedure

public SplitTableRegionProcedure(MasterProcedureEnv env,
 RegionInfo regionToSplit,
 byte[] splitRow)
                          throws IOException

Throws:

IOException

Method Details

acquireLock

protected Procedure.LockState acquireLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method if they need a lock on an Entity. A lock can be anything, and it is up to the implementor. The Procedure Framework will call this method just before it invokes Procedure.execute(Object). It calls Procedure.releaseLock(Object) after the call to execute.

If you need to hold the lock for the life of the Procedure -- i.e. you do not want any other Procedure interfering while this Procedure is running, see Procedure.holdLock(Object).

Example: in our Master we can execute request in parallel for different tables. We can create t1 and create t2 and these creates can be executed at the same time. Anything else on t1/t2 is queued waiting that specific table create to happen.

There are 3 LockState:

• LOCK_ACQUIRED should be returned when the proc has the lock and the proc is ready to execute.
• LOCK_YIELD_WAIT should be returned when the proc has not the lock and the framework should take care of readding the procedure back to the runnable set for retry
• LOCK_EVENT_WAIT should be returned when the proc has not the lock and someone will take care of readding the procedure back to the runnable set when the lock is available.

Overrides:

acquireLock in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Returns:

the lock state as described above.

releaseLock

protected void releaseLock(MasterProcedureEnv env)

Description copied from class: Procedure

The user should override this method, and release lock if necessary.

Overrides:

releaseLock in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

getDaughterOneRI

public RegionInfo getDaughterOneRI()

getDaughterTwoRI

public RegionInfo getDaughterTwoRI()

hasBestSplitRow

private boolean hasBestSplitRow()

checkSplittable

private void checkSplittable(MasterProcedureEnv env,
 RegionInfo regionToSplit)
                      throws IOException

Check whether the region is splittable

Parameters:

env - MasterProcedureEnv

regionToSplit - parent Region to be split

Throws:

IOException

getDaughterRegionIdTimestamp

private static long getDaughterRegionIdTimestamp(RegionInfo hri)

Calculate daughter regionid to use.

Parameters:

hri - Parent RegionInfo

Returns:

Daughter region id (timestamp) to use.

removeNonDefaultReplicas

private void removeNonDefaultReplicas(MasterProcedureEnv env)
                               throws IOException

Throws:

IOException

checkClosedRegions

private void checkClosedRegions(MasterProcedureEnv env)
                         throws IOException

Throws:

IOException

executeFromState

protected StateMachineProcedure.Flow executeFromState(MasterProcedureEnv env,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)
                                               throws InterruptedException

Description copied from class: StateMachineProcedure

called to perform a single step of the specified 'state' of the procedure

Specified by:

executeFromState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

state - state to execute

Returns:

Flow.NO_MORE_STATE if the procedure is completed, Flow.HAS_MORE_STATE if there is another step.

Throws:

InterruptedException

rollbackState

protected void rollbackState(MasterProcedureEnv env,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)
                      throws IOException,
InterruptedException

To rollback SplitTableRegionProcedure, an AssignProcedure is asynchronously submitted for parent region to be split (rollback doesn't wait on the completion of the AssignProcedure) . This can be improved by changing rollback() to support sub-procedures. See HBASE-19851 for details.

Specified by:

rollbackState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

state - state to rollback

Throws:

IOException - temporary failure, the rollback will retry later

InterruptedException

isRollbackSupported

protected boolean isRollbackSupported(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)

Description copied from class: StateMachineProcedure

Used by the default implementation of abort() to know if the current state can be aborted and rollback can be triggered.

Overrides:

isRollbackSupported in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

getState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState getState(int stateId)

Description copied from class: StateMachineProcedure

Convert an ordinal (or state id) to an Enum (or more descriptive) state object.

Specified by:

getState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

stateId - the ordinal() of the state enum (or state id)

Returns:

the state enum object

getStateId

protected int getStateId(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState state)

Description copied from class: StateMachineProcedure

Convert the Enum (or more descriptive) state object to an ordinal (or state id).

Specified by:

getStateId in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

state - the state enum object

Returns:

stateId the ordinal() of the state enum (or state id)

getInitialState

protected org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState getInitialState()

Description copied from class: StateMachineProcedure

Return the initial state object that will be used for the first call to executeFromState().

Specified by:

getInitialState in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Returns:

the initial state enum object

serializeStateData

protected void serializeStateData(ProcedureStateSerializer serializer)
                           throws IOException

Description copied from class: Procedure

The user-level code of the procedure may have some state to persist (e.g. input arguments or current position in the processing state) to be able to resume on failure.

Overrides:

serializeStateData in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

serializer - stores the serializable state

Throws:

IOException

deserializeStateData

protected void deserializeStateData(ProcedureStateSerializer serializer)
                             throws IOException

Description copied from class: Procedure

Called on store load to allow the user to decode the previously serialized state.

Overrides:

deserializeStateData in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

serializer - contains the serialized state

Throws:

IOException

toStringClassDetails

public void toStringClassDetails(StringBuilder sb)

Description copied from class: Procedure

Extend the toString() information with the procedure details e.g. className and parameters

Overrides:

toStringClassDetails in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Parameters:

sb - the string builder to use to append the proc specific information

getParentRegion

private RegionInfo getParentRegion()

getTableOperationType

public TableProcedureInterface.TableOperationType getTableOperationType()

Description copied from interface: TableProcedureInterface

Given an operation type we can take decisions about what to do with pending operations. e.g. if we get a delete and we have some table operation pending (e.g. add column) we can abort those operations.

Specified by:

getTableOperationType in interface TableProcedureInterface

Specified by:

getTableOperationType in class AbstractStateMachineRegionProcedure<org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>

Returns:

the operation type that the procedure is executing.

getProcedureMetrics

protected ProcedureMetrics getProcedureMetrics(MasterProcedureEnv env)

Description copied from class: Procedure

Override this method to provide procedure specific counters for submitted count, failed count and time histogram.

Overrides:

getProcedureMetrics in class Procedure<MasterProcedureEnv>

Parameters:

env - The environment passed to the procedure executor

Returns:

Container object for procedure related metric

getSplitRow

private byte[] getSplitRow()

prepareSplitRegion

public boolean prepareSplitRegion(MasterProcedureEnv env)
                           throws IOException

Prepare to Split region.

Parameters:

env - MasterProcedureEnv

Throws:

IOException

rollbackPrepareSplit

private void rollbackPrepareSplit(MasterProcedureEnv env)

Rollback prepare split region

Parameters:

env - MasterProcedureEnv

preSplitRegion

private void preSplitRegion(MasterProcedureEnv env)
                     throws IOException,
InterruptedException

Action before splitting region in a table.

Parameters:

env - MasterProcedureEnv

Throws:

IOException

InterruptedException

postRollBackSplitRegion

private void postRollBackSplitRegion(MasterProcedureEnv env)
                              throws IOException

Action after rollback a split table region action.

Parameters:

env - MasterProcedureEnv

Throws:

IOException

openParentRegion

private void openParentRegion(MasterProcedureEnv env)
                       throws IOException

Rollback close parent region

Throws:

IOException

createDaughterRegions

public void createDaughterRegions(MasterProcedureEnv env)
                           throws IOException

Create daughter regions

Throws:

IOException

deleteDaughterRegions

private void deleteDaughterRegions(MasterProcedureEnv env)
                            throws IOException

Throws:

IOException

splitStoreFiles

private Pair<List<org.apache.hadoop.fs.Path>,List<org.apache.hadoop.fs.Path>> splitStoreFiles(MasterProcedureEnv env,
 HRegionFileSystem regionFs)
                                                                                       throws IOException

Create Split directory

Parameters:

env - MasterProcedureEnv

Throws:

IOException

assertSplitResultFilesCount

private void assertSplitResultFilesCount(org.apache.hadoop.fs.FileSystem fs,
 int expectedSplitResultFileCount,
 org.apache.hadoop.fs.Path dir)
                                  throws IOException

Throws:

IOException

splitStoreFile

private Pair<org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path> splitStoreFile(HRegionFileSystem regionFs,
 StoreFileTracker tracker,
 byte[] family,
 HStoreFile sf)
                                                                          throws IOException

Throws:

IOException

preSplitRegionBeforeMETA

private void preSplitRegionBeforeMETA(MasterProcedureEnv env)
                               throws IOException,
InterruptedException

Post split region actions before the Point-of-No-Return step

Parameters:

env - MasterProcedureEnv

Throws:

IOException

InterruptedException

updateMeta

private void updateMeta(MasterProcedureEnv env)
                 throws IOException

Add daughter regions to META

Parameters:

env - MasterProcedureEnv

Throws:

IOException

preSplitRegionAfterMETA

private void preSplitRegionAfterMETA(MasterProcedureEnv env)
                              throws IOException,
InterruptedException

Pre split region actions after the Point-of-No-Return step

Parameters:

env - MasterProcedureEnv

Throws:

IOException

InterruptedException

postSplitRegion

private void postSplitRegion(MasterProcedureEnv env)
                      throws IOException

Post split region actions

Parameters:

env - MasterProcedureEnv

Throws:

IOException

getParentRegionServerName

private ServerName getParentRegionServerName(MasterProcedureEnv env)

createUnassignProcedures

private TransitRegionStateProcedure[] createUnassignProcedures(MasterProcedureEnv env)
                                                        throws IOException

Throws:

IOException

createAssignProcedures

private TransitRegionStateProcedure[] createAssignProcedures(MasterProcedureEnv env)
                                                      throws IOException

Throws:

IOException

getRegionReplication

private int getRegionReplication(MasterProcedureEnv env)
                          throws IOException

Throws:

IOException

writeMaxSequenceIdFile

private void writeMaxSequenceIdFile(MasterProcedureEnv env)
                             throws IOException

Throws:

IOException

abort

protected boolean abort(MasterProcedureEnv env)

Description copied from class: Procedure

The abort() call is asynchronous and each procedure must decide how to deal with it, if they want to be abortable. The simplest implementation is to have an AtomicBoolean set in the abort() method and then the execute() will check if the abort flag is set or not. abort() may be called multiple times from the client, so the implementation must be idempotent.

NOTE: abort() is not like Thread.interrupt(). It is just a notification that allows the procedure implementor abort.

Overrides:

abort in class StateMachineProcedure<MasterProcedureEnv,org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos.SplitTableRegionState>