Package org.apache.hadoop.hbase.regionserver

Class HRegion.BatchOperation<T>

java.lang.Object

org.apache.hadoop.hbase.regionserver.HRegion.BatchOperation<T>

Direct Known Subclasses:

HRegion.MutationBatchOperation, HRegion.ReplayBatchOperation

Enclosing class:

HRegion

private abstract static class HRegion.BatchOperation<T>
extends Object

Class that tracks the progress of a batch operations, accumulating status codes and tracking the index at which processing is proceeding. These batch operations may get split into mini-batches for processing.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static interface	HRegion.BatchOperation.Visitor	Visitor interface for batch operations

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	atomic
protected Durability	durability
protected final Map<byte[],List<ExtendedCell>>[]	familyCellMaps
protected int	nextIndexToProcess
protected final HRegion.ObservedExceptionsInBatch	observedExceptions
protected final T[]	operations
protected final HRegion	region
protected final Result[]	results
protected final OperationStatus[]	retCodeDetails
protected final WALEdit[]	walEditsFromCoprocessors

Constructor Summary

Constructors

Constructor	Description
BatchOperation(HRegion region, T[] operations)

Method Summary

Modifier and Type	Method	Description
protected void	addNonSkipWALMutationsToWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, List<ExtendedCell> cellsFromCP, Map<byte[],List<ExtendedCell>> familyCellMap)
protected void	applyFamilyMapToMemStore(Map<byte[],List<ExtendedCell>> familyMap, MemStoreSizing memstoreAccounting)	Atomically apply the given map of family->edits to the memstore.
List<Pair<NonceKey,WALEdit>>	buildWALEdits(MiniBatchOperationInProgress<Mutation> miniBatchOp)	Builds separate WALEdit per nonce by applying input mutations.
protected abstract void	cacheSkipWALMutationForRegionReplication(MiniBatchOperationInProgress<Mutation> miniBatchOp, List<Pair<NonceKey,WALEdit>> walEdits, Map<byte[],List<ExtendedCell>> familyCellMap)
abstract void	checkAndPrepare()	Validates each mutation and prepares a batch for write.
protected void	checkAndPrepareMutation(int index, long timestamp)
protected void	checkAndPrepareMutation(Mutation mutation, long timestamp)	Helper method that checks and prepares only one mutation.
protected abstract void	checkAndPreparePut(Put p)	Implement any Put request specific check and prepare logic here.
abstract void	closeRegionOperation()
void	completeMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry)	This method completes mini-batch operations by calling postBatchMutate() CP hook (if required) and completing mvcc.
protected MiniBatchOperationInProgress<Mutation>	createMiniBatch(int lastIndexExclusive, int readyToWriteCount)
protected WALEdit	createWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp)
protected static void	doAddCellsToWALEdit(WALEdit walEdit, List<ExtendedCell> cellsFromCP, Map<byte[],List<ExtendedCell>> familyCellMap)
private void	doFinishHotnessProtector(MiniBatchOperationInProgress<Mutation> miniBatchOp)
void	doPostOpCleanupForMiniBatch(MiniBatchOperationInProgress<Mutation> miniBatchOp, WALEdit walEdit, boolean success)
List<UUID>	getClusterIds()
abstract Mutation	getMutation(int index)
abstract Mutation[]	getMutationsForCoprocs()	This method is potentially expensive and useful mostly for non-replay CP path.
abstract long	getNonce(int index)
abstract long	getNonceGroup(int index)
abstract long	getOrigLogSeqNum()
(package private) boolean	isAtomic()
boolean	isDone()
abstract boolean	isInReplay()
boolean	isOperationPending(int index)
MiniBatchOperationInProgress<Mutation>	lockRowsAndBuildMiniBatch(List<Region.RowLock> acquiredRowLocks)	Creates Mini-batch of all operations [nextIndexToProcess, lastIndexExclusive) for which a row lock can be acquired.
abstract void	prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp, long timestamp, List<Region.RowLock> acquiredRowLocks)	If necessary, calls preBatchMutate() CP hook for a mini-batch and updates metrics, cell count, tags and timestamp for all cells of all operations in a mini-batch.
int	size()
abstract void	startRegionOperation()
void	visitBatchOperations(boolean pendingOnly, int lastIndexExclusive, HRegion.BatchOperation.Visitor visitor)	Helper method for visiting pending/ all batch operations
protected void	writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp, long writeNumber)
abstract MultiVersionConcurrencyControl.WriteEntry	writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp, MultiVersionConcurrencyControl.WriteEntry writeEntry, long now)	Write mini-batch operations to MemStore

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

operations

protected final T[] operations

retCodeDetails

protected final OperationStatus[] retCodeDetails

walEditsFromCoprocessors

protected final WALEdit[] walEditsFromCoprocessors

familyCellMaps

protected final Map<byte[],List<ExtendedCell>>[] familyCellMaps

results

protected final Result[] results

region

protected final HRegion region

nextIndexToProcess

protected int nextIndexToProcess

observedExceptions

protected final HRegion.ObservedExceptionsInBatch observedExceptions

durability

protected Durability durability

atomic

protected boolean atomic

Constructor Details

BatchOperation

public BatchOperation(HRegion region,
 T[] operations)

Method Details

visitBatchOperations

public void visitBatchOperations(boolean pendingOnly,
 int lastIndexExclusive,
 HRegion.BatchOperation.Visitor visitor)
                          throws IOException

Helper method for visiting pending/ all batch operations

Throws:

IOException

getMutation

public abstract Mutation getMutation(int index)

getNonceGroup

public abstract long getNonceGroup(int index)

getNonce

public abstract long getNonce(int index)

getMutationsForCoprocs

public abstract Mutation[] getMutationsForCoprocs()

This method is potentially expensive and useful mostly for non-replay CP path.

isInReplay

public abstract boolean isInReplay()

getOrigLogSeqNum

public abstract long getOrigLogSeqNum()

startRegionOperation

public abstract void startRegionOperation()
                                   throws IOException

Throws:

IOException

closeRegionOperation

public abstract void closeRegionOperation()
                                   throws IOException

Throws:

IOException

checkAndPrepare

public abstract void checkAndPrepare()
                              throws IOException

Validates each mutation and prepares a batch for write. If necessary (non-replay case), runs CP prePut()/preDelete()/preIncrement()/preAppend() hooks for all mutations in a batch. This is intended to operate on entire batch and will be called from outside of class to check and prepare batch. This can be implemented by calling helper method checkAndPrepareMutation(int, long) in a 'for' loop over mutations.

Throws:

IOException

checkAndPreparePut

protected abstract void checkAndPreparePut(Put p)
                                    throws IOException

Implement any Put request specific check and prepare logic here. Please refer to checkAndPrepareMutation(Mutation, long) for how its used.

Throws:

IOException

prepareMiniBatchOperations

public abstract void prepareMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 long timestamp,
 List<Region.RowLock> acquiredRowLocks)
                                         throws IOException

If necessary, calls preBatchMutate() CP hook for a mini-batch and updates metrics, cell count, tags and timestamp for all cells of all operations in a mini-batch.

Throws:

IOException

writeMiniBatchOperationsToMemStore

public abstract MultiVersionConcurrencyControl.WriteEntry writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 MultiVersionConcurrencyControl.WriteEntry writeEntry,
 long now)
                                                                                      throws IOException

Write mini-batch operations to MemStore

Throws:

IOException

writeMiniBatchOperationsToMemStore

protected void writeMiniBatchOperationsToMemStore(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 long writeNumber)
                                           throws IOException

Throws:

IOException

isDone

public boolean isDone()

size

public int size()

isOperationPending

public boolean isOperationPending(int index)

getClusterIds

public List<UUID> getClusterIds()

isAtomic

boolean isAtomic()

checkAndPrepareMutation

protected void checkAndPrepareMutation(Mutation mutation,
 long timestamp)
                                throws IOException

Helper method that checks and prepares only one mutation. This can be used to implement checkAndPrepare() for entire Batch. NOTE: As CP prePut()/preDelete()/preIncrement()/preAppend() hooks may modify mutations, this method should be called after prePut()/preDelete()/preIncrement()/preAppend() CP hooks are run for the mutation

Throws:

IOException

checkAndPrepareMutation

protected void checkAndPrepareMutation(int index,
 long timestamp)
                                throws IOException

Throws:

IOException

lockRowsAndBuildMiniBatch

public MiniBatchOperationInProgress<Mutation> lockRowsAndBuildMiniBatch(List<Region.RowLock> acquiredRowLocks)
                                                                 throws IOException

Creates Mini-batch of all operations [nextIndexToProcess, lastIndexExclusive) for which a row lock can be acquired. All mutations with locked rows are considered to be In-progress operations and hence the name MiniBatchOperationInProgress. Mini batch is window over HRegion.BatchOperation and contains contiguous pending operations.

Parameters:

acquiredRowLocks - keeps track of rowLocks acquired.

Throws:

IOException

createMiniBatch

protected MiniBatchOperationInProgress<Mutation> createMiniBatch(int lastIndexExclusive,
 int readyToWriteCount)

createWALEdit

protected WALEdit createWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp)

buildWALEdits

public List<Pair<NonceKey,WALEdit>> buildWALEdits(MiniBatchOperationInProgress<Mutation> miniBatchOp)
                                           throws IOException

Builds separate WALEdit per nonce by applying input mutations. If WALEdits from CP are present, they are merged to result WALEdit.

Throws:

IOException

addNonSkipWALMutationsToWALEdit

protected void addNonSkipWALMutationsToWALEdit(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 WALEdit walEdit,
 List<ExtendedCell> cellsFromCP,
 Map<byte[],List<ExtendedCell>> familyCellMap)

doAddCellsToWALEdit

protected static void doAddCellsToWALEdit(WALEdit walEdit,
 List<ExtendedCell> cellsFromCP,
 Map<byte[],List<ExtendedCell>> familyCellMap)

cacheSkipWALMutationForRegionReplication

protected abstract void cacheSkipWALMutationForRegionReplication(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 List<Pair<NonceKey,WALEdit>> walEdits,
 Map<byte[],List<ExtendedCell>> familyCellMap)

completeMiniBatchOperations

public void completeMiniBatchOperations(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 MultiVersionConcurrencyControl.WriteEntry writeEntry)
                                 throws IOException

This method completes mini-batch operations by calling postBatchMutate() CP hook (if required) and completing mvcc.

Throws:

IOException

doPostOpCleanupForMiniBatch

public void doPostOpCleanupForMiniBatch(MiniBatchOperationInProgress<Mutation> miniBatchOp,
 WALEdit walEdit,
 boolean success)
                                 throws IOException

Throws:

IOException

doFinishHotnessProtector

private void doFinishHotnessProtector(MiniBatchOperationInProgress<Mutation> miniBatchOp)

applyFamilyMapToMemStore

protected void applyFamilyMapToMemStore(Map<byte[],List<ExtendedCell>> familyMap,
 MemStoreSizing memstoreAccounting)

Atomically apply the given map of family->edits to the memstore. This handles the consistency control on its own, but the caller should already have locked updatesLock.readLock(). This also does not check the families for validity.

Parameters:

familyMap - Map of Cells by family