Package org.apache.hadoop.hbase.coprocessor.example.row.stats

Class RowStatisticsCompactionObserver

java.lang.Object

org.apache.hadoop.hbase.coprocessor.example.row.stats.RowStatisticsCompactionObserver

All Implemented Interfaces:

Coprocessor, MasterCoprocessor, MasterObserver, RegionCoprocessor, RegionObserver

@Private
public class RowStatisticsCompactionObserver
extends Object
implements RegionCoprocessor, RegionObserver, MasterCoprocessor, MasterObserver

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	RowStatisticsCompactionObserver.RowStatisticsScanner

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.Coprocessor

Coprocessor.State

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

RegionObserver.MutationType

Field Summary

Fields

Modifier and Type	Field	Description
private static final long	DEFAULT_MAX_BUCKET_SIZE
private static final org.slf4j.Logger	LOG
private long	maxCacheSize
private final RowStatisticsRecorder	recorder
private static final String	ROW_STATISTICS_DROPPED
private static final String	ROW_STATISTICS_PUT_FAILED
private Counter	rowStatisticsDropped
private Counter	rowStatisticsPutFailed
private static final ConcurrentMap<TableName,Long>	TABLE_COUNTERS

Fields inherited from interface org.apache.hadoop.hbase.Coprocessor

PRIORITY_HIGHEST, PRIORITY_LOWEST, PRIORITY_SYSTEM, PRIORITY_USER, VERSION

Constructor Summary

Constructors

Constructor	Description
RowStatisticsCompactionObserver()
RowStatisticsCompactionObserver(RowStatisticsRecorder recorder)

Method Summary

Modifier and Type	Method	Description
Optional<MasterObserver>	getMasterObserver()
Optional<RegionObserver>	getRegionObserver()
void	postStartMaster(ObserverContext<MasterCoprocessorEnvironment> ctx)	Called immediately after an active master instance has completed initialization.
InternalScanner	preCompact(ObserverContext<? extends RegionCoprocessorEnvironment> context, Store store, InternalScanner scanner, ScanType scanType, CompactionLifeCycleTracker tracker, CompactionRequest request)	Called prior to writing the StoreFiles selected for compaction into a new StoreFile.
void	start(CoprocessorEnvironment e)	Called by the CoprocessorEnvironment during it's own startup to initialize the coprocessor.
void	stop(CoprocessorEnvironment e)	Called by the CoprocessorEnvironment during it's own shutdown to stop the coprocessor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.Coprocessor

getServices

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.MasterObserver

postAbortProcedure, postAddReplicationPeer, postAddRSGroup, postAssign, postBalance, postBalanceRSGroup, postBalanceSwitch, postClearDeadServers, postCloneSnapshot, postCompletedCreateTableAction, postCompletedDeleteTableAction, postCompletedDisableTableAction, postCompletedEnableTableAction, postCompletedMergeRegionsAction, postCompletedModifyTableAction, postCompletedSnapshotAction, postCompletedSplitRegionAction, postCompletedTruncateTableAction, postCreateNamespace, postCreateTable, postDecommissionRegionServers, postDeleteNamespace, postDeleteSnapshot, postDeleteTable, postDisableReplicationPeer, postDisableTable, postEnableReplicationPeer, postEnableTable, postGetClusterMetrics, postGetConfiguredNamespacesAndTablesInRSGroup, postGetLocks, postGetNamespaceDescriptor, postGetProcedures, postGetReplicationPeerConfig, postGetRSGroupInfo, postGetRSGroupInfoOfServer, postGetRSGroupInfoOfTable, postGetTableDescriptors, postGetTableNames, postGetUserPermissions, postGrant, postHasUserPermissions, postIsRpcThrottleEnabled, postListDecommissionedRegionServers, postListNamespaceDescriptors, postListNamespaces, postListReplicationPeers, postListRSGroups, postListSnapshot, postListTablesInRSGroup, postLockHeartbeat, postMasterStoreFlush, postMergeRegions, postMergeRegionsCommitAction, postModifyColumnFamilyStoreFileTracker, postModifyNamespace, postModifyTable, postModifyTableStoreFileTracker, postMove, postMoveServers, postMoveServersAndTables, postMoveTables, postRecommissionRegionServer, postRegionOffline, postRemoveReplicationPeer, postRemoveRSGroup, postRemoveServers, postRenameRSGroup, postRequestLock, postRestoreSnapshot, postRevoke, postRollBackMergeRegionsAction, postRollBackSplitRegionAction, postSetNamespaceQuota, postSetRegionServerQuota, postSetSplitOrMergeEnabled, postSetTableQuota, postSetUserQuota, postSetUserQuota, postSetUserQuota, postSnapshot, postSwitchExceedThrottleQuota, postSwitchRpcThrottle, postTableFlush, postTransitReplicationPeerSyncReplicationState, postTruncateRegion, postTruncateRegionAction, postTruncateTable, postUnassign, postUpdateMasterConfiguration, postUpdateReplicationPeerConfig, postUpdateRSGroupConfig, preAbortProcedure, preAddReplicationPeer, preAddRSGroup, preAssign, preBalance, preBalanceRSGroup, preBalanceSwitch, preClearDeadServers, preCloneSnapshot, preCreateNamespace, preCreateTable, preCreateTableAction, preCreateTableRegionsInfos, preDecommissionRegionServers, preDeleteNamespace, preDeleteSnapshot, preDeleteTable, preDeleteTableAction, preDisableReplicationPeer, preDisableTable, preDisableTableAction, preEnableReplicationPeer, preEnableTable, preEnableTableAction, preGetClusterMetrics, preGetConfiguredNamespacesAndTablesInRSGroup, preGetLocks, preGetNamespaceDescriptor, preGetProcedures, preGetReplicationPeerConfig, preGetRSGroupInfo, preGetRSGroupInfoOfServer, preGetRSGroupInfoOfTable, preGetTableDescriptors, preGetTableNames, preGetUserPermissions, preGrant, preHasUserPermissions, preIsRpcThrottleEnabled, preListDecommissionedRegionServers, preListNamespaceDescriptors, preListNamespaces, preListReplicationPeers, preListRSGroups, preListSnapshot, preListTablesInRSGroup, preLockHeartbeat, preMasterInitialization, preMasterStoreFlush, preMergeRegions, preMergeRegionsAction, preMergeRegionsCommitAction, preModifyColumnFamilyStoreFileTracker, preModifyNamespace, preModifyTable, preModifyTableAction, preModifyTableStoreFileTracker, preMove, preMoveServers, preMoveServersAndTables, preMoveTables, preRecommissionRegionServer, preRegionOffline, preRemoveReplicationPeer, preRemoveRSGroup, preRemoveServers, preRenameRSGroup, preRequestLock, preRestoreSnapshot, preRevoke, preSetNamespaceQuota, preSetRegionServerQuota, preSetSplitOrMergeEnabled, preSetTableQuota, preSetUserQuota, preSetUserQuota, preSetUserQuota, preShutdown, preSnapshot, preSplitRegion, preSplitRegionAction, preSplitRegionAfterMETAAction, preSplitRegionBeforeMETAAction, preStopMaster, preSwitchExceedThrottleQuota, preSwitchRpcThrottle, preTableFlush, preTransitReplicationPeerSyncReplicationState, preTruncateRegion, preTruncateRegionAction, preTruncateTable, preTruncateTableAction, preUnassign, preUpdateMasterConfiguration, preUpdateReplicationPeerConfig, preUpdateRSGroupConfig

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.RegionCoprocessor

getBulkLoadObserver, getEndpointObserver

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

postAppend, postAppend, postAppendBeforeWAL, postBatchMutate, postBatchMutateIndispensably, postBulkLoadHFile, postCheckAndDelete, postCheckAndDelete, postCheckAndMutate, postCheckAndPut, postCheckAndPut, postClose, postCloseRegionOperation, postCommitStoreFile, postCompact, postCompactSelection, postDelete, postDelete, postExists, postFlush, postFlush, postGetOp, postIncrement, postIncrement, postIncrementBeforeWAL, postInstantiateDeleteTracker, postMemStoreCompaction, postMutationBeforeWAL, postOpen, postPut, postPut, postReplayWALs, postScannerClose, postScannerFilterRow, postScannerNext, postScannerOpen, postStartRegionOperation, postStoreFileReaderOpen, preAppend, preAppend, preAppendAfterRowLock, preBatchMutate, preBulkLoadHFile, preCheckAndDelete, preCheckAndDelete, preCheckAndDeleteAfterRowLock, preCheckAndDeleteAfterRowLock, preCheckAndMutate, preCheckAndMutateAfterRowLock, preCheckAndPut, preCheckAndPut, preCheckAndPutAfterRowLock, preCheckAndPutAfterRowLock, preClose, preCommitStoreFile, preCompactScannerOpen, preCompactSelection, preDelete, preDelete, preExists, preFlush, preFlush, preFlushScannerOpen, preGetOp, preIncrement, preIncrement, preIncrementAfterRowLock, preMemStoreCompaction, preMemStoreCompactionCompact, preMemStoreCompactionCompactScannerOpen, preOpen, prePrepareTimeStampForDeleteVersion, prePut, prePut, preReplayWALs, preScannerClose, preScannerNext, preScannerOpen, preStoreFileReaderOpen, preStoreScannerOpen, preWALAppend

Field Details

LOG

private static final org.slf4j.Logger LOG

DEFAULT_MAX_BUCKET_SIZE

private static final long DEFAULT_MAX_BUCKET_SIZE

See Also:

• Constant Field Values

TABLE_COUNTERS

private static final ConcurrentMap<TableName,Long> TABLE_COUNTERS

ROW_STATISTICS_DROPPED

private static final String ROW_STATISTICS_DROPPED

See Also:

• Constant Field Values

ROW_STATISTICS_PUT_FAILED

private static final String ROW_STATISTICS_PUT_FAILED

See Also:

• Constant Field Values

rowStatisticsDropped

private Counter rowStatisticsDropped

rowStatisticsPutFailed

private Counter rowStatisticsPutFailed

maxCacheSize

private long maxCacheSize

recorder

private final RowStatisticsRecorder recorder

Constructor Details

RowStatisticsCompactionObserver

@Private
public RowStatisticsCompactionObserver(RowStatisticsRecorder recorder)

RowStatisticsCompactionObserver

public RowStatisticsCompactionObserver()

Method Details

getRegionObserver

public Optional<RegionObserver> getRegionObserver()

Specified by:

getRegionObserver in interface RegionCoprocessor

getMasterObserver

public Optional<MasterObserver> getMasterObserver()

Specified by:

getMasterObserver in interface MasterCoprocessor

start

public void start(CoprocessorEnvironment e)
           throws IOException

Description copied from interface: Coprocessor

Called by the CoprocessorEnvironment during it's own startup to initialize the coprocessor.

Specified by:

start in interface Coprocessor

Throws:

IOException

stop

public void stop(CoprocessorEnvironment e)
          throws IOException

Description copied from interface: Coprocessor

Called by the CoprocessorEnvironment during it's own shutdown to stop the coprocessor.

Specified by:

stop in interface Coprocessor

Throws:

IOException

postStartMaster

public void postStartMaster(ObserverContext<MasterCoprocessorEnvironment> ctx)
                     throws IOException

Description copied from interface: MasterObserver

Called immediately after an active master instance has completed initialization. Will not be called on standby master instances unless they take over the active role.

Specified by:

postStartMaster in interface MasterObserver

Throws:

IOException

preCompact

public InternalScanner preCompact(ObserverContext<? extends RegionCoprocessorEnvironment> context,
 Store store,
 InternalScanner scanner,
 ScanType scanType,
 CompactionLifeCycleTracker tracker,
 CompactionRequest request)

Description copied from interface: RegionObserver

Called prior to writing the StoreFiles selected for compaction into a new StoreFile.

To override or modify the compaction process, implementing classes can wrap the provided InternalScanner with a custom implementation that is returned from this method. The custom scanner can then inspect Cells from the wrapped scanner, applying its own policy to what gets written.

If implementations are wrapping the passed in InternalScanner, they can also have their implementation implement Shipper and delegate to the original scanner. This will cause compactions to free up memory as they progress, which is especially important for people using off-heap memory pools.

Keep in mind that when Shipper.shipped() is called, any cell references you maintain in your implementation may get corrupted. As such you should make sure to deep clone any cells that you need to keep reference to across invocations of shipped.

Specified by:

preCompact in interface RegionObserver

Parameters:

context - the environment provided by the region server

store - the store being compacted

scanner - the scanner over existing data used in the store file rewriting

scanType - type of Scan

tracker - tracker used to track the life cycle of a compaction

request - the requested compaction

Returns:

the scanner to use during compaction. Should not be null unless the implementation is writing new store files on its own.