Package org.apache.hadoop.hbase.quotas

Class MasterQuotaManager

java.lang.Object

org.apache.hadoop.hbase.quotas.MasterQuotaManager

All Implemented Interfaces:

RegionStateListener

@Private
@Evolving
public class MasterQuotaManager
extends Object
implements RegionStateListener

Master Quota Manager. It is responsible for initialize the quota table on the first-run and provide the admin operations to interact with the quota table. TODO: FUTURE: The master will be responsible to notify each RS of quota changes and it will do the "quota aggregation" when the QuotaScope is CLUSTER.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	MasterQuotaManager.NamedLock<T>
private static interface	MasterQuotaManager.SetQuotaOperations	Encapsulates CRUD quota operations for some subject.
private static class	MasterQuotaManager.SizeSnapshotWithTimestamp	Holds the size of a region at the given time, millis since the epoch.

Field Summary

Fields

Modifier and Type	Field	Description
private static final Map<RegionInfo,Long>	EMPTY_MAP
private boolean	initialized
private static final org.slf4j.Logger	LOG
private final MasterServices	masterServices
private MasterQuotaManager.NamedLock<String>	namespaceLocks
private NamespaceAuditor	namespaceQuotaManager
private MasterQuotaManager.NamedLock<String>	regionServerLocks
private ConcurrentHashMap<RegionInfo,MasterQuotaManager.SizeSnapshotWithTimestamp>	regionSizes
private RpcThrottleStorage	rpcThrottleStorage
private MasterQuotaManager.NamedLock<TableName>	tableLocks
private MasterQuotaManager.NamedLock<String>	userLocks

Constructor Summary

Constructors

Constructor	Description
MasterQuotaManager(MasterServices masterServices)

Method Summary

Modifier and Type	Method	Description
void	addRegionSize(RegionInfo hri, long size, long time)
void	checkAndUpdateNamespaceRegionQuota(TableName tName, int regions)
void	checkNamespaceTableAndRegionQuota(TableName tName, int regions)
private void	checkQuotaSupport()
private void	createQuotaTable()
NamespaceAuditor	getNamespaceQuotaManager()
int	getRegionCountOfTable(TableName tName)	Returns cached region count, or -1 if quota manager is disabled or table status not found
(package private) void	initializeRegionSizes()
boolean	isExceedThrottleQuotaEnabled()
private boolean	isInViolationAndPolicyDisable(TableName tableName, QuotaObserverChore quotaObserverChore)	Method to check if a table is in violation and policy set on table is DISABLE.
boolean	isQuotaInitialized()
boolean	isRpcThrottleEnabled()
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsRpcThrottleEnabledResponse	isRpcThrottleEnabled(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsRpcThrottleEnabledRequest request)
void	onRegionMerged(RegionInfo mergedRegion)	Process region merge event.
void	onRegionSplit(RegionInfo hri)	Process region split event.
void	onRegionSplitReverted(RegionInfo hri)	Process region split reverted event.
void	processFileArchivals(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.FileArchiveNotificationRequest request, Connection conn, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs)
(package private) int	pruneEntriesOlderThan(long timeToPruneBefore, QuotaObserverChore quotaObserverChore)
void	removeNamespaceQuota(String namespace)
void	removeRegionSizesForTable(TableName tableName)	Removes each region size entry where the RegionInfo references the provided TableName.
void	removeTableFromNamespaceQuota(TableName tName)	Remove table from namespace quota.
void	setNamespaceQuota(String namespace, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
void	setNamespaceQuota(NamespaceDescriptor desc)
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaResponse	setQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
private void	setQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req, MasterQuotaManager.SetQuotaOperations quotaOps)
void	setRegionServerQuota(String regionServer, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
void	setTableQuota(TableName table, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
void	setUserQuota(String userName, String namespace, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
void	setUserQuota(String userName, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
void	setUserQuota(String userName, TableName table, org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
Map<RegionInfo,Long>	snapshotRegionSizes()
void	start()
void	stop()
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchExceedThrottleQuotaResponse	switchExceedThrottleQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchExceedThrottleQuotaRequest request)
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchRpcThrottleResponse	switchRpcThrottle(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchRpcThrottleRequest request)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

EMPTY_MAP

private static final Map<RegionInfo,Long> EMPTY_MAP

masterServices

private final MasterServices masterServices

namespaceLocks

private MasterQuotaManager.NamedLock<String> namespaceLocks

tableLocks

private MasterQuotaManager.NamedLock<TableName> tableLocks

userLocks

private MasterQuotaManager.NamedLock<String> userLocks

regionServerLocks

private MasterQuotaManager.NamedLock<String> regionServerLocks

initialized

private boolean initialized

namespaceQuotaManager

private NamespaceAuditor namespaceQuotaManager

regionSizes

private ConcurrentHashMap<RegionInfo,MasterQuotaManager.SizeSnapshotWithTimestamp> regionSizes

rpcThrottleStorage

private RpcThrottleStorage rpcThrottleStorage

Constructor Details

MasterQuotaManager

public MasterQuotaManager(MasterServices masterServices)

Method Details

start

public void start()
           throws IOException

Throws:

IOException

stop

public void stop()

isQuotaInitialized

public boolean isQuotaInitialized()

setQuota

public org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaResponse setQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                                                                                         throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setUserQuota

public void setUserQuota(String userName,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                  throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setUserQuota

public void setUserQuota(String userName,
 TableName table,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                  throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setUserQuota

public void setUserQuota(String userName,
 String namespace,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                  throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setTableQuota

public void setTableQuota(TableName table,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                   throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setNamespaceQuota

public void setNamespaceQuota(String namespace,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                       throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setRegionServerQuota

public void setRegionServerQuota(String regionServer,
 org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req)
                          throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

setNamespaceQuota

public void setNamespaceQuota(NamespaceDescriptor desc)
                       throws IOException

Throws:

IOException

removeNamespaceQuota

public void removeNamespaceQuota(String namespace)
                          throws IOException

Throws:

IOException

switchRpcThrottle

public org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchRpcThrottleResponse switchRpcThrottle(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchRpcThrottleRequest request)
                                                                                                           throws IOException

Throws:

IOException

isRpcThrottleEnabled

public org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsRpcThrottleEnabledResponse isRpcThrottleEnabled(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsRpcThrottleEnabledRequest request)
                                                                                                                 throws IOException

Throws:

IOException

isRpcThrottleEnabled

public boolean isRpcThrottleEnabled()
                             throws IOException

Throws:

IOException

switchExceedThrottleQuota

public org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchExceedThrottleQuotaResponse switchExceedThrottleQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SwitchExceedThrottleQuotaRequest request)
                                                                                                                           throws IOException

Throws:

IOException

isExceedThrottleQuotaEnabled

public boolean isExceedThrottleQuotaEnabled()
                                     throws IOException

Throws:

IOException

setQuota

private void setQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest req,
 MasterQuotaManager.SetQuotaOperations quotaOps)
               throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

checkNamespaceTableAndRegionQuota

public void checkNamespaceTableAndRegionQuota(TableName tName,
 int regions)
                                       throws IOException

Throws:

IOException

checkAndUpdateNamespaceRegionQuota

public void checkAndUpdateNamespaceRegionQuota(TableName tName,
 int regions)
                                        throws IOException

Throws:

IOException

getRegionCountOfTable

public int getRegionCountOfTable(TableName tName)
                          throws IOException

Returns cached region count, or -1 if quota manager is disabled or table status not found

Throws:

IOException

onRegionMerged

public void onRegionMerged(RegionInfo mergedRegion)
                    throws IOException

Description copied from interface: RegionStateListener

Process region merge event.

Specified by:

onRegionMerged in interface RegionStateListener

Throws:

IOException

onRegionSplit

public void onRegionSplit(RegionInfo hri)
                   throws IOException

Description copied from interface: RegionStateListener

Process region split event.

Specified by:

onRegionSplit in interface RegionStateListener

Parameters:

hri - An instance of RegionInfo

Throws:

IOException

removeTableFromNamespaceQuota

public void removeTableFromNamespaceQuota(TableName tName)
                                   throws IOException

Remove table from namespace quota.

Parameters:

tName - - The table name to update quota usage.

Throws:

IOException - Signals that an I/O exception has occurred.

getNamespaceQuotaManager

public NamespaceAuditor getNamespaceQuotaManager()

checkQuotaSupport

private void checkQuotaSupport()
                        throws IOException

Throws:

IOException

createQuotaTable

private void createQuotaTable()
                       throws IOException

Throws:

IOException

onRegionSplitReverted

public void onRegionSplitReverted(RegionInfo hri)
                           throws IOException

Description copied from interface: RegionStateListener

Process region split reverted event.

Specified by:

onRegionSplitReverted in interface RegionStateListener

Parameters:

hri - An instance of RegionInfo

Throws:

IOException - Signals that an I/O exception has occurred.

initializeRegionSizes

void initializeRegionSizes()

addRegionSize

public void addRegionSize(RegionInfo hri,
 long size,
 long time)

snapshotRegionSizes

public Map<RegionInfo,Long> snapshotRegionSizes()

pruneEntriesOlderThan

int pruneEntriesOlderThan(long timeToPruneBefore,
 QuotaObserverChore quotaObserverChore)

isInViolationAndPolicyDisable

private boolean isInViolationAndPolicyDisable(TableName tableName,
 QuotaObserverChore quotaObserverChore)

Method to check if a table is in violation and policy set on table is DISABLE.

Parameters:

tableName - tableName to check.

quotaObserverChore - QuotaObserverChore instance

Returns:

returns true if table is in violation and policy is disable else false.

removeRegionSizesForTable

public void removeRegionSizesForTable(TableName tableName)

Removes each region size entry where the RegionInfo references the provided TableName.

Parameters:

tableName - tableName.

processFileArchivals

public void processFileArchivals(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.FileArchiveNotificationRequest request,
 Connection conn,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.FileSystem fs)
                          throws IOException

Throws:

IOException