Package org.apache.hadoop.hbase.master.normalizer

Class SimpleRegionNormalizer

java.lang.Object

org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, ConfigurationObserver, RegionNormalizer

@LimitedPrivate("Configuration")
class SimpleRegionNormalizer
extends Object
implements RegionNormalizer, ConfigurationObserver

Simple implementation of region normalizer. Logic in use:

1. Get all regions of a given table
2. Get avg size S of the regions in the table (by total size of store files reported in RegionMetrics)
3. For each region R0, if R0 is bigger than S * 2, it is kindly requested to split.
4. Otherwise, for the next region in the chain R1, if R0 + R1 is smaller then S, R0 and R1 are kindly requested to merge.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private class	SimpleRegionNormalizer.NormalizeContext	Inner class caries the state necessary to perform a single invocation of computePlansForTable(TableDescriptor).
private static final class	SimpleRegionNormalizer.NormalizerConfiguration	Holds the configuration values read from Configuration.

Field Summary

Fields

Modifier and Type	Field	Description
(package private) static final boolean	DEFAULT_MERGE_ENABLED
(package private) static final int	DEFAULT_MERGE_MIN_REGION_AGE_DAYS
(package private) static final int	DEFAULT_MERGE_MIN_REGION_COUNT
(package private) static final int	DEFAULT_MERGE_MIN_REGION_SIZE_MB
(package private) static final long	DEFAULT_MERGE_REQUEST_MAX_NUMBER_OF_REGIONS_COUNT
(package private) static final boolean	DEFAULT_SPLIT_ENABLED
private static final org.slf4j.Logger	LOG
private MasterServices	masterServices
(package private) static final String	MERGE_ENABLED_KEY
(package private) static final String	MERGE_MIN_REGION_AGE_DAYS_KEY
(package private) static final String	MERGE_MIN_REGION_COUNT_KEY
(package private) static final String	MERGE_MIN_REGION_SIZE_MB_KEY
(package private) static final String	MERGE_REQUEST_MAX_NUMBER_OF_REGIONS_COUNT_KEY
(package private) static final String	MIN_REGION_COUNT_KEY	Deprecated. since 2.5.0 and will be removed in 4.0.0.
private SimpleRegionNormalizer.NormalizerConfiguration	normalizerConfiguration
(package private) static final String	SPLIT_ENABLED_KEY

Constructor Summary

Constructors

Constructor	Description
SimpleRegionNormalizer()

Method Summary

Modifier and Type	Method	Description
private List<NormalizationPlan>	computeMergeNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx)	Computes the merge plans that should be executed for this table to converge average region towards target average or target region count.
List<NormalizationPlan>	computePlansForTable(TableDescriptor tableDescriptor)	Computes a list of normalizer actions to perform on the target table.
private List<NormalizationPlan>	computeSplitNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx)	Computes the split plans that should be executed for this table to converge average region size towards target average or target region count.
private double	getAverageRegionSizeMb(List<RegionInfo> tableRegions, TableDescriptor tableDescriptor)	Also make sure tableRegions contains regions of the same table
org.apache.hadoop.conf.Configuration	getConf()
Period	getMergeMinRegionAge()	Return this instance's configured value for "hbase.normalizer.merge.min_region_age.days".
int	getMergeMinRegionCount()	Return this instance's configured value for "hbase.normalizer.merge.min.region.count".
long	getMergeMinRegionSizeMb()	Return this instance's configured value for "hbase.normalizer.merge.min_region_size.mb".
long	getMergeRequestMaxNumberOfRegionsCount()
private long	getRegionSizeMB(RegionInfo hri)	Returns size of region in MB and if region is not found than -1
private boolean	isLargeEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration, SimpleRegionNormalizer.NormalizeContext ctx, RegionInfo regionInfo)	Return true when regionInfo has a size that is sufficient to be considered for a merge operation, false otherwise.
private boolean	isMasterSwitchEnabled(MasterSwitchType masterSwitchType)
boolean	isMergeEnabled()	Return this instance's configured value for "hbase.normalizer.merge.enabled".
private static boolean	isOldEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration, SimpleRegionNormalizer.NormalizeContext ctx, RegionInfo regionInfo)	Return true when regionInfo has a creation date that is old enough to be considered for a merge operation, false otherwise.
boolean	isSplitEnabled()	Return this instance's configured value for "hbase.normalizer.split.enabled".
private static <T> void	logConfigurationUpdated(String key, T oldValue, T newValue)
private static boolean	logTraceReason(BooleanSupplier predicate, String fmtWhenTrue, Object... args)
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf)	This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
private static Period	parseMergeMinRegionAge(org.apache.hadoop.conf.Configuration conf)
private static int	parseMergeMinRegionCount(org.apache.hadoop.conf.Configuration conf)
private static long	parseMergeMinRegionSizeMb(org.apache.hadoop.conf.Configuration conf)
private static long	parseMergeRequestMaxNumberOfRegionsCount(org.apache.hadoop.conf.Configuration conf)
private boolean	proceedWithMergePlanning(TableDescriptor tableDescriptor)
private boolean	proceedWithSplitPlanning(TableDescriptor tableDescriptor)
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setMasterServices(MasterServices masterServices)	Set the master service.
(package private) void	shuffleNormalizationPlans(List<NormalizationPlan> plans)	This very simple method exists so we can verify it was called in a unit test.
private boolean	skipForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration, SimpleRegionNormalizer.NormalizeContext ctx, RegionInfo regionInfo)	Determine if a RegionInfo should be considered for a merge operation.
private static boolean	skipForSplit(RegionState state, RegionInfo regionInfo)	Determine if a region in RegionState should be considered for a split operation.
private static <T> void	warnInvalidValue(String key, T parsedValue, T settledValue)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

SPLIT_ENABLED_KEY

static final String SPLIT_ENABLED_KEY

See Also:

• Constant Field Values

DEFAULT_SPLIT_ENABLED

static final boolean DEFAULT_SPLIT_ENABLED

See Also:

• Constant Field Values

MERGE_ENABLED_KEY

static final String MERGE_ENABLED_KEY

See Also:

• Constant Field Values

DEFAULT_MERGE_ENABLED

static final boolean DEFAULT_MERGE_ENABLED

See Also:

• Constant Field Values

MIN_REGION_COUNT_KEY

@Deprecated
static final String MIN_REGION_COUNT_KEY

Deprecated.

since 2.5.0 and will be removed in 4.0.0. Use MERGE_MIN_REGION_COUNT_KEY instead.

See Also:

• HBASE-25745
• Constant Field Values

MERGE_MIN_REGION_COUNT_KEY

static final String MERGE_MIN_REGION_COUNT_KEY

See Also:

• Constant Field Values

DEFAULT_MERGE_MIN_REGION_COUNT

static final int DEFAULT_MERGE_MIN_REGION_COUNT

See Also:

• Constant Field Values

MERGE_MIN_REGION_AGE_DAYS_KEY

static final String MERGE_MIN_REGION_AGE_DAYS_KEY

See Also:

• Constant Field Values

DEFAULT_MERGE_MIN_REGION_AGE_DAYS

static final int DEFAULT_MERGE_MIN_REGION_AGE_DAYS

See Also:

• Constant Field Values

MERGE_MIN_REGION_SIZE_MB_KEY

static final String MERGE_MIN_REGION_SIZE_MB_KEY

See Also:

• Constant Field Values

DEFAULT_MERGE_MIN_REGION_SIZE_MB

static final int DEFAULT_MERGE_MIN_REGION_SIZE_MB

See Also:

• Constant Field Values

MERGE_REQUEST_MAX_NUMBER_OF_REGIONS_COUNT_KEY

static final String MERGE_REQUEST_MAX_NUMBER_OF_REGIONS_COUNT_KEY

See Also:

• Constant Field Values

DEFAULT_MERGE_REQUEST_MAX_NUMBER_OF_REGIONS_COUNT

static final long DEFAULT_MERGE_REQUEST_MAX_NUMBER_OF_REGIONS_COUNT

See Also:

• Constant Field Values

masterServices

private MasterServices masterServices

normalizerConfiguration

private SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration

Constructor Details

SimpleRegionNormalizer

public SimpleRegionNormalizer()

Method Details

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

parseMergeMinRegionCount

private static int parseMergeMinRegionCount(org.apache.hadoop.conf.Configuration conf)

parseMergeMinRegionAge

private static Period parseMergeMinRegionAge(org.apache.hadoop.conf.Configuration conf)

parseMergeMinRegionSizeMb

private static long parseMergeMinRegionSizeMb(org.apache.hadoop.conf.Configuration conf)

parseMergeRequestMaxNumberOfRegionsCount

private static long parseMergeRequestMaxNumberOfRegionsCount(org.apache.hadoop.conf.Configuration conf)

warnInvalidValue

private static <T> void warnInvalidValue(String key,
 T parsedValue,
 T settledValue)

logConfigurationUpdated

private static <T> void logConfigurationUpdated(String key,
 T oldValue,
 T newValue)

isSplitEnabled

public boolean isSplitEnabled()

Return this instance's configured value for "hbase.normalizer.split.enabled".

isMergeEnabled

public boolean isMergeEnabled()

Return this instance's configured value for "hbase.normalizer.merge.enabled".

getMergeMinRegionCount

public int getMergeMinRegionCount()

Return this instance's configured value for "hbase.normalizer.merge.min.region.count".

getMergeMinRegionAge

public Period getMergeMinRegionAge()

Return this instance's configured value for "hbase.normalizer.merge.min_region_age.days".

getMergeMinRegionSizeMb

public long getMergeMinRegionSizeMb()

Return this instance's configured value for "hbase.normalizer.merge.min_region_size.mb".

getMergeRequestMaxNumberOfRegionsCount

public long getMergeRequestMaxNumberOfRegionsCount()

setMasterServices

public void setMasterServices(MasterServices masterServices)

Description copied from interface: RegionNormalizer

Set the master service. Must be called before first call to RegionNormalizer.computePlansForTable(TableDescriptor).

Specified by:

setMasterServices in interface RegionNormalizer

Parameters:

masterServices - master services to use

computePlansForTable

public List<NormalizationPlan> computePlansForTable(TableDescriptor tableDescriptor)

Description copied from interface: RegionNormalizer

Computes a list of normalizer actions to perform on the target table. This is the primary entry-point from the Master driving a normalization activity.

Specified by:

computePlansForTable in interface RegionNormalizer

Parameters:

tableDescriptor - table descriptor for table which needs normalize

Returns:

A list of the normalization actions to perform, or an empty list if there's nothing to do.

getRegionSizeMB

private long getRegionSizeMB(RegionInfo hri)

Returns size of region in MB and if region is not found than -1

isMasterSwitchEnabled

private boolean isMasterSwitchEnabled(MasterSwitchType masterSwitchType)

proceedWithSplitPlanning

private boolean proceedWithSplitPlanning(TableDescriptor tableDescriptor)

proceedWithMergePlanning

private boolean proceedWithMergePlanning(TableDescriptor tableDescriptor)

getAverageRegionSizeMb

private double getAverageRegionSizeMb(List<RegionInfo> tableRegions,
 TableDescriptor tableDescriptor)

Also make sure tableRegions contains regions of the same table

Parameters:

tableRegions - regions of table to normalize

tableDescriptor - the TableDescriptor

Returns:

average region size depending on

See Also:

• TableDescriptor.getNormalizerTargetRegionCount()

skipForMerge

private boolean skipForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration,
 SimpleRegionNormalizer.NormalizeContext ctx,
 RegionInfo regionInfo)

Determine if a RegionInfo should be considered for a merge operation.

Callers beware: for safe concurrency, be sure to pass in the local instance of SimpleRegionNormalizer.NormalizerConfiguration, don't use this's instance.

computeMergeNormalizationPlans

private List<NormalizationPlan> computeMergeNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx)

Computes the merge plans that should be executed for this table to converge average region towards target average or target region count.

skipForSplit

private static boolean skipForSplit(RegionState state,
 RegionInfo regionInfo)

Determine if a region in RegionState should be considered for a split operation.

computeSplitNormalizationPlans

private List<NormalizationPlan> computeSplitNormalizationPlans(SimpleRegionNormalizer.NormalizeContext ctx)

Computes the split plans that should be executed for this table to converge average region size towards target average or target region count.
if the region is > 2 times larger than average, we split it. split is more high priority normalization action than merge.

isOldEnoughForMerge

private static boolean isOldEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration,
 SimpleRegionNormalizer.NormalizeContext ctx,
 RegionInfo regionInfo)

Return true when regionInfo has a creation date that is old enough to be considered for a merge operation, false otherwise.

isLargeEnoughForMerge

private boolean isLargeEnoughForMerge(SimpleRegionNormalizer.NormalizerConfiguration normalizerConfiguration,
 SimpleRegionNormalizer.NormalizeContext ctx,
 RegionInfo regionInfo)

Return true when regionInfo has a size that is sufficient to be considered for a merge operation, false otherwise.

Callers beware: for safe concurrency, be sure to pass in the local instance of SimpleRegionNormalizer.NormalizerConfiguration, don't use this's instance.

shuffleNormalizationPlans

void shuffleNormalizationPlans(List<NormalizationPlan> plans)

This very simple method exists so we can verify it was called in a unit test. Visible for testing.

logTraceReason

private static boolean logTraceReason(BooleanSupplier predicate,
 String fmtWhenTrue,
 Object... args)