Package org.apache.hadoop.hbase.master.janitor

Class CatalogJanitor

java.lang.Object

org.apache.hadoop.hbase.ScheduledChore

org.apache.hadoop.hbase.master.janitor.CatalogJanitor

All Implemented Interfaces:

Runnable

@Private
public class CatalogJanitor
extends ScheduledChore

A janitor for the catalog tables. Scans the hbase:meta catalog table on a period. Makes a lastReport on state of hbase:meta. Looks for unused regions to garbage collect. Scan of hbase:meta runs if we are NOT in maintenance mode, if we are NOT shutting down, AND if the assignmentmanager is loaded. Playing it safe, we will garbage collect no-longer needed region references only if there are no regions-in-transition (RIT).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static class	CatalogJanitor.SplitParentFirstComparator	Compare HRegionInfos in a way that has split parents sort BEFORE their daughters.

Field Summary

Fields

Modifier and Type	Field	Description
private final AtomicBoolean	alreadyRunning
static final int	DEFAULT_HBASE_CATALOGJANITOR_INTERVAL
private final AtomicBoolean	enabled
private CatalogJanitorReport	lastReport	Saved report from last hbase:meta scan to completion.
private static final org.slf4j.Logger	LOG
private final MasterServices	services

Constructor Summary

Constructors

Constructor	Description
CatalogJanitor(MasterServices services)

Method Summary

Modifier and Type	Method	Description
private static void	checkLog4jProperties()
private static Pair<Boolean,Boolean>	checkRegionReferences(MasterServices services, TableName tableName, RegionInfo region)	Checks if a region still holds references to parent.
protected void	chore()	The task to execute on each scheduled execution of the Chore
(package private) static boolean	cleanMergeRegion(MasterServices services, RegionInfo mergedRegion, List<RegionInfo> parents)	If merged region no longer holds reference to the merge regions, archive merge region on hdfs and perform deleting references in hbase:meta
private boolean	cleanParent(RegionInfo parent, Result rowContent)	If daughters no longer hold reference to the parents, delete the parent.
(package private) static boolean	cleanParent(MasterServices services, RegionInfo parent, Result rowContent)
boolean	getEnabled()
CatalogJanitorReport	getLastReport()	Returns Returns last published Report that comes of last successful scan of hbase:meta.
private static boolean	hasNoReferences(Pair<Boolean,Boolean> p)
protected boolean	initialChore()	Override to run a task before we start looping.
private static boolean	isMetaLoaded(AssignmentManager am)
private static boolean	isRIT(AssignmentManager am)
static void	main(String[] args)	For testing against a cluster.
int	scan()	Run janitorial scan of catalog hbase:meta table looking for garbage to collect.
protected CatalogJanitorReport	scanForReport()	Scan hbase:meta.
boolean	setEnabled(boolean enabled)
private void	updateAssignmentManagerMetrics()

Methods inherited from class org.apache.hadoop.hbase.ScheduledChore

cancel, cancel, choreForTesting, cleanup, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, isInitialChoreComplete, isScheduled, run, shutdown, shutdown, toString, triggerNow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

DEFAULT_HBASE_CATALOGJANITOR_INTERVAL

public static final int DEFAULT_HBASE_CATALOGJANITOR_INTERVAL

See Also:

• Constant Field Values

LOG

private static final org.slf4j.Logger LOG

alreadyRunning

private final AtomicBoolean alreadyRunning

enabled

private final AtomicBoolean enabled

services

private final MasterServices services

lastReport

private volatile CatalogJanitorReport lastReport

Saved report from last hbase:meta scan to completion. May be stale if having trouble completing scan. Check its date.

Constructor Details

CatalogJanitor

public CatalogJanitor(MasterServices services)

Method Details

initialChore

protected boolean initialChore()

Description copied from class: ScheduledChore

Override to run a task before we start looping.

Overrides:

initialChore in class ScheduledChore

Returns:

true if initial chore was successful

setEnabled

public boolean setEnabled(boolean enabled)

getEnabled

public boolean getEnabled()

chore

protected void chore()

Description copied from class: ScheduledChore

The task to execute on each scheduled execution of the Chore

Specified by:

chore in class ScheduledChore

isMetaLoaded

private static boolean isMetaLoaded(AssignmentManager am)

isRIT

private static boolean isRIT(AssignmentManager am)

scan

public int scan()
         throws IOException

Run janitorial scan of catalog hbase:meta table looking for garbage to collect.

Returns:

How many items gc'd whether for merge or split. Returns -1 if previous scan is in progress.

Throws:

IOException

scanForReport

protected CatalogJanitorReport scanForReport()
                                      throws IOException

Scan hbase:meta.

Returns:

Return generated CatalogJanitorReport

Throws:

IOException

getLastReport

public CatalogJanitorReport getLastReport()

Returns Returns last published Report that comes of last successful scan of hbase:meta.

cleanMergeRegion

static boolean cleanMergeRegion(MasterServices services,
 RegionInfo mergedRegion,
 List<RegionInfo> parents)
                         throws IOException

If merged region no longer holds reference to the merge regions, archive merge region on hdfs and perform deleting references in hbase:meta

Returns:

true if we delete references in merged region on hbase:meta and archive the files on the file system

Throws:

IOException

cleanParent

static boolean cleanParent(MasterServices services,
 RegionInfo parent,
 Result rowContent)
                    throws IOException

Throws:

IOException

cleanParent

private boolean cleanParent(RegionInfo parent,
 Result rowContent)
                     throws IOException

If daughters no longer hold reference to the parents, delete the parent.

Parameters:

parent - RegionInfo of split offlined parent

rowContent - Content of parent row in metaRegionName

Returns:

True if we removed parent from meta table and from the filesystem.

Throws:

IOException

hasNoReferences

private static boolean hasNoReferences(Pair<Boolean,Boolean> p)

Parameters:

p - A pair where the first boolean says whether or not the daughter region directory exists in the filesystem and then the second boolean says whether the daughter has references to the parent.

Returns:

True the passed p signifies no references.

checkRegionReferences

private static Pair<Boolean,Boolean> checkRegionReferences(MasterServices services,
 TableName tableName,
 RegionInfo region)
                                                    throws IOException

Checks if a region still holds references to parent.

Parameters:

tableName - The table for the region

region - The region to check

Returns:

A pair where the first boolean says whether the region directory exists in the filesystem and then the second boolean says whether the region has references to a parent.

Throws:

IOException

updateAssignmentManagerMetrics

private void updateAssignmentManagerMetrics()

checkLog4jProperties

private static void checkLog4jProperties()

main

public static void main(String[] args)
                 throws IOException

For testing against a cluster. Doesn't have a MasterServices context so does not report on good vs bad servers.

Throws:

IOException