Package org.apache.hadoop.hbase.master.cleaner

Class LogCleaner

java.lang.Object

org.apache.hadoop.hbase.ScheduledChore

org.apache.hadoop.hbase.master.cleaner.CleanerChore<BaseLogCleanerDelegate>

org.apache.hadoop.hbase.master.cleaner.LogCleaner

All Implemented Interfaces:

Runnable, ConfigurationObserver

@Private
public class LogCleaner
extends CleanerChore<BaseLogCleanerDelegate>
implements ConfigurationObserver

This Chore, every time it runs, will attempt to delete the WALs and Procedure WALs in the old logs folder. The WAL is only deleted if none of the cleaner delegates says otherwise.

See Also:

• BaseLogCleanerDelegate

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static final class	LogCleaner.CleanerContext

Field Summary

Fields

Modifier and Type	Field	Description
private long	cleanerThreadTimeoutMsec
static final int	DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE
(package private) static final long	DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC
private static final org.slf4j.Logger	LOG
static final String	OLD_WALS_CLEANER_THREAD_SIZE
static final String	OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC
private List<Thread>	oldWALsCleaner
private final LinkedBlockingQueue<LogCleaner.CleanerContext>	pendingDelete

Fields inherited from class org.apache.hadoop.hbase.master.cleaner.CleanerChore

CHORE_POOL_SIZE, cleanersChain, DEFAULT_CHORE_POOL_SIZE, DEFAULT_LOG_CLEANER_CHORE_DIRECTORY_SORTING, DEFAULT_LOG_CLEANER_CHORE_POOL_SIZE, excludeDirs, fs, LOG_CLEANER_CHORE_DIRECTORY_SORTING, LOG_CLEANER_CHORE_SIZE, params

Constructor Summary

Constructors

Constructor	Description
LogCleaner(int period, Stoppable stopper, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path oldLogDir, DirScanPool pool, Map<String,Object> params)

Method Summary

Modifier and Type	Method	Description
void	cancel(boolean mayInterruptIfRunning)
void	cleanup()	Override to run cleanup tasks when the Chore encounters an error and must stop running
private List<Thread>	createOldWalsCleaner(int size)
private void	deleteFile()
protected int	deleteFiles(Iterable<org.apache.hadoop.fs.FileStatus> filesToDelete)	Delete the given files
(package private) long	getCleanerThreadTimeoutMsec()
(package private) int	getSizeOfCleaners()
private void	interruptOldWALsCleaner()
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf)	This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
protected boolean	validate(org.apache.hadoop.fs.Path file)	Validate the file to see if it even belongs in the directory.

Methods inherited from class org.apache.hadoop.hbase.master.cleaner.CleanerChore

calculatePoolSize, chore, getChorePoolSize, getEnabled, initialChore, setEnabled, triggerCleanerNow

Methods inherited from class org.apache.hadoop.hbase.ScheduledChore

cancel, choreForTesting, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, isInitialChoreComplete, isScheduled, run, shutdown, shutdown, toString, triggerNow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

OLD_WALS_CLEANER_THREAD_SIZE

public static final String OLD_WALS_CLEANER_THREAD_SIZE

See Also:

• Constant Field Values

DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE

public static final int DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE

See Also:

• Constant Field Values

OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC

public static final String OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC

See Also:

• Constant Field Values

DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC

static final long DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC

See Also:

• Constant Field Values

pendingDelete

private final LinkedBlockingQueue<LogCleaner.CleanerContext> pendingDelete

oldWALsCleaner

private List<Thread> oldWALsCleaner

cleanerThreadTimeoutMsec

private long cleanerThreadTimeoutMsec

Constructor Details

LogCleaner

public LogCleaner(int period,
 Stoppable stopper,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path oldLogDir,
 DirScanPool pool,
 Map<String,Object> params)

Parameters:

period - the period of time to sleep between each run

stopper - the stopper

conf - configuration to use

fs - handle to the FS

oldLogDir - the path to the archived logs

pool - the thread pool used to scan directories

Method Details

validate

protected boolean validate(org.apache.hadoop.fs.Path file)

Description copied from class: CleanerChore

Validate the file to see if it even belongs in the directory. If it is valid, then the file will go through the cleaner delegates, but otherwise the file is just deleted.

Specified by:

validate in class CleanerChore<BaseLogCleanerDelegate>

Parameters:

file - full Path of the file to be checked

Returns:

true if the file is valid, false otherwise

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

deleteFiles

protected int deleteFiles(Iterable<org.apache.hadoop.fs.FileStatus> filesToDelete)

Description copied from class: CleanerChore

Delete the given files

Overrides:

deleteFiles in class CleanerChore<BaseLogCleanerDelegate>

Parameters:

filesToDelete - files to delete

Returns:

number of deleted files

cleanup

public void cleanup()

Description copied from class: ScheduledChore

Override to run cleanup tasks when the Chore encounters an error and must stop running

Overrides:

cleanup in class CleanerChore<BaseLogCleanerDelegate>

getSizeOfCleaners

int getSizeOfCleaners()

getCleanerThreadTimeoutMsec

long getCleanerThreadTimeoutMsec()

createOldWalsCleaner

private List<Thread> createOldWalsCleaner(int size)

interruptOldWALsCleaner

private void interruptOldWALsCleaner()

deleteFile

private void deleteFile()

cancel

public void cancel(boolean mayInterruptIfRunning)

Overrides:

cancel in class ScheduledChore