Package org.apache.hadoop.hbase

Class IntegrationTestBackupRestore

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestBackupRestore

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class IntegrationTestBackupRestore
extends IntegrationTestBase

An integration test to detect regressions in HBASE-7912. Create a table with many regions, load data, perform series backup/load operations, then restore and verify data

See Also:

• HBASE-7912
• HBASE-14123

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
private static String	BACKUP_ROOT_DIR
private static final String	CLASS_NAME
protected static final String	COLUMN_NAME
protected static final int	DEFAULT_NUM_ITERATIONS
protected static final int	DEFAULT_NUMBER_OF_TABLES
protected static final int	DEFAULT_REGION_COUNT
protected static final int	DEFAULT_REGIONSERVER_COUNT
protected static final int	DEFAULT_ROWS_IN_ITERATION
protected static Object	lock
protected static final org.slf4j.Logger	LOG
protected static final String	NUM_ITERATIONS_KEY
protected static final String	NUMBER_OF_TABLES_KEY
protected static int	numIterations
protected static int	numTables
protected static final String	REGION_COUNT_KEY
protected static int	regionsCountPerServer
protected static final String	REGIONSERVER_COUNT_KEY
protected static int	regionServerCount
protected static final String	ROWS_PER_ITERATION_KEY
protected static int	rowsInIteration
protected static final long	SLEEP_TIME_DEFAULT
protected static final String	SLEEP_TIME_KEY
protected long	sleepTime
protected static org.apache.hadoop.hbase.TableName[]	tableNames

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp, util

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
IntegrationTestBackupRestore()

Method Summary

Modifier and Type	Method	Description
protected void	addOptions()
private String[]	allIncremental(List<String> backupIds)
private String	backup(org.apache.hadoop.hbase.backup.BackupRequest request, org.apache.hadoop.hbase.backup.BackupAdmin client)
protected boolean	checkSucceeded(String backupId)	Check if backup is succeeded
private void	cleanUpBackupDir()
org.apache.hadoop.hbase.backup.RestoreRequest	createRestoreRequest(String backupRootDir, String backupId, boolean check, org.apache.hadoop.hbase.TableName[] fromTables, org.apache.hadoop.hbase.TableName[] toTables, boolean isOverwrite)	Get restore request.
private void	createTable(org.apache.hadoop.hbase.TableName tableName)
private void	createTables()
private void	deleteTablesIfAny()
private void	enableBackup(org.apache.hadoop.conf.Configuration conf)
private org.apache.hadoop.hbase.backup.BackupInfo	getBackupInfo(String backupId)
protected Set<String>	getColumnFamilies()	Provides the name of the CFs that are protected from random Chaos monkey activity (alter)
org.apache.hadoop.hbase.TableName	getTablename()	Provides the name of the table that is protected from random Chaos monkey activity
private void	loadData(org.apache.hadoop.hbase.TableName table, int numRows)
static void	main(String[] args)	Main method
private void	merge(String[] backupIds, org.apache.hadoop.hbase.backup.BackupAdmin client)
protected void	processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)
private void	restore(org.apache.hadoop.hbase.backup.RestoreRequest request, org.apache.hadoop.hbase.backup.BackupAdmin client)
private void	restoreVerifyTable(org.apache.hadoop.hbase.client.Connection conn, org.apache.hadoop.hbase.backup.BackupAdmin client, org.apache.hadoop.hbase.TableName table, String backupId, long expectedRows)
int	runTestFromCommandLine()	Returns status of CLI execution
private void	runTestMulti()
private void	runTestSingle(org.apache.hadoop.hbase.TableName table)
void	setUp()
void	setUpCluster()
void	setUpMonkey()
void	tearDown()
void	testBackupRestore()

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CLASS_NAME

private static final String CLASS_NAME

LOG

protected static final org.slf4j.Logger LOG

NUMBER_OF_TABLES_KEY

protected static final String NUMBER_OF_TABLES_KEY

See Also:

• Constant Field Values

COLUMN_NAME

protected static final String COLUMN_NAME

See Also:

• Constant Field Values

REGION_COUNT_KEY

protected static final String REGION_COUNT_KEY

See Also:

• Constant Field Values

REGIONSERVER_COUNT_KEY

protected static final String REGIONSERVER_COUNT_KEY

See Also:

• Constant Field Values

ROWS_PER_ITERATION_KEY

protected static final String ROWS_PER_ITERATION_KEY

See Also:

• Constant Field Values

NUM_ITERATIONS_KEY

protected static final String NUM_ITERATIONS_KEY

See Also:

• Constant Field Values

DEFAULT_REGION_COUNT

protected static final int DEFAULT_REGION_COUNT

See Also:

• Constant Field Values

DEFAULT_REGIONSERVER_COUNT

protected static final int DEFAULT_REGIONSERVER_COUNT

See Also:

• Constant Field Values

DEFAULT_NUMBER_OF_TABLES

protected static final int DEFAULT_NUMBER_OF_TABLES

See Also:

• Constant Field Values

DEFAULT_NUM_ITERATIONS

protected static final int DEFAULT_NUM_ITERATIONS

See Also:

• Constant Field Values

DEFAULT_ROWS_IN_ITERATION

protected static final int DEFAULT_ROWS_IN_ITERATION

See Also:

• Constant Field Values

SLEEP_TIME_KEY

protected static final String SLEEP_TIME_KEY

See Also:

• Constant Field Values

SLEEP_TIME_DEFAULT

protected static final long SLEEP_TIME_DEFAULT

See Also:

• Constant Field Values

rowsInIteration

protected static int rowsInIteration

regionsCountPerServer

protected static int regionsCountPerServer

regionServerCount

protected static int regionServerCount

numIterations

protected static int numIterations

numTables

protected static int numTables

tableNames

protected static org.apache.hadoop.hbase.TableName[] tableNames

sleepTime

protected long sleepTime

lock

protected static Object lock

BACKUP_ROOT_DIR

private static String BACKUP_ROOT_DIR

Constructor Details

IntegrationTestBackupRestore

public IntegrationTestBackupRestore()

Method Details

setUp

public void setUp()
           throws Exception

Overrides:

setUp in class IntegrationTestBase

Throws:

Exception

tearDown

public void tearDown()
              throws IOException

Throws:

IOException

setUpMonkey

public void setUpMonkey()
                 throws Exception

Overrides:

setUpMonkey in class IntegrationTestBase

Throws:

Exception

deleteTablesIfAny

private void deleteTablesIfAny()
                        throws IOException

Throws:

IOException

createTables

private void createTables()
                   throws Exception

Throws:

Exception

enableBackup

private void enableBackup(org.apache.hadoop.conf.Configuration conf)

cleanUpBackupDir

private void cleanUpBackupDir()
                       throws IOException

Throws:

IOException

testBackupRestore

public void testBackupRestore()
                       throws Exception

Throws:

Exception

runTestMulti

private void runTestMulti()
                   throws IOException

Throws:

IOException

createTable

private void createTable(org.apache.hadoop.hbase.TableName tableName)
                  throws Exception

Throws:

Exception

loadData

private void loadData(org.apache.hadoop.hbase.TableName table,
 int numRows)
               throws IOException

Throws:

IOException

backup

private String backup(org.apache.hadoop.hbase.backup.BackupRequest request,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
               throws IOException

Throws:

IOException

restore

private void restore(org.apache.hadoop.hbase.backup.RestoreRequest request,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
              throws IOException

Throws:

IOException

merge

private void merge(String[] backupIds,
 org.apache.hadoop.hbase.backup.BackupAdmin client)
            throws IOException

Throws:

IOException

runTestSingle

private void runTestSingle(org.apache.hadoop.hbase.TableName table)
                    throws IOException

Throws:

IOException

restoreVerifyTable

private void restoreVerifyTable(org.apache.hadoop.hbase.client.Connection conn,
 org.apache.hadoop.hbase.backup.BackupAdmin client,
 org.apache.hadoop.hbase.TableName table,
 String backupId,
 long expectedRows)
                         throws IOException

Throws:

IOException

allIncremental

private String[] allIncremental(List<String> backupIds)

checkSucceeded

protected boolean checkSucceeded(String backupId)
                          throws IOException

Check if backup is succeeded

Parameters:

backupId - pass backup ID to check status of

Returns:

status of backup

Throws:

IOException

getBackupInfo

private org.apache.hadoop.hbase.backup.BackupInfo getBackupInfo(String backupId)
                                                         throws IOException

Throws:

IOException

createRestoreRequest

public org.apache.hadoop.hbase.backup.RestoreRequest createRestoreRequest(String backupRootDir,
 String backupId,
 boolean check,
 org.apache.hadoop.hbase.TableName[] fromTables,
 org.apache.hadoop.hbase.TableName[] toTables,
 boolean isOverwrite)

Get restore request.

Parameters:

backupRootDir - directory where backup is located

backupId - backup ID

check - check the backup

fromTables - table names to restore from

toTables - new table names to restore to

isOverwrite - overwrite the table(s)

Returns:

an instance of RestoreRequest

setUpCluster

public void setUpCluster()
                  throws Exception

Specified by:

setUpCluster in class IntegrationTestBase

Throws:

Exception

runTestFromCommandLine

public int runTestFromCommandLine()
                           throws Exception

Returns status of CLI execution

Specified by:

runTestFromCommandLine in class IntegrationTestBase

Throws:

Exception

getTablename

public org.apache.hadoop.hbase.TableName getTablename()

Description copied from class: IntegrationTestBase

Provides the name of the table that is protected from random Chaos monkey activity

Specified by:

getTablename in class IntegrationTestBase

Returns:

table to not delete.

getColumnFamilies

protected Set<String> getColumnFamilies()

Description copied from class: IntegrationTestBase

Provides the name of the CFs that are protected from random Chaos monkey activity (alter)

Specified by:

getColumnFamilies in class IntegrationTestBase

Returns:

set of cf names to protect.

addOptions

protected void addOptions()

Overrides:

addOptions in class IntegrationTestBase

processOptions

protected void processOptions(org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine cmd)

Overrides:

processOptions in class IntegrationTestBase

main

public static void main(String[] args)
                 throws Exception

Main method

Parameters:

args - argument list

Throws:

Exception