Package org.apache.hadoop.hbase

Class IntegrationTestRegionReplicaReplication

java.lang.Object

org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.IntegrationTestBase

org.apache.hadoop.hbase.IntegrationTestIngest

org.apache.hadoop.hbase.IntegrationTestRegionReplicaReplication

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class IntegrationTestRegionReplicaReplication
extends IntegrationTestIngest

Integration test for testing async wal replication to secondary region replicas. Sets up a table with given region replication (default 2), and uses LoadTestTool client writer, updater and reader threads for writes and reads and verification. It uses a delay queue with a given delay ("read_delay_ms", default 5000ms) between the writer/updater and reader threads to make the written items available to readers. This means that a reader will only start reading from a row written by the writer / updater after 5secs has passed. The reader thread performs the reads from the given region replica id (default 1) to perform the reads. Async wal replication has to finish with the replication of the edits before read_delay_ms to the given region replica id so that the read and verify will not fail. The job will run for at least given runtime (default 10min) by running a concurrent writer and reader workload followed by a concurrent updater and reader workload for num_keys_per_server.

Example usage:

 hbase org.apache.hadoop.hbase.IntegrationTestRegionReplicaReplication
 -DIntegrationTestRegionReplicaReplication.num_keys_per_server=10000
 -Dhbase.IntegrationTestRegionReplicaReplication.runtime=600000
 -DIntegrationTestRegionReplicaReplication.read_delay_ms=5000
 -DIntegrationTestRegionReplicaReplication.region_replication=3
 -DIntegrationTestRegionReplicaReplication.region_replica_id=2
 -DIntegrationTestRegionReplicaReplication.num_read_threads=100
 -DIntegrationTestRegionReplicaReplication.num_write_threads=100
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	IntegrationTestRegionReplicaReplication.DelayingMultiThreadedUpdater	This extends MultiThreadedWriter to add a configurable delay to the keys written by the writer threads to become available to the MultiThradedReader threads.
static class	IntegrationTestRegionReplicaReplication.DelayingMultiThreadedWriter	This extends MultiThreadedWriter to add a configurable delay to the keys written by the writer threads to become available to the MultiThradedReader threads.

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

org.apache.hadoop.hbase.util.AbstractHBaseTool.OptionsOrderComparator

Field Summary

Fields

Modifier and Type	Field	Description
private static final String[]	DEFAULT_COLUMN_FAMILIES
private static final int	DEFAULT_REGION_REPLICATION
private static final String	OPT_READ_DELAY_MS
private static final int	SERVER_COUNT
private static final String	TEST_NAME

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

cluster, DEFAULT_NUM_KEYS_PER_SERVER, DEFAULT_NUM_READ_THREADS, DEFAULT_NUM_WRITE_THREADS, DEFAULT_RUN_TIME, HIPHEN, JUNIT_RUN_TIME, LOAD_TEST_TOOL_INIT_ARGS, loadTool, LOG, NUM_KEYS_PER_SERVER_KEY, NUM_READ_THREADS_KEY, NUM_WRITE_THREADS_KEY, RUN_TIME_KEY, util

Fields inherited from class org.apache.hadoop.hbase.IntegrationTestBase

CHAOS_MONKEY_PROPS, monkey, MONKEY_LONG_OPT, monkeyProps, monkeyToUse, NO_CLUSTER_CLEANUP_LONG_OPT, noClusterCleanUp

Fields inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

cmdLineArgs, conf, EXIT_FAILURE, EXIT_SUCCESS, LONG_HELP_OPTION, options, SHORT_HELP_OPTION

Constructor Summary

Constructors

Constructor	Description
IntegrationTestRegionReplicaReplication()

Method Summary

Modifier and Type	Method	Description
protected int	getMinServerCount()
static void	main(String[] args)
protected void	runIngestTest(long defaultRunTime, long keysPerServerPerIter, int colsPerKey, int recordSize, int writeThreads, int readThreads)
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	testIngest()

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestIngest

getArgsForLoadTestTool, getArgsForLoadTestToolInitTable, getColumnFamilies, getNumKeys, getTablename, initTable, internalRunIngestTest, runTestFromCommandLine, setUpCluster

Methods inherited from class org.apache.hadoop.hbase.IntegrationTestBase

addOptions, cleanUp, cleanUpCluster, cleanUpMonkey, cleanUpMonkey, doWork, getConf, getDefaultMonkeyFactory, getTestingUtil, loadMonkeyProperties, processBaseOptions, processOptions, setUp, setUpMonkey, startMonkey

Methods inherited from class org.apache.hadoop.hbase.util.AbstractHBaseTool

addOption, addOptNoArg, addOptNoArg, addOptWithArg, addOptWithArg, addRequiredOption, addRequiredOptWithArg, addRequiredOptWithArg, doStaticMain, getOptionAsDouble, getOptionAsInt, getOptionAsInt, getOptionAsLong, getOptionAsLong, newParser, parseArgs, parseInt, parseLong, printUsage, printUsage, processOldArgs, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

TEST_NAME

private static final String TEST_NAME

OPT_READ_DELAY_MS

private static final String OPT_READ_DELAY_MS

See Also:

• Constant Field Values

DEFAULT_REGION_REPLICATION

private static final int DEFAULT_REGION_REPLICATION

See Also:

• Constant Field Values

SERVER_COUNT

private static final int SERVER_COUNT

See Also:

• Constant Field Values

DEFAULT_COLUMN_FAMILIES

private static final String[] DEFAULT_COLUMN_FAMILIES

Constructor Details

IntegrationTestRegionReplicaReplication

public IntegrationTestRegionReplicaReplication()

Method Details

getMinServerCount

protected int getMinServerCount()

Overrides:

getMinServerCount in class IntegrationTestIngest

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class org.apache.hadoop.hbase.util.AbstractHBaseTool

testIngest

public void testIngest()
                throws Exception

Overrides:

testIngest in class IntegrationTestIngest

Throws:

Exception

runIngestTest

protected void runIngestTest(long defaultRunTime,
 long keysPerServerPerIter,
 int colsPerKey,
 int recordSize,
 int writeThreads,
 int readThreads)
                      throws Exception

Overrides:

runIngestTest in class IntegrationTestIngest

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception