Package org.apache.hadoop.hbase.mapreduce

Class TestImportExport

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestImportExport

public class TestImportExport
extends Object

Tests the table import and table export MR job functionality

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TestImportExport.MetadataController
private static class	TestImportExport.TableWALActionListener	This listens to the TestImportExport.TableWALActionListener.visitLogEntryBeforeWrite(RegionInfo, WALKey, WALEdit) to identify that an entry is written to the Write Ahead Log for the given table.

Field Summary

Fields

Modifier and Type	Field	Description
static final HBaseClassTestRule	CLASS_RULE
private static final String	EXPORT_BATCH_SIZE
private final org.apache.hadoop.hbase.TableName	EXPORT_TABLE
private static final byte[]	FAMILYA
private static final String	FAMILYA_STRING
private static final byte[]	FAMILYB
private static final String	FAMILYB_STRING
private static String	FQ_OUTPUT_DIR
private final org.apache.hadoop.hbase.TableName	IMPORT_TABLE
private static final org.slf4j.Logger	LOG
final org.junit.rules.TestName	name
private static final long	now
private static final String	OUTPUT_DIR
private static final byte[]	QUAL
private static final byte[]	ROW1
private static final byte[]	ROW2
private static final byte[]	ROW3
static final String	TEST_ATTR
static final String	TEST_TAG
static final byte	TEST_TAG_TYPE
protected static final HBaseTestingUtil	UTIL

Constructor Summary

Constructors

Constructor	Description
TestImportExport()

Method Summary

Modifier and Type	Method	Description
static void	afterClass()
void	announce()
static void	beforeClass()
private void	checkWhetherTagExists(org.apache.hadoop.hbase.TableName table, boolean tagExists)
void	cleanup()
private int	getCount(org.apache.hadoop.hbase.client.Table table, org.apache.hadoop.hbase.filter.Filter filter)	Count the number of keyvalues in the specified table with the given filter
protected boolean	runExport(String[] args)	Runs an export job with the specified command line args
protected void	runExportMain(String[] args)
(package private) boolean	runImport(String[] args)	Runs an import job with the specified command line args
void	testAddFilterAndArguments()	Test addFilterAndArguments method of Import This method set couple parameters into Configuration
void	testDurability()
void	testExportMain()	test main method.
void	testExportScan()
void	testExportScannerBatching()	Test export scanner batching
void	testImport94Table()	Test import data from 0.94 exported file
void	testImportMain()	test main method.
void	testKeyValueImporter()	Test map method of Importer
void	testMetaExport()	Test export hbase:meta table
void	testSimpleCase()	Test simple replication case with column mapping
void	testTagsAddition()	Add cell tags to delete mutations, run export and import tool and verify that tags are present in import table also.
void	testTagsWithEmptyCodec()	Set hbase.client.rpc.codec and hbase.client.default.rpc.codec both to empty string This means it will use no Codec.
void	testWithDeletes()
void	testWithFilter()	Create a simple table, run an Export Job on it, Import with filtering on, verify counts, attempt with invalid values.
void	testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

LOG

private static final org.slf4j.Logger LOG

UTIL

protected static final HBaseTestingUtil UTIL

ROW1

private static final byte[] ROW1

ROW2

private static final byte[] ROW2

ROW3

private static final byte[] ROW3

FAMILYA_STRING

private static final String FAMILYA_STRING

See Also:

• Constant Field Values

FAMILYB_STRING

private static final String FAMILYB_STRING

See Also:

• Constant Field Values

FAMILYA

private static final byte[] FAMILYA

FAMILYB

private static final byte[] FAMILYB

QUAL

private static final byte[] QUAL

OUTPUT_DIR

private static final String OUTPUT_DIR

See Also:

• Constant Field Values

FQ_OUTPUT_DIR

private static String FQ_OUTPUT_DIR

EXPORT_BATCH_SIZE

private static final String EXPORT_BATCH_SIZE

See Also:

• Constant Field Values

now

private static final long now

EXPORT_TABLE

private final org.apache.hadoop.hbase.TableName EXPORT_TABLE

IMPORT_TABLE

private final org.apache.hadoop.hbase.TableName IMPORT_TABLE

TEST_TAG_TYPE

public static final byte TEST_TAG_TYPE

See Also:

• Constant Field Values

TEST_ATTR

public static final String TEST_ATTR

See Also:

• Constant Field Values

TEST_TAG

public static final String TEST_TAG

See Also:

• Constant Field Values

name

public final org.junit.rules.TestName name

Constructor Details

TestImportExport

public TestImportExport()

Method Details

beforeClass

public static void beforeClass()
                        throws Throwable

Throws:

Throwable

afterClass

public static void afterClass()
                       throws Throwable

Throws:

Throwable

announce

public void announce()

cleanup

public void cleanup()
             throws Throwable

Throws:

Throwable

runExport

protected boolean runExport(String[] args)
                     throws Throwable

Runs an export job with the specified command line args

Returns:

true if job completed successfully

Throws:

Throwable

runExportMain

protected void runExportMain(String[] args)
                      throws Throwable

Throws:

Throwable

runImport

boolean runImport(String[] args)
           throws Throwable

Runs an import job with the specified command line args

Returns:

true if job completed successfully

Throws:

Throwable

testSimpleCase

public void testSimpleCase()
                    throws Throwable

Test simple replication case with column mapping

Throws:

Throwable

testMetaExport

public void testMetaExport()
                    throws Throwable

Test export hbase:meta table

Throws:

Throwable

testImport94Table

public void testImport94Table()
                       throws Throwable

Test import data from 0.94 exported file

Throws:

Throwable

testExportScannerBatching

public void testExportScannerBatching()
                               throws Throwable

Test export scanner batching

Throws:

Throwable

testWithDeletes

public void testWithDeletes()
                     throws Throwable

Throws:

Throwable

testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily

public void testWithMultipleDeleteFamilyMarkersOfSameRowSameFamily()
                                                            throws Throwable

Throws:

Throwable

testWithFilter

public void testWithFilter()
                    throws Throwable

Create a simple table, run an Export Job on it, Import with filtering on, verify counts, attempt with invalid values.

Throws:

Throwable

getCount

private int getCount(org.apache.hadoop.hbase.client.Table table,
 org.apache.hadoop.hbase.filter.Filter filter)
              throws IOException

Count the number of keyvalues in the specified table with the given filter

Parameters:

table - the table to scan

Returns:

the number of keyvalues found

Throws:

IOException

testImportMain

public void testImportMain()
                    throws Throwable

test main method. Import should print help and call System.exit

Throws:

Throwable

testExportScan

public void testExportScan()
                    throws Exception

Throws:

Exception

testExportMain

public void testExportMain()
                    throws Throwable

test main method. Export should print help and call System.exit

Throws:

Throwable

testKeyValueImporter

public void testKeyValueImporter()
                          throws Throwable

Test map method of Importer

Throws:

Throwable

testAddFilterAndArguments

public void testAddFilterAndArguments()
                               throws IOException

Test addFilterAndArguments method of Import This method set couple parameters into Configuration

Throws:

IOException

testDurability

public void testDurability()
                    throws Throwable

Throws:

Throwable

testTagsAddition

public void testTagsAddition()
                      throws Throwable

Add cell tags to delete mutations, run export and import tool and verify that tags are present in import table also.

Throws:

Throwable - throws Throwable.

checkWhetherTagExists

private void checkWhetherTagExists(org.apache.hadoop.hbase.TableName table,
 boolean tagExists)
                            throws IOException

Throws:

IOException

testTagsWithEmptyCodec

public void testTagsWithEmptyCodec()
                            throws Exception

Set hbase.client.rpc.codec and hbase.client.default.rpc.codec both to empty string This means it will use no Codec. Make sure that we don't return Tags in response.

Throws:

Exception - Exception