Package org.apache.hadoop.hbase.test

Class IntegrationTestBigLinkedList.Verify.VerifyReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.Verify.VerifyReducer

Enclosing class:

IntegrationTestBigLinkedList.Verify

public static class IntegrationTestBigLinkedList.Verify.VerifyReducer
extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Per reducer, we output problem rows as byte arrays so can be used as input for subsequent investigative mapreduce jobs. Each emitted value is prefaced by a one byte flag saying what sort of emission it is. Flag is the Count enum ordinal as a short.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields

Modifier and Type	Field	Description
private org.apache.hadoop.hbase.client.Connection	connection
private final org.apache.hadoop.io.BytesWritable	LOSTFAM
private ArrayList<byte[]>	refs
private AtomicInteger	rows
private final org.apache.hadoop.io.BytesWritable	UNREF

Constructor Summary

Constructors

Constructor	Description
VerifyReducer()

Method Summary

Modifier and Type	Method	Description
static byte[]	addPrefixFlag(int ordinal, byte[] r)	Returns new byte array that has ordinal as prefix on front taking up Bytes.SIZEOF_SHORT bytes followed by r
protected void	cleanup(org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context)
private StringBuilder	dumpExtraInfoOnRefs(org.apache.hadoop.io.BytesWritable key, org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context, List<byte[]> refs)	Dump out extra info around references if there are any.
static byte[]	getRowOnly(org.apache.hadoop.io.BytesWritable bw)	Returns Row bytes minus the type flag.
void	reduce(org.apache.hadoop.io.BytesWritable key, Iterable<org.apache.hadoop.io.BytesWritable> values, org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context)
protected void	setup(org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context)
static IntegrationTestBigLinkedList.Verify.VerifyCounts	whichType(byte[] bs)	Returns type from the Counts enum of this row.

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

refs

private ArrayList<byte[]> refs

UNREF

private final org.apache.hadoop.io.BytesWritable UNREF

LOSTFAM

private final org.apache.hadoop.io.BytesWritable LOSTFAM

rows

private AtomicInteger rows

connection

private org.apache.hadoop.hbase.client.Connection connection

Constructor Details

VerifyReducer

public VerifyReducer()

Method Details

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

InterruptedException

cleanup

protected void cleanup(org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context)
                throws IOException,
InterruptedException

Overrides:

cleanup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

InterruptedException

addPrefixFlag

public static byte[] addPrefixFlag(int ordinal,
 byte[] r)

Returns new byte array that has ordinal as prefix on front taking up Bytes.SIZEOF_SHORT bytes followed by r

whichType

public static IntegrationTestBigLinkedList.Verify.VerifyCounts whichType(byte[] bs)

Returns type from the Counts enum of this row. Reads prefix added by addPrefixFlag(int, byte[])

getRowOnly

public static byte[] getRowOnly(org.apache.hadoop.io.BytesWritable bw)

Returns Row bytes minus the type flag.

reduce

public void reduce(org.apache.hadoop.io.BytesWritable key,
 Iterable<org.apache.hadoop.io.BytesWritable> values,
 org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context)
            throws IOException,
InterruptedException

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

InterruptedException

dumpExtraInfoOnRefs

private StringBuilder dumpExtraInfoOnRefs(org.apache.hadoop.io.BytesWritable key,
 org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>.org.apache.hadoop.mapreduce.Reducer.Context context,
 List<byte[]> refs)
                                   throws IOException

Dump out extra info around references if there are any. Helps debugging.

Returns:

StringBuilder filled with references if any.

Throws:

IOException