Package org.apache.hadoop.hbase.client

Class BufferedMutatorImpl

java.lang.Object

org.apache.hadoop.hbase.client.BufferedMutatorImpl

All Implemented Interfaces:

Closeable, AutoCloseable, BufferedMutator

@Private
@Evolving
public class BufferedMutatorImpl
extends Object
implements BufferedMutator

Used to communicate with a single HBase table similar to Table but meant for batched, potentially asynchronous puts. Obtain an instance from a Connection and call close() afterwards. Provide an alternate to this implementation by setting BufferedMutatorParams.implementationClassName(String) or by setting alternate classname via the key {} in Configuration.

While this can be used across threads, great care should be used when doing so. Errors are global to the buffered mutator and the Exceptions can be thrown on any thread that causes the flush for requests.

Since:

1.0.0

See Also:

• ConnectionFactory
• Connection

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) class	BufferedMutatorImpl.QueueRowAccess

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.client.BufferedMutator

BufferedMutator.ExceptionListener

Field Summary

Fields

Modifier and Type	Field	Description
private final AsyncProcess	ap
private final boolean	cleanupPoolOnClose
private boolean	closed
private final org.apache.hadoop.conf.Configuration	conf
private final AtomicLong	currentWriteBufferSize
private final AtomicLong	executedWriteBufferPeriodicFlushes
private final AtomicLong	firstRecordInBufferTimestamp
private final BufferedMutator.ExceptionListener	listener
private static final org.slf4j.Logger	LOG
private final int	maxKeyValueSize
private final AtomicInteger	operationTimeout
private final ExecutorService	pool
private final AtomicInteger	rpcTimeout
private final TableName	tableName
private final AtomicInteger	undealtMutationCount	Count the size of writeAsyncBuffer.
private final ConcurrentLinkedQueue<Mutation>	writeAsyncBuffer
private final AtomicLong	writeBufferPeriodicFlushTimeoutMs
private Timer	writeBufferPeriodicFlushTimer
private final AtomicLong	writeBufferPeriodicFlushTimerTickMs
private final long	writeBufferSize

Fields inherited from interface org.apache.hadoop.hbase.client.BufferedMutator

CLASSNAME_KEY, MIN_WRITE_BUFFER_PERIODIC_FLUSH_TIMERTICK_MS

Constructor Summary

Constructors

Constructor	Description
BufferedMutatorImpl(ClusterConnection conn, BufferedMutatorParams params, AsyncProcess ap)
BufferedMutatorImpl(ClusterConnection conn, RpcRetryingCallerFactory rpcCallerFactory, RpcControllerFactory rpcFactory, BufferedMutatorParams params)

Method Summary

Modifier and Type	Method	Description
private void	checkClose()
void	close()	Performs a BufferedMutator.flush() and releases any resources held.
(package private) BufferedMutatorImpl.QueueRowAccess	createQueueRowAccess()
private AsyncProcessTask	createTask(BufferedMutatorImpl.QueueRowAccess access)
private void	doFlush(boolean flushAll)	Send the operations in the buffer to the servers.
void	flush()	Executes all the buffered, asynchronous Mutation operations and waits until they are done.
(package private) AsyncProcess	getAsyncProcess()
org.apache.hadoop.conf.Configuration	getConfiguration()	Returns the Configuration object used by this instance.
(package private) long	getCurrentWriteBufferSize()
protected long	getExecutedWriteBufferPeriodicFlushes()
TableName	getName()	Gets the fully qualified table name instance of the table that this BufferedMutator writes to.
(package private) ExecutorService	getPool()
(package private) int	getUnflushedSize()	Count the mutations which haven't been flushed
long	getWriteBufferPeriodicFlushTimeoutMs()	Returns the current periodic flush timeout value in milliseconds.
long	getWriteBufferPeriodicFlushTimerTickMs()	Returns the current periodic flush timertick interval in milliseconds.
long	getWriteBufferSize()	Returns the maximum size in bytes of the write buffer for this HTable.
private static RetriesExhaustedWithDetailsException	makeException(List<RetriesExhaustedWithDetailsException> errors)
void	mutate(List<? extends Mutation> ms)	Send some Mutations to the table.
void	mutate(Mutation m)	Sends a Mutation to the table.
void	setOperationTimeout(int operationTimeout)	Set operation timeout for this mutator instance
void	setRpcTimeout(int rpcTimeout)	Set rpc timeout for this mutator instance
void	setWriteBufferPeriodicFlush(long timeoutMs, long timerTickMs)	Sets the maximum time before the buffer is automatically flushed.
(package private) int	size()	Count the mutations which haven't been processed.
private void	timerCallbackForWriteBufferPeriodicFlush()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.client.BufferedMutator

disableWriteBufferPeriodicFlush, setWriteBufferPeriodicFlush

Field Details

LOG

private static final org.slf4j.Logger LOG

listener

private final BufferedMutator.ExceptionListener listener

tableName

private final TableName tableName

conf

private final org.apache.hadoop.conf.Configuration conf

writeAsyncBuffer

private final ConcurrentLinkedQueue<Mutation> writeAsyncBuffer

currentWriteBufferSize

private final AtomicLong currentWriteBufferSize

undealtMutationCount

private final AtomicInteger undealtMutationCount

Count the size of writeAsyncBuffer. The ConcurrentLinkedQueue.size() is NOT a constant-time operation.

writeBufferSize

private final long writeBufferSize

writeBufferPeriodicFlushTimeoutMs

private final AtomicLong writeBufferPeriodicFlushTimeoutMs

writeBufferPeriodicFlushTimerTickMs

private final AtomicLong writeBufferPeriodicFlushTimerTickMs

writeBufferPeriodicFlushTimer

private Timer writeBufferPeriodicFlushTimer

maxKeyValueSize

private final int maxKeyValueSize

pool

private final ExecutorService pool

rpcTimeout

private final AtomicInteger rpcTimeout

operationTimeout

private final AtomicInteger operationTimeout

cleanupPoolOnClose

private final boolean cleanupPoolOnClose

closed

private volatile boolean closed

ap

private final AsyncProcess ap

firstRecordInBufferTimestamp

private final AtomicLong firstRecordInBufferTimestamp

executedWriteBufferPeriodicFlushes

private final AtomicLong executedWriteBufferPeriodicFlushes

Constructor Details

BufferedMutatorImpl

BufferedMutatorImpl(ClusterConnection conn,
 BufferedMutatorParams params,
 AsyncProcess ap)

BufferedMutatorImpl

BufferedMutatorImpl(ClusterConnection conn,
 RpcRetryingCallerFactory rpcCallerFactory,
 RpcControllerFactory rpcFactory,
 BufferedMutatorParams params)

Method Details

checkClose

private void checkClose()

getPool

ExecutorService getPool()

getAsyncProcess

AsyncProcess getAsyncProcess()

getName

public TableName getName()

Description copied from interface: BufferedMutator

Gets the fully qualified table name instance of the table that this BufferedMutator writes to.

Specified by:

getName in interface BufferedMutator

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

Description copied from interface: BufferedMutator

Returns the Configuration object used by this instance.

The reference returned is not a copy, so any change made to it will affect this instance.

Specified by:

getConfiguration in interface BufferedMutator

mutate

public void mutate(Mutation m)
            throws InterruptedIOException,
RetriesExhaustedWithDetailsException

Description copied from interface: BufferedMutator

Sends a Mutation to the table. The mutations will be buffered and sent over the wire as part of a batch. Currently only supports Put and Delete mutations.

Specified by:

mutate in interface BufferedMutator

Parameters:

m - The data to send.

Throws:

InterruptedIOException

RetriesExhaustedWithDetailsException

mutate

public void mutate(List<? extends Mutation> ms)
            throws InterruptedIOException,
RetriesExhaustedWithDetailsException

Description copied from interface: BufferedMutator

Send some Mutations to the table. The mutations will be buffered and sent over the wire as part of a batch. There is no guarantee of sending entire content of mutations in a single batch; it will be broken up according to the write buffer capacity.

Specified by:

mutate in interface BufferedMutator

Parameters:

ms - The data to send.

Throws:

InterruptedIOException

RetriesExhaustedWithDetailsException

getExecutedWriteBufferPeriodicFlushes

protected long getExecutedWriteBufferPeriodicFlushes()

timerCallbackForWriteBufferPeriodicFlush

private void timerCallbackForWriteBufferPeriodicFlush()

close

public void close()
           throws IOException

Description copied from interface: BufferedMutator

Performs a BufferedMutator.flush() and releases any resources held.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface BufferedMutator

Specified by:

close in interface Closeable

Throws:

IOException - if a remote or network exception occurs.

createTask

private AsyncProcessTask createTask(BufferedMutatorImpl.QueueRowAccess access)

flush

public void flush()
           throws InterruptedIOException,
RetriesExhaustedWithDetailsException

Description copied from interface: BufferedMutator

Executes all the buffered, asynchronous Mutation operations and waits until they are done.

Specified by:

flush in interface BufferedMutator

Throws:

InterruptedIOException

RetriesExhaustedWithDetailsException

doFlush

private void doFlush(boolean flushAll)
              throws InterruptedIOException,
RetriesExhaustedWithDetailsException

Send the operations in the buffer to the servers.

Parameters:

flushAll - - if true, sends all the writes and wait for all of them to finish before returning. Otherwise, flush until buffer size is smaller than threshold

Throws:

InterruptedIOException

RetriesExhaustedWithDetailsException

makeException

private static RetriesExhaustedWithDetailsException makeException(List<RetriesExhaustedWithDetailsException> errors)

getWriteBufferSize

public long getWriteBufferSize()

Returns the maximum size in bytes of the write buffer for this HTable.

The default value comes from the configuration parameter hbase.client.write.buffer.

Specified by:

getWriteBufferSize in interface BufferedMutator

Returns:

The size of the write buffer in bytes.

setWriteBufferPeriodicFlush

public void setWriteBufferPeriodicFlush(long timeoutMs,
 long timerTickMs)

Description copied from interface: BufferedMutator

Sets the maximum time before the buffer is automatically flushed.

Specified by:

setWriteBufferPeriodicFlush in interface BufferedMutator

Parameters:

timeoutMs - The maximum number of milliseconds how long records may be buffered before they are flushed. Set to 0 to disable.

timerTickMs - The number of milliseconds between each check if the timeout has been exceeded. Must be 100ms (as defined in BufferedMutator.MIN_WRITE_BUFFER_PERIODIC_FLUSH_TIMERTICK_MS) or larger to avoid performance problems.

getWriteBufferPeriodicFlushTimeoutMs

public long getWriteBufferPeriodicFlushTimeoutMs()

Description copied from interface: BufferedMutator

Returns the current periodic flush timeout value in milliseconds.

Specified by:

getWriteBufferPeriodicFlushTimeoutMs in interface BufferedMutator

Returns:

The maximum number of milliseconds how long records may be buffered before they are flushed. The value 0 means this is disabled.

getWriteBufferPeriodicFlushTimerTickMs

public long getWriteBufferPeriodicFlushTimerTickMs()

Description copied from interface: BufferedMutator

Returns the current periodic flush timertick interval in milliseconds.

Specified by:

getWriteBufferPeriodicFlushTimerTickMs in interface BufferedMutator

Returns:

The number of milliseconds between each check if the timeout has been exceeded. This value only has a real meaning if the timeout has been set to > 0

setRpcTimeout

public void setRpcTimeout(int rpcTimeout)

Description copied from interface: BufferedMutator

Set rpc timeout for this mutator instance

Specified by:

setRpcTimeout in interface BufferedMutator

setOperationTimeout

public void setOperationTimeout(int operationTimeout)

Description copied from interface: BufferedMutator

Set operation timeout for this mutator instance

Specified by:

setOperationTimeout in interface BufferedMutator

getCurrentWriteBufferSize

long getCurrentWriteBufferSize()

size

int size()

Count the mutations which haven't been processed.

Returns:

count of undealt mutation

getUnflushedSize

int getUnflushedSize()

Count the mutations which haven't been flushed

Returns:

count of unflushed mutation

createQueueRowAccess

BufferedMutatorImpl.QueueRowAccess createQueueRowAccess()