Package org.apache.hadoop.hbase.wal

Class BoundedRecoveredHFilesOutputSink

java.lang.Object

org.apache.hadoop.hbase.wal.OutputSink

org.apache.hadoop.hbase.wal.BoundedRecoveredHFilesOutputSink

@Private
public class BoundedRecoveredHFilesOutputSink
extends OutputSink

A WALSplitter sink that outputs HFiles. Runs with a bounded number of HFile writers at any one time rather than let the count run up.

See Also:

• for a sink implementation that writes intermediate recovered.edits files.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.wal.OutputSink

OutputSink.WriterThread

Field Summary

Fields

Modifier and Type	Field	Description
private static final org.slf4j.Logger	LOG
private final AtomicInteger	openingWritersNum
private ConcurrentMap<String,Long>	regionEditsWrittenMap
private final WALSplitter	walSplitter

Fields inherited from class org.apache.hadoop.hbase.wal.OutputSink

closeCompletionService, closeThreadPool, entryBuffers, numThreads, reporter, splits, status, totalSkippedEdits

Constructor Summary

Constructors

Constructor	Description
BoundedRecoveredHFilesOutputSink(WALSplitter walSplitter, WALSplitter.PipelineController controller, EntryBuffers entryBuffers, int numWriters)

Method Summary

Modifier and Type	Method	Description
void	append(EntryBuffers.RegionEntryBuffer buffer)
List<org.apache.hadoop.fs.Path>	close()
private StoreFileWriter	createRecoveredHFileWriter(TableName tableName, String regionName, long seqId, String familyName, boolean isMetaTable)
int	getNumberOfRecoveredRegions()	Returns number of regions we've recovered
int	getNumOpenWriters()	Returns the number of currently opened writers
Map<String,Long>	getOutputCounts()	Returns a map from encoded region ID to the number of edits written out for that region.
boolean	keepRegionEvent(WAL.Entry entry)	Some WALEdit's contain only KV's for account on what happened to a region.
private boolean	writeRemainingEntryBuffers()	Write out the remaining RegionEntryBuffers and close the writers.

Methods inherited from class org.apache.hadoop.hbase.wal.OutputSink

finishWriterThreads, getTotalSkippedEdits, restartWriterThreadsIfNeeded, setReporter, setStatus, startWriterThreads, updateStatusWithMsg

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

walSplitter

private final WALSplitter walSplitter

regionEditsWrittenMap

private ConcurrentMap<String,Long> regionEditsWrittenMap

openingWritersNum

private final AtomicInteger openingWritersNum

Constructor Details

BoundedRecoveredHFilesOutputSink

public BoundedRecoveredHFilesOutputSink(WALSplitter walSplitter,
 WALSplitter.PipelineController controller,
 EntryBuffers entryBuffers,
 int numWriters)

Method Details

append

public void append(EntryBuffers.RegionEntryBuffer buffer)
            throws IOException

Specified by:

append in class OutputSink

Parameters:

buffer - A buffer of some number of edits for a given region.

Throws:

IOException - For any IO errors

close

public List<org.apache.hadoop.fs.Path> close()
                                      throws IOException

Specified by:

close in class OutputSink

Throws:

IOException

writeRemainingEntryBuffers

private boolean writeRemainingEntryBuffers()
                                    throws IOException

Write out the remaining RegionEntryBuffers and close the writers.

Returns:

true when there is no error.

Throws:

IOException

getOutputCounts

public Map<String,Long> getOutputCounts()

Description copied from class: OutputSink

Returns a map from encoded region ID to the number of edits written out for that region.

Specified by:

getOutputCounts in class OutputSink

getNumberOfRecoveredRegions

public int getNumberOfRecoveredRegions()

Description copied from class: OutputSink

Returns number of regions we've recovered

Specified by:

getNumberOfRecoveredRegions in class OutputSink

getNumOpenWriters

public int getNumOpenWriters()

Description copied from class: OutputSink

Returns the number of currently opened writers

Specified by:

getNumOpenWriters in class OutputSink

keepRegionEvent

public boolean keepRegionEvent(WAL.Entry entry)

Description copied from class: OutputSink

Some WALEdit's contain only KV's for account on what happened to a region. Not all sinks will want to get all of those edits.

Specified by:

keepRegionEvent in class OutputSink

Returns:

Return true if this sink wants to accept this region-level WALEdit.

createRecoveredHFileWriter

private StoreFileWriter createRecoveredHFileWriter(TableName tableName,
 String regionName,
 long seqId,
 String familyName,
 boolean isMetaTable)
                                            throws IOException

Returns:

Returns a base HFile without compressions or encodings; good enough for recovery given hfile has metadata on how it was written.

Throws:

IOException