Package org.apache.hadoop.hbase.replication.regionserver

Class WALEntryStream

java.lang.Object

org.apache.hadoop.hbase.replication.regionserver.WALEntryStream

All Implemented Interfaces:

Closeable, AutoCloseable

@Private
@Evolving
class WALEntryStream
extends Object
implements Closeable

Streaming access to WAL entries. This class is given a queue of WAL Path, and continually iterates through all the WAL WAL.Entry in the queue. When it's done reading from a Path, it dequeues it and starts reading from the next.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	WALEntryStream.HasNext

Field Summary

Fields

Modifier and Type	Field	Description
private final org.apache.hadoop.conf.Configuration	conf
private WAL.Entry	currentEntry
private org.apache.hadoop.fs.Path	currentPath
private long	currentPositionOfEntry
private long	currentPositionOfReader
private boolean	eofAutoRecovery
private final org.apache.hadoop.fs.FileSystem	fs
private static final org.slf4j.Logger	LOG
private final ReplicationSourceLogQueue	logQueue
private final MetricsSource	metrics
private WALTailingReader	reader
private WALTailingReader.State	state
private final WALFileLengthProvider	walFileLengthProvider
private final String	walGroupId

Constructor Summary

Constructors

Constructor	Description
WALEntryStream(ReplicationSourceLogQueue logQueue, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, long startPosition, WALFileLengthProvider walFileLengthProvider, MetricsSource metrics, String walGroupId)	Create an entry stream over the given queue at the given start position

Method Summary

Modifier and Type	Method	Description
private boolean	checkAllBytesParsed()
void	close()
private void	closeReader()
private long	currentTrailerSize()
private void	dequeueCurrentLog()
org.apache.hadoop.fs.Path	getCurrentPath()	Returns the Path of the current WAL
private org.apache.hadoop.fs.FileStatus	getCurrentPathFileStatus()
private String	getCurrentPathStat()
long	getPosition()	Returns the position of the last Entry returned by next()
WALEntryStream.HasNext	hasNext()	Try advance the stream if there is no entry yet.
private WALEntryStream.HasNext	lastAttempt()
WAL.Entry	next()	Returns the next WAL entry in this stream and advance the stream.
WAL.Entry	peek()	Returns the next WAL entry in this stream but does not advance.
private WALEntryStream.HasNext	prepareReader()
private Pair<WALTailingReader.State,Boolean>	readNextEntryAndRecordReaderPosition()	Returns whether the file is opened for writing.
private void	resetReader()
private void	setCurrentPath(org.apache.hadoop.fs.Path path)
private WALEntryStream.HasNext	tryAdvanceEntry()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

reader

private WALTailingReader reader

state

private WALTailingReader.State state

currentPath

private org.apache.hadoop.fs.Path currentPath

currentEntry

private WAL.Entry currentEntry

currentPositionOfEntry

private long currentPositionOfEntry

currentPositionOfReader

private long currentPositionOfReader

logQueue

private final ReplicationSourceLogQueue logQueue

walGroupId

private final String walGroupId

fs

private final org.apache.hadoop.fs.FileSystem fs

conf

private final org.apache.hadoop.conf.Configuration conf

walFileLengthProvider

private final WALFileLengthProvider walFileLengthProvider

metrics

private final MetricsSource metrics

eofAutoRecovery

private boolean eofAutoRecovery

Constructor Details

WALEntryStream

public WALEntryStream(ReplicationSourceLogQueue logQueue,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.conf.Configuration conf,
 long startPosition,
 WALFileLengthProvider walFileLengthProvider,
 MetricsSource metrics,
 String walGroupId)

Create an entry stream over the given queue at the given start position

Parameters:

logQueue - the queue of WAL paths

conf - the Configuration to use to create WALStreamReader for this stream

startPosition - the position in the first WAL to start reading at

walFileLengthProvider - provides the length of the WAL file

serverName - the server name which all WALs belong to

metrics - the replication metrics

Method Details

hasNext

public WALEntryStream.HasNext hasNext()

Try advance the stream if there is no entry yet. See the javadoc for WALEntryStream.HasNext for more details about the meanings of the return values.

You can call peek() or next() to get the actual WAL.Entry if this method returns WALEntryStream.HasNext.YES.

peek

public WAL.Entry peek()

Returns the next WAL entry in this stream but does not advance.

Must call hasNext() first before calling this method, and if you have already called next() to consume the current entry, you need to call hasNext() again to advance the stream before calling this method again, otherwise it will always return null

The reason here is that, we need to use the return value of hasNext() to tell upper layer to retry or not, so we can not wrap the hasNext() call inside peek() or next() as they have their own return value.

See Also:

• hasNext()
• next()

next

public WAL.Entry next()

Returns the next WAL entry in this stream and advance the stream. Will throw IllegalStateException if you do not call hasNext() before calling this method. Please see the javadoc of peek() method to see why we need this.

Throws:

IllegalStateException - Every time you want to call this method, please call hasNext() first, otherwise a IllegalStateException will be thrown.

See Also:

• hasNext()
• peek()

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

getPosition

public long getPosition()

Returns the position of the last Entry returned by next()

getCurrentPath

public org.apache.hadoop.fs.Path getCurrentPath()

Returns the Path of the current WAL

getCurrentPathStat

private String getCurrentPathStat()

setCurrentPath

private void setCurrentPath(org.apache.hadoop.fs.Path path)

resetReader

private void resetReader()
                  throws IOException

Throws:

IOException

prepareReader

private WALEntryStream.HasNext prepareReader()

lastAttempt

private WALEntryStream.HasNext lastAttempt()

tryAdvanceEntry

private WALEntryStream.HasNext tryAdvanceEntry()

getCurrentPathFileStatus

private org.apache.hadoop.fs.FileStatus getCurrentPathFileStatus()
                                                          throws IOException

Throws:

IOException

checkAllBytesParsed

private boolean checkAllBytesParsed()

dequeueCurrentLog

private void dequeueCurrentLog()

readNextEntryAndRecordReaderPosition

private Pair<WALTailingReader.State,Boolean> readNextEntryAndRecordReaderPosition()

Returns whether the file is opened for writing.

closeReader

private void closeReader()

currentTrailerSize

private long currentTrailerSize()