Package org.apache.hadoop.hbase.regionserver.wal

Class AsyncFSWAL

java.lang.Object
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL<WALProvider.AsyncWriter>
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL
All Implemented Interfaces:
Closeable, AutoCloseable, WALFileLengthProvider, WAL
@LimitedPrivate("Configuration") public class AsyncFSWAL extends AbstractFSWAL<WALProvider.AsyncWriter>
An asynchronous implementation of FSWAL.

Here 'waitingConsumePayloads' acts as the RingBuffer in FSHLog.

For append, we process it as follow:

  1. In the caller thread(typically, in the rpc handler thread):
    1. Insert the entry into 'waitingConsumePayloads'. Use ringbuffer sequence as txid.
    2. Schedule the consumer task if needed. See shouldScheduleConsumer() for more details.
  2. In the consumer task(executed in a single threaded thread pool)
    1. Poll the entry from waitingConsumePayloads and insert it into toWriteAppends
    2. Poll the entry from toWriteAppends, append it to the AsyncWriter, and insert it into unackedAppends
    3. If the buffered size reaches batchSize, or there is a sync request, then we call sync on the AsyncWriter.
    4. In the callback methods:
For sync, the processing stages are almost same. And different from FSHLog, we will open a new writer and rewrite unacked entries to the new writer and sync again if we hit a sync error.

Here we only describe the logic of doReplaceWriter. The main logic of rollWriter is same with FSHLog.
For a normal roll request(for example, we have reached the log roll size):

  1. In the log roller thread, we will set waitingRoll(int) to true and readyForRolling to false, and then wait on readyForRolling(see waitForSafePoint()).
  2. In the consumer thread, we will stop polling entries from waitingConsumePayloads if waitingRoll(int) is true, and also stop writing the entries in toWriteAppends out.
  3. If there are unflush data in the writer, sync them.
  4. When all out-going sync request is finished, i.e, the unackedAppends is empty, signal the readyForRollingCond.
  5. Back to the log roller thread, now we can confirm that there are no out-going entries, i.e., we reach a safe point. So it is safe to replace old writer with new writer now.
  6. Set writerBroken(int) and waitingRoll(int) to false.
  7. Schedule the consumer task.
  8. Schedule a background task to close the old writer.
For a broken writer roll request, the only difference is that we can bypass the wait for safe point stage.