Package org.apache.hadoop.hbase.regionserver.wal

Class AsyncFSWAL

java.lang.Object
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL<WALProvider.AsyncWriter>
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL
All Implemented Interfaces:
Closeable, AutoCloseable, WALFileLengthProvider, WAL
@LimitedPrivate("Configuration") public class AsyncFSWAL extends AbstractFSWAL<WALProvider.AsyncWriter>
An asynchronous implementation of FSWAL.

Here 'waitingConsumePayloads' acts as the RingBuffer in FSHLog.

For append, we process it as follow:

  1. In the caller thread(typically, in the rpc handler thread):
    1. Insert the entry into 'waitingConsumePayloads'. Use ringbuffer sequence as txid.
    2. Schedule the consumer task if needed. See AbstractFSWAL.shouldScheduleConsumer() for more details.
  2. In the consumer task(executed in a single threaded thread pool)
    1. Poll the entry from AbstractFSWAL.waitingConsumePayloads and insert it into AbstractFSWAL.toWriteAppends
    2. Poll the entry from AbstractFSWAL.toWriteAppends, append it to the AsyncWriter, and insert it into AbstractFSWAL.unackedAppends
    3. If the buffered size reaches AbstractFSWAL.batchSize, or there is a sync request, then we call sync on the AsyncWriter.
    4. In the callback methods:
For sync, the processing stages are almost same. And different from FSHLog, we will open a new writer and rewrite unacked entries to the new writer and sync again if we hit a sync error.

Here we only describe the logic of doReplaceWriter. The main logic of rollWriter is same with FSHLog.
For a normal roll request(for example, we have reached the log roll size):

  1. In the log roller thread, we will set AbstractFSWAL.waitingRoll(int) to true and AbstractFSWAL.readyForRolling to false, and then wait on AbstractFSWAL.readyForRolling(see AbstractFSWAL.waitForSafePoint()).
  2. In the consumer thread, we will stop polling entries from AbstractFSWAL.waitingConsumePayloads if AbstractFSWAL.waitingRoll(int) is true, and also stop writing the entries in AbstractFSWAL.toWriteAppends out.
  3. If there are unflush data in the writer, sync them.
  4. When all out-going sync request is finished, i.e, the AbstractFSWAL.unackedAppends is empty, signal the AbstractFSWAL.readyForRollingCond.
  5. Back to the log roller thread, now we can confirm that there are no out-going entries, i.e., we reach a safe point. So it is safe to replace old writer with new writer now.
  6. Set AbstractFSWAL.writerBroken(int) and AbstractFSWAL.waitingRoll(int) to false.
  7. Schedule the consumer task.
  8. Schedule a background task to close the old writer.
For a broken writer roll request, the only difference is that we can bypass the wait for safe point stage.