Package org.apache.hadoop.hbase.wal

Interface WAL

All Superinterfaces:
AutoCloseable, Closeable, WALFileLengthProvider
All Known Implementing Classes:
AbstractFSWAL, AsyncFSWAL, DisabledWALProvider.DisabledWAL, FSHLog
@Private @Evolving public interface WAL extends Closeable, WALFileLengthProvider
A Write Ahead Log (WAL) provides service for reading, writing waledits. This interface provides APIs for WAL users (such as RegionServer) to use the WAL (do append, sync, etc). Note that some internals, such as log rolling and performance evaluation tools, will use WAL.equals to determine if they have already seen a given WAL.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static class 
    WAL.Entry
    Utility class that lets us keep track of the edit with it's key.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    abortCacheFlush(byte[] encodedRegionName)
    Abort a cache flush.
    long
    appendData(RegionInfo info, WALKeyImpl key, WALEdit edits)
    Append a set of data edits to the WAL.
    long
    appendMarker(RegionInfo info, WALKeyImpl key, WALEdit edits)
    Append an operational 'meta' event marker edit to the WAL.
    void
    close()
    Caller no longer needs any edits from this WAL.
    void
    completeCacheFlush(byte[] encodedRegionName, long maxFlushedSeqId)
    Complete the cache flush.
    WALCoprocessorHost
    getCoprocessorHost()
    Returns Coprocessor host.
    long
    getEarliestMemStoreSeqNum(byte[] encodedRegionName, byte[] familyName)
    Gets the earliest unflushed sequence id in the memstore for the store.
    default void
    init()
    Used to initialize the WAL.
    void
    registerWALActionsListener(WALActionsListener listener)
    Registers WALActionsListener
    Map<byte[],List<byte[]>>
    rollWriter()
    Roll the log writer.
    Map<byte[],List<byte[]>>
    rollWriter(boolean force)
    Roll the log writer.
    void
    shutdown()
    Stop accepting new writes.
    default void
    skipRemoteWAL(boolean markerEditOnly)
    Tell the WAL that when creating new writer you can skip creating the remote writer.
    Long
    startCacheFlush(byte[] encodedRegionName, Map<byte[],Long> familyToSeq)
     
    Long
    startCacheFlush(byte[] encodedRegionName, Set<byte[]> families)
    WAL keeps track of the sequence numbers that are as yet not flushed im memstores in order to be able to do accounting to figure which WALs can be let go.
    void
    sync()
    Sync what we have in the WAL.
    default void
    sync(boolean forceSync)
     
    void
    sync(long txid)
    Sync the WAL if the txId was not already sync'd.
    default void
    sync(long txid, boolean forceSync)
     
    String
    toString()
    Human readable identifying information about the state of this WAL.
    boolean
    unregisterWALActionsListener(WALActionsListener listener)
    Unregisters WALActionsListener
    void
    updateStore(byte[] encodedRegionName, byte[] familyName, Long sequenceid, boolean onlyIfGreater)
    updates the seuence number of a specific store.

    Methods inherited from interface org.apache.hadoop.hbase.replication.regionserver.WALFileLengthProvider

    getLogFileSizeIfBeingWritten

  • Method Details

    • init

      default void init() throws IOException
      Used to initialize the WAL. Usually this is for creating the first writer.
      Throws:
      IOException

    • registerWALActionsListener

      void registerWALActionsListener(WALActionsListener listener)
      Registers WALActionsListener

    • unregisterWALActionsListener

      boolean unregisterWALActionsListener(WALActionsListener listener)
      Unregisters WALActionsListener

    • rollWriter

      Map<byte[],List<byte[]>> rollWriter() throws FailedLogCloseException, IOException
      Roll the log writer. That is, start writing log messages to a new file.

      The implementation is synchronized in order to make sure there's one rollWriter running at any given time.

      Returns:
      If lots of logs, flush the stores of returned regions so next time through we can clean logs. Returns null if nothing to flush. Names are actual region names as returned by RegionInfo.getEncodedName()
      Throws:
      FailedLogCloseException
      IOException

    • rollWriter

      Map<byte[],List<byte[]>> rollWriter(boolean force) throws IOException
      Roll the log writer. That is, start writing log messages to a new file.

      The implementation is synchronized in order to make sure there's one rollWriter running at any given time. If true, force creation of a new writer even if no entries have been written to the current writer

      Returns:
      If lots of logs, flush the stores of returned regions so next time through we can clean logs. Returns null if nothing to flush. Names are actual region names as returned by RegionInfo.getEncodedName()
      Throws:
      IOException

    • shutdown

      void shutdown() throws IOException
      Stop accepting new writes. If we have unsynced writes still in buffer, sync them. Extant edits are left in place in backing storage to be replayed later.
      Throws:
      IOException

    • close

      void close() throws IOException
      Caller no longer needs any edits from this WAL. Implementers are free to reclaim underlying resources after this call; i.e. filesystem based WALs can archive or delete files.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • appendData

      long appendData(RegionInfo info, WALKeyImpl key, WALEdit edits) throws IOException
      Append a set of data edits to the WAL. 'Data' here means that the content in the edits will also have transitioned through the memstore.

      The WAL is not flushed/sync'd after this transaction completes BUT on return this edit must have its region edit/sequence id assigned else it messes up our unification of mvcc and sequenceid. On return key will have the region edit/sequence id filled in.

      Parameters:
      info - the regioninfo associated with append
      key - Modified by this call; we add to it this edits region edit/sequence id.
      edits - Edits to append. MAY CONTAIN NO EDITS for case where we want to get an edit sequence id that is after all currently appended edits.
      Returns:
      Returns a 'transaction id' and key will have the region edit/sequence id in it.
      Throws:
      IOException
      See Also:

    • appendMarker

      long appendMarker(RegionInfo info, WALKeyImpl key, WALEdit edits) throws IOException
      Append an operational 'meta' event marker edit to the WAL. A marker meta edit could be a FlushDescriptor, a compaction marker, or a region event marker; e.g. region open or region close. The difference between a 'marker' append and a 'data' append as in appendData(RegionInfo, WALKeyImpl, WALEdit)is that a marker will not have transitioned through the memstore.

      The WAL is not flushed/sync'd after this transaction completes BUT on return this edit must have its region edit/sequence id assigned else it messes up our unification of mvcc and sequenceid. On return key will have the region edit/sequence id filled in.

      Parameters:
      info - the regioninfo associated with append
      key - Modified by this call; we add to it this edits region edit/sequence id.
      edits - Edits to append. MAY CONTAIN NO EDITS for case where we want to get an edit sequence id that is after all currently appended edits.
      Returns:
      Returns a 'transaction id' and key will have the region edit/sequence id in it.
      Throws:
      IOException
      See Also:

    • updateStore

      void updateStore(byte[] encodedRegionName, byte[] familyName, Long sequenceid, boolean onlyIfGreater)
      updates the seuence number of a specific store. depending on the flag: replaces current seq number if the given seq id is bigger, or even if it is lower than existing one

    • sync

      void sync() throws IOException
      Sync what we have in the WAL.
      Throws:
      when - timeout, it would throw WALSyncTimeoutIOException.
      IOException

    • sync

      void sync(long txid) throws IOException
      Sync the WAL if the txId was not already sync'd.
      Parameters:
      txid - Transaction id to sync to.
      Throws:
      when - timeout, it would throw WALSyncTimeoutIOException.
      IOException

    • sync

      default void sync(boolean forceSync) throws IOException
      Parameters:
      forceSync - Flag to force sync rather than flushing to the buffer. Example - Hadoop hflush vs hsync.
      Throws:
      when - timeout, it would throw WALSyncTimeoutIOException.
      IOException

    • sync

      default void sync(long txid, boolean forceSync) throws IOException
      Parameters:
      txid - Transaction id to sync to.
      forceSync - Flag to force sync rather than flushing to the buffer. Example - Hadoop hflush vs hsync.
      Throws:
      when - timeout, it would throw WALSyncTimeoutIOException.
      IOException

    • startCacheFlush

      Long startCacheFlush(byte[] encodedRegionName, Set<byte[]> families)
      WAL keeps track of the sequence numbers that are as yet not flushed im memstores in order to be able to do accounting to figure which WALs can be let go. This method tells WAL that some region is about to flush. The flush can be the whole region or for a column family of the region only.

      Currently, it is expected that the update lock is held for the region; i.e. no concurrent appends while we set up cache flush.

      Parameters:
      families - Families to flush. May be a subset of all families in the region.
      Returns:
      Returns HConstants.NO_SEQNUM if we are flushing the whole region OR if we are flushing a subset of all families but there are no edits in those families not being flushed; in other words, this is effectively same as a flush of all of the region though we were passed a subset of regions. Otherwise, it returns the sequence id of the oldest/lowest outstanding edit.
      See Also:

    • startCacheFlush

      Long startCacheFlush(byte[] encodedRegionName, Map<byte[],Long> familyToSeq)

    • completeCacheFlush

      void completeCacheFlush(byte[] encodedRegionName, long maxFlushedSeqId)
      Complete the cache flush.
      Parameters:
      encodedRegionName - Encoded region name.
      maxFlushedSeqId - The maxFlushedSeqId for this flush. There is no edit in memory that is less that this sequence id.
      See Also:

    • abortCacheFlush

      void abortCacheFlush(byte[] encodedRegionName)
      Abort a cache flush. Call if the flush fails. Note that the only recovery for an aborted flush currently is a restart of the regionserver so the snapshot content dropped by the failure gets restored to the memstore.
      Parameters:
      encodedRegionName - Encoded region name.

    • getCoprocessorHost

      WALCoprocessorHost getCoprocessorHost()
      Returns Coprocessor host.

    • getEarliestMemStoreSeqNum

      long getEarliestMemStoreSeqNum(byte[] encodedRegionName, byte[] familyName)
      Gets the earliest unflushed sequence id in the memstore for the store.
      Parameters:
      encodedRegionName - The region to get the number for.
      familyName - The family to get the number for.
      Returns:
      The earliest/lowest/oldest sequence id if present, HConstants.NO_SEQNUM if absent.

    • skipRemoteWAL

      default void skipRemoteWAL(boolean markerEditOnly)
      Tell the WAL that when creating new writer you can skip creating the remote writer.

      Used by sync replication for switching states from ACTIVE, where the remote cluster is broken.

    • toString

      String toString()
      Human readable identifying information about the state of this WAL. Implementors are encouraged to include information appropriate for debugging. Consumers are advised not to rely on the details of the returned String; it does not have a defined structure.
      Overrides:
      toString in class Object