Package org.apache.hadoop.hbase.regionserver.wal

Class WALUtil

java.lang.Object
org.apache.hadoop.hbase.regionserver.wal.WALUtil
@Private public class WALUtil extends Object
Helper methods to ease Region Server integration with the Write Ahead Log (WAL). Note that methods in this class specifically should not require access to anything other than the API found in WAL. For internal use only.

  • Field Details

  • Constructor Details

  • Method Details

    • writeCompactionMarker

      public static WALKeyImpl writeCompactionMarker(WAL wal, NavigableMap<byte[],Integer> replicationScope, RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c, MultiVersionConcurrencyControl mvcc) throws IOException
      Write the marker that a compaction has succeeded and is about to be committed. This provides info to the HMaster to allow it to recover the compaction if this regionserver dies in the middle. It also prevents the compaction from finishing if this regionserver has already lost its lease on the log.

      This write is for internal use only. Not for external client consumption.

      Parameters:
      mvcc - Used by WAL to get sequence Id for the waledit.
      Throws:
      IOException

    • writeFlushMarker

      public static WALKeyImpl writeFlushMarker(WAL wal, NavigableMap<byte[],Integer> replicationScope, RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f, boolean sync, MultiVersionConcurrencyControl mvcc) throws IOException
      Write a flush marker indicating a start / abort or a complete of a region flush

      This write is for internal use only. Not for external client consumption.

      Throws:
      IOException

    • writeRegionEventMarker

      public static WALKeyImpl writeRegionEventMarker(WAL wal, NavigableMap<byte[],Integer> replicationScope, RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor r, MultiVersionConcurrencyControl mvcc) throws IOException
      Write a region open marker indicating that the region is opened. This write is for internal use only. Not for external client consumption.
      Throws:
      IOException

    • writeBulkLoadMarkerAndSync

      public static WALKeyImpl writeBulkLoadMarkerAndSync(WAL wal, NavigableMap<byte[],Integer> replicationScope, RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor desc, MultiVersionConcurrencyControl mvcc) throws IOException
      Write a log marker that a bulk load has succeeded and is about to be committed. This write is for internal use only. Not for external client consumption.
      Parameters:
      wal - The log to write into.
      replicationScope - The replication scope of the families in the HRegion
      hri - A description of the region in the table that we are bulk loading into.
      desc - A protocol buffers based description of the client's bulk loading request
      Returns:
      walKey with sequenceid filled out for this bulk load marker
      Throws:
      IOException - We will throw an IOException if we can not append to the HLog.

    • writeMarker

      private static WALKeyImpl writeMarker(WAL wal, NavigableMap<byte[],Integer> replicationScope, RegionInfo hri, WALEdit edit, MultiVersionConcurrencyControl mvcc, Map<String,byte[]> extendedAttributes) throws IOException
      Throws:
      IOException

    • doFullMarkerAppendTransaction

      private static WALKeyImpl doFullMarkerAppendTransaction(WAL wal, NavigableMap<byte[],Integer> replicationScope, RegionInfo hri, WALEdit edit, MultiVersionConcurrencyControl mvcc, Map<String,byte[]> extendedAttributes, boolean sync) throws IOException
      A 'full' WAL transaction involves starting an mvcc transaction followed by an append, an optional sync, and then a call to complete the mvcc transaction. This method does it all. Good for case of adding a single edit or marker to the WAL.

      This write is for internal use only. Not for external client consumption.

      Returns:
      WALKeyImpl that was added to the WAL.
      Throws:
      IOException

    • getWALBlockSize

      public static long getWALBlockSize(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir) throws IOException
      Blocksize returned here is 2x the default HDFS blocksize unless explicitly set in Configuration. Works in tandem with hbase.regionserver.logroll.multiplier. See comment in AbstractFSWAL in Constructor where we set blocksize and logrollsize for why.
      Returns:
      Blocksize to use writing WALs.
      Throws:
      IOException

    • getWALBlockSize

      public static long getWALBlockSize(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, boolean isRecoverEdits) throws IOException
      Public because of FSHLog. Should be package-private
      Parameters:
      isRecoverEdits - the created writer is for recovered edits or WAL. For recovered edits, it is true and for WAL it is false.
      Throws:
      IOException

    • filterCells

      public static void filterCells(WALEdit edit, Function<Cell,Cell> mapper)

    • writeReplicationMarkerAndSync

      public static void writeReplicationMarkerAndSync(WAL wal, MultiVersionConcurrencyControl mvcc, RegionInfo regionInfo, byte[] rowKey, long timestamp) throws IOException
      Throws:
      IOException