Package org.apache.hadoop.hbase.wal

Class IOTestProvider

java.lang.Object
org.apache.hadoop.hbase.wal.IOTestProvider
All Implemented Interfaces:
org.apache.hadoop.hbase.wal.WALProvider
@Private public class IOTestProvider extends Object implements org.apache.hadoop.hbase.wal.WALProvider
A WAL Provider that returns a single thread safe WAL that optionally can skip parts of our normal interactions with HDFS.

This implementation picks a directory in HDFS based on the same mechanisms as the FSHLogProvider. Users can configure how much interaction we have with HDFS with the configuration property "hbase.wal.iotestprovider.operations". The value should be a comma separated list of allowed operations:

  • append : edits will be written to the underlying filesystem
  • sync : wal syncs will result in hflush calls
  • fileroll : roll requests will result in creating a new file on the underlying filesystem.
Additionally, the special cases "all" and "none" are recognized. If ommited, the value defaults to "all." Behavior is undefined if "all" or "none" are paired with additional values. Behavior is also undefined if values not listed above are included.

Only those operations listed will occur between the returned WAL and HDFS. All others will be no-ops.

Note that in the case of allowing "append" operations but not allowing "fileroll", the returned WAL will just keep writing to the same file. This won't avoid all costs associated with file management over time, becaue the data set size may result in additional HDFS block allocations.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    private static enum 
    IOTestProvider.AllowedOperations
     
    private static class 
    IOTestProvider.IOTestWAL
     
    private static class 
    IOTestProvider.IOTestWriter
    Presumes init will be called by a single thread prior to any access of other methods.

    Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.wal.WALProvider

    org.apache.hadoop.hbase.wal.WALProvider.AsyncWriter, org.apache.hadoop.hbase.wal.WALProvider.Writer, org.apache.hadoop.hbase.wal.WALProvider.WriterBase

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final String
    ALLOWED_OPERATIONS
     
    private org.apache.hadoop.conf.Configuration
    conf
     
    private org.apache.hadoop.hbase.wal.WALFactory
    factory
     
    protected AtomicBoolean
    initialized
     
    private List<org.apache.hadoop.hbase.regionserver.wal.WALActionsListener>
    listeners
     
    private org.apache.hadoop.hbase.regionserver.wal.FSHLog
    log
     
    private static final org.slf4j.Logger
    LOG
     
    private String
    providerId
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    IOTestProvider()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addWALActionsListener(org.apache.hadoop.hbase.regionserver.wal.WALActionsListener listener)
     
    void
    close()
     
    private org.apache.hadoop.hbase.regionserver.wal.FSHLog
    createWAL()
     
    long
    getLogFileSize()
     
    long
    getNumLogFiles()
     
    org.apache.hadoop.hbase.wal.WAL
    getWAL(org.apache.hadoop.hbase.client.RegionInfo region)
     
    List<org.apache.hadoop.hbase.wal.WAL>
    getWALs()
     
    void
    init(org.apache.hadoop.hbase.wal.WALFactory factory, org.apache.hadoop.conf.Configuration conf, String providerId, org.apache.hadoop.hbase.Abortable abortable)
     
    void
    shutdown()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.hbase.wal.WALProvider

    getWALFileLengthProvider

  • Field Details

  • Constructor Details

  • Method Details

    • init

      public void init(org.apache.hadoop.hbase.wal.WALFactory factory, org.apache.hadoop.conf.Configuration conf, String providerId, org.apache.hadoop.hbase.Abortable abortable) throws IOException
      Specified by:
      init in interface org.apache.hadoop.hbase.wal.WALProvider
      Parameters:
      factory - factory that made us, identity used for FS layout. may not be null
      conf - may not be null
      providerId - differentiate between providers from one facotry, used for FS layout. may be null
      Throws:
      IOException

    • getWALs

      public List<org.apache.hadoop.hbase.wal.WAL> getWALs()
      Specified by:
      getWALs in interface org.apache.hadoop.hbase.wal.WALProvider

    • createWAL

      private org.apache.hadoop.hbase.regionserver.wal.FSHLog createWAL() throws IOException
      Throws:
      IOException

    • getWAL

      public org.apache.hadoop.hbase.wal.WAL getWAL(org.apache.hadoop.hbase.client.RegionInfo region) throws IOException
      Specified by:
      getWAL in interface org.apache.hadoop.hbase.wal.WALProvider
      Throws:
      IOException

    • close

      public void close() throws IOException
      Specified by:
      close in interface org.apache.hadoop.hbase.wal.WALProvider
      Throws:
      IOException

    • shutdown

      public void shutdown() throws IOException
      Specified by:
      shutdown in interface org.apache.hadoop.hbase.wal.WALProvider
      Throws:
      IOException

    • getNumLogFiles

      public long getNumLogFiles()
      Specified by:
      getNumLogFiles in interface org.apache.hadoop.hbase.wal.WALProvider

    • getLogFileSize

      public long getLogFileSize()
      Specified by:
      getLogFileSize in interface org.apache.hadoop.hbase.wal.WALProvider

    • addWALActionsListener

      public void addWALActionsListener(org.apache.hadoop.hbase.regionserver.wal.WALActionsListener listener)
      Specified by:
      addWALActionsListener in interface org.apache.hadoop.hbase.wal.WALProvider