Package org.apache.hadoop.hbase.wal

Class AbstractFSWALProvider<T extends AbstractFSWAL<?>>

java.lang.Object
org.apache.hadoop.hbase.wal.AbstractFSWALProvider<T>
All Implemented Interfaces:
WALProvider
Direct Known Subclasses:
AsyncFSWALProvider, FSHLogProvider
@Private @Evolving public abstract class AbstractFSWALProvider<T extends AbstractFSWAL<?>> extends Object implements WALProvider
Base class of a WAL Provider that returns a single thread safe WAL that writes to Hadoop FS. By default, this implementation picks a directory in Hadoop FS based on a combination of
  • the HBase root directory
  • HConstants.HREGION_LOGDIR_NAME
  • the given factory's factoryId (usually identifying the regionserver by host:port)
It also uses the providerId to differentiate among files.

  • Field Details

  • Constructor Details

  • Method Details

    • init

      public void init(WALFactory factory, org.apache.hadoop.conf.Configuration conf, String providerId, Abortable abortable) throws IOException
      Description copied from interface: WALProvider
      Set up the provider to create wals. will only be called once per instance.
      Specified by:
      init in interface WALProvider
      Parameters:
      factory - factory that made us, identity used for FS layout. may not be null
      conf - may not be null
      providerId - differentiate between providers from one factory, used for FS layout. may be null
      Throws:
      IOException

    • getWALs

      public List<WAL> getWALs()
      Description copied from interface: WALProvider
      Returns the List of WALs that are used by this server
      Specified by:
      getWALs in interface WALProvider

    • getWAL

      public T getWAL(RegionInfo region) throws IOException
      Specified by:
      getWAL in interface WALProvider
      Parameters:
      region - the region which we want to get a WAL for it. Could be null.
      Returns:
      a WAL for writing entries for the given region.
      Throws:
      IOException

    • createWAL

      protected abstract T createWAL() throws IOException
      Throws:
      IOException

    • doInit

      protected abstract void doInit(org.apache.hadoop.conf.Configuration conf) throws IOException
      Throws:
      IOException

    • shutdown

      public void shutdown() throws IOException
      Description copied from interface: WALProvider
      persist outstanding WALs to storage and stop accepting new appends. This method serves as shorthand for sending a sync to every WAL provided by a given implementation. Those WALs will also stop accepting new writes.
      Specified by:
      shutdown in interface WALProvider
      Throws:
      IOException

    • close

      public void close() throws IOException
      Description copied from interface: WALProvider
      shutdown utstanding WALs and clean up any persisted state. Call this method only when you will not need to replay any of the edits to the WALs from this provider. After this call completes, the underlying resources should have been reclaimed.
      Specified by:
      close in interface WALProvider
      Throws:
      IOException

    • getNumLogFiles

      public long getNumLogFiles()
      iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the number of files (rolled and active). if either of them aren't, count 0 for that provider.
      Specified by:
      getNumLogFiles in interface WALProvider

    • getLogFileSize

      public long getLogFileSize()
      iff the given WALFactory is using the DefaultWALProvider for meta and/or non-meta, count the size of files (only rolled). if either of them aren't, count 0 for that provider.
      Specified by:
      getLogFileSize in interface WALProvider

    • getNumRolledLogFiles

      public static int getNumRolledLogFiles(WAL wal)
      returns the number of rolled WAL files.

    • getLogFileSize

      public static long getLogFileSize(WAL wal)
      returns the size of rolled WAL files.

    • getCurrentFileName

      public static org.apache.hadoop.fs.Path getCurrentFileName(WAL wal)
      return the current filename from the current wal.

    • requestLogRoll

      static void requestLogRoll(WAL wal)
      request a log roll, but don't actually do it.

    • extractFileNumFromWAL

      public static long extractFileNumFromWAL(WAL wal)
      It returns the file create timestamp from the file name. For name format see validateWALFilename(String) public until remaining tests move to o.a.h.h.wal
      Parameters:
      wal - must not be null
      Returns:
      the file number that is part of the WAL file name

    • validateWALFilename

      public static boolean validateWALFilename(String filename)
      A WAL file name is of the format: <wal-name>WAL_FILE_NAME_DELIMITER <file-creation-timestamp>[.meta]. provider-name is usually made up of a server-name and a provider-id
      Parameters:
      filename - name of the file to validate
      Returns:
      true if the filename matches an WAL, false otherwise

    • getWALDirectoryName

      public static String getWALDirectoryName(String serverName)
      Construct the directory name for all WALs on a given server. Dir names currently look like this for WALs: hbase//WALs/kalashnikov.att.net,61634,1486865297088.
      Parameters:
      serverName - Server name formatted as described in ServerName
      Returns:
      the relative WAL directory name, e.g. .logs/1.example.org,60030,12345 if serverName passed is 1.example.org,60030,12345

    • getWALArchiveDirectoryName

      public static String getWALArchiveDirectoryName(org.apache.hadoop.conf.Configuration conf, String serverName)
      Construct the directory name for all old WALs on a given server. The default old WALs dir looks like: hbase/oldWALs. If you config hbase.separate.oldlogdir.by.regionserver to true, it looks like hbase//oldWALs/kalashnikov.att.net,61634,1486865297088.
      Parameters:
      serverName - Server name formatted as described in ServerName
      Returns:
      the relative WAL directory name

    • getServerNameFromWALDirectoryName

      public static ServerName getServerNameFromWALDirectoryName(org.apache.hadoop.conf.Configuration conf, String path) throws IOException
      Pulls a ServerName out of a Path generated according to our layout rules. In the below layouts, this method ignores the format of the logfile component. Current format: [base directory for hbase]/hbase/.logs/ServerName/logfile or [base directory for hbase]/hbase/.logs/ServerName-splitting/logfile Expected to work for individual log files and server-specific directories.
      Returns:
      null if it's not a log file. Returns the ServerName of the region server that created this log file otherwise.
      Throws:
      IOException

    • getServerNameFromWALDirectoryName

      public static ServerName getServerNameFromWALDirectoryName(org.apache.hadoop.fs.Path logFile)
      This function returns region server name from a log file name which is in one of the following formats:
      • hdfs://<name node>/hbase/.logs/<server name>-splitting/...
      • hdfs://<name node>/hbase/.logs/<server name>/...
      Returns:
      null if the passed in logFile isn't a valid WAL file path

    • isMetaFile

      public static boolean isMetaFile(org.apache.hadoop.fs.Path p)

    • isMetaFile

      public static boolean isMetaFile(String p)
      Returns True if String ends in META_WAL_PROVIDER_ID

    • isArchivedLogFile

      public static boolean isArchivedLogFile(org.apache.hadoop.fs.Path p)

    • findArchivedLog

      public static org.apache.hadoop.fs.Path findArchivedLog(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) throws IOException
      Find the archived WAL file path if it is not able to locate in WALs dir.
      Parameters:
      path - - active WAL file path
      conf - - configuration
      Returns:
      archived path if exists, null - otherwise
      Throws:
      IOException - exception

    • recoverLease

      public static void recoverLease(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path path)

    • addWALActionsListener

      public void addWALActionsListener(WALActionsListener listener)
      Description copied from interface: WALProvider
      Add a WALActionsListener.

      Notice that you must call this method before calling WALProvider.getWAL(RegionInfo) as this method will not effect the WAL which has already been created. And as long as we can only it when initialization, it is not thread safe.

      Specified by:
      addWALActionsListener in interface WALProvider

    • getWALPrefixFromWALName

      public static String getWALPrefixFromWALName(String name)
      Get prefix of the log from its name, assuming WAL name in format of log_prefix.filenumber.log_suffix
      Parameters:
      name - Name of the WAL to parse
      Returns:
      prefix of the log
      See Also: