Package org.apache.hadoop.hbase.util

Class FSTableDescriptors

java.lang.Object
org.apache.hadoop.hbase.util.FSTableDescriptors
All Implemented Interfaces:
TableDescriptors
@Private public class FSTableDescriptors extends Object implements TableDescriptors
Implementation of TableDescriptors that reads descriptors from the passed filesystem. It expects descriptors to be in a file in the TABLEINFO_DIR subdir of the table's directory in FS. Can be read-only -- i.e. does not modify the filesystem or can be read and write.

Also has utility for keeping up the table descriptors tableinfo file. The table schema file is kept in the TABLEINFO_DIR subdir of the table directory in the filesystem. It has a TABLEINFO_FILE_PREFIX and then a suffix that is the edit sequenceid: e.g. .tableinfo.0000000003. This sequenceid is always increasing. It starts at zero. The table schema file with the highest sequenceid has the most recent schema edit. Usually there is one file only, the most recent but there may be short periods where there are more than one file. Old files are eventually cleaned. Presumption is that there will not be lots of concurrent clients making table schema edits. If so, the below needs a bit of a reworking and perhaps some supporting api in hdfs.

  • Field Details

  • Constructor Details

    • FSTableDescriptors

      public FSTableDescriptors(org.apache.hadoop.conf.Configuration conf) throws IOException
      Construct a FSTableDescriptors instance using the hbase root dir of the given conf and the filesystem where that root dir lives. This instance can do write operations (is not read only).
      Throws:
      IOException

    • FSTableDescriptors

      public FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir)

    • FSTableDescriptors

      public FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir, boolean fsreadonly, boolean usecache)

  • Method Details

    • tryUpdateMetaTableDescriptor

      public static void tryUpdateMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf) throws IOException
      Throws:
      IOException

    • tryUpdateAndGetMetaTableDescriptor

      public static TableDescriptor tryUpdateAndGetMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir) throws IOException
      Throws:
      IOException

    • getTableFamilyDescForMeta

      public static ColumnFamilyDescriptor getTableFamilyDescForMeta(org.apache.hadoop.conf.Configuration conf)

    • getReplBarrierFamilyDescForMeta

      public static ColumnFamilyDescriptor getReplBarrierFamilyDescForMeta()

    • createMetaTableDescriptorBuilder

      public static TableDescriptorBuilder createMetaTableDescriptorBuilder(org.apache.hadoop.conf.Configuration conf) throws IOException
      Throws:
      IOException

    • isUsecache

      protected boolean isUsecache()

    • get

      @Nullable public TableDescriptor get(TableName tableName)
      Get the current table descriptor for the given table, or null if none exists.

      Uses a local cache of the descriptor but still checks the filesystem on each call if fsvisited is not true, i.e, we haven't done a full scan yet, to see if a newer file has been created since the cached one was read.

      Specified by:
      get in interface TableDescriptors

    • getAll

      public Map<String,TableDescriptor> getAll() throws IOException
      Returns a map from table name to table descriptor for all tables.
      Specified by:
      getAll in interface TableDescriptors
      Returns:
      Map of all descriptors.
      Throws:
      IOException

    • getByNamespace

      public Map<String,TableDescriptor> getByNamespace(String name) throws IOException
      Find descriptors by namespace.
      Specified by:
      getByNamespace in interface TableDescriptors
      Returns:
      Map of all descriptors.
      Throws:
      IOException
      See Also:

    • update

      public void update(TableDescriptor td, boolean cacheOnly) throws IOException
      Description copied from interface: TableDescriptors
      Add or update descriptor
      Specified by:
      update in interface TableDescriptors
      Parameters:
      td - Descriptor to set into TableDescriptors
      cacheOnly - only add the given htd to cache, without updating the storage. For example, when creating table, we will write the descriptor to fs when creating the fs layout, so we do not need to update the fs again.
      Throws:
      IOException

    • updateTableDescriptor

      org.apache.hadoop.fs.Path updateTableDescriptor(TableDescriptor td) throws IOException
      Throws:
      IOException

    • remove

      public TableDescriptor remove(TableName tablename) throws IOException
      Removes the table descriptor from the local cache and returns it. If not in read only mode, it also deletes the entire table directory(!) from the FileSystem.
      Specified by:
      remove in interface TableDescriptors
      Throws:
      IOException

    • isTableDir

      public static boolean isTableDir(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
      Check whether we have a valid TableDescriptor.
      Throws:
      IOException

    • getTableDir

      private org.apache.hadoop.fs.Path getTableDir(TableName tableName)
      Return the table directory in HDFS

    • formatTableInfoSequenceId

      private static String formatTableInfoSequenceId(int number)
      Parameters:
      number - Number to use as suffix.
      Returns:
      Returns zero-prefixed decimal version of passed number (Does absolute in case number is negative).

    • getTableInfoSequenceIdAndFileLength

      static FSTableDescriptors.SequenceIdAndFileLength getTableInfoSequenceIdAndFileLength(org.apache.hadoop.fs.Path p)
      Returns the current sequence id and file length or 0 if none found.
      Parameters:
      p - Path to a .tableinfo file.

    • getTableInfoFileName

      static String getTableInfoFileName(int sequenceId, byte[] content)
      Returns Name of tableinfo file.

    • getTableDescriptorFromFs

      public static TableDescriptor getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path hbaseRootDir, TableName tableName) throws IOException
      Returns the latest table descriptor for the given table directly from the file system if it exists, bypassing the local cache. Returns null if it's not found.
      Throws:
      IOException

    • getTableDescriptorFromFs

      public static TableDescriptor getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
      Returns the latest table descriptor for the table located at the given directory directly from the file system if it exists.
      Throws:
      IOException

    • deleteMalformedFile

      private static void deleteMalformedFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file) throws IOException
      Throws:
      IOException

    • getTableDescriptorFromFs

      private static Optional<Pair<org.apache.hadoop.fs.FileStatus,TableDescriptor>> getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, boolean readonly) throws IOException
      Throws:
      IOException

    • deleteTableDescriptors

      public static void deleteTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
      Throws:
      IOException

    • deleteTableDescriptorFiles

      private static void deleteTableDescriptorFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, int maxSequenceId) throws IOException
      Deletes files matching the table info file pattern within the given directory whose sequenceId is at most the given max sequenceId.
      Throws:
      IOException

    • writeTableDescriptor

      private static org.apache.hadoop.fs.Path writeTableDescriptor(org.apache.hadoop.fs.FileSystem fs, TableDescriptor td, org.apache.hadoop.fs.Path tableDir, org.apache.hadoop.fs.FileStatus currentDescriptorFile) throws IOException
      Attempts to write a new table descriptor to the given table's directory. It begins at the currentSequenceId + 1 and tries 10 times to find a new sequence number not already in use.

      Removes the current descriptor file if passed in.

      Returns:
      Descriptor file or null if we failed write.
      Throws:
      IOException

    • createTableDescriptor

      public boolean createTableDescriptor(TableDescriptor htd) throws IOException
      Create new TableDescriptor in HDFS. Happens when we are creating table. Used by tests.
      Returns:
      True if we successfully created file.
      Throws:
      IOException

    • createTableDescriptor

      public boolean createTableDescriptor(TableDescriptor htd, boolean forceCreation) throws IOException
      Create new TableDescriptor in HDFS. Happens when we are creating table. If forceCreation is true then even if previous table descriptor is present it will be overwritten
      Returns:
      True if we successfully created file.
      Throws:
      IOException

    • createTableDescriptorForTableDirectory

      public boolean createTableDescriptorForTableDirectory(org.apache.hadoop.fs.Path tableDir, TableDescriptor htd, boolean forceCreation) throws IOException
      Create a new TableDescriptor in HDFS in the specified table directory. Happens when we create a new table during cluster start or in Clone and Create Table Procedures. Checks readOnly flag passed on construction.
      Parameters:
      tableDir - table directory under which we should write the file
      htd - description of the table to write
      forceCreation - if true,then even if previous table descriptor is present it will be overwritten
      Returns:
      true if the we successfully created the file, false if the file already exists and we weren't forcing the descriptor creation.
      Throws:
      IOException - if a filesystem error occurs

    • createTableDescriptorForTableDirectory

      public static boolean createTableDescriptorForTableDirectory(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, TableDescriptor htd, boolean forceCreation) throws IOException
      Create a new TableDescriptor in HDFS in the specified table directory. Happens when we create a new table snapshoting. Does not enforce read-only. That is for caller to determine.
      Parameters:
      fs - Filesystem to use.
      tableDir - table directory under which we should write the file
      htd - description of the table to write
      forceCreation - if true,then even if previous table descriptor is present it will be overwritten
      Returns:
      true if the we successfully created the file, false if the file already exists and we weren't forcing the descriptor creation.
      Throws:
      IOException - if a filesystem error occurs