Package org.apache.hadoop.hbase.regionserver

Class HMobStore

java.lang.Object
org.apache.hadoop.hbase.regionserver.HStore
org.apache.hadoop.hbase.regionserver.HMobStore
All Implemented Interfaces:
ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Store, StoreConfigInformation
@Private public class HMobStore extends HStore
The store implementation to save MOBs (medium objects), it extends the HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek to HBase is done. The store implements how we save MOBs by extending HStore. When a descriptor of a column family has the value "IS_MOB", it means this column family is a mob one. When a HRegion instantiate a store for this column family, the HMobStore is created. HMobStore is almost the same with the HStore except using different types of scanners. In the method of getScanner, the MobStoreScanner and MobReversedStoreScanner are returned. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.

  • Field Details

  • Constructor Details

  • Method Details

    • getConfiguration

      public org.apache.hadoop.conf.Configuration getConfiguration()
      Gets current config.

    • createScanner

      protected KeyValueScanner createScanner(Scan scan, ScanInfo scanInfo, NavigableSet<byte[]> targetCols, long readPt) throws IOException
      Gets the MobStoreScanner or MobReversedStoreScanner. In these scanners, a additional seeks in the mob files should be performed after the seek in HBase is done.
      Overrides:
      createScanner in class HStore
      Throws:
      IOException

    • createStoreEngine

      protected StoreEngine<?,?,?,?> createStoreEngine(HStore store, org.apache.hadoop.conf.Configuration conf, CellComparator cellComparator) throws IOException
      Creates the mob store engine.
      Overrides:
      createStoreEngine in class HStore
      Parameters:
      store - The store. An unfortunate dependency needed due to it being passed to coprocessors via the compactor.
      conf - Store configuration.
      cellComparator - KVComparator for storeFileManager.
      Returns:
      StoreEngine to use.
      Throws:
      IOException

    • getTempDir

      private org.apache.hadoop.fs.Path getTempDir()
      Gets the temp directory.
      Returns:
      The temp directory.

    • createWriterInTmp

      public StoreFileWriter createWriterInTmp(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction) throws IOException
      Creates the writer for the mob file in temp directory.
      Parameters:
      date - The latest date of written cells.
      maxKeyCount - The key count.
      compression - The compression algorithm.
      startKey - The start key.
      isCompaction - If the writer is used in compaction.
      Returns:
      The writer for the mob file.
      Throws:
      IOException

    • createWriter

      public StoreFileWriter createWriter(Date date, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) throws IOException
      Creates the writer for the mob file in the mob family directory.
      Parameters:
      date - The latest date of written cells.
      maxKeyCount - The key count.
      compression - The compression algorithm.
      startKey - The start key.
      isCompaction - If the writer is used in compaction.
      Returns:
      The writer for the mob file.
      Throws:
      IOException

    • createWriterInTmp

      public StoreFileWriter createWriterInTmp(String date, org.apache.hadoop.fs.Path basePath, long maxKeyCount, Compression.Algorithm compression, byte[] startKey, boolean isCompaction, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) throws IOException
      Creates the writer for the mob file in temp directory.
      Parameters:
      date - The date string, its format is yyyymmmdd.
      basePath - The basic path for a temp directory.
      maxKeyCount - The key count.
      compression - The compression algorithm.
      startKey - The start key.
      isCompaction - If the writer is used in compaction.
      Returns:
      The writer for the mob file.
      Throws:
      IOException

    • createWriterInTmp

      public StoreFileWriter createWriterInTmp(MobFileName mobFileName, org.apache.hadoop.fs.Path basePath, long maxKeyCount, Compression.Algorithm compression, boolean isCompaction, Consumer<org.apache.hadoop.fs.Path> writerCreationTracker) throws IOException
      Creates the writer for the mob file in temp directory.
      Parameters:
      mobFileName - The mob file name.
      basePath - The basic path for a temp directory.
      maxKeyCount - The key count.
      compression - The compression algorithm.
      isCompaction - If the writer is used in compaction.
      Returns:
      The writer for the mob file.
      Throws:
      IOException

    • commitFile

      public void commitFile(org.apache.hadoop.fs.Path sourceFile, org.apache.hadoop.fs.Path targetPath) throws IOException
      Commits the mob file.
      Parameters:
      sourceFile - The source file.
      targetPath - The directory path where the source file is renamed to.
      Throws:
      IOException

    • validateMobFile

      private void validateMobFile(org.apache.hadoop.fs.Path path) throws IOException
      Validates a mob file by opening and closing it.
      Parameters:
      path - the path to the mob file
      Throws:
      IOException

    • resolve

      public MobCell resolve(Cell reference, boolean cacheBlocks) throws IOException
      Reads the cell from the mob file, and the read point does not count. This is used for DefaultMobStoreCompactor where we can read empty value for the missing cell.
      Parameters:
      reference - The cell found in the HBase, its value is a path to a mob file.
      cacheBlocks - Whether the scanner should cache blocks.
      Returns:
      The cell found in the mob file.
      Throws:
      IOException

    • resolve

      public MobCell resolve(Cell reference, boolean cacheBlocks, boolean readEmptyValueOnMobCellMiss) throws IOException
      Reads the cell from the mob file with readEmptyValueOnMobCellMiss
      Parameters:
      reference - The cell found in the HBase, its value is a path to a mob file.
      cacheBlocks - Whether the scanner should cache blocks.
      readEmptyValueOnMobCellMiss - should return empty mob cell if reference can not be resolved.
      Returns:
      The cell found in the mob file.
      Throws:
      IOException

    • resolve

      public MobCell resolve(Cell reference, boolean cacheBlocks, long readPt, boolean readEmptyValueOnMobCellMiss) throws IOException
      Reads the cell from the mob file.
      Parameters:
      reference - The cell found in the HBase, its value is a path to a mob file.
      cacheBlocks - Whether the scanner should cache blocks.
      readPt - the read point.
      readEmptyValueOnMobCellMiss - Whether return null value when the mob file is missing or corrupt.
      Returns:
      The cell found in the mob file.
      Throws:
      IOException

    • getLocations

      public List<org.apache.hadoop.fs.Path> getLocations(TableName tableName) throws IOException
      Parameters:
      tableName - to look up locations for, can not be null
      Returns:
      a list of location in order of working dir, archive dir. will not be null.
      Throws:
      IOException

    • readCell

      private MobCell readCell(List<org.apache.hadoop.fs.Path> locations, String fileName, Cell search, boolean cacheMobBlocks, long readPt, boolean readEmptyValueOnMobCellMiss) throws IOException
      Reads the cell from a mob file. The mob file might be located in different directories. 1. The working directory. 2. The archive directory. Reads the cell from the files located in both of the above directories.
      Parameters:
      locations - The possible locations where the mob files are saved.
      fileName - The file to be read.
      search - The cell to be searched.
      cacheMobBlocks - Whether the scanner should cache blocks.
      readPt - the read point.
      readEmptyValueOnMobCellMiss - Whether return null value when the mob file is missing or corrupt.
      Returns:
      The found cell. Null if there's no such a cell.
      Throws:
      IOException

    • getPath

      public org.apache.hadoop.fs.Path getPath()
      Gets the mob file path.
      Returns:
      The mob file path.

    • updateCellsCountCompactedToMob

      public void updateCellsCountCompactedToMob(long count)

    • getCellsCountCompactedToMob

      public long getCellsCountCompactedToMob()

    • updateCellsCountCompactedFromMob

      public void updateCellsCountCompactedFromMob(long count)

    • getCellsCountCompactedFromMob

      public long getCellsCountCompactedFromMob()

    • updateCellsSizeCompactedToMob

      public void updateCellsSizeCompactedToMob(long size)

    • getCellsSizeCompactedToMob

      public long getCellsSizeCompactedToMob()

    • updateCellsSizeCompactedFromMob

      public void updateCellsSizeCompactedFromMob(long size)

    • getCellsSizeCompactedFromMob

      public long getCellsSizeCompactedFromMob()

    • updateMobFlushCount

      public void updateMobFlushCount()

    • getMobFlushCount

      public long getMobFlushCount()

    • updateMobFlushedCellsCount

      public void updateMobFlushedCellsCount(long count)

    • getMobFlushedCellsCount

      public long getMobFlushedCellsCount()

    • updateMobFlushedCellsSize

      public void updateMobFlushedCellsSize(long size)

    • getMobFlushedCellsSize

      public long getMobFlushedCellsSize()

    • updateMobScanCellsCount

      public void updateMobScanCellsCount(long count)

    • getMobScanCellsCount

      public long getMobScanCellsCount()

    • updateMobScanCellsSize

      public void updateMobScanCellsSize(long size)

    • getMobScanCellsSize

      public long getMobScanCellsSize()

    • getRefCellTags

      public byte[] getRefCellTags()