Package org.apache.hadoop.hbase.master.snapshot

Class MasterSnapshotVerifier

java.lang.Object
org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier
@Private @Unstable public final class MasterSnapshotVerifier extends Object
General snapshot verification on the master.

This is a light-weight verification mechanism for all the files in a snapshot. It doesn't attempt to verify that the files are exact copies (that would be paramount to taking the snapshot again!), but instead just attempts to ensure that the files match the expected files and are the same length.

Taking an online snapshots can race against other operations and this is an last line of defense. For example, if meta changes between when snapshots are taken not all regions of a table may be present. This can be caused by a region split (daughters present on this scan, but snapshot took parent), or move (snapshots only checks lists of region servers, a move could have caused a region to be skipped or done twice).

Current snapshot files checked:

  1. SnapshotDescription is readable
  2. Table info is readable
  3. Regions
  • Matching regions in the snapshot as currently in the table
  • RegionInfo matches the current and stored regions
  • All referenced hfiles have valid names
  • All the hfiles are present (either in .archive directory in the region)
  • All recovered.edits files are present (by name) and have the correct file size

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final org.slf4j.Logger
    LOG
     
    private MasterServices
    services
     
    private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription
    snapshot
     
    private TableName
    tableName
     
    private org.apache.hadoop.fs.FileSystem
    workingDirFs
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    MasterSnapshotVerifier(MasterServices services, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.FileSystem workingDirFs)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    private void
    verifyRegionInfo(RegionInfo region, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotRegionManifest manifest)
    Verify that the regionInfo is valid
    private void
    verifyRegions(SnapshotManifest manifest, boolean verifyRegions)
    Check that all the regions in the snapshot are valid, and accounted for.
    void
    verifySnapshot(org.apache.hadoop.fs.Path snapshotDir, boolean verifyRegions)
    Verify that the snapshot in the directory is a valid snapshot
    private void
    verifySnapshotDescription(org.apache.hadoop.fs.Path snapshotDir)
    Check that the snapshot description written in the filesystem matches the current snapshot
    private void
    verifyTableInfo(SnapshotManifest manifest)
    Check that the table descriptor for the snapshot is a valid table descriptor

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • MasterSnapshotVerifier

      public MasterSnapshotVerifier(MasterServices services, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.FileSystem workingDirFs)
      Parameters:
      services - services for the master
      snapshot - snapshot to check
      workingDirFs - the file system containing the temporary snapshot information

  • Method Details

    • verifySnapshot

      public void verifySnapshot(org.apache.hadoop.fs.Path snapshotDir, boolean verifyRegions) throws CorruptedSnapshotException, IOException
      Verify that the snapshot in the directory is a valid snapshot
      Parameters:
      snapshotDir - snapshot directory to check
      Throws:
      CorruptedSnapshotException - if the snapshot is invalid
      IOException - if there is an unexpected connection issue to the filesystem

    • verifySnapshotDescription

      private void verifySnapshotDescription(org.apache.hadoop.fs.Path snapshotDir) throws CorruptedSnapshotException
      Check that the snapshot description written in the filesystem matches the current snapshot
      Parameters:
      snapshotDir - snapshot directory to check
      Throws:
      CorruptedSnapshotException

    • verifyTableInfo

      private void verifyTableInfo(SnapshotManifest manifest) throws IOException
      Check that the table descriptor for the snapshot is a valid table descriptor
      Parameters:
      manifest - snapshot manifest to inspect
      Throws:
      IOException

    • verifyRegions

      private void verifyRegions(SnapshotManifest manifest, boolean verifyRegions) throws IOException
      Check that all the regions in the snapshot are valid, and accounted for.
      Parameters:
      manifest - snapshot manifest to inspect
      Throws:
      IOException - if we can't reach hbase:meta or read the files from the FS

    • verifyRegionInfo

      private void verifyRegionInfo(RegionInfo region, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotRegionManifest manifest) throws IOException
      Verify that the regionInfo is valid
      Parameters:
      region - the region to check
      manifest - snapshot manifest to inspect
      Throws:
      IOException