Class MetaTableAccessor
Read/write operations on hbase:meta
region as well as assignment information stored
to hbase:meta
.
Some of the methods of this class take ZooKeeperWatcher as a param. The only reason for this is when this class is used on client-side (e.g. HBaseAdmin), we want to use short-lived connection (opened before each operation, closed right after), while when used on HM or HRS (like in AssignmentManager) we want permanent connection.
HBASE-10070 adds a replicaId to HRI, meaning more than one HRI can be defined for the same table range (table, startKey, endKey). For every range, there will be at least one HRI defined which is called default replica.
Meta layout
For each table there is single row named for the table with a 'table' column family. The column family currently has one column in it, the 'state' column: table:state => contains table state Then for each table range ('Region'), there is a single row, formatted as: <tableName>,<startKey>,<regionId>,<encodedRegionName>. This row is the serialized regionName of the default region replica. Columns are: info:regioninfo => contains serialized HRI for the default region replica info:server => contains hostname:port (in string form) for the server hosting the default regionInfo replica info:server_<replicaId> => contains hostname:port (in string form) for the server hosting the regionInfo replica with replicaId info:serverstartcode => contains server start code (in binary long form) for the server hosting the default regionInfo replica info:serverstartcode_<replicaId> => contains server start code (in binary long form) for the server hosting the regionInfo replica with replicaId info:seqnumDuringOpen => contains seqNum (in binary long form) for the region at the time the server opened the region with default replicaId info:seqnumDuringOpen_<replicaId> => contains seqNum (in binary long form) for the region at the time the server opened the region with replicaId info:splitA => contains a serialized HRI for the first daughter region if the region is split info:splitB => contains a serialized HRI for the second daughter region if the region is split info:merge* => contains a serialized HRI for a merge parent region. There will be two or more of these columns in a row. A row that has these columns is undergoing a merge and is the result of the merge. Columns listed in marge* columns are the parents of this merged region. Example columns: info:merge0001, info:merge0002. You make also see 'mergeA', and 'mergeB'. This is old form replaced by the new format that allows for more than two parents to be merged at a time. TODO: Add rep_barrier for serial replication explaination. See SerialReplicationChecker.
The actual layout of meta should be encapsulated inside MetaTableAccessor methods, and should not leak out of it (through Result objects, etc)
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic interface
Implementations 'visit' a catalog table row but with close() at the end.(package private) static class
Collects all returned.(package private) static class
AMetaTableAccessor.Visitor
that collects content out of passedResult
.static class
A Visitor that skips offline regions and split parentsstatic enum
static final class
static class
A Visitor for a table.static interface
Implementations 'visit' a catalog table row. -
Field Summary
Modifier and TypeFieldDescriptionprivate static final byte
private static final org.slf4j.Logger
(package private) static final char
The delimiter for meta columns for replicaIds > 0private static final org.slf4j.Logger
static final byte[]
private static final byte
private static final Pattern
A regex for parsing server columns from meta. -
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionprivate static Put
addDaughtersToPut
(Put put, RegionInfo splitA, RegionInfo splitB) Adds split daughters to the Putstatic Put
addEmptyLocation
(Put p, int replicaId) static Put
addLocation
(Put p, ServerName sn, long openSeqNum, int replicaId) (package private) static Put
addMergeRegions
(Put put, Collection<RegionInfo> mergeRegions) static Put
addRegionInfo
(Put p, RegionInfo hri) private static Put
addRegionStateToPut
(Put put, int replicaId, RegionState.State state) static void
addRegionsToMeta
(Connection connection, List<RegionInfo> regionInfos, int regionReplication) Adds a hbase:meta row for each of the specified new regions.private static void
addRegionsToMeta
(Connection connection, List<RegionInfo> regionInfos, int regionReplication, long ts) Adds a hbase:meta row for each of the specified new regions.static void
addRegionToMeta
(Connection connection, RegionInfo regionInfo) Adds a (single) hbase:meta row for the specified new region and its daughters.static void
addReplicationBarrier
(Put put, long openSeqNum) See class comment on SerialReplicationCheckerprivate static void
addReplicationParent
(Put put, List<RegionInfo> parents) private static Put
addSequenceNum
(Put p, long openSeqNum, int replicaId) static void
addSplitsToParent
(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) Adds daughter region infos to hbase:meta row for the specified region.private static void
private static void
debugLogMutations
(List<? extends Mutation> mutations) private static void
deleteFromMetaTable
(Connection connection, List<Delete> deletes) Delete the passeddeletes
from thehbase:meta
table.private static void
deleteFromMetaTable
(Connection connection, Delete d) Delete the passedd
from thehbase:meta
table.static void
deleteMergeQualifiers
(Connection connection, RegionInfo mergeRegion) Deletes merge qualifiers for the specified merge region.static void
deleteRegionInfo
(Connection connection, RegionInfo regionInfo) Deletes the specified region from META.static void
deleteRegionInfos
(Connection connection, List<RegionInfo> regionsInfo) Deletes the specified regions from META.private static void
deleteRegionInfos
(Connection connection, List<RegionInfo> regionsInfo, long ts) Deletes the specified regions from META.static void
deleteTableState
(Connection connection, TableName table) Remove state for table from metafullScan
(Connection connection, MetaTableAccessor.QueryType type) Performs a full scan ofhbase:meta
.static void
fullScanMetaAndPrint
(Connection connection) fullScanRegions
(Connection connection) Performs a full scan ofhbase:meta
for regions.static void
fullScanRegions
(Connection connection, MetaTableAccessor.Visitor visitor) Performs a full scan ofhbase:meta
for regions.static void
fullScanTables
(Connection connection, MetaTableAccessor.Visitor visitor) Performs a full scan ofhbase:meta
for tables.private static Result
static List<RegionInfo>
getAllRegions
(Connection connection, boolean excludeOfflinedSplitParents) Lists all of the regions currently in META.static byte[]
Returns the column family used for meta columns.static Result
getCatalogFamilyRow
(Connection connection, RegionInfo ri) Returns Return theHConstants.CATALOG_FAMILY
row from hbase:meta.private static RegionInfo
getClosestRegionInfo
(Connection connection, TableName tableName, byte[] row) Returns Get closest metatable region row to passedrow
static PairOfSameType<RegionInfo>
getDaughterRegions
(Result data) Returns the daughter regions by reading the corresponding columns of the catalog table Result.private static List<RegionInfo>
getListOfRegionInfos
(List<Pair<RegionInfo, ServerName>> pairs) static List<RegionInfo>
getMergeRegions
(Cell[] cells) Returns Deserialized regioninfo values taken from column values that match the regex 'info:merge.*' in array ofcells
.static List<RegionInfo>
getMergeRegions
(Connection connection, RegionInfo regionInfo) Returns Return all regioninfos listed in the 'info:merge*' columns of theregionInfo
row.static Map<String,
RegionInfo> getMergeRegionsWithName
(Cell[] cells) Returns Deserialized values of <qualifier,regioninfo> pairs taken from column values that match the regex 'info:merge.*' in array ofcells
.static Table
getMetaHTable
(Connection connection) Callers should call close on the returnedTable
instance.static byte[]
getMetaKeyForRegion
(RegionInfo regionInfo) Returns the row key to use for this regionInfoprivate static Scan
getMetaScan
(org.apache.hadoop.conf.Configuration conf, int rowUpperLimit) static byte[]
getParentsBytes
(List<RegionInfo> parents) static Pair<RegionInfo,
ServerName> getRegion
(Connection connection, byte[] regionName) Deprecated.static RegionInfo
getRegionInfo
(Result data) Returns RegionInfo object from the column HConstants.CATALOG_FAMILY:HConstants.REGIONINFO_QUALIFIER of the catalog table Result.static RegionInfo
getRegionInfo
(Result r, byte[] qualifier) Returns the RegionInfo object from the columnHConstants.CATALOG_FAMILY
andqualifier
of the catalog table result.static byte[]
Returns the column qualifier for serialized region infostatic HRegionLocation
getRegionLocation
(Connection connection, byte[] regionName) Returns the HRegionLocation from meta for the given regionstatic HRegionLocation
getRegionLocation
(Connection connection, RegionInfo regionInfo) Returns the HRegionLocation from meta for the given regionprivate static HRegionLocation
getRegionLocation
(Result r, RegionInfo regionInfo, int replicaId) Returns the HRegionLocation parsed from the given meta row Result for the given regionInfo and replicaId.static RegionLocations
Returns an HRegionLocationList extracted from the result.static Result
getRegionResult
(Connection connection, RegionInfo regionInfo) Gets the result in hbase:meta for the specified region.private static byte[]
Returns the column qualifier for serialized region statestatic byte[]
getRegionStateColumn
(int replicaId) Returns the column qualifier for serialized region stateprivate static long
static long[]
getReplicationBarrier
(Connection conn, byte[] regionName) getReplicationBarrierResult
(Connection conn, TableName tableName, byte[] row, byte[] encodedRegionName) private static MetaTableAccessor.ReplicationBarrierResult
getReplicationBarrierResult
(Result result) static long[]
getReplicationBarriers
(Result result) static Scan
getScanForTableName
(org.apache.hadoop.conf.Configuration conf, TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table.static byte[]
getSeqNumColumn
(int replicaId) Returns the column qualifier for seqNum column for replicaIdprivate static long
getSeqNumDuringOpen
(Result r, int replicaId) The latest seqnum that the server writing to meta observed when opening the region.static byte[]
getServerColumn
(int replicaId) Returns the column qualifier for server column for replicaIdstatic ServerName
getServerName
(Result r, int replicaId) Returns aServerName
from catalog tableResult
.static byte[]
getServerNameColumn
(int replicaId) Returns the column qualifier for serialized region statestatic NavigableMap<RegionInfo,
Result> getServerUserRegions
(Connection connection, ServerName serverName) Get the user regions a given server is hosting.static byte[]
getStartCodeColumn
(int replicaId) Returns the column qualifier for server start code column for replicaIdgetTableEncodedRegionNameAndLastBarrier
(Connection conn, TableName tableName) getTableEncodedRegionNamesForSerialReplication
(Connection conn, TableName tableName) private static byte[]
Returns the column family used for table columns.static List<RegionInfo>
getTableRegions
(Connection connection, TableName tableName) Gets all of the regions of the specified table.static List<RegionInfo>
getTableRegions
(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) Gets all of the regions of the specified table.static List<Pair<RegionInfo,
ServerName>> getTableRegionsAndLocations
(Connection connection, TableName tableName) Do not use this method to get meta table regions, use methods in MetaTableLocator instead.static List<Pair<RegionInfo,
ServerName>> getTableRegionsAndLocations
(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) Do not use this method to get meta table regions, use methods in MetaTableLocator instead.static byte[]
getTableStartRowForMeta
(TableName tableName, MetaTableAccessor.QueryType type) Returns start row for scanning META according to query typestatic TableState
getTableState
(Connection conn, TableName tableName) Fetch table state for given table from META tablestatic TableState
Decode table state from META Result.private static byte[]
Returns the column qualifier for serialized table statestatic Map<TableName,
TableState> getTableStates
(Connection conn) Fetch table states from META tablestatic byte[]
getTableStopRowForMeta
(TableName tableName, MetaTableAccessor.QueryType type) Returns stop row for scanning META according to query typestatic ServerName
getTargetServerName
(Result r, int replicaId) Returns theServerName
from catalog tableResult
where the region is transitioning on.static boolean
hasMergeRegions
(Cell[] cells) Returns True if any merge regions present incells
; i.e.static boolean
hasMergeRegions
(Connection conn, RegionInfo regionInfo) Check whether the givenregionInfo
has any 'info:merge*' columns.private static boolean
isMergeQualifierPrefix
(Cell cell) Returns True if the column incell
matches the regex 'info:merge.*'.static Delete
makeDeleteFromRegionInfo
(RegionInfo regionInfo, long ts) Generates and returns a Delete containing the region info for the catalog tablestatic Put
makePutForReplicationBarrier
(RegionInfo regionInfo, long openSeqNum, long ts) static Put
makePutFromRegionInfo
(RegionInfo regionInfo) Generates and returns aPut
containing theRegionInfo
for the catalog table.static Put
makePutFromRegionInfo
(RegionInfo regionInfo, long ts) Generates and returns aPut
containing theRegionInfo
for the catalog table.static Put
makePutFromTableState
(TableState state, long ts) Construct PUT for given statestatic void
mergeRegions
(Connection connection, RegionInfo mergedRegion, Map<RegionInfo, Long> parentSeqNum, ServerName sn, int regionReplication) Merge regions into one in an atomic operation.(package private) static void
multiMutate
(Table table, byte[] row, List<Mutation> mutations) Performs an atomic multi-mutate operation against the given table.private static void
multiMutate
(Table table, byte[] row, Mutation... mutations) static void
overwriteRegions
(Connection connection, List<RegionInfo> regionInfos, int regionReplication) Overwrites the specified regions from hbase:meta.private static List<byte[]>
parseParentsBytes
(byte[] bytes) static RegionInfo
parseRegionInfoFromRegionName
(byte[] regionName) Returns an HRI parsed from this regionName.(package private) static int
parseReplicaIdFromServerColumn
(byte[] serverColumn) Parses the replicaId from the server column qualifier.private static void
static void
putsToMetaTable
(Connection connection, List<Put> ps) Put the passedps
to thehbase:meta
table.private static void
putToMetaTable
(Connection connection, Put p) Put the passedp
to thehbase:meta
table.static Result
scanByRegionEncodedName
(Connection connection, String regionEncodedName) Scans META table for a row whose key contains the specified regionEncodedName, returning a single relatedResult
instance if any row is found, null otherwise.(package private) static void
scanMeta
(Connection connection, byte[] startRow, byte[] stopRow, MetaTableAccessor.QueryType type, int maxRows, MetaTableAccessor.Visitor visitor) Performs a scan of META table.private static void
scanMeta
(Connection connection, byte[] startRow, byte[] stopRow, MetaTableAccessor.QueryType type, Filter filter, int maxRows, MetaTableAccessor.Visitor visitor, CatalogReplicaMode metaReplicaMode) private static void
scanMeta
(Connection connection, byte[] startRow, byte[] stopRow, MetaTableAccessor.QueryType type, MetaTableAccessor.Visitor visitor) static void
scanMeta
(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) Performs a scan of META table for given table starting from given row.private static void
scanMeta
(Connection connection, TableName table, MetaTableAccessor.QueryType type, int maxRows, MetaTableAccessor.Visitor visitor, CatalogReplicaMode metaReplicaMode) static void
scanMetaForTableRegions
(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName) static void
scanMetaForTableRegions
(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName, CatalogReplicaMode metaReplicaMode) static void
splitRegion
(Connection connection, RegionInfo parent, long parentOpenSeqNum, RegionInfo splitA, RegionInfo splitB, ServerName sn, int regionReplication) Splits the region into two in an atomic operation.private static void
updateLocation
(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) Updates the location of the specified region to be the specified server.static void
updateRegionLocation
(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.static void
updateRegionState
(Connection connection, RegionInfo ri, RegionState.State state) Update state column in hbase:meta.private static void
updateTableState
(Connection connection, TableState state) Update state of the table in meta.static void
updateTableState
(Connection conn, TableName tableName, TableState.State actual) Updates state in META Do not use.private static void
writeRegionName
(ByteArrayOutputStream out, byte[] regionName)
-
Field Details
-
LOG
-
METALOG
-
REPLICATION_PARENT_QUALIFIER
-
ESCAPE_BYTE
- See Also:
-
SEPARATED_BYTE
- See Also:
-
META_REPLICA_ID_DELIMITER
The delimiter for meta columns for replicaIds > 0- See Also:
-
SERVER_COLUMN_PATTERN
A regex for parsing server columns from meta. See above javadoc for meta layout
-
-
Constructor Details
-
MetaTableAccessor
public MetaTableAccessor()
-
-
Method Details
-
fullScanRegions
public static void fullScanRegions(Connection connection, MetaTableAccessor.Visitor visitor) throws IOException Performs a full scan ofhbase:meta
for regions.- Parameters:
connection
- connection we're usingvisitor
- Visitor invoked against each row in regions family.- Throws:
IOException
-
fullScanRegions
Performs a full scan ofhbase:meta
for regions.- Parameters:
connection
- connection we're using- Throws:
IOException
-
fullScanTables
public static void fullScanTables(Connection connection, MetaTableAccessor.Visitor visitor) throws IOException Performs a full scan ofhbase:meta
for tables.- Parameters:
connection
- connection we're usingvisitor
- Visitor invoked against each row in tables family.- Throws:
IOException
-
fullScan
private static List<Result> fullScan(Connection connection, MetaTableAccessor.QueryType type) throws IOException Performs a full scan ofhbase:meta
.- Parameters:
connection
- connection we're usingtype
- scanned part of meta- Returns:
- List of
Result
- Throws:
IOException
-
getMetaHTable
Callers should call close on the returnedTable
instance.- Parameters:
connection
- connection we're using to access Meta- Returns:
- An
Table
forhbase:meta
- Throws:
IOException
-
get
- Parameters:
t
- Table to use (will be closed when done).g
- Get to run- Throws:
IOException
-
getRegion
@Deprecated public static Pair<RegionInfo,ServerName> getRegion(Connection connection, byte[] regionName) throws IOException Deprecated.usegetRegionLocation(Connection, byte[])
insteadGets the region info and assignment for the specified region.- Parameters:
connection
- connection we're usingregionName
- Region to lookup.- Returns:
- Location and RegionInfo for
regionName
- Throws:
IOException
-
getRegionLocation
public static HRegionLocation getRegionLocation(Connection connection, byte[] regionName) throws IOException Returns the HRegionLocation from meta for the given region- Parameters:
connection
- connection we're usingregionName
- region we're looking for- Returns:
- HRegionLocation for the given region
- Throws:
IOException
-
getRegionLocation
public static HRegionLocation getRegionLocation(Connection connection, RegionInfo regionInfo) throws IOException Returns the HRegionLocation from meta for the given region- Parameters:
connection
- connection we're usingregionInfo
- region information- Returns:
- HRegionLocation for the given region
- Throws:
IOException
-
getCatalogFamilyRow
Returns Return theHConstants.CATALOG_FAMILY
row from hbase:meta.- Throws:
IOException
-
getMetaKeyForRegion
Returns the row key to use for this regionInfo -
parseRegionInfoFromRegionName
Returns an HRI parsed from this regionName. Not all the fields of the HRI is stored in the name, so the returned object should only be used for the fields in the regionName.- Throws:
IOException
-
getRegionResult
public static Result getRegionResult(Connection connection, RegionInfo regionInfo) throws IOException Gets the result in hbase:meta for the specified region.- Parameters:
connection
- connection we're usingregionInfo
- region we're looking for- Returns:
- result of the specified region
- Throws:
IOException
-
scanByRegionEncodedName
public static Result scanByRegionEncodedName(Connection connection, String regionEncodedName) throws IOException Scans META table for a row whose key contains the specified regionEncodedName, returning a single relatedResult
instance if any row is found, null otherwise.- Parameters:
connection
- the connection to query META table.regionEncodedName
- the region encoded name to look for at META.- Returns:
Result
instance with the row related info in META, null otherwise.- Throws:
IOException
- if any errors occur while querying META.
-
getMergeRegions
@Nullable public static List<RegionInfo> getMergeRegions(Connection connection, RegionInfo regionInfo) throws IOException Returns Return all regioninfos listed in the 'info:merge*' columns of theregionInfo
row.- Throws:
IOException
-
hasMergeRegions
Check whether the givenregionInfo
has any 'info:merge*' columns.- Throws:
IOException
-
getMergeRegionsWithName
Returns Deserialized values of <qualifier,regioninfo> pairs taken from column values that match the regex 'info:merge.*' in array ofcells
. -
getMergeRegions
Returns Deserialized regioninfo values taken from column values that match the regex 'info:merge.*' in array ofcells
. -
hasMergeRegions
Returns True if any merge regions present incells
; i.e. the column incell
matches the regex 'info:merge.*'. -
isMergeQualifierPrefix
Returns True if the column incell
matches the regex 'info:merge.*'. -
getAllRegions
public static List<RegionInfo> getAllRegions(Connection connection, boolean excludeOfflinedSplitParents) throws IOException Lists all of the regions currently in META.- Parameters:
connection
- to connect withexcludeOfflinedSplitParents
- False if we are to include offlined/splitparents regions, true and we'll leave out offlined regions from returned list- Returns:
- List of all user-space regions.
- Throws:
IOException
-
getTableRegions
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName) throws IOException Gets all of the regions of the specified table. Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table we're looking for- Returns:
- Ordered list of
RegionInfo
. - Throws:
IOException
-
getTableRegions
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) throws IOException Gets all of the regions of the specified table. Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table we're looking forexcludeOfflinedSplitParents
- If true, do not include offlined split parents in the return.- Returns:
- Ordered list of
RegionInfo
. - Throws:
IOException
-
getListOfRegionInfos
-
getTableStartRowForMeta
Returns start row for scanning META according to query type -
getTableStopRowForMeta
Returns stop row for scanning META according to query type -
getScanForTableName
public static Scan getScanForTableName(org.apache.hadoop.conf.Configuration conf, TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table. It doesn't specify any columns. This is a better alternative to just using a start row and scan until it hits a new table since that requires parsing the HRI to get the table name.- Parameters:
tableName
- bytes of table's name- Returns:
- configured Scan object
-
getMetaScan
-
getTableRegionsAndLocations
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, TableName tableName) throws IOException Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table we're looking for- Returns:
- Return list of regioninfos and server.
- Throws:
IOException
-
getTableRegionsAndLocations
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, @Nullable TableName tableName, boolean excludeOfflinedSplitParents) throws IOException Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table to work with, can be null for getting all regionsexcludeOfflinedSplitParents
- don't return split parents- Returns:
- Return list of regioninfos and server addresses.
- Throws:
IOException
-
getServerUserRegions
public static NavigableMap<RegionInfo,Result> getServerUserRegions(Connection connection, ServerName serverName) throws IOException Get the user regions a given server is hosting.- Parameters:
connection
- connection we're usingserverName
- server whose regions we're interested in- Returns:
- List of user regions installed on this server (does not include catalog regions).
- Throws:
IOException
-
fullScanMetaAndPrint
- Throws:
IOException
-
scanMetaForTableRegions
public static void scanMetaForTableRegions(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName, CatalogReplicaMode metaReplicaMode) throws IOException - Throws:
IOException
-
scanMetaForTableRegions
public static void scanMetaForTableRegions(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName) throws IOException - Throws:
IOException
-
scanMeta
private static void scanMeta(Connection connection, TableName table, MetaTableAccessor.QueryType type, int maxRows, MetaTableAccessor.Visitor visitor, CatalogReplicaMode metaReplicaMode) throws IOException - Throws:
IOException
-
scanMeta
private static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, MetaTableAccessor.QueryType type, MetaTableAccessor.Visitor visitor) throws IOException - Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, MetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) throws IOException Performs a scan of META table for given table starting from given row.- Parameters:
connection
- connection we're usingvisitor
- visitor to calltableName
- table withing we scanrow
- start scan from this rowrowLimit
- max number of rows to return- Throws:
IOException
-
scanMeta
static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, MetaTableAccessor.QueryType type, int maxRows, MetaTableAccessor.Visitor visitor) throws IOException Performs a scan of META table.- Parameters:
connection
- connection we're usingstartRow
- Where to start the scan. Pass null if want to begin scan at first row.stopRow
- Where to stop the scan. Pass null if want to scan all rows from the start onetype
- scanned part of metamaxRows
- maximum rows to returnvisitor
- Visitor invoked against each row.- Throws:
IOException
-
scanMeta
private static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, MetaTableAccessor.QueryType type, @Nullable Filter filter, int maxRows, MetaTableAccessor.Visitor visitor, CatalogReplicaMode metaReplicaMode) throws IOException - Throws:
IOException
-
getClosestRegionInfo
@NonNull private static RegionInfo getClosestRegionInfo(Connection connection, @NonNull TableName tableName, @NonNull byte[] row) throws IOException Returns Get closest metatable region row to passedrow
- Throws:
IOException
-
getCatalogFamily
Returns the column family used for meta columns.- Returns:
- HConstants.CATALOG_FAMILY.
-
getTableFamily
Returns the column family used for table columns.- Returns:
- HConstants.TABLE_FAMILY.
-
getRegionInfoColumn
Returns the column qualifier for serialized region info- Returns:
- HConstants.REGIONINFO_QUALIFIER
-
getTableStateColumn
Returns the column qualifier for serialized table state- Returns:
- HConstants.TABLE_STATE_QUALIFIER
-
getRegionStateColumn
Returns the column qualifier for serialized region state- Returns:
- HConstants.STATE_QUALIFIER
-
getRegionStateColumn
Returns the column qualifier for serialized region state- Parameters:
replicaId
- the replicaId of the region- Returns:
- a byte[] for state qualifier
-
getServerNameColumn
Returns the column qualifier for serialized region state- Parameters:
replicaId
- the replicaId of the region- Returns:
- a byte[] for sn column qualifier
-
getServerColumn
Returns the column qualifier for server column for replicaId- Parameters:
replicaId
- the replicaId of the region- Returns:
- a byte[] for server column qualifier
-
getStartCodeColumn
Returns the column qualifier for server start code column for replicaId- Parameters:
replicaId
- the replicaId of the region- Returns:
- a byte[] for server start code column qualifier
-
getSeqNumColumn
Returns the column qualifier for seqNum column for replicaId- Parameters:
replicaId
- the replicaId of the region- Returns:
- a byte[] for seqNum column qualifier
-
parseReplicaIdFromServerColumn
Parses the replicaId from the server column qualifier. See top of the class javadoc for the actual meta layout- Parameters:
serverColumn
- the column qualifier- Returns:
- an int for the replicaId
-
getServerName
Returns aServerName
from catalog tableResult
.- Parameters:
r
- Result to pull from- Returns:
- A ServerName instance or null if necessary fields not found or empty.
-
getTargetServerName
Returns theServerName
from catalog tableResult
where the region is transitioning on. It should be the same asgetServerName(Result,int)
if the server is at OPEN state.- Parameters:
r
- Result to pull the transitioning server name from- Returns:
- A ServerName instance or
getServerName(Result,int)
if necessary fields not found or empty.
-
getSeqNumDuringOpen
The latest seqnum that the server writing to meta observed when opening the region. E.g. the seqNum when the result ofgetServerName(Result, int)
was written.- Parameters:
r
- Result to pull the seqNum from- Returns:
- SeqNum, or HConstants.NO_SEQNUM if there's no value written.
-
getDaughterRegions
Returns the daughter regions by reading the corresponding columns of the catalog table Result.- Parameters:
data
- a Result object from the catalog table scan- Returns:
- pair of RegionInfo or PairOfSameType(null, null) if region is not a split parent
-
getRegionLocations
Returns an HRegionLocationList extracted from the result.- Returns:
- an HRegionLocationList containing all locations for the region range or null if we can't deserialize the result.
-
getRegionLocation
Returns the HRegionLocation parsed from the given meta row Result for the given regionInfo and replicaId. The regionInfo can be the default region info for the replica.- Parameters:
r
- the meta row resultregionInfo
- RegionInfo for default replicareplicaId
- the replicaId for the HRegionLocation- Returns:
- HRegionLocation parsed from the given meta row Result for the given replicaId
-
getRegionInfo
Returns RegionInfo object from the column HConstants.CATALOG_FAMILY:HConstants.REGIONINFO_QUALIFIER of the catalog table Result.- Parameters:
data
- a Result object from the catalog table scan- Returns:
- RegionInfo or null
-
getRegionInfo
Returns the RegionInfo object from the columnHConstants.CATALOG_FAMILY
andqualifier
of the catalog table result.- Parameters:
r
- a Result object from the catalog table scanqualifier
- Column family qualifier- Returns:
- An RegionInfo instance or null.
-
getTableState
@Nullable public static TableState getTableState(Connection conn, TableName tableName) throws IOException Fetch table state for given table from META table- Parameters:
conn
- connection to usetableName
- table to fetch state for- Throws:
IOException
-
getTableStates
Fetch table states from META table- Parameters:
conn
- connection to use- Returns:
- map {tableName -> state}
- Throws:
IOException
-
updateTableState
public static void updateTableState(Connection conn, TableName tableName, TableState.State actual) throws IOException Updates state in META Do not use. For internal use only.- Parameters:
conn
- connection to usetableName
- table to look for- Throws:
IOException
-
getTableState
Decode table state from META Result. Should contain cell from HConstants.TABLE_FAMILY- Returns:
- null if not found
- Throws:
IOException
-
makePutFromRegionInfo
Generates and returns aPut
containing theRegionInfo
for the catalog table.- Throws:
IllegalArgumentException
- when the provided RegionInfo is not the default replica.IOException
-
makePutFromRegionInfo
Generates and returns aPut
containing theRegionInfo
for the catalog table.- Throws:
IllegalArgumentException
- when the provided RegionInfo is not the default replica.IOException
-
makeDeleteFromRegionInfo
Generates and returns a Delete containing the region info for the catalog table -
addDaughtersToPut
private static Put addDaughtersToPut(Put put, RegionInfo splitA, RegionInfo splitB) throws IOException Adds split daughters to the Put- Throws:
IOException
-
putToMetaTable
Put the passedp
to thehbase:meta
table.- Parameters:
connection
- connection we're usingp
- Put to add to hbase:meta- Throws:
IOException
-
put
- Parameters:
t
- Table to usep
- put to make- Throws:
IOException
-
putsToMetaTable
Put the passedps
to thehbase:meta
table.- Parameters:
connection
- connection we're usingps
- Put to add to hbase:meta- Throws:
IOException
-
deleteFromMetaTable
Delete the passedd
from thehbase:meta
table.- Parameters:
connection
- connection we're usingd
- Delete to add to hbase:meta- Throws:
IOException
-
deleteFromMetaTable
private static void deleteFromMetaTable(Connection connection, List<Delete> deletes) throws IOException Delete the passeddeletes
from thehbase:meta
table.- Parameters:
connection
- connection we're usingdeletes
- Deletes to add to hbase:meta This list should support #remove.- Throws:
IOException
-
addRegionStateToPut
private static Put addRegionStateToPut(Put put, int replicaId, RegionState.State state) throws IOException Set the column value corresponding to thisreplicaId
'sRegionState
to the providedstate
. Mutates the providedPut
.- Throws:
IOException
-
updateRegionState
public static void updateRegionState(Connection connection, RegionInfo ri, RegionState.State state) throws IOException Update state column in hbase:meta.- Throws:
IOException
-
addSplitsToParent
public static void addSplitsToParent(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) throws IOException Adds daughter region infos to hbase:meta row for the specified region. Note that this does not add its daughter's as different rows, but adds information about the daughters in the same row as the parent. UsesplitRegion(Connection, RegionInfo, long, RegionInfo, RegionInfo, ServerName, int)
if you want to do that.- Parameters:
connection
- connection we're usingregionInfo
- RegionInfo of parent regionsplitA
- first split daughter of the parent regionInfosplitB
- second split daughter of the parent regionInfo- Throws:
IOException
- if problem connecting or updating meta
-
addRegionToMeta
Adds a (single) hbase:meta row for the specified new region and its daughters. Note that this does not add its daughter's as different rows, but adds information about the daughters in the same row as the parent. UsesplitRegion(Connection, RegionInfo, long, RegionInfo, RegionInfo, ServerName, int)
if you want to do that.- Parameters:
connection
- connection we're usingregionInfo
- region information- Throws:
IOException
- if problem connecting or updating meta
-
addRegionsToMeta
public static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication) throws IOException Adds a hbase:meta row for each of the specified new regions. Initial state for new regions is CLOSED.- Parameters:
connection
- connection we're usingregionInfos
- region information list- Throws:
IOException
- if problem connecting or updating meta
-
addRegionsToMeta
private static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication, long ts) throws IOException Adds a hbase:meta row for each of the specified new regions. Initial state for new regions is CLOSED.- Parameters:
connection
- connection we're usingregionInfos
- region information listts
- desired timestamp- Throws:
IOException
- if problem connecting or updating meta
-
addMergeRegions
- Throws:
IOException
-
mergeRegions
public static void mergeRegions(Connection connection, RegionInfo mergedRegion, Map<RegionInfo, Long> parentSeqNum, ServerName sn, int regionReplication) throws IOExceptionMerge regions into one in an atomic operation. Deletes the merging regions in hbase:meta and adds the merged region.- Parameters:
connection
- connection we're usingmergedRegion
- the merged regionparentSeqNum
- Parent regions to merge and their next open sequence id used by serial replication. Set to -1 if not needed by this table.sn
- the location of the region- Throws:
IOException
-
splitRegion
public static void splitRegion(Connection connection, RegionInfo parent, long parentOpenSeqNum, RegionInfo splitA, RegionInfo splitB, ServerName sn, int regionReplication) throws IOException Splits the region into two in an atomic operation. Offlines the parent region with the information that it is split into two, and also adds the daughter regions. Does not add the location information to the daughter regions since they are not open yet.- Parameters:
connection
- connection we're usingparent
- the parent region which is splitparentOpenSeqNum
- the next open sequence id for parent region, used by serial replication. -1 if not necessary.splitA
- Split daughter region AsplitB
- Split daughter region Bsn
- the location of the region- Throws:
IOException
-
updateTableState
Update state of the table in meta.- Parameters:
connection
- what we use for updatestate
- new state- Throws:
IOException
-
makePutFromTableState
Construct PUT for given state- Parameters:
state
- new state
-
deleteTableState
Remove state for table from meta- Parameters:
connection
- to use for deletiontable
- to delete state for- Throws:
IOException
-
multiMutate
- Throws:
IOException
-
multiMutate
Performs an atomic multi-mutate operation against the given table. Used by the likes of merge and split as these want to make atomic mutations across multiple rows.- Throws:
IOException
- even if we encounter a RuntimeException, we'll still wrap it in an IOE.
-
updateRegionLocation
public static void updateRegionLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.Uses passed catalog tracker to get a connection to the server hosting hbase:meta and makes edits to that region.
- Parameters:
connection
- connection we're usingregionInfo
- region to update location ofopenSeqNum
- the latest sequence number obtained when the region was opensn
- Server namemasterSystemTime
- wall clock time from master if passed in the open region RPC- Throws:
IOException
-
updateLocation
private static void updateLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException Updates the location of the specified region to be the specified server.Connects to the specified server which should be hosting the specified catalog region name to perform the edit.
- Parameters:
connection
- connection we're usingregionInfo
- region to update location ofsn
- Server nameopenSeqNum
- the latest sequence number obtained when the region was openmasterSystemTime
- wall clock time from master if passed in the open region RPC- Throws:
IOException
- In particular could throwConnectException
if the server is down on other end.
-
deleteRegionInfo
public static void deleteRegionInfo(Connection connection, RegionInfo regionInfo) throws IOException Deletes the specified region from META.- Parameters:
connection
- connection we're usingregionInfo
- region to be deleted from META- Throws:
IOException
-
deleteRegionInfos
public static void deleteRegionInfos(Connection connection, List<RegionInfo> regionsInfo) throws IOException Deletes the specified regions from META.- Parameters:
connection
- connection we're usingregionsInfo
- list of regions to be deleted from META- Throws:
IOException
-
deleteRegionInfos
private static void deleteRegionInfos(Connection connection, List<RegionInfo> regionsInfo, long ts) throws IOException Deletes the specified regions from META.- Parameters:
connection
- connection we're usingregionsInfo
- list of regions to be deleted from META- Throws:
IOException
-
overwriteRegions
public static void overwriteRegions(Connection connection, List<RegionInfo> regionInfos, int regionReplication) throws IOException Overwrites the specified regions from hbase:meta. Deletes old rows for the given regions and adds new ones. Regions added back have state CLOSED.- Parameters:
connection
- connection we're usingregionInfos
- list of regions to be added to META- Throws:
IOException
-
deleteMergeQualifiers
public static void deleteMergeQualifiers(Connection connection, RegionInfo mergeRegion) throws IOException Deletes merge qualifiers for the specified merge region.- Parameters:
connection
- connection we're usingmergeRegion
- the merged region- Throws:
IOException
-
addRegionInfo
- Throws:
IOException
-
addLocation
public static Put addLocation(Put p, ServerName sn, long openSeqNum, int replicaId) throws IOException - Throws:
IOException
-
writeRegionName
-
getParentsBytes
-
parseParentsBytes
-
addReplicationParent
- Throws:
IOException
-
makePutForReplicationBarrier
public static Put makePutForReplicationBarrier(RegionInfo regionInfo, long openSeqNum, long ts) throws IOException - Throws:
IOException
-
addReplicationBarrier
See class comment on SerialReplicationChecker- Throws:
IOException
-
addEmptyLocation
- Throws:
IOException
-
getReplicationBarrier
-
getReplicationBarriers
-
getReplicationBarrierResult
private static MetaTableAccessor.ReplicationBarrierResult getReplicationBarrierResult(Result result) -
getReplicationBarrierResult
public static MetaTableAccessor.ReplicationBarrierResult getReplicationBarrierResult(Connection conn, TableName tableName, byte[] row, byte[] encodedRegionName) throws IOException - Throws:
IOException
-
getReplicationBarrier
- Throws:
IOException
-
getTableEncodedRegionNameAndLastBarrier
public static List<Pair<String,Long>> getTableEncodedRegionNameAndLastBarrier(Connection conn, TableName tableName) throws IOException - Throws:
IOException
-
getTableEncodedRegionNamesForSerialReplication
public static List<String> getTableEncodedRegionNamesForSerialReplication(Connection conn, TableName tableName) throws IOException - Throws:
IOException
-
debugLogMutations
- Throws:
IOException
-
debugLogMutation
- Throws:
IOException
-
addSequenceNum
- Throws:
IOException
-
getRegionLocation(Connection, byte[])
instead