Package org.apache.hadoop.hbase
Class DistributedHBaseCluster
java.lang.Object
org.apache.hadoop.hbase.HBaseCluster
org.apache.hadoop.hbase.DistributedHBaseCluster
- All Implemented Interfaces:
Closeable
,AutoCloseable
,org.apache.hadoop.conf.Configurable
Manages the interactions with an already deployed distributed cluster (as opposed to a
pseudo-distributed, or mini/local cluster). This is used by integration and system tests.
-
Nested Class Summary
Modifier and TypeClassDescriptionprivate static class
-
Field Summary
Modifier and TypeFieldDescriptionprivate org.apache.hadoop.hbase.client.Admin
private ClusterManager
private final org.apache.hadoop.hbase.client.Connection
private final Set<org.apache.hadoop.hbase.ServerName>
List of RegionServers killed so far.Fields inherited from class org.apache.hadoop.hbase.HBaseCluster
conf, initialClusterStatus, LOG
-
Constructor Summary
ConstructorDescriptionDistributedHBaseCluster
(org.apache.hadoop.conf.Configuration conf, ClusterManager clusterManager) -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
Closes all the resources held open for this cluster.org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface
getAdminProtocol
(org.apache.hadoop.hbase.ServerName serverName) Returns an AdminProtocol interface to the regionserverorg.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService.BlockingInterface
getClientProtocol
(org.apache.hadoop.hbase.ServerName serverName) Returns a ClientProtocol interface to the regionserverorg.apache.hadoop.hbase.ClusterMetrics
Returns a ClusterStatus for this HBase clusterorg.apache.hadoop.hbase.ClusterMetrics
Returns a ClusterStatus for this HBase cluster as observed at the starting of the HBaseClusterorg.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService.BlockingInterface
Returns anMasterProtos.MasterService.BlockingInterface
to the active masterorg.apache.hadoop.hbase.ServerName
getServerHoldingRegion
(org.apache.hadoop.hbase.TableName tn, byte[] regionName) Get the ServerName of region server serving the specified regionboolean
boolean
isKilledRS
(org.apache.hadoop.hbase.ServerName serverName) Keeping track of killed servers and being able to check if a particular server was killed makes it possible to do fault tolerance testing for dead servers in a deterministic way.void
killDataNode
(org.apache.hadoop.hbase.ServerName serverName) Kills the datanode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.void
killJournalNode
(org.apache.hadoop.hbase.ServerName serverName) Kills the journalnode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.void
killMaster
(org.apache.hadoop.hbase.ServerName serverName) Kills the master process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.void
killNameNode
(org.apache.hadoop.hbase.ServerName serverName) Kills the namenode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.void
killRegionServer
(org.apache.hadoop.hbase.ServerName serverName) Kills the region server process if this is a distributed cluster, otherwise this causes the region server to exit doing basic clean up only.void
killZkNode
(org.apache.hadoop.hbase.ServerName serverName) Kills the zookeeper node process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.protected boolean
boolean
restoreClusterMetrics
(org.apache.hadoop.hbase.ClusterMetrics initial) Restores the cluster to given state if this is a real cluster, otherwise does nothing.protected boolean
restoreMasters
(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) protected boolean
restoreRegionServers
(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) void
resumeRegionServer
(org.apache.hadoop.hbase.ServerName serverName) Resume the region servervoid
setClusterManager
(ClusterManager clusterManager) void
shutdown()
Shut down the HBase clustervoid
startDataNode
(org.apache.hadoop.hbase.ServerName serverName) Starts a new datanode on the given hostname or if this is a mini/local cluster, silently logs warning message.void
startJournalNode
(org.apache.hadoop.hbase.ServerName serverName) Starts a new journalnode on the given hostname or if this is a mini/local cluster, silently logs warning message.void
startMaster
(String hostname, int port) Starts a new master on the given hostname or if this is a mini/local cluster, starts a master locally.void
startNameNode
(org.apache.hadoop.hbase.ServerName serverName) Starts a new namenode on the given hostname or if this is a mini/local cluster, silently logs warning message.void
startRegionServer
(String hostname, int port) Starts a new region server on the given hostname or if this is a mini/local cluster, starts a region server locally.void
startZkNode
(String hostname, int port) Starts a new zookeeper node on the given hostname or if this is a mini/local cluster, silently logs warning message.void
stopDataNode
(org.apache.hadoop.hbase.ServerName serverName) Stops the datanode if this is a distributed cluster, otherwise silently logs warning message.void
stopJournalNode
(org.apache.hadoop.hbase.ServerName serverName) Stops the journalnode if this is a distributed cluster, otherwise silently logs warning message.void
stopMaster
(org.apache.hadoop.hbase.ServerName serverName) Stops the given master, by attempting a gradual stop.void
stopNameNode
(org.apache.hadoop.hbase.ServerName serverName) Stops the namenode if this is a distributed cluster, otherwise silently logs warning message.void
stopRegionServer
(org.apache.hadoop.hbase.ServerName serverName) Stops the given region server, by attempting a gradual stop.void
stopZkNode
(org.apache.hadoop.hbase.ServerName serverName) Stops the region zookeeper if this is a distributed cluster, otherwise silently logs warning message.void
suspendRegionServer
(org.apache.hadoop.hbase.ServerName serverName) Suspend the region serverboolean
waitForActiveAndReadyMaster
(long timeout) Blocks until there is an active master and that master has completed initialization.void
waitForDataNodeToStart
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified datanode to join the clustervoid
waitForDataNodeToStop
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified datanode to stop the thread / process.void
waitForJournalNodeToStart
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified journalnode to join the clustervoid
waitForJournalNodeToStop
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified journalnode to stopvoid
waitForMasterToStop
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified master to stop the thread / process.void
waitForNameNodeToStart
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified namenode to join the clustervoid
waitForNameNodeToStop
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified namenode to stopvoid
waitForRegionServerToStop
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified region server to stop the thread / process.private void
waitForServiceToStart
(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) private void
waitForServiceToStop
(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) void
waitForZkNodeToStart
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified zookeeper node to join the clustervoid
waitForZkNodeToStop
(org.apache.hadoop.hbase.ServerName serverName, long timeout) Wait for the specified zookeeper node to stop the thread / process.void
Wait for HBase Cluster to shut down.Methods inherited from class org.apache.hadoop.hbase.HBaseCluster
getConf, getServerHoldingMeta, restoreInitialStatus, setConf, waitForActiveAndReadyMaster, waitForDatanodesRegistered, waitForNamenodeAvailable, waitForRegionServerToStart
-
Field Details
-
admin
-
connection
-
clusterManager
-
killedRegionServers
List of RegionServers killed so far. ServerName also comprises startCode of a server, so any restarted instances of the same server will have different ServerName and will not coincide with past dead ones. So there's no need to cleanup this list.
-
-
Constructor Details
-
DistributedHBaseCluster
public DistributedHBaseCluster(org.apache.hadoop.conf.Configuration conf, ClusterManager clusterManager) throws IOException - Throws:
IOException
-
-
Method Details
-
setClusterManager
-
getClusterManager
-
getClusterMetrics
Returns a ClusterStatus for this HBase cluster- Specified by:
getClusterMetrics
in classHBaseCluster
- Throws:
IOException
- See Also:
-
getInitialClusterMetrics
Description copied from class:HBaseCluster
Returns a ClusterStatus for this HBase cluster as observed at the starting of the HBaseCluster- Overrides:
getInitialClusterMetrics
in classHBaseCluster
- Throws:
IOException
-
close
Description copied from class:HBaseCluster
Closes all the resources held open for this cluster. Note that this call does not shutdown the cluster.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Specified by:
close
in classHBaseCluster
- Throws:
IOException
- See Also:
-
getAdminProtocol
public org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface getAdminProtocol(org.apache.hadoop.hbase.ServerName serverName) throws IOException Description copied from class:HBaseCluster
Returns an AdminProtocol interface to the regionserver- Specified by:
getAdminProtocol
in classHBaseCluster
- Throws:
IOException
-
getClientProtocol
public org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.ClientService.BlockingInterface getClientProtocol(org.apache.hadoop.hbase.ServerName serverName) throws IOException Description copied from class:HBaseCluster
Returns a ClientProtocol interface to the regionserver- Specified by:
getClientProtocol
in classHBaseCluster
- Throws:
IOException
-
startRegionServer
Description copied from class:HBaseCluster
Starts a new region server on the given hostname or if this is a mini/local cluster, starts a region server locally.- Specified by:
startRegionServer
in classHBaseCluster
- Parameters:
hostname
- the hostname to start the regionserver on- Throws:
IOException
- if something goes wrong
-
killRegionServer
Description copied from class:HBaseCluster
Kills the region server process if this is a distributed cluster, otherwise this causes the region server to exit doing basic clean up only.- Specified by:
killRegionServer
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
isKilledRS
Description copied from class:HBaseCluster
Keeping track of killed servers and being able to check if a particular server was killed makes it possible to do fault tolerance testing for dead servers in a deterministic way. A concrete example of such case is - killing servers and waiting for all regions of a particular table to be assigned. We can check for server column in META table and that its value is not one of the killed servers.- Specified by:
isKilledRS
in classHBaseCluster
-
stopRegionServer
Description copied from class:HBaseCluster
Stops the given region server, by attempting a gradual stop.- Specified by:
stopRegionServer
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
waitForRegionServerToStop
public void waitForRegionServerToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified region server to stop the thread / process.- Specified by:
waitForRegionServerToStop
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
suspendRegionServer
Description copied from class:HBaseCluster
Suspend the region server- Specified by:
suspendRegionServer
in classHBaseCluster
- Parameters:
serverName
- the hostname to suspend the regionserver on- Throws:
IOException
- if something goes wrong
-
resumeRegionServer
Description copied from class:HBaseCluster
Resume the region server- Specified by:
resumeRegionServer
in classHBaseCluster
- Parameters:
serverName
- the hostname to resume the regionserver on- Throws:
IOException
- if something goes wrong
-
startZkNode
Description copied from class:HBaseCluster
Starts a new zookeeper node on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startZkNode
in classHBaseCluster
- Parameters:
hostname
- the hostname to start the regionserver on- Throws:
IOException
- if something goes wrong
-
killZkNode
Description copied from class:HBaseCluster
Kills the zookeeper node process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killZkNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
stopZkNode
Description copied from class:HBaseCluster
Stops the region zookeeper if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopZkNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
waitForZkNodeToStart
public void waitForZkNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified zookeeper node to join the cluster- Specified by:
waitForZkNodeToStart
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
waitForZkNodeToStop
public void waitForZkNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified zookeeper node to stop the thread / process.- Specified by:
waitForZkNodeToStop
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
startDataNode
Description copied from class:HBaseCluster
Starts a new datanode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startDataNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
killDataNode
Description copied from class:HBaseCluster
Kills the datanode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killDataNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
stopDataNode
Description copied from class:HBaseCluster
Stops the datanode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopDataNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
waitForDataNodeToStart
public void waitForDataNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified datanode to join the cluster- Specified by:
waitForDataNodeToStart
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
waitForDataNodeToStop
public void waitForDataNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified datanode to stop the thread / process.- Specified by:
waitForDataNodeToStop
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
startNameNode
Description copied from class:HBaseCluster
Starts a new namenode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startNameNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
killNameNode
Description copied from class:HBaseCluster
Kills the namenode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killNameNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
stopNameNode
Description copied from class:HBaseCluster
Stops the namenode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopNameNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
waitForNameNodeToStart
public void waitForNameNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified namenode to join the cluster- Specified by:
waitForNameNodeToStart
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
waitForNameNodeToStop
public void waitForNameNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified namenode to stop- Specified by:
waitForNameNodeToStop
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
startJournalNode
Description copied from class:HBaseCluster
Starts a new journalnode on the given hostname or if this is a mini/local cluster, silently logs warning message.- Specified by:
startJournalNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
killJournalNode
Description copied from class:HBaseCluster
Kills the journalnode process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killJournalNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
stopJournalNode
Description copied from class:HBaseCluster
Stops the journalnode if this is a distributed cluster, otherwise silently logs warning message.- Specified by:
stopJournalNode
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
waitForJournalNodeToStart
public void waitForJournalNodeToStart(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified journalnode to join the cluster- Specified by:
waitForJournalNodeToStart
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
waitForJournalNodeToStop
public void waitForJournalNodeToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified journalnode to stop- Specified by:
waitForJournalNodeToStop
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
waitForServiceToStop
private void waitForServiceToStop(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
waitForServiceToStart
private void waitForServiceToStart(ClusterManager.ServiceType service, org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException - Throws:
IOException
-
getMasterAdminService
public org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService.BlockingInterface getMasterAdminService() throws IOExceptionDescription copied from class:HBaseCluster
Returns anMasterProtos.MasterService.BlockingInterface
to the active master- Specified by:
getMasterAdminService
in classHBaseCluster
- Throws:
IOException
-
startMaster
Description copied from class:HBaseCluster
Starts a new master on the given hostname or if this is a mini/local cluster, starts a master locally.- Specified by:
startMaster
in classHBaseCluster
- Parameters:
hostname
- the hostname to start the master on- Throws:
IOException
- if something goes wrong
-
killMaster
Description copied from class:HBaseCluster
Kills the master process if this is a distributed cluster, otherwise, this causes master to exit doing basic clean up only.- Specified by:
killMaster
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
stopMaster
Description copied from class:HBaseCluster
Stops the given master, by attempting a gradual stop.- Specified by:
stopMaster
in classHBaseCluster
- Throws:
IOException
- if something goes wrong
-
waitForMasterToStop
public void waitForMasterToStop(org.apache.hadoop.hbase.ServerName serverName, long timeout) throws IOException Description copied from class:HBaseCluster
Wait for the specified master to stop the thread / process.- Specified by:
waitForMasterToStop
in classHBaseCluster
- Throws:
IOException
- if something goes wrong or timeout occurs
-
waitForActiveAndReadyMaster
Description copied from class:HBaseCluster
Blocks until there is an active master and that master has completed initialization.- Specified by:
waitForActiveAndReadyMaster
in classHBaseCluster
- Parameters:
timeout
- the timeout limit in ms- Returns:
- true if an active master becomes available. false if there are no masters left.
- Throws:
IOException
-
getServerHoldingRegion
public org.apache.hadoop.hbase.ServerName getServerHoldingRegion(org.apache.hadoop.hbase.TableName tn, byte[] regionName) throws IOException Description copied from class:HBaseCluster
Get the ServerName of region server serving the specified region- Specified by:
getServerHoldingRegion
in classHBaseCluster
- Parameters:
tn
- Table name that has the region.regionName
- Name of the region in bytes- Returns:
- ServerName that hosts the region or null
- Throws:
IOException
-
waitUntilShutDown
Description copied from class:HBaseCluster
Wait for HBase Cluster to shut down.- Specified by:
waitUntilShutDown
in classHBaseCluster
-
shutdown
Description copied from class:HBaseCluster
Shut down the HBase cluster- Specified by:
shutdown
in classHBaseCluster
- Throws:
IOException
-
isDistributedCluster
- Overrides:
isDistributedCluster
in classHBaseCluster
- Returns:
- whether we are interacting with a distributed cluster as opposed to an in-process mini/local cluster.
-
restoreClusterMetrics
public boolean restoreClusterMetrics(org.apache.hadoop.hbase.ClusterMetrics initial) throws IOException Description copied from class:HBaseCluster
Restores the cluster to given state if this is a real cluster, otherwise does nothing. This is a best effort restore. If the servers are not reachable, or insufficient permissions, etc. restoration might be partial.- Overrides:
restoreClusterMetrics
in classHBaseCluster
- Returns:
- whether restoration is complete
- Throws:
IOException
-
restoreMasters
protected boolean restoreMasters(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) -
restoreRegionServers
protected boolean restoreRegionServers(org.apache.hadoop.hbase.ClusterMetrics initial, org.apache.hadoop.hbase.ClusterMetrics current) -
restoreAdmin
- Throws:
IOException
-