Package org.apache.hadoop.hbase.util

Class DynamicClassLoader

java.lang.Object

java.lang.ClassLoader

java.security.SecureClassLoader

java.net.URLClassLoader

org.apache.hadoop.hbase.util.ClassLoaderBase

org.apache.hadoop.hbase.util.DynamicClassLoader

All Implemented Interfaces:

Closeable, AutoCloseable

@Private
public class DynamicClassLoader
extends ClassLoaderBase

This is a class loader that can load classes dynamically from new jar files under a configured folder. The paths to the jar files are converted to URLs, and URLClassLoader logic is actually used to load classes. This class loader always uses its parent class loader to load a class at first. Only if its parent class loader can not load a class, we will try to load it using the logic here.

The configured folder can be a HDFS path. In this case, the jar files under that folder will be copied to local at first under ${hbase.local.dir}/jars/. The local copy will be updated if the remote copy is updated, according to its last modified timestamp.

We can't unload a class already loaded. So we will use the existing jar files we already know to load any class which can't be loaded using the parent class loader. If we still can't load the class from the existing jar files, we will check if any new jar file is added, if so, we will load the new jar file and try to load the class again. If still failed, a class not found exception will be thrown.

Be careful in uploading new jar files and make sure all classes are consistent, otherwise, we may not be able to load your classes properly.

Field Summary

Fields

Modifier and Type	Field	Description
private static final String	DYNAMIC_JARS_DIR
private static final String	DYNAMIC_JARS_DIR_KEY
private static final String	DYNAMIC_JARS_OPTIONAL_CONF_KEY
private static final boolean	DYNAMIC_JARS_OPTIONAL_DEFAULT
private HashMap<String,Long>	jarModifiedTime
private File	localDir
private static final org.slf4j.Logger	LOG
private org.apache.hadoop.fs.Path	remoteDir
private org.apache.hadoop.fs.FileSystem	remoteDirFs
private final boolean	useDynamicJars
private final boolean	userConfigUseDynamicJars

Fields inherited from class org.apache.hadoop.hbase.util.ClassLoaderBase

DEFAULT_LOCAL_DIR, LOCAL_DIR_KEY, parent

Constructor Summary

Constructors

Constructor	Description
DynamicClassLoader(org.apache.hadoop.conf.Configuration conf, ClassLoader parent)	Creates a DynamicClassLoader that can load classes dynamically from jar files under a specific folder.

Method Summary

Modifier and Type	Method	Description
private void	initTempDir(org.apache.hadoop.conf.Configuration conf)
Class<?>	loadClass(String name)
private void	loadNewJars()
private Class<?>	tryRefreshClass(String name)

Methods inherited from class java.net.URLClassLoader

addURL, close, definePackage, findClass, findResource, findResources, getPermissions, getResourceAsStream, getURLs, newInstance, newInstance

Methods inherited from class java.security.SecureClassLoader

defineClass, defineClass

Methods inherited from class java.lang.ClassLoader

clearAssertionStatus, defineClass, defineClass, defineClass, defineClass, definePackage, findClass, findLibrary, findLoadedClass, findResource, findSystemClass, getClassLoadingLock, getDefinedPackage, getDefinedPackages, getName, getPackage, getPackages, getParent, getPlatformClassLoader, getResource, getResources, getSystemClassLoader, getSystemResource, getSystemResourceAsStream, getSystemResources, getUnnamedModule, isRegisteredAsParallelCapable, loadClass, registerAsParallelCapable, resolveClass, resources, setClassAssertionStatus, setDefaultAssertionStatus, setPackageAssertionStatus, setSigners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

DYNAMIC_JARS_DIR

private static final String DYNAMIC_JARS_DIR

DYNAMIC_JARS_DIR_KEY

private static final String DYNAMIC_JARS_DIR_KEY

See Also:

• Constant Field Values

DYNAMIC_JARS_OPTIONAL_CONF_KEY

private static final String DYNAMIC_JARS_OPTIONAL_CONF_KEY

See Also:

• Constant Field Values

DYNAMIC_JARS_OPTIONAL_DEFAULT

private static final boolean DYNAMIC_JARS_OPTIONAL_DEFAULT

See Also:

• Constant Field Values

userConfigUseDynamicJars

private final boolean userConfigUseDynamicJars

useDynamicJars

private final boolean useDynamicJars

localDir

private File localDir

remoteDirFs

private org.apache.hadoop.fs.FileSystem remoteDirFs

remoteDir

private org.apache.hadoop.fs.Path remoteDir

jarModifiedTime

private HashMap<String,Long> jarModifiedTime

Constructor Details

DynamicClassLoader

public DynamicClassLoader(org.apache.hadoop.conf.Configuration conf,
 ClassLoader parent)

Creates a DynamicClassLoader that can load classes dynamically from jar files under a specific folder.

Parameters:

conf - the configuration for the cluster.

parent - the parent ClassLoader to set.

Method Details

initTempDir

private void initTempDir(org.apache.hadoop.conf.Configuration conf)

loadClass

public Class<?> loadClass(String name)
                   throws ClassNotFoundException

Overrides:

loadClass in class ClassLoader

Throws:

ClassNotFoundException

tryRefreshClass

private Class<?> tryRefreshClass(String name)
                          throws ClassNotFoundException

Throws:

ClassNotFoundException

loadNewJars

private void loadNewJars()