Package org.apache.hadoop.hbase.io.crypto.tls

Class X509Util

java.lang.Object

org.apache.hadoop.hbase.io.crypto.tls.X509Util

@Private
public final class X509Util
extends Object

Utility code for X509 handling Default cipher suites: Performance testing done by Facebook engineers shows that on Intel x86_64 machines, Java9 performs better with GCM and Java8 performs better with CBC, so these seem like reasonable defaults.

This file has been copied from the Apache ZooKeeper project.

See Also:

• Base revision

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	X509Util.ClientAuth	Enum specifying the client auth requirement of server-side TLS sockets created by this X509Util.

Field Summary

Fields

Modifier and Type	Field	Description
(package private) static final String	CONFIG_PREFIX
private static final String[]	DEFAULT_CIPHERS_JAVA11
private static final String[]	DEFAULT_CIPHERS_JAVA8
private static final String[]	DEFAULT_CIPHERS_JAVA9
private static final String[]	DEFAULT_CIPHERS_OPENSSL
static final int	DEFAULT_HANDSHAKE_DETECTION_TIMEOUT_MILLIS
static final int	DEFAULT_HBASE_SERVER_NETTY_TLS_WRAP_SIZE
static final String	DEFAULT_PROTOCOL
private static final char[]	EMPTY_CHAR_ARRAY
static final String	HBASE_CLIENT_NETTY_TLS_ENABLED
static final String	HBASE_CLIENT_NETTY_TLS_HANDSHAKETIMEOUT
static final String	HBASE_CLIENT_NETTY_TLS_VERIFY_SERVER_HOSTNAME
static final String	HBASE_SERVER_NETTY_TLS_CLIENT_AUTH_MODE
static final String	HBASE_SERVER_NETTY_TLS_ENABLED
static final String	HBASE_SERVER_NETTY_TLS_SUPPORTPLAINTEXT
static final String	HBASE_SERVER_NETTY_TLS_VERIFY_CLIENT_HOSTNAME
static final String	HBASE_SERVER_NETTY_TLS_WRAP_SIZE	Set the SSL wrapSize for netty.
private static final org.slf4j.Logger	LOG
static final String	TLS_CERT_RELOAD
static final String	TLS_CIPHER_SUITES
static final String	TLS_CONFIG_CLR
static final String	TLS_CONFIG_KEYSTORE_LOCATION
static final String	TLS_CONFIG_KEYSTORE_PASSWORD
static final String	TLS_CONFIG_KEYSTORE_TYPE
static final String	TLS_CONFIG_OCSP
static final String	TLS_CONFIG_PROTOCOL
static final String	TLS_CONFIG_REVERSE_DNS_LOOKUP_ENABLED
static final String	TLS_CONFIG_TRUSTSTORE_LOCATION
static final String	TLS_CONFIG_TRUSTSTORE_PASSWORD
static final String	TLS_CONFIG_TRUSTSTORE_TYPE
static final String	TLS_ENABLED_PROTOCOLS
static final String	TLS_USE_OPENSSL

Constructor Summary

Constructors

Modifier	Constructor	Description
private	X509Util()

Method Summary

Modifier and Type	Method	Description
private static boolean	configureOpenSslIfAvailable(org.apache.hbase.thirdparty.io.netty.handler.ssl.SslContextBuilder sslContextBuilder, org.apache.hadoop.conf.Configuration conf)	Adds SslProvider.OPENSSL if OpenSsl is available and enabled.
(package private) static X509KeyManager	createKeyManager(String keyStoreLocation, char[] keyStorePassword, String keyStoreType)	Creates a key manager by loading the key store from the given file of the given type, optionally decrypting it using the given password.
static org.apache.hbase.thirdparty.io.netty.handler.ssl.SslContext	createSslContextForClient(org.apache.hadoop.conf.Configuration config)
static org.apache.hbase.thirdparty.io.netty.handler.ssl.SslContext	createSslContextForServer(org.apache.hadoop.conf.Configuration config)
(package private) static X509TrustManager	createTrustManager(String trustStoreLocation, char[] trustStorePassword, String trustStoreType, boolean crlEnabled, boolean ocspEnabled, boolean verifyHostName, boolean allowReverseDnsLookup)	Creates a trust manager by loading the trust store from the given file of the given type, optionally decrypting it using the given password.
static void	enableCertFileReloading(org.apache.hadoop.conf.Configuration config, AtomicReference<FileChangeWatcher> keystoreWatcher, AtomicReference<FileChangeWatcher> trustStoreWatcher, Runnable resetContext)	Enable certificate file reloading by creating FileWatchers for keystore and truststore.
private static String[]	getCBCCiphers()
private static String[]	getCipherSuites(org.apache.hadoop.conf.Configuration config, boolean useOpenSsl)
(package private) static String[]	getDefaultCipherSuites(boolean useOpenSsl)
(package private) static String[]	getDefaultCipherSuitesForJavaVersion(String javaVersion)
private static String[]	getEnabledProtocols(org.apache.hadoop.conf.Configuration config)
private static String[]	getGCMCiphers()
private static String[]	getOpenSslFilteredDefaultCiphers()	Not all of our default ciphers are available in OpenSSL.
private static String[]	getTls13Ciphers()
private static void	handleWatchEvent(Path filePath, WatchEvent<?> event, Runnable resetContext)	Handler for watch events that let us know a file we may care about has changed on disk.
private static FileChangeWatcher	newFileChangeWatcher(String fileLocation, Runnable resetContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

EMPTY_CHAR_ARRAY

private static final char[] EMPTY_CHAR_ARRAY

CONFIG_PREFIX

static final String CONFIG_PREFIX

See Also:

• Constant Field Values

TLS_CONFIG_PROTOCOL

public static final String TLS_CONFIG_PROTOCOL

See Also:

• Constant Field Values

TLS_CONFIG_KEYSTORE_LOCATION

public static final String TLS_CONFIG_KEYSTORE_LOCATION

See Also:

• Constant Field Values

TLS_CONFIG_KEYSTORE_TYPE

public static final String TLS_CONFIG_KEYSTORE_TYPE

See Also:

• Constant Field Values

TLS_CONFIG_KEYSTORE_PASSWORD

public static final String TLS_CONFIG_KEYSTORE_PASSWORD

See Also:

• Constant Field Values

TLS_CONFIG_TRUSTSTORE_LOCATION

public static final String TLS_CONFIG_TRUSTSTORE_LOCATION

See Also:

• Constant Field Values

TLS_CONFIG_TRUSTSTORE_TYPE

public static final String TLS_CONFIG_TRUSTSTORE_TYPE

See Also:

• Constant Field Values

TLS_CONFIG_TRUSTSTORE_PASSWORD

public static final String TLS_CONFIG_TRUSTSTORE_PASSWORD

See Also:

• Constant Field Values

TLS_CONFIG_CLR

public static final String TLS_CONFIG_CLR

See Also:

• Constant Field Values

TLS_CONFIG_OCSP

public static final String TLS_CONFIG_OCSP

See Also:

• Constant Field Values

TLS_CONFIG_REVERSE_DNS_LOOKUP_ENABLED

public static final String TLS_CONFIG_REVERSE_DNS_LOOKUP_ENABLED

See Also:

• Constant Field Values

TLS_ENABLED_PROTOCOLS

public static final String TLS_ENABLED_PROTOCOLS

See Also:

• Constant Field Values

TLS_CIPHER_SUITES

public static final String TLS_CIPHER_SUITES

See Also:

• Constant Field Values

TLS_CERT_RELOAD

public static final String TLS_CERT_RELOAD

See Also:

• Constant Field Values

TLS_USE_OPENSSL

public static final String TLS_USE_OPENSSL

See Also:

• Constant Field Values

DEFAULT_PROTOCOL

public static final String DEFAULT_PROTOCOL

See Also:

• Constant Field Values

HBASE_SERVER_NETTY_TLS_ENABLED

public static final String HBASE_SERVER_NETTY_TLS_ENABLED

See Also:

• Constant Field Values

HBASE_SERVER_NETTY_TLS_CLIENT_AUTH_MODE

public static final String HBASE_SERVER_NETTY_TLS_CLIENT_AUTH_MODE

See Also:

• Constant Field Values

HBASE_SERVER_NETTY_TLS_VERIFY_CLIENT_HOSTNAME

public static final String HBASE_SERVER_NETTY_TLS_VERIFY_CLIENT_HOSTNAME

See Also:

• Constant Field Values

HBASE_SERVER_NETTY_TLS_SUPPORTPLAINTEXT

public static final String HBASE_SERVER_NETTY_TLS_SUPPORTPLAINTEXT

See Also:

• Constant Field Values

HBASE_SERVER_NETTY_TLS_WRAP_SIZE

public static final String HBASE_SERVER_NETTY_TLS_WRAP_SIZE

Set the SSL wrapSize for netty. This is only a maximum wrap size. Buffers smaller than this will not be consolidated, but buffers larger than this will be split into multiple wrap buffers. The netty default of 16k is not great for hbase which tends to return larger payloads than that, meaning most responses end up getting chunked up. This leads to more memory contention in netty's PoolArena. See https://github.com/netty/netty/pull/13551

See Also:

• Constant Field Values

DEFAULT_HBASE_SERVER_NETTY_TLS_WRAP_SIZE

public static final int DEFAULT_HBASE_SERVER_NETTY_TLS_WRAP_SIZE

See Also:

• Constant Field Values

HBASE_CLIENT_NETTY_TLS_ENABLED

public static final String HBASE_CLIENT_NETTY_TLS_ENABLED

See Also:

• Constant Field Values

HBASE_CLIENT_NETTY_TLS_VERIFY_SERVER_HOSTNAME

public static final String HBASE_CLIENT_NETTY_TLS_VERIFY_SERVER_HOSTNAME

See Also:

• Constant Field Values

HBASE_CLIENT_NETTY_TLS_HANDSHAKETIMEOUT

public static final String HBASE_CLIENT_NETTY_TLS_HANDSHAKETIMEOUT

See Also:

• Constant Field Values

DEFAULT_HANDSHAKE_DETECTION_TIMEOUT_MILLIS

public static final int DEFAULT_HANDSHAKE_DETECTION_TIMEOUT_MILLIS

See Also:

• Constant Field Values

DEFAULT_CIPHERS_JAVA8

private static final String[] DEFAULT_CIPHERS_JAVA8

DEFAULT_CIPHERS_JAVA9

private static final String[] DEFAULT_CIPHERS_JAVA9

DEFAULT_CIPHERS_JAVA11

private static final String[] DEFAULT_CIPHERS_JAVA11

DEFAULT_CIPHERS_OPENSSL

private static final String[] DEFAULT_CIPHERS_OPENSSL

Constructor Details

X509Util

private X509Util()

Method Details

getTls13Ciphers

private static String[] getTls13Ciphers()

getGCMCiphers

private static String[] getGCMCiphers()

getCBCCiphers

private static String[] getCBCCiphers()

getOpenSslFilteredDefaultCiphers

private static String[] getOpenSslFilteredDefaultCiphers()

Not all of our default ciphers are available in OpenSSL. Takes our default cipher lists and filters them to only those available in OpenSsl. Prefers TLS 1.3, then GCM, then CBC because GCM tends to be better and faster, and we don't need to worry about the java8 vs 9 performance issue if OpenSSL is handling it.

getDefaultCipherSuites

static String[] getDefaultCipherSuites(boolean useOpenSsl)

getDefaultCipherSuitesForJavaVersion

static String[] getDefaultCipherSuitesForJavaVersion(String javaVersion)

createSslContextForClient

public static org.apache.hbase.thirdparty.io.netty.handler.ssl.SslContext createSslContextForClient(org.apache.hadoop.conf.Configuration config)
                                                                                             throws X509Exception,
IOException

Throws:

X509Exception

IOException

configureOpenSslIfAvailable

private static boolean configureOpenSslIfAvailable(org.apache.hbase.thirdparty.io.netty.handler.ssl.SslContextBuilder sslContextBuilder,
 org.apache.hadoop.conf.Configuration conf)

Adds SslProvider.OPENSSL if OpenSsl is available and enabled. In order to make it available, one must ensure that a properly shaded netty-tcnative is on the classpath. Properly shaded means relocated to be prefixed with "org.apache.hbase.thirdparty" like the rest of the netty classes. We make available org.apache.hbase:hbase-openssl as a convenience module which one can use to pull in a shaded netty-tcnative statically linked against boringssl.

createSslContextForServer

public static org.apache.hbase.thirdparty.io.netty.handler.ssl.SslContext createSslContextForServer(org.apache.hadoop.conf.Configuration config)
                                                                                             throws X509Exception,
IOException

Throws:

X509Exception

IOException

createKeyManager

static X509KeyManager createKeyManager(String keyStoreLocation,
 char[] keyStorePassword,
 String keyStoreType)
                                throws KeyManagerException

Creates a key manager by loading the key store from the given file of the given type, optionally decrypting it using the given password.

Parameters:

keyStoreLocation - the location of the key store file.

keyStorePassword - optional password to decrypt the key store. If empty, assumes the key store is not encrypted.

keyStoreType - must be JKS, PEM, PKCS12, BCFKS or null. If null, attempts to autodetect the key store type from the file extension (e.g. .jks / .pem).

Returns:

the key manager.

Throws:

KeyManagerException - if something goes wrong.

createTrustManager

static X509TrustManager createTrustManager(String trustStoreLocation,
 char[] trustStorePassword,
 String trustStoreType,
 boolean crlEnabled,
 boolean ocspEnabled,
 boolean verifyHostName,
 boolean allowReverseDnsLookup)
                                    throws TrustManagerException

Creates a trust manager by loading the trust store from the given file of the given type, optionally decrypting it using the given password.

Parameters:

trustStoreLocation - the location of the trust store file.

trustStorePassword - optional password to decrypt the trust store (only applies to JKS trust stores). If empty, assumes the trust store is not encrypted.

trustStoreType - must be JKS, PEM, PKCS12, BCFKS or null. If null, attempts to autodetect the trust store type from the file extension (e.g. .jks / .pem).

crlEnabled - enable CRL (certificate revocation list) checks.

ocspEnabled - enable OCSP (online certificate status protocol) checks.

verifyHostName - if true, ssl peer hostname must match name in certificate

allowReverseDnsLookup - if true, allow falling back to reverse dns lookup in verifying hostname

Returns:

the trust manager.

Throws:

TrustManagerException - if something goes wrong.

getEnabledProtocols

private static String[] getEnabledProtocols(org.apache.hadoop.conf.Configuration config)

getCipherSuites

private static String[] getCipherSuites(org.apache.hadoop.conf.Configuration config,
 boolean useOpenSsl)

enableCertFileReloading

public static void enableCertFileReloading(org.apache.hadoop.conf.Configuration config,
 AtomicReference<FileChangeWatcher> keystoreWatcher,
 AtomicReference<FileChangeWatcher> trustStoreWatcher,
 Runnable resetContext)
                                    throws IOException

Enable certificate file reloading by creating FileWatchers for keystore and truststore. AtomicReferences will be set with the new instances. resetContext - if not null - will be called when the file has been modified.

Parameters:

keystoreWatcher - Reference to keystoreFileWatcher.

trustStoreWatcher - Reference to truststoreFileWatcher.

resetContext - Callback for file changes.

Throws:

IOException

newFileChangeWatcher

private static FileChangeWatcher newFileChangeWatcher(String fileLocation,
 Runnable resetContext)
                                               throws IOException

Throws:

IOException

handleWatchEvent

private static void handleWatchEvent(Path filePath,
 WatchEvent<?> event,
 Runnable resetContext)

Handler for watch events that let us know a file we may care about has changed on disk.

Parameters:

filePath - the path to the file we are watching for changes.

event - the WatchEvent.