Transport Layer Security (TLS)
Transport Layer Security (TLS, formerly Secure Sockets Layer/SSL) is the recommended way to to protect integrity and confidentiality while data is transferred over an untrusted network connection, and to identify the endpoint. At this chapter we describe the available libraries in Fedora as well as known pitfalls, and safe ways to write applications with them.
When using any library, in addition to this guide, it is recommended to consult the library' documentation.
Common Pitfalls
TLS implementations are difficult to use, and most of them lack a clean API design. The following sections contain implementation-specific advice, and some generic pitfalls are mentioned below.
-
Most TLS implementations have questionable default TLS cipher suites. Most of them enable anonymous Diffie-Hellman key exchange (but we generally want servers to authenticate themselves). Many do not disable ciphers which are subject to brute-force attacks because of restricted key lengths. Some even disable all variants of AES in the default configuration.
When overriding the cipher suite defaults, it is recommended to disable all cipher suites which are not present on a whitelist, instead of simply enabling a list of cipher suites. This way, if an algorithm is disabled by default in the TLS implementation in a future security update, the application will not re-enable it.
-
The name which is used in certificate validation must match the name provided by the user or configuration file. No host name canonicalization or IP address lookup must be performed.
-
The TLS handshake has very poor performance if the TCP Nagle algorithm is active. You should switch on the
TCP_NODELAYsocket option (at least for the duration of the handshake), or use the Linux-specificTCP_CORKoption.Приклад 1. Deactivating the TCP Nagle algorithmconst int val = 1; int ret = setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &val, sizeof(val)); if (ret < 0) { perror("setsockopt(TCP_NODELAY)"); exit(1); } -
Implementing proper session resumption decreases handshake overhead considerably. This is important if the upper-layer protocol uses short-lived connections (like most application of HTTPS).
-
Both client and server should work towards an orderly connection shutdown, that is send
close_notifyalerts and respond to them. This is especially important if the upper-layer protocol does not provide means to detect connection truncation (like some uses of HTTP). -
When implementing a server using event-driven programming, it is important to handle the TLS handshake properly because it includes multiple network round-trips which can block when an ordinary TCP
acceptwould not. Otherwise, a client which fails to complete the TLS handshake for some reason will prevent the server from handling input from other clients. -
Unlike regular file descriptors, TLS connections cannot be passed between processes. Some TLS implementations add additional restrictions, and TLS connections generally cannot be used across
forkfunction calls (seeforkas a Primitive for Parallelism).
OpenSSL Pitfalls
Some OpenSSL function use tri-state return
values. Correct error checking is extremely
important. Several functions return int
values with the following meaning:
-
The value
1indicates success (for example, a successful signature verification). -
The value
0indicates semantic failure (for example, a signature verification which was unsuccessful because the signing certificate was self-signed). -
The value
-1indicates a low-level error in the system, such as failure to allocate memory usingmalloc.
Treating such tri-state return values as booleans can lead to security vulnerabilities. Note that some OpenSSL functions return boolean results or yet another set of status indicators. Each function needs to be checked individually.
Recovering precise error information is difficult.
Obtaining OpenSSL error codes
shows how to obtain a more precise error code after a function
call on an SSL object has failed. However,
there are still cases where no detailed error information is
available (e.g., if SSL_shutdown fails
due to a connection teardown by the other end).
static void __attribute__((noreturn))
ssl_print_error_and_exit(SSL *ssl, const char *op, int ret)
{
int subcode = SSL_get_error(ssl, ret);
switch (subcode) {
case SSL_ERROR_NONE:
fprintf(stderr, "error: %s: no error to report\n", op);
break;
case SSL_ERROR_WANT_READ:
case SSL_ERROR_WANT_WRITE:
case SSL_ERROR_WANT_X509_LOOKUP:
case SSL_ERROR_WANT_CONNECT:
case SSL_ERROR_WANT_ACCEPT:
fprintf(stderr, "error: %s: invalid blocking state %d\n", op, subcode);
break;
case SSL_ERROR_SSL:
fprintf(stderr, "error: %s: TLS layer problem\n", op);
case SSL_ERROR_SYSCALL:
fprintf(stderr, "error: %s: system call failed: %s\n", op, strerror(errno));
break;
case SSL_ERROR_ZERO_RETURN:
fprintf(stderr, "error: %s: zero return\n", op);
}
exit(1);
}
The OPENSSL_config function is
documented to never fail. In reality, it can terminate the
entire process if there is a failure accessing the
configuration file. An error message is written to standard
error, but which might not be visible if the function is
called from a daemon process.
OpenSSL contains two separate ASN.1 DER decoders. One set
of decoders operate on BIO handles (the input/output stream
abstraction provided by OpenSSL); their decoder function
names start with d2i_ and end in
_fp or _bio (e.g.,
d2i_X509_fp or
d2i_X509_bio). These decoders must not
be used for parsing data from untrusted sources; instead,
the variants without the _fp and
_bio (e.g.,
d2i_X509) shall be used. The BIO
variants have received considerably less testing and are not
very robust.
For the same reason, the OpenSSL command line tools (such as
openssl x509) are generally generally less
robust than the actual library code. They use the BIO
functions internally, and not the more robust variants.
The command line tools do not always indicate failure in the
exit status of the openssl process.
For instance, a verification failure in openssl
verify result in an exit status of zero.
OpenSSL command-line commands, such as openssl
genrsa, do not ensure that physical entropy is used
for key generation—they obtain entropy from
/dev/urandom and other sources, but not
from /dev/random. This can result in
weak keys if the system lacks a proper entropy source (e.g., a
virtual machine with solid state storage). Depending on local
policies, keys generated by these OpenSSL tools should not be
used in high-value, critical functions.
The OpenSSL server and client applications (openssl
s_client and openssl s_server)
are debugging tools and should never be
used as generic clients. For instance, the
s_client tool reacts in a
surprising way to lines starting with R and
Q.
OpenSSL allows application code to access private key material over documented interfaces. This can significantly increase the part of the code base which has to undergo security certification.
GnuTLS Pitfalls
Older versions of GnuTLS had several peculiarities described in previous versions of this guide; as of GnuTLS 3.3.10, these issues are no longer applicable.
OpenJDK Pitfalls
The Java cryptographic framework is highly modular. As a result, when you request an object implementing some cryptographic functionality, you cannot be completely sure that you end up with the well-tested, reviewed implementation in OpenJDK.
OpenJDK (in the source code as published by Oracle) and other implementations of the Java platform require that the system administrator has installed so-called unlimited strength jurisdiction policy files. Without this step, it is not possible to use the secure algorithms which offer sufficient cryptographic strength. Most downstream redistributors of OpenJDK remove this requirement.
Some versions of OpenJDK use /dev/random
as the randomness source for nonces and other random data
which is needed for TLS operation, but does not actually
require physical randomness. As a result, TLS applications
can block, waiting for more bits to become available in
/dev/random.
NSS Pitfalls
NSS was not designed to be used by other libraries which can be linked into applications without modifying them. There is a lot of global state. There does not seem to be a way to perform required NSS initialization without race conditions.
If the NSPR descriptor is in an unexpected state, the
SSL_ForceHandshake function can succeed,
but no TLS handshake takes place, the peer is not
authenticated, and subsequent data is exchanged in the clear.
NSS disables itself if it detects that the process underwent a
fork after the library has been
initialized. This behavior is required by the PKCS#11 API
specification.
TLS Clients
Secure use of TLS in a client generally involves all of the following steps. (Individual instructions for specific TLS implementations follow in the next sections.)
-
The client must configure the TLS library to use a set of trusted root certificates. These certificates are provided by the system in various formats and files. These are documented in
update-ca-trustman page in Fedora. Portable applications should not hard-code any paths; they should rely on APIs which set the default for the system trust store. -
The client selects sufficiently strong cryptographic primitives and disables insecure ones (such as no-op encryption). Compression support and SSL version 3 or lower must be disabled (including the SSLv2-compatible handshake).
-
The client initiates the TLS connection. The Server Name Indication extension should be used if supported by the TLS implementation. Before switching to the encrypted connection state, the contents of all input and output buffers must be discarded.
-
The client needs to validate the peer certificate provided by the server, that is, the client must check that there is a cryptographically protected chain from a trusted root certificate to the peer certificate. (Depending on the TLS implementation, a TLS handshake can succeed even if the certificate cannot be validated.)
-
The client must check that the configured or user-provided server name matches the peer certificate provided by the server.
It is safe to provide users detailed diagnostics on certificate validation failures. Other causes of handshake failures and, generally speaking, any details on other errors reported by the TLS implementation (particularly exception tracebacks), must not be divulged in ways that make them accessible to potential attackers. Otherwise, it is possible to create decryption oracles.
|
Depending on the application, revocation checking (against certificate revocations lists or via OCSP) and session resumption are important aspects of production-quality client. These aspects are not yet covered. |
Implementation TLS Clients With OpenSSL
In the following code, the error handling is only exploratory. Proper error handling is required for production use, especially in libraries.
The OpenSSL library needs explicit initialization (see OpenSSL library initialization).
// The following call prints an error message and calls exit() if
// the OpenSSL configuration file is unreadable.
OPENSSL_config(NULL);
// Provide human-readable error messages.
SSL_load_error_strings();
// Register ciphers.
SSL_library_init();
After that, a context object has to be created, which acts as a factory for connection objects (OpenSSL client context creation). We use an explicit cipher list so that we do not pick up any strange ciphers when OpenSSL is upgraded. The actual version requested in the client hello depends on additional restrictions in the OpenSSL library. If possible, you should follow the example code and use the default list of trusted root certificate authorities provided by the system because you would have to maintain your own set otherwise, which can be cumbersome.
// Configure a client connection context. Send a hendshake for the
// highest supported TLS version, and disable compression.
const SSL_METHOD *const req_method = SSLv23_client_method();
SSL_CTX *const ctx = SSL_CTX_new(req_method);
if (ctx == NULL) {
ERR_print_errors(bio_err);
exit(1);
}
SSL_CTX_set_options(ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_COMPRESSION);
// Adjust the ciphers list based on a whitelist. First enable all
// ciphers of at least medium strength, to get the list which is
// compiled into OpenSSL.
if (SSL_CTX_set_cipher_list(ctx, "HIGH:MEDIUM") != 1) {
ERR_print_errors(bio_err);
exit(1);
}
{
// Create a dummy SSL session to obtain the cipher list.
SSL *ssl = SSL_new(ctx);
if (ssl == NULL) {
ERR_print_errors(bio_err);
exit(1);
}
STACK_OF(SSL_CIPHER) *active_ciphers = SSL_get_ciphers(ssl);
if (active_ciphers == NULL) {
ERR_print_errors(bio_err);
exit(1);
}
// Whitelist of candidate ciphers.
static const char *const candidates[] = {
"AES128-GCM-SHA256", "AES128-SHA256", "AES256-SHA256", // strong ciphers
"AES128-SHA", "AES256-SHA", // strong ciphers, also in older versions
"RC4-SHA", "RC4-MD5", // backwards compatibility, supposed to be weak
"DES-CBC3-SHA", "DES-CBC3-MD5", // more backwards compatibility
NULL
};
// Actually selected ciphers.
char ciphers[300];
ciphers[0] = '\0';
for (const char *const *c = candidates; *c; ++c) {
for (int i = 0; i < sk_SSL_CIPHER_num(active_ciphers); ++i) {
if (strcmp(SSL_CIPHER_get_name(sk_SSL_CIPHER_value(active_ciphers, i)),
*c) == 0) {
if (*ciphers) {
strcat(ciphers, ":");
}
strcat(ciphers, *c);
break;
}
}
}
SSL_free(ssl);
// Apply final cipher list.
if (SSL_CTX_set_cipher_list(ctx, ciphers) != 1) {
ERR_print_errors(bio_err);
exit(1);
}
}
// Load the set of trusted root certificates.
if (!SSL_CTX_set_default_verify_paths(ctx)) {
ERR_print_errors(bio_err);
exit(1);
}
A single context object can be used to create multiple
connection objects. It is safe to use the same
SSL_CTX object for creating connections
concurrently from multiple threads, provided that the
SSL_CTX object is not modified (e.g.,
callbacks must not be changed).
After creating the TCP socket and disabling the Nagle
algorithm (per Deactivating the TCP Nagle algorithm), the actual
connection object needs to be created, as show in OpenSSL client context creation. If
the handshake started by SSL_connect
fails, the ssl_print_error_and_exit
function from Obtaining OpenSSL error codes is called.
The certificate_validity_override
function provides an opportunity to override the validity of
the certificate in case the OpenSSL check fails. If such
functionality is not required, the call can be removed,
otherwise, the application developer has to implement it.
The host name passed to the functions
SSL_set_tlsext_host_name and
X509_check_host must be the name that was
passed to getaddrinfo or a similar name
resolution function. No host name canonicalization must be
performed. The X509_check_host function
used in the final step for host name matching is currently
only implemented in OpenSSL 1.1, which is not released yet.
In case host name matching fails, the function
certificate_host_name_override is called.
This function should check user-specific certificate store, to
allow a connection even if the host name does not match the
certificate. This function has to be provided by the
application developer. Note that the override must be keyed
by both the certificate and the host
name.
// Create the connection object.
SSL *ssl = SSL_new(ctx);
if (ssl == NULL) {
ERR_print_errors(bio_err);
exit(1);
}
SSL_set_fd(ssl, sockfd);
// Enable the ServerNameIndication extension
if (!SSL_set_tlsext_host_name(ssl, host)) {
ERR_print_errors(bio_err);
exit(1);
}
// Perform the TLS handshake with the server.
ret = SSL_connect(ssl);
if (ret != 1) {
// Error status can be 0 or negative.
ssl_print_error_and_exit(ssl, "SSL_connect", ret);
}
// Obtain the server certificate.
X509 *peercert = SSL_get_peer_certificate(ssl);
if (peercert == NULL) {
fprintf(stderr, "peer certificate missing");
exit(1);
}
// Check the certificate verification result. Allow an explicit
// certificate validation override in case verification fails.
int verifystatus = SSL_get_verify_result(ssl);
if (verifystatus != X509_V_OK && !certificate_validity_override(peercert)) {
fprintf(stderr, "SSL_connect: verify result: %s\n",
X509_verify_cert_error_string(verifystatus));
exit(1);
}
// Check if the server certificate matches the host name used to
// establish the connection.
// FIXME: Currently needs OpenSSL 1.1.
if (X509_check_host(peercert, (const unsigned char *)host, strlen(host),
0) != 1
&& !certificate_host_name_override(peercert, host)) {
fprintf(stderr, "SSL certificate does not match host name\n");
exit(1);
}
X509_free(peercert);
The connection object can be used for sending and receiving
data, as in Using an OpenSSL connection to send and receive data.
It is also possible to create a BIO object
and use the SSL object as the underlying
transport, using BIO_set_ssl.
const char *const req = "GET / HTTP/1.0\r\n\r\n";
if (SSL_write(ssl, req, strlen(req)) < 0) {
ssl_print_error_and_exit(ssl, "SSL_write", ret);
}
char buf[4096];
ret = SSL_read(ssl, buf, sizeof(buf));
if (ret < 0) {
ssl_print_error_and_exit(ssl, "SSL_read", ret);
}
When it is time to close the connection, the
SSL_shutdown function needs to be called
twice for an orderly, synchronous connection termination
(Closing an OpenSSL connection in an orderly fashion).
This exchanges close_notify alerts with the
server. The additional logic is required to deal with an
unexpected close_notify from the server.
Note that is necessary to explicitly close the underlying
socket after the connection object has been freed.
// Send the close_notify alert.
ret = SSL_shutdown(ssl);
switch (ret) {
case 1:
// A close_notify alert has already been received.
break;
case 0:
// Wait for the close_notify alert from the peer.
ret = SSL_shutdown(ssl);
switch (ret) {
case 0:
fprintf(stderr, "info: second SSL_shutdown returned zero\n");
break;
case 1:
break;
default:
ssl_print_error_and_exit(ssl, "SSL_shutdown 2", ret);
}
break;
default:
ssl_print_error_and_exit(ssl, "SSL_shutdown 1", ret);
}
SSL_free(ssl);
close(sockfd);
Closing an OpenSSL connection in an orderly fashion shows how to deallocate the context object when it is no longer needed because no further TLS connections will be established.
SSL_CTX_free(ctx);
Implementation TLS Clients With GnuTLS
This section describes how to implement a TLS client with full certificate validation (but without certificate revocation checking). Note that the error handling in is only exploratory and needs to be replaced before production use.
Before setting up TLS connections, a credentials objects has to be allocated and initialized with the set of trusted root CAs (Initializing a GnuTLS credentials structure).
// Load the trusted CA certificates.
gnutls_certificate_credentials_t cred = NULL;
int ret = gnutls_certificate_allocate_credentials (&cred);
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_certificate_allocate_credentials: %s\n",
gnutls_strerror(ret));
exit(1);
}
ret = gnutls_certificate_set_x509_system_trust(cred);
if (ret == 0) {
fprintf(stderr, "error: no certificates found in system trust store\n");
exit(1);
}
if (ret < 0) {
fprintf(stderr, "error: gnutls_certificate_set_x509_system_trust: %s\n",
gnutls_strerror(ret));
exit(1);
}
After the last TLS connection has been closed, this credentials object should be freed:
gnutls_certificate_free_credentials(cred);
During its lifetime, the credentials object can be used to initialize TLS session objects from multiple threads, provided that it is not changed.
Once the TCP connection has been established, the Nagle algorithm should be disabled (see Deactivating the TCP Nagle algorithm). After that, the socket can be associated with a new GnuTLS session object. The previously allocated credentials object provides the set of root CAs. Then the TLS handshake must be initiated. This is shown in Establishing a TLS client connection using GnuTLS.
// Create the session object.
gnutls_session_t session;
ret = gnutls_init(&session, GNUTLS_CLIENT);
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_init: %s\n",
gnutls_strerror(ret));
exit(1);
}
// Configure the cipher preferences.
const char *errptr = NULL;
ret = gnutls_priority_set_direct(session, "NORMAL", &errptr);
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_priority_set_direct: %s\n"
"error: at: \"%s\"\n", gnutls_strerror(ret), errptr);
exit(1);
}
// Install the trusted certificates.
ret = gnutls_credentials_set(session, GNUTLS_CRD_CERTIFICATE, cred);
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_credentials_set: %s\n",
gnutls_strerror(ret));
exit(1);
}
// Associate the socket with the session object and set the server
// name.
gnutls_transport_set_int(session, sockfd);
ret = gnutls_server_name_set(session, GNUTLS_NAME_DNS,
host, strlen(host));
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_server_name_set: %s\n",
gnutls_strerror(ret));
exit(1);
}
// Establish the session.
ret = gnutls_handshake(session);
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_handshake: %s\n",
gnutls_strerror(ret));
exit(1);
}
After the handshake has been completed, the server certificate
needs to be verified against the server’s hostname (Verifying a server certificate using GnuTLS). In
the example, the user-defined
certificate_validity_override function is
called if the verification fails, so that a separate,
user-specific trust store can be checked. This function call
can be omitted if the functionality is not needed.
// Obtain the server certificate chain. The server certificate
// itself is stored in the first element of the array.
unsigned certslen = 0;
const gnutls_datum_t *const certs =
gnutls_certificate_get_peers(session, &certslen);
if (certs == NULL || certslen == 0) {
fprintf(stderr, "error: could not obtain peer certificate\n");
exit(1);
}
// Validate the certificate chain.
unsigned status = (unsigned)-1;
ret = gnutls_certificate_verify_peers3(session, host, &status);
if (ret != GNUTLS_E_SUCCESS) {
fprintf(stderr, "error: gnutls_certificate_verify_peers3: %s\n",
gnutls_strerror(ret));
exit(1);
}
if (status != 0 && !certificate_validity_override(certs[0])) {
gnutls_datum_t msg;
#if GNUTLS_VERSION_AT_LEAST_3_1_4
int type = gnutls_certificate_type_get (session);
ret = gnutls_certificate_verification_status_print(status, type, &out, 0);
#else
ret = -1;
#endif
if (ret == 0) {
fprintf(stderr, "error: %s\n", msg.data);
gnutls_free(msg.data);
exit(1);
} else {
fprintf(stderr, "error: certificate validation failed with code 0x%x\n",
status);
exit(1);
}
}
An established TLS session can be used for sending and receiving data, as in Using a GnuTLS session.
char buf[4096];
snprintf(buf, sizeof(buf), "GET / HTTP/1.0\r\nHost: %s\r\n\r\n", host);
ret = gnutls_record_send(session, buf, strlen(buf));
if (ret < 0) {
fprintf(stderr, "error: gnutls_record_send: %s\n", gnutls_strerror(ret));
exit(1);
}
ret = gnutls_record_recv(session, buf, sizeof(buf));
if (ret < 0) {
fprintf(stderr, "error: gnutls_record_recv: %s\n", gnutls_strerror(ret));
exit(1);
}
In order to shut down a connection in an orderly manner, you
should call the gnutls_bye function.
Finally, the session object can be deallocated using
gnutls_deinit (see Closing a GnuTLS session in an orderly fashion).
// Initiate an orderly connection shutdown.
ret = gnutls_bye(session, GNUTLS_SHUT_RDWR);
if (ret < 0) {
fprintf(stderr, "error: gnutls_bye: %s\n", gnutls_strerror(ret));
exit(1);
}
// Free the session object.
gnutls_deinit(session);
Implementing TLS Clients With OpenJDK
The examples below use the following cryptographic-related classes:
import java.security.NoSuchAlgorithmException;
import java.security.NoSuchProviderException;
import java.security.cert.CertificateEncodingException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import sun.security.util.HostnameChecker;
If compatibility with OpenJDK 6 is required, it is necessary
to use the internal class
sun.security.util.HostnameChecker. (The
public OpenJDK API does not provide any support for dissecting
the subject distinguished name of an X.509 certificate, so a
custom-written DER parser is needed—or we have to use an
internal class, which we do below.) In OpenJDK 7, the
setEndpointIdentificationAlgorithm method
was added to the
javax.net.ssl.SSLParameters class,
providing an official way to implement host name checking.
TLS connections are established using an
SSLContext instance. With a properly
configured OpenJDK installation, the
SunJSSE provider uses the system-wide set
of trusted root certificate authorities, so no further
configuration is necessary. For backwards compatibility with
OpenJDK6, the TLSv1 provider has to
be supported as a fall-back option. This is shown in Setting up an SSLContext for OpenJDK TLS clients.
SSLContext for OpenJDK TLS clients// Create the context. Specify the SunJSSE provider to avoid
// picking up third-party providers. Try the TLS 1.2 provider
// first, then fall back to TLS 1.0.
SSLContext ctx;
try {
ctx = SSLContext.getInstance("TLSv1.2", "SunJSSE");
} catch (NoSuchAlgorithmException e) {
try {
ctx = SSLContext.getInstance("TLSv1", "SunJSSE");
} catch (NoSuchAlgorithmException e1) {
// The TLS 1.0 provider should always be available.
throw new AssertionError(e1);
} catch (NoSuchProviderException e1) {
throw new AssertionError(e1);
}
} catch (NoSuchProviderException e) {
// The SunJSSE provider should always be available.
throw new AssertionError(e);
}
ctx.init(null, null, null);
In addition to the context, a TLS parameter object will be needed which adjusts the cipher suites and