OpenSSL 3.0.0 Design (1) | Introduction, Terminology and Architecture

Translation｜Wang Zuxi (flower name: Jin Jiu)

The development engineer of Ant Group
is responsible for the development and maintenance of the localized cryptographic library Tongsuo** focusing on cryptography, high-performance network, network security and other fields

This article 4830 words read 13 minutes

This article translates the OpenSSL official website documentation: https://www.openssl.org/docs/OpenSSL300Design.html

Tongsuo-8.4.0 is developed based on OpenSSL-3.0.3, so this article is also applicable to Tongsuo developers. It is rich in content and worth reading!

Due to the length of the article, today’s release consists of three parts: introduction , terminology , and architecture . Subsequent content will be released in full every week. Please continue to pay attention to the official account of Tongsuo.

1►Introduction

This document outlines the design of OpenSSL 3.0, the release following 1.1.1. Reading this article assumes that you are already familiar with the document called "OpenSSL Strategic Architecture" and have a working knowledge of OpenSSL 1.1.x.

The OpenSSL 3.0 release has minimal impact on most existing applications; almost all applications that perform well will simply need to be recompiled.

Most of the changes in OpenSSL 3.0 are internal architectural reorganizations to facilitate a long-term supported cryptographic framework that better separates algorithm implementation from algorithm API. These architectural changes also help improve the maintainability of the OpenSSL FIPS Cryptographic Module 3.0.

APIs currently marked as deprecated will not be removed in OpenSSL 3.0 .

In OpenSSL 3.0, many additional low-level functions will be marked as deprecated API .

OpenSSL 3.0 will support applications with TLS connections in both FIPS mode (using the OpenSSL FIPS cryptographic module 3.0) and non-FIPS mode.

For more up-to-date information on version 3.0, please refer to the link at https://www.openssl.org/docs .

2► Terminology

The following terms are used in this document in alphabetical order and briefly defined as follows:

Algorithm : Sometimes called a cryptographic algorithm, is a method of performing a set of operations such as encryption or decryption . We use this term abstractly, and usually refer to an algorithm by a name (such as "aes-128-cbc") .
Algorithm implementation (Algorithm implementation) : sometimes referred to as implementation (implementation) , is the specific realization of the algorithm. This is mostly represented in code as a set of functions.
CAVS is a cryptographic algorithm verification system. This is a tool for testing cryptographic implementations for FIPS compliance.
CMVP is a cryptographic module verifier. This process verifies that the cryptographic implementation is FIPS compliant.
EVP is a set of APIs implemented by libcrypto that enable applications to perform cryptographic operations. The implementation of the EVP API uses Core and Provider components.
Core is a component in libcrypto that enables applications to access algorithm implementations provided by Providers.
CSP is a key security parameter (Critical Security Parameters) . This includes any information (such as private keys, passwords, PIN codes, etc.) that could compromise the security of the module in the event of unauthorized disclosure or modification .
Explicit fetch (Explicit Fetch) is a method to find the implementation of the algorithm. The application locates the implementation and provides search conditions through explicit calls.
FIPS is the Federal Information Processing Standards . This is a set of standards defined by the US government. In particular, the FIPS 140-2 standard applies to cryptographic software.
The FIPS module is a CMVP-validated implementation of FIPS-compliant cryptographic algorithms. In OpenSSL, FIPS modules are implemented as Providers and delivered as dynamically loadable modules.
Implicit fetch (Implicit Fetch) is a method to find the implementation of the algorithm. The application will not explicitly call to locate the implementation, so the default search criteria will be used.
Integrity Checks are tests that run automatically when a FIPS module is loaded. The module checks itself and verifies that it has not been maliciously modified.
KAS is a key agreement scheme (Key Agreement Scheme) . It is the method by which two communicating parties negotiate a shared secret key.
KAT is known answer test (Known Answer Tests) . It is a set of tests used to health check the FIPS module.
libcrypto is a shared library implemented by OpenSSL that provides various cryptography-related functions to applications.
libssl is a shared library implemented by OpenSSL that provides applications with the ability to create SSL/TLS connections, either as a client or as a server.
Library context (Library Context) is an opaque structure that holds the library's "global" data.
Operation is a type of function performed on data, such as calculating digests, encrypting, decrypting, and so on. An algorithm can provide one or more operations. For example, RSA provides asymmetric encryption, asymmetric decryption, signing, verification, and more.
Parameters (Parameters) is a set of key-value pairs that have nothing to do with implementation, and are used to pass object data between Core and Provider. For example, they can be used to transmit private key data.
POST refers to the FIPS module's power-on self-test (also known as power-on self-test) , which runs at installation, power-up (that is, every time the FIPS module is loaded for an application) , or on demand. These tests include Integrity Check and KAT. If KAT runs successfully at install time, it does not need to be run again at power-up, but integrity checks are always performed.
Properties (Properties) are used by Provider to describe the characteristics of its algorithm implementation. They are also used for application queries to find specific implementations.
A Provider is a unit that provides an implementation of one or more algorithms.
The Provider module is a Provider in the form of a dynamically loadable module.

3► Architecture

The architecture should have the following properties:

Common Services (Common Services) are building blocks shared by applications and Providers, such as BIO, X509, SECMEM, ASN.1, etc.
Providers implement cryptographic algorithms and supporting services. An algorithm may consist of multiple operations (eg RSA may have "encrypt", "decrypt", "sign", "verify", etc.) . Also, an operation (such as "signature") can be implemented by multiple algorithms, such as RSA and ECDSA. Provider contains the cryptographic primitive implementation of the algorithm. This release will include the following Providers:

a. Default Provider (Default) , which contains the current non -legacy OpenSSL cryptographic algorithms; this will be built-in (ie part of libcrypto) .

b. Legacy Provider (Legacy) , which contains the implementation of old algorithms (such as DES, MDC2, MD2, Blowfish, CAST) .

c. FIPS Provider, which implements the OpenSSL FIPS cryptographic module 3.0; it can be dynamically loaded at runtime.
Core enables applications (and other Providers) to access the operations provided by Providers. Core is the mechanism for the concrete implementation of positioning operations.
Protocol implementation, such as TLS, DTLS.

There are numerous references to the "EVP API" in this document. This refers to "application-level" operations such as signing with a public key, generating a digest, etc. These functions include EVP_DigestSign, EVP_Digest, EVP_MAC_init, etc. The EVP API also encapsulates the cryptographic objects used to perform these services, such as EVP_PKEY, EVP_CIPHER, EVP_MD, EVP_MAC, and so on. Provider implements backend functionality for the latter collection. Instances of these objects can be implicitly or explicitly bound to Providers according to the needs of the application. The Provider Design section below discusses this in detail.

The architecture has the following characteristics:

The EVP layer is a thin encapsulation of the operations implemented in the Provider, and most calls are passed directly, with little pre- or post-processing.
New EVP APIs will be provided to affect how the Core selects (or finds) the implementation of the operation to use in any given EVP call.
Passes information between libcrypto and Provider in an implementation-independent manner.
Legacy APIs (such as low-level cryptographic APIs that do not go through the EVP layer) will be deprecated . There are legacy APIs for non-legacy algorithms (eg AES is not a legacy algorithm, but AES_encrypt is a legacy API) .
The OpenSSL FIPS cryptographic module will be implemented as a dynamically loaded Provider, which will be self-contained (ie only depend on services provided by the system runtime library and core) .

Conceptual Component View

An overview of the conceptual components in the OpenSSL architecture is shown in the figure below. Note that the presence of a component in the diagram does not imply that the component is a public API or an end-user component intended for direct access or use.

The new components (which did not exist in the previous architecture) look like this:

Core: This is the basic component that connects a request for an operation (such as encryption) to a Provider that provides that operation. It provides the ability to locate an algorithm that implements a specified operation, given a set of properties that an implementation must satisfy. For example, the attribute of the encryption algorithm includes at least "fips".
Default Provider (Default Provider) : Implements a set of default algorithms.
FIPS Provider: Implements a set of FIPS-validated algorithms and provides access through the core. This includes the following support services:
POST: power-on self-test, including:

· KAT: Known Answer Test

· Integrity check
Low Level Implementations : The set of components that actually implement the cryptographic primitives to meet the self-contained requirements specified by FIPS.
Legacy Provider (Legacy Provider) : Provides the implementation of the old algorithm, which will be exposed through the EVP level API.
Third Party Provider (3rd Party Providers) : Ultimately, third parties may provide their own Providers. A third-party provider is like any other provider, implementing a set of algorithms, accessible through the Core, and visible to applications and other providers.
Null Provider (Null Provider) : A Provider that does nothing, which is very useful for testing the correct use of the library context.
Base Provider (Base Provider) : Provider for key serialization. It is required by the FIPS Provider because it does not itself contain a method for loading keys. The base Provider is also embedded in the default Provider.

packaged view

The individual components described in the conceptual component view above are physically packaged as:

Executable applications that can be used by users
Libraries for use by applications
Dynamically loadable modules for use by Core

OpenSSL 3.0 will offer several different packaging options (such as a single library called libcrypto containing everything except the FIPS Providers, and all Providers as separate dynamically loadable modules) .

Which dynamically loadable modules are registered, used or available can be configured at runtime, the following diagram describes the architecture in terms of physical packaging:

The physical packaging introduced in this release is as follows:

FIPS Module : This module contains a FIPS Provider that implements a set of FIPS-validated algorithms and is accessible through the Core. FIPS Provider is OpenSSL FIPS Cryptographic Module 3.0.

We don't try to prevent users from making mistakes, but we do consider typical user usage and "safety". By default, the FIPS Provider will be built and installed.

We will be able to perform security checks to detect if a user has modified the source code in a way that affects FIPS, and (best effort) prevent building the FIPS Provider if the override option is not provided.

We need to ensure that there is a mechanism by which end users can determine whether their use of a FIPS module complies with a formally validated allowable use.

The version of the FIPS module will align with the base OpenSSL version number upon which the version number at validation depends. Not all OpenSSL releases require an updated FIPS module. Therefore, when a new FIPS module version is released, there may be gaps or jumps in its version number compared to previous versions.
Legacy module : This module contains the implementation of the legacy algorithm.

The original plan was to build engines with Provider wrappers , so that when passing an ENGINE pointer to some functions, it would work as usual, and act as a Provider when acting as the default implementation. Investigations during development revealed that this approach has problematic edge cases. The current workaround is that there are currently two code paths when an EVP call is made. For engines support , old code using "legacy keys". The long-term plan is to remove engines and legacy code paths from the codebase . Once engines are removed , anything written as an engine needs to be rewritten as a Provider.

Next week we will bring the "Core and Provider Design" part, friends who can't wait, you can check the full document in Tongsuo Yuque: https://www.yuque.com/tsdoc/ts/openssl- 300-design #AP9Lt

Tongsuo/Tongsuo Star ✨: https://github.com/Tongsuo-Project/Tongsuo