[C++ Memory Management] Memory Pool

Article directory

1. C++17 memory_resource memory pool

C++17 introduces a new memory resource abstraction layer - memory_resource. Its main purpose is to decouple memory allocation and recycling from specific data structures, thereby allowing developers to use different memory management strategies and improve memory management flexibility. memory_resourceIs a base class that can be used to create and configure memory pools with different memory management behaviors.

1.1 memory_resource base class

memory_resourceIt is an abstract base class that provides three pure virtual functions that need to be implemented by subclasses:

do_allocate(size_t bytes, size_t alignment): Allocate memory according to the specified bytes and alignment.
do_deallocate(void* p, size_t bytes, size_t alignment): Release previously allocated memory.
do_is_equal(const memory_resource& other) const noexcept: Determine whether two memory resources are equivalent.

The following is memory_resourcea simple implementation of the base class:

class MyMemoryResource : public std::pmr::memory_resource
{
    
    
protected:
    void* do_allocate(size_t bytes, size_t alignment) override
    {
    
    
        return ::operator new(bytes);
    }

    void do_deallocate(void* p, size_t bytes, size_t alignment) override
    {
    
    
        ::operator delete(p);
    }

    bool do_is_equal(const memory_resource& other) const noexcept override
    {
    
    
        return this == &other;
    }
};

1.2 Usage of memory resources

To use it memory_resource, you usually need to create some containers through it. For example, we can memory_resourcecreate one with std::pmr::vector:

MyMemoryResource myMemoryResource;
std::pmr::vector<int> vec(&myMemoryResource);

In this way, vecwill be used myMemoryResourceto allocate and reclaim memory.

1.3 Global and local memory resources

C++ provides two types of memory resources: global memory resources and local memory resources. Global memory resources are std::pmr::get_default_resource()obtained through , and all std::pmrobjects that do not specify memory resources will use it. Local memory resources are std::pmrspecified when creating the object, and they will overwrite global memory resources.

For example:

std::pmr::vector<int> vec; // 使用全局内存资源
std::pmr::vector<int> vec(&myMemoryResource); // 使用本地内存资源

1.4 Other memory resource types

In addition to user-defined memory resources, C++ also provides several predefined memory resources:

std::pmr::new_delete_resource(): Use newand deletefor memory allocation and recycling.
std::pmr::null_memory_resource(): Any attempt to allocate memory will throw std::bad_allocan exception.
std::pmr::unsynchronized_pool_resourceand std::pmr::synchronized_pool_resource: memory pool resources, thread-unsafe and thread-safe versions respectively.

The following chapters will continue to introduce these two memory pool resources.

2. struct unsynchronized_pool_resource

unsynchronized_pool_resourcestd::pmr::memory_resourceIt is an implementation in C++17 . This memory resource management method exists in the form of a memory pool, and its main advantage is to reduce the overhead of memory allocation and recycling. This resource is designed to be thread-unsafe, so developers need to manage and synchronize it in a multi-threaded environment.

2.1 Basic principles of unsynchronized_pool_resource

unsynchronized_pool_resourceA series of memory pools are maintained internally, and each memory pool is responsible for managing memory blocks in a certain size range. When memory needs to be allocated, the memory pool resource selects an appropriate memory pool based on the requested size and allocates a memory block from it. If the corresponding memory pool has no available memory blocks, the memory pool resource will obtain more memory from the upstream memory resource.

2.2 Use of unsynchronized_pool_resource

To use unsynchronized_pool_resource, you need to create an unsynchronized_pool_resourceobject and use it as std::pmra memory resource for other objects. Here is an example:

std::pmr::unsynchronized_pool_resource pool;
std::pmr::vector<int> vec(&pool);

In this example, vecwill be used poolto allocate and deallocate memory.

2.3 Configuration of unsynchronized_pool_resource

unsynchronized_pool_resourceHas some configurable parameters including maximum and minimum block sizes, maximum number of blocks, etc. These parameters can std::pmr::pool_optionsbe set through the structure, which will be introduced in detail in the following chapters.

2.4 Thread safety of unsynchronized_pool_resource

As the name suggests, unsynchronized_pool_resourceit is thread-unsafe. If used in a multi-threaded environment unsynchronized_pool_resource, you must ensure that each thread uses its own unsynchronized_pool_resource, or is protected by some synchronization mechanism (such as a mutex) unsynchronized_pool_resource.

Although this increases unsynchronized_pool_resourcethe complexity of use, unsynchronized_pool_resourceit may be more efficient than a synchronized memory pool in some scenarios because it avoids the overhead of synchronization.

2.5 Comparison between unsynchronized_pool_resource and new/delete

Compared with using newand deleteperforming memory management, unsynchronized_pool_resourcethe main advantage is that it reduces the overhead of memory allocation and recycling. Especially for scenarios where small blocks of memory are frequently allocated and reclaimed, use unsynchronized_pool_resourcecan significantly improve performance.

In addition, because unsynchronized_pool_resourcememory blocks can be reclaimed and reused, it also reduces memory fragmentation, thereby improving memory utilization.

3. pool_options memory pool configuration

pool_optionsIs the structure used to configure unsynchronized_pool_resourceand in C++17 . synchronized_pool_resourceIt provides some parameters that can be used to adjust the behavior of the memory pool to adapt to different application scenarios.

3.1 Parameters of pool_options

pool_optionsThe structure includes the following parameters:

max_blocks_per_chunk: The maximum number of blocks that can be included in each block. This parameter is used to limit the amount of memory the memory pool obtains from upstream memory resources at one time. If set to 0, it means there is no limit.
largest_required_pool_block: Request the maximum block size that the memory pool can handle. This parameter is used to determine what size memory blocks the memory pool should manage.

Here is an example:

std::pmr::pool_options options;
options.max_blocks_per_chunk = 1024;
options.largest_required_pool_block = 512;

3.2 Use of pool_options

To use it , you need to pass it in as a parameter when pool_optionscreating unsynchronized_pool_resourceor . synchronized_pool_resourceHere is an example:

std::pmr::pool_options options;
options.max_blocks_per_chunk = 1024;
options.largest_required_pool_block = 512;

std::pmr::unsynchronized_pool_resource pool(options);

In this example, memory management poolwill be performed based on the parameters in .options

3.3 Impact of pool_options

By adjusting pool_optionsthe parameters in , you can influence the behavior of the memory pool to optimize memory usage.

For example, if you know that your application will frequently allocate and reclaim memory blocks of a certain size, you can set the largest_required_pool_blockmemory pool to specifically handle these memory blocks to improve memory utilization.

Similarly, if you know that your application will allocate a large number of memory blocks, you can max_blocks_per_chunkreduce the number of memory acquisitions from upstream memory resources by increasing the memory management overhead.

Overall, pool_optionsit provides a flexible way to configure the memory pool so that it can provide the optimal memory management strategy according to the needs of the application.

4. memory_resource

memory_resourceWe have already given a basic introduction to in the first part . memory_resourceIt is a memory resource abstraction provided in C++17. It is the base class for all memory resources. In this section, we will delve into memory_resourceseveral key methods of .

4.1 memory_resource::allocate()

This method is the interface used by users memory_resourceto request memory from. It receives two parameters: the number of bytes required and the alignment requirement. It will then call do_allocatethe method to allocate memory.

4.2 memory_resource::deallocate()

This method is used to return previously allocated memory to memory_resource. It receives three parameters: the memory pointer to be returned, the size of the memory block, and the alignment requirements. It will then call do_deallocatethe method to return the memory.

4.3 memory_resource::is_equal()

This method is used to determine memory_resourcewhether two are equal. By default, two memory_resourceare equal if and only if they are the same object.

4.4 memory_resource::do_allocate(), memory_resource::do_deallocate(), memory_resource::do_is_equal()

These three methods are memory_resourcepure virtual functions and must be implemented by subclasses. The specific memory allocation and recycling behavior, as well as the way the two memory resources are compared, depend on the implementation of these three methods.

For example, unsynchronized_pool_resourceand synchronized_pool_resourceimplement their own memory management strategies through these three methods.

4.5 How to use memory_resource

memory_resourceUsually not used directly, but as std::pmrparameters when creating other objects, such as std::pmr::vectoror std::pmr::string.

Here is an example:

std::pmr::synchronized_pool_resource pool;
std::pmr::vector<int> vec(&pool);

In this example, vecwill be used poolto allocate and deallocate memory.

memory_resourceProvides a general memory resource interface, allowing you to customize memory management strategies according to the needs of the application. By using it memory_resource, you can create data structures that are more memory efficient.

5. unsynchronized_pool_resource thread is not safe

unsynchronized_pool_resourceIt is a memory resource class provided by the C++17 standard library for allocating and reclaiming memory from a pre-allocated memory pool. The key point is that unsynchronized_pool_resourcethere is no guarantee of thread safety. This is its synchronized_pool_resourcemain difference from . We discuss this in detail next.

5.1 What is thread unsafe?

"Thread unsafe" means that if the same unsynchronized_pool_resourceinstance is used by multiple threads at the same time, without other synchronization mechanisms (such as mutex locks) to protect this instance, data races and undefined behavior may result. In other words, multiple threads cannot unsynchronized_pool_resourceread and write to an instance at the same time.

5.2 The thread of unsynchronized_pool_resource is not safe

Since unsynchronized_pool_resourceit does not provide an internal synchronization mechanism, special care needs to be taken when using it in a multi-threaded environment unsynchronized_pool_resource. Here are two common ways to use it:

Each thread uses its own unsynchronized_pool_resourceinstance. The advantage of this approach is that since there is no contention, each thread's memory operations can be as fast as possible. However, this approach may lead to inefficient memory usage because each thread needs to preallocate its own memory pool.
Use a shared unsynchronized_pool_resourceinstance and use external synchronization mechanisms (such as mutex locks) to protect this instance. The advantage of this approach is that memory can be used more efficiently because all threads acquire and reclaim memory from the same memory pool. However, this approach may cause performance degradation because threads need to wait for other threads to release the lock.

5.3 How to deal with the thread insecurity of unsynchronized_pool_resource

The best way to handle unsynchronized_pool_resourcethread unsafety depends on the specific needs of your application. If performance is the most important factor and higher memory usage is acceptable, then each thread using its own unsynchronized_pool_resourceinstance may be the best option. If memory efficiency is the most important factor and lower performance is acceptable, then using a shared unsynchronized_pool_resourceinstance and an external synchronization mechanism may be the best option.

Overall, unsynchronized_pool_resourceit provides an efficient memory management method, but you need to pay attention to thread safety issues when using it in a multi-threaded environment. With proper use unsynchronized_pool_resource, you can balance memory efficiency and performance based on your application's needs.

6. synchronized_pool_resource thread safety

synchronized_pool_resourceIt is a memory resource class provided in C++17. Similar to unsynchronized_pool_resource, it is designed to provide efficient memory allocation and recycling services. However, synchronized_pool_resourcethe main difference is that it is thread-safe.

6.1 What is thread safety?

In multi-threaded programming, if a function, function library or class can be called simultaneously in a multi-threaded environment without producing incorrect results or undefined behavior, then we call it thread-safe. Simply put, thread safety is to ensure the correctness of results and consistency of state in a multi-threaded environment.

6.2 Thread safety of synchronized_pool_resource

synchronized_pool_resourceIt is thread-safe and provides an internal synchronization mechanism to ensure that multiple threads can allocate and reclaim memory on it at the same time. That is, multiple threads can share an synchronized_pool_resourceinstance without worrying about data races or undefined behavior.

This feature makes synchronized_pool_resourceit ideal for handling memory management in multi-threaded environments.

6.3 How to use synchronized_pool_resource

Using synchronized_pool_resourceis very simple and unsynchronized_pool_resourceis similar to the usage of . You can synchronized_pool_resourcepass in a pool_optionsparameter to configure the memory pool when creating an instance, and then use allocate()the and deallocate()methods to allocate and reclaim memory.

Here is an example:

std::pmr::pool_options options;
options.max_blocks_per_chunk = 1024;
options.largest_required_pool_block = 512;

std::pmr::synchronized_pool_resource pool(options);

// 从 pool 中分配内存
void* p = pool.allocate(100);

// 将内存归还给 pool
pool.deallocate(p, 100);

In this example, even if multiple threads execute the above code at the same time, the correctness of each and operation synchronized_pool_resourcecan be guaranteed .allocate()deallocate()

synchronized_pool_resourceProvides a thread-safe and efficient memory management method. By using it synchronized_pool_resource, you can perform safe and fast memory allocation and deallocation in a multi-threaded environment, thereby improving the performance of your application.