Redis source code reading comprehension, and related language details---zmalloc.c

malloc is a very commonly used function to apply for memory. Redis has its own library file for memory application allocation. Today we will take a look at the content of the related code zmalloc.c. First of all, the code is still posted first.

#ifndef __ZMALLOC_H
#define __ZMALLOC_H

/* Double expansion needed for stringification of macro values. */
#define __xstr(s) __str(s)
#define __str(s) #s

#if defined(USE_TCMALLOC)
#define ZMALLOC_LIB ("tcmalloc-" __xstr(TC_VERSION_MAJOR) "." __xstr(TC_VERSION_MINOR))
#include <google/tcmalloc.h>
#if (TC_VERSION_MAJOR == 1 && TC_VERSION_MINOR >= 6) || (TC_VERSION_MAJOR > 1)
#define HAVE_MALLOC_SIZE 1
#define zmalloc_size(p) tc_malloc_size(p)
#else
#error "Newer version of tcmalloc required"
#endif

The first is the macro definition of some code, in which the code uses the writing method of #if defined(X). The purpose of this definition is to determine whether X has been compiled. If it has been compiled before, then the intermediate code will be executed. , if X is not defined, the middle part of the code will be ignored, and there is another way of writing, #ifdnf has the same function as the above way of writing, and we can also write it like this, #if! defined(X), and #ifndef both mean that if this has not been compiled before, then the intermediate code is executed.

There are some definitions behind, we can ignore them first and go directly to the definitions related to this article

void *zmalloc(size_t size);
void *zcalloc(size_t size);
void *zrealloc(void *ptr, size_t size);
void zfree(void *ptr);
char *zstrdup(const char *s);
size_t zmalloc_used_memory(void);
void zmalloc_enable_thread_safeness(void);
void zmalloc_set_oom_handler(void (*oom_handler)(size_t));
float zmalloc_get_fragmentation_ratio(void);
size_t zmalloc_get_rss(void);
size_t zmalloc_get_private_dirty(void);
void zlibc_free(void *ptr);

A new data type size_t appears in the definition. This data type can be simply understood as unsigned int, that is, this type has a size large enough to store integers. First, the first function is zmalloc. The source code is as follows

void *zmalloc(size_t size) {
    void *ptr = malloc(size+PREFIX_SIZE);

    if (!ptr) zmalloc_oom_handler(size);
#ifdef HAVE_MALLOC_SIZE
    update_zmalloc_stat_alloc (zmalloc_size (ptr));
    return ptr;
#else
    *((size_t*)ptr) = size;
    update_zmalloc_stat_alloc(size+PREFIX_SIZE);
    return (char*)ptr+PREFIX_SIZE;
#endif
}

Among them, size is the size of the data we need, and the following PREFIX_SIZE has different sizes in different environments. This parameter is a macro-defined variable, and the returned pointer is used to determine whether it is empty. If the returned value is empty pointer, then it will jump to the following zmalloc_oom_handler function. The main purpose of this function is to print the relevant error information. The code is as follows:

static void (*zmalloc_oom_handler)(size_t) = zmalloc_default_oom;
static void zmalloc_default_oom(size_t size) {
    fprintf(stderr, "zmalloc: Out of memory trying to allocate %zu bytes\n",
        size);
    fflush(stderr);
    abort();
}

In this way, we can see very clearly that zmalloc_oom_handler has actually become a function pointer, and the actual code executed is zmalloc_default_oom, the content of which is to print out information and terminate the current service (abort() function), then The function is the function defined by the macro

, both #if and #else involve a function - update_zmalloc_stat_alloc(), then let's take a look at the function performed by this function. First, I still paste the source code according to my habit

#define update_zmalloc_stat_alloc(__n) do { \
    size_t _n = (__n); \
    if (_n&(sizeof(long)-1)) _n += sizeof(long)-(_n&(sizeof(long)-1)); \
    if (zmalloc_thread_safe) { \
        update_zmalloc_stat_add(_n); \
    } else { \
        used_memory += _n; \
    } \
} while(0)

It is also a code compiled by a macro definition. The code in this block is very elegant. When I read it, I basically read the analysis of many people to understand it, so some of the next words may be translated by me, because I really don't dare to play the axe~, first of all, there is such a loop do{...}while(0) in this code. This part is a very cool way of writing. In a word, it can be summarized in one sentence - 它能确保宏的行为总是相同的，if you want to see a detailed explanation, You can move to this URL  and click the open link , and then this code is to determine whether the size of the allocated memory space is a multiple of 8. If the memory size is not a multiple of 8, add the corresponding offset to make it a multiple of 8. _n&7 is functionally equivalent to _n%8, but bitwise operations are significantly more efficient.

------------------------------------------Excerpt from dividing line----- -----------------------------------------

        malloc() itself can guarantee that the allocated memory is 8-byte aligned: if the memory you want to allocate is not a multiple of 8, then malloc will allocate a little more to make up a multiple of 8. So the real function of the update_zmalloc_stat_alloc function (or zmalloc() relative to malloc()) is not to perform 8-byte alignment (malloc has been guaranteed), its real purpose is to make the variable used_memory accurately maintain the actual allocated memory size .       

        The condition of the second if is an integer variable zmalloc_thread_safe. As the name suggests, its value indicates whether the operation is thread-safe or not. If it is not thread-safe (else), add n to the variable used_memory. used_memory is a global static variable defined in the zmalloc.c file, representing the size of the allocated memory. If it is memory safe use update_zmalloc_stat_add to add n to used_memory.

        update_zmalloc_stat_add is also a macro function (the high efficiency and fast speed of Redis, these macro functions can be described as indispensable). It is also a conditionally compiled macro, which has different definitions according to different macros. The  code is as follows

#define update_zmalloc_stat_add(__n) do { \
    pthread_mutex_lock(&used_memory_mutex); \
    used_memory += (__n); \
    pthread_mutex_unlock(&used_memory_mutex); \
} while(0)

pthread_mutex_lock() and pthread_mutex_unlock() use mutex to achieve thread synchronization. The former means locking and the latter means unlocking. They are thread synchronization functions defined by POSIX. After locking, the code behind it will only be executed once when multiple threads execute this code at the same time, that is, thread safety is achieved.

-----------------------------------------------------------------------------------------------------

      ------ This passage was reposted by me. Originally, I wanted to write this piece with my own understanding, but this blogger explained it very well, so I moved this passage directly, and also Good to have a clearer understanding.

The next functions include zcalloc() and zrealloc(). First of all, we can know that the prototypes of the two functions are calloc() and realloc(). Next, let’s take a look at the difference between each function in this chain and malloc(). The first is malloc() of calloc(), but the former needs to initialize the data in it after allocating memory, while malloc() is not needed, which means that the value allocated by malloc() is likely to be uncertain The value of , and calloc() is a definite value. The function of realloc() is to reallocate the allocated memory. It may be a bit of a mouthful to say, but the fundamental function is to change the original memory size. If you need to add data Then realloc() will first check whether the tail meets the requirements. If not, new memory will be opened up. The same three functions in redis are the same except for the different calls. Logic, I will not repeat the code to explain,

The next function is the function free() corresponding to malloc(). The zfree() of redis is used to release the requested dynamic memory. First, paste the code as follows:

void zfree(void *ptr) {
#ifndef HAVE_MALLOC_SIZE
    void *realptr;
    size_t oldsize;
#endif

    if (ptr == NULL) return;
#ifdef HAVE_MALLOC_SIZE
    update_zmalloc_stat_free(zmalloc_size(ptr));
    free(ptr);
#else
    realptr = (char*)ptr-PREFIX_SIZE;
    oldsize = *((size_t*)realptr);
    update_zmalloc_stat_free(oldsize+PREFIX_SIZE);
    free(realptr);
#endif
}

First of all, for the macro definition part, we can only focus on the #else part. Combined with the previous zmalloc() analysis, we can know that realptr means the previous real memory address, and then for the size of the real allocated memory oldsize = *((size_t*)realptr ); and then use the function to release the device, we can see that the author of redis is very clear in handling this part of the code. It really feels that the performance of redis is so good for nothing.

The other parts of the code are relatively simple, and they are posted below for everyone to understand:

size_t zmalloc_get_rss(void) {
    /* If we can't get the RSS in an OS-specific way for this system just
     * return the memory usage we estimated in zmalloc()..
     *
     * Fragmentation will appear to be always 1 (no fragmentation)
     * of course... */
    return zmalloc_used_memory();
}

size_t zmalloc_used_memory(void) {
    size_t um;

    if (zmalloc_thread_safe) {
#ifdef HAVE_ATOMIC
        um = __sync_add_and_fetch(&used_memory, 0);
#else
        pthread_mutex_lock(&used_memory_mutex);
        um = used_memory;
        pthread_mutex_unlock(&used_memory_mutex);
#endif
    }
    else {
        um = used_memory;
    }

    return one;
}

void zmalloc_enable_thread_safeness(void) {
    zmalloc_thread_safe = 1;
}