There are many options for using logging libraries in c++, such as spdlog, g3log, log4cxx, log4cplus, log4qt, etc. They are all easy to use and powerful, but some are a bit heavyweight in terms of the amount of source code. Here we introduce easylog, a lightweight log library with excellent performance, from Ali's Yalanting library. Only a few simple files, just include the header file when using it.

Introduction to easylog

easylog, a light-weight and high-performance C++ log library open sourced by Ali, requires a minimum compiler support for C++17. It belongs to a function integrated in Alibaba's Yalan Pavilion library, with a small amount of code, easy to use and powerful performance. It uses some new features above c++17, such as constexpr compile-time optimization, string view class std::string_view, and three-party libraries ConcurrentQueue (thread-safe lock-free queue), jkj::dragonbox (efficient floating-point number to string conversion library), efvalue::meta_string string metaprogramming, and thus has high performance.

GitHub - purecpp-org/easylog: a c++20 log lib

easylog implementation

The implementation idea of easylog is relatively simple and clear. It is to write the log to the queue, and not to directly operate the file. Start a thread to take data from the queue and write it to the file. Although the amount of code is small and the implementation idea is simple, many new features of C++ performance optimization are used in it, which is worth learning and learning from.

ConcurrentQueue

ConcurrentQueue is a thread-safe queue data structure that allows multiple threads to operate on the queue at the same time without additional synchronization mechanisms. It is one of the commonly used data structures in concurrent programming.

moodycamel::ConcurrentQueue A multi-producer, multi-consumer lock-free queue implemented in C++11.

Warehouse Address:

https://github.com/cameron314/concurrentqueue

The advantages are as follows:

1. Thread safety: ConcurrentQueue provides a built-in thread safety mechanism, which can perform enqueue and dequeue operations on multiple threads at the same time, without manually adding additional synchronization mechanisms (such as mutexes or semaphores) to protect shared resources. This simplifies the complexities of concurrent programming.

2. Efficient performance: ConcurrentQueue provides better performance in a concurrent environment. It uses some efficient algorithms and data structures, such as lock-free queues or fine-grained locking, to reduce competition and improve concurrency performance.

3. Low latency: Since ConcurrentQueue is designed to support high concurrency and low latency scenarios, it usually has low operational latency. This is useful for applications that need to respond quickly and handle a large number of concurrent requests.

4. Scalability: ConcurrentQueue can easily expand to more threads when needed without performance bottlenecks. It is suitable for high-concurrency and high-throughput applications and can be scaled horizontally based on demand.

ConcurrentQueue is a convenient, efficient, thread-safe queue data structure, suitable for concurrent programming scenarios, and can provide better performance and scalability.

dragonbox-library

jkj::dragonbox is a C++ library for efficient float-to-string conversions. It provides a fast and precise way to represent floating point numbers as decimal strings, suitable for various application scenarios such as number formatting, logging, etc.

Warehouse Address:

https://github.com/jk-jeon/dragonbox

The main features and uses of the jkj::dragonbox library:

1. Efficient performance: jkj::dragonbox uses some efficient algorithms and techniques to achieve fast conversion from floating point numbers to strings. It is faster than the conversion functions in the standard library in most cases, and has predictable performance.

2. Accuracy: The jkj::dragonbox library provides accurate conversion results, can retain all significant digits of floating-point numbers, and can correctly handle rounding errors when rounding.

3. Portability: The jkj::dragonbox library is a cross-platform C++ library that can be used on various operating systems and compilers. 4. Ease of use: The interface of the jkj::dragonbox library is simple and easy to use. You only need to include the corresponding header file and call the corresponding conversion function to complete the conversion from floating point numbers to strings.

The jkj::dragonbox library provides an efficient, accurate and portable floating-point-to-string conversion solution, suitable for application scenarios that require high performance and precision.

constexpr feature

constexpr is a keyword introduced by C++11, which is used to declare a compile-time constant (compile-time constant). It can be evaluated at compile time and optimized at compile time, providing the ability to perform calculations and initialization at compile time.

1. Constant expression: constexpr can be used to declare a constant expression, that is, an expression whose value can be determined at compile time. These expressions can be evaluated at compile time and do not need to be evaluated at run time.

2. Compile-time optimization: Constant expressions declared using constexpr can be optimized at compile time to improve program performance. The compiler can compute the result of a constexpr expression at compile time and replace it directly with the resulting value without having to do the computation at run time.

3. Type checking: constexpr can also be used to declare member variables of functions, constructors, member functions and classes. These declarations can be type-checked at compile time to ensure they meet the requirements of constexpr.

4. Array size: constexpr can be used to declare the size of the array, that is, the size of the array is determined at compile time. In this way, static checking can be performed at compile time to avoid problems such as array out-of-bounds. Using constexpr can improve the performance and readability of your code, while allowing more calculations and optimizations at compile time. The feature of its evaluation at compile time enables the calculation of some constants to be completed at compile time instead of at runtime, thus improving the efficiency of the program.

meta_string feature

GET_STRING macro definition

Used to generate a string prefix containing filename and line number information at compile time. Its function is to add a prefix including the file name and line number to each log message in scenarios such as log output, so as to locate the source of the log message.

#define TO_STR(s) #s

#define GET_STRING(filename, line)                              \
  [] {                                                          \
    constexpr auto path = refvalue::meta_string{filename};      \
    constexpr size_t pos =                                      \
        path.rfind(std::filesystem::path::preferred_separator); \
    constexpr auto name = path.substr<pos + 1>();               \
    constexpr auto prefix = name + ":" + TO_STR(line);          \
    return "[" + prefix + "] ";                                 \
  }()

In fact, it can be regarded as a metaprogramming tool that generates string literals at compile time. It provides an ability to create strings at compile time, and perform string-related operations and calculations at compile time. It defines a template structure for processing strings during compilation and provides some string manipulation functions.

#pragma once

#include <algorithm>
#include <array>
#include <cstddef>
#if __has_include(<span>)
#include <compare>
#include <concepts>
#include <span>
#endif
#include <string_view>
#include <utility>

namespace refvalue {
template <std::size_t N>
struct meta_string {
  std::array<char, N + 1> elements_;

  constexpr meta_string() noexcept : elements_{} {}

  constexpr meta_string(const char (&data)[N + 1]) noexcept {
    for (size_t i = 0; i < N + 1; i++) elements_[i] = data[i];
  }

#if __has_include(<span>)
  template <std::size_t... Ns>
  constexpr meta_string(std::span<const char, Ns>... data) noexcept
      : elements_{} {
    auto iter = elements_.begin();

    ((iter = std::copy(data.begin(), data.end(), iter)), ...);
  }
#endif

  template <std::size_t... Ns>
  constexpr meta_string(const meta_string<Ns>&... data) noexcept : elements_{} {
    auto iter = elements_.begin();

    ((iter = std::copy(data.begin(), data.end(), iter)), ...);
  }

#if __has_include(<span>)
  template <std::same_as<char>... Ts>
  constexpr meta_string(Ts... chars) noexcept requires(sizeof...(Ts) == N)
      : elements_{chars...} {}
#endif

  constexpr char& operator[](std::size_t index) noexcept {
    return elements_[index];
  }

  constexpr const char& operator[](std::size_t index) const noexcept {
    return elements_[index];
  }

  constexpr operator std::string_view() const noexcept {
    return std::string_view{elements_.data(), size()};
  }

  constexpr bool empty() const noexcept { return size() == 0; }

  constexpr std::size_t size() const noexcept { return N; }

  constexpr char& front() noexcept { return elements_.front(); }

  constexpr const char& front() const noexcept { return elements_.front(); }

  constexpr char& back() noexcept { return elements_[size() - 1]; }

  constexpr const char& back() const noexcept { return elements_[size() - 1]; }

  constexpr auto begin() noexcept { return elements_.begin(); }

  constexpr auto begin() const noexcept { return elements_.begin(); }

  constexpr auto end() noexcept { return elements_.begin() + size(); }

  constexpr auto end() const noexcept { return elements_.begin() + size(); }

  constexpr char* data() noexcept { return elements_.data(); }

  constexpr const char* data() const noexcept { return elements_.data(); };

  constexpr const char* c_str() const noexcept { return elements_.data(); }

  constexpr bool contains(char c) const noexcept {
    return std::find(begin(), end(), c) != end();
  }

  constexpr bool contains(std::string_view str) const noexcept {
    return str.size() <= size()
               ? std::search(begin(), end(), str.begin(), str.end()) != end()
               : false;
  }

  static constexpr size_t substr_len(size_t pos, size_t count) {
    if (pos >= N) {
      return 0;
    }
    else if (count == std::string_view::npos || pos + count > N) {
      return N - pos;
    }
    else {
      return count;
    }
  }

  template <size_t pos, size_t count = std::string_view::npos>
  constexpr meta_string<substr_len(pos, count)> substr() const noexcept {
    constexpr size_t n = substr_len(pos, count);

    meta_string<n> result;
    for (int i = 0; i < n; ++i) {
      result[i] = elements_[pos + i];
    }
    return result;
  }

  constexpr size_t rfind(char c) const noexcept {
    return std::string_view(*this).rfind(c);
  }

  constexpr size_t find(char c) const noexcept {
    return std::string_view(*this).find(c);
  }
};

template <std::size_t N>
meta_string(const char (&)[N]) -> meta_string<N - 1>;

#if __has_include(<span>)
template <std::size_t... Ns>
meta_string(std::span<const char, Ns>...) -> meta_string<(Ns + ...)>;
#endif

template <std::size_t... Ns>
meta_string(const meta_string<Ns>&...) -> meta_string<(Ns + ...)>;

#if __has_include(<span>)
template <std::same_as<char>... Ts>
meta_string(Ts...) -> meta_string<sizeof...(Ts)>;
#endif

#if __has_include(<span>)
template <std::size_t M, std::size_t N>
constexpr auto operator<=>(const meta_string<M>& left,
                           const meta_string<N>& right) noexcept {
  return static_cast<std::string_view>(left).compare(
             static_cast<std::string_view>(right)) <=> 0;
}
#endif

template <std::size_t M, std::size_t N>
constexpr bool operator==(const meta_string<M>& left,
                          const meta_string<N>& right) noexcept {
  return static_cast<std::string_view>(left) ==
         static_cast<std::string_view>(right);
}

template <std::size_t M, std::size_t N>
constexpr bool operator==(const meta_string<M>& left,
                          const char (&right)[N]) noexcept {
  return static_cast<std::string_view>(left) ==
         static_cast<std::string_view>(meta_string{right});
}

template <std::size_t M, std::size_t N>
constexpr auto operator+(const meta_string<M>& left,
                         const meta_string<N>& right) noexcept {
  return meta_string{left, right};
}

template <std::size_t M, std::size_t N>
constexpr auto operator+(const meta_string<M>& left,
                         const char (&right)[N]) noexcept {
  meta_string<M + N - 1> s;
  for (size_t i = 0; i < M; ++i) s[i] = left[i];
  for (size_t i = 0; i < N; ++i) s[M + i] = right[i];
  return s;
}

template <std::size_t M, std::size_t N>
constexpr auto operator+(const char (&left)[M],
                         const meta_string<N>& right) noexcept {
  meta_string<M + N - 1> s;
  for (size_t i = 0; i < M - 1; ++i) s[i] = left[i];
  for (size_t i = 0; i < N; ++i) s[M + i - 1] = right[i];
  return s;
}

#if __has_include(<span>)
template <meta_string S, meta_string Delim>
struct split_of {
  static constexpr auto value = [] {
    constexpr std::string_view view{S};
    constexpr auto group_count = std::count_if(S.begin(), S.end(),
                                               [](char c) {
                                                 return Delim.contains(c);
                                               }) +
                                 1;
    std::array<std::string_view, group_count> result{};

    auto iter = result.begin();

    for (std::size_t start_index = 0, end_index = view.find_first_of(Delim);;
         start_index = end_index + 1,
                     end_index = view.find_first_of(Delim, start_index)) {
      *(iter++) = view.substr(start_index, end_index - start_index);

      if (end_index == std::string_view::npos) {
        break;
      }
    }

    return result;
  }();
};

template <meta_string S, meta_string Delim>
inline constexpr auto&& split_of_v = split_of<S, Delim>::value;

template <meta_string S, meta_string Delim>
struct split {
  static constexpr std::string_view view{S};
  static constexpr auto value = [] {
    constexpr auto group_count = [] {
      std::size_t count{};
      std::size_t index{};

      while ((index = view.find(Delim, index)) != std::string_view::npos) {
        count++;
        index += Delim.size();
      }

      return count + 1;
    }();
    std::array<std::string_view, group_count> result{};

    auto iter = result.begin();

    for (std::size_t start_index = 0, end_index = view.find(Delim);;
         start_index = end_index + Delim.size(),
                     end_index = view.find(Delim, start_index)) {
      *(iter++) = view.substr(start_index, end_index - start_index);

      if (end_index == std::string_view::npos) {
        break;
      }
    }

    return result;
  }();
};

template <meta_string S, meta_string Delim>
inline constexpr auto&& split_v = split<S, Delim>::value;

template <meta_string S, char C>
struct remove_char {
  static constexpr auto value = [] {
    struct removal_metadata {
      decltype(S) result;
      std::size_t actual_size;
    };

    constexpr auto metadata = [] {
      auto result = S;
      auto removal_end = std::remove(result.begin(), result.end(), C);

      return removal_metadata{
          .result{std::move(result)},
          .actual_size{static_cast<std::size_t>(removal_end - result.begin())}};
    }();

    meta_string<metadata.actual_size> result;

    std::copy(metadata.result.begin(),
              metadata.result.begin() + metadata.actual_size, result.begin());

    return result;
  }();
};

template <meta_string S, char C>
inline constexpr auto&& remove_char_v = remove_char<S, C>::value;

template <meta_string S, meta_string Removal>
struct remove {
  static constexpr auto groups = split_v<S, Removal>;
  static constexpr auto value = [] {
    return []<std::size_t... Is>(std::index_sequence<Is...>) {
      return meta_string{std::span<const char, groups[Is].size()>{
          groups[Is].data(), groups[Is].size()}...};
    }
    (std::make_index_sequence<groups.size()>{});
  }();
};

template <meta_string S, meta_string Removal>
inline constexpr auto&& remove_v = remove<S, Removal>::value;
#endif
}  // namespace refvalue

The main functions of refvalue::meta_string are as follows:

1. String operations at compile time: refvalue::meta_string allows operations on strings at compile time, such as concatenation, interception, comparison, etc. This allows string-related calculations and processing to be done at compile time instead of at runtime.

2. Code generation: refvalue::meta_string can be used to generate code, especially string-related code. By generating string literals at compile time, you can include strings as part of your code and manipulate and process them during code generation.

3. Metadata processing: refvalue::meta_string can be used to process metadata of string type, such as class name, function name, etc. By generating string literals at compile time, these strings can be used in metaprogramming as names for types and identifiers.

meta_stringis designed to perform string manipulation during compile time to improve performance. Since the operation occurs at the compile time, it can avoid the string operation overhead at runtime, thereby improving the execution efficiency of the program. For example, using meta_string, you can concatenate strings, extract substrings, remove characters, etc., at compile time instead of at runtime. This makes the code more flexible and efficient.

In short, refvalue::meta_string is a metaprogramming tool for generating string literals at compile time and performing string-related operations and processing. It can be used to generate code, process metadata, and perform string calculations and operations at compile time, providing a capability for string processing at compile time.

It should be noted that some functions in this code may require the support of C++20 or later, such as features such as std::span and std::same_as. Therefore, when using these features, you need to ensure that your compiler supports them.

Example of using meta_string

#include <iostream>
#include "meta_string.hpp"

int main() {
    using namespace refvalue;

    constexpr auto str = meta_string("hello world");

    static_assert(str.size() == 6, "字符串长度不正确");

    std::cout << "字符串: " << str << std::endl;
    std::cout << "字符串长度: " << str.size() << std::endl;

    constexpr auto subStr = str.substr<3, 2>();

    std::cout << "子串: " << subStr << std::endl;
    std::cout << "子串长度: " << subStr.size() << std::endl;

    constexpr auto concatStr = str + meta_string(" 欢迎！");

    std::cout << "拼接后的字符串: " << concatStr << std::endl;

    constexpr auto removedStr = remove_v<concatStr, meta_string("迎")>;

    std::cout << "移除后的字符串: " << removedStr << std::endl;

    return 0;
}

In the above example, an meta_stringobject is created strand initialized, then we show some meta_stringoperations on the . Use std::cout to print the original string and its length, use substrthe function to extract a substring, specify the starting position and length, and print the substring and its length. We concatenate with another using +the operator and store the result in . Finally the substring "welcome" is removed from with the function and the result is stored in . keywords are used to ensure that these operations are evaluated at compile time. strmeta_stringconcatStrremove_vconcatStrremovedStrconstexpr

meta_stringAllows string manipulation at compile time, not runtime. This means that the result of string processing is determined before the program is executed, which can improve the performance and efficiency of the program. Since string processing occurs at compile time rather than runtime, the overhead of string manipulation during program execution is avoided, which reduces runtime computation and memory consumption. This is also the reason why it is used in easylog meta_string . For some scenes with fixed log format prefix characters such as date and code line number, using it can significantly improve performance.

type traits mechanism

The concept of type traits, literal translation is type extraction. According to the name, you can also guess that it is used to obtain the type. Before c++ 11, stl has already used related technologies. For example, iterators use related types to obtain. "STL Source Code Analysis" has a detailed introduction. If you are interested, you can go to have a look. C++ 11 even introduced a special header file <type_traits> to do things related to type traits.

Simply understood, type traits in C++ is a mechanism for querying and manipulating type information at compile time. They allow us to obtain properties and traits about types at compile time and program against them. Type traits can help us determine whether a type has certain characteristics, such as whether it is a pointer, reference, array, etc., whether it is a const or volatile qualifier, or whether it can perform operations such as copy construction or move construction. Through these traits, we can perform different processing and logic branches according to the characteristics of the type at compile time.

To put it simply, type traits is a tool for querying and manipulating type properties at compile time. They can help us program according to the characteristics of types and improve the reliability and flexibility of code.

template <typename Return>
struct function_traits<Return (*)() noexcept> {
  using parameters_type = void;
  using return_type = Return;
};

In the code snippet given above, a function_traitsspecialization of the template is defined for a function pointer with no parameters and a noexcept specifier. Return (*)() noexceptThis specialization is triggered when a template parameter matches the pattern. In the specialization, two type aliases are defined: parameters_typeand return_type. In this case, parameters_typeis set to void, indicating that the function has no parameters, and return_typeis set to a template parameter Return, indicating the return type of the function. This code is useful when you want to extract or manipulate information about a function's type, such as extracting the return type or checking whether a function has a specific signature. Type traits like function_traitsthis can be used in template metaprogramming or generic programming scenarios where compile-time type information is required.

Give a simple usage example to deepen understanding:

#include <iostream>
#include <type_traits>

// 类型特性，用于检查一个类型是否为指针
template <typename T>
struct is_pointer {
    static constexpr bool value = false;
};

template <typename T>
struct is_pointer<T*> {
    static constexpr bool value = true;
};

// 使用类型特性
int main() {
    std::cout << std::boolalpha;
    std::cout << "int* 是指针吗？" << is_pointer<int*>::value << std::endl;
    std::cout << "float* 是指针吗？" << is_pointer<float*>::value << std::endl;
    std::cout << "double 是指针吗？" << is_pointer<double>::value << std::endl;

    return 0;
}

In this example, a is_pointertype trait named is defined that checks whether a given type is a pointer. This attribute is defined as a struct template containing a static member variable valueset to false. is_pointer<T*>A partial specialization is then provided for where T*denotes a pointer type. In this partial specialization, valueset to true. mainUse the type attribute in the function is_pointerto check whether different types are pointers. Use std::cout to print the result. For int*and float*, the output will be truebecause they are pointer types, and for double, falsebecause it is not a pointer type. This example shows how to use type traits to perform type checking at compile time and provide information about types at compile time.

std::string_view

std::string_view is widely used in easylog.

std::string_view is a lightweight string view class introduced in C++17.

advantage

1. Zero-overhead abstraction: std::string_view does not own any data, it is just a view of existing string data. This means it can be constructed and passed around very efficiently, without the overhead of data copying.

2. Interoperate with std::string: std::string_view can be conveniently constructed from std::string and vice versa. This makes it ideal as a function argument that can accept std::string or std::string_view.

3. Security: Compared with C-style strings, std::string_view is type-safe and can avoid potential buffer overflow errors.

4. Flexibility: std::string_view can point to any continuous sequence of characters, not limited to std::string. This makes it very versatile and flexible.

Applicable scene

1. When you need a string abstraction, but don't need to own the underlying string data. For example, in a function parameter, you may only need to read string data, but not own or modify it.

2. When a string view is required, it can point to different string types (such as std::string, char*, const char*, etc.). 3. When you need to efficiently construct and pass string data to avoid unnecessary copy overhead.

4. When a type-safe string abstraction is required, it can replace C-style strings.

5. When a string view that can interoperate with std::string is required, to support functions that accept std::string or std::string_view.

In summary, std::string_view is a lightweight string abstraction that efficiently manipulates string data and provides type safety and interoperability with std::string. It's great as a function argument, or when you need a view of a string without owning the underlying data.

doctest unit testing framework

The doctest unit testing framework is used in easylog. doctest is a unit testing framework for C++. Compared with Google Test (gtest), its main advantages are:

1. Ease of use: the API of doctest is very simple, and only a few macro definitions and comments are needed to write test cases. In contrast, gtest's API is slightly more complex, requiring more functions and classes to define tests.

2. No need to link libraries: doctest is a header file library, no need to link any library files. gtest needs to link libgtest.a and libgtest_main.a two library files.

3. Support subtests: doctest supports organizing test cases into hierarchical subtests, which makes the tests more structured and readable. gtest does not currently support subtests.

4. Support a variety of assertions: doctest provides a variety of assertion macros to verify test conditions, including equality assertions, truth assertions, exception assertions, etc. gtest also provides a variety of assertions, but slightly fewer.

5. Support labeling and ignoring of test cases: doctest can add labels to test cases, and then choose to run only test cases with a certain label. It also supports ignoring specified test cases. gtest does not support these two features.

6. Support multiple test reports: doctest can generate test reports in multiple formats, including XML, JSON, JUnit, etc. gtest only supports reports in Google Test format.

7. Support random order of test cases: doctest can randomly change the execution order of test cases, which is useful when detecting bugs that depend on the order of test cases. gtest does not support randomized test order.

In general, doctest is a simple, easy-to-use and more powerful C++ unit testing framework. Compared with gtest, it has more features and advantages. It is easy to use and suitable for unit testing of some C++ projects.

easylog use

easylog is relatively simple to use, you can directly refer to the examples.

  std::string filename = "easylog.txt";
  std::filesystem::remove(filename);
  easylog::init_log(Severity::DEBUG, filename, true, 5000, 1, true);

  ELOG_INFO << 42 << " " << 4.5 << 'a' << Severity::DEBUG;

  ELOGV(INFO, "test");
  ELOGV(INFO, "it is a long string test %d %s", 2, "ok");

  int len = 42;
  ELOGV(INFO, "rpc header data_len: %d, buf sz: %lu", len, 20);

  ELOG(INFO) << "test log";
  easylog::flush();
  ELOG_INFO << "hello "
            << "easylog";

  ELOGI << "same";

  ELOG_DEBUG << "debug log";
  ELOGD << "debug log";

c++17 support

Note, the original warehouse does not support c++17 by default, and the minimum requirement is c++20. But several codes can be modified to support c++17.

The following defines are enabled in type_traits.h.

namespace std {
template <class T>
struct remove_cvref {
  typedef std::remove_cv_t<std::remove_reference_t<T>> type;
};
template <class T>
using remove_cvref_t = typename remove_cvref<T>::type;
}  // namespace std

Remove span-related header files and definitions from meta-string.h. Because <span> is a new feature of c++20.