After a year, acl cross-platform network communication and server framework library released a major version upgrade. The main improvements are as follows:
1. Refactor the network coroutine library, support more operating system platforms (Linux/FreeBSD/MacOS/Windows), and support more network event engines: epoll/select/poll/kqueue/iocp/Win32 GUI;
2. The acl service container is independent from the basic library to form an independent module, and at the same time, the service management tool chain is added;
In addition, a large number of new features and bug fixes are included.
1. About acl
acl includes the following rich common function libraries:
1. Server framework: including process pool mode, thread pool mode, non-blocking mode, UDP communication mode, trigger mode and coroutine mode
2. Network communication library: blocking/non-blocking network IO Library (lib_acl_cpp library has SSL communication capability by embedding polarssl)
3. HTTP network library: HTTP client/server library, HTTP server supports HttpServlet interface access mode; supports HTTP cookies/session; HTTP client supports connection pool and connection pool cluster; support Websocket
4, redis client library: realizes a total of more than 150 commands in 12 categories of redis client commands, supports connection pool and cluster connection pool mode
5, common network application library: SMTP client Side library/PING library/DNS parsing library/memcache client library/handler socket client library/beanstalk client library/disque client library
6. Mail parsing library: mime parsing library/RFC2047/RFC2048/mime base64/mime qp/ mime uucode, etc.
7. Event engine: support select, poll (for unix), epoll (for linux), kqueue (for bsd), devpoll (for solaris), iocp (for win32), win32 window message (for win32) event engine
8. Universal connection pool library: an efficient connection pool basic class library that supports rich
functions Support connection pool and connection pool cluster management functions
10. XML/json streaming parsing library: Different from other existing xml/json parsing libraries on the Internet, the xml/json parsing library in acl uses a finite state machine method to parse data, and the processing method is more flexible
. 11. Network coroutine library : This library transforms the blocking IO mode into a non-blocking mode through context switching and event triggering at the bottom layer, which facilitates programmers to quickly write high-concurrency, high-performance network services
12. It supports serialization and deserialization of C++ struct objects ( Serialize to Json object and deserialize from Json object)
Examples:
1. lib_acl/samples: There are a lot of samples based on lib_acl C library in
this directory 2. lib_protocol/samples: There are a lot of samples based on lib_protocol C library in this directory Example
3. lib_acl_cpp/samples: There are a large number of examples based on the lib_acl_cpp C++ library in
this directory. 4. lib_fiber/samples: There are a large number of coroutine-based network client/server sample
applications in this directory:
1. app/master: This directory is acl service container (acl_master) and common tools
2. app/redis_tools: Redis cluster management, monitoring and intelligent creation tools in this directory
3. app/wizard: wizard program for automatically generating acl server framework
4. app /gid: Service program and client library used to generate globally unique integer IDs
5. app/wizard_demo: Test programs for some servers6
, app/gson: json serialization and deserialization code generation
tool Version upgrade involves content
1) Basic C library: lib_acl
.feature: feature: Provide socket IO read and write registration interface, allow users to register IO process, so that ACL_VSTREAM stream uses user registered API process for network IO read and write process.
Feature: aio module is added acl_aio_last_nready, the event module adds acl_event_last_nready to indicate the number of events triggered by an event loop.
feature: acl_mbox.c supports using eventfd handle for IO notification
. feature: acl_udp_server.c supports binding UNIX domain sockets
. feature: udp in net The module supports binding UNIX domain sockets to transmit data
. Feature: acl_argv.c Add acl_argv_set to replace the value of the specified subscript position.
Feature: acl_udp_server.c Support IP address automatic discovery function, that is, when the network card address changes, it will automatically Add or delete listening addresses.
feature: acl_dlink.c binary block algorithm supports adding negative integers
. feature: master server template supports subsequent processing after receiving SIGTERM signal.
feature: acl_mbox_read function in acl_mbox.c The read timeout unit is seconds The level is adjusted to millisecond level
. Feature: Open acl_read_xxx_wait interface to allow millisecond level read timeout waiting
.feature: acl_inet_listen/acl_inet_bind/acl_vstream_listen_ex add flag
flag.feature: add configuration items master_reuseport and master_nonblock, master_fastopen to master server framework and server template to allow users to set listening mode through configuration
file.feature: acl_udp_server.c/acl_threads_server.c /acl_aio_server.c, acl_single_server.c/acl_trigger_server.c, add a method to get the configuration file path of the current
service.feature: master/template/acl_udp_server.c Add a configuration item to allow the created thread to be in non-"detached"
mode.feature : All server templates under master/template support the callback process after getting the SIGHUP signal.
Feature: The function event_new_kernel in event_kernel.c adds acl_close_on_exec control to prevent child processes from inheriting the epoll/devpoll/kqueue handle
. The address binding operation forms a separate method.
feature: add acl_host_addrinfo method to support general domain name resolution
. feature: acl_udp_server.c supports both daemon and alone operating modes.
feature: acl_udp_server.c supports binding fuzzy matching addresses
.feature: acl_master server framework allows no application service program at startup, so that users can add
dynamically.feature: ACL_TOKEN object adds support for acl_foreach, which can traverse all keywords.feature
: acl_token_tree.c adds acl_token_tree_word_remove
method.feature: acl_master support SO_REUSEPORT .feature in UDP mode
: The acl_master server framework supports switching the running identity of the child process before execvp the application after fork the child process. This feature will help the application service written in the go language to switch the running identity.
.feature: add inet_ntop/inet_pton for vc2003
.bugfix: acl_xml_parse.c macro IS_CDATA is wrong, --- by https://gitee.com/ghostshy
.bugfix: acl_read_wait.c does not need to set EPOLLHUP | EPOLLERR flag before calling epoll , this flag belongs to the return value.
bugfix: acl_inet_connect.c When the connection fails because acl_socket_close is called, the system error number will be reset on the WIN platform.
bugfix: acl_read_wait.c/acl_write_wait.c When calling the poll API, POLLHUP | POLLERR should not as input parameter
.bugfix: When opening a file with acl_vstream_fdopen, it should no longer judge whether it is a SOCKET type, otherwise the file handle type will be regarded as a SOCKET type, which will cause problems.
Bugfix: Refactor the function acl_ifconf_search in acl_ifconf.c, this function has some Compatibility issues.
Bugfix: Fix the self-awareness of network card IP address changes in
acl_udp_server.c. Bugfix: acl_ifconf_search configures addresses incorrectly
. Bugfix: acl_mylog.c When fork child processes, it may cause deadlock problems
. Bugfix: acl_events. The event_init function eventp->timer_keep in c is a function pointer and should not be set to 0, otherwise the acl_event_keep_timer function will crash
. Bugfix: acl_udp_server.c When the SIGHUP signal is received and fed back to the acl_master through the pipeline, the passed process ID is wrong
.bugfix: acl_dlink.c binary block algorithm When adding two consecutive data blocks, it should be merged.
bugfix: The thread lock inside acl_mylog.c adopts a recursive lock, otherwise, when a signal is received when logging, the log will be blocked. Cause deadlock.bugfix
: acl_aqueue_pop_timedwait function in acl_aqueue.c calculates the timeout time incorrectly.bugfix
: acl_host_port.c->host_port is incorrectly parsed under the windows platform
.bugfix: acl_aio_server.c/acl_threads_server.c is wrong under windows.
Bugfix: the parameter order of thread_callback in acl_threads_server.c is wrong when calling the callback function serv_timeout.
.bugfix: The implementation of acl_atomic_int64_fetch_add/acl_atomic_int64_add_fetch has a
problem.bugfix: acl_master When master_prefork > 0, if you stop a service module or reload acl_master after changing the listening address of the service module, acl_master will crash, the problem
has been
fixed.bugfix: acl_pthread There is a bug in acl_pthread_setspecific in .c
.bugfix: there is an illegal release of memory in acl_mylog.c/private_stream.c
2) Basic network protocol library: lib_protocol
.feature: "?" question mark in HTTP request uri is escaped to "%3F" "In the case of ", compatibility processing can be done.
Feature: The function http_hdr_res_parse in http_hdr_res.c cancels the check of http_status, so that the application can use the custom status code
3) C++ library:
lib_acl_cpp.feature: for HTTP request uri" ?" When the question mark is escaped as "%3F", compatibility processing can be done
.feature: Add the diff_xxx class object set to quickly compare the difference between two sets.
Feature: The atomic_long class adds the assignment copy overload function, while prohibiting the construction copy function.
Feature: Add redis_role, redis_sentinel class.
Feature: server_socket class support SO_REUSEPORT
.feature: aio_handle class adds the method last_nready to get the number of events triggered by this event
loop.feature: redis_cluster feature class supports redis.4.xx
.feature: adds tbox class for passing message objects in a multi-threaded environment
. feature: add a new class event_mutex, which supports thread-level or coroutine-level mutual exclusion in the form of atomic operations + IO events. feature
: thread_mutex takes whether to create recursive locks as a construction parameter option
. feature: add trigger.hpp file, Used for timing trigger operations
by time.feature: Added bitmap class in stdlib/ directory for bitmap operations.feature
:Added atomic operation class
atomic.feature:Added tcp connection pool communication
module.feature: polarssl_conf & polarssl_io Two classes support dynamic loading of polarssl dynamic library.
feature: redis_hash class adds several implementations for hmset method for the convenience of users
.feature: master_proc/master_aio/master_threads/master_udp/master_trigger add get_conf_path method to get the current server's configuration file
path.feature: all server templates support SIGHUP signal callback processing.feature
: db_sqlite class adds transaction processing interface--- by lindawei <[email protected]>
.feature: Add WebSocketServlet by "fuwangqin" <[email protected]>
.feature: ostream::write supports buffered data
writing.feature: db_pgsql.cpp supports UNIX domain socket connection postgresql database.feature
: Added support for
postgreqsl.bugfix: When the acl::thread object is reused (ie, call start again after the thread exits), thread_id_ needs to be set to 0 --- by [email protected]
.bugfix: thread_cond The internal calculation time of the method wait in .cpp is incorrect
. bugfix: serialize/gsoner.cpp does not support std::map type data objects optional
.bugfix: buf_ in server_socket is a fixed size of 64 bytes, for UNIX domain sockets is not enough, change its type to acl::string
.bugfix: In redis_client.cpp, when the connection requires authentication, if the authentication infinite loop state may be caused due to network or other reasons, add the flag bit member (authing_) to prevent circular authentication
. Bugfix: redis_client_cluster::set_all_slot method does not support connection Authentication function.
bugfix: polarssl_conf.cpp put the initialization of the library outside the constructor to cope with different usages of the application.
bugfix: some command classes in the redis module have problems with binary data support.
bugfix: polarssl_conf.cpp is in dynamic When loading the polarssl library, if the application defines a global acl::polarssl_conf object, because the __polarssl_path inside the polarssl_conf class is also static and global, the order in which the compiler constructs these two objects is to construct the application's polarssl_conf object first and then the polarssl_conf object. Construct __polarssl_path, which will cause polarssl_conf to crash during construction.
Bugfix: When overloading the < and > symbols in string.cpp, the minimum length value should be used for comparison, but it is mistakenly written to take the longest length for comparison, which is easy Causes illegal memory access
. Bugfix: In redis_hash.cpp, there is a wrong command word when calling the hmset command.
Bugfix: The reset method in http_request.cpp should add header_.reset() and client_->reset(), otherwise will cause memory leak
.bugfix: In websocket::make_frame_header, when payload_len is 126, there will be a memory cross-boundary problem--- found by "lindawei" <[email protected]>
.bugfix: There is a deletion problem in connect_manager::remove--- - "fuwangqin" <[email protected]>
.bugfix: In the gson template function in gson_helper.ipp, when the second parameter is std::list or std::vector, if the member of the element in the container is an object pointer The problem that the pointer object will be released twice due to the shallow copy -- "lindawei" <[email protected]>
.bugfix: escape in query.cpp should not escape \r\n when escaping
4) Network coroutine library: lib_fiber
.feature: acl coroutine library can run on
MacOS.feature: fiber_server.cpp supports choosing different event engines when coroutine scheduling.feature: Coroutine
event engine adds support for Winsock
IOCP.feature : Added API acl_fiber_schedule_init() to set the coroutine scheduler to auto-start
mode.feature: Added API acl_fiber_schedule_set_event() to explicitly set the coroutine event
engine.feature: fiber_io.c when there is no monitoring network handle and no timer task , the IO coroutine will automatically exit
.feature: The coroutine module can support the WIN32 window message engine, so that it can be seamlessly integrated with interface programming.feature
: acl_fiber_poll/acl_fiber_select supports the win32 platform.feature
: Added kqueue event engine to support FreeBSD, so that acl coroutines support FreeBSD
.feature : fiber_event.c adds a lock that supports coroutine/thread mutual exclusion at the same time, and the performance can reach tens of millions of QPS
. Feature: fiber.c, fiber_alloc needs to add signal processing.
Feature: EVENT::poll_list, epoll_list can Use FIFO or STACK
.feature: hook sendfile/sendfile64
.feature: add a C++ version of the coroutine timer class fiber_timer
.feature: fiber_server.c When running independently, you can also specify the master_reuseport option through the configuration
file.feature: master_fiber adds get_conf_path, Add acl_fiber_server_conf to acl_fiber_server.cpp. These two methods are used to obtain the current service configuration file path
. Feature: In master/fiber_server.c, the stack size of the fiber_sleep coroutine is allowed to be controlled by the configuration item; at the same time, the internal STACK_SIZE default value is set by 64 K is changed to 128 K, which is also
the default stack size of the coroutine created by the user level
.feature: fiber_server.c/master_fiber.cpp supports SIGHUP signal callback
process.feature: fiber_server.c supports multi-thread running
mode.feature: fiber_server.c coroutine server template adds a smooth exit mechanism.bugfix
: when all coroutines have not When acl_fiber_stop is called after execution, the cached coroutine object and the memory of the running coroutine object need to be released.
bugfix: acl_fiber_gethostbyname is unsafe in coroutine mode, there will be multiple coroutines sharing a thread local Variable problem, which leads to memory parsing failure. The solution is to use coroutine local variables
. Bugfix: Fixed the problem of FreeBSD kqueue.
Bugfix: The function epoll_event_wait of event_epoll.c does not handle the error condition of EPOLLERR|EPOLLHUP, which will cause the program to enter Infinite loop.bugfix
: event_select.c/event_poll.c When no network handle is monitored, Sleep should be called, because select/WSAPoll under Windows does not allow the input handle to be empty
. Bugfix: event_poll.c should handle exceptions ( POLLERR | POLLHUP | POLLNVAL)
.bugfix: fiber_io.c, fiber_io_loop is prone to 32-bit integer overflow, which will cause the timer to fail to trigger
. Bugfix: fiber.c, __thread_fiber->switched is originally a signed int type, it may be Coroutine scheduling problem due to overflow problem
3. Link address
Code cloud: https://gitee.com/acl-dev/acl
github: https://github.com/acl-dev/acl/
sourceforge: http://sourceforge.net/projects/acl/