Open Source Summer 2023 | Welcome to openEuler A-Tune SIG and Base-service SIG development tasks

Open Source Summer 2023 | Welcome to openEuler A-Tune SIG and Base-service SIG development tasks_tuning

 

Summer of Open Source is a series of summer activities initiated by the Institute of Software, Chinese Academy of Sciences and openEuler, which aims to encourage students to actively participate in the development and maintenance of open source software, and promote the vigorous development of excellent open source software communities. The event unites major open source communities to provide projects for the development and maintenance of important open source software, and open registration to college students around the world.

The openEuler community currently has 94 projects online! All college students are welcome to apply according to the direction they are interested in or good at

Today I will bring you the task introduction of A-Tune SIG and Base-service SIG. These two SIGs have released 11 projects in total, involving operating system, performance tuning and national secret security fields. The project tasks cover a wide range of difficulty levels. Moderate. Everyone is welcome to participate in the project, and we will provide you with rich learning resources and technical guidance.

Project 1: unbound uses libverto instead of libevent

project description:

libverto provides an asynchronous API interface that allows other libraries to expose an asynchronous interface to the application and to start or stop the application's main loop. libverto provides an asynchronous API interface for controlling the event loop of glib, libev, libevent, and tevent libraries for application calls.

Libevent is a lightweight high-performance network library designed to replace the event loop in event-driven network servers. It provides a unified response mechanism for three types of events and supports callbacks after IO events, signal events, and timer timeout events occur. function.

Since the libverto project is more flexible and supports more different event libraries, the project mainly replaces the dependency on libevent with libverto by modifying the dependency on libevent in the unbound software. The main objectives of the project are as follows:

Replace at least 50% of the interfaces used in libevent in unbound with interfaces using libverto, and keep the original software functioning normally.

Output standard:

1. The code is merged into the master branch of the unbound warehouse in the form of patch.

2. Alternative interfaces account for more than 50% of the original interfaces, and cover 80% of different interfaces. For irreplaceable interfaces, continue to use libevent, and provide corresponding material descriptions.

3. After the replacement, the unbound test cases and functions are the same as the original ones.

4. Well commented/documented.

skills requirement:

1. Familiar with C/C++ development language

2. Familiar with the operating system event library

Project home page:

unbound uses libverto instead of libevent · Issue #I6YM46 · openEuler/open-source-summer - Gitee.com

Project 2: libevhtp uses libhv instead of libevent

project description:

libhv is a cross-platform network library similar to libevent, libev, and libuv, which provides an easier-to-use interface and richer protocols (community address: https://github.com/ithewei/libhv).

Libevent is a lightweight high-performance network library designed to replace the event loop in event-driven network servers. It provides a unified response mechanism for three types of events and supports callbacks after IO events, signal events, and timer timeout events occur. function.

The main objectives of the project are as follows:

Replace at least 80% of the interfaces used in libevhtp to libevent with interfaces using libhv, and keep the original software functioning normally.

Output standard:

1. The code is merged into the master branch of the libevhtp warehouse in the form of patch.

2. The replaced interface accounts for more than 50% of the original used interface, and covers 80% of different interfaces. For irreplaceable interfaces, continue to use libevent, and provide corresponding material descriptions.

3. After the replacement, the unbound test cases and functions are the same as the original ones.

4. Detailed notes/documents.

skills requirement:

1. Familiar with C/C++ development language

2. Familiar with the operating system event library

Project Mentor:

libevhtp uses libhv instead of libevent · Issue #I6YLUY · openEuler/open-source-summer - Gitee.com

Project 3: nghttp2 uses libverto instead of libev

project description:

libverto provides an asynchronous API interface that allows other libraries to expose an asynchronous interface to the application and to start or stop the application's main loop. libverto provides an asynchronous API interface for controlling the event loop of glib, libev, libevent, and tevent libraries for application calls.

Libev is an event loop library that manages these event sources and provides them to programs by registering observations for events such as a file descriptor being read or a timeout occurring. The program is controlled by executing an event loop and communicating events through a callback mechanism. Features include pid watcher, absolute time based periodic timer, and epoll/kqueue/event ports/inotify/eventfd/signalfd backend support, fast timer management, time jump detection and correction, and ease of use.

Since the libverto project is more flexible and supports more different event libraries, the project mainly replaces the dependency on libev with libverto by modifying the dependency on libev in the nghttp2 software. The main objectives of the project are as follows:

Replace at least 50% of the interfaces used in libev in nghttp2 with interfaces using libverto, and keep the original software functioning normally.

Output standard:

1. The code is merged into the master branch of the nghttp2 warehouse in the form of patch.

2. The replaced interfaces account for more than 50% of the original used interfaces, and cover 80% of different interfaces. For irreplaceable interfaces, continue to use libev, and provide corresponding material descriptions.

3. After the replacement, the test cases and functions of nghttp2 are the same as the original ones.

4. Detailed notes/documents.

skills requirement:

1. Familiar with C/C++ development language

2. Familiar with the operating system event library

Project Mentor:

Project home page:

nghttp2 uses libverto instead of libev · Issue #I6YLTU · openEuler/open-source-summer - Gitee.com

Project 4: Provide a Rust-based OS boot tool

project description:

Background: The boot process of the operating system passes through the BIOS to boot grub, and then grub boots the kernel. Grub is a must-select boot tool for the server, and its security and reliability requirements are high. The existing grub is written in C language, and there is a certain degree of security. Hidden danger.
Solution: Use Rust to rewrite a lightweight, safe and reliable boot program to meet the system boot function in server scenarios.

Output standard:

Use Rust to rewrite a lightweight, safe and reliable boot program that complies with the UEFI specification, has multi-system boot capabilities, and supports booting the kernel installed on the local disk.

skills requirement:

1. Understand the UEFI specification
2. Understand the system boot process
3. Familiar with C, RUST

Project Mentor:

Project home page:

Provide a Rust-based OS boot tool Issue #I6YLML openEuler/open-source-summer - Gitee.com

Project 5: Provide an adaptation layer for libhv, compatible with libverto interface

project description:

libverto provides an asynchronous API interface that allows other libraries to expose an asynchronous interface to the application and to start or stop the application's main loop. libverto provides an asynchronous API interface for controlling the event loop of glib, libev, libevent, and tevent libraries for application calls.
libhv is a cross-platform network library similar to libevent, libev, and libuv, which provides an easier-to-use interface and richer protocols (community address: https://github.com/ithewei/libhv).
The project requires an adaptation layer component ifm_libverto to encapsulate libhv, and the encapsulated interface is compatible with libverto. The main objectives of the project are as follows:

An adaptation layer component ifm_libverto is provided on the basis of libhv, and the interface of libhv is encapsulated into a libverto compatible interface. For the interface that libhv does not support, you can call the interface of libverto to complete.

Except for the function definition and structure definition of the header file in the encapsulation layer, the rest of the content needs to be rewritten and scanned through code fragments.

By modifying the dependency of gssproxy, relying on ifm_libverto has no effect on the function of gssproxy.

Based on the existing benchmark test project or design a new performance test case, test the performance in ifm_libverto, and provide a performance comparison report.

Output standard:

1. Ifm_libverto needs to provide corresponding dynamic library files.

2. Perform adaptation verification on gssproxy, and support switching to ifm_libverto. After switching, gssproxy can be compiled and passed the test case.

3. The performance test benchmark does not need to be merged into the corresponding warehouse, and an independent warehouse can be provided for testing, but the corresponding performance test comparison needs to be provided when the patch is submitted.

4. Corresponding test cases need to be provided, and the test cases can reuse the existing libverto test code.

5. In addition to libverto, do not actively introduce other open source software, and the relevant codes need to be self-developed codes, and pass the commonly used CodeCheck and pclint scans.

6. Detailed notes/documents.

7. The code is merged into the ifm_nettle warehouse, and the relevant source code files need to be isolated from other modules.

skills requirement:

1. Familiar with C/C++ development language

2. Familiar with the operating system event library

Project Mentor:

Project home page:

Provide an adaptation layer for libhv, compatible with libverto interface · Issue #I6YH0K · openEuler/open-source-summer - Gitee.com

Project 6: Performance Tuning Visualization Tool

project description:

When users use or develop applications on openEuler, there will be performance bottlenecks, and it is necessary to analyze where the performance bottlenecks are and perform tuning. A-Tune currently has automated and intelligent tuning capabilities, but lacks analysis and visualization capabilities. Therefore, it is hoped that a bottleneck point identification, analysis, and visualization tool will be added to the A-Tune performance tuning warehouse. For example, collect the cpu data of the current system, and output flame graphs through open source tools to provide users with analysis of performance bottlenecks.

Output standard:

1. Select 3 dimensions from CPU, memory, disk (Disk I/O), network (Network I/O), kernel or other software and hardware performance bottlenecks to realize corresponding data collection and visualization functions.
2. The tool is required to provide command line/UI/or other simple and easy-to-operate methods for users to use. Advanced: The tool is required to provide command lines based on A-Tune's existing command line framework for users to use.
3. The tool is required to output a visual data interface, which can be html files, pictures, terminal printing, etc. for users to conduct intuitive performance analysis or reference.

skills requirement:

1. Master python and Go programming language

2. Familiar with linux operating system, have basic operating system knowledge (such as cpu, memory, network)

3. Familiar with A-Tune tuning tool

4. Ability to independently search for information and solve engineering problems.

Project Mentor:

Visualization Tool for Linux Performance Tuning · Issue #I6YG4L · openEuler/open-source-summer - Gitee.com

Project 7: Implementing a dichotomous profiling tool

project description:

If the performance of two versions of a software differs significantly, the tuner needs to analyze and find out the patch that has the greatest impact on performance between the two versions. There are often tens or hundreds of code submissions between the two versions. Taking python software as an example, from python3.9.0 to python3.9.1, there are 282 submissions for only a small version update, and it is difficult for tuning personnel to locate. Often requires a lot of repetitive, time-consuming manual work. It is hoped that a dichotomy performance analysis tool will be implemented. By continuously dichotomizing 282 submissions and then building a python package, the built package will be tested for performance, so that key performance patches can be found automatically.

Output standard:

Under the A-Tune warehouse, develop a dichotomous performance analysis tool, realize the input of the software package to be analyzed (such as python/glibc), any two commitids, and performance benchmark scripts, and output the commitid with the largest performance improvement

skills requirement:

1. Have the foundation of linux operating system development
2. Proficient in rpmbuild package or obs package method
3. Master at least one mainstream programming language (C/C++/python/Java/Go)

Project Mentor:

Project home page:

Implementing a binary performance analysis tool · Issue #I6YG3Z · openEuler/open-source-summer - Gitee.com

Project 8: Multisystem cross-validation performance tool

project description:

There is a common scenario in performance tuning—the performance of the same software on two different operating systems is quite different. Cross-test data such as startup parameters and system services to find configurations/parameters that affect performance. Based on this goal, an automated cross-validation tool is implemented.
Taking centos and openeuler two operating systems as examples, first test the baseline performance of the two operating systems respectively, then configure the sysctl configuration, startup item configuration, and system services on the centos system to the openeuler system in turn and test the configured performance data; The same is true for the openeuler system.

Output standard:

Provide an application and application testing method to realize the cross-delivery function of sysctl configuration, ulimit configuration, and system startup item parameter configuration on the two operating systems of openeuler and centos.

skills requirement:

1. Have the foundation of linux operating system development
2. Master at least one mainstream programming language (C/C++/python/Java/Go)

Project Mentor:

Project home page:

Multi-system cross-validation performance tool Issue #I6YFVD openEuler/open-source-summer - Gitee.com

Project 9: Design and Implementation of Dynamic Tuning Algorithm Based on Transfer Learning

project description:

A-Tune is an AI-based operating system performance tuning engine. A-Tune uses AI technology to make the operating system "understand" the business, simplify IT system tuning work, and allow applications to perform well. When we optimize for a specific scenario, there are usually some similar scenarios that have been tuned in the past. By using the data of similar historical tuning scenarios, the tuning efficiency can be improved. Based on A-Tune's current tuning capabilities, this project designs a transfer learning algorithm to achieve dynamic tuning; supports storage and query of historical tuning data, and has a reasonable architecture design; compared with A-Tune's existing tuning algorithm, the efficiency is improved by 30 %; Applicable to mainstream big data scenarios (eg, Mysql). (Algorithm implementation can provide guidance)

Output standard:

1. Innovative solution: Based on A-Tune’s current tuning capabilities, design a transfer learning algorithm to achieve dynamic tuning; support storage and query of historical tuning data, with reasonable architecture design; 2. Technology leadership: Compared with A-
Tune The efficiency of Tune's existing tuning algorithm is increased by 30%, which is suitable for mainstream big data scenarios (eg, Mysql)

skills requirement:

1. Have a certain understanding of machine learning models
2. Have a basic understanding of linux operating system development
3. Have some understanding of A-Tune tuning function

Project Mentor:

Project home page:

Design and Implementation of Dynamic Tuning Algorithm Based on Migration Learning · Issue #I6YIF9 · openEuler/open-source-summer - Gitee.com

Project 10: A-Tune recognizes application scenarios and dynamically recommends tuning parameters

project description:

When the user needs to adjust the performance of a piece of software, the following situation may occur: the user only understands the parameter function and value range of the current software, but does not know the kernel mode parameters, and cannot give the tuning range of the kernel mode parameters. At this time, A-Tune can add new functions: 1. Identify the current application scenario according to the type of tuning software; 2. According to the result of the application scenario identification, recommend to the user the kernel state parameters that need to be tuned.

Output standard:

1. Realize common scene awareness of software models, such as computing, storage, network and other software load resources. (Currently A-Tune has a certain capability)
2. Realize the correct classification of software models (Currently A-Tune has a certain capability)
3. Realize parameter recommendation in corresponding scenarios

skills requirement:

1. Have the foundation of linux operating system development
2. Proficient in python language, understand go language
3. Have AI/machine learning foundation

Project Mentor:

Project home page:

A-Tune recognizes application scenarios and dynamically recommends tuning parameters · Issue #I6YH50 · openEuler/open-source-summer - Gitee.com

Project 11: RPM signature and verification support national secret algorithm

project description:

Background: The RPM packages released by each Linux distribution in the RPM software warehouse will be digitally signed, so that users can perform digital signature verification on the RPM packages obtained from the Internet to confirm that the RPM packages are credible and complete.

Requirements: At present, each release version mainly uses RSA certificate and SHA256 hash algorithm for digital signature of RPM package. my country is promoting the commercial use of national secret algorithm. This project needs to use national secret algorithm SM2 (elliptic curve public key cryptography algorithm) and SM3 (hash algorithm) realizes the digital signature of the RPM package, and can realize the national secret algorithm digital signature verification of the RPM package when using dnf to install the software package.

Output standard:

1. Realize the RPM package signature and support the use of SM2 digital certificate and SM3 hash algorithm2
. Implement rpm -K to support the verification of RPM packages signed with the national secret algorithm3.
Implement management functions such as gpg key import and export and add support for the national secret
algorithm4 . Implement dnf to support the verification of RPM packages signed with the national secret algorithm, and does not affect the verification of RPM packages signed with other algorithms

skills requirement:

1. Familiar with python development language
2. Familiar with C/C++ development language
3. Familiar with the use of package management tools such as dnf and rpm
4. Familiar with Linux system’s RPM package signature management process
5. Understand encryption and decryption algorithm knowledge

Project Mentor:

Project home page:

RPM signature and verification support national secret algorithm Issue #I6Y72Q openEuler/open-source-summer - Gitee.com

Event Schedule and Participation

Currently in the process of student registration, communication with tutors, and submission of project applications.

Open Source Summer 2023 | Welcome to openEuler A-Tune SIG and Base-service SIG development tasks_Application_02

In order to let everyone know more about the projects of A-Tune SIG and Base-service SIG, openEuler will hold an online live broadcast on May 23 (next Tuesday).

Time: May 23 (next Tuesday)

Venue: openEuler Station B

Welcome to follow openEuler Station B, make an appointment for live streaming

Guess you like

Origin blog.csdn.net/openEuler_/article/details/130728892