Into the algorithm engineer (the road of algorithm engineer development and practice)

Copyright statement: This article is the original article of the blogger and follows the CC 4.0 BY-SA copyright agreement. Please attach the original source link and this statement for reprinting.
Link to this article: https://blog.csdn.net/weixin_37737254/article/details/110508319

algorithm engineer

Author：louwill

Machine Learning Lab

As the position of algorithm engineer is based on different business scenarios and application directions, their respective jobs are relatively different. So it is difficult to have a generalized algorithm engineer technology stack. For example, there are machine vision algorithm posts for image orientation, natural language processing algorithm posts for text orientation, and speech recognition algorithm posts for speech.

This article only sorts out the common, basic and necessary R&D skills of algorithm engineers. In other words, no matter which business scenario you are doing in algorithm work, these basic research and development skills must be known.

This set of skills list mainly includes two types, one is theoretical skills, and the other is programming languages ​​and tools.

data analysis

Data analysis is the basic ability of algorithm engineers. All current algorithms are based on data-driven, and data analysis can be regarded as a precondition for algorithm research. The main technology stacks include:

EDA

统计分析

统计绘图与数据可视化

sql

...

Data Structures and Algorithms

Data structure and algorithm is one of the four basic courses for computer practitioners. A deep understanding of commonly used data structures and basic algorithms is the basic skill of algorithm engineers. At present, the data structure and algorithm questions of the interview test are essential items for both large and small companies in the industry. Brushing leetcode is a must for the algorithm before finding a job.

data structure

线性表

数组

链表

字符串

栈

队列

堆

哈希表

树

图

Basic algorithm

二分查找

排序

递归

回溯

分治

双指针

深度优先搜索

广度优先搜索

动态规划

...

Machine learning

Machine learning is the core competence of Algorithm Post. For machine learning algorithm engineers, as the algorithm posts become more and more inwardly written, it is a common requirement to have a deep understanding and manual push of commonly used machine learning models and algorithms. Commonly used machine learning models include:

单模型

    线性回归

    逻辑回归

    Lasso

    Ridge

    knn

    ID3

    C4.5

    CART

    感知机

    神经网络

    SVM

集成学习

    GBDT

    AdaBoost

    XGBoost

    LightGBM

    CatBoost

    Random Forest

无监督模型

    kmeans

    层次聚类

    谱聚类

    PCA

    SVD

    LDA

概率模型

    朴素贝叶斯

    贝叶斯网络

    EM

    MCMC

    最大熵

    CRF

    HMM

Machine learning model map

Deep learning

Deep learning is one of the most popular directions of machine learning, and different application fields also have their own focus. For example, image algorithms are more focused on CNN, and text algorithms are more focused on RNN. However, the commonly used network models and development history should be clear.

DNN

    MLP

    BP

CNN

    分类

    检测

    分割

    AlexNet/VGG/GoogleNet/ResNet/DenseNet

    ...

RNN

    LSTM

    Transformer

    Attention

    Bert

    XLNet

    ...

GNN

...

Linux

Linux is the main tool for algorithm engineers, and basic and common commands must be proficient.

Linux

Shell

Vim

Git

版本管理

远程仓库

分支管理

...

Python

I won't say much about the importance of Python. For the complete tutorial, please refer to teacher Liao Xuefeng:

https://www.liaoxuefeng.com/wiki/1016959663602400

The third-party libraries that algorithm personnel should be proficient in include:

sklearn

tensorflow

keras

pytorch

cv2

nltk

...

C++

C++ is the same as Python, and even more important than Python. C++ is essential for the industrial deployment and landing of model algorithms. C++ online tutorial reference: https://www.w3cschool.cn/cpp/

SQL

SQL is the basic research and development tool for algorithm personnel. The ability to write proficient SQL scripts is also a very important manifestation of the ability.

增删改查

MySQL

Oracle

SQLite

...

Docker

Docker is an open source application container engine that allows developers to package their applications and dependent packages into a lightweight, portable container, and then publish to any popular Linux machine, which can also be virtualized.

reptile

For algorithm engineers, crawlers can independently write scripts to obtain some structured data on web pages. Python-based crawlers should master:

前端知识

    html

    css

    javascript

Python第三方库

    urllib

    beautifulsoup

    request

    lxml

    scrapy