First public account: [NLP from entry to giving up]-daily update technology sharing
Let me introduce myself briefly. I work in a social company as an NLP algorithm engineer. I mainly do text classification, sequence labeling, and text similarity. I have also done search/recommendation needs.
I have been working for almost a year and a half. During this year and a half, while working, I have been thinking about what is the core competitiveness of an algorithm engineer and how can I avoid being instilled when I just started. The 35-year-old midlife crisis.
I have seen a lot of other people’s experience summaries, combined with my own work experience, and slowly gained a little bit of my own insights. I want to share them and hope it will be helpful to everyone.
Let me use a big vernacular to talk about my own summary of core competitiveness: different scenarios, choose the right method, and get the greatest effect.
This sentence is very simple, but it is not that simple to do it! ! This requires our many years of experience accumulation and summary.
Next, I will simulate a work scenario, and talk about where the core competitiveness is reflected, and how to continuously improve my ability! !
0. Background
If you now receive a request from a new business party, what is your first step?
Open the refrigerator and put the elephant... ah, no
1. Demand Qualitative
In the first step, we need to qualitatively and set the direction for this requirement. Is it a text classification task? Name body recognition? Or machine translation? In different directions, we will use different methods and focus. Of course, in many cases a requirement is not just a task, but in most cases it is a combination of multiple tasks. This is for the time being.
2. Research
Set a good direction, such as a task of text classification, the next step is to do research.
The survey is mainly divided into two parts: data and model.
Research data: For an algorithm engineer, processing data definitely occupies most of the day. Many people ridicule themselves as SQL BOY. But what I want to say is that data determines the upper limit of the model. This sentence is absolutely true. Only by truly understanding our data can we achieve good results in the tasks that follow.
You need to know the approximate magnitude of your data; there are not many irregular data; whether data enhancement is needed or not, which data enhancement technology may be effective for the current data; what features may be useful in combination Wait, these all require careful consideration when investigating data.
Here is an insert, if the data is very irregular, at this time you have to think about whether to start from the model or the data itself, and what is the appropriate method for each. For example, you can look at the characteristics of these irregular data can be used. For example, is it possible to use word-based models without word segmentation, etc.?
In the process of data processing, because many company data now exist in hive, we will use big data development tools such as MapReduce, Hive, Spark, etc. Mastering these tools will definitely allow you to easily handle data processing.
Research model: The purpose is to use suitable models according to different requirements. You need to make decisions based on various conditions. If the business side needs high accuracy, choose which model can meet the requirements. If the business side needs high response speed, which model is appropriate. For the current data, which model might work well.
Or take our text classification requirement as an example. If speed is important, will FastText be better, or what about TextCNN? If accuracy is required, can Bert? Or how about Albert and FastBert?
In this process, you need to read a lot of papers and blogs, look at other people's experiences and summaries, and help yourself make the final decision.
3. Train/optimize model
Model training and optimization generally take up too much time. There is a sentence that I forgot where I saw it. It is said that it took two weeks to process the data and two hours to train the model. It may sound a bit exaggerated, but it does so. In this process, your main job is to use code to implement the model, think about how to run this model more efficiently, and how to get better results?
For example, to train a model using GPU, you need to be familiar with the knowledge of model/data parallelization. For example, for a specific task, modify the loss function, modify the optimization function, and so on.
4. Deploy and go online
We need to provide an interface for developers to process data and return results. At this time, you have to think about the response speed of your interface? How much concurrency can be? In view of this situation, you can do a simple stress test yourself. Generally speaking, you need to master common tools such as grpc, kafka, flask, nginx. Of course, you don't need to be proficient in these. If the demand side has high requirements for the interface, you can use some mature open source frameworks, so don't panic.
Well, that's about it. We have great patience throughout the process, and we have to keep accumulating experience from the process.
5. Summary
In summary, if you want to improve your core competitiveness, you must do two things: algorithm + engineering.
For algorithms, go deep into the bottom layer and get your hands dirty. The algorithm model focuses on quality, not quantity. Since you want to make a model, you must figure it out thoroughly, break it apart and smash it, don't be paradoxical.
For engineering, you need big data development capabilities and model training and deployment capabilities.
Both must be grasped, both hands must be hard, in order to be safe. In fact, in this respect, I did not do well. People are lazy. The principle is set, but when you do it, it is a short man of action... Then I learned from the pain and tried to force myself, so Two warehouses were built a few days ago.
One is a warehouse of various interview questions about NLP (this is quite a lot of my own notes, upload it slowly, and I will review it myself):
https://github.com/DA-southampton/NLP_ability
A warehouse is about how various models are actually implemented in major companies. The address is here:
https://github.com/DA-southampton/Tech_Aarticle
For this warehouse, I did not expect that the number of stars in these two days is quite large for me.
I think for myself, maybe everyone has such a pain point, only knowing what the model is is of no use, and knowing how the model is really used is the kingly way.
So interested friends can go and take a look. If I can bring you a little help, I will be very happy. I will continue to update, and strive for every article with new thinking instead of old wine in new bottles.
After finishing work, click like, so that I can drink a few more bottles in Paradise Supermarket happily! Bow thanks!