Java Transformation Outsourcing five big data architecture, is so domineering
Introduction
Qiaoer Ye (a pseudonym) is my study group of veterans, we knew last year, he was five years old bird Java experience. Late last year, when he had just started to learn a large data called me, some of the issues on the learning path counseling, it had just the transformation of big data I gave him a few less mature advice, and provides some sort of my own data, earlier this year a successful transition, and data from the development to the present framework, it is regressed.
Personal introduction I Qiaoer Ye, specialist, coordinates of second-tier cities. August 14 graduate started in Java development, a monthly salary of 3800, foreign resident in more than two years, beginning 17 came into contact with large data, the end of 18 began learning system, this year began a formal full-time big data development and architecture work-related. Salary increase of 40%, the current annual salary of 200,000.
Why I would like to turn Big Data
Today, almost a year's time from the distance determined to leave the company last year. This year, his mind and mentality have taken place in no small change. The theme of this section is why I want to turn big data?
In fact, before meeting Feng brother, at the beginning of 2017, I was on a business trip in Yunnan resident when he started in the large contact data. I remember it was New Year's Day from the beginning, was to buy a few pieces of dark horse that shipping in 2016. We understand big data, when just opened my door to the New World, from the perspective of salary, generally higher than Java, which I can not refuse. I had a great look at this is to see the extent of salary to go.
**
back to Chengdu Corporation in the mid-17 to around 18 years in April, after another into the company's other business lines, but working with large data content and no major errors, no access. During this time, also from time to time to check back at night video learning, then learning bad habits, take a look at a lot of content, and not to write some code, make some notes to understand them. Of course, the absorption effect will be more worse.
In fact, this time there is a problem with real-time calculation of business, starting with the front desk was scheduled task to trigger a request and then come back inside from a database query, to run for some time, more slowly, because many pages once a minute, to read a dozen interface page to complete the assembly.
Since this is the time the data has characteristics, that is, can be assembled by the time dimension, daily data from hours to statistics, it can be accumulated from the days of the month, year from the month cumulative. In this way, the request comes in, only the data other than the day pulled within a database, data from other time periods are taken out from the cache assembly. This time transformation, basically solved frequently check the library, as well as the problem of slow. But when the cold start caching how to do this, is a problem, we were very simple, visit the page again.
Then I was not satisfied in the day to see the article Storm found this thing, the first contact with something real-time computing. The big data science curriculum at this time, do not see here, and then look through, and found this explanation. Door to a new world, looked a little bit.
Read it again, so cool, we want to transform the functions of the above, let me engage the boss. At this alternative was Kafka Storm, is relatively common solution, but these things are for the first time in our company use. Many people only heard kafka, never used, not to mention the Storm. Kafka data source to this level, either by writing to deploy an application to a client side database collected from people of good api written by Kafka Kafka, the canal is to use either to resolve binglog people's logs, which send to Kafka.
The next step is to write code that storm, and I remember finished a business presentation with them after these things, when including the then technical director, including force are relatively ignorant. Because they did not come into contact with large assembly data related.
But the matter does not end, because the way I was transferred to other lines of business to support the go, this thing they did not pick up, first, no one will be full, and the second is in front of that transformation again perfectly adequate, speed what the requirements are met. But this thing for me, there is not a small gain, to be later determined to do big data foreshadowed.
On May 18, the way they hand pick a company with a large detached data line of business, there are almost 12 people following way. Content is more work on the coordination and management. Technically thing is basically not much. This feature team is to collect party data, and then internalized into our big-screen applications.
The amount of data part of the project, and some also on the table million. In line with some of the features of big data. Routines can use big data to do, if I do today. I will use this set to do big data acquisition, processing, and collection of this set do. In a Java project inside to do before returning the data collection process directly resulting data becomes needed. The latter needs change, these data have to calculate the new data from the new take back, to change the code, and re-deploy.
There is a deep impression of the project, data processing work overtime this weekend, has been half past three on Monday got to go back, this is the norm. At that time, in fact, there is a warehouse concept points, but do not know how landing. Looking for people to learn, baa.
Almost time came in October with the team the past few months, basically spent in overtime but I also realized that the lack of technology, plus handle a variety of co-ordination on the project every day, a lot of things out overtime just do false data (meaning customers) come out so that the leaders of various projects to see it. For us technology is really a waste of your youth.
One might argue that management go bad? Do not they have control of a dozen people yet? Indeed, management is the direction you want to do later. Only current, the way mentally immature, do not be afraid of a bad example, and secondly, technology management, technology first, and then the management. This is the company I spent a decade older brother who saw, he began as manager of the technical department, but no technical ability, I also recruited him go. Not that he was bad, but this technology is not able to manage a technology research and development department.
Four years have passed, and now, I had that leader, with more than 50 people now look like the company's R & D department, his management skills are the last two years to learn, and to be the leader of that decade, the edge of the non-core sectors. In fact, from that point, I saw a technology company's management, what is the most important skill?
We can think of it, if the company because of some factors to attrition, what kind of person would first consider?
In fact, considerations on future career development, I think I was in before age 30, should be important is the accumulation of technological capabilities. And from that time 30 years old, and 5 years.
Considering these aspects, there is going renovated, once decorated to loans, will spend a lot of money, according to the current salary level is not played, behind these do want to get rid of large data they might have in the future delayed, time is money ah.
Finally, in December 2018 No. 5, I bare quit, or at such layoffs everywhere environment. But it does do not learn from me, if you can not meet your economic strength next six months of expenses, then do not do it, because you will face great psychological pressure.
**
From the company's day at noon, the skies are blue, the color looks good ginkgo children, young married woman to go directly to the school, where she prepared the second study section.
That night, I told the young married woman sitting with their school library, I feel good.
In fact, in the previous two days I deliberately to practice my inattention problems, is impetuous, wanted to quietly heart. If you are restless, impulsive, you do not do anything.
Then re-set a bit in recent years to their deficiencies. There are two big problems is learning modalities, there is no record of knowledge (recorded here is that he wrote, organize their own, not your collection, cut others) and the formation of habits of knowledge. Then it is impetuous, which also want to learn, that want to learn, look here, look there. In the end, no one's done.
By tomato clock and meditation help, I was able to focus on some of the attention. Then began to learn the contents of large data, programs or courses to go with, or that of a dark horse. But not to say that all finish, because time does not too enough, I plan to look like March of the year after, can first get the interview.
Before learning styles have changed with, say a little more obvious is when learning principles, I will review in the evening when the draw drawing himself again, and of course the draw when thinking back to him, and then all of the code knock again. Those schematics painted himself into now able to remember.
About a week or so, the planet met Feng brother, probably introduced to him about the situation, chatted and gave some advice. Although I did not fully given route in accordance with the go, but the route and the information he provided to me referential significance, the most important thing is to give my " confidence ", so I determined to do it is to go right.
Speaking in front of a system of things, say a point, the JVM is knowledge, because this one is did not dare think of something, think hard. Indeed it is difficult, not your back a little face questions able to get. At that time I saw the class to explain the JVM knowledge of the Internet, this class is a class inside a chapter architecture buy, the teacher speaks well, then combine "in-depth understanding of JVM Virtual Machines," this book, read the memory layout, garbage collection and several other questions are often asked a few chapters, put together a few articles, after more than a week, found that many of the JVM interview knowledge can answer come up. And inside the brain can associate these things, it is no feeling before.
Not only is the JVM, any technology, we can hope to learn of the system. This is when the manufacturers of the interview, I believe you have a profound experience.
Probably back in the library for a month's appearance, from her own school back there to live over here, and then insisted nine in the morning to start learning, from noon to rest half past twelve at night to 10:00 so, basically in accordance with the schedule to the University.
March continued to look, in fact, before the April entry, not to interview time, according to study time will come, but also from time to time in the morning to go to community basketball court to play ball one hour to ease the pressure. Walk around the park or get up early to go outside, stay awake, thinking.
Interview, to be honest, I was not very good. Later this interview veterans consult Feng brother, help me from resume writing to interviewing delivery finished re-set all sort out a bit, have to say the experience is real wealth, these methods behind me interview very helpful.
This year's market for low-educated people, is really hard, but if we continue, still have a chance. There are several companies, to interview more than one hour talk did not result. Many direct throw trash, did not have the opportunity. But eventually I received two, although you can also get a better feel of the offer, but went anyway, because it can not afford to drag, and must get back to the blood. Doing payroll, up 40% on the basis of the way before, and now there are almost 20 million a year.
Although there is no good, but I entered this industry Yeah, back can continue to accumulate, to where they want to Charge.
Current state
Now the situation a little easier, but more surprisingly, I have only one person with system development capabilities, the other partners are doing little data governance, data modeling and development. Because we are part of a data service, a lot of these things provide external interface development also fell on my head, but still can get, did not increase over the class!
In addition to start a company that eight hours, almost a week now basically learning time is about 15 to 20 hours. Basically learning will come back at night for two hours, because my goal is to go to Internet companies. During that time something in front of school is to achieve a common company's needs, but if you do not have the knowledge into the system, is not the line, now also a piece to add.
I posted a link I # this year's Ali side # experience. To know the peace column "Notes of an Alibaba side of experience" (https://zhuanlan.zhihu.com/p/60828766) have almost 1.2K praise, we feel the inside of the process. Very simple, but there is no strong accumulation, or not, I am a counterexample.
Some thoughts and suggestions
1. If you want a good to turn big data, it would immediately begin. No idea of the route, ask someone who can ask. Of course, with online training courses to go, beep beep miles miles above a little more, if you want you can find. The key is that you do not just wait and see ah, ah have to do it, to learn it.
2. For people in transition, the items on the resume is very important, it can be said to be a stepping stone. Although many items online, but you have based on your actual come, not seek more refinement here.
3. In addition to the strength of knowledge, resume writing and interview the whole process are also part of the overall strength of the beginning of the interview he did not pay attention to this, that strength on the line, resulting in wasted a lot of opportunities.
4. In any case, hope that we can take the time to think about how they wanted to go, what kind of capabilities they need to have it will not be abandoned by the rapid development of society, how old will not have a so-called crisis.
5. Do not be in your own comfort zone for too long, or else get out, out "mixed", sooner or later have to repay! Flew like a good move, do not make excuses!
6. Lastly, I hope everyone has their own core competitiveness, but also hope that my 35-year-old crisis is not because of the pressure, but not because of fear of layoffs.
--end--
This article from the blog article multiple platforms OpenWrite release!