On top of the wave, OpenAI could be the first $10 trillion company in history

the gold rush era was a lot like

If you went to California to pan for gold at that time, a lot of people would die, but people who sold spoons and shovels would always make money. The so-called shove and pick business.

Large models are platform opportunities. According to our judgment in the past few days, the model-first platform will be larger than the information-first platform. The platform has the following characteristics:

• ① It works out of the box;
• ② There must be a sufficiently simple and good business model. The platform allows developers to live on it and earn enough money to support themselves, otherwise it is not called a platform;
• ③ He has his own killer app. ChatGPT itself is a killer app. Today, the platform company is you in the Apple ecosystem. No matter how well you do, as long as you become a big Apple, you will be confiscated, because it needs to use your underlying things, so you are the platform. Platforms generally have their anchor points and strong support points, and there are many opportunities for long-term OpenAI equipment—maybe this is the first $10 trillion company in history .

This is a fierce competitive platform battle, and a company with a large volume in the future. Competition in this field is fierce. The price is too big (the price is too big), it would be a pity to miss it. Anyway have to try.

Inflection point The marginal cost of obtaining information begins to become a fixed cost.

We must remember that anything that changes society or industry is always a structural change. This structural change is often a large cost, from marginal costs to fixed costs.

For example, Lu Qi studied at CMU and drove out of Pittsburgh. A map costs 3 dollars, and obtaining information is very expensive. Today I want a map, there is still a price, but it has become a fixed price. Google pays an average of US$1 billion a year to make a map, but each user wants to obtain map information, basically at a cost of 0. In other words, when the cost of obtaining information becomes zero, it must have changed all industries. This is what happened in the past 20 years. Today, it is basically free information everywhere (free information is everywhere).

Today's 2022-2023 inflection point is large models

What is the inflection point in 2022-2023 today? It is unstoppable, overwhelming, and why? exactly the same. The cost of the model goes from marginal to fixed, because there is a thing called a large model.

The cost of the model begins to change from marginal to fixed, and the large model is the core of technology and the basis of industrialization. Once OpenAI is set up, the development speed will climb rapidly. Why is the model so important and this inflection point is so important, because the model has an intrinsic relationship with people. Each of us is a composite of models. There are three models of people:

• Cognitive model, we can see, hear, think, and plan;

• Task model, we can climb stairs, move chairs and peel eggs;

• Domain model, some of us are doctors, some are lawyers, and some are code farmers.

That's all. All our contributions to society are a combination of these three models. Everyone does not make money by the strength of their hands and legs, but by their brains.

In the future, the only value is how much insight you have

Human society is driven by technology. From the agricultural era, people used tools to do simple labor. The biggest problem was that people were bound to the land. People lacked mobility and freedom. The biggest change that industrial development has had on people is that people can move around and go to cities and factories. The early industrial system was dominated by physical labor and supplemented by mental labor, but with mechanization, electrification, and electronicization, human physical labor declined. After the information age, people are mainly engaged in mental work, and the economy has shifted from a commodity economy to a service economy—code farmers, designers, and analysts have become typical occupations in our era.

This inflection point of large models will basically affect all people and blue-collar workers in the service economy, because they are models. Unless you have unique insights, there are large models of services you are engaged in today. Typical careers for the next era, we think of as entrepreneurs and scientists.

Trinity Structural Evolution Model

The essence is that any complex system, including a person, a company, a society, and even the digital system of digitalization itself, is a complex system. The "Holy Trinity" includes:

• "information" system (subsystem of information), which obtains information from the environment;

• "Model" system (subsystem of model), which expresses information, performs reasoning and planning;

• "Action" system (subsystem of action), we finally interact with the environment to achieve what humans want to achieve.
  

What is the next inflection point?

The next inflection point will be combination: "mobility" everywhere (autonomous driving, robotics, spatial computing). That is to say, people need to act in physical space, and its cost also moves from marginal to fixed. 20 years later, everything in this house has robotic arms, automated things. Anything I need, press a button, the software can move, and I still need to find someone today.

So, which companies can reach the next inflection point and stand at the next inflection point? I think Tesla has a high probability. Its autopilot and robots are very powerful now. Microsoft climbed up with OpenAI today, but how to stand at the next inflection point?

Three inflection points:

• ① Information is everywhere today. In the next 15-20 years, models are knowledge and will be everywhere. In the future, open it on your mobile phone, any internet connection, and the model will come over. It teaches you how to answer legal questions and how to do medical tests. No matter what kind of model can be everywhere.

• ② In the future, automated and autonomous actions can be ubiquitous.

• ③ People and digital technologies co-evolve. Sam has been saying a lot lately that it has to co-evolve to achieve general intelligence (AGI). The four elements of general intelligence are: emergence + agency + affordance + embodiment.

To sum up, we analyze the future from the fundamental trinity structure, and we can clearly see the inflection point we are facing today from the historical inflection point in the past. The essence is that the cost of the model changes from marginal to fixed, and one or even many great companies will be born. There is no doubt that OpenAI is in the lead.

Although it is a bit early, I personally think that OpenAI will definitely be bigger than Google in the future. It's just 1 times bigger, 5 times bigger or 10 times bigger.

OpenAI

This engine is basically a model system (model system), and its core is the model architecture Transformer, which is the sequence model (sequence model): sequence in, sequence out, encode, decode and decode only. But the final core is GPT, which is the Transformer after pre-training, which can highly compress information. Ilya has a belief: if you can compress information efficiently, you must have acquired knowledge, otherwise you cannot compress information. So, if you compress the information efficiently, you got to have some knowledge (you have to have some knowledge).

Ilya firmly believes in GPT3, 3.5, and of course GPT-4, which already has a world model in it. Although what you do is predict next word (predict the next keyword), this is just an optimization method. It has already expressed the information of the world, and it can continuously improve the model ability, especially in the sub- Generalization in concept space. Knowledge graphs really don't work. If any student makes a knowledge map, I will tell you seriously, you should not use a knowledge map. I have also been doing knowledge graphs for more than 20 years, just don't do that. Just pretty bad. It does not work at all. You should use Transformer. (Don't do that. It sucks. It doesn't work at all. You should use a Transformer.)

What's more important is to use enhanced learning, coupled with human feedback, to align with human values. Because GPT has been in use for more than 4 years, the knowledge has been encapsulated in it, and it was really impossible and difficult to use in the past.

The biggest one is alignment engineering, especially instruction following and natural language alignment. Of course, it can also be aligned with codes, tables, and charts.

It is very difficult to make a large model, and the most difficult thing is infra (infrastructure). When I was at Microsoft, each of our servers did not use a network card, but an FPGA. The bandwidth speed of network IO is infinite bandwidth technology (Infiniband), and the memory is directly accessed between servers. Why? Because Transformer is a density model, it is not only a matter of computing power, but also requires extremely high bandwidth. If you think that GPT-4 needs 24,000 to 25,000 cards for training, imagine how many people in the world can do this kind of system. All data and data center network architectures are different. It is not a three-tier architecture, it must be an east-west network architecture. So a lot of work to do here.

Tokens are very important. There may be 40-50 certain tokens in the world, which are language tokens and modals, and now there are more tokens. Of course, the parameters of more models are now miniaturized and localized, and the expertise in the task field can be integrated into these large models. Its maneuverability mainly depends on prompts and debugging, especially adjusting according to instructions, or aligning to debug, or in-context learning (context learning), which has been implemented relatively clearly. Its operability is getting stronger and stronger. Scalability is basically enough.

For the first time in history, GPT can reach 100 million active users in two months, and it cannot be stopped. Why?

① It encapsulates all knowledge in the world.

② It has strong enough learning and reasoning ability. GPT-3 ability is between high school students and college students. GPT-4 is not only admitted to Stanford, but also ranked very high in Stanford.

③ Its field is wide enough, its knowledge is deep enough, and it is easy to use. The biggest breakthrough in natural language is easy to use. The scalability is also good enough. Of course, it is still very expensive, like more than 20,000 cards, and it takes several months to train such a large project. But it’s not so ridiculously expensive—Google can do it, Microsoft can do it, several big Chinese companies can do it, and start-up companies can do it with financing.

Taken together, the critical point of the paradigm has arrived. The turning point has come.

A few words. I have been doing natural language for more than 20 years. The original natural language processing has 14 tasks. I can find out verbs, find out nouns, and analyze sentences clearly. Even if the analysis is clear, you know that this is an adjective, this is a verb, and this is a noun—then this noun is a pack of cigarettes? Or your uncle? Or a grave? Still a movie? No idea (do not know). What you need is knowledge. Natural language processing is never useful without knowledge.

The only way to make natural language work is you have knowledge (the only way to make natural language processing effective is that you have knowledge). It happens that Transformer compresses so much knowledge together, which is its biggest breakthrough.

The future is an era where models are everywhere

What OpenAI needs to do in the next 2-3 years is to make the model more sparse. Now it requires too much bandwidth. It needs to lengthen the attention window, or the function of recursion causality reasoning, including brainstorming and other work. Of course, there are some grounding things, including sub-symbols and sub-concepts. More modalities, more token space, more model stability, more latent space (e.g. Latent Space alignment), more computation, more infrastructure tools. 2-3 years are basically full. In other words, we probably know what is needed to continue to make this engine bigger.

Advice for Entrepreneurs

The internal structure of a startup is a combination of people and things. A person is the founder/founding team at the beginning; he has original intentions, internal driving force and external driving force; he can think independently and judge the future; he can be action-oriented and solve problems; he can be demand-oriented and find value; Communicate for resources. The next step is product market matching, this part is research and development technology, research and development of products, and delivery of products. The business model is to receive money, grow more, reach more customers, raise more money, and reach the value of the future. Organizationally, through system construction, develop future-oriented talents, organizational structure and cultural values, etc. It's all the sum of one company.

Our advice to every student is not to act rashly, but to think first.

• ① Don't be exaggerated, don't be too hot. Personally, I am most opposed to rubbing the heat. If you want to make a big model, you have to think about what you want to do. What is really going on in the big model has an essential relationship with which dimension or dimensions your entrepreneurial direction is in. Rushing is the worst behavior and will waste opportunities.

• ② Be diligent in learning at this stage. The new paradigm has multiple dimensions and considerable complexity. You need to read the papers you should read, especially now that the development is too fast and the uncertainty is great. My judgments all have a certain grayscale, and I can’t say that I can see them clearly, but this is roughly the result. Learning takes time and I highly recommend it.

• ③ Be action-oriented after thinking clearly, and take action decisively and in a planned manner. If this change has a structural impact on your industry, if you don’t advance, you will retreat. If you don't move forward, there is no way out, and today's position cannot be maintained. You can only take action if your industry is directly affected.

Each company is a collection of capabilities.

① In terms of product development capabilities, if your company focuses on software, there is no doubt that it will have an impact on you, and the long-term impact will be enormous. Especially if you are working on the C-end, the design of the user experience must have an impact, and you have to seriously consider what to do in the future today.

② If your company develops technology by itself, it will have local and indirect effects in the short term, it can help you think about technology design. The research and development of long-term core technology will also be affected. Today's chip design is a large number of tools, and in the future, large models will definitely affect chip development. Similarly, proteins are protein structural designs. No matter what you do, future technology will affect it. There is no direct impact in the short term, but it may have a significant impact in the long term.

③ The ability to meet demand. To meet demand basically requires reaching users, and the supply chain or operation and maintenance will definitely be affected. The operation and maintenance of software can be done for you with GPT, but the supply chain of hardware may not be. In the long run, there are opportunities for change, because the upstream and downstream structures will change. You have to judge whether your structure in this industry will change.

④ Exploration of business value, reaching users, financing, all of which can help you think and iterate.

about people and organization.

① Let’s talk about the founder first. Today, the founder has strong technical ability, which seems to be very powerful and important, but the future is really not important. Technology ChatGPT can help you in the future. As a founder, what is becoming more and more important and valuable is your will and heart. Willpower is a unique judgment and belief about the future, with persistence and strong tenacity. This is an increasingly important core competency for future founders.

② For start-up teams, tools can help explore directions, accelerate iterations of ideas, iterations of products, and even resource acquisition.

③ For the cultivation of future talents, on the one hand, learn tools, think and explore opportunities, and cultivate your own prompt engineer (prompt engineer) in a long-term and appropriate time.

④ Finally, when it comes to the construction of organizational culture, it is necessary to think more deeply, prepare early, and seize the opportunities of the times. Especially considering that there are already co-pilots for many functions, whether it is writing code or doing design, how can they be coordinated?

We are faced with the opportunity of such an era. It is both an opportunity and a challenge. We recommend that you think holistically about this opportunity.

Lesson: Don't be a Lost Generation

Looking back at history, only 2% of the early beneficiaries of the industrial revolution were Watt and Fulton, and the other 98% were only late beneficiaries, and there was a transition period of about 50-70 years. Those who are eliminated have no chance.

In the same way, why did the "Occupy Wall Street" incident break out in the United States? Because after the third industrial revolution, the lost generation thought they were the 98% who were eliminated, and the wealth was taken away by the 2%.

But another fact behind this is that 1% of the people in the United States pay 50% of the country's taxes. The money of the rich is not put in the safe, but invested and reproduced as a social resource.

The only difference is: who controls these resources and is responsible for these roles.

Everyone must deeply understand the necessity of becoming the 2% of people, there is no middle ground! Either be Occupy Wall Street, or be the 98%.

The Industrial Revolution is the biggest event in the world

Before the Industrial Revolution, there was no essential change in per capita GDP, whether in the East or the West. However, after the Industrial Revolution, GDP per capita increased by leaps and bounds. Throughout Europe, it increased by 50 times in 200 years; in China, it increased by more than 50 times in just 40 years. (In 1978, the per capita GDP was about 200 US dollars, almost expelled from the ball, and now it is more than 10,000 US dollars.) Therefore, the achievements of any princes and generals in ancient and modern China and abroad are not worth mentioning compared with the industrial revolution. The occurrence of the industrial revolution is the result of science promoting technology and then transforming it into productivity. This is an important manifestation of technology in economic and social life.

Not only that, but also the life span of a person.

Before liberation, China's average life expectancy was 39 years old. Now our life expectancy is 80 years old, which has tripled within 70 years since the founding of the People's Republic of China. This is a very remarkable thing.

When will human life expectancy roughly double? Basically, we have to go back to before farming began, about tens of thousands of years ago.

The average life expectancy has doubled in tens of thousands of years, and doubled again in just 70 years. This is thanks to the industrial revolution. Without the industrial revolution, there would be no such thing.

World civilization has various forces, and art is also a force, so why are science and technology so important? Because it can bring a superimposed progress.

What is superimposed progress? Today is 1, tomorrow is 2, the day after tomorrow is 4, and the day after tomorrow is 8. This is called superimposed progress.

reference

• https://mp.weixin.qq.com/s/_ZvyxRpgIA4L4pqfcQtPTQ
• https://www.sohu.com/a/466624040_121118995
• https://www.sohu.com/a/326413586_124422f
• https://www.sohu.com/a/326413586_124422