Translator: Mingming Ruyue
The cost of artificial intelligence
While I firmly believe that open source AI will continue to thrive, I also think that in the coming years companies and governments will have an increasing incentive to limit the adoption of new neural network architectures and techniques. This article aims to anticipate and outline the potentially disruptive changes that could occur this decade, and to propose some ideas and solutions to help the open source community adapt to this challenge of assumptions about the future. Predicting the future is very difficult, and many of the predictions I make may not come true, or may not turn out the way I envisioned them. However, that doesn't mean thinking about the future is useless. Instead, thinking about possible futures can help us actively adapt to a changing world. As the world continues to evolve at breakneck speed, the ability to envision and prepare for possible new worlds will become increasingly important.
In the near future, we will witness fundamental changes in the way we interact with people, the way labor is exchanged, and even the way society is organized. Personalized AI entities (called "Ghosts") promise to become potentially personalized beings for each person and connect us into a network of other AI systems around the world. These AI systems will provide us with multiple services, and we can think of them as tools to expand our cognition, not just assistants. Businesses and organizations will likely also have their own "Ghosts" to improve collaboration among members. In addition to the social aspect, associative memory networks with recurrent connections may enable AI systems to have memory capabilities. These individual "Ghosts" may even form unique identities. Additionally, AI systems utilizing consensus algorithms may emerge, leading to the development of decentralized autonomous AI. While this has yet to materialize, we can already envision some upcoming economic changes and trends.
The exchange value of AI-generated services will depend on the energy required to provide the service (i.e. the cost of running the relevant models), coupled with the information asymmetry of AI within the market. Those services that are easily copied by AI will have a lower exchange value, resulting in less residual value for the AI owner. This trend will have a large impact on service-based economies, where a substantial reduction in the residual value of most services can be expected. We can therefore expect, for some time, that populations and governments in many Western countries will react in similar ways to the way textile workers opposed machine production in the nineteenth century. With diminishing returns across industries, many parts of the world will adopt monopoly licensing strategies and authoritarian rules to limit access to AI.
In those regions where AI is more aggressively advanced, open-source AI is likely to thrive in services with low exchange value but high use value, such as those that are easily replaced by AI. However, the economic incentives will be different for services where small improvements can substantially increase the value of use. In this type of service, we can expect the trend of "the strong get stronger". More advanced AI systems will be able to create more residual value. As a result, there may be less incentive to innovate to share techniques and model architectures. But it is important to note that residual value can only be obtained in the system when there is an asymmetry between the system participants. In the context of AI, this asymmetry may take the form of information asymmetry, which involves controlling and limiting others' access to information and knowledge.
Information asymmetries among some actors may lead governments to impose restrictions, such as imposing intellectual property protection, licensing regimes, or access control measures; such information asymmetries require political action, not technical means alone. However, technology can address the information asymmetry caused by resource asymmetry. Specifically, current artificial neural networks are usually trained in a dense manner, i.e., all units within the network are activated once an input is provided. For an architecture like Transformer (used in services like ChatGPT), the computational cost of information propagation increases significantly due to its "self-attention mechanism". High computational complexity leads to high energy consumption.
Considering the resource-intensive training of large-scale language models (LLMs), we can foresee that the development and management of AI systems will be concentrated in a small number of closed-source entities in the case of “the strongest get stronger”. These entities will be encouraged to treat their model weights and architecture as proprietary, as more confidentiality will lead to higher profits. Unfortunately, resource constraints also mean that for smaller entities such as researchers, nonprofits, or startups, it is often not feasible to train large language models from scratch because of the prohibitive energy costs involved. Consequently, most open-source LLM efforts rely on fine-tuning pre-existing models, which is more economical and energy-efficient. Based on these trends, I believe we should prioritize reducing the cost of collectively training and running large-scale deep learning models in order to maintain the competitiveness and quality of open source AI.
Sparse Activation Tensor && Cryptographic Ghost Proof
This part contains a lot of formulas, and the CSDN display effect is not good, see the CSDN public account:
https://mp.weixin.qq.com/s/oeKsd-gLyD2_prsmwzOT-g
to reflect
In addition to openness and trustlessness, integrating CGPs has the potential to bring together the fields of artificial intelligence and decentralized ledger technology, paving the way for the emergence of autonomous AI systems . Autonomous AI System is a permissionless peer-to-peer AI protocol utilizing consensus algorithms. Rather than changing entries in the ledger, these agreements spread ideas. To ensure efficient operation and prevent denial of service (DoS) attacks, these protocols may rely on mutual credit or currency. The technical details as well as the economic and social implications of these autonomous systems will be explored in future research.
As engineers and AI researchers, we must recognize the inherently political nature of technology. Seemingly small engineering decisions can lead to profound societal shifts. For example, deep learning models with dense activations may lead to centralized forms of social organization, while deep learning models with sparse activations may lead to decentralized forms of social organization. Interdisciplinary thinking is needed now more than ever. For example, it is worth considering the impact of decentralized AI on governance. How will AI systems affect the nation? Will they empower authoritarian states, or help develop stronger democracies? What about the company's organizational structure and minimum viable size? It is very interesting to imagine the possibility of creating scalable direct democracies by connecting each citizen's personalized AI entity (Ghost) to a shared network. These ideas are worthy of further exploration in the future.
I hope this post has piqued your interest in exploring the synergies between AI and peer-to-peer technologies. I firmly believe that the next few years will see significant progress at the intersection of AI and database systems, especially in the field of sparse activation tensors. Furthermore, the fusion of artificial intelligence and peer-to-peer systems, as well as the development of sparsely activated associative memory networks, will undoubtedly lead to significant advances. I encourage you to explore these fascinating topics and contribute to open source AI. Remember, the Free/Open Source Software (FLOSS) movement is not just about code sharing, it's about empowering the global community.
Original: https://blog.opncbr.com/post/open_source_ai/