Following the release of GPT-4 this spring, OpenAI created another sleepless night.
In the past year, ChatGPT has definitely been the hottest word in the entire technology field.
At 02:00 a.m. on November 7, Beijing time, OpenAI’s first DevDay event officially started. The Keynote main forum session was lectured by Sam Altman and broadcast live on YouTube. Together with the on-site demonstration, it showed the release of multiple new products for a full 45 minutes. The content was compact and sincere. I watched this 45-minute keynote speech video in its entirety. The content is very rich. Let me help you sort out the information worth paying attention to. Highlight summary [all key points]:
-
GPT4 supports 128k contexts.
-
New GPT is only 1/3 the price.
-
Customized GPT & GPTs app store.
-
GPT4 opens up the networking mode, plug-in mode, code execution mode, and Dall-E model.
-
GPT adds log probability output.
-
Added json format support.
As the highly anticipated first developer conference, Altman brought us a series of new AI models and tools this time, including the following:
- New GPT-4 Turbo model
- More controllable output: function call enhancements, JSON schema
- Opening new APIs: DALLE-3, GPT-4 Turbo with vision, TTS and Whisper V3
- GPT-4 fine-tuning and customizing models
- GPTs: Create a customized version of ChatGPT
- GPT Store will be launched soon
- Assistants API: an experience closer to AI agents
1. GPT-4 Turbo officially released, supporting 128k context window
The biggest change of GPT-4 Turbo released today is that the knowledge base has been updated as of April 2023. Compared with previous versions that only included world knowledge before September 2021, GPT-4 Turbo has an updated knowledge base.
In addition, GPT-4 Turbo supports 128k context windows, which is equivalent to more than 300 pages of text. Altman also emphasized that after optimizing the model performance, the cost of using Token can be greatly reduced compared with GPT-4.
GPT-4 Turbo also incorporates JSON mode, which ensures that models will respond with valid JSON. The new API parameter response_format enables the model to limit its output to produce a syntactically correct JSON object. JSON mode allows developers to generate JSON in the API in the dialog window outside of function calls.
There have been rumors on the Internet that GPT-5 will be officially released at this developer day, but in the end only the arrival of GPT-4 Turbo was witnessed. Although it cannot eliminate developers' regret that GPT-5 cannot be released in time, we can also feel OpenAI's caution and restraint in major version releases.
2. GPT store is about to be launched: OpenAI’s ecological ambitions
The technology circle often compares the emergence of ChatGPT to major cross-generational events such as Apple's release of the iPhone. In the GPTs part of this conference, you can feel OpenAI’s ambition to build a larger ecosystem through model technology.
According to Sam Altman’s explanation, GPTs are customized versions of ChatGPT for specific purposes. You can create your own GPTs by relying entirely on natural language input without any coding experience. On-site OpenAI provides custom GPT examples: Canva and ZapierAI. Through very simple interaction, custom GPT generation can be achieved.
Currently, some customized GPTs already support ChatGPT Plus and enterprise user trials.
In addition to introducing the GPTs ecosystem in detail in the main forum, OpenAI has published a blog article on the official website to introduce in detail the characteristics and design concepts of GPTs. An official video released shows how to use a pet doctor GPT to solve the daily care problems of dogs.
At present, several companies such as Amgen, Bain, and Square have begun to use the customized GPT provided by GPT to enter the business, and it is expected to be fully open to API users and enterprise client users in the near future.
At the same time, later this month, OpenAI will launch a GPT store function, mainly used to share customized GPT assistants built by users.
3. Assistants API: Agent first step
The Assistants API is the first step in helping developers build Agents into their own programs, a purpose-built artificial intelligence product that has specific instructions, leverages additional knowledge, and can call models and tools to perform tasks. The new Assistants API provides new features like code interpreters and retrieval and function calls to handle a lot of the heavy lifting you previously had to do yourself and enable you to build high-quality AI applications.
Code Interpreter: Write and run Python code in a sandbox execution environment that can generate graphs and charts, and process files with different data and formats. It allows developers to run code iteratively to solve complex coding and mathematical problems and more.
Retrieval: Enhance the assistant with knowledge outside the model, such as proprietary domain data, product information, or user-provided documentation. This means developers do not need to compute and store document embeddings, or implement chunking and search algorithms. The Assistants API optimizes the retrieval techniques to use based on experience building knowledge retrieval in ChatGPT.
Function Call: Enables the assistant to call a function you define and incorporate the function response into its message.
You can currently try the Assistants API beta by going to the Assistants Playground.
Developers can try out the Assistants API beta without writing any code in the Assistants playground: https://platform.openai.com/playground?mode=assistant
4. Multi-modal capabilities continue to advance and blossom in an all-round way
Multimodality is a technology that the current model team is focusing on and developing. Developers can generate human-quality speech from text through the text-to-speech API.
Developers can integrate DALL·E 3 directly into their applications and products via the Image API and specify DALL·E-3 as a model. Companies such as Snap, Coca-Cola and Shutterstock already use DALL·E 3 to generate images and designs for their clients and campaigns.
Image
Similar to previous versions of DALL·E, the API includes built-in auditing capabilities to help developers protect their applications from abuse. Different formats and quality options are currently available, starting at $0.04 per generated image, check out the DALL·E 3 Getting Started Guide in the API.
GPT-4 Turbo with vision, developers can access it through gpt-4-vision-preview in the API. OpenAI plans to provide visual support for the main GPT-4 Turbo model, and the price depends on the size of the input image. For example, a 1080×1080 pixel image will cost $0.00765.
5. Other updates
Copyright Shield
OpenAI said that they are committed to protecting customers through the built-in copyright protection measures in the system, so they have launched a new copyright protection measure - Copyright Shield. If you face a legal claim regarding copyright infringement, they will step in to protect the customer and pay the resulting costs. This applies to generally available features of ChatGPT Enterprise and our developer platform.
Whisper v3 和 Consistency Decoder
Whisper is an open source speech-to-text model developed by OpenAI. At this developer conference, Altman officially announced a new version of Whisper large-v3, which is characterized by improved recognition performance in various languages. They also plan to support Whisper v3 in the API in the near future.
Content reference:
Press conference replay: https://www.youtube.com/watch?v=U9mJuUkhUzk
DevDay：https://openai.com/blog/new-models-and-developer-products-announced-at-devday
GPTs：https://openai.com/blog/introducing-gpts
Specially invited on-site observation team: Keven Liu
500px: https://500px.com/rongxinliu
Github: https://github.com/rongxin-liu
Note: The content comes from the Internet and will be deleted without infringement!