A few weeks ago, Microsoft Research released a paper titled Orca: Progressive Learning from Complex Explanation Traces of GPT-4. The paper describes the development of a new artificial intelligence model designed to solve challenging problems and explain its reasoning to humans. What I found most interesting about the paper was the description of case studies comparing the performance of four AI models in response to the problems they were asked to solve. Twelve of these experiments are described in the paper.
Orca, a model with 13 billion parameters, can handle diverse and challenging tasks and explain its reasoning to humans, the researchers said. It uses interpreted trajectories as learning material and leverages a massive collection called Flan 2022 (containing more than 1,000 tasks and 10,000 instructions) for complex tasks. In experiments described in the paper, Orca outperformed other open-source models in most domains, and matched GPT-4 on tasks such as natural language inference and image captioning. This may demonstrate the potential of explanation-based learning.
test
In tests, the researchers compared Orca to the following AI models:
Text-Davinci-003: It is a powerful model aimed at language tasks with better quality, longer output, and consistent instruction follow across multiple languages.
ChatGPT: The most powerful GPT-3.5 model and an improvement on text-davinci-003. It's optimized for chat and trained using conversations with humans.
GPT-4: The latest model in the GPT family, exhibiting human-level performance on various professional and academic benchmarks. It is optimized for chat and can perform more complex tasks.
Vicuna: An open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. It has become the leading open source language model.
Experimental results
Trigonometry Problem Solving - Known Right Triangles