At today's Bund Conference sub-forum, Ant Group officially opened up its large code model CodeFuse.
According to reports, this is an exclusive large-scale code generation model developed by Ant. It can provide intelligent suggestions and real-time support based on developer input, helping developers automatically generate code, automatically add comments, automatically generate test cases, repair and optimize code, etc. to improve R&D efficiency.
CodeFuse is developed based on the large basic Ant model and is now available on Hugging Face: https://huggingface.co/codefuse-ai .
The content of this open source includes:
- MFT (Multi-Task Fine-tuning) framework, also known as MFTcoder ;
- Two data sets used to enhance the coding capabilities of LLMs , including Code Exercise and Evol-Instruction ;
- A faster and more reliable deployment framework based on FasterTransformer ;.
The resulting model collection includes CodeFuse-13B and CodeFuse-CodeLlama-34B , which support a variety of code-related tasks, such as code completion, text-to-code, unit test generation, etc.
It is worth mentioning that CodeFuse-CodeLlama-34B is based on CodeLlama as the basic model and uses the above-mentioned MFT framework for fine-tuning, achieving a high score of 74.4% (greedy decoding) in the HumanEval Python pass@1 evaluation , even exceeding GPT -4 (67%) performance .
According to the official introduction, CodeFuse’s mission is to develop large-scale code language models (Code LLMs) specifically designed to support the entire software development life cycle, covering key stages such as design, requirements, coding, testing, deployment, and operation and maintenance.