Lamini: Large language model fine-tuning framework

Lamini: Large language model fine-tuning framework

Fine-tuning is one of the most difficult parts of the large language model (LLM) development life cycle. This process is especially challenging if we are talking about techniques such as reinforcement learning with human feedback (RLHF), which require particularly complex workflows. Recently, I've seen a lot of new open source projects being born that try to simplify the fine-tuning process in LLM. One of the most concerned is undoubtedly Lamini .

What is Lamini

Lamini is a powerful tool that allows developers of all backgrounds to train high-performance LLMs, including models as good as ChatGPT, with a few lines of code from the Lamini library. Lamini includes optimizations beyond what is currently available to developers, ranging from complex RLHF to simple error mitigation. Lamini can also easily compare multiple base models, whether closed-source from OpenAI or open-source on HuggingFace, with just one line of code.

Lamini includes the following key capabilities:

• The Lamini library includes optimized prompt tuning and input and output, which can be tried directly on Lamini's playground.
• Fine-tuning and RLHF are possible with access to the advanced Lamini library in just a few lines of code.
• Managed Data Generators support building blocks for creating the data needed for training instructions to follow LLM.
• Follow LLM with instructions in just a few lines of code.

The process of using Lamini can be summarized as the following workflow.

1. Easy model tuning : The Lamini library provides an API that allows developers to easily tune models, including ChatGPT and other open source models. The library's API supports rapid tuning across different models, allowing developers to switch between OpenAI and open-source models with a single line of code. The Lamini library optimizes hints, allowing developers to leverage different models without worrying about how to format hints for each model.
2. Build large input-output pair datasets : Developing large input-output pair datasets is critical for training models. High-quality datasets help models learn how to follow instructions or respond in JSON format. Lamini publishes a repository that uses the Lamini library to generate 50,000 data samples from 100 data samples for consumption by the Lamini engine. This repository includes an open source dataset of 50,000 entries. How to generate the dataset is described in detail below.
3. Fine-tuning models on large datasets : In addition to the data generator, Lamini also released an LLM that uses the Lamini library to fine-tune on generated data. Developers can programmatically fine-tune their models by accessing this feature. Alternatively, fine-tune by calling OpenAI's fine-tuning API.
4. Perform RLHF : Lamini makes it easy for developers to run RLHF on fine-tuned models without requiring a large team of machine learning and human labeling experts.
5. Deploying the model to the cloud : Once the developer has fine-tuned the model, deploying it to the cloud is simple. Developers can click on the API in their product or feature to deploy their models.

Use Lamini

Lamini provides a simple programming model for fine-tuning the model, as shown in the following code:

class Animal(Type):
    name: str = Context("name of the animal")
    n_legs: int = Context("number of legs that animal has")

class Speed(Type):
    speed: float = Context("how fast something can run")

llama_animal = Animal(name="Larry", n_legs=4)
centipede_animal = Animal(name="Cici", n_legs=100)

my_data = [llama_animal, centipede_animal]

dog_animal = Animal(name="Nacho", n_legs=4)
dog_speed = Story(story="There once was a cute doggo named Nacho. She was a golden retriever who liked to run. All four of her paws were adorable.")

my_data.append([dog_animal, dog_speed])

Another interesting feature of Lamini is its support for batch processing, which can execute fine-tuning jobs as batches.

job = llm.submit_job(self, input, output_type, *args, **kwargs)

Furthermore, Lamini allows adding variants for output creation.

ad_copy = llm(input=aspects, output_type=AdCopy, random=True)

Or deduplicate data.

ad_copies = llm.sample(input=aspects, output_type=AdCopy, n=5)

data generator

One of the main components of the Lamini framework is the Lamini Data Generator, a powerful LLM pipeline designed to improve LLM performance by using a set of response datasets of more than 100 instructions to generate over 50,000 pairs of new instruction data.

This pipeline uses the Lamini library to call different but similar LLMs to generate different command and response pairs to train your LLM to follow commands better.

Lamini provides default settings for build pipelines using open source LLMs named Lamini Openand . Lamini InstructAs of the current release, the framework is using EleutherAI's Pythia for Lamini Opento generate further commands, and Databricks' Dolly for Lamini Instructto generate the corresponding responses for those commands.

Summarize

Switching LLMs with Lamini can be done with just a few lines of code.

Lamini is dedicated to solving one of the most difficult challenges in LLM-driven development. The framework provides a simple and consistent programming model to abstract the fine-tuning process across different LLMs. We will likely see Lamini incorporated into different LLM frameworks in the near future.