An attempt to fine-tune collection questions and answers using openai

1. Install openai dependencies

pip install --upgrade openai

2. Set openai api key

import os
 
# Note. alternatively you can set a temporary env variable like this:
os.environ["OPENAI_API_KEY"] = "sk-xxx"
if os.getenv("OPENAI_API_KEY") is not None:
    print("OPENAI_API_KEY is ready")
else:
    print("OPENAI_API_KEY environment variable not found")

3. Prepare training and verification set files, such as: train.json, val.json, which must contain the keywords "prompt" and "completion", that is, questions and answers.

4. Convert the json file into jsonl, that is, into row json format. During the conversion process, follow the recommended option Y to save fine-tuning costs.

openai tools fine_tunes.prepare_data -f train.json

 5. Submit fine-tuning training

 openai api fine_tunes.create -t train_prepared.jsonl -v val_prepared.jsonl -m davinci

6. Check whether the training is completed

openai api fine_tunes.follow -i ft-8Jj9rTcj2Ov0r9CZzoC7r2ag

 7. Verify the training effect

openai api completions.create -m davinci:ft-personal-2023-05-11-05-26-32 -p <YOUR_PROMPT>

 8. Conclusion

The effect has not reached the expiration date, and the result of the question output cannot be controlled.