I have a lot of free time these days. I have been unemployed since August and have had a nightmare of job offers being revoked. I wanted to take a step back and work on different, interactive projects that could be used to solve everyday problems. NLP has always interested me. I've always wanted to use Python and my journey began. If you read my recent tutorial on building an NLP API, we used spaCy and FastAPI. In this tutorial, I want to show you how to make an awesome English to French translation app using a pretrained Transformer model from Hugging Face (specifically the Helsinki-NLP/opus-mt-en-fr model) ( citation) below). First, we can use python's input() function to create an awesome terminal application, then we'll use the Streamlit library to add a full UI, no front-end skills required! I'll break down all dependencies, imports, and code line by line to give you an idea of how the app works.
start
Before we start, we need to add some dependencies. I will demonstrate this using PyTorch (torch)
pip3 install transformers sentencepiece sacremoses torch
Dependency breakdown:
Transformers: where do we get our models and datasets from
Transformers provides APIs and tools to easily download and train state-of-the-art pretrained models. Using pretrained models can reduce computational costs, carbon footprint, and save time and resources required to train models from scratch. These models support common tasks in different ways,
sentence:
SentencePiece is an unsupervised text tokenizer and detokenizer