snowboy customizes wake-up words to realize voice wake-up [Voice Assistant]

background

The main goal of this series is to initially complete the basic functions of a smart speaker, including voice wake-up, speech recognition (speech to text), processing user requests (such as checking the weather, etc., mainly achieved through Rasa's own defined intent), speech synthesis (text to speech) )Function.

Speech recognition and speech synthesis are implemented offline.

environment

Operating system: Ubuntu 22.04.3 LTS

CPU：Intel® Core™ i5-1035G1 CPU @ 1.00GHz × 8

Memory: 16.0 GiB

Graphics card: not used, this series uses CPU solution

Reference article

snowboy github address (no longer updated): https://github.com/Kitt-AI/snowboy

After snowboy was not maintained, seasalt-ai was maintained for a period of time (mainly providing wake word training based on docker), so you can download its source code directly.

GitHub - seasalt-ai/snowboy: DNN based hotword and wake word detection toolkit (model generation included)

docker：Install Docker Desktop on Ubuntu | Docker Documentation

practice

necessary conditions

Can be installed via apt

# 安装之前先更新
apt-get upgrade
apt-get update

 python3.10 (comes with the system), python3-pip

# 参考安装python
apt-get install python3-pip

python -m pip install --upgrade pip

git, git-lfs, vim, gcc, g++ (version 11.4+), cmake (version 3.22+), swig...

If an error is reported during the installation process, there is a high probability that the installation package is still missing. Find the corresponding package and install it again and try again.

# 参考安装，安装之前可以先查一下本机之前安装过没有
apt install git

apt install git-lfs

apt install gcc

apt install g++

apt install cmake

apt install vim

apt install swig

apt install libatlas-base-dev

Audio related driver installation

apt-get install python3-pyaudio sox

pip install pyaudio

pip install scipy

Test microphone

rec t.wav

Download and install snowboy

mkdir /home/test

cd /home/test/

git clone https://github.com/seasalt-ai/snowboy.git

cd snowboy/swig/Python3/

make

After executing make, the library files of snowboy will be generated to implement the voice wake-up function.

ll

# Makefile
# snowboydetect.py
# _snowboydetect.so*
# snowboy-detect-swig.cc
# snowboy-detect-swig.i
# snowboy-detect-swig.o

Custom wake word

mkdir /home/test/snowboy/model

Wake word generation using wukong-robot

Snowboy Personal Wake Word

After generation, you only need to put the downloaded pmdl file in the model directory. There is no need to go through the following docker training wake-up words.

docker installation

Install Docker Engine on Ubuntu | Docker Documentation 

If you have installed docker before, you can skip it. If it is not installed on your machine, just follow the steps to install it.

Use docker to train custom wake words

Prepare the audio file for the wake word

cd /home/test/snowboy/model

rec -r 16000 -c 1 -b 16 -e signed-integer -t wav record1.wav

rec -r 16000 -c 1 -b 16 -e signed-integer -t wav record2.wav

rec -r 16000 -c 1 -b 16 -e signed-integer -t wav record3.wav

After recording, check whether there are any problems with the following sounds. If the command reports an error, you may need to install less packages or reopen the terminal.

Edit the Dockerfile, because I found that the official website is either slow or other errors are reported.

vim Dockerfile

FROM ubuntu:16.04

RUN apt update && apt --yes --force-yes install wget unzip build-essential python python-dev virtualenv portaudio19-dev

RUN wget https://bootstrap.pypa.io/pip/2.7/get-pip.py

RUN python2 get-pip.py

RUN wget https://github.com/seasalt-ai/snowboy/archive/master.zip && unzip master.zip
RUN cd snowboy-master/ && \
    virtualenv -p python2 venv/snowboy && \
    . venv/snowboy/bin/activate && \
    cd examples/Python && \
    pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

RUN apt -y remove wget unzip build-essential portaudio19-dev && apt -y autoremove && apt clean && rm -rf /var/lib/apt/lists/*

CMD cd snowboy-master/ && \
    . venv/snowboy/bin/activate && \
    cd examples/Python && \
    python generate_pmdl.py -r1=model/record1.wav -r2=model/record2.wav -r3=model/record3.wav -lang=en -n=model/hotword.pmdl

Save after editing, then build the image

docker build -t snowboy-pmdl .

After the build is completed, run the container to generate a custom wake word model.

docker run -it -v $(pwd)/model:/snowboy-master/examples/Python/model snowboy-pmdl

After executing the command, a hotword.pmdl model file will be generated in the model folder.

test

After generating the model test effect, shout the wake-up word and then hear the ding, it is considered completed.

cd home/test/snowboy/examples/Python3

python demo.py ../../model/hotword.pmdl