YoMo is a programming framework for assisting developers to easily build a distributed cloud system (Geo-Distributed Cloud System). YoMo's communication layer is built on the QUIC protocol, which brings high-speed data transmission and built-in "streaming function" of Streaming Serverless, which greatly improves the development experience of distributed cloud systems. The distributed cloud system built by YoMo provides an ultra-high-speed communication mechanism between near-field computing power and terminals, and has a wide range of application scenarios in Metaverse, VR/AR, IoT and other fields.
YoMo is written in Go language, and the Streaming Serverless part uses Golang's plugins and shared libraries to dynamically load user code, but it also brings some limitations to developers. Combined with the rigid need for isolation in serverless architectures, this makes WebAssembly an excellent choice for running user-defined functions.
For example, in the process of real-time AI inference in AR/VR and smart factories, cameras can send real-time unstructured data to computing nodes in near-field MEC (multi-access edge computing) devices through YoMo, and automatically execute managed AI inference function. When the AI inference is completed, YoMo sends the AI calculation results to the end device in real time.
The challenge for YoMo, however, is to consolidate and manage handler functions written by multiple external developers in edge computing nodes. This requires runtime isolation of these functions without sacrificing performance. Traditional software container solutions, such as Docker, are not up to the task because they are too heavy and slow to handle real-time tasks.
WebAssembly provides a lightweight, high-performance software container. It is very suitable as a runtime for YoMo data processing handler functions.
In this article, we'll show you how to create a Rust function for Tensorflow-based image recognition, compile it to WebAssembly, and run it as a streaming data handler using YoMo. We use WasmEdge as the WebAssembly runtime because WasmEdge provides the best performance and highest flexibility compared to other WebAssembly runtimes. WasmEdge is the only WebAssembly VM with stable support for Tensorflow. YoMo manages WasmEdge VM instances and WebAssembly bytecode applications inside containers through WasmEdge's Golang API .
GitHub source code: https://github.com/yomorun/yomo-wasmedge-tensorflow
Ready to work
Obviously, Golang needs to be installed. We assume you have it installed.
The Golang version needs to be newer than 1.15 for our example to work.
Also, the YoMo CLI application needs to be installed. It schedules and coordinates data flow and handler function calls.
$ go install github.com/yomorun/cli/yomo@latest
$ yomo version
YoMo CLI version: v0.0.5
Next, install the WasmEdge and Tensorflow shared libraries. WasmEdge is the leading WebAssembly runtime, hosted by CNCF. We'll use it to embed and run WebAssembly programs from YoMo.
# Install WasmEdge
$ wget https://github.com/second-state/WasmEdge-go/releases/download/v0.8.1/install_wasmedge.sh
$ chmod +x ./install_wasmedge.sh
$ sudo ./install_wasmedge.sh /usr/local
# Install WasmEdge Tensorflow extension
$ wget https://github.com/second-state/WasmEdge-go/releases/download/v0.8.1/install_wasmedge_tensorflow_deps.sh
$ wget https://github.com/second-state/WasmEdge-go/releases/download/v0.8.1/install_wasmedge_tensorflow.sh
$ chmod +x ./install_wasmedge_tensorflow_deps.sh
$ chmod +x ./install_wasmedge_tensorflow.sh
$ sudo ./install_wasmedge_tensorflow_deps.sh /usr/local
$ sudo ./install_wasmedge_tensorflow.sh /usr/local
# Install WasmEdge Images extension
$ wget https://github.com/second-state/WasmEdge-go/releases/download/v0.8.1/install_wasmedge_image_deps.sh
$ wget https://github.com/second-state/WasmEdge-go/releases/download/v0.8.1/install_wasmedge_image.sh
$ chmod +x ./install_wasmedge_image_deps.sh
$ chmod +x ./install_wasmedge_image.sh
$ sudo ./install_wasmedge_image_deps.sh /usr/local
$ sudo ./install_wasmedge_image.sh /usr/local
Finally, since our demo WebAssembly function is written in Rust, you will also need to install the Rust compiler and the rustwasmc toolchain.
Other parts of the demo can fork and clone the source code repo .
$ git clone https://github.com/yomorun/yomo-wasmedge-tensorflow.git
Image classification function
The image recognition functions that handle the YoMo image stream are written in Rust. It uses the WasmEdge Tensorflow API to process input images.
#[wasm_bindgen]
pub fn infer(image_data: &[u8]) -> String {
// Load the TFLite model and its meta data (the text label for each recognized object number)
let model_data: &[u8] = include_bytes!("lite-model_aiy_vision_classifier_food_V1_1.tflite");
let labels = include_str!("aiy_food_V1_labelmap.txt");
// Pre-process the image to a format that can be used by this model
let flat_img = wasmedge_tensorflow_interface::load_jpg_image_to_rgb8(image_data, 192, 192);
// Run the TFLite model using the WasmEdge Tensorflow API
let mut session = wasmedge_tensorflow_interface::Session::new(&model_data, wasmedge_tensorflow_interface::ModelType::TensorFlowLite);
session.add_input("input", &flat_img, &[1, 192, 192, 3])
.run();
let res_vec: Vec<u8> = session.get_output("MobilenetV1/Predictions/Softmax");
// Find the object index in res_vec that has the greatest probability
// Translate the probability into a confidence level
// Translate the object index into a label from the model meta data food_name
ret_str = format!(
"It {} a <a href='https://www.google.com/search?q={}'>{}</a> in the picture",
confidence, food_name, food_name
);
return ret_str;
}
You can compile this function to WebAssembly bytecode using the rustwasmc tool.
Here, we require Rust compiler version 1.50 or earlier in order for WebAssembly functions to work with WasmEdge's Golang API. Once the interface type specification is finalized and supported, we will catch up with the latest Rust compiler release .
$ rustup default 1.50.0
$ cd flow/rust_mobilenet_food
$ rustwasmc build `--enable-ext`
# The output WASM will be pkg/rust_mobilenet_food_lib_bg.wasm.
# Copy the wasm bytecode file to the flow/ directory
$ cp pkg/rust_mobilenet_food_lib_bg.wasm ../
Integrate with YoMo
On the YoMo side, we use the WasmEdge Golang API to start and run the WasmEdge virtual machine for the image recognition function. The app.go file is as follows in the source code project:
package main
... ...
var (
vm *wasmedge.VM
vmConf *wasmedge.Configure
counter uint64
)
func main() {
// Initialize WasmEdge's VM
initVM()
defer vm.Delete()
defer vmConf.Delete()
// Connect to Zipper service
cli, err := client.NewServerless("image-recognition").Connect("localhost", 9000)
if err != nil {
log.Print("❌ Connect to zipper failure: ", err)
return
}
defer cli.Close()
cli.Pipe(Handler)
}
// Handler process the data in the stream
func Handler(rxStream rx.RxStream) rx.RxStream {
stream := rxStream.
Subscribe(ImageDataKey).
OnObserve(decode).
Encode(0x11)
return stream
}
// decode Decode and perform image recognition
var decode = func(v []byte) (interface{}, error) {
// get image binary
p, _, _, err := y3.DecodePrimitivePacket(v)
if err != nil {
return nil, err
}
img := p.ToBytes()
// recognize the image
res, err := vm.ExecuteBindgen("infer", wasmedge.Bindgen_return_array, img)
return hash, nil
}
... ...
// initVM initialize WasmEdge's VM
func initVM() {
wasmedge.SetLogErrorLevel()
vmConf = wasmedge.NewConfigure(wasmedge.WASI)
vm = wasmedge.NewVMWithConfig(vmConf)
var wasi = vm.GetImportObject(wasmedge.WASI)
wasi.InitWasi(
os.Args[1:], /// The args
os.Environ(), /// The envs
[]string{".:."}, /// The mapping directories
[]string{}, /// The preopens will be empty
)
/// Register WasmEdge-tensorflow and WasmEdge-image
var tfobj = wasmedge.NewTensorflowImportObject()
var tfliteobj = wasmedge.NewTensorflowLiteImportObject()
vm.RegisterImport(tfobj)
vm.RegisterImport(tfliteobj)
var imgobj = wasmedge.NewImageImportObject()
vm.RegisterImport(imgobj)
/// Instantiate wasm
vm.LoadWasmFile("rust_mobilenet_food_lib_bg.wasm")
vm.Validate()
vm.Instantiate()
}
run
Finally, we fire up YoMo and see the entire data processing pipeline in action. Start the YoMo CLI application from the project folder. The yaml file defines the port on which YoMo should listen and the workflow handler that triggers incoming data. Note that the stream name image-recognitionmatches the data handler app.go mentioned above .
$ yomo serve -c ./zipper/workflow.yaml
Start the handler program by running the app.go program mentioned above.
$ cd flow
$ go run --tags "tensorflow image" app.go
Start a mock data source by sending data to YoMo . A video is a series of picture frames. The WasmEdge function in app.go will be called for each picture frame in the video.
# Download a video file
$ wget -P source 'https://github.com/yomorun/yomo-wasmedge-tensorflow/releases/download/v0.1.0/hot-dog.mp4'
# Stream the video to YoMo
$ go run ./source/main.go ./source/hot-dog.mp4
You can see the output of the WasmEdge handler function in the console. It will print the name of the object detected in each picture frame of the video.
Looking to the future
This article discusses how to use the WasmEdge Tensorflow API in the YoMo framework and the Golang SDK to process image streams in near real-time.
In partnership with YoMo, we will soon deploy WasmEdge in real production in smart factories for various assembly line tasks. WasmEdge is a software runtime for edge computing!