Machine Learning (1): Intelligent recognition of pet pedigree based on TensorFlow

Humans like to bring everything into the category of the contempt chain, and pets are no exception. Generally speaking, owning a purebred pet can make the owner occupy the cloud of the contempt chain, and then despise those hybrid or stray pets. A professional identification agency has even been developed that can issue a "Certificate of Pedigree". However, the conventional methods for the identification of various purebreds are exquisite: such as eye size, color, nose characteristics, body length, tail characteristics, hair, etc. Of course, it also includes some more mysterious characteristics: the personality, temperament and so on of the pet family. Aside from "black magic", since it is based on the identification of biological shape characteristics, the need to determine whether a purebred is an image recognition service is essentially an image recognition service.

Hello TensorFlow

Tensorflow is not a Machine Learning specific library, instead, is a general purpose computation library that represents computations with graphs.

TensorFlow open source software library (Apache 2.0 license), originally developed by the Google Brain team. TensorFlow provides a series of algorithm models and programming interfaces, allowing us to quickly build a machine learning-based intelligent service. For developers, there are currently four programming interfaces to choose from:

C++ source code: Tensorflow core is written based on C++ and supports operations at all levels from high to low;
Python bindings & Python library: Benchmark C++ implementation, support Python to call C++ functions;
Java bindings;
Go binding;

Here is a simple example:

Environmental preparation

Install the TensorFlow C library, including a header file c_api.h and libtensorflow.so

wget https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-1.5.0.tar.gz

## options
TF_TYPE="cpu" # Change to "gpu" for GPU support
TF_VERSION='1.5.0'
curl -L \
  "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-${TF_TYPE}-$(go env GOOS)-x86_64-${TF_VERSION}.tar.gz" |

Install the Go language environment, reference: Play with programming language: Golang
安装 Tensorflow Go binding library

go get github.com/tensorflow/tensorflow/tensorflow/go
go get github.com/tensorflow/tensorflow/tensorflow/go/op

Download the model (demo model), including a label file label_strings.txt and graph.pb

mkdir model
wget https://storage.googleapis.com/download.tensorflow.org/models/inception5h.zip -O model/inception.zip
unzip model/inception.zip -d model
chmod -R 777 model

Tensorflow Model Function

//Loading TensorFlow model
func loadModel() error {
  // Load inception model
  model, err := ioutil.ReadFile("./model/tensorflow_inception_graph.pb")
  if err != nil {
    return err
  }
  graph = tf.NewGraph()
  if err := graph.Import(model, ""); err != nil {
    return err
  }
  // Load labels
  labelsFile, err := os.Open("./model/imagenet_comp_graph_label_strings.txt")
  if err != nil {
    return err
  }
  defer labelsFile.Close()
  scanner := bufio.NewScanner(labelsFile)
  // Labels are separated by newlines
  for scanner.Scan() {
    labels = append(labels, scanner.Text())
  }
  if err := scanner.Err(); err != nil {
    return err
  }
  return nil
}

Classifying Workflow

The main process of image recognition based on the Tensorflow model is as follows:

Image conversion (Convert to tensor)
Image normalization ( Normalize )
Image classification ( Classifying )

func recognizeHandler(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
  // Read image
  imageFile, header, err := r.FormFile("image")
  // Will contain filename and extension
  imageName := strings.Split(header.Filename, ".")
  if err != nil {
    responseError(w, "Could not read image", http.StatusBadRequest)
    return
  }
  defer imageFile.Close()
  var imageBuffer bytes.Buffer
  // Copy image data to a buffer
  io.Copy(&imageBuffer, imageFile)

  // ...

  tensor, err := makeTensorFromImage(&imageBuffer, imageName[:1][0])
  if err != nil {
    responseError(w, "Invalid image", http.StatusBadRequest)
    return
  }

  // ...
}

函数 makeTensorFromImage() which runs an image tensor through the normalization graph.

func makeTensorFromImage(imageBuffer *bytes.Buffer, imageFormat string) (*tf.Tensor, error) {
  tensor, err := tf.NewTensor(imageBuffer.String())
  if err != nil {
    return nil, err
  }
  graph, input, output, err := makeTransformImageGraph(imageFormat)
  if err != nil {
    return nil, err
  }
  session, err := tf.NewSession(graph, nil)
  if err != nil {
    return nil, err
  }
  defer session.Close()
  normalized, err := session.Run(
    map[tf.Output]*tf.Tensor{input: tensor},
    []tf.Output{output},
    nil)
  if err != nil {
    return nil, err
  }
  return normalized[0], nil
}

The function maketransformimagegraph() adjusts the pixel value of the graph to 224x224 to meet the model input parameter requirements.

func makeTransformImageGraph(imageFormat string) (graph *tf.Graph, input, output tf.Output, err error) {
  const (
    H, W  = 224, 224
    Mean  = float32(117)
    Scale = float32(1)
  )
  s := op.NewScope()
  input = op.Placeholder(s, tf.String)
  // Decode PNG or JPEG
  var decode tf.Output
  if imageFormat == "png" {
    decode = op.DecodePng(s, input, op.DecodePngChannels(3))
  } else {
    decode = op.DecodeJpeg(s, input, op.DecodeJpegChannels(3))
  }
  // Div and Sub perform (value-Mean)/Scale for each pixel
  output = op.Div(s,
    op.Sub(s,
      // Resize to 224x224 with bilinear interpolation
      op.ResizeBilinear(s,
        // Create a batch containing a single image
        op.ExpandDims(s,
          // Use decoded pixel values
          op.Cast(s, decode, tf.Float),
          op.Const(s.SubScope("make_batch"), int32(0))),
        op.Const(s.SubScope("size"), []int32{H, W})),
      op.Const(s.SubScope("mean"), Mean)),
    op.Const(s.SubScope("scale"), Scale))
  graph, err = s.Finalize()
  return graph, input, output, err
}

Finally, input the formatted image tensor into the Inception model graph for operation.

session, err := tf.NewSession(graph, nil)
if err != nil {
  log.Fatal(err)
}
defer session.Close()
output, err := session.Run(
  map[tf.Output]*tf.Tensor{
    graph.Operation("input").Output(0): tensor,
  },
  []tf.Output{
    graph.Operation("output").Output(0),
  },
  nil)
if err != nil {
  responseError(w, "Could not run inference", http.StatusInternalServerError)
  return
}

Testing

func main() {
  if err := loadModel(); err != nil {
    log.Fatal(err)
    return
  }
  r := httprouter.New()
  r.POST("/recognize", recognizeHandler)
  err := http.ListenAndServe(":8080", r)
  if err != nil {
    log.Println(err)
    return
  }
}

Identifying Cases: Black Swans

$ curl localhost:8080/recognize -F 'image=@../data/IMG_3560.png'
{
  "filename":"IMG_3000.png",
  "labels":[
    {"label":"black swan","probability":0.98746836,"Percent":"98.75%"},
    {"label":"oystercatcher","probability":0.0040768473,"Percent":"0.41%"},
    {"label":"American coot","probability":0.002185003,"Percent":"0.22%"},
    {"label":"black stork","probability":0.0011524856,"Percent":"0.12%"},
    {"label":"redshank","probability":0.0010183558,"Percent":"0.10%"}]
}

From the above case, we can find that the estimated probability value of this service for black swan images is 98.75%, which is very accurate; but for the other two images of pet dogs, the highest estimated probability value is only about 30%, although It has not been recognized as a cat or a wolf, but it is still a long way from the desired usability (ignoring the complexity of the species itself for the time being). Mainly because we are still using a very "original" model. If we need to serve niche areas (pets, but also other things), we need to enhance optimization through training (Training Models), or introduce richer labels. , a more suitable model. Of course, there will also be poor sample quality in the training process, wrong samples and various noises will also affect the accuracy.