Baumer Industrial Camera How Baumer Industrial Camera combines NEOAPI SDK and OpenCV to acquire images and perform edge detection on images (C++)

​

Baumer industrial cameras

Baumer Industrial Cameras Baumer cameras are high-performance, high-quality industrial cameras that can be used in a variety of applications such as object detection, counting and recognition, motion analysis and image processing.

Baumer's 10GbE cameras have excellent image processing performance and can transmit high-resolution images in real time. In addition, the camera features fast data transfer, low power consumption, easy integration, and high scalability.
​Baumer
Industrial Camera NEOAPI SDK is the latest software development kit (SDK) for Baumer industrial cameras. It provides developers with a series of APIs and tools for communicating and controlling Baumer industrial cameras. The control method is extremely convenient, similar to Halcon's camera assistant control method. ​

OpenCV, full name Open Source Computer Vision Library, is an open source cross-platform computer vision and machine vision library. The library provides a wealth of image processing and computer vision algorithms, covering image processing, feature detection, target recognition, motion tracking and other fields, and is suitable for various platforms, including Windows, Linux, Android and iOS. OpenCV, initiated by Intel Corporation and open source, is a widely used tool in the field of computer vision.

Note: This article is based on Baumer's NEOAPI SDK and OpenCV using C++ language to implement camera edge detection on images.

Technical background of Baumer industrial camera using OpenCV for edge detection of images

When industrial cameras use OpenCV for image edge detection, the following technical background is involved:

1. Edge detection algorithm: OpenCV contains many classic edge detection algorithms, such as Sobel, Canny, Laplacian, etc. These algorithms can be used to identify edge areas in images and help extract important feature information in images.

2. Image gradient: The core idea of ​​edge detection is to use the gradient changes between pixels in the image to identify edges. The Sobel and Laplacian operators can be used to calculate horizontal and vertical gradients in images respectively, while the Canny algorithm combines multiple technologies to achieve more accurate edge detection.

3. Thresholding: In edge detection, it is very important to set appropriate thresholds to process gradient values. This helps filter out the effects of noise or non-edge areas, thereby improving edge detection accuracy.

4. Real-time and efficiency: Industrial cameras usually require image processing in real-time, so its computational complexity and real-time performance need to be considered when selecting an edge detection algorithm. OpenCV provides an edge detection algorithm optimized for performance, suitable for the needs of real-time applications of industrial cameras.

5. Application scenarios: Application scenarios for industrial cameras using OpenCV for edge detection include product quality inspection, positioning, measurement, etc. Through edge detection, the characteristics of the product surface can be effectively extracted and analyzed to achieve automated detection and analysis of the product.

To sum up, when industrial cameras use OpenCV for image edge detection, they need to have an in-depth understanding of edge detection algorithm principles, image gradient calculations, threshold processing and other technical background knowledge to achieve accurate extraction and analysis of edge areas in images.

Use OpenCV to build edge detection function in NEOAPI SDK

After the camera is connected, you can use OpenCV in the NEOAPI SDK to establish an edge detection function. The C++ calling code is as follows:

#include <iostream>
#include <thread>
#include <vector>
#include <map>
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/core/ocl.hpp"
#include "neoapi/neoapi.hpp"


class EdgeDetector {
    
    
 public:
    explicit EdgeDetector(NeoAPI::NeoString serialnumber) {
    
    
        camera_.Connect(serialnumber);
        camera_.f().ExposureTime.Set(2500);
        // cv mats will created by width and height -> there is no space for chunk -> disable chunk
        camera_.DisableChunk();
        try {
    
    
            camera_.f().PixelFormat.Set(NeoAPI::PixelFormat::BayerRG8);
        } catch (NeoAPI::FeatureAccessException&) {
    
    
            camera_.f().PixelFormat.Set(NeoAPI::PixelFormat::Mono8);
        }
        pixel_format_ = camera_.f().PixelFormat.Get();
        identifier_ = serialnumber;
    }

    ~EdgeDetector() {
    
    
        Stop();
        FreeCamBuffers();
    }

    // setups the edge detector to do processing with the requested type
    void Setup(MemoryMode memtype) {
    
    
        cv::ocl::setUseOpenCL(MemoryMode::cpu != memtype);
        if (cv::ocl::Device::getDefault().hostUnifiedMemory()) {
    
      // do not use svm functions if this failes
            try {
    
    
                cv::ocl::Context::getDefault().setUseSVM(MemoryMode::shared == memtype);
            }
            catch(...) {
    
    
                memtype = MemoryMode::cpu;
                std::cout << "SVM Error: falling back to cpu memory!" << std::endl;
            }
        }

        SetupBuffers(3, memtype);
        camera_.SetUserBufferMode();
    }

    // single edge detection on a given image
    void Detect(const NeoAPI::Image& image, bool show_image) {
    
    
        cv::UMat *img_mat = &(image.GetUserBuffer<CamBuffer*>()->gpu_mat_);
        if (NeoAPI::PixelFormat::BayerRG8 == pixel_format_) {
    
    
            cv::cvtColor(*img_mat, grey_mat_, cv::COLOR_BayerRG2GRAY);
        } else {
    
    
            grey_mat_ = *img_mat;
        }
        cv::GaussianBlur(grey_mat_, gauss_mat_, cv::Size(5, 5), 0);
        cv::Sobel(gauss_mat_, sobel_mat_, sobel_mat_.depth(), 1, 1, 5);
        if (show_image) {
    
    
            cv::imshow(identifier_, sobel_mat_);
            cv::pollKey();
        }
        ++frames_;
    }

    // returns the number of processed images since last call
    size_t ProcessedFrames() {
    
    
        size_t ret = frames_;
        frames_ = 0;
        return ret;
    }

    // return the cameras serial number
    const cv::String& GetIdentifier() {
    
    
        return identifier_;
    }

    // starts a seperate thread that will do edge detection continouosly
    void Start(bool show_images) {
    
    
        run_ = true;
        detect_thread_ = std::thread(&EdgeDetector::Detect_, this, show_images);
    }

    // stops a previous started continouosly edge detection
    void Stop() {
    
    
        run_ = false;
        if (detect_thread_.joinable()) {
    
    
            detect_thread_.join();
        }
    }

 private:
    void FreeCamBuffers() {
    
    
        while (!buffers_.empty()) {
    
    
            delete buffers_.back();
            buffers_.pop_back();
        }
    }

    void SetupBuffers(size_t count, MemoryMode memtype) {
    
    
        int width = static_cast<int>(camera_.f().Width.Get());
        int height = static_cast<int>(camera_.f().Height.Get());
        FreeCamBuffers();
        for (size_t i = 0; i < count; ++i) {
    
    
            buffers_.push_back(new CamBuffer(width, height, memtype));
            camera_.AddUserBuffer(buffers_.back());
        }
        // allocate processing matrices because operations cannot run in place
        // some opencv versions use the wrong constructor -> create the mats objects with explicit memory type
        grey_mat_ = cv::UMat();
        gauss_mat_ = cv::UMat();
        sobel_mat_ = cv::UMat();
        grey_mat_.create(cv::Size(width, height), CV_8UC1, static_cast<cv::UMatUsageFlags>(memtype));
        gauss_mat_.create(cv::Size(width, height), CV_8UC1, static_cast<cv::UMatUsageFlags>(memtype));
        sobel_mat_.create(cv::Size(width, height), CV_8UC1, static_cast<cv::UMatUsageFlags>(memtype));
    }

    void Detect_(bool show_images) {
    
    
        try {
    
    
            while (run_) {
    
    
                NeoAPI::Image image = camera_.GetImage();
                if (image.IsEmpty()) {
    
    
                    std::cout << identifier_ << " Error during acquisition!" << std::endl;
                    break;
                } else {
    
    
                    Detect(image, show_images);
                }
            }
            if (show_images) {
    
    
                cv::destroyWindow(identifier_);
            }
        } catch (NeoAPI::NeoException& exc) {
    
    
            std::cout << identifier_ << " error: " << exc.GetDescription() << std::endl;
        } catch (cv::Exception& exc) {
    
    
            std::cout << identifier_ <<  "cv error:" << exc.msg << std::endl;
        }
    }

    NeoAPI::Cam camera_;
    std::vector<CamBuffer*> buffers_;
    cv::String identifier_;
    cv::UMat grey_mat_;
    cv::UMat gauss_mat_;
    cv::UMat sobel_mat_;
    std::thread detect_thread_;
    size_t frames_ {
    
    0};
    NeoAPI::PixelFormat pixel_format_;
    bool run_ {
    
    false};
};

Use the edge detection function in NEOAPI SDK to detect edges on images

After the camera is connected, you can use the previously designed function in the NEOAPI SDK to perform edge detection on the image. The C++ calling code is as follows:

#include <iostream>
#include <thread>
#include <vector>
#include <map>
#include "opencv2/imgproc.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/core/ocl.hpp"
#include "neoapi/neoapi.hpp"


void PrintMetrics(const std::vector<EdgeDetector*>& devices, size_t duration) {
    
    
    for (size_t secs = 0; secs < duration; ++secs) {
    
    
        std::this_thread::sleep_for(std::chrono::milliseconds(1000));  // print every second metrics
        for (auto device : devices) {
    
    
            std::cout << device->GetIdentifier() << " fps: " << device->ProcessedFrames() << std::endl;
        }
    }
}

void FindDevices(std::vector<EdgeDetector*> *devices) {
    
    
    for (auto device : NeoAPI::CamInfoList::Get()) {
    
    
        try {
    
    
            devices->push_back(new EdgeDetector(device.GetSerialNumber()));
        }
        catch (NeoAPI::NeoException& exc) {
    
    
            std::cout << exc.GetDescription() << std::endl;
        }
    }
    std::cout << devices->size() << " device(s) connected!" << std::endl;
}

void GetGpuCapabilities(std::map<MemoryMode, std::string> *memtypes) {
    
    
    (*memtypes)[MemoryMode::cpu] = "cpu";
    if (cv::ocl::haveOpenCL()) {
    
    
        (*memtypes)[MemoryMode::gpu] = "gpu";
        if (cv::ocl::Device::getDefault().hostUnifiedMemory()) {
    
    
            (*memtypes)[MemoryMode::shared] = "gpu with shared memory";
        }
    }
}

void RunDetection(const std::vector<EdgeDetector*> &devices,
                  const std::map<MemoryMode, std::string> &memtypes,
                  bool show_images) {
    
    
    if (devices.size()) {
    
    
        for (auto memtype : memtypes) {
    
    
            std::cout << "Next run will be processed on " << memtype.second << std::endl;
            for (auto device : devices) {
    
    
                device->Setup(memtype.first);
            }

            for (auto device : devices) {
    
    
                device->Start(show_images);
            }

            // run the detection for given time in seconds and print status informations
            PrintMetrics(devices, 5);

            for (auto device : devices) {
    
    
                device->Stop();
            }
        }
    }
}

void FreeDevices(std::vector<EdgeDetector*> *devices) {
    
    
    while (devices->size()) {
    
    
        delete devices->back();
        devices->pop_back();
    }
}

int main(int argc, char *argv[]) {
    
    
    /* Showing the images have a high impact on processing speed.
       For better comparision show_images should be disabled. */
    bool show_images = ((argc > 1) && argv);
    std::map<MemoryMode, std::string> memtypes;
    std::vector<EdgeDetector*> devices;

    // look if the gpu supports opencl and shared memory
    GetGpuCapabilities(&memtypes);

    // find all connected cameras
    FindDevices(&devices);

    // edge detection processing on all connected cameras
    RunDetection(devices, memtypes, show_images);

    // cleanup
    FreeDevices(&devices);

    return 0;
}

Advantages of industrial camera edge detection on images through OpenCV

Industrial cameras using OpenCV for image edge detection have multiple advantages, including but not limited to the following:

1. Rich algorithm support: OpenCV provides a variety of classic edge detection algorithms, such as Sobel, Canny, Laplacian, etc., which are suitable for edge recognition needs in different scenarios. This means that industrial cameras can select the most suitable algorithm for a specific application to obtain the best edge detection results.

2. Efficient real-time performance: OpenCV optimizes multiple edge detection algorithms to perform well in real-time processing scenarios of industrial cameras. This means that industrial cameras can quickly and accurately perform edge detection on images, which is suitable for real-time monitoring, automated inspection and other applications.

3. Flexible parameter settings: OpenCV's edge detection algorithm usually has adjustable parameters, such as thresholds, convolution kernel size, etc., allowing engineers to optimize parameters according to specific scenarios and needs to obtain the best edge detection results.

4. Cross-platform and ease of use: OpenCV is a cross-platform open source library that can run on multiple operating systems and provides easy-to-use APIs and documentation. This facilitates the development and deployment of industrial cameras, and also provides support for algorithm debugging and optimization.

5. Combined with other image processing functions: OpenCV not only provides edge detection algorithms, but also contains rich image processing functions, such as filtering, morphological operations, etc. Industrial cameras can combine these features with edge detection to achieve more complex image processing tasks.

In summary, industrial cameras use OpenCV for image edge detection with multiple advantages such as rich algorithm support, efficient real-time performance, flexible parameter settings, cross-platform and ease of use, and combination with other image processing functions. It provides industrial detection and Automated production and other fields provide powerful image processing solutions.

Industrial cameras use OpenCV to perform edge detection on images.

Industrial cameras use OpenCV to perform edge detection on images and have a wide range of applications in various industries, including but not limited to the following aspects:

1. Manufacturing industry: Industrial cameras combined with OpenCV's edge detection function can be used in product quality inspection, surface defect detection, dimensional measurement and other fields. Through edge detection, features and defects on the product surface can be quickly and accurately identified to achieve automated quality control.

2. Intelligent manufacturing: In the field of intelligent manufacturing, edge detection by industrial cameras through OpenCV can be used in visual navigation, automated assembly and other scenarios to help robots and automated production lines achieve precise positioning and operation.

3. Medical image analysis: The combination of industrial cameras and OpenCV can be used for edge feature extraction and analysis of medical images to assist doctors in disease diagnosis and treatment.

4. Intelligent transportation: In the field of intelligent transportation, the edge detection implemented by industrial cameras through OpenCV can be used for vehicle identification, pedestrian detection, traffic monitoring and other tasks to improve the intelligence and safety of the transportation system.

5. Agricultural field: Industrial cameras combined with OpenCV's edge detection are used in agricultural fields such as fruit and vegetable quality inspection, pest and disease detection, and crop growth monitoring.

6. Security monitoring: Industrial cameras can use OpenCV for edge detection in the field of security monitoring, such as face recognition, behavior analysis, etc., to help improve the accuracy and efficiency of the monitoring system.

In summary, industrial cameras using OpenCV to perform edge detection on images have important application value in many industries such as manufacturing, intelligent manufacturing, medical care, intelligent transportation, agriculture, and security monitoring, providing efficient images for these fields. Process and analyze solutions.