Sapera Vision: Acquisition, Control, Image Processing and Artificial Intelligence (AI)

AI Inference Speed Improvement
• Sapera Processing now supports NVIDIA RTX 4000 series to speed up AI inference.
• Introducing new GPU Service to manage GPU parameters for optimal performance (i.e., favour speed over power saving).

AI Rotated Object Detection
• Inference engine helps Astrocyte object detection models with rotation.
• Rotation offered as an additional angle field in the output results of object detection.
• Innovative demo “CProAIRotatedObjectDetectionDemo” to illustrate new functionality.

AI Demos
• Substantial simplification of AI C++ demos to put importance on the inference code.
• Introduction of C# equivalent for most C++ AI demos.

RGB Packed (24-bit)
• New pixel format RGB888 (24-bit BGR packed) available.
• All handling functions for this new pixel format added to the CProImage class.
• Currently only supported by the AI inference tool.

Spinnaker
• All acquisition-enabled demos to support Spinnaker 4.0 (in addition to Sapera LT).
• Live acquisition and processing from GigE/USB via selection of Spinnaker or Sapera LT.
• Single demo code with plug-in architecture sitting on Spinnaker and Sapera LT.

Other General Improvements
• Improved management of AI cache (to be cleared on new installation).
• Added support for Dot Code standard in Quick Barcode tool.
• Improved Data Matrix grading speed via a new parameter to disable fine tuning of corners.
• All non-interactive demos ported from MFC to Console and Qt.
• New overload of BuildModel function for optimised inference engine initialisation.

Rotated Object Detection
• Object detection algorithm (YOLOX) now supports rotation.
• Rotation available at labelling (rotated bounding boxes) and inference (output angle).
• Example dataset “Hardware” adapted with rotated rectangle annotations.

New Segmentation Algorithm
• New algorithm (MANet) introduced in addition to the existing algorithm (DeepLabV3).
• New algorithm provides higher accuracy, faster training time and faster inference time.
• A new example dataset “Scratches” (locate thin scratches on brushed metal) is available.

Speed Improvement
• Now supporting NVIDIA RTX 4000 series to speed up training and inference.
• Training on multiple GPUs available (in automatic training mode) to reduce train time.
• Introducing new GPU Service to manage GPU parameters for optimal performance.
• Training time up to 30% faster with introduction of Mixed-Precision quantization.

Other General Improvements
• Various UI improvements to increase ease-of-use.
• Support of 24-bit RGB images at inference.
• Automatic training if now the default training mode.
• Function for merging two datasets available. Useful to delegate annotations to a group.
• Explicit image shuffle mechanism at dataset creation reducing variation in model accuracy between training sessions.
• Improved visualisation of results for the object detection inference.
• User defined threshold is now propagated to inter-tile annotation overlap threshold.
• Region size and tile size are now user editable when step size is different than one.
• Detection of already running version of cortex and logging more information at cortex start.
• Standardised image augmentation functions across model types.
• Added filtering of invalid segmentation annotations.
• Added a cortex shutdown option (to be used for troubleshooting only).

SIU-032-23
• Improved logs by selectively removing most of the GPU polling info that were polluting the logs.
• Allow overwriting of model name on model save.
• In classification model type the confidence scores are now displayed for all categories.
• All categories now displayed along with confidence scores in continual learning predictions.
• Added the build number with which a model has been trained to the model information.
• Display all training metrics on a single plot to reduce GUI clutter.
• Added a function to export labelled data as a CSV file.

Sapera Processing the inference class supports object detection models with and without rotation. There are new fields in the result class to access the angle and the coordinates of the rotated rectangle. A new example dataset and a new demo are available to show labelling, training and inference using rotated object detection.
In Astrocyte the current object detection algorithm (YOLOX) was adapted to support rotation. At labelling time bounding boxes can be drawn using the graphic tool and rotated to precisely fit the shape and orientation of the object. At training time, the algorithm is automatically selected (regular or rotation-enabled) depending on whether the labels are rotated or not). At inference time the predictions are output as rotated rectangles.

To ensure optimal and stable inference time Dalsa have introduced a new GPU Service to manage the parameters of the GPU. When your application is started the GPU driver is configured into the “performance mode” to favour optimal speed (the default mode being “power saving”). The initial mode is restored when application is closed. The new GPU Service is installed by both Astrocyte and Sapera Processing installers.

In addition to speed improvement at inference time the training can also run faster by using multiple GPUs. When using automatic training mode, you can select all GPUs available in the system such that the training operation is parallelised across GPUs.

Finally, in Sapera Vision Software Edition 2023-12, Dalsa introduces “Mixed-Precision” for quantization which uses a combination of 16-bit and 32-bit weights that allows accelerating the training time by up to 30%.