iDS NXT Ocean: A simplified Artificial Intelligence workflow experience

In this, the fourth and final editorial, we will explain how to upload an inference model to your iDS NXT camera to use for a real-life application.

The first step is to open the NXT cockpit application on your local PC, then by clicking "connect" we select our camera and log in. Once logged in, you can now navigate to the configuration tab and click the app manager on the toolbar. As we have already built both a classification and object detection neural network in this series of editorials, we can choose either for this last step: choose either classifier or object detector vision app from the app manager.

For the purpose of this example, click the classifier app and set the status to running. You will now see the classifier V-app button on the toolbar. The next step is to click on the classifier V-app button as it opens the V-app configuration page.

Camera settings are found in the camera tab and the V-app section has some new controls and functionality. You should now upload the classifier neural network to the camera. After this, click "install CNN." The next step is to navigate to the location where the CNN was downloaded from the web-based application, NXT Lighthouse. When here, select the file you wish to use and click open. After a few seconds a message underneath the main image window will state that the file was successfully uploaded.

Click on the CNN tab drop down menu, and under configurables you can select the newly loaded CNN. If the camera is in free run mode (or in triggered mode and receiving triggers) you will see images being captured and the camera immediately starts classifying. The results of each classification will appear underneath the image. Regions of Interest can be used to focus the CNN on a specific part of the image rather than classifying the whole image. This should improve accuracy. The ROI tool can be used to set up an ROI. By clicking add current ROI you can add multiple Regions of Interest.

Then click on the results thumbnail at the bottom of the screen. The ROI results are revealed along with a confidence score (see image) on the target objects classification. The results text under the main image window will also provide a count for each class recognized.

To run the object detection CNN, you must disable the classifier V-app by setting the status to "not running" and set the status for the object Detector V-app to "running."

Now click the Object Detector icon on the left toolbar to open the object detector page. Under the V-app section now click install CNN, navigate to the CNN file and click "open." Once cockpit reports that the file was successfully uploaded you can select the CNN from the list under configurables and immediately the object detection process begins. By clicking on the ROI thumbnail at the bottom of the screen we can see the object detection results with a bounding box around each object detected along with a confidence score. Under configurables it is possible to set a detection threshold to help filter out any false positive cases, or you could lower the confidence threshold to help identify objects that are difficult to detect.