Deploying to OAK Devices with Roboflow's Python Package

OAK: OpenCV AI Kit

The Luxonis OAK (OpenCV AI Kit) won an award as the 2022 Edge AI and Vision Product of the Year in the Cameras and Sensors category.

Follow along with our Quick Start Guide if you haven't yet trained a model with Roboflow Train.

If you'd like to access Accurate Train to train your model, contact our sales team today.

• Roboflow Train: Choosing Your Computer Vision Model Size

After training your custom model with Roboflow Train, you're ready to get set up for deployment!

Don't yet have an OAK device? Head over to the Luxonis & Roboflow OAK Store to purchase them with a 10% discount.

Setting up DepthAI and OpenVINO™ in your Python environment

We will need to set up DepthAI and OpenVINO™, Intel's open-source toolkit for optimizing and deploying AI inference, on our host system (i.e desktop computer, laptop, Raspberry Pi) and within a Python environment. This is required for the OAK device to operate (display the video feed) when connected to the host system.

1. macOS (M1 Chip) | Additional instructions are available here
2. macOS (Intel Chip)
3. Windows 10
4. Linux (Ubuntu)
5. Raspberry Pi
6. NVIDIA Jetson Nano/Xavier

Installing the Roboflow OAK Python pip Package

• roboflowoak (PyPi)

# enter the line below in your terminal within your chosen python environment
pip install roboflowoak

If you're using Anaconda, you will first need to install pip

# enter the line below in your terminal within your chosen python environment
conda install pip
# upon completion of installation, install the Roboflow OAK Python pip package
pip install roboflowoak

If you installed python with conda, don't forget to activate the python environment you installed the package to if you haven't already.

# change [replace-with-env-name] to the environment you installed roboflowoak to
# example: environment name is roboflowoak -- change [replace-with-env-name] to roboflowoak 
conda activate [replace-with-env-name]

Python Script for Running Inference with your Model

1. Copy/paste the script below into VSCode, XCode, PyCharm, Spyder (or another code editor)
2. Update the values for model/project [name], model version, api_key, and device_name within the "rf" object.
3. Save the python file to a directory - be sure to note the directory name and file name as we'll need these later for the deployment to work.

from roboflowoak import RoboflowOak
import cv2
import time
import numpy as np

if __name__ == '__main__':
    # instantiating an object (rf) with the RoboflowOak module
    rf = RoboflowOak(model="[YOUR-MODEL-ID]", confidence=0.05, overlap=0.5, version="[YOUR-MODEL-VERSION-#]", api_key="[YOUR-PRIVATE_API_KEY]", rgb=True, depth=True, device=None, device_name="[CHOOSE-A-DEVICE-NAME]", blocking=True)
    while True:
        t0 = time.time()
        result, frame, raw_frame, depth = rf.detect(visualize=True)
        predictions = result["predictions"]
        #{
        #    predictions:
        #    [ {
        #        x: (middle),
        #        y:(middle),
        #        width:
        #        height:
        #        depth: ###->
        #        confidence:
        #        class:
        #        mask: {
        #    ]
        #}
        #frame - frame after preprocs, with predictions
        #raw_frame - original frame from your OAK
        #depth - depth map for raw_frame, center-rectified to the center camera
    
        t = time.time()-t0
        print("INFERENCE TIME IN MS ", 1/t)
        print("PREDICTIONS ", [p.json() for p in predictions])
    
        # setting parameters for depth calculation
        max_depth = np.amax(depth)
        cv2.imshow("depth", depth/max_depth)
        # displaying the video feed as successive frames
        cv2.imshow("frame", frame)
    
        # how to close the OAK inference window / stop inference: CTRL+q or CTRL+c
        if cv2.waitKey(1) == ord('q'):
            break

Remember: Do not reveal your private API Key to anyone. You can revoke your API key to receive a new one if it is compromised.

Running Inference: Deployment

1. On your host device, re-open your terminal and change to the directory (within your terminal) in which you saved your Python script (file).
2. Connect your OAK device to the host system with a USB-C to USB-C or USB-C to USB-A cable (or with an ethernet cord if you're using an OAK PoE device).
3. Enter the code below (after replacing the placeholder text with the path to your Python script)

python3 [YOUR-PYTHON-FILE].py

Voilà! It works!

Face Detection model running on the OAK.

Where do we go from here? - Model Improvements

You may notice low confidence or false detections when you first begin using your model. Use Roboflow's Python package for help in implementing active learning to programmatically sample new images and increase the size of your dataset and improve your model. This will make quickly reducing the occurrence of false detections and improving the confidence level of detections a more seamless process.

• More on Active Learning: Roboflow's Python Package for Computer Vision