Deploying to OAK Devices with Roboflow's Python Package

OAK: OpenCV AI Kit

The Luxonis OAK (OpenCV AI Kit) won an award as the 2022 Edge AI and Vision Product of the Year in the Cameras and Sensors category.

Follow along with our Quick Start Guide if you haven't yet trained a model with Roboflow Train.

If you'd like to access Accurate Train to train your model, contact our sales team today.

• Roboflow Train: Choosing Your Computer Vision Model Size

After training your custom model with Roboflow Train, you're ready to get set up for deployment!

Don't yet have an OAK device? Head over to the Luxonis & Roboflow OAK Store to purchase them with a 10% discount.

Setting up DepthAI in your Python environment

We will need to set up DepthAI and OpenCV - In your Terminal, enter:

python3 -m pip install depthai opencv-python

Installing the Roboflow OAK Python pip Package

• roboflowoak (PyPi)

# enter the line below in your terminal within your chosen python environment
pip install roboflowoak==0.0.5
## uncomment and run the following line instead if you previously installed roboflowoak version 0.0.6
# pip install -lv roboflowoak==0.0.5

If you're using Anaconda, you will first need to install pip

# enter the line below in your terminal within your chosen python environment
conda install pip
# upon completion of installation, install the Roboflow OAK Python pip package
pip install roboflowoak==0.0.5

If you installed python with conda, don't forget to activate the python environment you installed the package to if you haven't already.

# change [replace-with-env-name] to the environment you installed roboflowoak to
# example: environment name is roboflowoak -- change [replace-with-env-name] to roboflowoak 
conda activate replace-with-env-name

Python Script for Running Inference with your Model

1. Copy/paste the script below into VSCode, XCode, PyCharm, Spyder (or another code editor)
2. Update the values for model/project [name], model version, api_key, and device_name within the "rf" object.
3. Save the python file to a directory - be sure to note the directory name and file name as we'll need these later for the deployment to work.

• NOTE: If you are deploying to an OAK device without Depth capabilities, set depth=False when instantiating (creating) the rf object. OAK's with Depth have a "D" attached to the model name, i.e OAK-D and OAK-D-Lite.
  • also be sure to comment out max_depth = np.amax(depth) and cv2.imshow("depth", depth/max_depth)

from roboflowoak import RoboflowOak
import cv2
import time
import numpy as np

if __name__ == '__main__':
    # instantiating an object (rf) with the RoboflowOak module
    rf = RoboflowOak(model="YOUR-MODEL-ID", confidence=0.05, overlap=0.5, version="YOUR-MODEL-VERSION-#", api_key="YOUR-PRIVATE_API_KEY", rgb=True, depth=True, device=None, device_name="CHOOSE-A-DEVICE-NAME", blocking=True)
    while True:
        t0 = time.time()
        result, frame, raw_frame, depth = rf.detect()
        predictions = result["predictions"]
        #{
        #    predictions:
        #    [ { 
        #        x: (middle),
        #        y:(middle),
        #        width: ,
        #        height: ,
        #        depth: ###->,
        #        confidence: ,
        #        class: ,
        #        mask: { }
        #       }
        #    ]
        #}
        # frame - frame after preprocs, with predictions
        # raw_frame - original frame from your OAK
        # depth - depth map for raw_frame, center-rectified to the center camera
        # To access specific values within "predictions" use:
        # p.json()[a] for p in predictions
        # set "a" to the index you are attempting to access
        # Example: accessing the "y"-value:
        # p.json()['y'] for p in predictions
    
        t = time.time()-t0
        print("INFERENCE TIME IN MS ", 1/t)
        print("PREDICTIONS ", [p.json() for p in predictions])
    
        # setting parameters for depth calculation
        # comment out the following 2 lines out if you're using an OAK without Depth
        max_depth = np.amax(depth)
        cv2.imshow("depth", depth/max_depth)
        # displaying the video feed as successive frames
        cv2.imshow("frame", frame)
    
        # how to close the OAK inference window / stop inference: CTRL+q or CTRL+c
        if cv2.waitKey(1) == ord('q'):
            break

Remember: Do not reveal your private API Key to anyone. You can  revoke your API key to receive a new one if it is compromised.

Running Inference: Deployment

1. On your host device, re-open your terminal and change to the directory (within your terminal) in which you saved your Python script (file).
2. Connect your OAK device to the host system with a USB-C to USB-C or USB-C to USB-A cable (or with an ethernet cord if you're using an OAK PoE device).
3. Enter the code below (after replacing the placeholder text with the path to your Python script)

python3 [YOUR-PYTHON-FILE].py

Voilà! It works!

Face Detection model running on the OAK.

Where do we go from here? - Model Improvements

You may notice low confidence or false detections when you first begin using your model. Use Roboflow's Python package for help in implementing active learning to programmatically sample new images and increase the size of your dataset and improve your model. This will help you to quickly reduce the occurrence of false detections and improve the confidence level of detections in a more seamless process.

• More on Active Learning: Roboflow's Python Package for Computer Vision