Deploying to OAK Devices with Roboflow's Python Package
OAK: OpenCV AI Kit
The Luxonis OAK (OpenCV AI Kit) won an award as the 2022 Edge AI and Vision Product of the Year in the Cameras and Sensors category.
Follow along with our Quick Start Guide if you haven't yet trained a model with Roboflow Train.
If you'd like to access Accurate Train to train your model, contact our sales team today.
- Roboflow Train: Choosing Your Computer Vision Model Size
After training your custom model with Roboflow Train, you're ready to get set up for deployment!
Don't yet have an OAK device? Head over to the Luxonis & Roboflow OAK Store to purchase them with a 10% discount.
Setting up DepthAI and OpenVINO™ in your Python environment
We will need to set up DepthAI and OpenVINO™, Intel's open-source toolkit for optimizing and deploying AI inference, on our host system (i.e desktop computer, laptop, Raspberry Pi) and within a Python environment. This is required for the OAK device to operate (display the video feed) when connected to the host system.
- macOS (M1 Chip) | Additional instructions are available here
- macOS (Intel Chip)
- Windows 10
- Linux (Ubuntu)
- Raspberry Pi
- NVIDIA Jetson Nano/Xavier
Since the OpenVINO™ 2022.1 release, the following development tools: Model Optimizer, Post-Training Optimization Tool, Model Downloader and other Open Model Zoo tools, Accuracy Checker, and Annotation Converter are not part of the installer. These tools are now only available on pypi.org. (source)
Installing the Roboflow OAK Python pip Package
# enter the line below in your terminal within your chosen python environment pip install roboflowoak
If you're using Anaconda, you will first need to install pip
# enter the line below in your terminal within your chosen python environment conda install pip # upon completion of installation, install the Roboflow OAK Python pip package pip install roboflowoak
If you installed python with conda, don't forget to activate the python environment you installed the package to if you haven't already.
# change [replace-with-env-name] to the environment you installed roboflowoak to # example: environment name is roboflowoak -- change [replace-with-env-name] to roboflowoak conda activate [replace-with-env-name]
Python Script for Running Inference with your Model
- Copy/paste the script below into VSCode, XCode, PyCharm, Spyder (or another code editor)
- Update the values for model/project [name], model version, api_key, and device_name within the "rf" object.
- Save the python file to a directory - be sure to note the directory name and file name as we'll need these later for the deployment to work.
from roboflowoak import RoboflowOak import cv2 import time import numpy as np if __name__ == '__main__': # instantiating an object (rf) with the RoboflowOak module rf = RoboflowOak(model="[YOUR-MODEL-ID]", confidence=0.05, overlap=0.5, version="[YOUR-MODEL-VERSION-#]", api_key="[YOUR-PRIVATE_API_KEY]", rgb=True, depth=True, device=None, device_name="[CHOOSE-A-DEVICE-NAME]", blocking=True) while True: t0 = time.time() result, frame, raw_frame, depth = rf.detect(visualize=True) predictions = result["predictions"] #{ # predictions: # [ { # x: (middle), # y:(middle), # width: # height: # depth: ###-> # confidence: # class: # mask: { # ] #} #frame - frame after preprocs, with predictions #raw_frame - original frame from your OAK #depth - depth map for raw_frame, center-rectified to the center camera t = time.time()-t0 print("INFERENCE TIME IN MS ", 1/t) print("PREDICTIONS ", [p.json() for p in predictions]) # setting parameters for depth calculation max_depth = np.amax(depth) cv2.imshow("depth", depth/max_depth) # displaying the video feed as successive frames cv2.imshow("frame", frame) # how to close the OAK inference window / stop inference: CTRL+q or CTRL+c if cv2.waitKey(1) == ord('q'): break
Remember: Do not reveal your private API Key to anyone. You can revoke your API key to receive a new one if it is compromised.
Running Inference: Deployment
- On your host device, re-open your terminal and change to the directory (within your terminal) in which you saved your Python script (file).
- Connect your OAK device to the host system with a USB-C to USB-C or USB-C to USB-A cable (or with an ethernet cord if you're using an OAK PoE device).
- Enter the code below (after replacing the placeholder text with the path to your Python script)
python3 [YOUR-PYTHON-FILE].py
Voilà! It works!
Face Detection model running on the OAK.
Where do we go from here? - Model Improvements
You may notice low confidence or false detections when you first begin using your model. Use Roboflow's Python package for help in implementing active learning to programmatically sample new images and increase the size of your dataset and improve your model. This will make quickly reducing the occurrence of false detections and improving the confidence level of detections a more seamless process.
- More on Active Learning: Roboflow's Python Package for Computer Vision