A Guide for Model Production

This documentation is intended to aggregate resources from Roboflow's blog, YouTube channel, and other information from our team, to promote the success of all Roboflow users..

Written by Mohamed Traore

Last published at: May 4th, 2022

Key Metrics


mAP, Precision and Recall scores are provided for all models trained with Roboflow Train, and through the notebooks in the Roboflow Model Zoo.

mAP

Precision

  • A measure of how precise a model is at prediction time. True positives are divided by all positives that have been guessed.

Recall

  • A measure of performance for a prediction system. Recall is used to assess whether a prediction system is guessing enough. True positives are divided by all possible true positives.

Prediction

  • An attempt by a model to replicate the ground truth. A prediction usually contains a confidence value for each class.
  • Formats available for importing and exporting on Roboflow

Key Tools for Dataset Quality Control

Dataset Health Check
Dataset Health Check, Guided Walkthrough

"Garbage in, garbage out."
This old machine learning adage conveys a salient machine learning point: unless input data is of high quality, model accuracy — even with the best computer vision architectures — will suffer. Understanding what preprocessing and augmentation are at their core enables data scientists to get the most out of their input data.
Preprocessing

Preprocessing steps are image changes that are done to all images, like resizing or converting to grayscale. Preprocessing steps will be applied to training, validation, and testing sets.

Augmentation

Image augmentation steps are image changes designed only to increase dataset size for better performance, like randomly altering brightness or rotation. Image augmentation steps will be applied only to training images.

Bounding Box Level Augmentation

level augmentation generates new training data by only altering the content of a source image’s bounding boxes. In doing so, developers have greater control over creating training data that is more suitable to their problem’s conditions.

Learning Data Augmentation Strategies for Object Detection - Google Research Team (2019)

One-Click Model Training (Roboflow Train)

Introduction (Video)

Third-Party Training Integrations (Pro)
AWS, Azure Custom Vision, Google Cloud Vision AutoML
  • AWS & Azure Custom Vision require Third-Party keys to be entered into your project's workspace. These access points must be set up on your target AWS or Azure account to enable integration with Roboflow.

Zero Shot Object Tracking

Model Production Tips

Choosing the Right Problem Statement (Roboflow Blog)

Characteristics: Specific, Achievable, Measurable
  • What will our model experience "in the wild…?"
Through the dataset assembly process, our laser focus might miss a whole host of edge cases that we would naturally consider out-of-scope for our model. While out-of-scope instances are intuitive to us, our model has no way of knowing anything beyond the scope of what it has been shown.

Augmentations for Aerial Datasets (Roboflow's Blog)

Small  (Roboflow's Blog) | Video
 

Active Learning Tips (Roboflow's Blog) | Video

A Primer on Transfer Learning (Roboflow's Blog)
Using pre-trained weights to bootstrap your model’s learning. You are “transferring” the knowledge learned on another dataset and then “fine-tuning” it to learn about your new domain.
Merging Datasets (Roboflow's Documentation)