Train a Custom YOLOS Transformer Model

This tutorial highlights the combination of a suite of best-in-class machine learning tools - Hugging Face Transformers, Roboflow, Weights and Biases, and AWS Sagemaker Serverless - to create a YOLOS object detection labeling, training, and deployment pipeline.

Roboflow Model Library (Model Zoo): YOLOS

The YOLOS model was proposed in You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection by Yuxin Fang, Bencheng Liao, Xinggang Wang, Jiemin Fang, Jiyang Qi, Rui Wu, Jianwei Niu, Wenyu Liu. YOLOS proposes to just leverage the plain Vision Transformer (ViT) for object detection, inspired by DETR. It turns out that a base-sized encoder-only Transformer can also achieve 42 AP on COCO, similar to DETR and much more complex frameworks such as Faster R-CNN. - https://huggingface.co/docs/transformers/main/en/model_doc/yolos

Roboflow, Hugging Face Transformers, Weights & Biases, and AWS Sagemaker Model Pipeline

Workflow:

1. Install YOLOS Environment
2. Gather and Label Our Dataset in Roboflow
3. Download Our Dataset and Prepare Data For Training
4. Configure Our YOLOS Model from Hugging Face Transformers; Hugging Face YOLOS listing || Original YOLOS Github Repository
5. Run Custom YOLOS Training in PyTorch Lightning
6. Visualize YOLOS Training in Weights and Biases
7. Evaluate Custom YOLOS Model
8. Visualize Test Inference
9. Deploy YOLOS Model to AWS Sagemaker Serverless
10. Sample images for Active Learning with Roboflow to improve our model

• Once you copy your dataset export code (export in COCO JSON format), continue on with the video tutorial to train and deploy your model

Train and Deploy YOLOS Transformer on a Custom Dataset (Roboflow Blog)

(with the help of Roboflow's python package for easy data management)

• YOLOS Model Training Notebook (Roboflow Model Zoo)
• YOLOS, GitHub Repository

Don't forget to save your model weights!

How to Save and Load Your Model Weights in Google Colab