How do I visualize false positives identified during evaluation?

You can filter the dataset view for samples with high false positive counts or use the SDK's evaluation patches feature to create a view focused specifically on individual errors.

Which evaluation methods are supported?

The skill supports COCO and Open Images for detections, simple, top-k, and binary for classifications, and ActivityNet for temporal detections in video.

Can I evaluate segmentation masks?

Yes, it supports segmentation evaluation using the simple method, with optional bandwidth parameters to focus specifically on boundary pixel accuracy.

Does this skill handle mAP computation automatically?

Yes, for detection and temporal tasks, you can set the compute_mAP parameter to true to generate mean Average Precision, mean Average Recall, and PR curves.

Do I need to manually install the evaluation plugin?

The skill includes a directive to check for the @voxel51/evaluation plugin and will guide you through the download and enabling process if it is missing.

FiftyOne Model Evaluation

Name: FiftyOne Model Evaluation
Author: voxel51

byvoxel51

•

Data Science & ML

Evaluates computer vision model predictions against ground truth using industry-standard protocols like COCO and Open Images.

About

The fiftyone-model-evaluation skill streamlines the process of assessing machine learning models within the FiftyOne ecosystem. It provides a standardized workflow to compute critical performance metrics such as mAP, precision, recall, and confusion matrices across various tasks including object detection, classification, segmentation, and regression. By automating plugin management and providing specific directives for evaluation parameters, this skill enables developers to identify model failures, analyze true/false positives, and visualize performance gaps directly within the FiftyOne App or through the Python SDK.

Key Features

Standardized evaluation protocols including COCO, Open Images, and ActivityNet
Support for multiple CV tasks: detection, classification, segmentation, and regression
3 GitHub stars
Automated computation of mAP, precision, recall, and confusion matrices
Interactive visualization of True Positives, False Positives, and False Negatives
Seamless integration between FiftyOne App operators and Python SDK views

Use Cases

Analyzing classification errors through confusion matrices and per-class performance metrics
Benchmarking object detection models using COCO-style mAP at various IoU thresholds
Comparing performance between multiple model versions to identify specific failure scenarios

About

Key Features

Standardized evaluation protocols including COCO, Open Images, and ActivityNet
Support for multiple CV tasks: detection, classification, segmentation, and regression
3 GitHub stars
Automated computation of mAP, precision, recall, and confusion matrices
Interactive visualization of True Positives, False Positives, and False Negatives
Seamless integration between FiftyOne App operators and Python SDK views

Use Cases

Analyzing classification errors through confusion matrices and per-class performance metrics
Benchmarking object detection models using COCO-style mAP at various IoU thresholds
Comparing performance between multiple model versions to identify specific failure scenarios