Perception models

The models that turn pixels into detections. This is where the capability lives and where the recurring engineering cost sits.

Start with the licence, not the model

Ultralytics is AGPL-3.0. The AGPL's network clause reaches a hosted service, not only a distributed binary, so a customer-facing product built on it either carries source obligations or needs their commercial licence. Either is a valid choice. Making it accidentally, eighteen months and 20,000 labelled frames later, is not.

Start on an Apache-2.0 detector. The accuracy difference on agricultural imagery is small; the licence difference is not.

The dataset is the asset

A public weed dataset trained on another country's soil, camera, growth stage and light will disappoint you. The model is commodity; the labelled data from your own fields is not, and it is the only thing here a competitor cannot buy.

Note the pattern: the capabilities that need no labels are the ones to ship first. Change detection against a field's own history is unsupervised and works from day one. See disease detection.

Labelling

Two practices that halve the cost:

Assisted labelling. Use a segmentation model to propose masks, and have the human correct rather than draw. For leafy targets this is a large multiplier.

Ground truth from the field. Every time a scout walks to a flagged patch and records what it was, that is a labelled example with a location and a date. Build the capture form into the scouting route on day one, or you will be labelling from scratch a year later.

Training

That last row matters commercially. A model with a slightly worse mAP that produces a prescription 2% closer on treated area is the better model for this product, because treated area is what the customer pays for and what the dashboard reports.

Deployment

Quantisation to INT8 typically costs under a point of accuracy and buys 2 to 4x throughput. Always verify on your own held-out farm rather than trusting the tooling's report.

Tiled inference

Agricultural frames are large and targets are small. Naive resizing destroys them.

1. Slice the frame into overlapping tiles at the model's native input size.
2. Run inference per tile.
3. Merge with non-maximum suppression across tile boundaries.
4. Project to ground coordinates.
5. Deduplicate across frame overlap.

SAHI (MIT) implements steps 1 to 3. Steps 4 and 5 are yours, and step 5 is where double counting quietly ruins a yield estimate. Validate against hand-counted quadrats every season.

Counting: use density, not boxes

For dense objects like wheat ears, drawing a box around each of 400 objects in a frame is unaffordable. Point-supervised density map regression needs one click per object and predicts a count directly. It is the right architecture for yield forecasting and it costs a fraction of the labelling.

Change detection needs no model at all

For disease and pest, the strongest first version is statistical:

1. Store every flight's calibrated index raster, aligned, keyed by field and date.
2. For each pixel, compute a z-score against that field's own history at the same growth stage.
3. Threshold, take connected components, rank by area times severity.

No labels, no training, and it directly implements "compared against the field's own history, so a wet spring baseline does not read as disease". Add supervised models later, to name the cause rather than to find the patch.

Confidence, and what to do with it

Every detection carries a confidence. This repository stores it on the Detection row and the field map varies marker opacity with it.

The escalation queue is a feature, not an admission. It is what lets you promise autonomy without promising infallibility, and one reviewer can clear a day's ambiguous findings across many farms in an hour.