Research record
Two numbers measured on two different sets are not a comparison. That rule governs this page: benchmarks are frozen before models are trained, splits are by source video and never by frame, held-out videos are excluded from every training source, and negatives of different kinds are never pooled. The results that failed those rules are listed here too.
The cascade
Each row is a design decision we would have got wrong by following the default. The right-hand column is what the measurement actually said.
| Decision | The default | What we measured |
|---|---|---|
| Frame preprocessing | Centre-crop to the encoder's square input | Cropping discards 33–44% of a 16:9 tank frame. Letterboxing instead fixed field performance — the failure was aspect ratio, not architecture. |
| Motion gate | Per-video normalised motion | Normalisation rescales a static video until lamp flicker reads as movement. We gate on the absolute changed-pixel fraction, with the burned-in timestamp masked out. |
| Presence detector | Zero-shot CLIP similarity | A trained probe on the same frozen features reaches AUC 0.745 against zero-shot 0.450. The representation is reusable; the zero-shot decision rule is not. |
| Open-vocabulary detector as a filter | Prompt a detector, threshold the score | AUC 0.759 versus human labels — real ranking signal, but no usable operating point: any threshold keeping 95% of the animals also passes 70% of the negatives. |
| When to spend the big model | Run the VLM on everything | Probe frames reject 60% of videos with no full decode; 42.7% of surviving clips still hold no animal. The teacher runs last, on what is left, for $0.0006 a clip. |
Frozen benchmark suite
SEG-TEST is 122 human-drawn mask frames from 5 held-out videos plus verified empty-tank negatives. SKEL-50 is 50 frozen frames from 20 videos. Neither is ever regenerated to suit a result.
| Segmentation model / mode | Mask IoU (mean) | Median | Area error | Presence AUC |
|---|---|---|---|---|
| Shipped — 3.2 M LR-ASPP @ 768² | 0.6415 | 0.7193 | 1.05% | 0.794 |
| Previous headline @ 512² | 0.6075 | 0.6661 | 1.07% | 0.718 |
| + temporal fusion (EMA) | 0.5471 | 0.6000 | 1.00% | 0.9685 |
| + temporal fusion (optical flow) | 0.5109 | 0.5505 | 1.06% | 0.9495 |
| Teacher architecture, zero-shot per frame | 0.374 | — | — | — |
Per frame and zero-shot, the teacher scores 0.374 against 0.6415 for the 3.2 M student — but on the 21 of 122 frames where the teacher clears its own confidence gate it wins, 0.726 to 0.657. The teacher is high-precision and low-recall; distillation converted a sparse, high-quality signal into dense coverage. That is the actual claim, and it is more interesting than the headline.
Smoothing the probability map over time costs 0.10 of mask IoU and buys +0.17 of presence AUC. Optical flow is worse than plain EMA on both axes, so the mechanism is smoothing, not motion compensation. We fuse when the output is a presence gate and don't when it is a mask.
macro-F1 on one frozen test split, held out by source video · "no octopus" is one of the six classes
The pattern across every ablation: everything that adds real information helps; everything that reshuffles existing information does not. Representation +0.087 › class-weight tempering +0.017 › head capacity ≈ 0 › resampling ≈ 0 › feature-space augmentation negative.
| Configuration | Macro-F1 | Note |
|---|---|---|
| Majority-class baseline | 0.100 | predicts "no octopus" always (43.1% accuracy) |
| CLIP probe + motion channels | 0.530 | frozen CLIP ViT-B/32 |
| DINOv2 representation | 0.577 | image backbone swap |
| VideoMAE representation | 0.588 | video backbone |
| Mask-guided crop | 0.618 | pixels spent on the animal |
| 5-member ensemble (shipped) | 0.665 | 75.4% accuracy |
Results that did not survive their own controls
Each of these looked like a win first. They are published because a negative result is what stops the same idea being retried — by us, or by anyone reading the release.
The model scored 0.49 against auto-labels and 0.70 against clean ones — so cleaner labels had to be the lever. They were not: those clean frames were in the training set. Measured leak-free, upgrading the teacher moved held-out IoU from 0.494 to 0.506. A whole planned work-stream cancelled by one split.
Ten segmentation channels lifted macro-F1 from 0.606 to 0.649, a margin 15× the seed noise. Split by whether the segmenter had seen the video: +0.064 on seen videos, −0.025 on unseen. The gain was the leak. The pre-registered control is the only reason it was not reported as a win.
Head width and dropout: +0.0017 against a seed standard deviation of 0.0040 — and 1024 hidden units is the worst setting on both backbones. Upsampling ≈ loss weighting. Feature-space mixup and Gaussian noise are monotonically negative. The model was never capacity- or overfitting-limited.
Balancing was supposed to rescue the starved human interaction class. Unweighted training gives it F1 0.44; weighted gives 0.37 — and the best overall test scores too. We did not adopt it: validation ranks it worst, and taking it would have been selection on the test set.
Probed on wild YouTube footage, both presence signals fall to near chance, and the failure mode is mislocation rather than silence. We took the result at face value and positioned the system as tank instrumentation, not a wild-footage tool.
A classifier over pooled frozen CLIP features reached 45% validation accuracy — below the 50% majority baseline, with per-class F1 near zero on everything but presence. Static appearance cannot carry behaviour. That null is what sent us to video backbones and mask geometry.
Stated plainly
A welfare system that overstates its certainty is worse than no system. These are the live limits of the work as it stands.
One animal. Every behavioural finding describes a single captive Octopus vulgaris. Nothing here is a species-level claim.
The activity budget is the soft finding. Resting ranges 16–73% across the seven analysed days. The circadian profile (4 of 4 days) and the human-presence response (7 of 7 days, p = 0.0078) are the robust ones.
The teacher is not ground truth. Its labels are only moderately self-consistent across passes (κ 0.41–0.59 by field), so every downstream accuracy inherits that bound. Human labels collected so far were partly anchored on the model's answer, which flatters the teacher.
Infrared is out of scope for masks. The colour-trained segmenter over-segments bright tools under IR, so our largest deployment camera is currently excluded from mask-derived features.
Kinematics are uncalibrated. Arm-tip speeds are pixels per second in crop space. The ordering of behaviours is the result; the units are not physical.
Open release
Code under Apache-2.0, labels and benchmarks under CC-BY-4.0. Vote distributions are released rather than majority labels alone, because the margin predicts human agreement and a collapsed label throws that away.
with the full five-pass vote distribution for presence and behaviour
five disjoint frame samplings per clip, so caption stability is measurable
hand-drawn masks and behaviour labels, held out of every training source
presence gate, segmenter, ethogram ensemble, caption LoRA and its 4-bit build
We are looking for partner sites with cameras already running, and for collaborators who want the behavioural record their footage already carries.