RETRAINING-FREE OBJECT AUTHORIZATION FOR VIDEO SURVEILLANCE: AN EMPIRICAL STUDY OF EMBEDDING VERIFICATION

Abstract

Surveillance detectors report what an object is but not whether it is registered with the site, so an unattended bag and an enrolled bag look identical to the system. This paper studies ISS-RUO, a retraining-free authorization layer in which a YOLOv8 detector localizes objects, a frozen ImageNet-pretrained EfficientNetB0 embeds each region, and the maximum cosine similarity to an enrolled database is thresholded to label the object AUTHORIZED or UNREGISTERED; new objects are enrolled simply by adding images. The layer is evaluated with disjoint enrolment, validation, and test partitions, thresholds tuned on validation data only, and five repeated class splits. On object-centred images (Caltech-256; 20 authorized and 20 unseen classes) it reaches an AUROC of 0.892 ± 0.017 and a balanced accuracy of 0.800 ± 0.017, comparable to a linear classifier with softmax rejection (AUROC 0.897 ± 0.023) and better than CLIP embeddings (0.841 ± 0.038); a fixed threshold of 0.75 fails (accuracy 0.361), and enrolling one image per class lowers AUROC to 0.769. On multi-object COCO scenes, detector-based localization is essential (AUROC 0.900 versus 0.760 for whole-scene embeddings), but the detector found only about 60% of annotated objects, and when authorization coincides with the detector’s own classes its label rule (F1 0.932) far outperforms embedding verification (F1 0.567). On a two-core CPU, processing took about 272 ms per frame, and frame skipping cut CPU time by 78%. The results indicate where embedding based authorization is useful and where it is not.