Now You See It: Evaluating a Small Object Detection Model for Uncrewed Surface Vehicle Operations

As Uncrewed Surface Vehicle (USV) operations scale in size, distance, and complexity, they increasingly rely on artificial intelligence (AI) algorithms to maintain situational awareness, and operators must understand how algorithm performance may change as camera configuration is adjusted to meet operational needs. This thesis proposes distance-based evaluation strategies that characterize object detection performance in terms directly applicable to USV operations. In this research, a YOLOv5s model was trained to detect small maritime obstacles, focusing on aids to navigation, and the lobster pot floats common in the Gulf of Maine. Three test videos were collected, each capturing a buoy approaching the camera from 100 meters to 4 meters under fair weather conditions, with known distance measurements for each frame. The experiments evaluated detection confidence across varying camera resolutions and video bitrates, examined the impact of incongruent model input configuration and camera resolution, and explored whether model architecture could inform object detection distances. Results demonstrated that changes to image resolution produce more predictable effects on detection distance than bitrate changes and that reducing the image size consistently degrades performance, while bitrate reductions below 2500 kbps result in inconsistent detections over the 100-meter range. Additional findings revealed that excessive padding of image frames severely degrades detection, particularly for distant objects, and that conservative maximum detection boundary can be calculated from the model's smallest anchor box, a pinhole camera model, and known sizes of small objects.