Viegut, R.A., E.B. Webb, A.H. Raedeke, Z. Tang, Y. Zhang, Z. Zhai, Z Liu, S. Wang J. Zheng and Y. Shang. Detection probability and bias in machine-learning-based unoccupied aerial system non-breeding waterfowl surveys. Drones
Abstract
Unoccupied aerial systems (UAS) may provide cheaper, safer, and more accurate and precise alternatives to traditional waterfowl survey techniques while also reducing disturbance to waterfowl. We evaluated availability and perception bias on machine-learning-based nonbreeding waterfowl count estimates derived from aerial imagery collected using a DJI Mavic Pro 2 on Missouri Department of Conservation intensively managed wetland Conservation Areas within the Upper Mississippi River Conservation Priority Area across Missouri, USA. UAS imagery was collected using a proprietary software for automated flight path planning in a back-and-forth transect flight pattern at 10 m/s, no earlier than two hours after sunrise and ending by 1:00 pm at 15 - 90 m in altitude above ground level (AGL) (ground sampling distance 0.38-2.29 cm/pixel). Waterfowl in images were labeled using LabelMe by trained labelers. These same images were simultaneously analyzed using a computer algorithm developed to detect and classify waterfowl in aerial images by species and sex. We developed and evaluated three generalized linear mixed models with Bernoulli distributions: one to model the probability that a bird present in the image area was visible in the image (availability), one to model the probability that a bird visible in the image was detected by the algorithm, and one to model the probability that a machine-learning generated detection was actually a false-positive and not an actual bird. Variation in waterfowl availability was best explained by the interaction of vegetation cover type, sky condition, survey altitude, and individual bird characteristics of species and sex, with more complex and taller vegetation cover types reducing availability by up to 70 percent at higher survey altitudes. The probability of the algorithm correctly detecting available birds showed no pattern within vegetation cover type, survey altitude, or sky condition, with the algorithm correctly detecting 85 percent of available birds. The probability of the algorithm generating incorrect false-positive detections was best explained by vegetation cover types with features similar in size and shape to the birds, especially lotus (Nelumbo lutea), with up to 58 percent of detections being false-positive detections in lotus cover types. Overall, the algorithm achieved counts an average of 6.70 percent greater than the human labeled counts, and upon applying correction factors using a modified Horvitz-Thompson estimator, the corrected estimates were an average of 5.59 percent lower than the human labeled counts. Our results indicate that vegetation cover type, sky condition, and survey altitude influence the availability and detection of waterfowl in UAS surveys; however, using well-trained machine learning algorithms may produce accurate counts per image under a variety of survey conditions.