Automated detection of wildlife in proximity to marine renewable energy infrastructure using machine learning of underwater imagery

Abstract

Environmental interactions of marine renewable energy (MRE) projects are challenging to monitor, and key questions remain about their potential impacts. Processing large volumes of environmental data acquired from submarine monitoring and the use of machine learning to identify presence and interactions of marine wildlife with MRE infrastructure are powerful tools for assessing the environmental response to MRE infrastructure. The use of automated image analysis for species identification and enumeration using algorithms like convolutional neural networks can vastly reduce the time required to extract usable data from submarine imagery compared to manual expert processing. We present a novel industry-ready image processing workflow for automated wildlife detection developed using 1000+ hours of underwater video footage obtained by Nova Innovation Ltd. from their operational tidal stream turbine array at Bluemull Sound in Shetland, Scotland. The objective of this work was to develop a workflow and associated algorithms to automatically filter many hours of underwater video, remove unwanted footage, and extract only video containing marine mammals, diving birds or fish. The workflow includes object detection through advanced image analysis, image classification using machine learning, statistical analyses such as quantification of data storage reduction and number of detections, and automated production of a summary report. Blind tests were undertaken on a subset of videos to quantify and iteratively improve the accuracy of the results. The final iteration of the workflow delivered an accuracy of 80% for the identification of marine mammals, diving birds and fish when a three-category (wildlife, algae, and background) classification system was used. The accuracy rose to 95% when a two-category system was used, and objects were classified simply as ‘target’ or ‘non-target’. The entire workflow can be run from video inception to production of an automated results report in approximately 30 minutes, dependent on size of input data, when environmental conditions such as water clarity and key species of interest are familiar to the algorithm. The accuracy and runtime speed of the workflow can be improved through expanding the training dataset of images used in the development of this initial tool by including additional species and water conditions. Application of this workflow significantly reduces manual processing and interpretation time, which can be a significant burden on project developers. Automated processing provides a subset for more focused manual scrutiny and analysis, while reducing the overall size of dataset requiring storage. Auto-reporting can be used to provide outputs for marine regulators to meet monitoring reporting conditions of project licences. Integration of this workflow with automated passive acoustic monitoring systems can provide a holistic environmental monitoring approach using both underwater imagery and acoustics.