An open-source, citizen science and machine learning approach to analyse subsea movies

• Main Authors: Victor Anton, Jannes Germishuys, Per Bergström, Mats Lindegarth, Matthias Obst
• Format: Article
• Language: English
• Published: Pensoft Publishers 2021-02-01
• Series: Biodiversity Data Journal
• Subjects: marine biodiversity; autonomous underwater vehicle
• Online Access: https://bdj.pensoft.net/article/60548/download/pdf/

This paper describes a data system to analyse large amounts of subsea movie data for marine ecological research. The system consists of three distinct modules for data management and archiving, citizen science, and machine learning in a high performance computation environment. It allows scientists to upload underwater footage to a customised citizen science website hosted by Zooniverse, where volunteers from the public classify the footage. Classifications with high agreement among citizen scientists are then used to train machine learning algorithms. An application programming interface allows researchers to test the algorithms and track biological objects in new footage. We tested our system using recordings from remotely operated vehicles (ROVs) in a Marine Protected Area, the Kosterhavet National Park in Sweden. Results indicate a strong decline of cold-water corals in the park over a period of 15 years, showing that our system allows to effectively extract valuable occurrence and abundance data for key ecological species from underwater footage. We argue that the combination of citizen science tools, machine learning, and high performance computers are key to successfully analyse large amounts of image data in the future, suggesting that these services should be consolidated and interlinked by national and international research infrastructures.Novel information system to analyse marine underwater footage.