于红

• 1:大连海洋大学信息工程学院
• 2:设施渔业教育部重点实验室
• 3:辽宁省海洋信息技术重点实验室

摘要(Abstract)：

水产动物目标探测与追踪研究对智慧渔业、精准养殖具有重要的意义,本文全面梳理了基于声学传感器、光学摄像机及声学与光学结合的水产动物数据采集技术,系统阐述了数据集构建、数据预处理、水产动物目标识别、水产动物目标跟踪等技术研究进展及其在海洋渔业领域的应用现状,分析了水产动物目标探测与追踪技术研究及应用中面临的问题和挑战,对未来的研究方向进行了展望,提出了加快水产动物水下目标探测与追踪数据集及开放共享服务平台构建、实现水产动物精准探测与实时追踪关键技术突破、推动基于水产动物精准探测与实时追踪的智能监控设备研发等建议,旨在为加速推动智慧渔业、精准养殖及提升海洋渔业现代化水平提供参考。

关键词(KeyWords)： 水产动物;水下目标识别;水下目标跟踪;深度学习

Abstract:

Keywords:

基金项目(Foundation): 辽宁省重点研发计划项目(2020JH2/10100043);; 辽宁省高等学校海洋产业技术研究院项目(2018-CY-04)

作者(Author): 于红

DOI: 10.16535/j.cnki.dlhyxb.2020-263

参考文献(References)：

• [1]JUELL J E,WESTERBERG H.An ultrasonic telemetric system for automatic positioning of individual fish used to track Atlantic salmon(Salmo salar L.) in a sea cage[J].Aquacultural Engineering,1993,12(1):1-18.
• [2]LUBIS M Z,MANIK H M.Acoustic systems(split beam echo sounder) to determine abundance of fish in marine fisheries[J].Journal of Geoscience,Engineering,Environment,and Technology,2017,2(1):76-83.
• [3]BELCHER E,HANOT W,BURCH J.Dual-frequency identification sonar(DIDSON)[C]//Proceedings of the 2002 International Symposium on Underwater Technology.Tokyo,Japan:IEEE,2002:187-192,doi:10.1109/UT.2002.1002424.
• [4]段延娥,李道亮,李振波,等.基于计算机视觉的水产动物视觉特征测量研究综述[J].农业工程学报,2015,31(15):1-11.DUAN Y E,LI D L,LI Z B,et al.Review on visual characteristic measurement research of aquatic animals based on computer vision[J].Transactions of the Chinese Society of Agricultural Engineering,2015,31(15):1-11.(in Chinese)
• [5]何佳,黄志涛,宋协法,等.基于计算机视觉技术的水产养殖中鱼类行为识别与量化研究进展[J].渔业现代化,2019,46(3):7-14.HE J,HUANG Z T,SONG X F,et al.Research progress on recognition and quantification of fish behavior in aquaculture based on computer vision technology[J].Fishery Modernization,2019,46(3):7-14.(in Chinese)
• [6]林森,赵颍.水下光学图像中目标探测关键技术研究综述[J].激光与光电子学进展,2020,57(6):060002.LIN S,ZHAO Y.Review on key technologies of target exploration in underwater optical images[J].Laser&Optoelectronics Progress,2020,57(6):060002.(in Chinese)
• [7]郭戈,王兴凯,徐慧朴.基于声呐图像的水下目标检测、识别与跟踪研究综述[J].控制与决策,2018,33(5):906-922.GUO G,WANG X K,XU H P.Review on underwater target detection,recognition and tracking based on sonar image[J].Control and Decision,2018,33(5):906-922.(in Chinese)
• [8]荆丹翔,周晗昀,韩军,等.基于成像声呐DIDSON的水域内鱼群数量估计方法[J].应用声学,2019,38(4):705-711.JING D X,ZHOU H Y,HAN J,et al.Fish abundance estimation based on an imaging sonar[J].Applied Acoustics,2019,38(4):705-711.(in Chinese)
• [9]张翔,沈蔚,童剑锋,等.基于DIDSON双频识别声纳技术的青草沙水库鱼类资源量评估[J].上海海洋大学学报,2017,26(4):561-569.ZHANG X,SHEN W,TONG J F,et al.Evaluation of fish resources in Qingcaosha Reservoir based on dual frequency identification sonar technology[J].Journal of Shanghai Ocean University,2017,26(4):561-569.(in Chinese)
• [10]徐建瑜,崔绍荣,苗香雯,等.计算机视觉技术在水产养殖中的应用与展望[J].农业工程学报,2005,21(8):174-178.XU J Y,CUI S R,MIAO X W,et al.Application and prospect of computer vision technology in aquaculture[J].Transactions of the Chinese Society of Agricultural Engineering,2005,21(8):174-178.(in Chinese)
• [11]BANAN A,NASIRI A,TAHERI-GARAVAND A.Deep learningbased appearance features extraction for automated carp species identification[J].Aquacultural Engineering,2020,89:102053.
• [12]刘世晶,王帅,陈军,等.基于改进主成分分析和Ada Boost算法的运动虾苗识别方法[J].农业工程学报,2017,33(1):212-218.LIU S J,WANG S,CHEN J,et al.Moving larval shrimps recognition based on improved principal component analysis and AdaBoost[J].Transactions of the Chinese Society of Agricultural Engineering,2017,33(1):212-218.(in Chinese)
• [13]ZHOU Y Z,YU H,WU J F,et al.Fish density estimation with multi-scale context enhanced convolutional neural network[J].Journal of Communications and Information Networks,2019,4(3):80-88.
• [14]MAO J F,XIAO G,SHENG W G,et al.Research on realizing the3D occlusion tracking location method of fish’s school target[J].Neurocomputing,2016,214:61-79.
• [15]PAUTSINA A,CISAR P,STYS D,et al.Infrared reflection system for indoor 3D tracking of fish[J].Aquacultural Engineering,2015,69:7-17.
• [16]PETTERSEN R,BRAA H L,GAWEL B A,et al.Detection and classification of Lepeophterius salmonis(Kroyer,1837) using underwater hyperspectral imaging[J].Aquacultural Engineering,2019,87:102025.
• [17]FU X P,SHANG X D,SUN X D,et al.Underwater hyperspectral target detection with band selection[J].Remote Sensing,2020,12(7):1056.
• [18]TERAYAMA K,SHIN K,MIZUNO K,et al.Integration of sonar and optical camera images using deep neural network for fish monitoring[J].Aquacultural Engineering,2019,86:102000.
• [19]TANG C,ZHANG G,HU H G,et al.An improved YOLOv3 algorithm to detect molting in swimming crabs against a complex background[J].Aquacultural Engineering,2020,91:102115.
• [20]MENG L,HIRAYAMA T,OYANAGI S.Underwater-drone with panoramic camera for automatic fish recognition based on deep learning[J].IEEE Access,2018,6:17880-17886.
• [21]FISHER R B,CHEN-BURGER Y H,GIORDANO D,et al.Fish4Knowledge:collecting and analyzing massive coral reef fish video data[M].Cham:Springer International Publishing,2016.
• [22]SPAMPINATO C,PALAZZO S,BOOM B,et al.Overview of the life CLEF 2014 fish task[C]//Proceedings of Working Notes for CLEF 2014 Conference.Sheffield,UK,2014:616-624.
• [23]HE J Y,VAN OSSENBRUGGEN J,DE VRIES A P.Fish4label:accomplishing an expert task without expert knowledge[R/OL].(2013-05-15)[2020-09-01].http://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/PAPERS/oair2013-demo.pdf.
• [24]GARCIA V,SANT’ANA D A,ZANONI V A G,et al.A new image dataset for the evaluation of automatic fingerlings counting[J].Aquacultural Engineering,2020,89:102064.
• [25]WANG Z H,LIU C W,WANG S J,et al.UDD:an underwater open-sea farm object detection dataset for underwater robot picking[EB/OL].(2003-05-08)[2020-09-01].https://arxiv.org/pdf/2003.01446.pdf.
• [26]罗强,吴俊峰,于红,等.一种基于改进直方图均衡化的显著图提取算法[J].小型微型计算机系统,2018,39(5):1092-1096.LUO Q,WU J F,YU H,et al.Saliency detection algorithm based on improved histogram equalization[J].Journal of Chinese Computer Systems,2018,39(5):1092-1096.(in Chinese)
• [27]LIU Z Y,LI X,FAN L Z,et al.Measuring feeding activity of fish in RAS using computer vision[J].Aquacultural Engineering,2014,60:20-27.
• [28]杨卫中,徐银丽,乔曦,等.基于对比度受限直方图均衡化的水下海参图像增强方法[J].农业工程学报,2016,32(6):197-203.YANG W Z,XU Y L,QIAO X,et al.Method for image intensification of underwater sea cucumber based on contrast-limited adaptive histogram equalization[J].Transactions of the Chinese Society of Agricultural Engineering,2016,32(6):197-203.(in Chinese)
• [29]赵德安,刘晓洋,孙月平,等.基于机器视觉的水下河蟹识别方法[J].农业机械学报,2019,50(3):151-158.ZHAO D A,LIU X Y,SUN Y P,et al.Detection of underwater crabs based on machine vision[J].Transactions of the Chinese Society for Agricultural Machinery,2019,50(3):151-158.(in Chinese)
• [30]赵德安,曹硕,孙月平,等.基于联动扩展神经网络的水下自由活蟹高效检测器[J].农业机械学报,2020,51(19):163-174.ZHAO D A,CAO S,SUN Y P,et al.Small-sized efficient detector for underwater freely live crabs based on compound scaling neural network[J].Transactions of the Chinese Society for Agricultural Machinery,2020,51(19):163-174.(in Chinese)
• [31]张俊龙,曾国荪,覃如符.基于深度学习的海底观测视频中鱼类的识别方法[J].计算机应用,2019,39(2):376-381.ZHANG J L,ZENG G S,QIN R F.Fish recognition method for submarine observation video based on deep learning[J].Journal of Computer Applications,2019,39(2):376-381.(in Chinese)
• [32]YE X C,LI Z,SUN B L,et al.Deep joint depth estimation and color correction from monocular underwater images based on unsupervised adaptation networks[J].IEEE Transactions on Circuits and Systems for Video Technology,2019,doi:10.1109/TCSVT.2019.2958950.
• [33]HOU M J,LIU R S,FAN X,et al.Joint residual learning for underwater image enhancement[C]//Proceedings of 2018 25th IEEE International Conference on Image Processing.Athens,Greece:IEEE,2018:4043-4047.
• [34]李娟,朱学岩,葛凤丽,等.基于计算机视觉的水下海参识别方法研究[J].中国农机化学报,2020,41(7):171-177.LI J,ZHU X Y,GE F L,et al.Research on underwater sea cucumber identification based on computer vision[J].Journal of Chinese Agricultural Mechanization,2020,41(7):171-177.(in Chinese)
• [35]SAID A B,HADJIDJ R,MELKEMI K E,et al.Multispectral image denoising with optimized vector non-local mean filter[J].Digital Signal Processing,2016,58:115-126.
• [36]DE ROBERTIS A,HIGGINBOTTOM I.A post-processing technique to estimate the signal-to-noise ratio and remove echosounder background noise[J].ICES Journal of Marine Science,2007,64(6):1282-1291,[37]孙建伟,于红,吴俊峰,等.一种使用多特征的鱼类图像检索方法[J].大连海洋大学学报,2016,31(4):449-455.SUN J W,YU H,WU J F,et al.A fish image retrieval method using multiple features[J].Journal of Dalian Ocean University,2016,31(4):449-455.(in Chinese)
• [38]ZHANG D,LILLYWHITE K D,LEE D J,et al.Automatic shrimp shape grading using evolution constructed features[J].Computers and Electronics in Agriculture,2014,100:116-122.
• [39]徐凤强,董鹏,王辉兵,等.基于水下机器人的海产品智能检测与自主抓取系统[J].北京航空航天大学学报,2019,45(12):2393-2402.XU F Q,DONG P,WANG H B,et al.Intelligent detection and autonomous capture system of seafood based on underwater robot[J].Journal of Beijing University of Aeronautics and Astronautics,2019,45(12):2393-2402.(in Chinese)
• [40]杨眉,魏鸿磊,华顺刚.一种基于神经网络的扇贝图像识别方法[J].大连海洋大学学报,2014,29(1):70-74.YANG M,WEI H L,HUA S G.A scallop image recognition method based on a neural network[J].Journal of Dalian Ocean University,2014,29(1):70-74.(in Chinese)
• [41]刘萍,杨鸿波,宋阳.改进YOLOv3网络的海洋生物识别算法[J].计算机应用研究,2020,37(sup 1):394-397.LIU P,YANG H B,SONG Y.Improved-YOLOv3-based marine biology recognition algorithm[J].Application Research of Computers,2020,37(sup 1):394-397.(in Chinese)
• [42]张美玲,吴俊峰,于红,等.基于颜色四通道及空间金字塔的鱼类图像检索[J].计算机应用,2019,39(5):1466-1472.ZHANG M L,WU J F,YU H,et al.Fish image retrieval algorithm based on color four channels and spatial pyramid[J].Journal of Computer Applications,2019,39(5):1466-1472.(in Chinese)
• [43]SALMAN A,JALAL A,SHAFAIT F,et al.Fish species classification in unconstrained underwater environments based on deep learning[J].Limnology and Oceanography:Methods,2016,14(9):570-585.
• [44]CHEN F D,XU J P,WEI Y,et al.Establishing an eyeball-weight relationship for Litopenaeus vannamei using machine vision technology[J].Aquacultural Engineering,2019,87:102014.
• [45]LEE D J,XIONG G M,LANE R M,et al.An efficient shape analysis method for shrimp quality evaluation[C]//Proceedings of2012 12th International Conference on Control Automation Robotics&Vision.Guangzhou:IEEE,2012:865-870.
• [46]王斌,徐建瑜,王春琳.基于计算机视觉的梭子蟹蜕壳检测及不同背景对蜕壳的影响[J].渔业现代化,2016,43(2):11-16.WANG B,XU J Y,WANG C L.Computer-vision based molting detection of Portunus tritubercularus and effects of different backgrounds on molting[J].Fishery Modernization,2016,43(2):11-16.(in Chinese)
• [47]郭常有,曹广斌,韩世成,等.基于图像处理技术的扇贝尺寸及位置自动识别[J].大连海洋大学学报,2012,27(6):578-582.GUO C Y,CAO G B,HAN S C,et al.Automatic measurement of size and location in scallop based on image processing[J].Journal of Dalian Ocean University,2012,27(6):578-582.(in Chinese)
• [48]XU F Q,DING X Y,PENG J J,et al.Real-time detecting method of marine small object with underwater robot vision[C]//Proceedings of 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans.Kobe,Japan:IEEE,2018:1-4.
• [49]LIU R S,FAN X,ZHU M,et al.Real-world underwater enhancement:challenges,benchmarks,and solutions under natural light[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,doi:10.1109/TCSVT.2019.2963772.
• [50]ROVA A,MORI G,DILL L M.One fish,two fish,butterfish,trumpeter:recognizing fish in underwater video[R/OL].(2007-05-16)[2020-09-01].http://www.osti.gov/cgi-bin/eprints/redirect Eprints Url?http%3A%2F%2Fwww.sfu.ca%2Fbiology%2Ffaculty%2Fdill%2Fpublications%2Frova_fis.pdf.
• [51]THARWAT A,HEMEDAN A A,HASSANIEN A E,et al.A biometric-based model for fish species classification[J].Fisheries Research,2018,204:324-336.
• [52]张美玲,吴俊峰,于红,等.一种基于HSVG-SURF特征的鱼类图像检索算法[J].小型微型计算机系统,2018,39(9):2085-2089.ZHANG M L,WU J F,YU H,et al.Fish image retrieval algorithm based on HSVG four channel SURF feature[J].Journal of Chinese Computer Systems,2018,39(9):2085-2089.(in Chinese)
• [53]HARBITZ A.Estimation of shrimp(Pandalus borealis) carapace length by image analysis[J].ICES Journal of Marine Science,2007,64(5):939-944.
• [54]HRIDAYAMI P,PUTRA I K G D,WIBAWA K S.Fish species recognition using VGG16 deep convolutional neural network[J].Journal of Computing Science and Engineering,2019,13(3):124-130.
• [55]王柯力,袁红春.基于迁移学习的水产动物图像识别方法[J].计算机应用,2018,38(5):1304-1308,1326.WANG K L,YUAN H C.Aquatic animal imang classification method based on transfer learning[J].Journal of Computer Applications,2018,38(5):1304-1308,1326.(in Chinese)
• [56]陈英义,龚川洋,刘烨琦,等.基于FTVGG16卷积神经网络的鱼类识别方法[J].农业机械学报,2019,50(5):223-231.CHEN Y Y,GONG C Y,LIU Y Q,et al.Fish identification method based on FTVGG16 convolutional neural network[J].Transactions of the Chinese Society for Agricultural Machinery,2019,50(5):223-231.(in Chinese)
• [57]李庆忠,李宜兵,牛炯.基于改进YOLO和迁移学习的水下鱼类目标实时检测[J].模式识别与人工智能,2019,32(3):193-203.LI Q Z,LI Y B,NIU J.Real-time detection of underwater fish based on improved YOLO and transfer learning[J].Pattern Recognition and Artificial Intelligence,2019,32(3):193-203.(in Chinese)
• [58]贾宇霞,樊帅昌,易晓梅.基于显著性增强和迁移学习的鱼类识别研究[J].渔业现代化,2020,47(1):38-46.JIA Y X,FAN S C,YI X M.Fish recognition based on significant enhancement and transfer learning[J].Fishery Modernization,2020,47(1):38-46.(in Chinese)
• [59]ALLKEN V,HANDEGARD N O,ROSEN S,et al.Fish species identification using a convolutional neural network trained on synthetic data[J].ICES Journal of Marine Science,2019,76(1):342-349.
• [60]SIDDIQUI S A,SALMAN A,MALIK M I,et al.Automatic fish species classification in underwater videos:exploiting pre-trained deep neural network models to compensate for limited labelled data[J].ICES Journal of Marine Science,2018,75(1):374-389.
• [61]TAMOU A B,BENZINOU A,NASREDDINE K,et al.Underwater live fish recognition by deep learning[C]//Proceedings of the8th International Conference on Image and Signal Processing.Cherbourg,France:Springer,2018:275-283.
• [62]李均鹏,祝开艳,杨澍.基于迁移学习的复杂场景海洋鱼类识别方法[J].计算机应用与软件,2019,36(9):168-174.LI J P,ZHU K Y,YANG S.Ocean fish recognition incomplex scene based on transfer learning[J].Computer Applications and Software,2019,36(9):168-174.(in Chinese)
• [63]ZHOU J,CLARK C M.Autonomous fish tracking by ROV using monocular camera[R/OL].(2006-07-07)[2020-09-01].http://www.hmc.edu/lair/publications/2006/zhou_crv_2006.pdf.
• [64]袁永明,施珮.基于图像处理的鱼群运动监测方法研究[J].南方水产科学,2018,14(5):109-114.YUAN Y M,SHI P.Study on fish movement monitoring method based on image processing[J].South China Fisheries Science,2018,14(5):109-114.(in Chinese)
• [65]CHUANG M C,HWANG J N,WILLIAMS K,et al.Tracking live fish from low-contrast and low-frame-rate stereo videos[J].IEEE Transactions on Circuits and Systems for Video Technology,2015,25(1):167-179.
• [66]WANG G A,HWANG J N,WILLIAMS K,et al.Closed-loop tracking-by-detection for ROV-based multiple fish tracking[C]//Proceedings of 2016 ICPR 2nd Workshop on Computer Vision for Analysis of Underwater Imagery.Cancun,Mexico:IEEE,2016:7-12.
• [67]CHUANG M C,HWANG J N,YE J H,et al.Underwater fish tracking for moving cameras based on deformable multiple kernels[J].IEEE Transactions on Systems,Man,and Cybernetics:Systems,2017,47(9):2467-2477.
• [68]PANG B,LI Y Z,ZHANG Y F,et al.Tube TK:adopting tubes to track multi-object in a one-step training model[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA,USA:IEEE,2020:6307-6317.
• [69]BRASG,LEAL-TAIXL.Learning a neural solver for multiple object tracking[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA,USA:IEEE,2020:6246-6256,doi:10.1109/CVPR42600.2020.00628.
• [70]WANG G T,LUO C,SUN X Y,et al.Tracking by instance detection:a meta-learning approach[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA,USA:IEEE,2020:6287-6296,doi:10.1109/CVPR42600.2020.00632.
• [71]GUO D Y,WANG J,CUI Y,et al.Siam CAR:siamese fully convolutional classification and regression for visual tracking[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA,USA:IEEE,2020:6268-6276,doi:10.1109/CVPR42600.2020.00630.
• [72]DAI K N,ZHANG Y H,WANG D,et al.High-performance longterm tracking with meta-updater[C]//Proceedings of 2020IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA,USA:IEEE,2020:6297-6306,doi:10.1109/CVPR42600.2020.00633.
• [73]SADOUL B,MENGUES P E,FRIGGENS N C,et al.A new method for measuring group behaviours of fish shoals from recorded videos taken in near aquaculture conditions[J].Aquaculture,2014,430:179-187.
• [74]陈彩文,杜永贵,周超,等.基于图像纹理特征的养殖鱼群摄食活动强度评估[J].农业工程学报,2017,33(5):232-237.CHEN C W,DU Y G,ZHOU C,et al.Evaluation of feeding activity of shoal based on image texture[J].Transactions of the Chinese Society of Agricultural Engineering,2017,33(5):232-237.(in Chinese)
• [75]陈明,张重阳,冯国富,等.基于特征加权融合的鱼类摄食活动强度评估方法[J].农业机械学报,2020,51(2):245-253.CHEN M,ZHANG C Y,FENG G F,et al.Intensity assessment method of fish feeding activities based on feature weighted fusion[J].Transactions of the Chinese Society of Agricultural Engineering,2020,51(2):245-253.(in Chinese)
• [76]ZHAO J,BAO W J,ZHANG F D,et al.Modified motion influence map and recurrent neural network-based monitoring of the local unusual behaviors for fish school in intensive aquaculture[J].Aquaculture,2018,493:165-175.
• [77]TSAO S L,TIAN W M,SHEN C Y,et al.Benthic fish behavior characterization with a mechanically scanned imaging sonar[J].Aquacultural Engineering,2019,84:1-11.
• [78]INA Y,TAKAGI T,MIYASHITA S,et al.Analysis of sinking death using video images of the swimming performance of Pacific bluefin tuna(Thunnus orientalis) larvae[J].Aquacultural Engineering,2020,90:102099.
• [79]王振宇,张国胜,包林,等.利用水下摄影测量技术测量鱼类体长的可行性研究[J].大连海洋大学学报,2018,33(2):251-257.WANG Z Y,ZHANG G S,BAO L,et al.Feasibility of measurement of fish body length using underwater photogrammetry[J].Journal of Dalian Ocean University,2018,33(2):251-257.(in Chinese)
• [80]HUANG I W,HWANG J N,ROSE C S.Chute based automated fish length measurement and water drop detection[C]//Proceedings of 2016 IEEE International Conference on Acoustics,Speech and Signal Processing.Shanghai:IEEE,2016:1906-1910.