SOLUTION 02 / Digital Transformation

A fish counting system
that reduces the burden on workers
for sustainable fisheries

Research & Development Center, Yanmar Marine Systems Co., Ltd.
This system uses images from a camera installed underwater to count the number of tuna in real time. The company has released an automated fish counting system that achieves 98% or better accuracy. In addition to minimizing the burden on workers, this system also contributes to improving the efficiency of fishery operations and to digitalization of the marine field.

ISSUE

Development was started based on consultations from customers regarding their concerns.

In fish farming, it is essential to know the number of fish inside each net in order to control production volume and the amount of feed used, and also to submit reports and other materials to the Fisheries Agency. In particular for farming of tuna, where the price per fish is high, more accurate counting also results in more efficient operations.

The reason for the start of the automated fish counting system project was the concern raised by customers who were engaged in tuna farming over the difficulty in counting the number of fish for management. At that time, two main methods were used to count fish at most fish farms. The first was installing a camera in the net, or having a diver carry a camera, to capture images, and then visually counting each individual fish. The second was having a diver enter the net and count the number of dead fish, and subtracting that number from the previous total for management. Both methods place a large burden on the person performing the work, and the level of accuracy varies.

In order to address these concerns from the customers, Yanmar Marine Systems Co., Ltd. (hereafter, YMS) focused on the Yanmar image recognition technologies. It launched a full-scale project aimed to applying these technologies to create a system capable of counting the number of tuna in real time with an accuracy of 98% or higher.
Creating this system required a total solution involving both hardware and software. In cooperation with the Yanmar Research & Development Center, which possesses a higher level of image recognition technologies than other companies, the project was carried out using an integrated in-house system. As a result, coordination was also conducted smoothly and the project succeeded in improving the count accuracy. A test in an actual environment was carried out in June 2020. Adjustments were then made and new tests carried out in October and November. The results achieved the target of 98% or higher accuracy, and the product was released in December of that year.

During research and development, it was necessary to identify the actual number of fish in order to check the count accuracy. For this purpose, the project members also visually counted 18,000 tuna over 24 hours. They understood the large burden placed on the farm workers, and renewed their conviction that releasing a system which could count the fish would make the workers' jobs easier.

Times when counting the number of fish is necessary
• When catching natural small tuna (young Pacific bluefin tuna) and putting them in the nets
• When dividing the fish into multiple nets during the growth phase
• When conducting counting work for daily inventory management and managing the amount of feed used

SOLUTION

Achieving 98% or higher accuracy and reducing work burdens with functions that can be remotely operated to adjust them to the optimal imaging conditions

The automated fish counting system consists of the camera, video cable provided with the camera, dedicated image processing PC, and special software. The camera is installed at the exit of the passage net connecting one farming net to another. It captures images of the fish moving, and recognizes the count in real time. There is no other system in the world capable of real-time counting using simultaneous imaging and count recognition.

Image of how the automated fish counting system is used

This system is the first YMS product related to digital transformation. It was a new endeavor for the company and involved repeated trial and error, however the provision of valuable image data from the farmers proved to be a large foothold that allowed us to move forward with research and development. At the initial basic research stage, it became clear that the imaging conditions had a large effect on the count accuracy, and this led to the largest feature of this system: the functions which enable adjustment by remote operations in order to obtain clear images.

Image of the automated fish counting system

Before beginning imaging and counting with the camera installed underwater, these functions detect disturbances such as direct sunlight, floating debris, and obstructions that affect automated analysis, and provide alert notifications. Based on the detected information, the person performing the work can use remote operations to adjust the field of view and color to the optimal conditions. Because the number of fish is automatically counted in a clear image, this yields higher accuracy. The persons working on the farm are also freed from burdensome work such as adjusting the camera underwater and visually counting fish.
We are planning to provide two types: a real-time counting type and a type which counts using images after they were captured. It will be possible to install the type that best matches the conditions of use and user preferences.

RESULT & FUTURE

Aiming for a future of sustainable fisheries

The automated fish counting system which provides 98% or higher accuracy and was created by carefully resolving each issue one at a time is the first YMS product related to digital transformation. Research and development were carried out through a continuous series of trial and error using actual images. The project members described their satisfaction, "As we were starting from zero, I initially did not think that we would make it this far". They are planning to continue with further research into mechanisms for maintaining accuracy without human involvement, and for using the system with fish other than tuna, in order to make it a long-selling product that satisfies the customers. They are also working to improve the level of customer service, and are planning continual updates with an eye towards also deploying the system overseas.

As indicated in Sustainable Development Goal (SDG) No. 14, "Conserve and sustainably use the oceans, seas and marine resources for sustainable development", the marine field has become an area of particular importance on a global scale in recent years. There is a growing trend towards demanding resource management, and it is necessary also for tuna farming to submit accurate reports of the numbers of fish when using natural small tuna (young Pacific bluefin tuna) as the fish stock. As a result, this system can be described as necessary technology for preserving the world's marine resources.

In Japan, digitalization has not progressed as far in the marine field compared to other fields. While it is an industry that supports the food that is essential for human living, the number of people engaged in fishing and fishery work in Japan is declining. Finding ways to improve operating efficiency with a small number of people is essential in order for the fishery industry to continue into the future. Yanmar will continue to support the fishery industry and primary industries by producing smart products such as the automated fish counting system.

FUTUREBreaking through barriers to innovation with a powerful research and development system

In the technical language used in the technology management field, there are large barriers at each stage when working to commercialize the results from R&D at a company. These are known individually as the "Devil River", "Valley of Death", and "Darwinian Sea". One member described how in this project, a team of professionals from different departments including Planning, Research, Development, and Service worked together toward achieving clear goals, and were able to overcome these barriers which often get in the way when pursuing innovation.

The first barrier to innovation is known as the "Devil River". It is a barrier that prevents the transition from basic research to development, and results in a project concluding without being able to utilize the research results in development. In this project, we were able to smoothly pass the baton to development because we had a clear target of 98% or higher accuracy, and because we received video that was provided by fish farmers.
The second barrier is the "Valley of Death". It is a barrier that prevents research that has advanced to the stage of development from transitioning to conditions which would allow the sale of an actual system. In this project, we were able to bring a prototype system into the field and conduct tests under a variety of the conditions which our customers face. Through repeated machine learning, we were able to develop a system that can withstand use in the field.
In the field of UI design as well, where we had no prior experience, we were able to carry out development by cooperating with the Design Strategy Group and cooperating companies. The resulting UI was repeatedly improved based on actual use by the customers.
The third barrier is the "Darwinian Sea". This is a barrier that occurs at the phase when a product attempts to gain a share of the market after its release. It is the phase that we are now facing. We are expanding our future prospects by continually making updates and creating products that are recognized by the customers in order to overcome this barrier.

* Affiliated divisions are current as of April 2023.

SOLUTION 02 Other Solution Technology

Robot Tractor which improves efficiency and precision while keeping the burden on the workers at a minimum.
Auto navigation and docking system

Auto navigation and docking system
to reduce workloads and create a safe and secure future