New Fruit Harvesting Prototype Robot Developed--Nearly Equals Speed of Human Picker Agricultural Research Institute, Ritsumeikan University, Denso

A research team including Shinnosuke Kusaba, Chief of the Institute of Fruit Tree and Tea Science, National Agriculture and Food Research Organization (NARO), Visiting Professor Takanori Fukao of the Ritsumeikan University Research Organization of Science and Technology, and Section Leader Hideyuki Nishino of the FA Division, Denso Corporation has announced the development of a prototype robot that can harvest nine types of fruit, such as apples and pears, at almost the same speed as a human picker. They also developed a V-shaped tree appropriate for the mechanizing and automation of harvesting work. The robot can be installed on a converted golf cart capable of self driving, enabling harvesting work to take place with almost no human intervention. The robot can also carry out pesticide application and weeding, achieving a reduction of 50% labor in total. The team aims for practical application of the robot in two years. The technology is expected to make a significant contribution to the reduction of labor in fruit cultivation.

There is an increasing need to reduce labor and adopt machines in Japan's fruit industry due to aging farmers and a lack of new labor. However, fruit trees take on complex shapes in three dimensions, requiring all cultivation work to be carried out by hand, including pollination and picking.

To date, the research group has focused on the development of trees that are as flat in profile as possible so that all fruit varieties can be managed using machinery, while also developing a variety of solutions adapted to such trees, including self driving vehicles, automated pesticide spraying systems, automated mowers, and robot fruit harvesters. Meanwhile, Denso has hitherto worked on the development of robots for tomato harvesting.

For this research, the group developed a V shaped tree form that is both condusive to automated harvesting and also easier for human pickers. They then developed a prototype harvesting robot capable of harvesting fruits that do not require the use of clippers, such as apples and both Western and Japanese pears.

The prototype fruit harvesting robot combines the robot, which has two robotic arms on the left side in the direction of travel, with a container system and a self driving platform. The system assumes harvesting of apples and Western style pears (individually wrapped while still on the tree during growth), while it is able of measuring the correct harvest timing for Japanese pears to harvest only ripe fruit. The ripeness of the fruit is determined using four RGB-D cameras attached to the robot arm. These are used to recognize the bottom portion, less effected by sunlight bleaching, via deep learning to process information on the surrounding colors and data obtained via the fruit picker. The cameras are able to capture both visual images and distance measurements, allowing the system to be used for recognizing fruits ready for harvest. The prototype system was able to determine fruit ripeness with a 90% accuracy irrespective of the time of day or night.

For the actual harvesting, the fruit is gripped by a robotic hand, comprised of three soft fingers at the end of each robotic arm, and then twisted. The picked fruit is then placed on a conveyor belt which carries it to the container system at the front for packing into partitioned containers. The container is then replaced automatically when full.

The two robotic arms on the system are prevented from intersecting using an algorithm for trajectory planning, and as a result, the system can achieve a harvest rate of one fruit per 11 seconds that rivals that of a human picker.

The fruit harvesting robot and container system is pulled by a self driving platform - modified golf cart. It recognizes rows of V shaped fruit trees for automated harvesting work. Once the robot harvests all the fruit in the row of trees that its arms can reach, it turns around and proceeds back down the row, harvesting the other side. Approximately 4 meters is required for turning. The fruit cannot be harvested, however, when it grows on the far side of a branch. Each container can currently contain approximately 16 fruits, and the system can carry 4 rows of 5 containers each. Once all the containers are full, the system moves to a predetermined unloading spot where a human unloads the fruit. Under the self-driving system, the system stops automatically if hazards are found in its path. Other safety features include having the robot stop operating if a container door is open. The team says they have kept the cost of the prototype low by using many of the same components used in Denso's existing tomato harvesting robot.

In addition to harvesting, the prototype can also carry out weeding and pesticide application work, with a total expected reduction in labor of 50%.

The team will continue to carry out field tests in preparation for practical use to measure the system's durability and cost.