CASE STUDY

Industry: Warehouse/Logistics
Product: Helios 3D Time-of-Flight Camera
SDK: Vision Software by Florinco

  1. Home
  2. Case Studies
  3. Autonomous Forklifts Stack Cheese Boxes Using 3D Time-of-Flight

Autonomous Forklifts Stack Cheese Boxes
Using 3D Time-of-Flight

Aging cheese over a long period of time is not as easy as it may seem. Large stacks of cheese need to be stored in a place of proper temperature and humidity to allow for optimum conditions throughout their ripening process. Some cheeses are aged for a month and some for several years.

At a cheese factory in the Netherlands, 12 self-driving forklifts have the task to stack and de-stack cheese boxes in a fully automated process. The autonomous forklifts provided by Toyota Material Handling move racks of cheese 24/7 between production and ripening halls. These racks are roughly 3 meters high and can be stacked up to 3 levels. Before placing one rack on top of another, the exact position where to place it, needs to be determined.

Challenge

The automation of accurately positioning stacked boxes that have shiny surfaces and metal edges posed some challenges with 3D camera technology. Initially the forklifts were using a different 3D Time-of-Flight camera which, in the first implementation, did not work adequately due to the reflective surfaces of the metal racks. Over time the paint on the racks would wear off and newer models of racks had no paint but were galvanized. Both, however, resulted in insufficient measurement quality of the shiny metal surfaces which hindered the forklifts to precisely position and stack the boxes.

The fact that the ToF cameras also had to measure from an elevated angle, made the highly reflective surfaces an even bigger challenge because there was no alternative position for the cameras to give a better viewing angle. The 3D point clouds generated by the ToF camera needed to be analyzed, filtered and enhanced in order to generate a reliable depth image. Stereo vision cameras using projected light were not an option due to their pricing and bigger size.

Challenge: ToF cameras had to measure from an elevated angle, challenged by the highly reflective surfaces of the racks.

Solution

After evaluating all available solutions, LUCID’s Helios Time-of-Flight cameras showed the best results in the desired price range. Though the highly reflective surfaces still impacted the measurement quality, the resulting point clouds were sufficient to measure the relative racks positions. This time, two Helios 3D ToF cameras were used per forklift, which measure the relative location of the rack that the truck is holding and the rack on which it is to be placed. Based on the relative position, the fork lifter carefully stacks the racks.

Florinco, a developer of smart software solutions, has developed vision algorithms to extract certain features from the images to come up with more reliable measurements. The Helios ToF cameras were connected to an industrial PC with an Intel Atom processor, running Linux. This PC was connected to the forklift’s internal network to communicate the measurement results to the central processor of the forklift. Since the time of this system’s implementation, the 1st generation Helios camera didn’t offer IP67 rating, the ToF cameras were placed inside an autoVimation Orca housing. In the meantime, LUCID has introduced the next generation Helios2 ToF camera featuring a robust IP67-rated enclosure with lens protection and exceptional 3D depth data with sub-millimeter precision.

LUCID's first generation Helios ToF cameras were connected to the forklift’s internal network, offering the best 3D imaging results for the desired price range.

The first generation Helios ToF cameras were placed inside an autoVimation Orca housing for IP67 protection.

Conclusion

The Helios family of 3D Time of Flight cameras are an effective solution for many autonomous warehouse processes. Using the Helios camera, Florinco was able to overcome specular reflection from the metal racks to create autonomous forklifts without any need for manual control. Before the new solution was introduced, 10% of the rides had to be done manually. With the new solution, 99.9% of the rides go correctly and fully automated.