This technology is a software package that processes raw images captured from a depth sensor to create a more complete representation of the scene geometry to enable robotic manipulation.
Robotic tasks such as object grasping or manipulation require precise understanding of an object’s 3D shape and location. However, current approaches for automated mapping of the shapes, sizes, and geometries of 3D objects from depth sensor images are unable to fully depict geometric scenes, especially areas containing poorly illuminated or partially obstructed objects. As such, there is a need for improved methods to render 3D objects to help robots successfully grasp objects and navigate tricky environments.
This technology is a software platform that processes raw depth sensor images to create a complete 3D representation of objects within a geometric scene. This software utilizes machine learning, in the form of a convolutional neural network, to fill in portions of objects that are missing due to poor lighting or poor orientation with respect to the sensor. Specifically, the images are processed through five separate layers to generate high-resolution completions that are suitable for many downstream robotic tasks, including grasping, manipulation, and obstacle avoidance. Consequently, this technology provides better guidance for automated systems by realizing a more accurate understanding of the 3D geometries of objects in a surrounding scene.
A prototype of this technology has been validated for geometric scene and object completion, as well as guiding robotic grasping tasks.
Patent Pending
IR CU18079
Licensing Contact: Ron Katz