George Kantor:
A perception pipeline for modeling plant geometry and mechanics (Invited Talk)

George Kantor is a Research Professor at Carnegie Mellon University’s Robotics Institute.  He has over 20 years of experience research in developing and deploying robotic technologies for real-world applications in military, agriculture, mining, and scientific exploration. His technical interests lie in position estimation and mapping for mobile robots, control of robotic systems with nontrivial dynamics, off-road autonomous driving, and deep learning for image analysis and sensor fusion “on the edge”. In the agriculture domain, his group has developed mobile robots for in-field phenotyping, perception pipelines for creating dense 3D plant models, and reinforcement learning approaches for robotic manipulation in tree canopies. 

This talk presents a robotic perception pipeline for that generates high resolution point cloud models of plants in the field, extracts plant topology from the point cloud, fits rigid cylinder primitives to the topology, then finally models tree dynamics with torsional springs and dampers between the cylinders. A proof-of-concept result is shown using dormant season grapevines. The talk concludes by discussing the potential to use the resulting dynamic models to train reinforcement learning controllers for robotic canopy interaction tasks.