To gain experience in robotics, we built a robotic arm pick-and-place system that combines reinforcement learning with computer vision for autonomous object manipulation. Our system integrates YOLOv5 for real-time object detection with a custom convolutional neural network for scene interpretation, allowing the robotic arm to identify, localize, and manipulate objects effectively.
We used a 7-degree-of-freedom robotic arm with a two-finger gripper in a PyBullet simulation environment. What makes our approach innovative is its hybrid control architecture – the reinforcement learning policy handles high-level motion planning (approaching and aligning with objects), while programmed logic executes the critical grasp and lift phases. This combination addresses the challenge of sparse rewards in reinforcement learning while ensuring reliable execution.
Through significant performance optimization, we achieved an impressive 85× improvement in processing speed (from 1 FPS to 85 FPS), enabling training of 50,000 episodes in just 10 minutes. The result is a highly reliable system with a 100% grasp success rate that can effectively perform pick-and-place operations on multiple objects. Our project demonstrates how combining perception-driven learning with structured control logic creates a fast, stable, and scalable solution for robotic manipulation tasks.
Drawing on expertise from my computer vision internship with Wildlife Imaging Systems, I implemented and integrated the vision components of our robotic system. I developed the 3D triangulation pipeline that transforms 2D image coordinates from the YOLOv5 object detection system into precise 3D world coordinates, with an error of less than a centimeter.
I am used to working on individual projects, so my team's main issue was communication and coordination of tasks. We sometimes fell short in these areas, which caused frustration, confusion, and sometimes racing to meet deadlines.
By improving our communication practices and coordinating more effectively as a team, we successfully overcame these challenges and created a final product that made us all genuinely proud of our accomplishment.
As a result of overcoming these challenges successfully, I have gained invaluable teamwork skills that have significantly enhanced my ability to collaborate effectively on complex technical projects with team members.
To learn more about this project, please check out our final report!