Mohamed Eyad
  1. Projects
  2. Collaborative assembly

BSc graduation project · 2023 · Grade A+

Human-robot collaborative assembly with an ABB IRB 120

A low-cost human-robot collaboration cell for industrial assembly: the robot finds and handles parts while a person works next to it, and re-plans around them.

  • WhereRobotics Lab, Ain Shams University, Cairo
  • RobotABB IRB 120 with a custom end effector
  • Perception2 × Intel RealSense D435i · YOLOv7
  • GradeA+
Demo video

Overview

The goal was a human-robot collaboration cell that is affordable to build and safe to share: the robot detects and handles the parts of an assembly such as a gearbox, while a person works in the same space.

ABB IRB 120 arm with a blue custom gripper above an assembly table, with live part and number detection on screen.
The assembly cell: ABB IRB 120 with the custom end effector, and live part detection on screen.

How it works

  • Robot control: the ABB IRB 120 is driven from ROS through its EGM and RWS interfaces, with MoveIt for trajectory planning and a custom end effector for part handling.
  • Perception: two Intel RealSense D435i cameras. A custom YOLOv7 detector recognizes gears, bolts and nuts (97% accuracy), and circle and Hough-line detection align the parts.
  • Safety: MediaPipe human-pose estimation and an OctoMap of the workspace let the robot avoid people and re-plan its path on the fly.
  • Interaction: voice commands (Pocketsphinx) to start and stop the robot, and a Tkinter GUI for monitoring and control.
RViz view: the robot and coloured OctoMap voxels where people stand next to the table
OctoMap of the people in the workspace, used for obstacle avoidance and re-planning
Model of the ABB IRB 120 with the red custom end effector
Robot model with the custom end effector

Demo

Collaborative assembly demo · Open in Google Drive

Tech stack

  • ROS
  • MoveIt
  • ABB EGM / RWS
  • Python
  • YOLOv7
  • OpenCV
  • MediaPipe
  • OctoMap
  • Intel RealSense D435i
  • Pocketsphinx
  • Tkinter