This work was completed for the lab as a team project by a group of 3.
My contribution to this work : My major contributions include development of the Vision system - Calibration, Workspace Reconstruction, Block Color and Size Detection, development of Forward and Inverse Kinematics
Developing the algorithms and software to enable a robot arm in a workstation equipped with a camera to perform various pick and place tasks
Doing so, involves developing algorithms for
Camera Calibration, Workspace Reconstruction
Blocks - Color, Size, Stacks detection
Forward and Inverse Kinematics of the robot
Path Planning for the robot trajectory
In this project we develop the higher level software stack for the ReactorX-200 robot arm to perform various pick and place operations of varying size blocks. The workspace also has a mounted RGB-D camera, using which we developed a vision system to detect the 3D location, color, size of the blocks. The two sub-systems are integrated using a state machine.
Calibration using a checkerboard
Use of APRIL Tags and SolvePnP for workspace reconstruction
Use of both RGB and HSV color spaces to achieve block color detections
Using DH Parameters for Kinematics
RRT Algorithm based Path Planning
Calibration was performed using checkerboards and Workspace reconstruction using APRIL Tags. The outcome was the generation of intrinsic and extrinsic camera matrices. The calibration process was integrated into the state machine. The used mask and the generated workspace is shown here
We developed an algorithm that fuses both RGB and HSV colorspace information to robustly detect colors of different blocks (many of which are worn out). This involved a lot of tuning to figure out the bandwidth. Also, the '0' and '1' indicate the size of the block. The position information is passed on to the plannning module.
The arm has 5 Degrees of Freedom and thus can't access any arbitrary point in space. So, we developed a two tier system wherein, the preferred means to pick a block is from above it. If that's not possible, it arm tries to pick it up from the side, as shown. We also do this after placing a block.
And every time we pick the block, we first grip the block, release, rotate arm by 90 degrees, and pick again. This helps align the block with the gripper so that the grip is perfect.
We also derive mechanisms to handle singularities and unreachable poses as detailed in our documentation of the project
We perform multiple tasks using the developed software, including sorting in the rainbow order, and stacking blocks to heights.
