Video Presentation
Motivation
Often Robotic arms might struggle with interaction with the environment. Without much feedback, rather the embodiment, they may not be able to execute operations that require them to perform highly without the visual and sensory inputs.
Hence we proceeded to design and implement a haptic robotic arm that can identify and classify a preset group of objects using a computer vision-based system and apply different pressures on the designated object after locating its orientation in space.
Academic Goals:
We also wished to deepen our understanding of Sensors and Computer Vision, which led us to select this topic.
Objectives
Software Objectives
Train YOLO to identify and differentiate between the following objects Sponge Ball, Rubber Ball and Golf ball
Integrate the YOLO output to Arduino (Future Work)
Electro- Mechanical Objectives
Program servos of a robot arm to perform a set path
Output force data when picking up an object
Validate the output of an accelerometer
System Decomposition
Methodology
Sections
The electro-mechanical system includes all the work done on the physical system such as using an Arduino to collect accelerometer and pressure data and actuating the robot arm
-
- Beginner
-
- Intermediate
-
- Advanced
The software section contains of Computer Vision and identification of the object in front.
The section was divided into four sections as follows
A: Beginners
- Computer Vision basics
- Installations (Covered in sections)
- Taking Pictures for dataset Generation!
- Basic git and bash commands
B: Intermediate
- Image and Videos Processing – Implementation
- Understanding YOLO algorithm
- Creating Custom Object Dataset in Yolo format
- Label Studio and Annotations
C: Advanced
- Running Darknet framework on Jetson
- Preparing and Training your model on Jetson
- Preparing and Training your model on Google Colab
- Running YOLO with custom dataset
D: Expert(Future Work)
- The future work to be done here is to integrate the YOLO output with the Arduino. After identification of the object, the program should pass it to arduino, arduino should take the feedback and change its course of action
Conclusions and Outputs
Successful Identification of the Objects on YOLO : Images
Haptic Arm Picking Up Objects
About the Authors
This page was written by Ankitha Durvasula and Rucha Patil.
