YOLO Pose → Hand Raise → Mouse Click
I connected a yolo framework built with a generic camera and tweak the instructions so that you can control a CAD program with your gestures. I love TinkerCAD, so of course I ran it into there. But really it could be used to control anything.
This project turns a hand poses into a real OS mouse click using a standard webcam and a YOLO pose model. Works in any application: Tinkercad, CAD tools, browsers, Notepad, anything.Â
Real-Time Gesture Control for Windows (Python + YOLO + OpenCV + PyAutoGUI)
This project turns a right-hand raise into a real OS mouse click using a standard webcam and a YOLO pose model. Works in any application: Tinkercad, CAD tools, browsers, Notepad, anything.
✅ Features
Real-time YOLO pose estimation
Detects when your right wrist rises above your right shoulder
Converts gesture → left mouse click
Debounced (no spamming)
Lightweight & runs on any modern Windows laptop
100% offline after first model download
🧠How It Works
YOLO pose models output 17 COCO keypoints:
Right shoulder index = 6
Right wrist index = 10
Gesture rule:
If wrist.y < shoulder.y − threshold → CLICK
Skeleton overlay and debugging text are shown for transparency.