Project Overview

H.O.L.L.Y. (Holographic, Obscure, Little Logical Yapper) is a productivity device that transforms any flat surface into an interactive touchscreen using computer vision and projection mapping. The system tracks hand gestures in real-time, allowing users to interact with a projected interface through simple point-and-click motions. Built with Python and open-source libraries on a Raspberry Pi, H.O.L.L.Y. demonstrates an accessible approach to hands-free computing.

Key Achievements

Optimized Computer Vision Pipeline

Engineered a computer vision pipeline that achieves 40% reduction in gesture detection latency compared to the initial implementation. The optimization focused on multi-threaded frame processing and efficient algorithms for the Raspberry Pi's ARM architecture. Implemented frame buffering to maintain smooth interactions during processing, compensating for the limited GPU support on Raspberry Pi hardware.

Lens Correction Algorithms

Implemented lens correction algorithms that compensate for distortion in the camera and projector optics. Achieved 12% improvement in tracking accuracy over baseline by calibrating the camera-projector system using checkerboard patterns and computing distortion coefficients. The correction ensures detected hand positions accurately map to projected interface elements.

Voice Assistant Integration

Integrated language models via HuggingFace to create a voice-activated assistant. Combined speech recognition, natural language processing, and text-to-speech synthesis for basic voice interactions and system control.

Technical Implementation

System Architecture

• Vision Module: OpenCV and MediaPipe Hands for hand detection and tracking
• Calibration: Camera-projector calibration for lens distortion correction
• UI Framework: Kivy-based projection interface
• Voice Assistant: Speech recognition and TTS pipeline

Computer Vision Pipeline

Hand Detection & Tracking

• MediaPipe Hands: 21 hand landmark detection
• Kalman Filtering: Smooth trajectory prediction to reduce jitter

Lens Correction

• Camera Calibration: Checkerboard-based calibration
• Distortion Correction: Brown-Conrady model for radial distortion

Gesture Recognition

• Point: Tracks midpoint between thumb and index finger for cursor control
• Click: Pinching motion (distance threshold) for button activation

Voice Assistant

• Google Speech API: Speech-to-text
• HuggingFace Transformers: Basic conversational responses
• pyttsx3: Text-to-speech synthesis

User Interface

• Virtual Desktop: Launcher with application icons
• Calculator: Virtual calculator with gesture input
• Settings: System configuration interface

Hardware Configuration

• Projector: Full-HD LED projector
• Camera: USB camera with wide-angle lens
• Microphone: USB microphone
• Speaker: Bluetooth speaker
• Processing Unit: Raspberry Pi

Performance Metrics

• Latency Reduction: 40% improvement
• Tracking Accuracy: 12% improvement with lens correction
• Response Time: ~50ms end-to-end

Learning Outcomes

• Computer vision algorithms and real-time image processing
• Camera calibration and geometric transformations
• Performance optimization for embedded systems
• Hardware-software integration