AI-Tracked PTZ Camera System
Computer Vision & PID Control
The Objective
For my onboarding at Ulysses, I developed a PID-tuned Pan-Tilt-Zoom (PTZ) camera system. The goal was to mount a camera on two servos, detect a target using a custom YOLOv5 machine learning model, and dynamically center the target. This classic control theory problem taught me about the practical application of Ziegler-Nichols PID tuning. I also learned how to train a custom YOLO model to detect arbitrary objects. A major takeaway was learning proper data collection and curation. I went with the manual approach of labeling the first few hundred images, then used the model to auto-label over 1,000 more, manually verifying them for fine-tuning.
Technical Details
The architecture involves a camera, two servos, a Raspberry Pi 5, and an Arduino Nano. I deliberately utilized Commercial Off-The-Shelf (COTS) components to enable rapid prototyping. The Raspberry Pi processes the camera inputs, runs the YOLO inference, and calculates the error between the center of the camera and the detected target. It streams this feed and transmits the error margins to the Arduino Nano over a custom UART packet protocol. The Arduino parses the error data and executes the PID control loop to drive the servos, dynamically re-centering the target and sustaining the tracking loop.
Initial version demonstration
The Benchy Target
Testing & Iteration
I later found out that this specific control problem was simple enough to solve relying heavily on the Kp term, allowing the Kd and Ki terms to be effectively zero. The main video showcases my final version, which tracks accurately using a predominantly Proportional-tuned loop.