Accuracy Without Wearables
Rep counts had to hold up across lighting, camera angles, and body types. A demo that only works in a studio would not survive real homes.
iOS & AI Developer
AI Fitness Challenge Platform
Arjun partnered with a fitness product team to build an iOS app that counts push-ups from the live camera and turns workouts into multiplayer challenges. The engagement focused on on-device pose detection, real-time leaderboards, and a social loop that works without wearable hardware.
About the client
Users wanted accurate rep counting and social motivation without buying extra sensors. Camera-first fitness apps often look good in a tripod demo and then fail in real lighting, angles, and body types.
Arjun implemented on-device Vision and Core ML for pose estimation and rep counting, with Firebase for real-time leaderboards and challenges.
Challenges
Battery And Thermal Limits
Processing every camera frame at full resolution would overheat the phone and kill retention. Detection needed a sustainable frame rate.
Instant Social Feedback
Multiplayer challenges and leaderboards feel broken if counts take seconds to appear. Friends finishing a set at the same moment also create write conflicts.
Privacy Expectations
Users are more willing to point a camera at themselves when processing stays on-device. Uploading workout video was not an acceptable default.
Scope Across An Exercise Library
Shipping every movement on day one would delay a stable pipeline. The product needed one reliable exercise and one social loop first.
Solutions
Arjun processed the camera feed on-device with Vision and Core ML, then synced counts and challenge state through Firestore with optimistic updates.
On-device pose estimation
Used Apple's Vision framework and Core ML to estimate pose and count reps on the phone, so video does not need to leave the device for the core experience.
Live camera preview and overlays
Built SwiftUI camera views with motion overlays so users can see what the model is tracking and calibrate form before a set starts.
Real-time multiplayer sync
Synced challenge state and scores via Firebase with optimistic UI, so leaderboards feel instant even when two people finish together.
Performance-aware detection
Downsampled the camera feed and ran detection at a sustainable FPS to protect battery life and avoid thermal throttling on longer sessions.
Firebase auth and challenge data
Used Firebase Auth and Firestore for users, rooms, and challenge history so social workouts persist across sessions.
Project tech stack
SwiftUI
Vision / Core ML
Firebase
Features
Live camera rep counting
The app counts push-ups from the camera in real time, without a wearable or a trainer in the room.
Motion detection overlays
On-screen guidance helps users stay in frame and understand why a rep was counted or rejected.
Multiplayer challenges
Friends can compete in the same challenge, with counts syncing live instead of waiting for a post-workout upload.
Leaderboards
Rankings update from Firestore so social motivation stays in the loop during the workout, not only after it.
On-device privacy
Pose estimation runs on the phone, which is both a performance choice and a product selling point.
Results
40%
Challenge engagement hike
Multiplayer challenge engagement hiked by 40% after live camera counting, rooms, and leaderboards ran in one loop instead of a solo counter with no social reason to return.
18%
User retention lift
7-day user retention improved by 18% as friends, scores, and rematches gave people a next session beyond the first workout.
38%
Higher workout completion
Workout completion rose by 38% once reps were counted automatically on-device, without wearables or manual logging.
29%
Better rep-count accuracy
Rep-count accuracy improved by 29% across varied lighting and camera angles after pose estimation ran through Vision / Core ML with on-screen calibration.
45%
Fewer heat/battery drop-offs
Session drop-off from heat and battery drain fell by 45% by downsampling the camera feed and running detection at a sustainable frame rate.