Squat Form Analysis Using MediaPipe
Disclaimer: The accuracy of the squat form analysis can be affected by various factors such as camera angle, lighting conditions, occlusions, and the quality of the input video. For best results, ensure that the camera is positioned at an appropriate angle and the subject is clearly visible.
Introduction to MediaPipe
MediaPipe is an open-source framework developed by Google for building multimodal (e.g., video, audio, and sensor) machine learning pipelines. One of its solutions, MediaPipe Pose, is designed for high-fidelity body pose tracking, which can detect 33 key body landmarks in real-time.
Extraction of 33 Key Points
MediaPipe Pose detects 33 key body landmarks, each with x, y, z coordinates, and a visibility score. These landmarks represent various parts of the body, such as the nose, eyes, ears, shoulders, elbows, wrists, hips, knees, ankles, and feet.
Pose Landmarks
Image source: Official MediaPipe website
Key Landmarks for Squat Analysis
For squat analysis, we focus on the following key landmarks:
- Shoulder (left and right)
- Hip (left and right)
- Knee (left and right)
- Ankle (left and right)
These landmarks are essential for calculating the joint angles that are critical for analyzing squat form.
Calculation of Joint Angles
Knee Angle
The knee angle is formed by the hip, knee, and ankle. It is calculated using the arctangent function.
Example Calculation:
To calculate the knee angle, we use the coordinates of the hip, knee, and ankle:
- Hip (H): (x1, y1)
- Knee (K): (x2, y2)
- Ankle (A): (x3, y3)The formula to calculate the angle is:
\(θ knee = π 180 ×(arctan2(y3−y2,x3−x2)−arctan2(y1−y2,x1−x2)) \)If the calculated angle exceeds 180 degrees, it is adjusted:
\(if θ knee >180 then θ knee =360−θ knee \)
Hip Angle
The hip angle is formed by the shoulder, hip, and knee. It is calculated similarly.
Example Calculation:
To calculate the hip angle, we use the coordinates of the shoulder, hip, and knee:
- Shoulder (S):(x1, y1)
- Hip (H):(x2, y2)
- Knee (K):(x3, y3)The formula to calculate the angle is:
If the calculated angle exceeds 180 degrees, it is adjusted:
Back Angle
The back angle is formed by the shoulder, hip, and ankle. It is calculated as:
Example Calculation:
To calculate the back angle, we use the coordinates of the shoulder, hip, and ankle:
- Shoulder (S): (x1, y1)
- Hip (H):(x2, y2)
- Ankle (A):(x3, y3)The formula to calculate the angle is:
\( \theta_{\text{back}} = \left| \frac{180}{\pi} \times \left( \arctan2(y3 — y2, x3 — x2) — \arctan2(y1 — y2, x1 — x2) \right) \right| \)If the calculated angle exceeds 180 degrees, it is adjusted:
\( \text{if } \theta_{\text{back}} > 180 \text{ then } \theta_{\text{back}} = 360 — \theta_{\text{back}} \)
Form Issue Detection
Ideal Angle Ranges
To determine if the squat form is correct, the calculated angles are compared to predefined ideal ranges:
- Knee Angle: 90 to 110 degrees
- Hip Angle: 80 to 100 degrees
- Back Angle: 70 to 90 degrees
Detection of Deviations
If the calculated angles fall outside these ideal ranges, form issues are detected:
- Knees Bending Too Much: Knee angle < 90 degrees
- Hips Bending Too Much: Hip angle < 80 degrees
- Back Leaning Too Much: Back angle < 70 degrees
Providing Suggestions
Based on the detected form issues, specific suggestions are provided to improve squat form. These suggestions are aimed at correcting the deviations and ensuring proper technique.
Example Suggestions
1. Knees Bending Too Much:
— Issue: Knee angle < 90 degrees
— Suggestion: “Watch your knee bend. Ensure your knees do not bend excessively and maintain a proper alignment with your toes.”
2. Hips Bending Too Much:
— Issue: Hip angle < 80 degrees
— Suggestion: “Keep your hips higher. Avoid excessive hip bending by maintaining a more upright posture.”
3. Back Leaning Too Much:
— Issue: Back angle < 70 degrees
— Suggestion: “Maintain a straighter back. Focus on keeping your back straight and avoid leaning forward excessively.”
Conclusion
By extracting key body landmarks using MediaPipe Pose, calculating joint angles, and comparing them to ideal ranges, the application can effectively detect form issues during squats. Based on these detections, specific suggestions are provided to help users improve their squat technique. The final workout report offers a comprehensive summary of the performance and areas for improvement, ensuring users can make informed adjustments to their training regimen.
References
- MediaPipe Pose Documentation
- Google AI Edge — MediaPipe Pose Landmarker
Learnings from Squat Form Analysis App Development
1. Resource Management: Streamlit apps running on CPU can face resource constraints with heavy models like MediaPipe. Implementing caching strategies, such as using Streamlit’s `@st.cache_resource` decorator, optimizes resource usage and improves performance.
2. Performance Optimization: Repeated model initialization can slow down the app. Caching the pose detection model ensures it is initialized only once and reused across frames, enhancing overall stability and preventing memory overload.
Key Learning: Caching the MediaPipe Model
One of the most significant learnings was the importance of caching the MediaPipe pose detection model. This can be implemented using Streamlit’s `@st.cache_resource` decorator:
@st.cache_resourcedef load_pose_model(): mp_pose = mp.solutions.pose return mp_pose.Pose(static_image_mode=False, min_detection_confidence=0.5, min_tracking_confidence=0.5)
# Use the cached model in your main app
pose_model = load_pose_model()By caching the model, we ensure that:
1. The model is initialized only once, reducing computational overhead.
2. The same model instance is reused across different runs of the app.
3. Resource usage is optimized, preventing frequent out-of-memory issues.
This approach significantly improves the app’s performance and stability, especially when processing multiple frames or handling continuous video input.
Thank you for reading this far. Happy learning!
#ComputerVision #FitnessTech #AI #MediaPipe #SquatForm #MachineLearning