Project case study · Full-stack and machine learning development
ThirdEyeAI – Real-time Engagement Detection & Meeting Platform
An AI-powered learning platform combining a WebRTC meeting engine, privacy-first engagement detection, instructor analytics, and natural-language session insights.
What it is
ThirdEyeAI explores how online learning platforms can provide useful engagement signals without sending participant video to a central inference server.
The application combines a meeting experience, browser-based engagement analysis, instructor-facing analytics, and natural-language exploration of session data.
Problem and approach
Problem
Remote instructors have limited visibility into participant engagement, while conventional video-analysis systems can create privacy and infrastructure concerns.
Solution
Run engagement inference in each participant's browser and share only derived engagement information with the instructor-facing analytics experience.
How it works
Meeting session
Participants join a WebRTC-based meeting and interact through the browser.
Private inference
TensorFlow.js performs engagement inference on the client so raw video does not need to be uploaded for analysis.
Instructor insights
Derived session signals are summarized for instructors and can be explored through a retrieval-augmented natural-language workflow.
How I built it
- 1.
Model development
Developed and evaluated a hybrid ensemble using a 1D CNN and 1D ResNet with the DAiSEE dataset.
- 2.
Browser integration
Moved inference into TensorFlow.js and integrated it with the meeting experience.
- 3.
Analytics workflow
Connected engagement results to instructor visualizations and a retrieval-augmented query flow.
Challenges and decisions
Balancing useful engagement analytics with participant privacy.
Approach: Kept video processing in the browser and shared derived signals instead of raw frames.
Lesson: System architecture can make privacy a default property rather than an optional policy.
Skills and technologies
What I learned
- How client-side inference changes privacy and scaling tradeoffs.
- How to connect machine-learning output to an understandable product experience.
- How retrieval can make structured analytics accessible through natural language.
Outcomes
- 94.25%
- engagement-detection accuracy
- Client-side
- privacy-first inference