I built an AI Tool to detect facial expressions while you watch a video!
Rohan Sawant
CEO & Founder | AI, LLM & NLP Consulting | Training, Deploying & Scaling GenAI Applications
Hemuer : Laugh when everyone laughs, smile when everyone smiles!
Live Demo Link ????
Hemeur is an AI Tool to record facial expressions of users as they watch a video and then visualize the funniest parts of it!
It watches users as they watch a video, and logs when the users smile.
Insights can be gathered from collected data!
Check it out on Github!
Architectural Overview
Working
1. Camera detects the expression of the viewer
face-api.js with TensorflowJS detects the expressions, sends a POST to the backend.
2. Write expressions to MQ
NodeJS writes the expressions to RabbitMQ
3. Write expressions to the database
Data from the MQ is now moved over to the database for storage and popped off the queue.
4. Insights are generated from the collected data
The expressions and their positions in the video are noted down, and visualized in the UI.
Tech Stack
- Frontend - face-api.js in TensorFlow.js, JavaScript and JQuery, BootStrap
- Backend - NodeJS Express, mongoose and amqplib
- Database - MongoDB
- Message Queue - RabbitMQ on CloudAMQP
- Hosting - Heroku - Free Tier
- Local Development - Docker and Docker Compose
Features
Screen jiggles when you giggle!
Chat Panel, pops every time someone laughs (or talks)
Links above the messages, allow to skip to the funny bits
1. Privacy First
Facial Recognition is done in the browser itself. No video ever leaves your device.
2. Scalable
Hemuer is powered by RabbitMQ, this adds real-time chat support! Transactions go to the MQ first, then are stored in the database.
This keeps things up and running even during high traffic!
3. Data-Driven
The Stats page displays Smiles vs their Position in Video.
This can be used to find, seek and skip to the funniest bits of the video.
4. Open Source
This is where I nag you for stars... ??
Can I get a star?
Future Scope
- A Chrome Extension for YouTube?
- A Plugin for Streaming Services to analyze how people react to their content?
- A Tool for Focus Groups and Scientific Studies?
- Something creepy straight out of 1984?
Getting Started
Prerequisites
- Docker and Docker Compose needs to be installed on your machine
How to run?
- Clone the repo from https://github.com/CT83/Hemuer-AI-Expression-Detector
- docker-compose up --build
- Visit https://localhost:3000/
- Smile. ??
Credits
- Vincent Mühler for the Face Recognition API