Making a voice-based browser friendly game

I spent last weekend coding up a game: an Indian version of Flappy Bird called Chidia Udd.

The catch?

I can't write frontend to save my life, so I had to navigate on two fronts: tackling React(frontend framework) side while building the game logic in parallel.

Hurdle #1: Game Technicalities

Game physics is important. The gravity, bird speed, pipe speed and jump strength all should be really fine tuned to make your game not stuck in a limbo.

Hurdle #2: WebSpeech API

The game is voice-based, operating on limited specific commands that move the bird or change its avatar. The Web Speech API wasn't ideal for real-time recognition in my use case, as players would repeat the same command. I couldn't afford to drop any events as it affects gameplay. I fetched interim events without waiting for players to complete their sentences, resulting in duplicate events.

A temporary solution was to throttle commands and cap them to a maximum threshold for a given time interval. So even if you say "udd" three times within half a second, it's considered one event. (h/t: Nikhil Srivastava)

Hurdle #3: Distribution

Getting traction for a game which people have iterated or played over multiple times is tough. To tackle this i built public leaderboard where players can claim bragging rights for their scores. A fun message is auto-generated that you can share on Twitter, tagging the user whose score you just beat. This definitely helped the game gain traction on Twitter.

Hurdle #4: User Feedback

Taking player feedback and introducing touch/keyboard mode. Game's difficulty increases as the mode is changed to keyboard/touch.

Results:

As of now the game as around 40 players, which i feel is really really cool.

Next steps:

I really want to deprecate keyboard/touch mode and make this a voice-based game only. For that i need to find a very low latency , mobile browser friendly, audio transcribing model.

I will be putting my deep learning skills to use by building a small scale real-time audio transcription model from scratch. It'll learn to identify a very limited set of words.

Ensuring the model is small enough to work well with mobile browsers.

Game link: Play Chidia Udd!

You can comment your scores below haha!

Below are some of the interim iterations of the game.