Agasthya Kothuri

Agasthya Kothuri About

PhoneFight

A browser sword-fighting game where the phone in your pocket is the sword.

Phone Motion sword

WindowVR

Webcam head tracking that turns a laptop screen into a window onto a 3D room.

Webcam Head-tracked window

Pixel-art hand drawing a gesture in the air

Gesture Recognition

Draw a shape in the air with a Wii Remote, and any program can react to it.

Wii Remote Gesture input

PocketWand

Turns any phone into a motion controller for Python, with nothing to install.

Phone Python controller

Wii Remote Handler

A Python package that lets a modern Windows computer talk to the Wii Remote.

Wii Remote Windows API

The nutrition mini-game: sorting foods into groups

Nutrition Games

A published study of what teens don't know about food, and a game built to teach it.

Lessons Targeted game

About

Augmenting what already exists.

I'm Agasthya Kothuri. I design and build new ways to interact with computers, using things people already own: phones, webcams and game controllers. No new hardware, nothing to install.

Email GitHub LinkedIn

All projects 06

01Game
PhoneFight
A browser sword-fighting game where the phone in your pocket is the sword.
02Prototype
WindowVR
Webcam head tracking that turns a laptop screen into a window onto a 3D room.
03Tool
Gesture Recognition
Draw a shape in the air with a Wii Remote, and any program can react to it.
04Library
PocketWand
Turns any phone into a motion controller for Python, with nothing to install.
05Library
Wii Remote Handler
A Python package that lets a modern Windows computer talk to the Wii Remote.
06Research
Nutrition Games
A published study of what teens don't know about food, and a game built to teach it.

01Close

PhoneFight

Swing the phone in your pocket like a sword to deflect arrows and slice projectiles, right in the browser.

Phone Motion sword

Play in browser ↗Source ↗

Existing
A phone and a laptop
Everyone has both, and the phone already senses its rotation hundreds of times a second.
Friction
Motion games need kit
Swinging a sword normally takes a console, a VR headset or an app install.
Augmentation
The phone becomes the controller
Scan a QR code and the browser streams the phone's gyroscope to the game over WebRTC.
Result
Slice, deflect, survive
Fast edge swings slice, flat hits deflect, and the waves keep getting harder, all in a browser tab.

Start screen with QR pairing

Wave 1: launchers ring the arena

The phone is the sword

Playing on a laptop

Gameplay

Highlights

How it works

Pairing. The game creates a room with a short code and shows a QR code. The phone opens the controller page and connects peer to peer over WebRTC, using PeerJS as the broker.

Sensor stream. The controller page reads the phone's orientation and gyroscope and streams them to the game.

Arm model. The phone only reports rotation, not position, so an arm model turns rotation into a hand position. Raising the blade lifts the hand, and holding it level and still puts it into a high guard.

Slice or deflect. There's no trained model here. The game measures the blade's speed at the point of contact: an edge moving faster than about 300°/s mid-blade slices, and anything else deflects.

Collisions. Every projectile is a capsule, and hits are checked with segment-to-segment distance over 10 substeps per frame, so fast swings never pass straight through.

Rendering. Three.js draws the arena, the sword and its trail, and sliced projectiles really split into two halves.

Built with

Three.js, WebRTC, PeerJS

Needs

A laptop or desktop browser
A phone with motion sensors

Next projectWindowVR →

02Close

WindowVR

Head tracking from a laptop webcam turns the screen into a window onto a 3D room.

Webcam Head-tracked window

Try it ↗Source ↗

Existing
A webcam and a flat screen
Nearly every laptop has both. Nobody uses the webcam while they look at the screen.
Friction
The screen ignores you
It shows the same picture wherever your head is, so 3D scenes look like flat posters.
Augmentation
Head-tracked perspective
Track the eyes, then redraw the room from exactly where they are, every frame.
Result
The screen becomes a window
Targets appear to float in front of the glass. It runs live in the browser.

Demo

Tracking overlay with distance and viewport readout

Settings for other screens and webcams

WindowVR recreates Johnny Chung Lee's famous Wii Remote head-tracking demo using nothing but a laptop's webcam.

How it works

Face tracking. MediaPipe's face landmark model finds the eyes in each webcam frame. The eye point comes from the eye corners rather than the irises, so glancing around doesn't move the view.

Distance. The on-camera gap between the eyes, together with the webcam's field of view and an average eye separation, gives your distance from the screen.

Smoothing and prediction. A One Euro filter smooths the eye point when you're still and follows closely when you move fast. Its filtered velocity predicts ahead to hide latency.

Off-axis projection. Three.js renders the room with an asymmetric camera frustum built from the eye point and the screen's physical size. That's what makes the screen behave like glass instead of a camera that swivels.

Known Mac screens and webcams are detected automatically, with adjustable settings for other hardware. No install: Three.js and MediaPipe load straight from a CDN.

Built with

MediaPipe, Three.js, One Euro filter

Needs

A laptop or desktop browser
A webcam

Next projectGesture Recognition →

03Close

Gesture Recognition

An LSTM network turns a Wii Remote into a real-time gesture controller that any program can listen to.

Wii Remote Gesture input

Source ↗

Existing
A Wii Remote
A cheap, widely owned controller with an accelerometer and a gyroscope inside.
Friction
Raw motion means nothing
Software sees streams of numbers, not "circle" or "swipe left".
Augmentation
An LSTM that reads gestures
Recorded, labelled gestures train a network that recognises nine shapes live.
Result
Gestures for any app
Predictions above 85% confidence go out over UDP, so a game or UI in any language can react.

Hold the trigger, draw a shape in the air, and a neural network recognises it and sends it to any program listening on the network. It recognises nine gestures out of the box: bolt, circle, m, swipe up, down, left and right, trash, and z.

How it works

Connect and calibrate. The remote connects through my Wii Remote Handler, flashes an LED to confirm the link, and turns on the Motion Plus gyroscope. An optional guided calibration finds the sensors' zero offsets.

Capture. Holding the B trigger records a gesture, and letting go ends it. Each sample holds acceleration on three axes, roll, pitch and yaw, and the time since the last sample.

Collect data. In Data Collection mode, each gesture is labelled in the console and saved as JSON, which builds the training set.

Train. An LSTM network learns from the recorded sequences, using data augmentation and K-fold cross-validation.

Predict. Gestures take different amounts of time, so each sequence is timestamped relative to its start and padded to a fixed length before it goes into the model.

Broadcast. A recognised gesture is sent to 127.0.0.1:4242 over UDP.

Built with

Python, LSTM, UDP

Needs

A Wii Remote with Wii Motion Plus
A Windows computer with Bluetooth

Next projectPocketWand →

04Close

PocketWand

Turn any phone into a motion controller for your own Python program, with no app to install.

Phone Python controller

Source ↗

Existing
Any phone
Every phone has motion sensors and a web browser.
Friction
Phone controllers need apps
Usually that means an app install and a shared network, and campus Wi-Fi blocks devices from talking to each other.
Augmentation
One line of Python
Room() starts a server and a tunnel and prints a QR code. The phone streams its orientation back over a WebSocket.
Result
A controller for your program
Orientation as a quaternion, a matrix or Euler angles, from several phones at once, on any network.

Try it

pip install "pocketwand[demo]"
pocketwand demo

A window opens with a QR code. Scan it, allow motion access, and the phone on screen copies yours.

Use it in your own game

import pocketwand

room = pocketwand.Room()          # starts the server and tunnel, prints a QR code
while True:
    c = room.controller(0)        # never None; an empty slot reads as disconnected
    if c.connected:
        yaw, pitch, roll = c.euler

How it works

Server and tunnel. Room() runs a small web server and opens a free Cloudflare quick tunnel to it, so the phone gets the HTTPS page iOS requires before it will share motion data.

Stream. The phone sends its orientation back over a WebSocket. Both sides only make outgoing connections, so firewalls and campus client isolation don't get in the way.

Frames. Orientation is rotated into a world frame (+X right, +Y towards your screen, +Z up). Recentring only changes heading, so tilt always matches gravity.

Built with

Python, WebSockets, Cloudflare Tunnel

Needs

Python 3.10+
A phone with motion sensors

Next projectWii Remote Handler →

05Close

Wii Remote Handler

A Python package that gives a modern Windows computer full access to the Wii Remote.

Wii Remote Windows API

Source ↗

Existing
A 2006 controller
The Wii Remote packs an accelerometer, a gyroscope, an IR camera, LEDs, a speaker and rumble.
Friction
Outdated libraries
Most Wii Remote packages no longer work on a modern Windows computer.
Augmentation
A complete Python package
Built on hidapi, it decodes the remote's packed HID reports into real-world values.
Result
The hardware layer
Full access from Python, and the base layer of my gesture recognition project.

What you can read

What you can control

I got tired of digging through outdated Wii Remote packages looking for one that still worked, so I built my own.

How it works

It's built on the hidapi library, which handles the connection between HID (Human Interface Device) hardware and a Windows PC.

To keep latency low, the Wii Remote sends its data as packed bytes, which have to be decoded into real-world values. The WiiBrew community reverse-engineered and documented almost all of this format, and their work made this package possible.

It can also read and write raw HID reports, so it can be adapted to features the package doesn't wrap yet.

Built with

Python, hidapi

Needs

Windows 10 or later
A Wii Remote

Next projectNutrition Games →

06Close

Nutrition Games

Co-authored a published study of teen nutrition knowledge, then built the game that teaches the gaps it found.

Lessons Targeted game

Read the paper ↗DOI ↗

Existing
Nutrition lessons
Students already learn about food groups at school.
Friction
Gaps nobody measured
In our study of 50 students, 42% scored below 50% and none scored above 79%.
Augmentation
A game aimed at the gaps
I analysed where understanding was weakest and built a game around exactly those concepts.
Result
Published research
Published in the International Journal of Science and Research, October 2025.

The published abstract

"Nutrition Games: A Tool to Improve Nutritional Knowledge of Children", with Dr. C. Anjali Devi.

What I did

Analysed the gaps. We assessed what 9th and 10th grade students knew about food groups, and I analysed the data to find where their understanding was weakest.

Built the game. I used those findings and my game development experience to build a gamified learning website that strengthens the concepts students struggled with.

Wrote it up. I wrote up the findings for the lead PhD researcher, and the paper was published in IJSR.

Role: Co-author and game developer
Year: 2025
Built with: Research, Game design

Next projectPhoneFight →