A journey through code, creativity, and continuous learning.
Top projects
Shrota: Timestamp-Grounded Transcripts for Code-Mixed Audio
A pipeline that compiles multi-speaker Hindi/English lecture and podcast audio once into a frozen canonical transcript where every word carries start/end timestamps and a speaker label, then serves subtitles, cited chapter summaries and moment-level search from that single artifact. faster-whisper large-v3 with an OOM fallback ladder, a local WeSpeaker ONNX plus spectral-clustering diarizer (9.1% DER), and hybrid BM25 + BGE-M3 retrieval with cross-encoder rerank (80% Recall@5 on 60 gold questions), all sequenced to fit a 4 GB laptop GPU at 2.1 GB peak VRAM. Evaluation is fully generated and reports the misses honestly, including alignment drift and Hinglish WER.
StreamForge: Long-Form Audio to Searchable Structure
A Go control plane (ffmpeg HLS segmenting, Redis work queue with worker leases, idempotent retries) dispatching to Python GPU workers, turning an hour of multi-speaker Hinglish cricket audio into structured, citable content in ~7 minutes on one consumer GPU. faster-whisper large-v3 plus WhisperX alignment and pyannote diarization; a lexicon bias pass cut entity WER from 31% to 12%. Hybrid BM25 + dense retrieval over Qdrant with cross-encoder rerank, every answer citing a word-level timestamp and abstaining below the evidence threshold. Shipped as FastAPI microservices on Kubernetes via GitLab CI, gated by a 40-clip golden set that fails the build on a >3% regression.
HealthByte: Honest Health & Nutrition Engine
A personalized nutrition engine that classifies food with EfficientNet-B0 and reports calories as uncertainty bands, not false-precision numbers. Stores biometric time-series in SQLite, pulls activity via Google Fit OAuth, and surfaces only statistically honest correlations (Spearman with Benjamini-Hochberg FDR control). FastAPI service that picks honest over impressive.
image-N: Multimodal RAG From Scratch
A multimodal Retrieval-Augmented Generation system built in pure Python with no framework glue and no API keys. FAISS vector search, CLIP image retrieval, and local Ollama LLMs behind a FastAPI service, with every stage (chunking, embedding, retrieval, generation) readable top to bottom.
Holmes: AI-Content Provenance Detector
A misinformation and AI-content detector that fuses four independent signals (a DeBERTa-v3 text classifier, C2PA/EXIF metadata, SynthID, and image forensics) into one calibrated confidence with an uncertainty band, instead of a binary real-or-fake verdict. FastAPI plus ONNX, deployable to AWS Lambda.
NOIR: Autonomous Multimodal Robot Assistant
A mobile robot running three concurrent real-time loops (vision, voice, hardware control) in Python, with a fully local voice pipeline: openWakeWord hotword, Faster-Whisper STT on CUDA, Gemma via Ollama, and Piper TTS. ESP32 firmware on FreeRTOS drives an L298N H-bridge over TCP, with a Flutter app for live telemetry and manual control.
NyxCore: On-Device TinyML Watchdog for ROS 2
An independent ESP32-S3 watchdog that monitors a robot's DDS streams and keeps reporting even when the main compute node stalls or crashes. Runs anomaly inference in pure C with no ML runtime: a Mahalanobis distance scorer and a sub-400-byte autoencoder baked into firmware, shipped as a managed micro-ROS lifecycle node on FreeRTOS.
Path Tracer: From-Scratch C++17 and CUDA Renderer
A Monte Carlo path tracer written twice with zero external dependencies: a single-file C++17 CPU reference and a CUDA port mapping one pixel to one GPU thread. Implements Lambertian, metal, and dielectric materials from the math (Snell refraction, Schlick Fresnel), with reproducible seeded renders and a thin-lens depth-of-field camera.