Run a <400ms latency Voice Agent on just 4GB VRAM. Fully offline, no API keys required. Optimized for GTX 1650 and edge robotics with zero-copy inference. (Apache 2.0)

NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN [IEEE TMC 2025]

Implementation of "Low-Cost and Effective Fault-Tolerance Enhancement Techniques for Emerging Memories-Based Deep Neural Networks." 2021 58th ACM/IEEE Design Automation Conference (DAC).

High-performance late-interaction retrieval engine for on-prem AI. ColBERT/ColPali multi-vector search with Rust fused MaxSim, Triton GPU kernels, ROQ quantization, LEMUR routing, WAL-backed CRUD, and a FastAPI server — single machine, CPU or GPU.

RAG with Binary Quantization for enhanced performance

Batched QLoRA fine-tuning of FLAN-T5-Large for three-way stance classification, with systematic evaluation of clustering, embedding probes, and full model inference