I’m an AI Cloud Engineer and Architect interested mostly in LLM inference, Agentic AI and Cloud.

My open-source projects include Cognitora inference (KV-aware orchestration above vLLM, SGLang, TensorRT-LLM and llama.cpp, on bare metal, Kubernetes or cloud), Frink (a pure-Rust GGUF engine, checked against llama.cpp on 100 architectures), GPUlayer (runs PyTorch workloads on any GPU you can reach, from a laptop to cloud VMs created per job), and the research repo LASA (jailbreak transfer from base models to fine-tunes).

This blog is where I write about cloud architecture, inference, GPUs, distributed systems and engineering leadership.

🕹️ Learned to code on a Commodore, and I’ve basically been hitting RUN ever since.

Certifications

NVIDIA · AI Infrastructure & Operations · Agentic AI · Gen AI LLMs
AWS · Generative AI Developer Professional · Solutions Architect Associate
Google Cloud · Professional Cloud Developer · Associate Cloud Engineer
Databricks · credentials wallet

Stack

Languages · Go Rust Python TypeScript JavaScript Bash SQL C++
Infra & DevOps · Neovim Git gitgui cmux Terraform Ansible Docker Kubernetes Helm Containerd Firecracker Cosign Prometheus etcd
AI & inference · vLLM SGLang llama.cpp TensorRT--LLM MLX Ollama Frink Qdrant HNSW FAISS cuVS OpenAI Anthropic OpenRouter
GPU & accelerators · CUDA Metal Apple Silicon NEON AVX2 SIMD GGUF
Backend & data · FastAPI Actix Tonic gRPC QUIC libp2p BitTorrent PostgreSQL MongoDB RocksDB
Web · Next.js Angular React GraphQL Tailwind CSS shadcn/ui
Cloud · AWS Google Cloud Vercel Fly.io
Security · Semgrep Bandit Gitleaks Trivy
APIs · Binance Facebook Meta LinkedIn X Pinterest Telegram

Notable Projects

  • Frink Crates.io - Pure-Rust GGUF inference engine: quantized CPU, Apple Metal and CUDA kernels, mixture-of-experts support, and an OpenAI-compatible server. Every speed claim is benchmarked head-to-head against llama.cpp on the same host and file. Write-ups: a Rust GGUF engine, measured against llama.cpp and Frink on Metal, at parity with llama.cpp and past it.

  • GPUlayer PyPI - One layer between your code and every GPU you can reach: your laptop, servers that join with gpulayer join, Kubernetes pools, or cloud VMs created per job. Runs commands, Python functions, multi-node PyTorch training and long-running services on one queue, verifies each worker’s CUDA/MPS/MLX stack before placing work, explains why a job can’t be placed, and exposes it all to coding agents (Claude Code, Pi, MCP). Docs: gpulayer.dev.

  • LASA - Lineage Attack Surface Analyzer, a defensive research harness that asks whether jailbreak evidence from an open-weight base model can predict which fine-tuned derivatives inherit the same weaknesses. Attacks are searched only on the source model, frozen, then measured on related checkpoints against attack-free, random-suffix and mismatched-suffix controls, with a committed study plan and an independent HarmBench classifier. Paper: preprint (PDF).

  • gitgui - A git GUI that runs inside the terminal, next to your coding agent. One Rust binary paints an iced interface as pixels into a cmux, Ghostty or kitty pane over the kitty graphics protocol, or opens a desktop window where the terminal has no graphics: commit graph, staging by file, hunk or line, a three-way conflict resolver, history rewriting, a built-in editor, draggable panes, and a socket so agents like Pi drive it. Write-up: gitgui: a real git GUI inside cmux, next to Pi.

  • VittoriaDB - Zero-configuration embedded vector database with HNSW indexing, ACID storage, and REST API. Single Go binary for local AI development.

  • PeerClaw - Decentralized P2P AI agent network in a single static binary. BitTorrent shape applied to inference: peers share compute, agents earn and spend tokens for it.

  • DistX Crates.io - High-performance vector database written in Rust. Features HNSW indexing with SIMD optimizations, Qdrant-compatible REST API, and gRPC support.

  • s0-cli - LLM-driven CLI agent that finds security vulnerabilities and AI-slop patterns: stub auth, hallucinated imports, dummy crypto, prompt-injection sinks. Hybrid of classic scanners (semgrep, bandit, gitleaks, trivy) with LLM triage on top, plus a self-optimizing meta-harness loop. Write-up: the meta-harness that tunes its own scanner.

  • Cognitora inference - Open-source, datacenter-scale LLM orchestration above vLLM, SGLang, TensorRT-LLM, and llama.cpp: KV-aware routing, prefill/decode disaggregation, multi-tier KV cache, static Rust binaries for bare metal, Kubernetes, or cloud. Write-ups: the orchestrator and admission control for distributed LLM inference.

  • Rondine 🐦 - Hardware-aware local LLM launcher. Detects the machine, picks a model that fits, and applies tuned configs for Mac, NVIDIA GPUs and DGX Spark instead of leaving you to do flag archaeology. Write-up: Rondine: hardware-aware local LLM serving.

  • MARS - GPU-resident multimodal memory substrate for real-time embodied AI. Episode-scoped retrieval as a CUDA kernel-level primitive: 197 µs p99 at N=1M with perfect cross-modal hit@15, 33× faster than FAISS-Flat-GPU on the same hardware. Companion paper: MARS: Episode-Scoped GPU Retrieval for Real-Time Embodied AI.

Connect

Feel free to reach out or follow my work: