About

Hi, I'm Rahul.

I'm a principal engineer working on a competitive-intelligence platform that crawls the web at scale on AWS. My days go into crawl scheduling, data pipelines, cloud security and helping teams ship distributed systems that stay up.

This site is where I write up what I learn: the patterns that held up in production, the ones that didn't, and how AI agents are changing the way we collect web data. Everything here comes from building real systems, not from summarising docs.

I also build things. The CrawlPilot Chrome extension, InstaInsights and the tools on this site started as problems I had myself. You can see them all on the products page, or start with the blog.