SaaS · Small businesses · 2023 · In production

BotBrain

A no-code builder that trains an AI agent on any website to answer visitors, book appointments and capture leads.

0:34

of chats resolved without a human: 68%
visitor conversations handled: 400k
median time to a full reply: 2.1s
leads captured for businesses: 30.5k

The problem

Small businesses missed questions and leads that arrived outside office hours, and the chatbot tools they tried needed scripted flows or a developer to set up. The answers were already on their websites, but nothing could use them.

Who it’s for: Small and mid-sized businesses, from clinics and online shops to estate agents, schools and IT firms, and the visitors who reach them through their websites.
My role: AI/ML Full Stack Engineer
Team: 1 full-stack engineer, 1 CEO
Timeline: Jul 2023 – Aug 2024

Stack

AI

GPT-4o
LangGraph
LangChain
Pinecone

Backend

Python
Django
PostgreSQL
Redis
Django Channels

Integrations

Google Calendar
Calendly
Stripe
ScrapingBee

Infra

NGINX
Gunicorn
Ubuntu
Sentry
GitHub Actions

How it works

Train

Site crawler
Files and Q&A
Stealth proxy

Index

Embeddings
Pinecone namespaces

Chat

Website widget
Context builder

Agent

GPT-4o agent
Tool calls

Act

Calendar booking
CRM webhook
Human handoff

Step 01 / 05

The owner adds a website

Ten crawler threads walk the site from its sitemap, retrying blocked pages through a proxy; files, text and Q&A pairs go in alongside.

Step 02 / 05

Pages are indexed mid-crawl

Every three seconds, newly crawled pages are embedded and written to the chatbot’s own Pinecone namespace, so it can answer before the crawl ends.

Step 03 / 05

A visitor asks a question

The widget sends it with the last 20 exchanges, and the three closest knowledge chunks are added to the prompt.

Step 04 / 05

The agent decides what to do

GPT-4o answers from the site’s knowledge, or calls a tool to book a slot, share a Calendly link or alert the owner.

Step 05 / 05

The work gets done

Bookings land in Google Calendar, form leads post to the owner’s CRM webhook, and flagged chats wait for a human.

Decisions

Train while crawling

Chose Embedding pages while the crawl still runs over crawling the whole site, then embedding it.

A trainer thread indexes new pages every three seconds, so a chatbot answers from its first pages long before a 200-page crawl finishes.

Trade-off: Two threads share crawl state, and a final pass has to catch pages that land after the last tick.

A namespace per chatbot

Chose Serverless Pinecone with one namespace per chatbot over a self-hosted RediSearch index on the app server.

It keeps each customer’s knowledge apart and takes vector search off the single server that already runs the web app, Redis and workers.

Trade-off: Every search became a network hop, so I later cached the client per process to remove a 1–2s round trip.

Proxy only when blocked

Chose Direct fetches with a stealth-proxy fallback on 403, 429 or 503 over routing every page through the scraping proxy.

Pages that respond within five seconds cost nothing to fetch; only sites that block bots use paid ScrapingBee requests, capped at ten at once.

Trade-off: Blocked pages wait out the direct attempt first, and there are two fetch paths to maintain.

What I’d do next

I’d stream replies token by token from day one; the widget still waits for the full answer before showing anything.
I’d put an eval set of real visitor questions in CI before changing prompts, so every prompt branch is tested on each release.
I’d chunk pages smaller, with overlap and a relevance cut-off; today chunks run to 2,000 tokens and the top three always go in.

Results

Chats resolved without a human
Before: 0%After: 68%
Support tickets per client a month
Before: 320After: 130
Time to launch a client’s chatbot
Before: 5 daysAfter: 3 days
Customer satisfaction (CSAT)
Before: 3.4/5After: 4.4/5