Sync website pages to a Supabase vector store for n8n chatbots

Every chatbot recipe on this site starts with a knowledge field: paste your FAQ text in and the bot answers from it. That works up to a few thousand words. Past that, answers get vaguer because the model has more to sift through on every turn.

This recipe is the first half of the fix. It reads your website, splits the text into chunks, creates an embedding for each chunk and stores them in a Supabase table, then refreshes them weekly. The Supabase knowledge base chatbot is the second half: it searches that table instead of reading a pasted block.

When you need this

If your knowledge text is under a couple of thousand words and answers are accurate, you do not need it yet. A vector store adds infrastructure for no real gain on a small FAQ. Reach for it when:

• Answers get vaguer, or start blending two topics together.
• Your content has grown to a full docs site, a large catalogue or dozens of articles.
• You want the bot to cite where an answer came from, because each stored chunk carries its source URL.

How it works

• A manual run or a weekly schedule starts the workflow.
• It reads your sitemap, or a list of URLs you paste, and loops over the pages one at a time.
• Each page is fetched and stripped to readable text. Pages with almost no text are skipped so a bad crawl cannot wipe good chunks.
• Old chunks for that URL are deleted, the text is split into overlapping chunks, and OpenAI creates an embedding for each chunk.
• Each chunk is written to your Supabase table, and a summary reports how many pages and chunks were processed.

Ingestion and retrieval never run at the same time. If ingestion has not run, or failed, retrieval returns nothing and the bot says it does not know, rather than erroring.

What it costs

The workflow is free to download and use. OpenAI embeddings are billed per token, separately from chat completions. Embedding a typical small site costs a small amount, and each weekly refresh re-embeds every page. Supabase's own plan limits apply.

Set it up

1. Prepare Supabase. In the SQL editor, run the SQL below. It enables pgvector, creates the documents table and creates the match_documents function the chatbot calls later.
2. Import the workflow. Download the file with the button at the top of this page and import it with Workflows → Import from File.
3. Create two credentials. Create a Supabase API credential using your project URL as the host and your service role key (not the anon key; ingestion has to bypass row-level security to write). Create an OpenAI API credential with your key. If n8n asks which domains a credential may be used with, allow your Supabase project domain and api.openai.com.
4. Attach them. Attach the Supabase credential to Delete existing chunks for this URL and Upsert chunk into Supabase, and the OpenAI credential to Generate embedding.
5. Fill in Ingestion settings. Set source_type to sitemap or manual_urls, then fill in sitemap_url or manual_urls (one URL per line). Set supabase_url to your project URL. Leave table_name as documents unless you changed it. chunk_size 500 and chunk_overlap 50 are sensible defaults.
6. Run it once. Click Execute on the Click to run setup trigger. Check the Ingestion summary output for pages and chunks processed, then look at the table in Supabase: you should see rows with real text, not empty strings.
7. Turn on the schedule. Activate the workflow and the Weekly re-ingest trigger refreshes your content every week. Re-running is safe: each page's old chunks are deleted before the new ones are written, so nothing duplicates.

The SQL for step 1:

create extension if not exists vector;

create table documents (
  id bigserial primary key,
  source_url text not null,
  chunk_index int not null,
  content text not null,
  embedding vector(1536),
  metadata jsonb,
  ingested_at timestamptz default now()
);

create index on documents using hnsw (embedding vector_cosine_ops);

create or replace function match_documents (
  query_embedding vector(1536),
  match_count int default null,
  filter jsonb default '{}',
  match_threshold float default 0.75
) returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
language plpgsql
as $$
begin
  return query
  select
    documents.id,
    documents.content,
    documents.metadata,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where documents.metadata @> filter
    and 1 - (documents.embedding <=> query_embedding) > match_threshold
  order by documents.embedding <=> query_embedding
  limit match_count;
end;
$$;

Troubleshooting

Ingested pages have hardly any content

The text-stripping step is a rough pass: it removes script, style, nav, header and footer blocks, then strips remaining tags. Pages that render their content with JavaScript, or sites with unusual markup, may need that Strip to readable text node tuned. Check one page's output there before assuming the rest of the pipeline is broken.

My sitemap produced nothing, or the wrong pages

The workflow reads every <loc> entry in the sitemap you give it. A sitemap index that points to other sitemaps is not followed, so give it the child sitemap URL, or use manual_urls.

Re-running created duplicate chunks

Each page's chunks should be deleted before new ones are written. If you see duplicates, check that Delete existing chunks for this URL ran successfully. A failed delete followed by a successful insert is the usual cause.

I get a 401 from Supabase

Check the Supabase credential uses your project URL as the host and the service role key, not the anon key.

Take it further

Connect the table to a chatbot with the Supabase knowledge base chatbot. To ingest from Notion or Google Drive instead of a website, replace the fetch and parse steps; everything from the chunking step onward stays the same.