AI-Powered Competitor Price RAG Pipeline with Logging

Overview

This workflow implements a Retrieval-Augmented Generation (RAG) pipeline for processing and analyzing competitor pricing data. It is triggered via a webhook (e.g., by an external scraper), splits incoming text into chunks, generates embeddings using OpenAI, stores them in a Supabase vector store, and then allows an Anthropic-powered AI agent to query that context. Results are appended to a Google Sheet for record-keeping, and any errors trigger a Slack alert to a designated channel.

This setup is ideal for teams that need to automate the ingestion of structured or unstructured competitor data, enable natural language queries against that data, and maintain an audit log—all without manual intervention.

Node-by-Node Breakdown

• Sticky Note — A visual note in the editor (not executed).
• Webhook Trigger (No auth) — Listens for incoming POST requests at the path /webhook/competitor-price-scraper. This is the entry point; an external service (e.g., a custom scraper) should send data here.
• Text Splitter (No auth) — Splits the incoming text into chunks of 400 characters with 40 character overlap. This ensures the embeddings are meaningful and context windows are manageable.
• Embeddings (API Key auth — OpenAI) — Uses the text-embedding-3-small model to convert each text chunk into a vector embedding. Requires an OpenAI API key.
• Supabase Insert (API Key auth — Supabase) — Inserts the generated embeddings into the Supabase vector store under the index competitor_price_scraper. This stores the data for future retrieval.
• Supabase Query (API Key auth — Supabase) — Queries the same Supabase vector index to retrieve relevant chunks based on a question or input from the agent.
• Vector Tool (No auth) — Wraps the Supabase query as a tool that the AI agent can call to fetch contextual data.
• Window Memory (No auth) — Provides conversational memory (buffer window) so the agent can maintain context across multiple interactions.
• Chat Model (API Key auth — Anthropic) — The language model that powers the AI agent. Uses Anthropic’s Claude model (selected via credentials). Requires an Anthropic API key.
• RAG Agent (No auth) — The core AI agent that receives the incoming data, uses the vector tool to retrieve relevant context, and generates a response using the chat model. The system message is set to "You are an assistant for Competitor Price Scraper".
• Append Sheet (OAuth2 — Google Sheets) — Appends the agent’s response (from the RAG Agent node) to a Google Sheet named Log. You must specify the Google Sheet ID and sheet name in the node parameters.
• Slack Alert (OAuth2 — Slack) — Triggered only on error from the RAG Agent. Sends a message to the #alerts channel with the error details.

Setup Instructions

1. Accounts & Credentials – You will need:
   • An OpenAI account (for embeddings). Create an API key and add it in n8n as an OpenAI credential.
   • An Anthropic account (for the chat model). Create an API key and add it as an Anthropic credential.
   • A Supabase project. Enable the pgvector extension, create a table for embeddings, and get the Supabase URL + anon/public API key. Add these as a Supabase credential in n8n.
   • A Google Cloud project with the Sheets API enabled. Generate OAuth2 credentials (or use a service account) and add them as a Google Sheets OAuth2 credential in n8n.
   • A Slack workspace. Create a Slack app with chat:write permission and install it to your workspace. Add the bot token as a Slack OAuth2 credential in n8n.
2. Configure the Webhook – The webhook path is /webhook/competitor-price-scraper. You can change this as needed. Expose this endpoint (e.g., via n8n’s public URL or a tunnel) so your scraper can POST data to it.
3. Set the Google Sheet ID and sheet name – In the Append Sheet node, replace SHEET_ID with the actual ID of your Google Sheet and ensure the sheet is named Log (or update the parameter).
4. Adjust Slack Channel – In the Slack Alert node, set the channel name (default #alerts) to one that exists in your Slack workspace.
5. Test the workflow – Send a sample POST request with some competitor price text to the webhook URL. Check the Google Sheet for logged responses and test error handling by sending malformed data.

Use Cases & Variations

• Competitive Intelligence – Automate the ingestion of price lists, product descriptions, or news articles from competitors. Use the AI agent to answer questions like “Which competitor has the lowest price on product X?”.
• Document Q&A – Adapt this pipeline for any document type (PDFs, support tickets, internal memos). Replace the webhook with a file trigger and use the same RAG flow.
• Multi-Source Ingestion – Add an HTTP Request node before the Text Splitter to fetch data from an API or RSS feed instead of relying on a webhook.
• Custom Output – Instead of a Google Sheet, send the agent’s response via email, SMS, or directly into a CRM.
• Scheduled Runs – Swap the Webhook Trigger for a Schedule Trigger to run the ingestion on a regular basis (e.g., daily).

{
  "name": "Competitor Price Scraper",
  "nodes": [
    {
      "parameters": {
        "content": "Automated workflow: Competitor Price Scraper",
        "height": 530,
        "width": 1100,
        "color": 5
      },
      "id": "3658f18f-6ed3-4c14-b12d-9ec7fadc03f9",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        -480,
        -240
      ]
    },
    {
      "parameters": {
        "httpMethod": "POST",
        "path": "competitor-price-scraper"
      },
      "id": "9b5a4057-4b5f-44f7-aea9-847d23a8f821",
      "name": "Webhook Trigger",
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 1,
      "position": [
        -300,
        0
      ]
    },
    {
      "parameters": {
        "chunkSize": 400,
        "chunkOverlap": 40
      },
      "id": "3b015342-843e-40e8-8164-734611f2c7aa",
      "name": "Text Splitter",
      "type": "@n8n/n8n-nodes-langchain.textSplitterCharacterTextSplitter",
      "typeVersion": 1,
      "position": [
        -130,
        0
      ]
    },
    {
      "parameters": {
        "model": "text-embedding-3-small",
// ... truncated (copy to see full JSON)