Web Page Scraper via FireCrawl (Reusable Sub-Workflow)

High-Level Summary

This workflow provides a reusable web scraping sub-workflow that accepts a target URL via an incoming execution request and returns the page content converted to Markdown using the FireCrawl API. It's designed to be called from other n8n workflows or AI agents, enabling seamless integration of web content extraction into larger automations. By centralizing the scraping logic, you avoid duplicating API setup across multiple workflows.

Workflow Nodes

• Execute Workflow Trigger (No auth) — This node acts as the entry point, triggered only when another workflow calls this one via the "Execute Workflow" node. It expects incoming JSON with a query.url property containing the URL to scrape.
• FireCrawl (HTTP Request) (Header Auth) — Sends a POST request to the FireCrawl /v1/scrape endpoint. The request body is built dynamically using the expression:

  {
    "url": "{{ $json.query.url }}",
    "formats": ["markdown"]
  }
  

  Authentication uses a generic Header Auth credential. You must configure a credential with a custom header (typically Authorization or x-api-key) and the value of your FireCrawl API key.
• Edit Fields (Set) (No auth) — Extracts the scraped content from the FireCrawl response ($json.data.markdown) and maps it to a new field named response. This clean output can then be used by the calling workflow.
• Sticky Note — Provides inline documentation within the editor, explaining the expected input format ({"url": "..."}) and that this workflow is intended for reuse by AI agents and other workspaces.

Setup Instructions

1. FireCrawl Account: Sign up at firecrawl.dev and obtain an API key (usually a Bearer token).
2. n8n Credential: In n8n, create a new credential of type Header Auth. Set the header name (e.g., Authorization) and header value to Bearer YOUR_FIRECRAWL_API_KEY. If FireCrawl requires a different header (e.g., x-api-key), adjust accordingly.
3. Import the Workflow: Copy this workflow JSON into your n8n instance. Ensure the FireCrawl HTTP Request node is connected to the credential you just created.
4. Test: Use the Execute Workflow node in another workflow (or use the built-in execution panel) to send the payload:

   {
     "query": {
       "url": "https://en.wikipedia.org/wiki/Linux"
     }
   }
   

   The workflow will return the page content as Markdown.

Use Cases and Variations

• AI Agent Content Ingestion: Connect this sub-workflow to an AI agent node (e.g., OpenAI, LangChain) to let the agent fetch and analyze web pages in real time.
• Automated Web Monitoring: Trigger this workflow on a schedule or via webhook to scrape specific URLs and store the Markdown in a database or send it to Slack.
• Multi-URL Scraping: Modify the incoming payload to accept an array of URLs, then add a loop node to iterate and scrape each one.
• Output Format: Change the formats parameter in the HTTP Request body from ["markdown"] to ["html"] or ["screenshot"] to get different response types.
• Error Handling: Add an Error Trigger or IF node to handle cases where FireCrawl returns an error (e.g., invalid URL or rate limit).

{
  "id": "7DPLpEkww5Uctcml",
  "meta": {
    "instanceId": "75d76ac1fb686d403c2294ca007b62282f34c3e15dc3528cc1dbe36a827c0c6e"
  },
  "name": "get_a_web_page",
  "tags": [
    {
      "id": "7v5QbLiQYkQ7zGTK",
      "name": "tools",
      "createdAt": "2025-01-08T16:33:21.887Z",
      "updatedAt": "2025-01-08T16:33:21.887Z"
    }
  ],
  "nodes": [
    {
      "id": "290cc9b8-e4b1-4124-ab0e-afbb02a9072b",
      "name": "Execute Workflow Trigger",
      "type": "n8n-nodes-base.executeWorkflowTrigger",
      "position": [
        -460,
        -100
      ],
      "parameters": {},
      "typeVersion": 1
    },
    {
      "id": "f256ed59-ba61-4912-9a75-4e7703547de5",
      "name": "FireCrawl",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        -220,
        -100
      ],
      "parameters": {
        "url": "https://api.firecrawl.dev/v1/scrape",
        "method": "POST",
        "options": {},
        "jsonBody": "={\n \"url\": \"{{ $json.query.url }}\",\n \"formats\": [\n \"markdown\"\n ]\n} ",
        "sendBody": true,
        "sendHeaders": true,
        "specifyBody": "json",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth",
        "headerParameters": {
          "parameters": [
            {}
          ]
        }
      },
// ... truncated (copy to see full JSON)