n8nauto
Get Started

API Schema Extractor

Automatically extract API schemas and operations from web documentation with this powerful n8n workflow. This manual trigger workflow orchestrates a sophisticated process, beginning with a Web Search For API Schema using httpRequest to locate relevant documentation. It then Scrapes Webpage Contents via another httpRequest, processes the information into a list, and intelligently splits the text into manageable chunks using the Recursive Character Text Splitter1. These chunks are loaded by the Default Data Loader and then evaluated by the Has API Documentation? text classifier, with successful runs proceeding to embed the content using Embeddings Google Gemini and store it in Qdrant. The workflow further leverages Google Gemini Chat Model and a second Embeddings Google Gemini node to Extract API Operations, performing a Search in Relevant Docs within Qdrant to gather comprehensive details. This entire process is designed to help developers, data scientists, and businesses quickly understand and integrate with new APIs by automating the tedious and time-consuming task of manually sifting through documentation, saving significant development time and accelerating project timelines. The extracted information can then be used for various purposes, including generating API clients, creating internal documentation, or populating API directories, all without writing a single line of code for the extraction process itself.

88 nodesmanual trigger208 views0 copiesOther
Google DriveQdrantGoogle Sheets

Workflow JSON

{"nodes": [{"id": "2498bb93-176f-458c-acee-f541859df770", "name": "When clicking \u2018Test workflow\u2019", "type": "n8n-nodes-base.manualTrigger", "position": [2460, 2820], "parameters": {}, "typeVersion": 1}, {"id": "c08bcf84-9336-44f9-b452-0c9469f18f48", "name": "Web Search For API Schema", "type": "n8n-nodes-base.httpRequest", "onError": "continueRegularOutput", "position": [3100, 3820], "parameters": {"url": "https://api.apify.com/v2/acts/serping~fast-google-search-results-scraper/run-sync-get-dataset-items", "method": "POST", "options": {}, "sendBody": true, "authentication": "genericCredentialType", "bodyParameters": {"parameters": [{"name": "searchTerms", "value": "={{\n[\n `site:${$json.data.url.replace(/^http[s]:\\/\\//, '').replace(/\\/$/, '').replace('www.', '')} \"${$json.data.service}\" api developer (intext:reference OR intext:resource) (-inurl:support OR -inurl:help) (inurl:api OR intitle:api) -filetype:pdf`\n]\n}}"}, {"name": "resultsPerPage", "value": "={{ 10 }}"}]}, "genericAuthType": "httpHeaderAuth"}, "typeVersion": 4.2}, {"id": "d5b19e3a-acd0-4b06-8d77-42de1f797dba", "name": "Scrape Webpage Contents", "type": "n8n-nodes-base.httpRequest", "position": [3940, 3720], "parameters": {"url": "https://api.apify.com/v2/acts/apify~web-scraper/run-sync-get-dataset-items", "options": {"batching": {"batch": {"batchSize": 2, "batchInterval": 30000}}}, "jsonBody": "={\n \"startUrls\": [\n {\n \"url\": \"{{ $json.source.link }}\",\n \"method\": \"GET\"\n }\n ],\n \"breakpointLocation\": \"NONE\",\n \"browserLog\": false,\n \"closeCookieModals\": false,\n \"debugLog\": false,\n \"downloadCss\": false,\n \"downloadMedia\": false,\n \"excludes\": [\n {\n \"glob\": \"/**/*.{png,jpg,jpeg,pdf}\"\n }\n ],\n \"headless\": true,\n \"ignoreCorsAndCsp\": false,\n \"ignoreSslErrors\": false,\n \n \"injectJQuery\": true,\n \"keepUrlFragments\": false,\n \"linkSelector\": \"a[href]\",\n \"maxCrawlingDepth\": 1,\n \"maxPagesPerCrawl\": 1,\n \"maxRequestRetries\": 1,\n \"maxResultsPerCrawl\": 1,\n \"pageFunction\": \"// The function accepts a single argument: the \\\"context\\\" object.\\n// For a complete list of its properties and functions,\\n// see https://apify.com/apify/web-scraper#page-function \\nasync function pageFunction(context) {\\n\\n await new Promise(res => { setTimeout(res, 6000) });\\n // This statement works as a breakpoint when you're trying to debug your code. Works only with Run mode: DEVELOPMENT!\\n // debugger; \\n\\n // jQuery is handy for finding DOM elements and extracting data from them.\\n // To use it, make sure to enable the \\\"Inject jQuery\\\" option.\\n const $ = context.jQuery;\\n const title = $('title').first().text();\\n\\n // Clone the body to avoid modifying the original content\\n const bodyClone = $('body').clone();\\n bodyClone.find('iframe, img, script, style, object, embed, noscript, svg, video, audio').remove();\\n const body = bodyClone.html();\\n\\n // Return an object with the data extracted from the page.\\n // It will be stored to the resulting dataset.\\n return {\\n url: context.request.url,\\n title,\\n body\\n };\\n}\",\n \"postNavigationHooks\": \"// We need to return array of (possibly async) functions here.\\n// The functions accept a single argument: the \\\"crawlingContext\\\" object.\\n[\\n async (crawlingContext) => {\\n // ...\\n },\\n]\",\n \"preNavigationHooks\": \"// We need to return array of (possibly async) functions here.\\n// The functions accept two arguments: the \\\"crawlingContext\\\" object\\n// and \\\"gotoOptions\\\".\\n[\\n async (crawlingContext, gotoOptions) => {\\n // ...\\n },\\n]\\n\",\n \"proxyConfiguration\": {\n \"useApifyProxy\": true\n },\n \"runMode\": \"PRODUCTION\",\n \n \"useChrome\": false,\n \"waitUntil\": [\n \"domcontentloaded\"\n ],\n \"globs\": [],\n \"pseudoUrls\": [],\n \"proxyRotation\": \"RECOMMENDED\",\n \"maxConcurrency\": 50,\n \"pageLoadTimeoutSecs\": 60,\n \"pageFunctionTimeoutSecs\": 60,\n \"maxScrollHeightPixels\": 5000,\n \"customData\": {}\n}", "sendBody": true, "sendQuery": true, "specifyBody": "json", "authentication": "genericCredentialType", "genericAuthType": "httpQueryAuth", "queryParameters": {"parameters": [{"name": "memory", "value": "2048"}]}}, "typeVersion": 4.2}, {"id": "5853ba7e-4068-4792-be5c-b8cf81ee89cb", "name": "Results to List", "type": "n8n-nodes-base.splitOut", "position": [3460, 3720], "parameters": {"options": {}, "fieldToSplitOut": "origin_search.results"}, "typeVersion": 1}, {"id": "8ed2e8ec-b2e3-474b-b19d-f38b518f274b", "name": "Recursive Character Text Splitter1", "type": "@n8n/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitter", "position": [5800, 4020], "parameters": {"options": {}, "chunkSize": 4000}, "typeVersion": 1}, {"id": "e2a8137b-7da3-4032-bca2-c14465356f02", "name": "Content Chunking @ 50k Chars", "type": "n8n-nodes-base.set", "position": [5380, 3740], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "7753a4f4-3ec2-4c05-81df-3d5e8979a478", "name": "=data", "type": "array", "value": "={{ new Array(Math.round($json.content.length / Math.min($json.content.length, 50000))).fill('').map((_,idx) => $json.content.substring(idx * 50000, idx * 50000 + 50000)) }}"}, {"id": "7973bcb4-f239-4619-85fc-c76e20386375", "name": "service", "type": "string", "value": "={{ $json.service }}"}, {"id": "b46e44bc-ad01-4cf0-8b07-25eeb1fb5874", "name": "url", "type": "string", "value": "={{ $json.url }}"}]}}, "typeVersion": 3.3}, {"id": "6ef5866a-d992-4472-9221-27efbec8e7be", "name": "Split Out Chunks", "type": "n8n-nodes-base.splitOut", "position": [5540, 3740], "parameters": {"include": "allOtherFields", "options": {}, "fieldToSplitOut": "data"}, "typeVersion": 1}, {"id": "5e43b4d8-cebf-43ed-866d-0b4cb2997853", "name": "Default Data Loader", "type": "@n8n/n8n-nodes-langchain.documentDefaultDataLoader", "position": [5800, 3900], "parameters": {"options": {"metadata": {"metadataValues": [{"name": "service", "value": "={{ $json.service }}"}, {"name": "url", "value": "={{ $json.url }}"}]}}, "jsonData": "={{ $json.data }}", "jsonMode": "expressionData"}, "typeVersion": 1}, {"id": "d4b34767-be50-44ee-b778-18842034c276", "name": "Set Embedding Variables", "type": "n8n-nodes-base.set", "position": [4980, 3580], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "4008ae44-7998-4a6f-88c9-686f8b02e92b", "name": "content", "type": "string", "value": "={{ $json.body }}"}, {"id": "f7381ac6-ef40-463c-ad2b-df2c31d3e828", "name": "service", "type": "string", "value": "={{ $('EventRouter').first().json.data.service }}"}, {"id": "7eae99fd-75c7-4974-a128-641b8ada0cc2", "name": "url", "type": "string", "value": "={{ $json.url }}"}]}}, "typeVersion": 3.4}, {"id": "109b6c3a-9b16-40cc-9186-5045df387b52", "name": "Execute Workflow Trigger", "type": "n8n-nodes-base.executeWorkflowTrigger", "position": [2420, 4200], "parameters": {}, "typeVersion": 1}, {"id": "31556ff2-6358-4bd4-8ec4-2797d993256e", "name": "Execution Data", "type": "n8n-nodes-base.executionData", "position": [2620, 4200], "parameters": {"dataToSave": {"values": [{"key": "eventType", "value": "={{ $json.eventType }}"}, {"key": "executedById", "value": "={{ $json.executedById }}"}, {"key": "service", "value": "={{ $json.data.service }}"}]}}, "typeVersion": 1}, {"id": "b65b3d4d-f667-4f8f-a06f-847c3d7b83e0", "name": "EventRouter", "type": "n8n-nodes-base.switch", "position": [2800, 4200], "parameters": {"rules": {"values": [{"outputKey": "research", "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"operator": {"type": "string", "operation": "equals"}, "leftValue": "={{ $json.eventType }}", "rightValue": "research"}]}, "renameOutput": true}, {"outputKey": "extraction", "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "5418515e-ef6a-42e0-aeb9-8d0d35b898ca", "operator": {"name": "filter.operator.equals", "type": "string", "operation": "equals"}, "leftValue": "={{ $json.eventType }}", "rightValue": "extract"}]}, "renameOutput": true}, {"outputKey": "generate", "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "0135165e-d211-44f3-92a4-a91858a57d99", "operator": {"name": "filter.operator.equals", "type": "string", "operation": "equals"}, "leftValue": "={{ $json.eventType }}", "rightValue": "generate"}]}, "renameOutput": true}]}, "options": {}}, "typeVersion": 3.2}, {"id": "541f7d9b-c8ff-44dc-8618-8550dbf0b951", "name": "Google Gemini Chat Model", "type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini", "position": [4460, 3740], "parameters": {"options": {}, "modelName": "models/gemini-1.5-flash-latest"}, "typeVersion": 1}, {"id": "617d6139-8417-4ecb-8f7c-558cd1c38ac3", "name": "Successful Runs", "type": "n8n-nodes-base.filter", "position": [4100, 3720], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "cac77cce-0a5c-469e-ba80-9fb026f04b18", "operator": {"type": "string", "operation": "exists", "singleValue": true}, "leftValue": "={{ $json.body }}", "rightValue": ""}]}}, "typeVersion": 2.2, "alwaysOutputData": true}, {"id": "1115db69-b414-46cd-a9a1-565ae98cbd91", "name": "For Each Document...", "type": "n8n-nodes-base.splitInBatches", "position": [5180, 3580], "parameters": {"options": {}}, "typeVersion": 3}, {"id": "3f0e3764-2479-4d74-aca8-c3e830eac423", "name": "Embeddings Google Gemini", "type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini", "position": [5680, 3900], "parameters": {"modelName": "models/text-embedding-004"}, "typeVersion": 1}, {"id": "87d42766-d1a2-406d-b01c-044fd2fc8910", "name": "Has API Documentation?", "type": "@n8n/n8n-nodes-langchain.textClassifier", "position": [4460, 3580], "parameters": {"options": {"fallback": "discard"}, "inputText": "={{\n$json.body\n .replaceAll('\\n', '')\n .substring(0, 40000)\n}}", "categories": {"categories": [{"category": "contains_api_schema_documentation", "description": "True if this document contains REST API schema documentation or definitions"}]}}, "typeVersion": 1, "alwaysOutputData": true}, {"id": "55939b49-d91c-42a1-9770-48cbe4008c9a", "name": "Store Document Embeddings", "type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant", "position": [5700, 3740], "parameters": {"mode": "insert", "options": {}, "qdrantCollection": {"__rl": true, "mode": "id", "value": "={{ $('EventRouter').first().json.data.collection }}"}}, "typeVersion": 1}, {"id": "3e1da749-b8b9-42cb-818b-eabf4b114abb", "name": "Embeddings Google Gemini1", "type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini", "position": [3700, 4520], "parameters": {"modelName": "models/text-embedding-004"}, "typeVersion": 1}, {"id": "be0906d4-351f-4b3b-9f32-8e5ee68083c5", "name": "Google Gemini Chat Model1", "type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini", "position": [4600, 4240], "parameters": {"options": {}, "modelName": "models/gemini-1.5-pro-002"}, "typeVersion": 1}, {"id": "886415d5-c888-4b97-9fb5-02e6a14df4cc", "name": "Extract API Operations", "type": "@n8n/n8n-nodes-langchain.informationExtractor", "position": [4600, 4100], "parameters": {"text": "={{ $json.documents }}", "options": {"systemPromptTemplate": "=You have been given an extract of a webpage which should contain a list of web/REST api operations.\nStep 1. Extract all REST (eg. GET,POST,PUT,DELETE) API operation endpoints from the page content and generate appropriate labels for the resource, operation, description, method for each.\n* \"resource\" refers to the API group, for example: \"/v1/api/indicators/list\" and \"/v1/api/indicators/create\" will both have the resource name of \"indicators\". Use the following template \"<domain>\" eg. \"entities\", \"posts\", \"credentials\".\n* \"operation\" refers to the action performed, use the following template \"<verb> <entity>\" eg. \"List entities\", \"Create post\", \"Update credentials\"\n* only use one HTTP verb for \"method\"\n* \"description\" should be limited to one sentence.\n* Examples of API urls: \"/api/\", \"/api/v1/\", \"/v1/api\". API urls should not end with \"htm\" or html\".\n* Extract a maximum of 15 endpoints.\n* If the page content contains no api operations, return an empty array."}, "schemaType": "manual", "inputSchema": "{\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"resource\": { \"type\": \"string\" },\n \"operation\": { \"type\": \"string\" },\n \"description\": { \"type\": \"string\" },\n \"url\": { \"type\": \"string\" },\n \"method\": { \"type\": \"string\" },\n \"documentation_url\": { \"type\": \"string\" }\n }\n }\n}"}, "typeVersion": 1}, {"id": "76470e34-7c1f-44ce-81e2-047dcca3fa32", "name": "Search in Relevant Docs", "type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant", "position": [3700, 4380], "parameters": {"mode": "load", "topK": 5, "prompt": "={{ $json.query }}", "options": {"searchFilterJson": "={{\n{\n \"must\": [\n {\n \"key\": \"metadata.service\",\n \"match\": {\n \"value\": $('EventRouter').first().json.data.service\n }\n }\n ]\n}\n}}"}, "qdrantCollection": {"__rl": true, "mode": "id", "value": "={{ $('EventRouter').first().json.data.collection }}"}}, "typeVersion": 1}, {"id": "49ca6a35-5b89-4ed5-bbab-250e09b4222f", "name": "Wait", "type": "n8n-nodes-base.wait", "position": [3780, 3160], "webhookId": "e9ad3ef0-7403-4e65-b0a4-4afdfb0cbc6d", "parameters": {"amount": 0}, "typeVersion": 1.1}, {"id": "800cb05b-f5d1-47c8-869e-921915929f34", "name": "Remove Dupes", "type": "n8n-nodes-base.removeDuplicates", "position": [3780, 3720], "parameters": {"compare": "selectedFields", "options": {}, "fieldsToCompare": "source.link"}, "typeVersion": 2}, {"id": "d8203c40-aa0b-44b9-8dfd-aea250c8d109", "name": "Filter Results", "type": "n8n-nodes-base.filter", "position": [3620, 3720], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "42872456-411b-4d86-a9dd-b907d001ea1c", "operator": {"name": "filter.operator.equals", "type": "string", "operation": "equals"}, "leftValue": "={{ $json.type }}", "rightValue": "normal"}]}}, "typeVersion": 2.2}, {"id": "5714dc09-fd67-4285-9434-ac97cd80dec1", "name": "Research", "type": "n8n-nodes-base.executeWorkflow", "onError": "continueErrorOutput", "position": [3460, 2980], "parameters": {"mode": "each", "options": {"waitForSubWorkflow": true}, "workflowId": {"__rl": true, "mode": "id", "value": "={{ $workflow.id }}"}}, "typeVersion": 1.1}, {"id": "2a2d3271-b0b6-4a1a-94e1-9b01399ba88f", "name": "Has Results?", "type": "n8n-nodes-base.if", "position": [3280, 3820], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "1223d607-45a8-44b1-b510-56fdbe013eba", "operator": {"type": "array", "operation": "exists", "singleValue": true}, "leftValue": "={{ $jmespath($json, 'origin_search.results') }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "b953082c-2d37-4549-80a7-d60535b8580e", "name": "Response Empty", "type": "n8n-nodes-base.set", "position": [3460, 3900], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52", "name": "response", "type": "string", "value": "no web results"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "41e9c328-d145-4b71-93bb-e2c448a14be0", "name": "Response OK", "type": "n8n-nodes-base.set", "position": [5380, 3580], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "79598789-4468-4565-828f-fedc48be15c3", "name": "response", "type": "string", "value": "ok"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "5d0a7556-def9-4c70-8828-40b4d22904de", "name": "Combine Docs", "type": "n8n-nodes-base.aggregate", "position": [4020, 4380], "parameters": {"options": {}, "aggregate": "aggregateAllItemData"}, "typeVersion": 1}, {"id": "39bd90b4-e0f5-49b0-b4a7-55a3ae8eccb2", "name": "Template to List", "type": "n8n-nodes-base.splitOut", "position": [3280, 4200], "parameters": {"options": {"destinationFieldName": "query"}, "fieldToSplitOut": "queries"}, "typeVersion": 1}, {"id": "51a1da10-5ad0-4bac-9bec-55b5af3da702", "name": "Query Templates", "type": "n8n-nodes-base.set", "position": [3100, 4200], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "e2a02550-8f53-4f8d-bb83-68ee3606736e", "name": "queries", "type": "array", "value": "=[\n\"What are the core functionalities, essential features, or primary use cases of {{ $json.data.service }}?\",\n\"Is there an API overview or API categories for {{ $json.data.service }}? What main APIs are listed or mentioned?\",\n\"What industry does {{ $json.data.service }} operate in? What is the most important of the services in the industry? Return the important service as the function.\",\n\"What REST apis (GET, POST, DELETE, PATCH) and/or operations can you identify for {{ $json.data.service }}?\",\n\"Does {{ $json.data.service }} have any CURL examples? If you can, identify one such example and explain what it does.\"\n]"}]}}, "executeOnce": true, "typeVersion": 3.3}, {"id": "414091b7-114b-4fc3-9755-2f87cfef239e", "name": "Google Gemini Chat Model2", "type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini", "position": [3700, 4240], "parameters": {"options": {}, "modelName": "models/gemini-1.5-pro-002"}, "typeVersion": 1}, {"id": "1f0f45ff-3bc9-4786-92e1-319244d020c0", "name": "For Each Template...", "type": "n8n-nodes-base.splitInBatches", "position": [3460, 4200], "parameters": {"options": {}}, "typeVersion": 3}, {"id": "2e577e62-7f89-4c99-b540-ce8c44f19a55", "name": "Query & Docs", "type": "n8n-nodes-base.set", "position": [4180, 4380], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "fdaea3de-3c9a-4f26-b7dc-769e534006a9", "name": "query", "type": "string", "value": "={{ $('For Each Template...').item.json.query }}"}, {"id": "88198374-d2f9-4ae7-b262-d3b2e630e0ac", "name": "documents", "type": "string", "value": "={{ $json.data.map(item => item.document.pageContent.replaceAll('\\n', ' ')).join('\\n---\\n') }}"}]}}, "typeVersion": 3.4}, {"id": "548d51fd-9740-4b4c-9c81-db62d2b31053", "name": "Identify Service Products", "type": "@n8n/n8n-nodes-langchain.informationExtractor", "position": [3700, 4100], "parameters": {"text": "={{ $json.query }}", "options": {"systemPromptTemplate": "=Use the following document to answer the user's question:\n```\n{{ $json.documents.replace(/[\\{\\}]/g, '') }}\n```"}, "attributes": {"attributes": [{"name": "product_or_solution", "required": true, "description": "A product or solution offered by the service"}, {"name": "description", "required": true, "description": "description of what the product or solution of the service does"}]}}, "typeVersion": 1}, {"id": "aa7041e9-4ac8-47f9-b98e-cf57873922bb", "name": "Extract API Templates", "type": "n8n-nodes-base.set", "position": [4180, 4200], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "e2a02550-8f53-4f8d-bb83-68ee3606736e", "name": "query", "type": "string", "value": "=I'm interested in {{ $json.output.product_or_solution }} apis which {{ $json.output.description }} What are the GET, POST, PATCH and/or DELETE endpoints of the {{ $json.output.product_or_solution }} api?"}]}}, "typeVersion": 3.3}, {"id": "e2b371c1-52af-4e57-877c-6933ba84e2d5", "name": "Embeddings Google Gemini2", "type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini", "position": [4600, 4520], "parameters": {"modelName": "models/text-embedding-004"}, "typeVersion": 1}, {"id": "d808c591-34e2-455f-96b1-3689d950608d", "name": "Search in Relevant Docs1", "type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant", "position": [4600, 4380], "parameters": {"mode": "load", "topK": 20, "prompt": "={{ $json.query }}", "options": {"searchFilterJson": "={{\n{\n \"must\": [\n {\n \"key\": \"metadata.service\",\n \"match\": {\n \"value\": $('EventRouter').first().json.data.service\n }\n }\n ]\n}\n}}"}, "qdrantCollection": {"__rl": true, "mode": "id", "value": "={{ $('EventRouter').first().json.data.collection }}"}}, "typeVersion": 1}, {"id": "222bde31-57fa-46c4-a23b-ec2d1b3c7e2d", "name": "Combine Docs1", "type": "n8n-nodes-base.aggregate", "position": [4920, 4380], "parameters": {"options": {}, "aggregate": "aggregateAllItemData"}, "typeVersion": 1}, {"id": "57677d83-a79a-4b71-9977-ee2324f5d593", "name": "Query & Docs1", "type": "n8n-nodes-base.set", "position": [5080, 4380], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "fdaea3de-3c9a-4f26-b7dc-769e534006a9", "name": "query", "type": "string", "value": "={{ $('For Each Template...1').item.json.query }}"}, {"id": "88198374-d2f9-4ae7-b262-d3b2e630e0ac", "name": "documents", "type": "string", "value": "={{\n$json.data\n .map(item =>\n`url: ${item.document.metadata.url}\ncontent: ${item.document.pageContent}`\n )\n .join('\\n---\\n')\n .replaceAll('\\n\\n', '\\n')\n}}"}]}}, "typeVersion": 3.4}, {"id": "124c3b07-3210-4190-8865-e18017fc9e6c", "name": "For Each Template...1", "type": "n8n-nodes-base.splitInBatches", "position": [4380, 4200], "parameters": {"options": {}}, "typeVersion": 3}, {"id": "8ea4a5da-c471-4201-a08b-9c18ed08ddc7", "name": "Merge Lists", "type": "n8n-nodes-base.code", "position": [4920, 4200], "parameters": {"jsCode": "return $input.all().flatMap(input => input.json.output) || [];"}, "typeVersion": 2, "alwaysOutputData": true}, {"id": "0e38cd3c-c843-4f6d-bdb6-901a8c12acbf", "name": "Remove Duplicates", "type": "n8n-nodes-base.removeDuplicates", "position": [5280, 4200], "parameters": {"compare": "selectedFields", "options": {}, "fieldsToCompare": "method, url"}, "typeVersion": 2}, {"id": "8f127f7a-e351-4b30-82dd-1f785be4a765", "name": "Append Row", "type": "n8n-nodes-base.googleSheets", "position": [5440, 4200], "parameters": {"columns": {"value": {"url": "={{ $json.url }}", "method": "={{ $json.method }}", "service": "={{ $('EventRouter').first().json.data.service }}", "resource": "={{ $json.resource }}", "operation": "={{ $json.operation }}", "description": "={{ $json.description }}", "documentation_url": "={{ $json.documentation_url }}"}, "schema": [{"id": "service", "type": "string", "display": true, "required": false, "displayName": "service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "resource", "type": "string", "display": true, "required": false, "displayName": "resource", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "operation", "type": "string", "display": true, "required": false, "displayName": "operation", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "description", "type": "string", "display": true, "required": false, "displayName": "description", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "url", "type": "string", "display": true, "required": false, "displayName": "url", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "method", "type": "string", "display": true, "required": false, "displayName": "method", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "documentation_url", "type": "string", "display": true, "required": false, "displayName": "documentation_url", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": []}, "options": {"useAppend": true}, "operation": "append", "sheetName": {"__rl": true, "mode": "list", "value": 1042334767, "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=1042334767", "cachedResultName": "Extracted API Operations"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "d9f490e2-320e-4dc1-af8f-ac7f6a61568d", "name": "Response OK1", "type": "n8n-nodes-base.set", "position": [5600, 4200], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "79598789-4468-4565-828f-fedc48be15c3", "name": "response", "type": "string", "value": "ok"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "7780b6ee-0fde-40bb-aef6-e67b883645e1", "name": "Has Operations?", "type": "n8n-nodes-base.if", "position": [5080, 4200], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "a95420a7-6265-4ea3-9c01-82c2d7aeb4f8", "operator": {"type": "object", "operation": "notEmpty", "singleValue": true}, "leftValue": "={{ $input.first().json }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "6589673d-984d-4a1e-a655-1bc19d2b154e", "name": "Response Empty1", "type": "n8n-nodes-base.set", "position": [5280, 4380], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52", "name": "response", "type": "string", "value": "no api operations found"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "c5dc3eac-a3a5-481d-a8bc-8b653d88143d", "name": "Research Pending", "type": "n8n-nodes-base.googleSheets", "position": [3180, 2980], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Research...').item.json.row_number }}", "Stage 1 - Research": "=pending"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "39bceadb-6c3b-4b52-82b9-bdcecd9a164a", "name": "Research Result", "type": "n8n-nodes-base.googleSheets", "position": [3620, 2980], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Research...').item.json.row_number }}", "Stage 1 - Research": "={{ $json.response }}"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "0bd07f31-1c51-45aa-8316-b658aa214293", "name": "Research Error", "type": "n8n-nodes-base.googleSheets", "position": [3620, 3160], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Research...').item.json.row_number }}", "Stage 1 - Research": "=error"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "0385784f-95ef-46c3-82c4-50fcf7146736", "name": "Extract Pending", "type": "n8n-nodes-base.googleSheets", "position": [4160, 2980], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Extract...').item.json.row_number }}", "Stage 2 - Extraction": "pending"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "executeOnce": false, "typeVersion": 4.5}, {"id": "21c1e982-25a6-4a00-b8d3-6c299c452106", "name": "Research Event", "type": "n8n-nodes-base.set", "position": [3320, 2980], "parameters": {"mode": "raw", "options": {}, "jsonOutput": "={{\n{\n \"eventType\": \"research\",\n \"createdAt\": $now.toISO(),\n \"executedById\": $execution.id,\n \"data\": {\n \"row_number\": $('For Each Research...').item.json.row_number,\n \"service\": $('For Each Research...').item.json.Service,\n \"url\": $('For Each Research...').item.json.Website,\n \"collection\": \"api_schema_crawler_and_extractor\"\n }\n}\n}}"}, "typeVersion": 3.4}, {"id": "c83f99f1-e28f-4c15-aff8-da25bb5dfe3b", "name": "Extract Event", "type": "n8n-nodes-base.set", "position": [4300, 2980], "parameters": {"mode": "raw", "options": {}, "jsonOutput": "={{\n{\n \"eventType\": \"extract\",\n \"createdAt\": $now.toISO(),\n \"executedById\": $execution.id,\n \"data\": {\n \"row_number\": $('For Each Extract...').item.json.row_number,\n \"service\": $('For Each Extract...').item.json.Service,\n \"url\": $('For Each Extract...').item.json.Website,\n \"collection\": \"api_schema_crawler_and_extractor\"\n }\n}\n}}"}, "typeVersion": 3.4}, {"id": "88c3caec-75f7-47a1-9b50-1246c457c2b4", "name": "Extract", "type": "n8n-nodes-base.executeWorkflow", "onError": "continueErrorOutput", "position": [4440, 2980], "parameters": {"mode": "each", "options": {"waitForSubWorkflow": true}, "workflowId": {"__rl": true, "mode": "id", "value": "={{ $workflow.id }}"}}, "typeVersion": 1.1}, {"id": "2342b7ff-b00d-439a-a859-63fd0a6bac3a", "name": "Extract Result", "type": "n8n-nodes-base.googleSheets", "position": [4600, 2980], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Extract...').item.json.row_number }}", "Stage 2 - Extraction": "={{ $json.response }}"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "d4c423c9-1d6a-4a69-9302-92ec79734d61", "name": "Extract Error", "type": "n8n-nodes-base.googleSheets", "position": [4600, 3160], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Extract...').item.json.row_number }}", "Stage 2 - Extraction": "error"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "f64254d6-4493-4aaf-8160-35e8ff4fdc34", "name": "Get API Operations", "type": "n8n-nodes-base.googleSheets", "position": [3100, 4740], "parameters": {"options": {}, "filtersUI": {"values": [{"lookupValue": "={{ $json.data.service }}", "lookupColumn": "service"}]}, "sheetName": {"__rl": true, "mode": "list", "value": 1042334767, "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=1042334767", "cachedResultName": "Extracted API Operations"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "fa748b63-3d2b-4cf3-b1fb-1bd953e5054b", "name": "Contruct JSON Schema", "type": "n8n-nodes-base.code", "position": [3280, 4740], "parameters": {"jsCode": "const service = {\n documentation_url: $('EventRouter').first().json.data.url,\n endpoints: [],\n};\n\nconst resources = Array.from(new Set($input.all().map(item => item.json.resource.toLowerCase().trim())));\n\nfor (const resource of resources) {\n const resourceLabel = resource.replace('api', '').trim();\n if (!resourceLabel) continue;\n const endpoint = {\n resource: resourceLabel[0].toUpperCase() + resourceLabel.substring(1, resourceLabel.length)\n };\n const operations = $input.all()\n .filter(item => item.json.resource.toLowerCase().trim() === resource)\n .map(item => item.json);\n endpoint.operations = operations.map(op => ({\n \"operation\": op.operation[0].toUpperCase() + op.operation.substring(1, op.operation.length),\n \"description\": op.description.match(/(^[^\\.]+.)/)[0],\n \"ApiUrl\": op.url,\n \"method\": op.method.toUpperCase(),\n \"method_documentation_url\": op.documentation_url || ''\n }));\n service.endpoints.push(endpoint);\n}\n\nreturn service;"}, "typeVersion": 2}, {"id": "e60b7ccb-baa2-4095-8425-0e20bcdbfdd2", "name": "Upload to Drive", "type": "n8n-nodes-base.googleDrive", "position": [3640, 4740], "parameters": {"name": "={{ $json.filename }}", "content": "={{ $json.data }}", "driveId": {"__rl": true, "mode": "list", "value": "My Drive"}, "options": {}, "folderId": {"__rl": true, "mode": "list", "value": "149rBJYv9RKQx-vQO2qKUGfUzxk_J4lfw", "cachedResultUrl": "https://drive.google.com/drive/folders/149rBJYv9RKQx-vQO2qKUGfUzxk_J4lfw", "cachedResultName": "63. API Schema Extractor Remake"}, "operation": "createFromText"}, "typeVersion": 3}, {"id": "f90546e6-3610-4198-87fc-96d7e2b6bc57", "name": "Set Upload Fields", "type": "n8n-nodes-base.set", "position": [3460, 4740], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "3c7d4946-c385-4aff-93ec-ae0850964099", "name": "filename", "type": "string", "value": "={{\n $('EventRouter').first().json.data.service\n .replace(/\\W+/, '_')\n .toLowerCase()\n}}_api_operations_{{ $now.format('yyyyMMddhhmmss') }}.json"}, {"id": "4a7a9fae-7267-4ef6-ae33-ac4cd9777ee9", "name": "data", "type": "string", "value": "={{ JSON.stringify($json, null, 4) }}"}]}}, "typeVersion": 3.4}, {"id": "c814b48d-2005-4150-a481-956f0b9506a5", "name": "Response OK2", "type": "n8n-nodes-base.set", "position": [3820, 4740], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "79598789-4468-4565-828f-fedc48be15c3", "name": "response", "type": "object", "value": "={{\n({\n id: $json.id,\n filename: $('Set Upload Fields').item.json.filename\n}).toJsonString()\n}}"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "4b1efa99-e8c8-49f5-8db8-916b8dde838d", "name": "Generate Event", "type": "n8n-nodes-base.set", "position": [5300, 2980], "parameters": {"mode": "raw", "options": {}, "jsonOutput": "={{\n{\n \"eventType\": \"generate\",\n \"createdAt\": $now.toISO(),\n \"executedById\": $execution.id,\n \"data\": {\n \"row_number\": $('For Each Generate...').item.json.row_number,\n \"service\": $('For Each Generate...').item.json.Service,\n \"url\": $('For Each Generate...').item.json.Website,\n \"collection\": \"api_schema_crawler_and_extractor\"\n }\n}\n}}"}, "typeVersion": 3.4}, {"id": "49b82a1a-d51e-4caf-b7ab-8d27d0585b60", "name": "Generate Pending", "type": "n8n-nodes-base.googleSheets", "position": [5160, 2980], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Generate...').item.json.row_number }}", "Stage 3 - Output File": "pending"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "executeOnce": false, "typeVersion": 4.5}, {"id": "7d1a937c-49cc-40d7-b2ca-d315c5efca93", "name": "Generate", "type": "n8n-nodes-base.executeWorkflow", "onError": "continueErrorOutput", "position": [5440, 2980], "parameters": {"mode": "each", "options": {"waitForSubWorkflow": true}, "workflowId": {"__rl": true, "mode": "id", "value": "={{ $workflow.id }}"}}, "typeVersion": 1.1}, {"id": "f35d843d-6c40-4725-b73f-8ca1a8e219bb", "name": "Generate Error", "type": "n8n-nodes-base.googleSheets", "position": [5600, 3160], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Generate...').item.json.row_number }}", "Stage 3 - Output File": "error"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "e2f1f8e8-6852-4f19-98ec-85d9bd42729c", "name": "Generate Result", "type": "n8n-nodes-base.googleSheets", "position": [5600, 2980], "parameters": {"columns": {"value": {"row_number": "={{ $('For Each Generate...').item.json.row_number }}", "Output Destination": "={{ $json.response.filename }}", "Stage 3 - Output File": "ok"}, "schema": [{"id": "Service", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Service", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Website", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Website", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 1 - Research", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 1 - Research", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 2 - Extraction", "type": "string", "display": true, "removed": true, "required": false, "displayName": "Stage 2 - Extraction", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Stage 3 - Output File", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Stage 3 - Output File", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "Output Destination", "type": "string", "display": true, "removed": false, "required": false, "displayName": "Output Destination", "defaultMatch": false, "canBeUsedToMatch": true}, {"id": "row_number", "type": "string", "display": true, "removed": false, "readOnly": true, "required": false, "displayName": "row_number", "defaultMatch": false, "canBeUsedToMatch": true}], "mappingMode": "defineBelow", "matchingColumns": ["row_number"]}, "options": {}, "operation": "update", "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "typeVersion": 4.5}, {"id": "00c5b05b-fd70-4d58-8fc6-4e9b8d689a43", "name": "Get All Extract", "type": "n8n-nodes-base.googleSheets", "position": [3620, 2820], "parameters": {"options": {}, "filtersUI": {"values": [{"lookupValue": "=ok", "lookupColumn": "Stage 1 - Research"}, {"lookupValue": "={{ \"\" }}", "lookupColumn": "Stage 2 - Extraction"}]}, "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "executeOnce": true, "typeVersion": 4.5, "alwaysOutputData": true}, {"id": "c477ea01-028d-4e69-b772-adb8c03d1522", "name": "Get All Research", "type": "n8n-nodes-base.googleSheets", "position": [2640, 2820], "parameters": {"options": {}, "filtersUI": {"values": [{"lookupValue": "={{ \"\" }}", "lookupColumn": "Stage 1 - Research"}]}, "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "credentials": {"googleSheetsOAuth2Api": {"id": "", "name": "[Your googleSheetsOAuth2Api]"}}, "typeVersion": 4.5, "alwaysOutputData": true}, {"id": "60ba84c1-40cf-492f-bf52-c9edf5925646", "name": "For Each Research...", "type": "n8n-nodes-base.splitInBatches", "position": [3020, 2820], "parameters": {"options": {}}, "typeVersion": 3}, {"id": "5365cd1a-c7f8-40fb-84b3-9e5306ecf462", "name": "For Each Extract...", "type": "n8n-nodes-base.splitInBatches", "position": [4000, 2820], "parameters": {"options": {}}, "typeVersion": 3}, {"id": "d7a0743f-5f83-4c9b-b11c-85e2df3a4ecc", "name": "Wait1", "type": "n8n-nodes-base.wait", "position": [4780, 3160], "webhookId": "e9ad3ef0-7403-4e65-b0a4-4afdfb0cbc6d", "parameters": {"amount": 0}, "typeVersion": 1.1}, {"id": "ec09ac70-5e05-463c-9d30-027e691a36b4", "name": "All Research Done?", "type": "n8n-nodes-base.if", "position": [2800, 2820], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "8d4b0159-af18-445e-a9ee-bd7952d8e0bd", "operator": {"type": "object", "operation": "empty", "singleValue": true}, "leftValue": "={{ $input.first().json }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "cd892e11-b4de-42f1-bab9-4bd783494c8a", "name": "All Extract Done?", "type": "n8n-nodes-base.if", "position": [3780, 2820], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "8d4b0159-af18-445e-a9ee-bd7952d8e0bd", "operator": {"type": "object", "operation": "empty", "singleValue": true}, "leftValue": "={{ $input.first().json }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "426091fb-d0eb-4589-8f2f-2bbeb9174cfc", "name": "Get All Generate", "type": "n8n-nodes-base.googleSheets", "position": [4600, 2820], "parameters": {"options": {}, "filtersUI": {"values": [{"lookupValue": "=ok", "lookupColumn": "Stage 1 - Research"}, {"lookupValue": "=ok", "lookupColumn": "Stage 2 - Extraction"}, {"lookupValue": "={{ \"\" }}", "lookupColumn": "Stage 3 - Output File"}]}, "sheetName": {"__rl": true, "mode": "list", "value": "gid=0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0", "cachedResultName": "Sheet1"}, "documentId": {"__rl": true, "mode": "list", "value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0", "cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk", "cachedResultName": "API Schema Crawler & Extractor"}}, "executeOnce": true, "typeVersion": 4.5}, {"id": "01e91cf6-5bd5-4891-ba1f-95176e444fe6", "name": "All Generate Done?", "type": "n8n-nodes-base.if", "position": [4780, 2820], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "8d4b0159-af18-445e-a9ee-bd7952d8e0bd", "operator": {"type": "object", "operation": "empty", "singleValue": true}, "leftValue": "={{ $input.first().json }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "08f3505d-aad8-475a-bf08-e3da12798367", "name": "For Each Generate...", "type": "n8n-nodes-base.splitInBatches", "position": [5000, 2820], "parameters": {"options": {}}, "typeVersion": 3}, {"id": "1a1b30bd-91ab-41bd-9ead-39d24fc2643f", "name": "Wait2", "type": "n8n-nodes-base.wait", "position": [5780, 3160], "webhookId": "e9ad3ef0-7403-4e65-b0a4-4afdfb0cbc6d", "parameters": {"amount": 0}, "typeVersion": 1.1}, {"id": "8f2be6bb-ab65-4c92-9ca1-d7ffa936a2a3", "name": "Has Results?1", "type": "n8n-nodes-base.if", "position": [4260, 3720], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "1223d607-45a8-44b1-b510-56fdbe013eba", "operator": {"type": "array", "operation": "notEmpty", "singleValue": true}, "leftValue": "={{ $input.all().filter(item => item.json.body) }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "82fe66bf-4348-4673-8c64-3415f642fb4b", "name": "Response Scrape Error", "type": "n8n-nodes-base.set", "position": [4460, 3900], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52", "name": "response", "type": "string", "value": "web scraping error"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "3625591b-cb48-4131-ae8a-56d1e132bb5a", "name": "Has Results?3", "type": "n8n-nodes-base.if", "position": [4780, 3580], "parameters": {"options": {}, "conditions": {"options": {"version": 2, "leftValue": "", "caseSensitive": true, "typeValidation": "strict"}, "combinator": "and", "conditions": [{"id": "1223d607-45a8-44b1-b510-56fdbe013eba", "operator": {"type": "array", "operation": "notEmpty", "singleValue": true}, "leftValue": "={{ $input.all().filter(item => item.json.body) }}", "rightValue": ""}]}}, "typeVersion": 2.2}, {"id": "f82a4a25-5f93-4ba4-baae-08283c4ccadd", "name": "Response No API Docs", "type": "n8n-nodes-base.set", "position": [4980, 3740], "parameters": {"options": {}, "assignments": {"assignments": [{"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52", "name": "response", "type": "string", "value": "no api docs in web results"}]}}, "executeOnce": true, "typeVersion": 3.4}, {"id": "4c3bb934-966c-445a-893f-0676a59140ee", "name": "Sticky Note", "type": "n8n-nodes-base.stickyNote", "position": [3020, 2580], "parameters": {"width": 620, "height": 180, "content": "## Stage 1 - Research for API Documentation\n- Fetch a list of services pending research from Database (Google Sheet)\n- Uses a search engine (Google) to find API Documentation for each service\n- Uses Webscraper (Apify) to read the contents of search results to filter irrelevant pages\n- Stores webpage contents and metadata into Vector Store (Qdrant)"}, "typeVersion": 1}, {"id": "bc269a57-f353-4cc8-bd2e-43236fa55d39", "name": "Sticky Note1", "type": "n8n-nodes-base.stickyNote", "position": [4000, 2580], "parameters": {"width": 760, "height": 180, "content": "## Stage 2 - Extract API Operations From Documentation\n- Fetch a list of services pending extraction from Database (Google Sheet)\n- Query Vector store (Qdrant) to figure out service's products, solutions and offerings\n- Query Vector store (Qdrant) again for API documentation relevant to these products, solutions and offerings\n- Extract any API operations found in the API documentation results using LLM (Gemini)\n- Store extracted API operations into Database (Google Sheet)"}, "typeVersion": 1}, {"id": "d2dcad47-f655-4a15-ac92-6dab05eea4e1", "name": "Sticky Note2", "type": "n8n-nodes-base.stickyNote", "position": [5000, 2580], "parameters": {"width": 740, "height": 180, "content": "## Stage 3 - Generate Custom Schema From API Operations\n- Fetch a list of services pending generation from Database (Google Sheet)\n- Fetch all API operations for each service from Database (Google Sheet)\n- Use Code node to combine and group all API operations for a service and convert to a custom schema\n- Upload the resulting custom schema to file storage (Google Drive)"}, "typeVersion": 1}, {"id": "d1e1a271-4260-49c3-bda6-2864605c7365", "name": "Sticky Note3", "type": "n8n-nodes-base.stickyNote", "position": [3100, 3680], "parameters": {"color": 5, "width": 180, "height": 80, "content": "## Stage 1 - Subworkflow"}, "typeVersion": 1}, {"id": "1e50f04a-94ff-48b4-aa99-cd1d4f1d12be", "name": "Sticky Note4", "type": "n8n-nodes-base.stickyNote", "position": [3100, 4080], "parameters": {"color": 5, "width": 180, "height": 80, "content": "## Stage 2 - Subworkflow"}, "typeVersion": 1}, {"id": "f8334dbd-b542-404a-b4fc-6cf7cc07730d", "name": "Sticky Note5", "type": "n8n-nodes-base.stickyNote", "position": [3100, 4620], "parameters": {"color": 5, "width": 180, "height": 80, "content": "## Stage 3 - Subworkflow"}, "typeVersion": 1}], "pinData": {"Execute Workflow Trigger": [{"data": {"url": "https://www.formstack.com/", "service": "Formstack", "collection": "api_schema_crawler_and_extractor", "row_number": 2}, "createdAt": "2024-12-07T12:22:35.344-05:00", "eventType": "research", "executedById": "10234"}]}, "connections": {"Wait": {"main": [[{"node": "For Each Research...", "type": "main", "index": 0}]]}, "Wait1": {"main": [[{"node": "For Each Extract...", "type": "main", "index": 0}]]}, "Wait2": {"main": [[{"node": "For Each Generate...", "type": "main", "index": 0}]]}, "Extract": {"main": [[{"node": "Extract Result", "type": "main", "index": 0}], [{"node": "Extract Error", "type": "main", "index": 0}]]}, "Generate": {"main": [[{"node": "Generate Result", "type": "main", "index": 0}], [{"node": "Generate Error", "type": "main", "index": 0}]]}, "Research": {"main": [[{"node": "Research Result", "type": "main", "index": 0}], [{"node": "Research Error", "type": "main", "index": 0}]]}, "Append Row": {"main": [[{"node": "Response OK1", "type": "main", "index": 0}]]}, "EventRouter": {"main": [[{"node": "Web Search For API Schema", "type": "main", "index": 0}], [{"node": "Query Templates", "type": "main", "index": 0}], [{"node": "Get API Operations", "type": "main", "index": 0}]]}, "Merge Lists": {"main": [[{"node": "Has Operations?", "type": "main", "index": 0}]]}, "Combine Docs": {"main": [[{"node": "Query & Docs", "type": "main", "index": 0}]]}, "Has Results?": {"main": [[{"node": "Results to List", "type": "main", "index": 0}], [{"node": "Response Empty", "type": "main", "index": 0}]]}, "Query & Docs": {"main": [[{"node": "For Each Template...", "type": "main", "index": 0}]]}, "Remove Dupes": {"main": [[{"node": "Scrape Webpage Contents", "type": "main", "index": 0}]]}, "Combine Docs1": {"main": [[{"node": "Query & Docs1", "type": "main", "index": 0}]]}, "Extract Error": {"main": [[{"node": "Wait1", "type": "main", "index": 0}]]}, "Extract Event": {"main": [[{"node": "Extract", "type": "main", "index": 0}]]}, "Has Results?1": {"main": [[{"node": "Has API Documentation?", "type": "main", "index": 0}], [{"node": "Response Scrape Error", "type": "main", "index": 0}]]}, "Has Results?3": {"main": [[{"node": "Set Embedding Variables", "type": "main", "index": 0}], [{"node": "Response No API Docs", "type": "main", "index": 0}]]}, "Query & Docs1": {"main": [[{"node": "For Each Template...1", "type": "main", "index": 0}]]}, "Execution Data": {"main": [[{"node": "EventRouter", "type": "main", "index": 0}]]}, "Extract Result": {"main": [[{"node": "Wait1", "type": "main", "index": 0}]]}, "Filter Results": {"main": [[{"node": "Remove Dupes", "type": "main", "index": 0}]]}, "Generate Error": {"main": [[{"node": "Wait2", "type": "main", "index": 0}]]}, "Generate Event": {"main": [[{"node": "Generate", "type": "main", "index": 0}]]}, "Research Error": {"main": [[{"node": "Wait", "type": "main", "index": 0}]]}, "Research Event": {"main": [[{"node": "Research", "type": "main", "index": 0}]]}, "Extract Pending": {"main": [[{"node": "Extract Event", "type": "main", "index": 0}]]}, "Generate Result": {"main": [[{"node": "Wait2", "type": "main", "index": 0}]]}, "Get All Extract": {"main": [[{"node": "All Extract Done?", "type": "main", "index": 0}]]}, "Has Operations?": {"main": [[{"node": "Remove Duplicates", "type": "main", "index": 0}], [{"node": "Response Empty1", "type": "main", "index": 0}]]}, "Query Templates": {"main": [[{"node": "Template to List", "type": "main", "index": 0}]]}, "Research Result": {"main": [[{"node": "Wait", "type": "main", "index": 0}]]}, "Results to List": {"main": [[{"node": "Filter Results", "type": "main", "index": 0}]]}, "Successful Runs": {"main": [[{"node": "Has Results?1", "type": "main", "index": 0}]]}, "Upload to Drive": {"main": [[{"node": "Response OK2", "type": "main", "index": 0}]]}, "Generate Pending": {"main": [[{"node": "Generate Event", "type": "main", "index": 0}]]}, "Get All Generate": {"main": [[{"node": "All Generate Done?", "type": "main", "index": 0}]]}, "Get All Research": {"main": [[{"node": "All Research Done?", "type": "main", "index": 0}]]}, "Research Pending": {"main": [[{"node": "Research Event", "type": "main", "index": 0}]]}, "Split Out Chunks": {"main": [[{"node": "Store Document Embeddings", "type": "main", "index": 0}]]}, "Template to List": {"main": [[{"node": "For Each Template...", "type": "main", "index": 0}]]}, "All Extract Done?": {"main": [[{"node": "Get All Generate", "type": "main", "index": 0}], [{"node": "For Each Extract...", "type": "main", "index": 0}]]}, "Remove Duplicates": {"main": [[{"node": "Append Row", "type": "main", "index": 0}]]}, "Set Upload Fields": {"main": [[{"node": "Upload to Drive", "type": "main", "index": 0}]]}, "All Generate Done?": {"main": [[], [{"node": "For Each Generate...", "type": "main", "index": 0}]]}, "All Research Done?": {"main": [[{"node": "Get All Extract", "type": "main", "index": 0}], [{"node": "For Each Research...", "type": "main", "index": 0}]]}, "Get API Operations": {"main": [[{"node": "Contruct JSON Schema", "type": "main", "index": 0}]]}, "Default Data Loader": {"ai_document": [[{"node": "Store Document Embeddings", "type": "ai_document", "index": 0}]]}, "For Each Extract...": {"main": [[{"node": "Get All Generate", "type": "main", "index": 0}], [{"node": "Extract Pending", "type": "main", "index": 0}]]}, "Contruct JSON Schema": {"main": [[{"node": "Set Upload Fields", "type": "main", "index": 0}]]}, "For Each Document...": {"main": [[{"node": "Response OK", "type": "main", "index": 0}], [{"node": "Content Chunking @ 50k Chars", "type": "main", "index": 0}]]}, "For Each Generate...": {"main": [[], [{"node": "Generate Pending", "type": "main", "index": 0}]]}, "For Each Research...": {"main": [[{"node": "Get All Extract", "type": "main", "index": 0}], [{"node": "Research Pending", "type": "main", "index": 0}]]}, "For Each Template...": {"main": [[{"node": "Identify Service Products", "type": "main", "index": 0}], [{"node": "Search in Relevant Docs", "type": "main", "index": 0}]]}, "Extract API Templates": {"main": [[{"node": "For Each Template...1", "type": "main", "index": 0}]]}, "For Each Template...1": {"main": [[{"node": "Extract API Operations", "type": "main", "index": 0}], [{"node": "Search in Relevant Docs1", "type": "main", "index": 0}]]}, "Extract API Operations": {"main": [[{"node": "Merge Lists", "type": "main", "index": 0}]]}, "Has API Documentation?": {"main": [[{"node": "Has Results?3", "type": "main", "index": 0}]]}, "Scrape Webpage Contents": {"main": [[{"node": "Successful Runs", "type": "main", "index": 0}]]}, "Search in Relevant Docs": {"main": [[{"node": "Combine Docs", "type": "main", "index": 0}]]}, "Set Embedding Variables": {"main": [[{"node": "For Each Document...", "type": "main", "index": 0}]]}, "Embeddings Google Gemini": {"ai_embedding": [[{"node": "Store Document Embeddings", "type": "ai_embedding", "index": 0}]]}, "Execute Workflow Trigger": {"main": [[{"node": "Execution Data", "type": "main", "index": 0}]]}, "Google Gemini Chat Model": {"ai_languageModel": [[{"node": "Has API Documentation?", "type": "ai_languageModel", "index": 0}]]}, "Search in Relevant Docs1": {"main": [[{"node": "Combine Docs1", "type": "main", "index": 0}]]}, "Embeddings Google Gemini1": {"ai_embedding": [[{"node": "Search in Relevant Docs", "type": "ai_embedding", "index": 0}]]}, "Embeddings Google Gemini2": {"ai_embedding": [[{"node": "Search in Relevant Docs1", "type": "ai_embedding", "index": 0}]]}, "Google Gemini Chat Model1": {"ai_languageModel": [[{"node": "Extract API Operations", "type": "ai_languageModel", "index": 0}]]}, "Google Gemini Chat Model2": {"ai_languageModel": [[{"node": "Identify Service Products", "type": "ai_languageModel", "index": 0}]]}, "Identify Service Products": {"main": [[{"node": "Extract API Templates", "type": "main", "index": 0}]]}, "Store Document Embeddings": {"main": [[{"node": "For Each Document...", "type": "main", "index": 0}]]}, "Web Search For API Schema": {"main": [[{"node": "Has Results?", "type": "main", "index": 0}]]}, "Content Chunking @ 50k Chars": {"main": [[{"node": "Split Out Chunks", "type": "main", "index": 0}]]}, "When clicking \u2018Test workflow\u2019": {"main": [[{"node": "Get All Research", "type": "main", "index": 0}]]}, "Recursive Character Text Splitter1": {"ai_textSplitter": [[{"node": "Default Data Loader", "type": "ai_textSplitter", "index": 0}]]}}}

How to Import This Workflow

  1. 1Copy the workflow JSON above using the Copy Workflow JSON button.
  2. 2Open your n8n instance and go to Workflows.
  3. 3Click Import from JSON and paste the copied workflow.

Don't have an n8n instance? Start your free trial at n8nautomation.cloud

Related Templates

Visualize your SQL Agent queries with OpenAI and Quickchart.io

Visualize your SQL Agent queries with OpenAI and Quickchart.io empowers you to instantly transform complex SQL Agent query results into insightful charts and graphs, all through a simple chat interface. This workflow connects an OpenAI Chat Model to interpret your chat messages, determine if a chart is needed using a Text Classifier, and then leverages Quickchart.io by generating a chart definition with structured output via an HTTP Request node. It automates the entire process from receiving a chat message to extracting the user's question, passing it to an AI Agent, and then conditionally generating and displaying a chart, saving significant time and effort for data analysts, developers, and business intelligence professionals who frequently need to visualize their SQL data. By automating chart generation, this workflow eliminates the manual steps of data extraction, chart selection, and configuration, allowing users to quickly gain visual insights from their SQL Agent queries without needing to switch between multiple tools or possess advanced charting skills.

19 nodes

Analyze the sentiment of feedback and send a message on Mattermost

Automatically analyze customer feedback sentiment and notify your team on Mattermost with this n8n workflow. This automation connects Typeform submissions directly to Google Cloud Natural Language for sentiment analysis, then routes the results to your Mattermost channel. When a user submits a form through Typeform, the workflow extracts the feedback, sends it to Google Cloud Natural Language for sentiment scoring, and based on that score, sends a message to a designated Mattermost channel, allowing for immediate team awareness of positive, negative, or neutral feedback. This workflow is ideal for product managers, customer support teams, or marketing departments who need to quickly gauge public opinion or customer satisfaction from surveys, support tickets, or product reviews, enabling faster response times to critical feedback and reducing the manual effort of monitoring and reporting. It saves significant time and ensures that important customer insights are never missed, fostering a more responsive and customer-centric operation.

5 nodes

Qualify replies from Pipedrive persons with AI

Automate the qualification of inbound email replies from Pipedrive contacts using artificial intelligence. This workflow connects Gmail and OpenAI to your Pipedrive CRM, streamlining your lead nurturing process. When a new email arrives in either of your specified Gmail inboxes (Email box 1 or Email box 2), the workflow searches for the sender as a person in Pipedrive. It then retrieves their full person details and sends the email content to OpenAI for an AI-powered assessment of interest (Is interested?). Based on OpenAI's response, if the person is deemed interested, a new deal is automatically created in Pipedrive, ensuring hot leads are immediately acted upon. This is ideal for sales teams, marketers, and business development professionals who receive a high volume of email replies and need to quickly identify and prioritize genuinely interested prospects, saving significant manual review time and accelerating sales cycles.

11 nodes

Ready to automate with n8n?

Get affordable managed n8n hosting with 24/7 support.