BigWolf AI Audit
Introduction
Client Background
For this project, our client needed a robust, scalable alternative to Stack-AI for document-based compliance audits. One of the primary concerns they shared with us about the alternative option was that it should be able to handle large volumes of files per project stage and return precise and traceable results.
Challenges
The original solution, built in Stack-AI, used a 5-node LLM pipeline to analyze documents stored in Supabase for 400+ compliance requirements. However, Stack-AI posed several limitations that included:
- Bottlenecks in storage and retrieval.
- Lack of control over model calls and outputs.
- Poor scalability for large document sets.
Given the client’s requirements for the project and their business objectives, we decided the goal was to fully replicate this workflow using Supabase, N8N, and direct LLM API access to regain control and improve performance. Some of the key challenges that we faced during the project included:
- Embedding and storing large documents with metadata.
- Enabling filtered queries based on UUIDs for Organization, Project, and Stage.
- Identifying whether compliance requirements were met, using multiple LLM steps.
- Returning structured JSON outputs with file-level traceability and evidence highlights.
- Logging every aspect of execution, such as inputs, token usage, timestamps, errors, etc.
System Design
For this project, we architected the new system around four main layers that included the vector database setup, LLM workflow orchestration, N8N automations, and structured output delivery. Details for the system design are provided below:
Embedding and Vector Storage
Converted all project documents stored in Supabase Storage into OpenAI embeddings, inserting resulting vectors into a dedicated pgvector table with metadata including file path, project UUID, and stage. Enabled search filtering based on UUIDs and active/inactive document status for efficient retrieval.
Multi-Step LLM Workflow
Recreated the original 5-node Stack-AI pipeline inside n8n using HTTP requests and API calls. Each LLM node was assigned a specialized task (requirement matching, evidence extraction, probability scoring), with refined prompts and question structures tailored to individual compliance checks.
n8n Automation (Cloud Workflow)
Configured webhook triggers to process incoming JSON requests, applying filters before launching the LLM pipeline. Returned structured JSON containing requirement compliance scores, extracted evidence, document paths, and Supabase file links for downstream processing.
Output, Logging and Monitoring
Stored all pipeline outputs in Supabase for long-term tracking and introduced a detailed logging framework capturing run number, user, timestamp, token usage, execution time, input/output JSON, and status/error flags for debugging and performance analytics.
Development Process
We completed the work in modular phases to allow rapid prototyping and future scalability. For the embedding layer, we batch-processed documents into embeddings and stored them with metadata. After that, we built Supabase queries for fast filtering by project UUID, stage, and activity status. Once that was complete, we moved onto LLM replication and testing and rebuilt the five LLM stages using OpenAI and Anthropic APIs, used real compliance prompts to test result quality, and calibrated response formatting for consistent JSON output.
As for the n8n integration, we created a central cloud workflow (WPF1) to orchestrate the entire process. In addition, we configured the input via webhook, output to endpoint, and Supabase DB and implemented retries and error-handling within each node. Lastly for user testing we piloted the workflow using actual project UUIDs and document sets, reviewed the JSON output and file traceability with the client, and confirmed accuracy and engagement with the client’s downstream.
Conclusion
This project highlights how fast, modular AI systems can be built by combining Supabase, N8N, and direct LLM APIs. By replicating and enhancing the Stack-AI workflow, a scalable, auditable solution for real-time document analysis was created.
Launch and Results
We ensured that the workflow was live within days and successfully replaced the Stack-AI while improving control and flexibility. The key outcomes of the project included:
- 93.4% parity with the original Stack-AI LLM workflow
- Instant query response for requirements across project stages
- Detailed, traceable outputs including file links and evidence text
- Comprehensive run logs for auditing and debugging
- Scalable design ready for new LLM providers (xAI, Anthropic, Pinecone)
Parity with the original Stack-AI LLM workflow
Improvement in response time for queries
Improvement in comprehensive log runs for auditing
Scalable design ready for new LLM providers