> ## Documentation Index > Fetch the complete documentation index at: https://docs.digcrate.app/llms.txt > Use this file to discover all available pages before exploring further. # Architecture > How Digcrate's components fit together ## Overview Digcrate is a Next.js application with a Convex real-time backend. The browser communicates with serverless API routes on Vercel, which in turn talk to Convex (database), Anthropic (AI), and MCP tool servers (music data). ``` Browser (Next.js App Router) │ ├── Clerk (auth middleware) │ ├── POST /api/chat ─────────────────────── Vercel serverless │ │ │ │ │ agentic-loop.ts │ │ │ │ │ │ Anthropic SDK OpenRouter │ │ │ │ │ tool_use calls │ │ │ │ │ crate-cli MCP servers │ │ (Discogs, MusicBrainz, Last.fm, …) │ │ │ └── SSE stream ──────────────────── Browser │ (thinking, tool_start, tool_end, answer_token, done) │ └── Convex subscriptions (real-time) (sessions, messages, artifacts, playlists, usage) ``` *** ## Tech stack | Layer | Technology | Purpose | | ------------------ | -------------------------------- | ---------------------------------------------------------------------------------------------------- | | Framework | Next.js 16 (App Router) | SSR, API routes, Turbopack dev | | Deployment | Vercel | Serverless functions | | Auth | Clerk | User sign-in, OAuth | | Connected services | Auth0 Token Vault | OAuth connections for Spotify, Slack, Google | | Database | Convex | Real-time sessions, messages, playlists, collections, influence graph, subscriptions, skills, shares | | Dynamic UI | OpenUI (`@openuidev/react-lang`) | Agent-generated interactive components | | Agent | Anthropic SDK + agentic loop | Tool-use loop with crate-cli MCP servers | | Billing | Stripe | Checkout, billing portal, webhooks | | Styling | Tailwind CSS v4 | Dark theme, responsive mobile | | Audio | YouTube IFrame API | Persistent player bar | | Analytics | PostHog | Product analytics, LLM observability | *** ## Project structure ``` crate-web/ ├── convex/ # Convex backend (schema + query/mutation functions) │ ├── schema.ts # Database schema (26 tables) │ ├── sessions.ts # Chat session CRUD │ ├── messages.ts # Message persistence │ ├── playlists.ts # Playlist management │ ├── subscriptions.ts # Stripe subscription state │ ├── usage.ts # Usage tracking and quotas │ ├── userSkills.ts # Custom skill CRUD │ ├── shares.ts # Published Deep Cut shares │ └── users.ts # User sync (Clerk → Convex) ├── src/ │ ├── app/ │ │ └── api/ │ │ ├── chat/route.ts # SSE streaming — agentic loop entry point │ │ ├── auth0/ # OAuth connect, callback, status │ │ ├── stripe/ # Checkout, billing portal, webhooks │ │ ├── skills/ # Custom skill management │ │ └── keys/route.ts # User API key management │ ├── components/ │ │ ├── workspace/ # Chat panel, Deep Cuts panel, workspace shell │ │ ├── sidebar/ # Desktop and mobile sidebar │ │ ├── settings/ # Connected services, plan, skills, keys │ │ └── player/ # YouTube audio player │ └── lib/ │ ├── agentic-loop.ts # Core agent loop (Anthropic + OpenRouter) │ ├── openui/ # 27+ OpenUI component definitions and registry │ ├── web-tools/ # Connected service tools (Spotify, Slack, Google) │ ├── auth0-token-vault.ts # OAuth token exchange via Auth0 Management API │ └── plans.ts # Subscription tiers and rate limiting ``` *** ## The agentic loop Every user message goes through a multi-turn tool-use loop implemented in `src/lib/agentic-loop.ts`. 1. The browser sends a `POST` to `/api/chat` with the message, session history, model choice, and API key. 2. The route handler builds the tool list from `crate-cli` MCP servers and calls `agenticLoop()`. 3. The loop calls the Anthropic (or OpenRouter) API with the message and available tools. 4. If the model returns `tool_use` blocks, the loop executes each tool call against the appropriate MCP server. 5. Tool results are fed back to the model as `tool_result` blocks. Steps 3–5 repeat up to 25 turns. 6. Each turn emits Server-Sent Events to the browser: `thinking`, `tool_start`, `tool_end`, `answer_token`, and finally `done`. 7. The browser's chat panel renders tokens as they arrive and parses OpenUI components from the final answer. The loop supports both Anthropic (direct SDK) and OpenRouter (OpenAI-compatible API) with the same event format. The `useOpenRouter` flag on `AgenticLoopOptions` selects the path. *** ## Key data flows ### Authentication ``` Request arrives → Clerk middleware checks session token → Authenticated: request proceeds with userId → Unauthenticated: redirect to /sign-in ``` User records are synced from Clerk to Convex via a webhook at `/api/webhooks/clerk`. The Convex `users` table stores the Clerk user ID, email, encrypted API keys, and Stripe customer ID. ### Agent query ``` POST /api/chat → Validate Clerk session, load subscription + usage → Check quota (free: 10/month, pro: 50/month, team: 200/month pooled) → Build tool groups from crate-cli MCP servers → agenticLoop() → Anthropic SDK → tool calls → MCP servers → Stream CrateEvents as SSE → On done: write message + artifacts to Convex ``` ### Real-time state (Convex) The browser subscribes to Convex queries that push updates in real time: | Subscription | What updates | | --------------------------- | ----------------------- | | `sessions.listByUser` | Sidebar session list | | `messages.listBySession` | Chat message history | | `artifacts.listBySession` | Deep Cuts panel | | `playlists.listByUser` | Playlist library | | `usage.getForCurrentPeriod` | Usage meter in settings | ### Connected services (Auth0 Token Vault) ``` User clicks "Connect Spotify" in Settings → GET /api/auth0/connect?service=spotify → Generates CSRF nonce, redirects to Auth0 /authorize → User grants permissions on Auth0 / Spotify consent screen → Auth0 redirects to /api/auth0/callback → Callback extracts Auth0 user ID, sets secure cookie Agent calls read_spotify_library tool → getTokenVaultToken("spotify", auth0UserId) → Fetches Management API token (cached 23 hours) → GET /api/v2/users/{userId}?fields=identities → Returns access_token from the Spotify identity → Tool calls Spotify Web API with the token ``` *** ## Database schema The Convex database has 24 tables. Key tables: | Table | Purpose | | ------------------------------------- | --------------------------------------------------------------- | | `users` | User accounts with Clerk ID, encrypted keys, Stripe customer ID | | `crates` | Named session folders for organizing research | | `sessions` | Chat sessions (title, crate membership, starred, archived) | | `messages` | Chat messages (user and assistant roles) | | `artifacts` | Saved research components (Deep Cuts) | | `toolCalls` | Tool execution log per session | | `playerQueue` | Audio player queue tracks per session | | `playlists` / `playlistTracks` | User playlists with track metadata | | `collection` | Vinyl collection records | | `subscriptions` | Stripe subscription state | | `usageEvents` | Per-user, per-period query count | | `userSkills` | Custom slash commands | | `shares` | Published Deep Cut share records | | `influenceArtists` / `influenceEdges` | Cached influence graph nodes and edges | | `telegraphAuth` / `telegraphEntries` | Telegraph publishing credentials and posts | | `tumblrAuth` / `tumblrPosts` | Tumblr publishing credentials and posts | | `orgKeys` | Team-level shared API keys by domain |