How Constellation structured a commercial gateway for LLMs, RAG and fine-tuning

Constellation was created to solve a different problem from a traditional chatbot: how to turn language models into a commercial product distributed through API, a first-party app and partner channels, while keeping control over authentication, models, context, billing and operations. The goal was to build an infrastructure layer to sell and operate LLMs with governance.

The platform was designed as a monorepo with a core API on Cloudflare Workers, a first-party Expo app, a distribution worker for OpenRouter and marketplaces, fine-tune/LoRA operations and Cloudflare security automation. This separation made it possible to keep the critical AI runtime apart from product experiences and external channels without duplicating business rules.

Chat Completions-compatible LLM gateway

At the core of the project, I structured an API compatible with the chat/completions pattern, protected by API keys, multi-tenant authentication and usage metering. The gateway exposes public aliases such as giovannicocco/legal-2.5, giovannicocco/sales-2.4, giovannicocco/gdpr-1.0, giovannicocco/contracts-1.6 and other domain-specific models, while keeping raw provider model IDs isolated from the public interface.

This design makes it possible to sell models by use case without tying the customer to the provider's technical ID. If the internal infrastructure changes, the public alias remains stable. I also implemented controls to avoid exposing provider model IDs when the product strategy requires a closed catalog.

RAG by alias and asynchronous ingestion

The knowledge layer was structured with document upload, R2, queues, embeddings and Vectorize. Each alias can point to a specific rag_profile, such as legal, sales, contracts, compliance, support, procurement, real estate or finance. In practice, this separates knowledge bases by tenant and by usage profile, preventing context from being mixed across customers or models.

The ingestion pipeline runs asynchronously: the file lands in R2, becomes a queue job, goes through text extraction, chunking, embedding and vector indexing. For rich documents, the architecture supports Markdown conversion through Workers AI when available; for simpler formats, it falls back to raw text. At runtime, chat retrieves the most relevant snippets and injects controlled context into the model prompt.

LoRA, fine-tuning and model catalog

Beyond RAG, Constellation gained an operational workspace for LoRA fine-tuning. I created an alias registry that connects public alias, base model, RAG profile and fine-tune adapter. With that, the team can create, list, upload adapters and generate the environment maps used by the gateway.

This layer prepares the platform to evolve from prompts and RAG into domain-specialized models. A sales alias, for example, can start by using the same base model as the others and later receive its own LoRA without breaking the API contract, billing logic or customer experience.

Distribution, billing and partner channels

I also structured a distribution worker for channels such as OpenRouter and marketplaces. It receives OpenAI-format payloads, normalizes input and output, applies a shared secret and forwards the request to the core with channel, provider and external request headers. This makes direct API, app, OpenRouter and marketplace traffic measurable separately.

Monetization was designed with hybrid billing: app subscription, token-based direct API billing through Stripe Billing for LLM Tokens, prepaid credits and local settlement for partners. The runtime records usage, cost, balance, channel, provider and billing events, creating the foundation for financial operations, reporting and governance.

The result was complete LLM infrastructure: a domain model catalog, market-compatible API, RAG by profile, a path to LoRA/fine-tuning, multi-platform app, marketplace distribution, channel-level metering and billing. Constellation moved beyond an AI interface and became a platform to package, operate and monetize language models.