Architecture — Yesper

A traffic study, a bill of quantities, a requirements review: thousands of pieces that have to stay consistent with each other and the regulations. Yesper is built for that job — specialist agents, Swedish authority data, document pipelines and long-lived project memory, coordinated by an orchestrator that decides who does what for every request.

The job

A deliverable

Dozens of documents. Thousands of requirements.

Everything has to stay consistent.

Change one piece and the others have to follow, across documents, phases and regulations. Holding it all together is the hard part.

YESPER

The system

A production AI system for construction and bid work

Your team works in a project workspace

Project leads Estimators Specialists Reviewers

The orchestrator decides who handles what

Orchestrator agent

Routes, delegates, and assembles results from many specialists

Not a single LLM that answers. A control plane that chooses the right tools, runs them in parallel, returns the results to the loop, and knows when it's done. On hard calls it consults an independent model from a second provider before moving on.

A fleet of specialist agents — 30+ and growing

Project intelligence

Project agent— reasons across all project files
Project file explorer
Project file indexer— semantic search / RAG
Drawing analysis
Bill of Quantities (BoQ)— extraction
Document type assessment
Image analysis
File comparison
BIM / CAD takeoff— IFC & DWG → Bill of Quantities

Swedish authority data

Allabolag— company register
Trafikverket
SGU— geological data
Naturvårdsverket
AMA— construction standard
PBL— planning & building law
Lantmäteriet— property data
Mapbox & geodata
Weather

Document management

Excel / xlsx agent
MS Office agent
File conversion— PDF, docx, xlsx…
File chunking
SharePoint sync

Workflow automation

Workflow agent
Checklist agent— structured runs
Composer agent
Template agent
Plan agent
Tender comparison— side-by-side bid evaluation

Research & web

Research agent
Web agent— browses and extracts
Code execution — works in the original files (xlsx, docx, pdf)
Reference library— curated knowledge

Deliverables & artefacts

Artifact editor
Web artifact creator
Content export— Word, Excel, PDF
Real-time streaming
Image generation — generated images into decks and documents

Long-lived project memory — not a stateless chat

Auto-built understanding

Built automatically: a company-wide workspace profile and a per-project 60-second brief, with phase detection across Anbud → Projektering → Produktion → Överlämning.

Per-organization database

Cosmos DB, org-isolated. Enterprise customers get their own tenant.

Indexed project files

Continuously chunked, embedded, and searchable in every conversation.

Agent threads & artefacts

Every run, every deliverable — saved and traceable.

Persistent agents

Their own identity, state, and history. Each agent owns a role — researcher, comparator, site engineer — and accumulates context over time.

Enterprise integrations & infrastructure

Auth0 SSO SharePoint sync MCP server (pluggable tools) Azure Service Bus (asynchronous events) EU data residency (Azure) Model-agnostic (Anthropic, Azure OpenAI, Google)

30+specialist agents

8Swedish authority integrations

Multi-tenantdata isolation per organization

Real-timestreaming & collaboration

Asynchronouspipelines & long-running work

An orchestrated agent loop with multi-model routing, multi-tenant Cosmos DB, event-driven async pipelines via Azure Service Bus, Cosmos change-feed + SignalR for real-time, and a fleet of specialist agents exposed as tool calls.

01Clients

Web app

React SPA · SSE streaming

MCP clients

Claude Desktop, Cursor, IDE

API consumers

Authenticated · enterprise

02Edge & API

HTTP API gateway

A modular REST surface — one endpoint group per capability, with SSE streaming for live responses.

Orchestration
Projects & files
Artifacts
Checklists & workflows
Memory
Prompt library
+ more

Auth & authz

Auth0 SSO with scoped authorization at workspace, project, and organization level.

MCP server

Exposes Yesper tools to external LLM clients via the Model Context Protocol.

03Orchestrator

Tool-calling loop with multi-model routing

Routing

Skill routing
Source routing
Agent exclusions
Capability discovery

Loop control

Subtask delegation
Task tracking
Progress reminders
Retry & backoff
Cross-model advisor

Streaming

Stream adapter
Smooth text output
Answer formatting
SSE → client

Model backends

Claude

Opus · Sonnet · Haiku · Vertex AI (EU)

Azure OpenAI

EU deployment

Google

Gemini · Vertex AI (EU)

04Agent fleet

Project intelligence

project_agent — cross-file reasoning
project_explorer_agent
project_file_researcher
project_file_indexer — RAG
blueprint_analyzer
bill_of_quantities
document_type_assessor
image_analyzer
file_comparator_agent

Swedish authority data

allabolag_agent
trafikverket_agent
sgu_agent
nvv_agent
ama_agent
mapbox_agent · geo_agent
weather_agent

Document pipeline

xlsx_agent · excel_converter
msoffice_agent
file_converter
file_chunker
sharepoint_sync
project_file_storage — Azure Blob

Workflow & structure

workflow_agent
checklist_agent
composer_agent
template_agent
plan_agent
memory_agent

Research & execution

research_agent
web_agent — headless browsing
code_execution_agent
reference_library
prompt_library

Artifacts & delivery

artifact · artifact_editor_agent
web_artifact_creator_agent
editor_agent
content_generation
content_export — Word/Excel/PDF
bessy — enrichment pipeline

Embedded skills — composable capability modules the orchestrator loads on demand

ifc-to-boq dwg-to-boq comparing-tenders analyzing-pdf-drawings filling-pdf-forms presenting-maps web-presentation-builder creating-web-apps + user-created skills

File-type-aware and deferred-loaded — only the skills a task needs are pulled into context.

05Document intelligence pipeline

1 · Extract

Swedish-tuned OCR · layout · vision

2 · Classify

14 doc types · SS 32266 · BSAB codes

3 · Enrich

Blueprints · BoQ tables

4 · Chunk

Page / row-tracked segments

5 · Embed

1536-dim vectors

6 · Index

Hybrid: vector + full-text + rerank

Multi-tier OCR fallback, automatic document classification, and query expansion across synonyms, abbreviations, and formal codes (e.g. "bygglov" → AFC.181).

06Data & events

Data plane

multi-tenant · org-isolated

Cosmos DB — default

Shared multi-tenant containers. Partition keys per org.

Cosmos DB — per-org

Dedicated databases for enterprise customers. Same code, different routing.

Azure Blob

Raw files · revisions · large binaries.

Cosmos change feed + SignalR

Change feed → realtime hub → SSE to client.

Redis streams

Low-latency fan-out for real-time text streaming.

Vector index

Chunked, embedded, semantically searchable via RAG tools.

Layered memory

Workspace + project tiers, auto-refreshed on indexing. Thread summaries searchable via hybrid vector + keyword.

Project tables

Agent-writable typed tables — structured data queryable per project.

Event plane

Azure Service Bus

Orchestration runs Agent tasks & schedules Checklist & workflow runs File indexing Thread summarization Project & workspace exploration Enrichment pipelines Artifact & file lifecycle 20+ domain events

Each capability publishes and subscribes to its own domain events. Delivery, retry, and dead-lettering are handled by Azure Service Bus.

07External providers

Model providers

Anthropic Claude (Opus / Sonnet / Haiku) — Vertex AI (EU)
Azure OpenAI (EU deployment)
Google Vertex AI — Gemini & image models (EU)
Per-agent model routing

Swedish authorities & open data

Allabolag
Trafikverket
SGU
Naturvårdsverket
AMA
PBL
Lantmäteriet
Mapbox
Weather API

Enterprise

Auth0
SharePoint Graph API
Outbound email
Continuous evals

08Cross-cutting

Observability

Structured logging, distributed tracing, and continuous model evaluations.

loggingtracingevals

Data residency & encryption

Customer data stays in Azure's EU region (Sweden Central); model inference runs on EU infrastructure across Azure and Google Cloud. Encrypted at rest with AES-256 and in transit with TLS 1.2+.

AES-256TLS 1.2+EU only

Resilience & durability

Long-running work survives restarts and provider outages — checkpointed execution, heartbeats with automatic failover, retries with backoff, and dead-letter handling.

checkpointsheartbeatsauto-failover

Secrets & config

Secrets held in a managed key vault and accessed via managed identity — never in environment variables or source.

managed key vaultmanaged identity

Behind one text field

Every question runs the whole system.

Each request runs an orchestrated loop of dozens of specialist agents, three model providers, a multi-tenant data model, an event plane with 20+ topics, and an index pipeline that keeps every project file semantically searchable. The UI is simple. What's behind it isn't.

A deliverable is thousands of moving pieces. This is the system that holds them together.

A deliverable

A production AI system for construction and bid work

Project intelligence

Swedish authority data

Document management

Workflow automation

Research & web

Deliverables & artefacts

Project intelligence

Swedish authority data

Document pipeline

Workflow & structure

Research & execution

Artifacts & delivery

Data plane

Event plane