Pi-AI Provider Migration: Decision Report — Why We're Not Migrating

Prepared for: AGNT Stakeholders
Prepared by: Engineering Team
Date: February 26, 2026
Status: ❌ NOT RECOMMENDED — Do Not Proceed

---

Executive Summary

We conducted a thorough technical evaluation of migrating our LLM provider system to @mariozechner/pi-ai (the unified multi-provider library from the Pi/OpenClaw project). After analyzing our current maintenance costs, the migration effort, ongoing post-migration costs, and the risk profile of the dependency, we recommend against migration.

The core finding: our current system costs ~27 hours/year to maintain (and declining), while migration would require ~160 hours upfront and ~26 hours/year ongoing (and potentially increasing). The investment never pays for itself.

We recommend three targeted improvements to our existing system instead, requiring ~9 hours of one-time effort, which address our two stated pain points (provider breaking changes and adding new providers) with zero production risk.

---

Table of Contents

1. Background & Motivation
2. What Pi-AI Is
3. Our Current System
4. Quantified Maintenance Costs: Current System
5. Migration Cost Estimate
6. Post-Migration Ongoing Costs
7. ROI Analysis
8. Breaking Changes & Compatibility Risks
9. Pi-AI Stability Assessment
10. Pain Point Analysis: Does Pi-AI Actually Solve Our Problems?
11. Alternative: Targeted Improvements
12. Decision Matrix
13. Final Recommendation
14. Appendix A: File Impact Inventory
15. Appendix B: Pi-AI Breaking Change Log
16. Appendix C: Provider Name Mapping Issues

---

1. Background & Motivation

The question of migrating to Pi-AI has been raised by stakeholders who observed the project's rapid growth (98,000+ GitHub stars, 350,000+ downloads, 300+ contributors). The stated motivations for considering migration are:

1. Occasional breaking changes from upstream providers (e.g., model deprecations, SDK updates, auth changes) — estimated at 4–6 incidents per year
2. Adding new providers — we add approximately 2–3 new providers per year
3. General appeal of reducing code — our provider layer spans ~2,750 lines across 20+ files

This report evaluates whether Pi-AI actually solves these problems cost-effectively.

---

2. What Pi-AI Is

@mariozechner/pi-ai is the unified LLM API layer extracted from the Pi monorepo (github.com/badlogic/pi-mono). Key facts:

Attribute	Detail
Author	Mario Zechner (creator of libGDX)
License	MIT
Current Version	0.55.1 (pre-1.0)
Initial Release	November 26, 2025 (~3 months ago)
npm Weekly Downloads	~1,248 (for the full @mariozechner/pi CLI)
Language	TypeScript (ships compiled JS)
API Types Supported	openai-completions, openai-responses, anthropic-messages, google-generative-ai, google-vertex, bedrock-converse-stream, plus Gemini CLI and Antigravity
Providers Covered	20+ (via ~6 API implementations)
Image Generation	❌ Not supported (tool-calling models only)
Maturity	Pre-1.0 with frequent breaking changes

Architecture Strength

Pi-AI's core innovation is API-centric rather than provider-centric architecture. Instead of writing a separate implementation per provider, it recognizes that most providers implement one of ~6 API protocols. A new OpenAI-compatible provider requires zero new code — just a model definition pointing to openai-completions with the correct baseUrl.

This is an elegant design. The question is whether adopting it justifies the migration cost.

---

3. Our Current System

Our provider layer consists of:

Component	Files	~LOC	Purpose
LlmService.js	1	200	Client factory (switch statement)
ProviderRegistry.js	1	350	Capabilities map, model fetching
LlmExecutionService.js	1	450	Tool-calling loop, caching, metrics
CustomOpenAIProviderService.js	1	350	DB-backed custom user providers
llmAdapters.js	1	~3,000	Streaming adapters per provider
Provider singletons (providers/*.js)	14	~1,400	Model-fetching singletons
Total	19	~5,750

Consumer files that directly import from this system: ~12 files including StreamEngine.js, OrchestratorService.js, PluginGenerator.js, AutonomousMessageService.js, ModelRoutes.js, AgentService.js, tools.js, and agnt-agent.js.

Current Providers (14)

OpenAI, Anthropic, Google (Gemini), Groq, xAI (Grok), DeepSeek, Cerebras, OpenRouter, Together AI, MiniMax, Kimi (Moonshot), ZAI, Local (Ollama/LM Studio), OpenAI Codex, Claude Code.

---

4. Quantified Maintenance Costs: Current System

Adding a New Provider: ~20 Minutes

Our providers follow an identical pattern. Adding a new OpenAI-compatible provider involves:

Step	Time
Copy an existing provider file, update URL and fallback models	10 min
Add baseURL entry + case block in LlmService.js	2 min
Add capabilities entry in ProviderRegistry.js	5 min
Add import + route in ModelRoutes.js	2 min
Basic smoke test	1 min
Total	~20 min

At 2–3 new providers per year, this represents ~1 hour/year.

Handling Provider Breaking Changes: ~4 Hours/Year

Incident Type	Frequency	Time Per Fix	Annual Cost
Model deprecation/rename	4–6/year	5 min	0.5 hrs
Base URL change	1–2/year	2 min	0.1 hrs
SDK major version bump	2–3/year	30–60 min	2 hrs
Auth flow change	1–2/year	30 min	1 hr
Fallback model list update	4–6/year	5 min	0.5 hrs
Total			~4 hrs

Total Annual Maintenance

Category	Hours/Year
Breaking change fixes	4
Adding new providers	1
Updating fallback model lists	2
General improvements and cleanup	15
Testing after changes	5
Total	~27

Important trend: this number is declining. The system is stabilizing. Most providers have been added. Most SDK breaking changes have already been handled. The 14 providers we have cover the vast majority of user needs.

---

5. Migration Cost Estimate

One-Time Investment: ~160 Hours

Task	Hours (Est.)	Risk Level
Learn Pi-AI API surface, types, streaming events	8–12	Low
Build PiAiBridge.js adapter layer with provider name mapping	8–16	Medium
Rewrite streaming pipeline (StreamEngine.js — 5 createLlmClient calls, callback → async iterable)	20–30	High
Rewrite llmAdapters.js (~3,000 lines, callback model → async iterable event model)	20–40	High
Rewrite PluginGenerator.js (7 createLlmClient calls, uses raw OpenAI SDK)	8–16	Medium
Migrate tool schemas (OpenAI-native format → TypeBox)	8–12	Medium
Provider name mapping across DB, frontend, auth system (grokai→xai, gemini→google)	8–12	Medium
Keep OAuth flows working (Codex, Claude Code — completely different auth model)	12–20	High
Keep image generation registry (Pi-AI doesn't support it)	4–8	Medium
Integration testing across all 14 providers	16–24	Medium
Fix production regressions and edge cases	8–16	High
Total	120–200
Midpoint estimate	~160

At a blended engineering rate, this represents approximately 4–5 weeks of full-time work for a single developer, or 6–8 weeks accounting for context-switching and other responsibilities.

---

6. Post-Migration Ongoing Costs

Task	Hours/Year	Notes
Pi-AI npm updates with breaking changes	8–15	Pre-1.0 library; 7 documented breaking changes in 3 months
Maintain image generation registry (still ours)	2	Pi-AI doesn't cover DALL-E, Gemini Image, Grok Image
Maintain OAuth adapter layer	3	Our auth ≠ Pi-AI auth; must bridge
Maintain custom provider DB bridge	2	CustomOpenAIProviderService → Pi-AI Model objects
Debug issues inside Pi-AI black box	5–10	Loss of visibility into provider-specific quirks
Total	~20–32
Midpoint estimate	~26

Key Observation

Post-migration annual costs (~26 hrs) are approximately the same as our current annual costs (~27 hrs), with the critical difference that:

• Our current costs are declining as the system matures
• Post-migration costs may be increasing as Pi-AI's pre-1.0 churn continues
• We lose control — when Pi-AI breaks, we wait for their npm publish instead of fixing it ourselves in 5 minutes

---

7. ROI Analysis

Cumulative Hours Over 5 Years

Year	Current System (cumulative)	Pi-AI Path (cumulative)
0 (migration year)	27	186
1	54	212
2	81	238
3	108	264
4	135	290
5	162	316

Break-Even Analysis

Even assuming the most optimistic savings of ~5 hours/year after migration, the 160-hour upfront investment would take 32 years to recoup. This is before accounting for:

• Current costs trending downward (system stabilizing)
• Post-migration costs potentially trending upward (pre-1.0 dependency churn)
• Risk of production incidents during migration
• Opportunity cost of 4–5 weeks of engineering time

Conclusion: The migration never achieves positive ROI.

---

8. Breaking Changes & Compatibility Risks

Critical Issues (Would Cause Runtime Failures)

1. Provider Name Mismatch

Our system and Pi-AI use different names for the same providers:

Our Name	Pi-AI Name	Stored In
grokai	xai	User DB, frontend, auth configs
gemini	google	User DB, frontend, auth configs
zai	Not supported	Must create custom model objects
local	Not supported	Must create custom model objects

These names are persisted in databases, user conversation histories, frontend components, and authentication provider configurations. Renaming them is a cross-stack migration affecting backend, frontend, and stored data.

2. OAuth Auth Flow Incompatibility

We have three special OAuth auth flows with custom token management:

Provider	Our System	Pi-AI System
openai-codex	CodexAuthManager.getAccessToken() → OpenAI SDK apiKey	loginOpenAICodex() → OAuth credentials in auth.json
openai-codex-cli	Custom chatgpt.com/backend-api/codex endpoint	openai-codex-responses API type with its own session/transport
claude-code	ClaudeCodeAuthManager with custom headers (anthropic-beta, x-app)	loginAnthropic() with Pi-AI's own ANTHROPIC_OAUTH_TOKEN

Our auth managers have custom token refresh logic, session management, and error recovery that Pi-AI does not replicate. This is the single highest-risk area of the migration.

3. Streaming Architecture Mismatch

Our system uses a callback-based streaming model:

// AGNT: callback pattern
const result = await adapter.callStream(messages, tools, onChunk);
// onChunk receives text deltas

Pi-AI uses an async iterable event stream:

// Pi-AI: async iterable pattern
for await (const event of stream(model, context)) {
  switch (event.type) {
    case 'text_delta': /* ... */
    case 'toolcall_start': /* ... */
    case 'done': /* ... */
  }
}

This requires rewriting StreamEngine.js (5 createLlmClient calls), llmAdapters.js (~3,000 lines), and every file that consumes streaming responses. This is a fundamental architectural change, not a surface-level refactor.

4. Tool Schema Format Incompatibility

Our tools use OpenAI-native JSON Schema format. Pi-AI uses TypeBox schemas. Every tool definition would need conversion, and we have dozens of tool schemas across the system.

5. Response Message Shape

Our adapters return { responseMessage, toolCalls } matching native SDK formats. Pi-AI returns a different AssistantMessage with typed content block arrays (TextBlock, ToolCallBlock, ThinkingBlock). The 8 files that consume adapter responses would all need updates.

High-Risk Issues (Would Cause Partial Failures)

• Image generation not supported — Pi-AI explicitly excludes non-tool-calling models. DALL-E, Gemini Image, and Grok Image are not covered. We must maintain a separate image generation registry.
• Static model lists vs. live API fetching — Our system fetches models live from provider APIs; Pi-AI uses auto-generated static lists from models.dev. Users would see fewer models until Pi-AI publishes npm updates.
• Custom provider DB persistence — Pi-AI's custom model system is runtime-only (in-memory). Our CustomOpenAIProviderService.js with encrypted API keys and per-user isolation must be retained and bridged.
• Loss of Cerebras SDK options — Our Cerebras integration uses the native Cerebras SDK with warmTCPConnection: false. Pi-AI routes Cerebras through generic openai-completions, losing SDK-specific options.

Medium-Risk Issues (Behavioral Differences)

• Environment variable conflicts — Pi-AI auto-reads OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. from environment. If set on our servers for other purposes, Pi-AI may use them unexpectedly.
• Error message differences — Our adapters throw AGNT-specific error messages that our error handling middleware catches. Pi-AI throws different error types with different messages.
• Debugging visibility — Provider quirks are handled inside Pi-AI's compiled code. When something breaks, we debug a black box instead of our own source files.

---

9. Pi-AI Stability Assessment

Release Velocity

Pi-AI has published 47 versions from v0.9.4 (Nov 26, 2025) to v0.55.1 (Feb 26, 2026) — that's 47 releases in 93 days, averaging one release every 2 days.

Documented Breaking Changes

From the official CHANGELOG, Pi-AI has had 7 documented breaking change releases in 3 months:

Version	Date	Breaking Change
0.13.0	Dec 6, 2025	Added required totalTokens field to Usage type
0.17.0	Dec 9, 2025	Removed provider-level tool argument validation
0.28.0	Dec 25, 2025	Removed all OAuth storage functions, renamed getApiKey → getEnvApiKey, removed setApiKey/resolveApiKey
0.31.0	Jan 2, 2026	Moved entire Agent API to separate package (@mariozechner/pi-agent-core)
0.37.0	Jan 5, 2026	Removed per-thinking-level model variants for Codex
0.38.0	Jan 8, 2026	Removed Codex model aliases (gpt-5, gpt-5-mini, etc.)
0.50.0	Jan 26, 2026	Refactored API streaming dispatch, moved env key resolution

Rate: ~2.3 breaking changes per month.

This is expected for a pre-1.0 project under active development. However, it means that if we pin to the current version, we lose the upstream bug fixes and new model additions that are the primary appeal. If we don't pin, we absorb ~2 breaking changes per month in a dependency that touches our most critical path — LLM communication.

Maintainer Risk

The project is primarily authored by Mario Zechner (@badlogic), with ~12 active contributors. As of February 26, 2026, the repo is on "OSS Vacation" (PRs auto-closed until March 2). While the project is healthy and growing, it is still a single-maintainer-led open source project. Our critical infrastructure would depend on one person's continued investment.

---

10. Pain Point Analysis: Does Pi-AI Actually Solve Our Problems?

Pain Point 1: "Occasional breaking changes from providers"

Dimension	Current System	After Pi-AI Migration
Who fixes it?	Us — typically in 5–30 min	Pi-AI team — we wait for npm publish
Time to resolution	Immediate (we control the code)	Hours to days (dependency on external team)
Fix quality	Tailored to our exact needs	Generic fix for all Pi-AI users
If upstream doesn't fix it?	N/A	Fork Pi-AI or patch locally — worse position
Annual effort	~4 hours	Same issues exist, different owner

Verdict: Provider breaking changes don't disappear — they shift from our 5-minute fixes to waiting on someone else's release schedule. We lose control and gain latency. Pi-AI does not solve this problem; it transfers responsibility with worse SLA.

Pain Point 2: "Adding new providers"

Dimension	Current System	After Pi-AI Migration
If Pi-AI already has the provider	N/A	getModel('provider', 'model') — 1 line
If Pi-AI doesn't have it	~20 min (copy-paste pattern)	Build custom Model object (~15 lines)
If provider isn't OpenAI-compatible	Write adapter in llmAdapters.js	Same — Pi-AI needs a new API type too
Frequency	~2–3/year	Same

Verdict: We save ~15 minutes per new provider IF Pi-AI already supports it. At 2–3 providers/year, that's 30–45 minutes saved annually — for a 160-hour upfront investment. Pi-AI marginally improves this, but the ROI is absurd.

---

11. Alternative: Targeted Improvements

Instead of migrating, we recommend three focused improvements that address the stated pain points:

Improvement 1: Provider Template Generator Script (~2 hours, one-time)

Create a CLI script that auto-generates all boilerplate for a new provider:

node scripts/generate-provider.js \
  --name deepinfra \
  --url https://api.deepinfra.com/v1 \
  --models "meta-llama/Llama-3.1-70B,mistralai/Mixtral-8x7B"

This would auto-generate the provider singleton, update LlmService.js, update ProviderRegistry.js, and update ModelRoutes.js.

Result: "Add a new provider" goes from 20 minutes to 2 minutes.

Improvement 2: Consolidate Provider Singletons (~3 hours, one-time)

All 14 provider files are identical except for baseURL and getFallbackModels(). Replace them with a single GenericProviderService class and a configuration object:

// providers/index.js — replaces 14 files with 1
const providers = {
  openai: new GenericProvider('https://api.openai.com/v1', ['gpt-4.1', 'gpt-4o']),
  groq: new GenericProvider('https://api.groq.com/openai/v1', ['llama-3.3-70b-versatile']),
  deepseek: new GenericProvider('https://api.deepseek.com', ['deepseek-chat']),
  // ... all 14 providers in one file
};

Result: 14 files → 1 file. Adding a provider becomes a single line.

Improvement 3: Centralized Provider Quirks Map (~4 hours, one-time)

Borrow Pi-AI's compat concept without the dependency:

const PROVIDER_COMPAT = {
  kimi: { userAgent: 'KimiCLI/0.77', mapDeveloperRole: true },
  openrouter: { extraHeaders: { 'HTTP-Referer': '...', 'X-Title': '...' } },
  cerebras: { warmTCP: false },
};

Result: Provider quirks are centralized instead of scattered across switch statements. New quirks = one line in a config object.

Total Investment: ~9 Hours

These three improvements deliver ~80% of Pi-AI's structural benefits (cleaner code, faster provider additions, centralized quirks) with 0% of the migration risk.

---

12. Decision Matrix

Factor	Keep Current + Improvements	Pi-AI Migration
Upfront cost	~9 hours	~160 hours
Annual ongoing cost	~20 hrs (declining)	~26 hrs (uncertain trajectory)
Break-even point	Immediate	Never
Production risk	None	High (streaming rewrite, auth bridging)
Control over fixes	Full	Partial (wait for npm publish)
"Add provider" time	~2 min (with generator)	~1 min (if Pi-AI has it)
"Handle break" time	5–30 min (immediate)	Unknown (external dependency)
Image generation	Fully supported	Must maintain separate registry
OAuth flows	Working, proven	Must bridge, high risk
Code reduction	14 files → 1 + improvements	14 files → 0 + bridge layer
Debugging visibility	Full (our source code)	Partial (compiled dependency)
Dependency risk	None (first-party code)	Pre-1.0 dep, ~2 breaking changes/month

---

13. Final Recommendation

❌ Do Not Migrate to Pi-AI

The migration does not achieve positive ROI at any time horizon. It introduces significant production risk, adds a volatile pre-1.0 dependency to our most critical code path, and does not meaningfully solve our two stated pain points (provider breaking changes and adding new providers).

✅ Implement Targeted Improvements Instead

Invest ~9 hours in three focused improvements:

1. Provider template generator script (2 hrs)
2. Consolidate 14 provider singletons into 1 (3 hrs)
3. Centralized provider quirks map (4 hrs)

These deliver the structural benefits without migration risk.

📋 Revisit Conditions

We should revisit this decision if any of the following change:

• Pi-AI reaches v1.0 with a stable API and semantic versioning guarantees
• We need to support 30+ providers (our current architecture scales linearly; Pi-AI's scales better)
• We are building a new product from scratch where Pi-AI could be adopted as a foundation rather than a migration
• Our maintenance costs reverse trend and start increasing significantly (>50 hrs/year)

---

Appendix A: File Impact Inventory

Complete list of files that would require modification in a Pi-AI migration:

File	Changes Required	Risk
LlmService.js	Replace createLlmClient() switch with Pi-AI getModel()	Medium
LlmExecutionService.js	Rewrite tool-calling loop for Pi-AI event model	High
ProviderRegistry.js	Replace static capabilities with Pi-AI getModels() + image gen sub-registry	Medium
CustomOpenAIProviderService.js	Bridge DB records → Pi-AI Model objects	Medium
llmAdapters.js (~3,000 lines)	Rewrite callback streaming → async iterable events	Critical
StreamEngine.js (5 createLlmClient calls)	Rewrite streaming pipeline	Critical
OrchestratorService.js	Update provider/model resolution	Medium
PluginGenerator.js (7 createLlmClient calls)	Replace raw OpenAI SDK usage with Pi-AI	High
AutonomousMessageService.js	Update adapter consumption	Medium
ModelRoutes.js (imports all 14 providers)	Replace with Pi-AI model listing	Low
AgentService.js	Update OpenAI import	Low
tools.js	Keep image gen, update text model resolution	Medium
agnt-agent.js	Update adapter consumption	Medium
14 provider singletons	Delete (replaced by Pi-AI model registry)	Low
Frontend provider dropdowns	Map grokai→xai, gemini→google	Medium
Database migration	Rename provider names in stored records	High
Auth provider configs	Map provider IDs	Medium

---

Appendix B: Pi-AI Breaking Change Log

Documented breaking changes from the official CHANGELOG (v0.9.4 through v0.55.1):

Version	Date	Breaking Change Summary
0.13.0	2025-12-06	Required totalTokens field added to Usage type
0.17.0	2025-12-09	Provider-level tool argument validation removed; must use validateToolCall()
0.28.0	2025-12-25	OAuth storage completely removed; getApiKey → getEnvApiKey; setApiKey/resolveApiKey deleted; OAuthCredentials type changed
0.31.0	2026-01-02	Entire Agent API (agentLoop, AgentContext, AgentTool, etc.) moved to @mariozechner/pi-agent-core
0.37.0	2026-01-05	Per-thinking-level model variants removed for Codex
0.38.0	2026-01-08	Codex model aliases removed (gpt-5, gpt-5-mini, gpt-5-nano, codex-mini-latest, etc.)
0.50.0	2026-01-26	API streaming dispatch refactored to registry pattern; env key resolution moved; Azure responses reworked

Rate: 7 breaking changes in 93 days = ~2.3 per month or ~27 per year (projected).

---

Appendix C: Provider Name Mapping Issues

If migration were pursued, the following provider name conflicts would need resolution across the full stack:

AGNT Internal Name	Pi-AI Name	Used In Frontend	Stored in DB	Auth Provider ID
grokai	xai	✅	✅	✅
gemini	google	✅	✅	✅
zai	(not in Pi-AI)	✅	✅	✅
local	(not in Pi-AI)	✅	✅	❌
openai-codex	openai (subtype)	✅	✅	✅
openai-codex-cli	openai-codex	✅	✅	✅
claude-code	anthropic (subtype)	✅	✅	✅

Renaming these would require:

• Database migration for all stored conversations and provider configs
• Frontend component updates
• Auth provider configuration updates
• API endpoint backward compatibility layer
• User-facing documentation updates

---

This report was prepared based on a thorough review of the Pi-AI source code (CHANGELOG, api-registry, models, stream, types, provider implementations), our complete AGNT provider codebase, and quantitative analysis of maintenance effort.