LLM Infrastructure and Tooling: Routing, Profiling, Pilot Apps
Learn how OpenRouter failover hides provider chaos, why SDPA kernel choice reshapes performance, and how ATL Saathi drafts curriculum.
Tue July 14, 2026
•
2 min read
The latest updates focus on infrastructure choices for LLM routing and new capabilities for data application development. OpenRouter releases detailed comparisons against LiteLLM and Portkey to help teams select gateways based on data residency, operational overhead, and compliance needs. Concurrently, Datasette introduces a plugin for hosting sandboxed HTML applications, enabling secure, client-side SQL interactions within a constrained environment. These developments provide clearer pathways for managing provider costs, governance, and custom data interfaces.
1. OpenRouter vs LiteLLM Gateway Comparison
- OpenRouter operates as a managed gateway on Cloudflare’s edge, while LiteLLM is a self-hosted proxy requiring PostgreSQL, Redis, and Docker infrastructure.
- OpenRouter charges a 5.5% platform fee on credit purchases with a $0.80 minimum, whereas LiteLLM is free to self-host but incurs infrastructure costs.
- LiteLLM offers six routing modes including weighted pick and latency-based selection, while OpenRouter uses an Auto Router powered by NotDiamond for provider selection.
- OpenRouter holds SOC 2 Type 2 certification and supports Zero Data Retention, while LiteLLM Enterprise adds RBAC and SSO without holding independent compliance certifications.
- Impact: Teams with strict data residency requirements or high model spend exceeding $3,600 monthly may find LiteLLM’s self-hosted model more cost-effective and compliant.
2. OpenRouter vs Portkey Gateway Comparison
- OpenRouter provides a unified API across 70+ providers with automatic failover, while Portkey acts as a control plane for existing provider keys.
- Portkey has been acquired by Palo Alto Networks and offers HIPAA compliance and custom BAAs on its Enterprise tier.
- OpenRouter streams traces to external platforms like Datadog and Langfuse, whereas Portkey provides in-product logs and dashboards with tiered retention.
- The two systems are compatible, allowing Portkey to route through OpenRouter as an upstream provider for combined governance and routing.
- Impact: Organizations needing integrated governance and prompt management over existing provider accounts should evaluate Portkey, while those seeking managed routing without key management should choose OpenRouter.
3. Datasette Apps Plugin Launch
- The new datasette-apps plugin enables self-contained HTML and JavaScript applications to run in a sandboxed iframe within Datasette.
- Applications are restricted by an immutable Content-Security-Policy header and MessageChannel() transport to prevent data exfiltration and unauthorized access.
- Apps can execute read-only SQL queries directly or use stored queries for write operations if explicitly allow-listed by administrators.
- The plugin includes a copyable prompt to facilitate LLM-assisted app generation and integrates with Datasette Agent for editing capabilities.
- Impact: Developers can now build secure, client-side data visualization tools and interfaces that interact with Datasette’s SQLite backend without server-side code deployment.
Sources
This post was generated with the assistance of AI and reviewed through automated processes. AI can make mistakes. Readers should consult the original sources linked for complete context and verification.
