LLM Infrastructure and Tooling: Routing, Profiling, Pilot Apps

OpenRouter’s documentation update focuses on routing DeepSeek models through multiple providers while keeping a single integration surface, including cost, speed controls, and reliability mechanisms for agentic workloads. Hugging Face’s PyTorch profiling series adds developer-focused traces for attention variants across SDPA backends and explains why kernel choice changes performance. Google DeepMind describes a Gemini-powered “ATL Saathi” pilot for educators, with features built around NotebookLM content organization and project generation workflows.

1. OpenRouter routing layer for DeepSeek models

• “DeepSeek” is served on OpenRouter by 16 different companies, with per-provider pricing and throughput varying by about 4x and 4–57 tokens per second.
• OpenRouter default load balancing deprioritizes providers with a significant outage in the last 30 seconds and biases selection toward cheaper providers using an inverse square of price weighting.
• Developers can control routing with provider fields such as sort, max_price, order/only/ignore, plus routing helpers including sticky routing, provider fallbacks, and Fusion.
• Impact: One DeepSeek call can include failover, provider pinning, and version switching behind a single model slug with a 5.5% pay-as-you-go platform fee.

2. Profiling in PyTorch (Part 3): attention traces across SDPA backends

• The series profiles naive causal attention and shows that replacing out-of-place masked_fill with in-place masked_fill_ removes a Memcpy kernel from the masking step.
• For torch.nn.functional.scaled_dot_product_attention with the SDPA math backend, the trace shows 20 GPU kernels per attention forward, with behavior tied to FP32 upcasting and _safe_softmax and causal mask materialization.
• The SDPA efficient and flash backends show single fused attention kernels per forward (e.g., fmha_cutlassF... for efficient and a pytorch_flash kernel for flash), while the cuDNN backend also produces one generated attention kernel but moves work into a CPU path for plan selection.
• Impact: Attention performance debugging can be driven by reading which kernels are launched and whether the trace indicates fused Tensor Core paths versus reference-style math decomposition.

3. Google DeepMind launches ATL Saathi, a Gemini-powered educator assistant pilot

• ATL Saathi is a Gemini-powered web application providing every Tinkering Lab educator with a 24/7 planning and training assistant.
• The app organizes ATL training modules and curriculum materials in NotebookLM and provides summarized modules, AI-generated infographics, video overviews, and interactive quizzes for 12 core modules.
• It includes an advanced project generation interface for 10 core modules that supports “push” (generate grade-appropriate project ideas) and “pull” (step-by-step assembly instructions, wiring diagrams, and safety precautions for student problem statements).
• Impact: Educators in an initial cohort of 100 pilot schools can generate curriculum-aligned learning materials and project plans through a Gemini-powered workflow.

Sources

• OpenRouter: https://openrouter.ai/blog/insights/why-openrouter-for-deepseek/
• Huggingface: https://huggingface.co/blog/torch-attention-profile
• GoogleDeepmind: https://deepmind.google/blog/empowering-indias-next-generation-of-innovators-with-atl-saathi/