Agents, Image Detail Tuning, and SWE-bench Reality Checks

Managed agents in Gemini API are getting additional operational controls for running longer tasks, calling external tools, and keeping network access working across interactions. OpenRouter is also sharing benchmark results that relate image “detail” settings to accuracy, reasoning effort, cost, and latency. OpenAI is publishing an audit of SWE-bench Pro data quality that estimates a substantial fraction of tasks may be broken.

1. Expanding Managed Agents in Gemini API (background execution, remote MCP, custom tools, credential refresh)

• Adds background execution for managed agents by passing background: true to run interactions asynchronously.
• Adds integration with remote Model Context Protocol (MCP) servers via an mcp_server tool at interaction time.
• Adds custom function calling that transitions to requires_action for client-side business logic execution.
• Adds network credential refresh by reusing environment_id with a new network configuration on a subsequent interaction.
• Impact: Developers can run managed-agent workloads asynchronously, connect them to external MCP tools from an isolated sandbox, and refresh network credentials without rebuilding their agent flow.

2. Choosing Optimal Image Input Detail Level in LLMs (OpenRouter benchmarks)

• Reports that image detail setting affects accuracy and cost, with results varying by model and whether the model uses reasoning.
• Shows an example where detail: low vs auto on gpt-5.5 scored 65.2% vs 79.0% and cost 5.1¢ vs 4.5¢ per question on MMMU-Pro Vision.
• States that for reasoning models, cost changes are driven more by reasoning effort than by image token savings, and recommends keeping images clear (auto or high) while adjusting reasoning effort.
• Impact: Developers can tune image “detail” and reasoning effort separately to avoid situations where downscaled inputs increase reasoning cost and reduce accuracy.

3. OpenAI audit: SWE-bench Pro dataset issues and retraction of prior recommendation

• Reports evidence that a significant portion of SWE-bench Pro tasks have breaking issues, estimated at ~30%.
• Finds that the pipeline flagged 200 broken tasks (27.4%) and human annotation identified 249 broken tasks (34.1%).
• Lists four categories of issues: overly strict tests, underspecified prompts, low-coverage tests, and misleading prompts.
• Impact: Developers using SWE-bench Pro results for model capability or safety decisions may need to re-check evaluation validity and consider benchmark alternatives.

Sources

• GoogleAI: https://blog.google/innovation-and-ai/technology/developers-tools/expanding-managed-agents-gemini-api/
• OpenRouter: https://openrouter.ai/blog/insights/image-detail-low-cost/
• OpenAI: https://openai.com/index/separating-signal-from-noise-coding-evaluations