内容来源：https://blog.google/innovation-and-ai/technology/developers-tools/agentic-vision-gemini-3-flash/

内容总结：

谷歌发布Gemini 3 Flash模型全新“自主视觉”功能，将图像理解从静态识别升级为动态推理过程。传统AI模型通常只能对图像进行一次性整体分析，容易忽略细节而被迫猜测。而新功能通过结合视觉推理与代码执行，使模型能够像调查员一样主动规划步骤，如放大、检查和处理图像，从而让回答基于确凿的视觉证据。

该技术引入了“思考-行动-观察”的自主循环机制：模型先分析问题与图像并制定分步计划；随后生成并执行Python代码来操作图像（如裁剪、标注）或进行分析计算；最后将处理后的图像纳入上下文，在更佳语境下生成最终答案。测试显示，启用代码执行后，模型在多数视觉基准任务上的性能提升5%-10%。

目前，该功能已应用于多个实际场景：在建筑图纸审核平台PlanCheckSolver.com中，通过迭代检查高分辨率图像，使合规验证准确率提升5%；在图像标注任务中，模型可直接在图像上绘制标记框，避免计数错误；在可视化数据分析中，能自动生成专业图表，替代传统概率猜测。

谷歌表示，未来将进一步提升功能的自动化程度，拓展更多工具支持（如图像搜索），并将该能力扩展至更多尺寸的模型。开发者现已可通过Google AI Studio和Vertex AI的Gemini API使用此功能，也可在Gemini应用中选择“思考”模式进行体验。

中文翻译：

Gemini 3 Flash 引入“自主视觉”能力

像 Gemini 这样的前沿 AI 模型，通常以单一、静态的“一瞥”来感知世界。如果它们错过了某些细微细节——比如微芯片上的序列号或远处的路牌——就不得不进行猜测。

Gemini 3 Flash 中的“自主视觉”功能，将图像理解从静态行为转变为自主过程。它将视觉视为一次主动调查。通过将视觉推理与代码执行（这是“自主视觉”首批支持的工具之一）相结合，模型能够制定计划，逐步放大、检查并处理图像，从而将答案建立在视觉证据之上。

在 Gemini 3 Flash 中启用代码执行功能，使其在大多数视觉基准测试中的质量持续提升 5-10%。

自主视觉：一项新的前沿 AI 能力

“自主视觉”为图像理解任务引入了自主的“思考、行动、观察”循环：

• 思考：模型分析用户查询和初始图像，制定多步骤计划。
• 行动：模型生成并执行 Python 代码，以主动操作图像（例如裁剪、旋转、标注）或分析图像（例如运行计算、统计边界框等）。
• 观察：处理后的图像会被添加到模型的上下文窗口中。这使得模型能够在生成最终答案前，在更优的上下文中检查新数据。

“自主视觉”实战应用

通过在 API 中启用代码执行，您可以解锁许多新功能，其中许多已在 Google AI Studio 的演示应用中重点展示。从 Gemini 应用这样的大型产品到小型初创公司，开发者们已经开始集成此功能以解锁众多用例，包括：

1. 缩放与检查
Gemini 3 Flash 经过训练，可在检测到细微细节时进行隐式放大。
AI 驱动的建筑图纸验证平台 PlanCheckSolver.com，通过启用 Gemini 3 Flash 的代码执行功能来迭代检查高分辨率输入，将准确率提高了 5%。后台日志视频演示了这一自主过程：Gemini 3 Flash 生成 Python 代码来裁剪和分析特定区域（例如屋顶边缘或建筑部分）作为新图像。通过将这些裁剪图重新加入其上下文窗口，模型将推理过程视觉化，以确认是否符合复杂的建筑规范。

2. 图像标注
“自主视觉”允许模型通过标注图像与其环境互动。Gemini 3 Flash 不仅能描述所见，还能执行代码直接在画布上绘制，以支撑其推理。
在下面的示例中，模型被要求在 Gemini 应用中计算一只手的手指数。为避免计数错误，它使用 Python 在识别出的每个手指上绘制边界框和数字标签。这个“视觉草稿本”确保了其最终答案基于像素级的精确理解。

3. 视觉数学与绘图
“自主视觉”可以解析高密度表格，并执行 Python 代码将发现结果可视化。
标准大语言模型在进行多步骤视觉算术时经常产生幻觉。Gemini 3 Flash 通过将计算卸载到确定性的 Python 环境中来绕过此问题。在下面来自 Google AI Studio 演示应用的示例中，模型识别原始数据，编写代码将先前的最佳结果归一化为 1.0，并生成专业的 Matplotlib 条形图。这用可验证的执行取代了概率性猜测。

未来展望
“自主视觉”的旅程才刚刚开始。

• 更多隐式代码驱动行为：目前，Gemini 3 Flash 擅长隐式决定何时放大查看小细节。虽然其他能力（如旋转图像或进行视觉数学计算）目前仍需明确的提示来触发，但我们正致力于在未来的更新中使这些行为完全隐式化。
• 更多工具：我们也在探索如何为 Gemini 模型配备更多工具，包括网络搜索和反向图片搜索，以进一步夯实其对世界的理解。
• 更多模型尺寸：此外，我们还计划将这一能力扩展到 Flash 之外的其他模型尺寸中。

如何开始使用
“自主视觉”功能现已通过 Google AI Studio 和 Vertex AI 中的 Gemini API 提供。它也开始在 Gemini 应用中逐步推出（通过在模型下拉菜单中选择“思考”来访问）。开发者可以在 Google AI Studio 中尝试演示，或在 AI Studio Playground 中通过开启“工具”下的“代码执行”来体验此功能。阅读开发者文档以了解更多信息（Vertex AI 开发文档）。

英文来源：

Introducing Agentic Vision in Gemini 3 Flash
Frontier AI models like Gemini typically process the world in a single, static glance. If they miss a fine-grained detail — like a serial number on a microchip or a distant street sign — they are forced to guess.
Agentic Vision in Gemini 3 Flash converts image understanding from a static act into an agentic process. It treats vision as an active investigation. By combining visual reasoning with code execution, one of the first tools supported by Agentic Vision, the model formulates plans to zoom in, inspect and manipulate images step-by-step, grounding answers in visual evidence.
Enabling code execution with Gemini 3 Flash delivers a consistent 5-10% quality boost across most vision benchmarks.
Agentic Vision: a new frontier AI capability
Agentic Vision introduces an agentic Think, Act, Observe loop into image understanding tasks:

• Think: the model analyzes the user query and the initial image, formulating a multi-step plan.
• Act: The model generates and executes Python code to actively manipulate images (e.g. cropping, rotating, annotating) or analyze them (e.g. running calculations, counting bounding boxes, etc).
• Observe: The transformed image is appended to the model's context window. This allows the model to inspect the new data with better context before generating a final response.
  Agentic Vision in action
  By enabling code execution in the API, you can unlock many new behaviors, many of which are highlighted in our demo app in Google AI Studio. From big products like the Gemini app to smaller startups, developers have already started integrating the capability to unlock many use cases, including:
  1. Zooming and inspecting
     Gemini 3 Flash is trained to implicitly zoom when detecting fine-grained details.
     PlanCheckSolver.com, an AI-powered building plan validation platform, improved accuracy by 5% by enabling code execution with Gemini 3 Flash to iteratively inspect high-resolution inputs. The video of the backend logs demonstrate this agentic process: Gemini 3 Flash generates Python code to crop and analyze specific patches (e.g., roof edges or building sections) as new images. By appending these crops back into its context window, the model visually grounds its reasoning to confirm compliance with complex building codes.
  2. Image annotation
     Agentic Vision allows the model to interact with its environment by annotating images. Instead of just describing what it sees, Gemini 3 Flash can execute code to draw directly on the canvas to ground its reasoning.
     In the example below, the model is asked to count the digits on a hand in the Gemini app. To avoid counting errors, it uses Python to draw bounding boxes and numeric labels over each finger it identifies. This "visual scratchpad" ensures that its final answer is based on pixel-perfect understanding.
  3. Visual math and plotting
     Agentic Vision can parse high-density tables and execute Python code to visualize the findings.
     Standard LLMs often hallucinate during multi-step visual arithmetic. Gemini 3 Flash bypasses this by offloading computation to a deterministic Python environment. In the example below from our demo app in Google AI Studio, the model identifies the raw data, writes code to normalize prior SOTA to 1.0 and generates a professional Matplotlib bar chart. This replaces probabilistic guessing with verifiable execution.
     What’s next
     We are just getting started with Agentic Vision.
• More Implicit Code-Driven Behaviors: Today, Gemini 3 Flash excels at implicitly deciding when to zoom in on small details. While other capabilities, such as rotating images or performing visual math, currently require an explicit prompt nudge to trigger, we are working to make these behaviors fully implicit in future updates.
• More Tools: We are also exploring how to equip Gemini models with even more tools, including web and reverse image search, to ground its understanding of the world even further.
• More Model Sizes: Additionally, we also plan to expand this capability to our other model sizes beyond just Flash.
  How to get started
  Agentic Vision is available today via the Gemini API in Google AI Studio and Vertex AI. It is also starting to roll out in the Gemini app (access by selecting Thinking from the model drop-down). Developers can try the demo in Google AI Studio, or experiment with the feature in the AI Studio Playground by turning on "Code Execution" under Tools. Read the developer docs to learn more for (Vertex AI dev docs).