surge-ai/handbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Handbook.md benchmark tasks

Browse Source
This commit is contained in:
DerekSurge
2026-06-24 12:44:34 -07:00
commit 25c9eda5ca
1800 changed files with 323575 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+4
View File
@@ -0,0 +1,4 @@
OPENROUTER_API_KEY=your-openrouter-key-here
# Use anthropic API directly, openrouter for other models.
# reasoning settings were not supported by openrouter at eval time.
ANTHROPIC_API_KEY=your-anthropic-key-here
.gitignore
+16
View File
@@ -0,0 +1,16 @@
# Python
.venv/
__pycache__/
*.pyc
*.egg-info/
.pytest_cache/
.ruff_cache/
# Local env (do NOT commit your real API keys)
.env
# Harbor run outputs
jobs/
# OS junk
.DS_Store
LICENSE
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
README.md
+35
View File
@@ -0,0 +1,35 @@
# HANDBOOK.md Benchmark Tasks
This repo contains HANDBOOK.md tasks and the environment they run in. Tasks are in Harbor format. More information on the benchmark can be found [here](https://surgehq.ai/blog/handbook-md).
## Overview
- `agent_harness/` — Agent harness used to evaluate HANDBOOK.md. Built with [OpenHands/software-agent-sdk](https://github.com/OpenHands/software-agent-sdk).
- `docker/` — build context for the `handbook` base image. Contains mock services and the
bundled agent harness.
- `tasks/` — one subdirectory per benchmark task.
- `.env.example` — template for required environment variables.
## Quick start
```bash
# 1. Build the base image (build context is self-contained)
docker build -t handbook_base docker/
# 2. Install Harbor + the bundled agent harness into an isolated venv
uv venv .venv --python 3.13
uv pip install --python .venv/bin/python harbor -e ./agent_harness
# 3. Run a single task locally
.venv/bin/harbor run -p tasks/<task_name> \
--agent-import-path agent_harness.openhands_agent:OpenHandsAgent \
-m anthropic/claude-opus-4-8 -n 1 \
--env-file .env
```
## License
Copyright 2026 Surge AI.
Licensed under the Apache License, Version 2.0. See [`LICENSE`](LICENSE) for the
full text.
agent_harness/README.md
+4
View File
@@ -0,0 +1,4 @@
# agent-harness
Internal Harbor agent package for the SOP benchmark distribution. See the
top-level [README](../README.md) for setup and smoke-test instructions.
agent_harness/pyproject.toml
+21
View File
@@ -0,0 +1,21 @@
[project]
name = "agent-harness"
version = "0.1.0"
description = "Harbor agent harness for Syntara tasks"
requires-python = ">=3.12"
dependencies = [
"harbor",
"deepdiff",
"litellm",
# NOTE: openhands-sdk is NOT a host dep. openhands_agent.py runs the agent
# loop in-container via the vendored openhands_runner.py, installed into an
# isolated venv baked into the syntara image (see docker/Dockerfile, which
# pins the SDK). The host side only does exec/upload/download.
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/agent_harness"]
agent_harness/src/agent_harness/__init__.py
View File
agent_harness/src/agent_harness/openhands_agent.py
+254
View File
@@ -0,0 +1,254 @@
"""Harbor agent that runs the OpenHands SDK agent loop *inside* the task
container.
It execs the vendored Python OpenHands runner (``/app/openhands-runner``)
in-container, driving the agent loop through the ``BaseEnvironment``
exec/upload/download contract. That makes it work on any harbor environment —
including Modal sandboxes, where the host can't reach the in-sandbox MCP port.
This agent targets the **single-container** task shape only (as staged for
Modal, and its local equivalent). It
launches the MCP proxy in-container itself; it does not handle the compose
host-side path.
Container prerequisites (baked into the ``syntara`` base image at build time):
- the runner at ``/app/openhands-runner/openhands_runner.py``
- an isolated venv with ``openhands-sdk`` at ``/app/openhands-runner/.venv``
- the syntara MCP proxy launcher at ``/app/scripts/start.sh``
Usage:
harbor run -p .modal_tasks/some_task \
--agent-import-path agent_harness.openhands_agent:OpenHandsAgent \
-m anthropic/claude-opus-4-8 -e modal --ek registry_secret=aws-ecr-secret
"""
from __future__ import annotations
import asyncio
import json
import os
import shlex
import time
from pathlib import Path
from harbor.agents.base import BaseAgent
from harbor.environments.base import BaseEnvironment
from harbor.models.agent.context import AgentContext
MAX_TOOL_CALLS = 200
# In-container locations.
RUNNER_DIR = "/app/openhands-runner"
RUNNER_PATH = f"{RUNNER_DIR}/openhands_runner.py"
RUNNER_PYTHON = f"{RUNNER_DIR}/.venv/bin/python"
PROXY_START_CMD = "/app/scripts/start.sh --method http --port 8000"
MCP_URL = "http://localhost:8000/mcp"
HEALTH_URL = "http://localhost:8000/health"
REMOTE_LOG_DIR = "/logs/agent"
REMOTE_CONFIG_PATH = "/tmp/openhands_run_config.json"
REMOTE_TRAJECTORY_PATH = f"{REMOTE_LOG_DIR}/trajectory.json"
REMOTE_RUNNER_LOG_PATH = f"{REMOTE_LOG_DIR}/run-openhands.log"
REMOTE_PROXY_LOG_PATH = f"{REMOTE_LOG_DIR}/mcp-proxy.log"
# Timeouts (seconds).
PROBE_TIMEOUT = 15
HEALTH_TIMEOUT = 180
# Generous ceiling for the runner exec; the effective limit is harbor's [agent]
# timeout_sec (× --agent-timeout-multiplier).
AGENT_EXEC_TIMEOUT = 7200
# Host env vars forwarded into the container for the runner. The runner resolves
# LLM credentials from these per provider (see openhands_runner._resolve_llm_auth).
FORWARDED_ENV_VARS = (
"ANTHROPIC_API_KEY",
"ANTHROPIC_BASE_URL",
"OPENAI_API_KEY",
"OPENAI_BASE_URL",
"OPENROUTER_API_KEY",
"GEMINI_API_KEY",
)
def _forwarded_env() -> dict[str, str]:
return {k: v for k in FORWARDED_ENV_VARS if (v := os.environ.get(k))}
FORWARDED_LLM_KWARGS = (
"reasoning_effort",
"reasoning_summary",
# Opt-in: --ak log_completions=true makes the runner dump each LLM call's
# request payload (including the kwargs actually sent, e.g. reasoning_effort)
# into the trial's agent/ log dir for verification.
"log_completions",
)
class OpenHandsAgent(BaseAgent):
"""Runs a single OpenHands SDK agent loop inside the task container."""
def __init__(self, *args, **kwargs) -> None:
# Pull the allow-listed LLM kwargs out of harbor's ``--ak`` kwargs;
# BaseAgent would otherwise drop them into **kwargs and discard them.
# Forwarded to the runner via the run config (see run()).
self.llm_kwargs = {
k: kwargs.pop(k) for k in FORWARDED_LLM_KWARGS if k in kwargs
}
super().__init__(*args, **kwargs)
@staticmethod
def name() -> str:
return "openhands-agent"
def version(self) -> str | None:
return "0.2.0"
async def setup(self, environment: BaseEnvironment) -> None:
probe = await environment.exec(
f"test -f {shlex.quote(RUNNER_PATH)} && test -x {shlex.quote(RUNNER_PYTHON)}",
timeout_sec=PROBE_TIMEOUT,
)
if probe.return_code != 0:
raise RuntimeError(
f"container is missing the OpenHands runner at {RUNNER_DIR}. "
"Rebuild the base image so the runner is staged into it."
)
if not await self._proxy_healthy(environment):
self.logger.info("Launching MCP proxy in-container")
launch = (
f"mkdir -p {REMOTE_LOG_DIR} && "
f"nohup {PROXY_START_CMD} </dev/null "
f">>{REMOTE_PROXY_LOG_PATH} 2>&1 &"
)
result = await environment.exec(launch, timeout_sec=PROBE_TIMEOUT)
if result.return_code != 0:
raise RuntimeError(
f"failed to launch MCP proxy (rc={result.return_code}): "
f"{(result.stderr or '').strip()}"
)
await self._wait_for_proxy(environment)
async def run(
self,
instruction: str,
environment: BaseEnvironment,
context: AgentContext,
) -> None:
task_dir = Path(environment.environment_dir).parent
sp_path = task_dir / "system_prompt.md"
system_prompt = sp_path.read_text()
config = {
"instruction": instruction,
"systemPrompt": system_prompt,
"model": self.model_name,
"mcpUrl": MCP_URL,
"maxToolCalls": MAX_TOOL_CALLS,
"llmKwargs": self.llm_kwargs,
# Host-visible log dir, so opt-in completion logs land where the
# trial syncs them back (see openhands_runner log_completions).
"logDir": REMOTE_LOG_DIR,
}
config_path = (self.logs_dir / "openhands_run_config.json").resolve()
config_path.write_text(json.dumps(config, indent=2))
await environment.upload_file(str(config_path), REMOTE_CONFIG_PATH)
self.logger.info("Running OpenHands runner in-container (model=%s)", self.model_name)
# The runner writes the trajectory JSON to a dedicated file (argv[2]);
# the OpenHands SDK's human-readable transcript + our logs go to stdout/
# stderr, which we capture together in the runner log. Keeping the two
# streams separate is essential — the SDK prints to stdout, so a stdout
# redirect would corrupt the machine-readable trajectory.
command = (
f"mkdir -p {REMOTE_LOG_DIR} && "
f"{shlex.quote(RUNNER_PYTHON)} {shlex.quote(RUNNER_PATH)} "
f"{shlex.quote(REMOTE_CONFIG_PATH)} {shlex.quote(REMOTE_TRAJECTORY_PATH)} "
f"> {REMOTE_RUNNER_LOG_PATH} 2>&1"
)
result = await environment.exec(
command=command,
env=_forwarded_env(),
timeout_sec=AGENT_EXEC_TIMEOUT,
)
runner_log = await self._download_quiet(
environment, REMOTE_RUNNER_LOG_PATH, "run-openhands.log"
)
trajectory = await self._download_quiet(
environment, REMOTE_TRAJECTORY_PATH, "trajectory.json"
)
if result.return_code != 0:
log_tail = "<runner log unavailable>"
if runner_log is not None:
log_tail = runner_log.read_text(errors="replace")[-2000:]
raise RuntimeError(
f"OpenHands runner exited {result.return_code}\n"
f"--- exec stderr (last 2000) ---\n{(result.stderr or '')[-2000:]}\n"
f"--- runner log tail ---\n{log_tail}"
)
if trajectory is None or trajectory.stat().st_size == 0:
raise RuntimeError(
"OpenHands runner produced no trajectory output; "
f"see {self.logs_dir} for the runner log"
)
traj = json.loads(trajectory.read_text())
context.n_input_tokens = traj.get("input_tokens")
context.n_output_tokens = traj.get("output_tokens")
context.n_cache_tokens = traj.get("cache_tokens")
context.cost_usd = traj.get("cost_usd")
context.metadata = {
"agent_id": traj.get("agent_id"),
"model": traj.get("model"),
"n_tool_calls": traj.get("n_tool_calls"),
"stopped_reason": traj.get("stopped_reason"),
"final_output_chars": len(traj.get("final_output") or ""),
"error_message": traj.get("error_message"),
}
# Surface genuine agent-loop errors as a trial exception so broken runs
# aren't averaged into results.
if traj.get("stopped_reason") == "error":
raise RuntimeError(
"OpenHands agent loop ended with an error after "
f"{traj.get('n_tool_calls') or 0} tool call(s): "
f"{traj.get('error_message') or '<no error message>'}"
)
async def _proxy_healthy(self, environment: BaseEnvironment) -> bool:
result = await environment.exec(
f"curl -fsS -o /dev/null --max-time 5 {shlex.quote(HEALTH_URL)}",
timeout_sec=PROBE_TIMEOUT,
)
return result.return_code == 0
async def _wait_for_proxy(self, environment: BaseEnvironment) -> None:
deadline = time.monotonic() + HEALTH_TIMEOUT
while time.monotonic() < deadline:
if await self._proxy_healthy(environment):
self.logger.info("MCP proxy healthy")
return
await asyncio.sleep(2)
tail = await environment.exec(
f"tail -c 2000 {REMOTE_PROXY_LOG_PATH} 2>/dev/null || true",
timeout_sec=PROBE_TIMEOUT,
)
raise RuntimeError(
f"MCP proxy at {HEALTH_URL} never became healthy within {HEALTH_TIMEOUT}s"
f"\n--- proxy log tail ---\n{(tail.stdout or '').strip() or '<no log>'}"
)
async def _download_quiet(
self, environment: BaseEnvironment, remote_path: str, filename: str
) -> Path | None:
"""Download a container file into logs_dir; None if unavailable."""
target = self.logs_dir / filename
try:
await environment.download_file(remote_path, target)
except Exception as e:
self.logger.warning("could not download %s: %s", remote_path, e)
return None
return target if target.exists() else None
agent_harness/src/agent_harness/openhands_runner.py
+359
View File
@@ -0,0 +1,359 @@
"""In-container OpenHands agent runner.
Runs the OpenHands SDK agent loop *inside* the task container, talking to the
syntara MCP proxy at ``localhost:8000/mcp`` (reachable in-container on every
harbor environment, including Modal sandboxes where the host can't reach the
port).
Baked into the ``syntara`` image at build time (this
single file is staged to ``/app/openhands-runner/openhands_runner.py`` next to an
isolated venv holding ``openhands-sdk``). The host-side ``OpenHandsAgent`` uploads
a config JSON, execs this script, and downloads the trajectory it writes.
Contract:
- argv[1] = path to a config JSON: {instruction, systemPrompt, model, mcpUrl,
maxToolCalls}.
- argv[2] = path to write the result/trajectory JSON to. This is a **dedicated
file**, NOT stdout: the OpenHands SDK prints a human-readable transcript to
stdout, so the machine-readable result must go to its own file for the host
to parse it.
- The process exits 0 even when the agent loop errored — the error is recorded
in the trajectory's ``stopped_reason``/``error_message`` so the host decides
whether to fail the trial.
"""
from __future__ import annotations
import json
import os
import sys
from typing import Any
MCP_TOOL_TIMEOUT = 300
WORKSPACE_DIR = "/tmp/openhands_workspace"
STATE_DIR = "/tmp/openhands_state"
def _log(msg: str) -> None:
print(msg, file=sys.stderr, flush=True)
def _resolve_llm_kwargs(config: dict[str, Any]) -> tuple[dict[str, Any], str | None]:
"""Split LLM kwargs into ``(constructor_kwargs, reasoning_effort)``.
``reasoning_effort`` is pulled out so it can be applied post-construction.
openhands.sdk otherwise won't accept `max` as a value.
"""
llm_kwargs = dict(config.get("llmKwargs") or {})
effort = llm_kwargs.pop("reasoning_effort", None)
if effort is not None:
effort = str(effort).strip().lower() or None
return llm_kwargs, effort
def _resolve_llm_auth(model_name: str) -> tuple[str, str | None, str | None]:
"""Map harbor's ``provider/model`` id to a litellm (model, api_key, base_url).
``openrouter/anthropic/<model>`` is rewritten to ``anthropic/<model>`` (dots
-> dashes) so Anthropic models route through the Anthropic provider, never
OpenRouter. To send Anthropic traffic through an LLM proxy, point
``ANTHROPIC_BASE_URL`` at the proxy endpoint and set ``ANTHROPIC_API_KEY`` to
the proxy token.
"""
if "/" not in model_name:
raise ValueError(
f"model must be in 'provider/model' form; got {model_name!r}"
)
provider, model = model_name.split("/", 1)
if provider == "openrouter" and model.startswith("anthropic/"):
provider = "anthropic"
model = model.split("/", 1)[1].replace(".", "-")
if provider == "anthropic":
litellm_model = f"anthropic/{model}"
return (
litellm_model,
os.environ.get("ANTHROPIC_API_KEY"),
os.environ.get("ANTHROPIC_BASE_URL"),
)
_provider_key_env = {
"openai": "OPENAI_API_KEY",
"openrouter": "OPENROUTER_API_KEY",
"google": "GEMINI_API_KEY",
"gemini": "GEMINI_API_KEY",
}
api_key = os.environ.get(_provider_key_env.get(provider, ""))
base_url = os.environ.get("OPENAI_BASE_URL") if provider == "openai" else None
return model_name, api_key, base_url
def _usage(*llms: Any) -> dict[str, int | float | None]:
"""Pull token/cost totals from one or more LLMs' metrics, tolerant of API drift.
Sums across every LLM passed in (e.g. the agent loop plus the condenser) so
the reported cost and cache usage reflect *all* model traffic for the trial,
not just the main agent call. Each field stays ``None`` if no LLM exposed it,
so a metrics-shape change degrades to null rather than crashing the trial.
"""
input_tokens = output_tokens = cache_tokens = None
cost: float | None = None
def _accumulate(current: int | float | None, value: Any) -> int | float | None:
if not isinstance(value, (int, float)):
return current
return value if current is None else current + value
for llm in llms:
metrics = getattr(llm, "metrics", None)
if metrics is None:
continue
cost = _accumulate(cost, getattr(metrics, "accumulated_cost", None))
token_usage = getattr(metrics, "accumulated_token_usage", None)
if token_usage is None:
continue
input_tokens = _accumulate(input_tokens, getattr(token_usage, "prompt_tokens", None))
output_tokens = _accumulate(output_tokens, getattr(token_usage, "completion_tokens", None))
cache_tokens = _accumulate(cache_tokens, getattr(token_usage, "cache_read_tokens", None))
return {
"input_tokens": input_tokens,
"output_tokens": output_tokens,
"cache_tokens": cache_tokens,
# accumulated_cost is 0.0 when litellm has no pricing for the model
# (e.g. an aliased model behind a proxy); treat that as "unknown" so
# downstream cost reporting can fall back to provider dashboards.
"cost_usd": cost if cost else None,
}
# Tool observations larger than this (in chars) are kept intact. The OpenHands
# SDK otherwise hard-truncates every tool result at DEFAULT_TEXT_CONTENT_LIMIT
# (50_000) — see openhands/sdk/utils/truncate.py — which silently drops the tail
# of large reads (a 24-page SOP PDF is ~56 KB, big spreadsheet dumps similar).
# The worldbench baseline feeds tool outputs to the model untruncated (it only
# shortens them when building compaction summaries), so an aggressive per-call
# cap is a parity gap that costs OpenHands rubric points on SOP-heavy tasks.
# 1 MB clears every realistic task read with headroom while still guarding
# against a pathological multi-MB blob blowing the context window.
TOOL_OBSERVATION_CHAR_LIMIT = 1_000_000
def _raise_tool_observation_limit() -> None:
"""Lift the SDK's hardcoded 50K tool-observation truncation cap.
The limit is a module-level constant (the per-message ``enable_truncation``
field is deprecated), and ``message.py`` binds it at import time, so both the
source module and that imported reference must be patched.
"""
import openhands.sdk.llm.message as _message
import openhands.sdk.utils.truncate as _truncate
_truncate.DEFAULT_TEXT_CONTENT_LIMIT = TOOL_OBSERVATION_CHAR_LIMIT
_message.DEFAULT_TEXT_CONTENT_LIMIT = TOOL_OBSERVATION_CHAR_LIMIT
def run(config: dict[str, Any]) -> dict[str, Any]:
from pydantic import SecretStr
from openhands.sdk import (
LLM,
Agent,
Conversation,
ConversationExecutionStatus,
Event,
LLMSummarizingCondenser,
MessageEvent,
)
from openhands.sdk.event import ActionEvent, AgentErrorEvent
from openhands.sdk.llm.message import content_to_str
_raise_tool_observation_limit()
model_name = config["model"]
litellm_model, api_key, base_url = _resolve_llm_auth(model_name)
llm_kwargs, reasoning_effort = _resolve_llm_kwargs(config)
# Opt-in per-call request logging (--ak log_completions=true). Point the
# folder at the trajectory's log dir so it's downloaded with the trial. Each
# logged payload records the kwargs actually sent to the model (e.g.
# reasoning_effort), giving a verifiable record of what was requested.
if llm_kwargs.pop("log_completions", False):
log_root = config.get("logDir") or WORKSPACE_DIR
llm_kwargs["log_completions"] = True
llm_kwargs["log_completions_folder"] = os.path.join(log_root, "completions")
_log(f"completion logging enabled -> {llm_kwargs['log_completions_folder']}")
_log(
f"Running OpenHands loop (model={litellm_model}, "
f"base_url={base_url or 'default'})"
)
# timeout=600 matches worldbench's ~10-min per-call ceiling (anthropic-sdk.ts
# uses `timeout: 600000 - 1`). The OpenHands LLM default is 300s, and
# litellm.Timeout isn't in its retryable set, so a slow proxy call surfaces as
# a fatal RuntimeError and drops the trial — doubling the ceiling matches the
# baseline and cuts those spurious exclusions.
llm = LLM(
usage_id="agent",
model=litellm_model,
api_key=SecretStr(api_key) if api_key else None,
base_url=base_url,
timeout=600,
**llm_kwargs,
)
# Context compaction, for parity with worldbench (which summarizes old turns
# to stay within the window). We use the SDK defaults (event-count trigger at
# max_size=240, keep_first=2): for these tasks neither harness's compaction
# actually fires, but this gives the *reactive* recovery path — on a real
# context-window overflow the SDK emits a CondensationRequest and retries with
# a summarized history instead of hard-erroring the trial. The summarizer runs
# on the same model/proxy as the agent (worldbench summarizes with its agent
# model too); a separate LLM instance just keeps its metrics distinct.
condenser_llm = LLM(
usage_id="condenser",
model=litellm_model,
api_key=SecretStr(api_key) if api_key else None,
base_url=base_url,
timeout=600,
**llm_kwargs,
)
if reasoning_effort is not None:
if litellm_model.startswith("anthropic/"):
llm.reasoning_effort = reasoning_effort
condenser_llm.reasoning_effort = reasoning_effort
else:
rb = {"reasoning": {"effort": reasoning_effort}}
llm.litellm_extra_body = {**llm.litellm_extra_body, **rb}
condenser_llm.litellm_extra_body = {**condenser_llm.litellm_extra_body, **rb}
# Pass the office-assistant prompt as `system_prompt` (verbatim full
# replacement) rather than an AgentContext suffix — otherwise OpenHands'
# default SWE-coding system prompt dominates and diverges from worldbench,
# which uses the office-assistant prompt as *the* system message.
agent = Agent(
llm=llm,
tools=[],
mcp_config={
"mcpServers": {
"syntara": {
"url": config["mcpUrl"],
"timeout": MCP_TOOL_TIMEOUT,
}
}
},
system_prompt=config["systemPrompt"],
condenser=LLMSummarizingCondenser(llm=condenser_llm),
)
n_tool_calls = 0
n_agent_errors = 0
final_output = ""
error_message: str | None = None
def callback(event: Event) -> None:
nonlocal n_tool_calls, n_agent_errors, final_output, error_message
if isinstance(event, ActionEvent):
n_tool_calls += 1
elif isinstance(event, AgentErrorEvent):
n_agent_errors += 1
error_message = getattr(event, "error", None) or str(event)
elif isinstance(event, MessageEvent) and event.source == "agent":
text = "".join(content_to_str(event.to_llm_message().content))
if text:
final_output = text
conversation = Conversation(
agent=agent,
callbacks=[callback],
workspace=WORKSPACE_DIR,
persistence_dir=STATE_DIR,
max_iteration_per_run=int(config["maxToolCalls"]),
)
saw_error = False
try:
conversation.send_message(config["instruction"])
conversation.run()
except Exception as e: # provider/transport/loop failure
saw_error = True
error_message = error_message or str(e)
_log(f"OpenHands conversation.run() raised: {e}")
status = getattr(conversation.state, "execution_status", None)
status_val = getattr(status, "value", status)
infra_failure = saw_error or status_val == ConversationExecutionStatus.ERROR.value
did_work = n_tool_calls > 0 or bool(final_output)
if saw_error:
# conversation.run() raised (e.g. litellm.Timeout / provider error): the
# loop was cut short and never completed, so this is a transport/infra
# failure even if the agent had already made tool calls.
stopped_reason = "error"
elif infra_failure and not did_work:
# ERROR status with nothing produced — genuine failure.
stopped_reason = "error"
elif status_val == ConversationExecutionStatus.STUCK.value:
stopped_reason = "stuck"
elif status_val == ConversationExecutionStatus.FINISHED.value:
stopped_reason = "end_turn"
else:
stopped_reason = "max_tool_calls"
usage = _usage(llm, condenser_llm)
return {
"agent_id": "openhands_sdk",
"model": litellm_model,
"final_output": final_output,
"n_tool_calls": n_tool_calls,
"n_agent_errors": n_agent_errors,
"stopped_reason": stopped_reason,
"error_message": error_message,
"input_tokens": usage["input_tokens"],
"output_tokens": usage["output_tokens"],
"cache_tokens": usage["cache_tokens"],
"cost_usd": usage["cost_usd"],
}
def main() -> None:
if len(sys.argv) != 3:
_log("usage: openhands_runner.py <config.json> <output.json>")
sys.exit(2)
config_path, output_path = sys.argv[1], sys.argv[2]
config = json.loads(open(config_path).read())
try:
result = run(config)
except Exception as e:
# Hard failure before/around the loop — still emit a trajectory so the
# host can surface it as an errored (excluded) trial rather than a crash.
result = {
"agent_id": "openhands_sdk",
"model": config.get("model"),
"final_output": "",
"n_tool_calls": 0,
"stopped_reason": "error",
"error_message": str(e),
"input_tokens": None,
"output_tokens": None,
"cache_tokens": None,
"cost_usd": None,
}
os.makedirs(os.path.dirname(output_path) or ".", exist_ok=True)
with open(output_path, "w") as f:
json.dump(result, f, indent=2)
_log(
f"wrote trajectory to {output_path} "
f"(stopped_reason={result.get('stopped_reason')}, "
f"n_tool_calls={result.get('n_tool_calls')})"
)
if __name__ == "__main__":
main()
agent_harness/src/agent_harness/sop_verifier.py
+135
View File
@@ -0,0 +1,135 @@
#!/usr/bin/env python3
"""Run bundled SOP Python verifiers inside a Harbor task container."""
from __future__ import annotations
import json
import shutil
import sys
import tempfile
import traceback
from pathlib import Path
from typing import Any
WORKDIR = Path("/workdir")
DATA_DIR = Path("/data")
INITIAL_DATA_DIR = Path("/initial_data")
TESTS_DIR = Path("/tests")
VERIFIER_DIR = Path("/logs/verifier")
SERVICE_COMPAT_FILES: dict[str, tuple[str, tuple[str, ...]]] = {
"slack": ("slack.json", ("slack.json", "slack_data.json")),
"google_mail": ("inbox.json", ("inbox.json", "mailbox.json")),
"google_calendar": ("calendar_data.json", ("calendar_data.json", "calendar.json")),
"jira": ("jira_state.json", ("jira_state.json", "jira_data.json")),
"shopify": ("shopify_data.json", ("shopify_data.json",)),
}
def _state_path(service: str, seed_name: str) -> Path | None:
candidates = [
DATA_DIR / service / "final.json",
DATA_DIR / service / seed_name,
INITIAL_DATA_DIR / service / seed_name,
]
return next((p for p in candidates if p.is_file()), None)
def _build_compat_external_services(dest: Path) -> None:
dest.mkdir(parents=True, exist_ok=True)
for service, (seed_name, compat_names) in SERVICE_COMPAT_FILES.items():
src = _state_path(service, seed_name)
if src is not None:
for compat_name in compat_names:
shutil.copy2(src, dest / compat_name)
def _coerce_result(raw: Any) -> dict[str, Any]:
if isinstance(raw, dict):
passed = bool(raw.get("pass", raw.get("passed", False)))
score = raw.get("score", 1.0 if passed else 0.0)
try:
score = float(score)
except (TypeError, ValueError):
score = 1.0 if passed else 0.0
return {
"pass": passed,
"score": max(0.0, min(1.0, score)),
"feedback": str(raw.get("feedback", "")),
}
passed = bool(raw)
return {"pass": passed, "score": 1.0 if passed else 0.0, "feedback": str(raw)}
def _run_one(rubric: dict[str, Any], external_services_path: Path) -> dict[str, Any]:
rubric_id = str(rubric.get("id") or "rubric")
code = rubric.get("verifier_code")
if not isinstance(code, str) or not code.strip():
return {
"id": rubric_id,
"pass": False,
"score": 0.0,
"feedback": "rubric has no verifier_code",
}
namespace: dict[str, Any] = {"__builtins__": __builtins__}
try:
exec(compile(code, f"<{rubric_id}>", "exec"), namespace)
verify = namespace.get("verify")
if not callable(verify):
raise RuntimeError("verifier_code did not define verify()")
result = _coerce_result(verify(str(WORKDIR), str(external_services_path)))
return {"id": rubric_id, **result}
except Exception:
return {
"id": rubric_id,
"pass": False,
"score": 0.0,
"feedback": traceback.format_exc(),
}
def main() -> None:
rubrics_path = TESTS_DIR / "rubrics.json"
if not rubrics_path.is_file():
print("[sop-verifier] ERROR: rubrics.json not found", file=sys.stderr)
sys.exit(1)
rubrics = json.loads(rubrics_path.read_text())
if not isinstance(rubrics, list):
print("[sop-verifier] ERROR: rubrics.json must be a list", file=sys.stderr)
sys.exit(1)
with tempfile.TemporaryDirectory(prefix="sop-external-services-") as tmp:
compat_dir = Path(tmp)
_build_compat_external_services(compat_dir)
results = [_run_one(r, compat_dir) for r in rubrics]
total = len(results)
passed = sum(1 for r in results if r.get("pass"))
average_score = round(
sum(float(r.get("score", 0.0)) for r in results) / total,
4,
) if total else 0.0
print(f"[sop-verifier] {passed}/{total} rubrics passed; score={average_score:.2f}")
for result in results:
status = "PASS" if result.get("pass") else "FAIL"
feedback = str(result.get("feedback", "")).replace("\n", " ")[:500]
print(f" [{status}] {result.get('id')}: {feedback}")
output = {
"passed": passed == total,
"rubrics_passed": passed,
"rubrics_total": total,
"score": average_score,
"rubric_results": results,
}
(TESTS_DIR / "results.json").write_text(json.dumps(output, indent=2) + "\n")
VERIFIER_DIR.mkdir(parents=True, exist_ok=True)
(VERIFIER_DIR / "reward.txt").write_text(str(average_score))
if __name__ == "__main__":
main()
agent_harness/uv.lock
Generated
+3099
View File
File diff suppressed because it is too large Load Diff
docker/.dockerignore
View File
docker/Dockerfile
+88
View File
@@ -0,0 +1,88 @@
# syntax=docker/dockerfile:1.7
FROM python:3.13-slim
ARG GIT_SHA=unknown
ENV WORKDIR=/workdir
RUN groupadd -g 1000 model && \
useradd -m -u 1000 -g 1000 model && \
gpasswd -d model root || true
RUN apt-get update && \
apt-get install -y --no-install-recommends \
awscli sudo curl zip unzip poppler-utils imagemagick && \
apt-get clean
RUN pip install --no-cache-dir \
uv==0.11.8 \
pdf2image==1.16.3 \
pandas==2.2.3 \
numpy==2.4.3
WORKDIR /app
# Deps layer: copy only manifests/lockfiles so `uv sync` stays cached when
# source changes. The list below is explicit by design —
# `just check-dockerfile-manifests` (also run in CI lint) fails if a new
# package gets added without being listed here. All packages are Python
# (jira/slack were ported from TS).
COPY pyproject.toml uv.lock /app/
COPY packages/core/pyproject.toml /app/packages/core/
COPY packages/google_calendar/pyproject.toml /app/packages/google_calendar/
COPY packages/google_mail/pyproject.toml /app/packages/google_mail/
COPY packages/jira/pyproject.toml /app/packages/jira/
COPY packages/mcp_proxy/pyproject.toml /app/packages/mcp_proxy/
COPY packages/read_file_safe/pyproject.toml /app/packages/read_file_safe/
COPY packages/shopify/pyproject.toml /app/packages/shopify/
COPY packages/slack/pyproject.toml /app/packages/slack/
COPY packages/syntara/pyproject.toml /app/packages/syntara/
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-install-workspace --all-extras
# Source layer.
COPY . /app
# Finalize workspace package install. External deps were cached above, so this
# step only links editable workspace packages and is fast.
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --all-packages --all-extras
# OpenHands runner: a single file that must be staged into the build context at
# openhands-runner/openhands_runner.py before `docker build` runs (it lands at
# /app/openhands-runner/openhands_runner.py via the source-layer COPY above).
# The Harbor OpenHandsAgent execs this runner in-container, so the image carries
# it with openhands-sdk in an isolated venv (kept separate from /app/.venv so its
# deps never collide with the MCP env).
RUN test -f /app/openhands-runner/openhands_runner.py || { \
echo "openhands_runner.py missing from build context —" \
"stage it into openhands-runner/ before building" >&2; exit 1; }
# This isolated, eval-only runner venv is not bound by the repo-wide
# `[tool.uv] exclude-newer` supply-chain cutoff (docker/pyproject.toml, ~14 days
# back): openhands-sdk and its deps (litellm, etc.) release continuously and are
# newer than the cutoff. Override it wholesale for just this install with a fixed
# date so resolution is reproducible. Bump the date whenever you bump the pin.
RUN --mount=type=cache,target=/root/.cache/uv \
uv venv /app/openhands-runner/.venv --python 3.13 && \
uv pip install --python /app/openhands-runner/.venv/bin/python \
--exclude-newer 2026-06-17T00:00:00Z \
openhands-sdk==1.28.1
# Verifier deps: tests/test.sh pip-installs these at runtime; baking them in
# makes the verifier fast and network-independent.
RUN pip install --no-cache-dir openpyxl pdfplumber python-docx
RUN mkdir -p ${WORKDIR} && chown -R model:model ${WORKDIR}
# Lock down sibling MCP-server source from the model user (uid 1000). syntara
# is the only server that runs as model; the rest run as root and never need
# to be readable by the agent. syntara eagerly imports its full dependency
# tree (incl. read_file_safe) before drop_privileges, so nothing else under
# /app/packages needs to stay readable post-setuid. /app/packages itself stays
# traversable so agents get a clean Permission denied rather than "directory
# doesn't exist".
RUN find /app/packages -mindepth 1 -maxdepth 1 -type d ! -name syntara \
-exec chown -R root:root {} + -exec chmod -R go-rwx {} +
RUN echo "$GIT_SHA" > /app/.git-sha
WORKDIR ${WORKDIR}
docker/openhands-runner/openhands_runner.py
+359
View File
@@ -0,0 +1,359 @@
"""In-container OpenHands agent runner.
Runs the OpenHands SDK agent loop *inside* the task container, talking to the
syntara MCP proxy at ``localhost:8000/mcp`` (reachable in-container on every
harbor environment, including Modal sandboxes where the host can't reach the
port).
Baked into the ``syntara`` image at build time (this
single file is staged to ``/app/openhands-runner/openhands_runner.py`` next to an
isolated venv holding ``openhands-sdk``). The host-side ``OpenHandsAgent`` uploads
a config JSON, execs this script, and downloads the trajectory it writes.
Contract:
- argv[1] = path to a config JSON: {instruction, systemPrompt, model, mcpUrl,
maxToolCalls}.
- argv[2] = path to write the result/trajectory JSON to. This is a **dedicated
file**, NOT stdout: the OpenHands SDK prints a human-readable transcript to
stdout, so the machine-readable result must go to its own file for the host
to parse it.
- The process exits 0 even when the agent loop errored — the error is recorded
in the trajectory's ``stopped_reason``/``error_message`` so the host decides
whether to fail the trial.
"""
from __future__ import annotations
import json
import os
import sys
from typing import Any
MCP_TOOL_TIMEOUT = 300
WORKSPACE_DIR = "/tmp/openhands_workspace"
STATE_DIR = "/tmp/openhands_state"
def _log(msg: str) -> None:
print(msg, file=sys.stderr, flush=True)
def _resolve_llm_kwargs(config: dict[str, Any]) -> tuple[dict[str, Any], str | None]:
"""Split LLM kwargs into ``(constructor_kwargs, reasoning_effort)``.
``reasoning_effort`` is pulled out so it can be applied post-construction.
openhands.sdk otherwise won't accept `max` as a value.
"""
llm_kwargs = dict(config.get("llmKwargs") or {})
effort = llm_kwargs.pop("reasoning_effort", None)
if effort is not None:
effort = str(effort).strip().lower() or None
return llm_kwargs, effort
def _resolve_llm_auth(model_name: str) -> tuple[str, str | None, str | None]:
"""Map harbor's ``provider/model`` id to a litellm (model, api_key, base_url).
``openrouter/anthropic/<model>`` is rewritten to ``anthropic/<model>`` (dots
-> dashes) so Anthropic models route through the Anthropic provider, never
OpenRouter. To send Anthropic traffic through an LLM proxy, point
``ANTHROPIC_BASE_URL`` at the proxy endpoint and set ``ANTHROPIC_API_KEY`` to
the proxy token.
"""
if "/" not in model_name:
raise ValueError(
f"model must be in 'provider/model' form; got {model_name!r}"
)
provider, model = model_name.split("/", 1)
if provider == "openrouter" and model.startswith("anthropic/"):
provider = "anthropic"
model = model.split("/", 1)[1].replace(".", "-")
if provider == "anthropic":
litellm_model = f"anthropic/{model}"
return (
litellm_model,
os.environ.get("ANTHROPIC_API_KEY"),
os.environ.get("ANTHROPIC_BASE_URL"),
)
_provider_key_env = {
"openai": "OPENAI_API_KEY",
"openrouter": "OPENROUTER_API_KEY",
"google": "GEMINI_API_KEY",
"gemini": "GEMINI_API_KEY",
}
api_key = os.environ.get(_provider_key_env.get(provider, ""))
base_url = os.environ.get("OPENAI_BASE_URL") if provider == "openai" else None
return model_name, api_key, base_url
def _usage(*llms: Any) -> dict[str, int | float | None]:
"""Pull token/cost totals from one or more LLMs' metrics, tolerant of API drift.
Sums across every LLM passed in (e.g. the agent loop plus the condenser) so
the reported cost and cache usage reflect *all* model traffic for the trial,
not just the main agent call. Each field stays ``None`` if no LLM exposed it,
so a metrics-shape change degrades to null rather than crashing the trial.
"""
input_tokens = output_tokens = cache_tokens = None
cost: float | None = None
def _accumulate(current: int | float | None, value: Any) -> int | float | None:
if not isinstance(value, (int, float)):
return current
return value if current is None else current + value
for llm in llms:
metrics = getattr(llm, "metrics", None)
if metrics is None:
continue
cost = _accumulate(cost, getattr(metrics, "accumulated_cost", None))
token_usage = getattr(metrics, "accumulated_token_usage", None)
if token_usage is None:
continue
input_tokens = _accumulate(input_tokens, getattr(token_usage, "prompt_tokens", None))
output_tokens = _accumulate(output_tokens, getattr(token_usage, "completion_tokens", None))
cache_tokens = _accumulate(cache_tokens, getattr(token_usage, "cache_read_tokens", None))
return {
"input_tokens": input_tokens,
"output_tokens": output_tokens,
"cache_tokens": cache_tokens,
# accumulated_cost is 0.0 when litellm has no pricing for the model
# (e.g. an aliased model behind a proxy); treat that as "unknown" so
# downstream cost reporting can fall back to provider dashboards.
"cost_usd": cost if cost else None,
}
# Tool observations larger than this (in chars) are kept intact. The OpenHands
# SDK otherwise hard-truncates every tool result at DEFAULT_TEXT_CONTENT_LIMIT
# (50_000) — see openhands/sdk/utils/truncate.py — which silently drops the tail
# of large reads (a 24-page SOP PDF is ~56 KB, big spreadsheet dumps similar).
# The worldbench baseline feeds tool outputs to the model untruncated (it only
# shortens them when building compaction summaries), so an aggressive per-call
# cap is a parity gap that costs OpenHands rubric points on SOP-heavy tasks.
# 1 MB clears every realistic task read with headroom while still guarding
# against a pathological multi-MB blob blowing the context window.
TOOL_OBSERVATION_CHAR_LIMIT = 1_000_000
def _raise_tool_observation_limit() -> None:
"""Lift the SDK's hardcoded 50K tool-observation truncation cap.
The limit is a module-level constant (the per-message ``enable_truncation``
field is deprecated), and ``message.py`` binds it at import time, so both the
source module and that imported reference must be patched.
"""
import openhands.sdk.llm.message as _message
import openhands.sdk.utils.truncate as _truncate
_truncate.DEFAULT_TEXT_CONTENT_LIMIT = TOOL_OBSERVATION_CHAR_LIMIT
_message.DEFAULT_TEXT_CONTENT_LIMIT = TOOL_OBSERVATION_CHAR_LIMIT
def run(config: dict[str, Any]) -> dict[str, Any]:
from pydantic import SecretStr
from openhands.sdk import (
LLM,
Agent,
Conversation,
ConversationExecutionStatus,
Event,
LLMSummarizingCondenser,
MessageEvent,
)
from openhands.sdk.event import ActionEvent, AgentErrorEvent
from openhands.sdk.llm.message import content_to_str
_raise_tool_observation_limit()
model_name = config["model"]
litellm_model, api_key, base_url = _resolve_llm_auth(model_name)
llm_kwargs, reasoning_effort = _resolve_llm_kwargs(config)
# Opt-in per-call request logging (--ak log_completions=true). Point the
# folder at the trajectory's log dir so it's downloaded with the trial. Each
# logged payload records the kwargs actually sent to the model (e.g.
# reasoning_effort), giving a verifiable record of what was requested.
if llm_kwargs.pop("log_completions", False):
log_root = config.get("logDir") or WORKSPACE_DIR
llm_kwargs["log_completions"] = True
llm_kwargs["log_completions_folder"] = os.path.join(log_root, "completions")
_log(f"completion logging enabled -> {llm_kwargs['log_completions_folder']}")
_log(
f"Running OpenHands loop (model={litellm_model}, "
f"base_url={base_url or 'default'})"
)
# timeout=600 matches worldbench's ~10-min per-call ceiling (anthropic-sdk.ts
# uses `timeout: 600000 - 1`). The OpenHands LLM default is 300s, and
# litellm.Timeout isn't in its retryable set, so a slow proxy call surfaces as
# a fatal RuntimeError and drops the trial — doubling the ceiling matches the
# baseline and cuts those spurious exclusions.
llm = LLM(
usage_id="agent",
model=litellm_model,
api_key=SecretStr(api_key) if api_key else None,
base_url=base_url,
timeout=600,
**llm_kwargs,
)
# Context compaction, for parity with worldbench (which summarizes old turns
# to stay within the window). We use the SDK defaults (event-count trigger at
# max_size=240, keep_first=2): for these tasks neither harness's compaction
# actually fires, but this gives the *reactive* recovery path — on a real
# context-window overflow the SDK emits a CondensationRequest and retries with
# a summarized history instead of hard-erroring the trial. The summarizer runs
# on the same model/proxy as the agent (worldbench summarizes with its agent
# model too); a separate LLM instance just keeps its metrics distinct.
condenser_llm = LLM(
usage_id="condenser",
model=litellm_model,
api_key=SecretStr(api_key) if api_key else None,
base_url=base_url,
timeout=600,
**llm_kwargs,
)
if reasoning_effort is not None:
if litellm_model.startswith("anthropic/"):
llm.reasoning_effort = reasoning_effort
condenser_llm.reasoning_effort = reasoning_effort
else:
rb = {"reasoning": {"effort": reasoning_effort}}
llm.litellm_extra_body = {**llm.litellm_extra_body, **rb}
condenser_llm.litellm_extra_body = {**condenser_llm.litellm_extra_body, **rb}
# Pass the office-assistant prompt as `system_prompt` (verbatim full
# replacement) rather than an AgentContext suffix — otherwise OpenHands'
# default SWE-coding system prompt dominates and diverges from worldbench,
# which uses the office-assistant prompt as *the* system message.
agent = Agent(
llm=llm,
tools=[],
mcp_config={
"mcpServers": {
"syntara": {
"url": config["mcpUrl"],
"timeout": MCP_TOOL_TIMEOUT,
}
}
},
system_prompt=config["systemPrompt"],
condenser=LLMSummarizingCondenser(llm=condenser_llm),
)
n_tool_calls = 0
n_agent_errors = 0
final_output = ""
error_message: str | None = None
def callback(event: Event) -> None:
nonlocal n_tool_calls, n_agent_errors, final_output, error_message
if isinstance(event, ActionEvent):
n_tool_calls += 1
elif isinstance(event, AgentErrorEvent):
n_agent_errors += 1
error_message = getattr(event, "error", None) or str(event)
elif isinstance(event, MessageEvent) and event.source == "agent":
text = "".join(content_to_str(event.to_llm_message().content))
if text:
final_output = text
conversation = Conversation(
agent=agent,
callbacks=[callback],
workspace=WORKSPACE_DIR,
persistence_dir=STATE_DIR,
max_iteration_per_run=int(config["maxToolCalls"]),
)
saw_error = False
try:
conversation.send_message(config["instruction"])
conversation.run()
except Exception as e: # provider/transport/loop failure
saw_error = True
error_message = error_message or str(e)
_log(f"OpenHands conversation.run() raised: {e}")
status = getattr(conversation.state, "execution_status", None)
status_val = getattr(status, "value", status)
infra_failure = saw_error or status_val == ConversationExecutionStatus.ERROR.value
did_work = n_tool_calls > 0 or bool(final_output)
if saw_error:
# conversation.run() raised (e.g. litellm.Timeout / provider error): the
# loop was cut short and never completed, so this is a transport/infra
# failure even if the agent had already made tool calls.
stopped_reason = "error"
elif infra_failure and not did_work:
# ERROR status with nothing produced — genuine failure.
stopped_reason = "error"
elif status_val == ConversationExecutionStatus.STUCK.value:
stopped_reason = "stuck"
elif status_val == ConversationExecutionStatus.FINISHED.value:
stopped_reason = "end_turn"
else:
stopped_reason = "max_tool_calls"
usage = _usage(llm, condenser_llm)
return {
"agent_id": "openhands_sdk",
"model": litellm_model,
"final_output": final_output,
"n_tool_calls": n_tool_calls,
"n_agent_errors": n_agent_errors,
"stopped_reason": stopped_reason,
"error_message": error_message,
"input_tokens": usage["input_tokens"],
"output_tokens": usage["output_tokens"],
"cache_tokens": usage["cache_tokens"],
"cost_usd": usage["cost_usd"],
}
def main() -> None:
if len(sys.argv) != 3:
_log("usage: openhands_runner.py <config.json> <output.json>")
sys.exit(2)
config_path, output_path = sys.argv[1], sys.argv[2]
config = json.loads(open(config_path).read())
try:
result = run(config)
except Exception as e:
# Hard failure before/around the loop — still emit a trajectory so the
# host can surface it as an errored (excluded) trial rather than a crash.
result = {
"agent_id": "openhands_sdk",
"model": config.get("model"),
"final_output": "",
"n_tool_calls": 0,
"stopped_reason": "error",
"error_message": str(e),
"input_tokens": None,
"output_tokens": None,
"cache_tokens": None,
"cost_usd": None,
}
os.makedirs(os.path.dirname(output_path) or ".", exist_ok=True)
with open(output_path, "w") as f:
json.dump(result, f, indent=2)
_log(
f"wrote trajectory to {output_path} "
f"(stopped_reason={result.get('stopped_reason')}, "
f"n_tool_calls={result.get('n_tool_calls')})"
)
if __name__ == "__main__":
main()
docker/packages/core/core/__init__.py
View File
docker/packages/core/core/_token.py
+39
View File
@@ -0,0 +1,39 @@
"""Proxy auth token capture for core.
The proxy injects ``MCP_PROXY_TOKEN`` into core's env so the viewer's
``ProxyTokenMiddleware`` can authenticate inbound non-MCP requests.
This code captures the token at import time, makes it available via ``get_proxy_token()``,
and pops it from ``os.environ`` with ``@capture_proxy_token``, so it doesn't reach the agent's bash/python.
"""
from __future__ import annotations
import os
from collections.abc import Callable
from functools import wraps
_TOKEN: str = ""
def capture_proxy_token[**P, R](fn: Callable[P, R]) -> Callable[P, R]:
"""Decorator: before running ``fn``, pop ``MCP_PROXY_TOKEN`` from
``os.environ`` and stash it for later reads via :func:`get_proxy_token`.
Apply this to ``core.server.main()`` — the first thing that runs
when core starts. The pop happens before any tool can be invoked
so subprocesses can't recover the token from ``/proc/$PPID/environ``.
"""
@wraps(fn)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
global _TOKEN
_TOKEN = os.environ.pop("MCP_PROXY_TOKEN", "")
return fn(*args, **kwargs)
return wrapper
def get_proxy_token() -> str:
"""Return the captured proxy auth token, or ``""`` if not yet captured."""
return _TOKEN
docker/packages/core/core/async_tool_guard.py
+59
View File
@@ -0,0 +1,59 @@
"""Boot-time guard: every registered MCP tool must be async.
Sync (`def`) tools make FastMCP run pydantic argument validation in an anyio
worker threadpool. Under concurrent calls the shared pydantic-core validator is
re-entered across threads and panics with ``pyo3_runtime.PanicException:
dictionary changed size during iteration``, which tears down the
StreamableHTTP task group and turns every later request into a 500. Async
(`async def`) tools validate on the single-threaded event loop and are safe.
This guard runs at server boot (and therefore during ``mcp-proxy gen``, which
boots every server): a non-conformant package fails fast instead of shipping a
server that can crash under load. The same check is intentionally duplicated in
each package because the bundled prod images install only that package's own
dependencies, so there is no shared runtime module to import.
"""
from __future__ import annotations
import asyncio
import inspect
from functools import partial
from typing import Any
def _unwrap(fn: Any) -> Any:
while isinstance(fn, partial):
fn = fn.func
return fn
def find_sync_tools(mcp: Any) -> list[str]:
"""Return the names of registered tools whose function is not a coroutine."""
# mcp.server.fastmcp.FastMCP exposes a synchronous tool manager.
manager = getattr(mcp, "_tool_manager", None)
if manager is not None and hasattr(manager, "list_tools"):
return sorted(t.name for t in manager.list_tools() if not inspect.iscoroutinefunction(_unwrap(t.fn)))
# fastmcp.FastMCP exposes an async API.
async def _collect() -> list[str]:
sync: list[str] = []
for descriptor in await mcp.list_tools():
tool = await mcp.get_tool(descriptor.name)
if not inspect.iscoroutinefunction(_unwrap(tool.fn)):
sync.append(descriptor.name)
return sorted(sync)
return asyncio.run(_collect())
def assert_tools_async(mcp: Any) -> None:
"""Raise if any registered tool is synchronous."""
sync = find_sync_tools(mcp)
if sync:
raise RuntimeError(
"MCP tools must be async (`async def`). Sync tools run pydantic argument "
"validation in a worker threadpool and can trigger a pydantic-core "
"'dictionary changed size during iteration' panic under concurrent calls, "
"which kills the server. Make these tools async: " + ", ".join(sync)
)
docker/packages/core/core/privilege.py
+95
View File
@@ -0,0 +1,95 @@
"""Workdir/ownership helpers for core's root-server + per-exec-drop model.
Configured via ``packages/core/mcp.json``'s ``env`` block:
"env": {"SETUID": "1000", "SETGID": "1000", "HOME": "/home/model"}
**The core server process stays root.** Sibling servers (slack, jira,
google_mail, grading, …) also run as root because their tools just call mocked
APIs over HTTP — no shell, no file IO, no adversarial surface. Core is
different: it hands the agent ``bash``, so each of *those commands* is dropped
to ``SETUID``/``SETGID`` at spawn time
(``core.tools.sandbox._privilege_drop_kwargs``), while the server itself
keeps root. Keeping the server root (rather than dropping the whole process to
uid 1000, as it used to) is what lets the Dockerfile close ``/opt/venv`` to uid
1000: the server reads its venv as root, and the agent — a *different* effective
uid per command — cannot. ``/app`` stays sealed to ``0700 root:root`` and the
venv lives outside ``/app`` at ``/opt/venv``. See ``docker/base/Dockerfile`` and
the README's "Sandbox model" section.
This module keeps the bits that still run in the root server: provisioning
``WORKDIR`` and restoring ownership of files copied out of the root-owned source
tree into the model-owned workdir. The actual privilege drop now happens
per-command in ``sandbox`` — there is intentionally no function here that drops
the whole process.
When the process isn't root (local dev, CI, tests) these helpers are no-ops:
they can't ``mkdir`` at ``/`` or ``chown``, and the same ``mcp.json`` ships
``SETUID``/``SETGID`` everywhere harmlessly.
"""
from __future__ import annotations
import os
from core.tools import sandbox
def _target_ids() -> tuple[int, int] | None:
"""Return ``(uid, gid)`` from ``SETUID``/``SETGID`` env, or ``None`` if neither is set.
A missing one becomes ``-1`` (the ``os.chown`` sentinel for "leave unchanged").
"""
setuid = os.environ.get("SETUID")
setgid = os.environ.get("SETGID")
if setuid is None and setgid is None:
return None
uid = int(setuid) if setuid is not None else -1
gid = int(setgid) if setgid is not None else -1
return uid, gid
def ensure_workdir() -> None:
"""Make sure WORKDIR exists with the right owner BEFORE we drop privileges.
Some deploy environments don't pre-provision /workdir (the Dockerfile +
start.sh do, but bundle launchers / Modal workflows may not). The setup
hook needs to write into it, and the server needs to chdir there — so we
create it as root and chown to SETUID/SETGID before the drop, since uid
1000 can't mkdir at /.
Non-root callers (CI, local dev, tests) can't mkdir at / and can't chown,
so this is a no-op — WORKDIR there is either pre-provisioned (Docker, test
fixtures pointing at tmp dirs) or genuinely absent, in which case any
caller that actually needs the dir will fail loudly on its own.
"""
if os.geteuid() != 0:
return
workdir = sandbox.WORKDIR
os.makedirs(workdir, exist_ok=True)
ids = _target_ids()
if ids is None:
return
os.chown(workdir, *ids)
def chown_tree_to_target(path: str | os.PathLike[str]) -> None:
"""Recursively chown ``path`` to ``SETUID``/``SETGID``. No-op when not root.
Used by the setup hook after copying root-owned source files into
model-owned WORKDIR — ``shutil.copytree`` runs as root (so it can read
locked-down sources) and the new entries inherit root ownership; this
restores them to the unprivileged target user so the model can edit them.
Symlinks are chowned without following.
"""
if os.geteuid() != 0:
return
ids = _target_ids()
if ids is None:
return
uid, gid = ids
root_str = os.fspath(path)
os.chown(root_str, uid, gid, follow_symlinks=False)
for root, dirs, files in os.walk(root_str):
for name in dirs + files:
os.chown(os.path.join(root, name), uid, gid, follow_symlinks=False)
docker/packages/core/core/server.py
+56
View File
@@ -0,0 +1,56 @@
"""Core standalone MCP server entry point.
This module exposes core's tools as an MCP server that can be launched as a
subprocess and proxied by mcp_proxy or any other MCP proxy.
"""
import os
from fastmcp import FastMCP
from fastmcp.tools.function_tool import FunctionTool
import core.tools as tools
from core._token import capture_proxy_token
from core.async_tool_guard import assert_tools_async
from core.privilege import ensure_workdir
from core.tools import sandbox
from core.viewer import run_http_server
def build_app() -> FastMCP:
"""Build a FastMCP app exposing all core tools."""
app = FastMCP("core")
for tool_name in sorted(tools.__all__):
tool_fn = getattr(tools, tool_name, None)
if callable(tool_fn):
# output_schema=None matches the shipped mcp-tools.generated.json,
# which has never advertised output schemas.
app.add_tool(FunctionTool.from_function(fn=tool_fn, name=tool_name, output_schema=None))
return app
@capture_proxy_token
def main() -> None:
# The server intentionally keeps running as root: it needs to read the
# locked-down /app tree, and a root server lets us close /opt/venv to uid
# 1000. Privilege is dropped per agent command instead — see
# core.tools.sandbox._privilege_drop_kwargs (the chokepoint every
# bash/file-tool subprocess flows through).
ensure_workdir()
# Land the running server in the agent's workdir so any tool that doesn't
# pass cwd= explicitly resolves relative paths under /workdir. Skip when
# the dir doesn't exist (CI / local dev where /workdir isn't provisioned).
if os.path.isdir(sandbox.WORKDIR):
os.chdir(sandbox.WORKDIR)
app = build_app()
assert_tools_async(app)
port = os.environ.get("PORT")
if port:
run_http_server(app, int(port))
else:
app.run(show_banner=False)
if __name__ == "__main__":
main()
docker/packages/core/core/setup.py
+130
View File
@@ -0,0 +1,130 @@
"""Copy uploaded context files into the agent's workdir.
Lookup order for the source directory:
1. Unified bundle: $BUNDLEDIR/files/ (when a trajectory bundle is mounted)
2. Task-specific: {WORLDBENCH_ROOT}/tasks/setup_data/{WORLDBENCH_TASK_ID}/
3. Generic: {WORLDBENCH_ROOT}/setup_data/
BUNDLEDIR is set by the parent process (``scripts/start.sh`` for local
dev, the production harness in production) and points at the unpacked bundle root.
For (2)/(3) the source files live under a ``files/`` subdirectory; for (1)
the bundle root already has ``files/`` as a peer of ``services/``, ``memory/``,
and ``meta/``. In all cases, the contents of ``files/`` are copied into
``sandbox.WORKDIR`` (``/workdir``).
"""
import os
import shutil
import sys
from pathlib import Path
from core.privilege import chown_tree_to_target, ensure_workdir
from core.tools import sandbox
def _resolve_files_dir(world_root: Path, task_id: str | None) -> Path | None:
"""Return the source ``files/`` directory, or None if no setup data exists."""
bundle_dir = os.environ.get("BUNDLEDIR")
if bundle_dir:
bundle_files = Path(bundle_dir) / "files"
if bundle_files.is_dir():
return bundle_files
task_setup_dir = world_root / "tasks" / "setup_data" / task_id if task_id else None
if task_setup_dir and task_setup_dir.exists():
setup_dir = task_setup_dir
else:
setup_dir = world_root / "setup_data"
if not setup_dir.exists():
return None
files_dir = setup_dir / "files"
return files_dir if files_dir.is_dir() else None
def main() -> None:
# Setup runs entirely as root: the source tree (e.g. /app/setup_data/files/,
# extracted by the production harness as root) is not readable by uid 1000, so we copy
# before dropping privileges and then chown the result to SETUID/SETGID so
# the model user can modify it. This script then exits — the actual MCP
# server is a separate process invocation that does its own privilege drop.
ensure_workdir()
task_id = os.environ.get("WORLDBENCH_TASK_ID")
world_root = Path(os.environ.get("WORLDBENCH_ROOT", os.getcwd()))
files_dir = _resolve_files_dir(world_root, task_id)
if files_dir is None:
print("No setup data found, skipping")
sys.exit(0)
# Refuse symlinks and hardlinks anywhere in the source tree, and refuse
# symlinks already present at the destination. We run as root here (so
# we can read locked-down sources the model user can't see) and chown
# the destination to uid 1000 afterwards. Three ways the protected
# tree could be leaked back to the model otherwise:
# 1. Source symlink `files/rubrics -> /app/tasks` — shutil.copytree's
# default `symlinks=False` dereferences it and materializes the
# target into /workdir.
# 2. Source hardlink `files/foo` to `/app/packages/grading/x.py` —
# copytree sees a regular file with the same inode and writes a
# new copy into /workdir.
# 3. Destination symlink `/workdir/context -> /app/packages/grading`
# planted by a prior model run — copytree's `dirs_exist_ok=True`
# follows it and writes bundle contents *into* the protected
# directory as root.
workdir = Path(sandbox.WORKDIR)
_reject_unsafe_source(files_dir)
workdir.mkdir(parents=True, exist_ok=True)
_reject_dest_symlinks(workdir)
shutil.copytree(files_dir, workdir, dirs_exist_ok=True)
chown_tree_to_target(workdir)
print(f"Copied files from {files_dir} to {workdir}")
def _reject_unsafe_source(root: Path) -> None:
"""Exit non-zero if any entry under ``root`` is a symlink or a hardlink.
Hardlinks are detected via ``st_nlink > 1`` on regular files. Files in
the bundle should be unique copies; multi-link inodes mean the bundle
points at something we didn't create, which might be a protected path
on the same filesystem.
"""
if root.is_symlink():
print(f"Refusing to copy symlinked setup root: {root}", file=sys.stderr)
sys.exit(1)
for dirpath, dirnames, filenames in os.walk(root, followlinks=False):
for name in dirnames + filenames:
entry = Path(dirpath) / name
if entry.is_symlink():
print(f"Refusing to copy symlink in setup data: {entry}", file=sys.stderr)
sys.exit(1)
for name in filenames:
entry = Path(dirpath) / name
# lstat — don't follow symlinks (already rejected above, but be defensive).
st = entry.lstat()
if st.st_nlink > 1:
print(f"Refusing to copy hardlinked file in setup data: {entry}", file=sys.stderr)
sys.exit(1)
def _reject_dest_symlinks(root: Path) -> None:
"""Exit non-zero if any entry under destination ``root`` is a symlink.
A symlink planted in /workdir by a prior model run would let
``copytree(..., dirs_exist_ok=True)`` write through it as root, into
paths the model user can't normally write — or, after chown, read.
"""
if root.is_symlink():
print(f"Refusing to copy into symlinked workdir: {root}", file=sys.stderr)
sys.exit(1)
for dirpath, dirnames, filenames in os.walk(root, followlinks=False):
for name in dirnames + filenames:
entry = Path(dirpath) / name
if entry.is_symlink():
print(f"Refusing to copy over symlink in workdir: {entry}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
docker/packages/core/core/state_codec.py
+27
View File
@@ -0,0 +1,27 @@
"""State codec for core.
Syntara on main is file-sandbox-backed — no database entities to snapshot.
``export_state``/``import_state`` exist for uniformity with the other MCP
servers (the proxy validates every server has them); round-trip is
trivially empty.
"""
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, ConfigDict
class SyntaraState(BaseModel):
"""Empty state — core has no DB to snapshot."""
model_config = ConfigDict(extra="allow")
def state_to_json() -> dict[str, Any]:
return {}
def state_from_json(data: dict[str, Any]) -> None:
_ = data
docker/packages/core/core/tools/__init__.py
+22
View File
@@ -0,0 +1,22 @@
from .bash import bash
from .echo import echo
from .list_files import listFiles
from .prepare_grading_context import prepareGradingContext
from .read_file import readFile
from .read_media import readMedia
from .read_pdf import readPDF
from .state import export_state, import_state
from .write_file import writeFile
__all__ = [
"bash",
"echo",
"export_state",
"import_state",
"listFiles",
"prepareGradingContext",
"readFile",
"readMedia",
"readPDF",
"writeFile",
]
docker/packages/core/core/tools/bash.py
+25
View File
@@ -0,0 +1,25 @@
import asyncio
from typing import Annotated, Any
from pydantic import Field
from core.tools.sandbox import DEFAULT_TIMEOUT_SECONDS, MAX_TIMEOUT_SECONDS, run_in_sandbox
async def bash(
command: Annotated[str, Field(description="The bash command to execute")],
timeout_seconds: Annotated[
int | None,
Field(
description=f"Timeout in seconds (default {DEFAULT_TIMEOUT_SECONDS}, max {MAX_TIMEOUT_SECONDS})",
ge=1,
le=MAX_TIMEOUT_SECONDS,
),
] = None,
) -> dict[str, Any]:
"""Execute a bash command in an isolated directory."""
timeout = timeout_seconds or DEFAULT_TIMEOUT_SECONDS
# run_in_sandbox blocks (subprocess.run) for up to `timeout` seconds; offload
# to a worker thread so a slow command doesn't stall the event loop and block
# other concurrent tool calls. Validation already ran on the loop.
return await asyncio.to_thread(run_in_sandbox, ["bash", "-c", command], timeout)
docker/packages/core/core/tools/echo.py
+8
View File
@@ -0,0 +1,8 @@
from typing import Annotated
from pydantic import Field
async def echo(message: Annotated[str, Field(description="The message to echo")]) -> str:
"""Echoes a message"""
return message + message
docker/packages/core/core/tools/list_files.py
+80
View File
@@ -0,0 +1,80 @@
import asyncio
from typing import Annotated, Any
from pydantic import Field
from core.tools.sandbox import DEFAULT_TIMEOUT_SECONDS, run_in_sandbox
# Emit NUL-terminated records "d <name>\0" or "f <name>\0". NUL separation
# preserves filenames containing newlines, which are valid POSIX. dotglob
# includes hidden files (matching os.listdir); nullglob makes empty dirs
# produce no output.
_LIST_SCRIPT = (
"shopt -s dotglob nullglob; "
'if [ ! -e "$1" ]; then printf "No such directory: %s" "$1" >&2; exit 1; fi; '
'if [ ! -d "$1" ]; then printf "Not a directory: %s" "$1" >&2; exit 1; fi; '
'cd -- "$1" || exit 1; '
"for e in *; do "
'if [ -d "$e" ]; then printf "d %s\\0" "$e"; '
'else printf "f %s\\0" "$e"; fi; '
"done"
)
async def listFiles(
directory: Annotated[
str,
Field(
description=(
"Directory to list, relative to the sandbox root. Use /workdir/ prefix or an "
"absolute path within the sandbox to be explicit. Defaults to the sandbox root."
),
),
] = ".",
) -> dict[str, Any]:
"""List files and subdirectories in a directory inside the sandbox."""
# Binary mode so non-UTF-8 filenames don't raise UnicodeDecodeError before
# we get a chance to handle them. We decode each name individually below.
# Offload the blocking subprocess call to a worker thread so it doesn't stall
# the event loop; the fast result parsing below stays on the loop.
result = await asyncio.to_thread(
run_in_sandbox,
["bash", "-c", _LIST_SCRIPT, "--", directory],
DEFAULT_TIMEOUT_SECONDS,
text=False,
)
stderr_bytes = result.get("stderr") or b""
stderr_str = stderr_bytes.decode("utf-8", errors="replace") if isinstance(stderr_bytes, bytes) else stderr_bytes
if result["returncode"] != 0:
return {
"directory": directory,
"files": [],
"directories": [],
"returncode": result["returncode"],
"stderr": stderr_str or result.get("error", ""),
}
files: list[str] = []
directories: list[str] = []
for entry in result["stdout"].split(b"\x00"):
if not entry:
continue
kind, _, name_bytes = entry.partition(b" ")
name = name_bytes.decode("utf-8", errors="replace")
if kind == b"d":
directories.append(name)
elif kind == b"f":
files.append(name)
files.sort()
directories.sort()
return {
"directory": directory,
"files": files,
"directories": directories,
"returncode": 0,
"stderr": "",
}
docker/packages/core/core/tools/prepare_grading_context.py
+538
View File
@@ -0,0 +1,538 @@
"""Prepare grading context for LLM-based rubric evaluation.
Collects file evidence from the sandbox, formats as XML, and assembles
the full text_for_grading string by prepending file evidence to the
agent's final output.
"""
import asyncio
import io
import logging
import os
from core.tools import sandbox
logger = logging.getLogger(__name__)
# Default cap on the number of files included as evidence. Sized to comfortably
# cover a task's read-only input documents PLUS the agent's deliverables: the
# walk is flat + alphabetical, so input files can otherwise consume every slot
# and silently drop the agent-created files the rubric actually grades. When the
# cap IS hit, the dropped files are surfaced both in a log line and as a
# <files_omitted_due_to_max_files> marker inside the <workspace_files> XML, so an
# empty or partial listing is never mistaken for "the agent created nothing."
DEFAULT_MAX_FILES = 100
# Per-file content budget that ends up in the grading prompt. The ceiling on the
# whole evidence block is DEFAULT_MAX_FILES * this, so the constraint isn't memory
# but the grader's context window and cost — every byte here is fed to the rubric
# LLM. 250 KB gives large multi-tab spreadsheets and multi-slide decks room to
# show every tab/slide (the per-sheet/per-slide budgeting below distributes it),
# while keeping the worst case bounded. Realistic tasks have a handful of
# substantive files, so actual prompts stay far below the 100-file ceiling.
DEFAULT_MAX_CONTENT_BYTES = 250_000
TEXT_EXTENSIONS = {
"txt",
"csv",
"json",
"jsonl",
"py",
"js",
"ts",
"md",
"html",
"xml",
"yaml",
"yml",
"toml",
"cfg",
"ini",
"log",
"sh",
"bash",
"sql",
"r",
"rb",
"java",
"c",
"cpp",
"h",
"hpp",
"css",
"scss",
"tex",
"rst",
}
SPECIAL_EXTENSIONS = {"pdf", "docx", "xlsx", "pptx"}
ALL_SUPPORTED_EXTENSIONS = TEXT_EXTENSIONS | SPECIAL_EXTENSIONS
# Special formats (PDF/Office) can't be truncated without corrupting the
# container, so extraction needs the whole file in memory — but an unbounded
# read lets one giant agent-generated file OOM grading before the rubric runs.
# Cap the raw read: files over this size are recorded as evidence (so the rubric
# still knows they exist) but not extracted. Plain text is already bounded by
# max_content_bytes, so it needs no separate cap.
MAX_SPECIAL_FILE_BYTES = 50 * 1024 * 1024
DEFAULT_EXCLUDE_PATTERNS = {
"__pycache__",
".git",
".venv",
"node_modules",
".mypy_cache",
".pytest_cache",
".ruff_cache",
}
def _is_excluded(relative_path: str, exclude_patterns: set[str]) -> bool:
parts = relative_path.split(os.sep)
return any(part in exclude_patterns for part in parts)
def _within_workdir(path: str, workdir_root: str) -> bool:
"""True if ``path`` resolves to ``workdir_root`` or somewhere beneath it.
``realpath`` collapses ``..`` and follows symlinks, so a symlink inside the
workdir pointing at ``/app`` resolves outside ``workdir_root`` and is
rejected. The core server runs as root, so this is what stops
``prepareGradingContext`` from being turned into a read oracle for the
locked-down ``/app`` tree.
"""
canonical = os.path.realpath(path)
return canonical == workdir_root or canonical.startswith(workdir_root + os.sep)
def _get_extension(file_path: str) -> str:
ext = os.path.splitext(file_path)[1].lower()
return ext.lstrip(".")
def _extract_text(raw: bytes, file_size: int, max_bytes: int) -> tuple[str | None, str, bool]:
try:
truncated = file_size > max_bytes
content = raw[:max_bytes].decode("utf-8", errors="replace")
if truncated:
content += f"\n\n[... truncated at {max_bytes:,} bytes, total size: {file_size:,} bytes]"
return content, "read", truncated
except Exception:
return None, "read", False
def _extract_pdf(raw: bytes, max_bytes: int) -> tuple[str | None, str, bool]:
try:
from pypdf import PdfReader
reader = PdfReader(io.BytesIO(raw))
parts: list[str] = []
total_bytes = 0
truncated = False
for page in reader.pages:
text = page.extract_text() or ""
text_bytes = len(text.encode("utf-8"))
if total_bytes + text_bytes > max_bytes:
remaining = max_bytes - total_bytes
parts.append(text[:remaining])
truncated = True
break
parts.append(text)
total_bytes += text_bytes
content = "\n".join(parts)
if truncated:
content += f"\n\n[... truncated at {max_bytes:,} bytes]"
return content, "pypdf", truncated
except Exception:
return None, "pypdf", False
def _docx_table_lines(table) -> list[str]:
"""Return one tab-separated line per row, recursing into nested tables.
A cell can hold both paragraphs and further tables, so we join the cell's
own paragraph text and then descend into any tables it nests — `cell.text`
alone flattens only the direct paragraphs and silently drops nested tables.
"""
lines: list[str] = []
for row in table.rows:
cells: list[str] = []
nested: list[str] = []
for cell in row.cells:
cells.append("\n".join(p.text for p in cell.paragraphs))
for inner in cell.tables:
nested.extend(_docx_table_lines(inner))
lines.append("\t".join(cells))
lines.extend(nested)
return lines
def _extract_docx(raw: bytes, max_bytes: int) -> tuple[str | None, str, bool]:
try:
from docx import Document
doc = Document(io.BytesIO(raw))
lines = [p.text for p in doc.paragraphs]
# Paragraphs alone miss text inside tables, which agents routinely use
# for structured output (reports, spreadsheets exported as docx, etc.).
for table in doc.tables:
lines.extend(_docx_table_lines(table))
text = "\n".join(lines)
text_bytes = len(text.encode("utf-8"))
truncated = text_bytes > max_bytes
if truncated:
content = text[:max_bytes]
content += f"\n\n[... truncated at {max_bytes:,} bytes]"
else:
content = text
return content, "python-docx", truncated
except Exception:
return None, "python-docx", False
def _extract_xlsx(raw: bytes, max_bytes: int) -> tuple[str | None, str, bool]:
try:
from openpyxl import load_workbook
wb = load_workbook(io.BytesIO(raw), read_only=True, data_only=True)
parts: list[str] = []
total_bytes = 0
truncated = False
# Budget the bytes PER SHEET instead of letting the first sheet drain the
# whole allowance. A multi-tab workbook used to truncate mid-first-sheet
# and never emit the later tabs at all — so a grader couldn't even tell
# those sheets existed. We give each not-yet-visited sheet an equal slice
# of the remaining budget, recomputed after every sheet so unused bytes
# from small/empty sheets roll forward to later ones.
sheet_names = wb.sheetnames
for index, sheet_name in enumerate(sheet_names):
ws = wb[sheet_name]
# Always emit the header so every tab is visible, even one whose rows
# get fully truncated away.
header = f"--- Sheet: {sheet_name} ---"
parts.append(header)
total_bytes += len(header.encode("utf-8")) + 1
remaining_sheets = len(sheet_names) - index
sheet_budget = max(0, (max_bytes - total_bytes) // remaining_sheets)
sheet_used = 0
sheet_truncated = False
for row in ws.iter_rows(values_only=True):
line = "\t".join(str(cell) if cell is not None else "" for cell in row)
# Skip fully-empty rows: openpyxl's read-only dimensions can be
# inflated, so a sheet may report thousands of blank trailing rows
# that would otherwise burn the budget producing only tabs.
if not line.strip():
continue
line_bytes = len(line.encode("utf-8")) + 1 # +1 for newline
if sheet_used + line_bytes > sheet_budget:
sheet_truncated = True
truncated = True
break
parts.append(line)
sheet_used += line_bytes
total_bytes += line_bytes
if sheet_truncated:
parts.append(f"[... '{sheet_name}' truncated at {sheet_budget:,} bytes for this sheet]")
wb.close()
content = "\n".join(parts)
if truncated:
content += f"\n\n[... some sheets truncated; per-sheet budget of ~{max_bytes:,} total bytes reached]"
return content, "openpyxl", truncated
except Exception:
return None, "openpyxl", False
def _pptx_shape_lines(shapes) -> list[str]:
"""Return text lines for a collection of shapes, recursing into groups.
Pulls text frames and table cells, descending into grouped shapes.
"""
lines: list[str] = []
for shape in shapes:
if shape.shape_type == 6: # MSO_SHAPE_TYPE.GROUP
lines.extend(_pptx_shape_lines(shape.shapes))
continue
if shape.has_text_frame:
text = shape.text_frame.text
if text.strip():
lines.append(text)
if shape.has_table:
for row in shape.table.rows:
cells = [cell.text for cell in row.cells]
lines.append("\t".join(cells))
return lines
def _extract_pptx(raw: bytes, max_bytes: int) -> tuple[str | None, str, bool]:
try:
from pptx import Presentation
prs = Presentation(io.BytesIO(raw))
parts: list[str] = []
total_bytes = 0
truncated = False
for index, slide in enumerate(prs.slides, start=1):
# Per-slide marker so the grader sees the slide count, which is often
# itself part of the rubric.
header = f"--- Slide {index} ---"
lines = _pptx_shape_lines(slide.shapes)
if slide.has_notes_slide:
notes = slide.notes_slide.notes_text_frame.text
if notes.strip():
lines.append(f"[Notes] {notes}")
block = "\n".join([header, *lines])
block_bytes = len(block.encode("utf-8")) + 1
if total_bytes + block_bytes > max_bytes:
remaining = max_bytes - total_bytes
if remaining > 0:
parts.append(block[:remaining])
truncated = True
break
parts.append(block)
total_bytes += block_bytes
content = "\n".join(parts)
if truncated:
content += f"\n\n[... truncated at {max_bytes:,} bytes]"
return content, "python-pptx", truncated
except Exception:
return None, "python-pptx", False
def _walk_dir(directory: str) -> list[str]:
results: list[str] = []
for root, _dirs, files in os.walk(directory):
for name in files:
results.append(os.path.join(root, name))
results.sort()
return results
def _format_evidence_as_xml(
evidence: list[dict],
directory: str | None = None,
dropped_paths: list[str] | None = None,
) -> str:
# Emit the container even with no included files when some were dropped by
# the cap, so the grader sees the truncation notice rather than a bare
# final_output. Only an empty listing AND nothing dropped means "no files."
if not evidence and not dropped_paths:
return ""
dir_attr = f' directory="{directory}"' if directory else ""
truncated_attr = ""
if dropped_paths:
truncated_attr = f' truncated_to_max_files="true" dropped_file_count="{len(dropped_paths)}"'
parts: list[str] = [f"<workspace_files{dir_attr}{truncated_attr}>"]
for item in evidence:
attrs = f'path="{item["path"]}" type="{item["extension"]}" size_bytes="{item["size_bytes"]}"'
if item.get("extraction_method"):
attrs += f' extraction_method="{item["extraction_method"]}"'
if item.get("truncated"):
attrs += ' truncated="true"'
if item["content"] is not None:
parts.append(f" <file {attrs}>")
parts.append(item["content"])
parts.append(" </file>")
else:
parts.append(f" <file {attrs} />")
if dropped_paths:
# Surface the names of files that were walked but excluded by the cap.
# An absent file here is genuinely absent; a file listed here exists but
# its contents were not included, so a rubric must not conclude it is
# missing from the workspace.
parts.append(f' <files_omitted_due_to_max_files count="{len(dropped_paths)}">')
for path in dropped_paths:
parts.append(f' <omitted_file path="{path}" />')
parts.append(" </files_omitted_due_to_max_files>")
parts.append("</workspace_files>")
return "\n".join(parts)
async def prepareGradingContext(
final_output: str,
directory: str | None = None,
extensions: list[str] | None = None,
exclude_patterns: list[str] | None = None,
max_files: int = DEFAULT_MAX_FILES,
max_content_bytes: int = DEFAULT_MAX_CONTENT_BYTES,
) -> str:
"""Prepare the text_for_grading string for rubric evaluation.
Collects file evidence from the sandbox, formats as XML,
and prepends to the agent's final_output.
Args:
final_output: The agent's final response text.
directory: Root directory to search. Defaults to the sandbox directory.
extensions: Only include files with these extensions (without dot).
exclude_patterns: Directory/file name patterns to skip.
max_files: Maximum number of files to include as evidence. Eligible files
beyond this cap are omitted but reported (logged and listed in the
XML under <files_omitted_due_to_max_files>) rather than dropped
silently. Defaults to DEFAULT_MAX_FILES.
max_content_bytes: Maximum bytes of content to read per file.
Returns:
The assembled text_for_grading string.
"""
# Walks the sandbox tree and reads/parses files (text/PDF/docx) — all
# blocking — so run the whole body in a worker thread to free the loop.
return await asyncio.to_thread(
_prepare_grading_context_sync,
final_output,
directory,
extensions,
exclude_patterns,
max_files,
max_content_bytes,
)
def _prepare_grading_context_sync(
final_output: str,
directory: str | None = None,
extensions: list[str] | None = None,
exclude_patterns: list[str] | None = None,
max_files: int = DEFAULT_MAX_FILES,
max_content_bytes: int = DEFAULT_MAX_CONTENT_BYTES,
) -> str:
if directory is None:
directory = sandbox.WORKDIR
if not os.path.isdir(directory):
return final_output
# Grading only ever needs the agent's deliverables, which live under the
# sandbox workdir. The server runs as root, so confine the walk to the
# workdir — anything outside it (a caller passing /app, or a symlink that
# escapes) is refused so this can't exfiltrate the locked-down tree.
workdir_root = os.path.realpath(sandbox.WORKDIR)
if not _within_workdir(directory, workdir_root):
logger.warning(
"prepareGradingContext: directory %r is outside the sandbox workdir %r; refusing to read it",
directory,
sandbox.WORKDIR,
)
return final_output
excl = set(exclude_patterns) if exclude_patterns else DEFAULT_EXCLUDE_PATTERNS
ext_filter: set[str] | None = None
if extensions:
ext_filter = {e.lower().lstrip(".") for e in extensions}
unsupported = ext_filter - ALL_SUPPORTED_EXTENSIONS
if unsupported:
raise ValueError(
f"Unsupported file extension(s): {', '.join('.' + e for e in sorted(unsupported))}. "
f"Supported extensions: {', '.join('.' + e for e in sorted(ALL_SUPPORTED_EXTENSIONS))}"
)
all_files = _walk_dir(directory)
evidence: list[dict] = []
# Files that passed every filter (would have been included) but fell outside
# the max_files cap. Tracked so the cap is observable rather than silent: a
# rubric seeing one of these must not conclude the file is absent.
dropped_paths: list[str] = []
for file_path in all_files:
# Skip symlinks (and any other entry) whose real target escapes the
# workdir — they'd otherwise let a root read reach outside the sandbox.
if not _within_workdir(file_path, workdir_root):
continue
relative_path = os.path.relpath(file_path, directory)
if _is_excluded(relative_path, excl):
continue
ext = _get_extension(file_path)
if ext not in ALL_SUPPORTED_EXTENSIONS:
continue
if ext_filter and ext not in ext_filter:
continue
# Cap is checked AFTER filtering so it counts only eligible files, and
# records the overflow instead of silently breaking out of the walk.
if len(evidence) >= max_files:
dropped_paths.append(file_path)
continue
# Read bytes AS THE SANDBOX USER. The server is root, so opening the file
# in-process would both bypass /app's permissions and be a symlink-swap
# TOCTOU oracle; reading as uid 1000 closes both. Office/PDF formats can't
# be truncated (zip/PDF bytes corrupt) so extraction needs the whole file;
# plain text is bounded by max_content_bytes.
read_limit = MAX_SPECIAL_FILE_BYTES if ext in SPECIAL_EXTENSIONS else max_content_bytes
# For special formats, probe the size first (limit=0 reads no bytes) so an
# oversized file is recorded as evidence and skipped — without streaming
# MAX_SPECIAL_FILE_BYTES through the pipe just to discard it, which would
# let a few huge agent files waste GBs and risk OOMing grading.
if ext in SPECIAL_EXTENSIONS:
try:
size_bytes, _h, _r, _s = sandbox.agent_read_window(file_path, offset=0, limit=0, sniff=0)
except sandbox.AgentReadError:
continue
if size_bytes > MAX_SPECIAL_FILE_BYTES:
evidence.append(
{
"path": file_path,
"extension": ext,
"size_bytes": size_bytes,
"content": None,
"truncated": True,
"extraction_method": (
f"skipped: {size_bytes:,} bytes exceeds the {MAX_SPECIAL_FILE_BYTES:,}-byte grading read cap"
),
}
)
continue
try:
size_bytes, _header, raw, _start = sandbox.agent_read_window(file_path, offset=0, limit=read_limit, sniff=0)
except sandbox.AgentReadError:
continue
if ext == "pdf":
content, method, truncated = _extract_pdf(raw, max_content_bytes)
elif ext == "docx":
content, method, truncated = _extract_docx(raw, max_content_bytes)
elif ext == "xlsx":
content, method, truncated = _extract_xlsx(raw, max_content_bytes)
elif ext == "pptx":
content, method, truncated = _extract_pptx(raw, max_content_bytes)
else:
content, method, truncated = _extract_text(raw, size_bytes, max_content_bytes)
item: dict = {
"path": file_path,
"extension": ext,
"size_bytes": size_bytes,
"content": content,
"truncated": truncated,
}
if ext in SPECIAL_EXTENSIONS:
item["extraction_method"] = method
evidence.append(item)
if dropped_paths:
logger.warning(
"prepareGradingContext hit max_files=%d in %s: included %d file(s), "
"omitted %d eligible file(s) from grading evidence: %s",
max_files,
directory,
len(evidence),
len(dropped_paths),
dropped_paths,
)
xml = _format_evidence_as_xml(evidence, directory, dropped_paths)
if xml:
return f"{xml}\n\n{final_output}"
return final_output
docker/packages/core/core/tools/read_file.py
+65
View File
@@ -0,0 +1,65 @@
import asyncio
import os
from typing import Annotated
from pydantic import Field
from core.tools import sandbox
from read_file_safe import (
DEFAULT_READ_LIMIT_BYTES,
HEADER_SNIFF_BYTES,
ReadFileSafeResult,
assemble_read_result,
error_result,
)
async def readFile(
file_path: Annotated[
str,
Field(
description="Path to the file. Use /workdir/ prefix for sandbox files, or an absolute path within the sandbox."
),
],
offset: Annotated[
int,
Field(
description="Byte offset to start reading from. Use 0 for the beginning or a previous call's next_offset to continue.",
ge=0,
),
] = 0,
limit: Annotated[
int | None,
Field(
description=(
f"Maximum number of bytes to read. Defaults to {DEFAULT_READ_LIMIT_BYTES}. "
"Pass null to read from offset to end of file."
),
ge=1,
),
] = DEFAULT_READ_LIMIT_BYTES,
) -> ReadFileSafeResult:
"""Read a file from the sandbox, with optional offset/limit (bytes) for paginating large files."""
workdir = os.path.realpath(sandbox.WORKDIR)
# os.path.join discards the workdir prefix if file_path is absolute (e.g.
# "/etc/hostname" or "/workdir/foo" in production where /workdir is real),
# so absolute paths pass through to the host and FS permissions enforce
# access.
resolved_path = os.path.join(workdir, file_path)
# The core server runs as root, so we must NOT open() agent-supplied
# paths in-process — that would read /app, /opt/venv, etc. Read the bytes as
# the unprivileged sandbox user (filesystem perms gate access, TOCTOU-safe),
# then run the normal decode/binary-sniff/pagination on what came back.
# The read spawns a subprocess and blocks; offload to a worker thread to
# keep the event loop free.
def _read() -> ReadFileSafeResult:
try:
total_bytes, header, raw, start = sandbox.agent_read_window(
resolved_path, offset=offset, limit=limit, sniff=HEADER_SNIFF_BYTES
)
except sandbox.AgentReadError as e:
return error_result(file_path, offset, str(e))
return assemble_read_result(file_path, total_bytes, header, raw, start, limit)
return await asyncio.to_thread(_read)
docker/packages/core/core/tools/read_media.py
+461
View File
@@ -0,0 +1,461 @@
import asyncio
import base64
import io
import mimetypes
import os
import re
from typing import Annotated
from urllib.parse import quote
from fastmcp.tools import ToolResult
from mcp.types import AudioContent, BlobResourceContents, EmbeddedResource, ImageContent, TextContent
from PIL import Image, ImageOps
from pydantic import AnyUrl, Field
from pypdf import PdfReader, PdfWriter
from core.tools import sandbox
MAX_MEDIA_FILE_BYTES = 20 * 1024 * 1024
# PDFs pass through unmodified, and Anthropic caps the total request at 32 MB
# base64 — beyond 10 MiB a single document plus history risks killing the run.
MAX_PDF_FILE_BYTES = 10 * 1024 * 1024
# Larger PDFs/page counts must be read in page ranges (mirrors Claude Code's Read tool).
MAX_PDF_PAGES_PER_READ = 20
# Anthropic rejects images over 2000px on either edge once a request carries
# >20 images, and downscales to ~2576px server-side anyway.
MAX_IMAGE_DIMENSION_PX = 2000
# Keeps the base64 form under the strictest per-image provider cap (5 MB).
MAX_ENCODED_IMAGE_BYTES = int(3.75 * 1024 * 1024)
JPEG_QUALITY_LADDER = (85, 70, 55)
SUPPORTED_IMAGE_MIME_TYPES = {
"image/gif",
"image/jpeg",
"image/png",
"image/webp",
}
PDF_MIME_TYPE = "application/pdf"
# Pinned per extension to Gemini's documented inlineData mimes — Python's
# mimetypes would guess non-canonical types like audio/x-wav, video/quicktime.
AV_MIME_TYPES_BY_EXTENSION = {
".wav": "audio/wav",
".mp3": "audio/mp3",
".aif": "audio/aiff",
".aiff": "audio/aiff",
".aac": "audio/aac",
".ogg": "audio/ogg",
".flac": "audio/flac",
".mp4": "video/mp4",
".mpeg": "video/mpeg",
".mpg": "video/mpeg",
".mov": "video/mov",
".avi": "video/avi",
".flv": "video/x-flv",
".webm": "video/webm",
".wmv": "video/wmv",
".3gp": "video/3gpp",
".3gpp": "video/3gpp",
}
SUPPORTED_AUDIO_MIME_TYPES = {m for m in AV_MIME_TYPES_BY_EXTENSION.values() if m.startswith("audio/")}
SUPPORTED_MEDIA_MIME_TYPES = SUPPORTED_IMAGE_MIME_TYPES | {PDF_MIME_TYPE} | set(AV_MIME_TYPES_BY_EXTENSION.values())
WORKDIR_PREFIX = "/workdir/"
def _error_result(*, file_path: str, mime_type: str | None, stderr: str) -> ToolResult:
label = mime_type or "unknown"
return ToolResult(
content=[
TextContent(
type="text",
text=f"readMedia failed for {file_path} ({label}): {stderr}",
)
]
)
def _media_mime_type(file_path: str) -> str | None:
_root, ext = os.path.splitext(file_path)
av_mime = AV_MIME_TYPES_BY_EXTENSION.get(ext.lower())
if av_mime is not None:
return av_mime
mime_type, _encoding = mimetypes.guess_type(file_path)
if mime_type in SUPPORTED_MEDIA_MIME_TYPES:
return mime_type
return None
def _resolve_read_path(file_path: str, workdir: str) -> str:
if file_path.startswith(WORKDIR_PREFIX):
# lstrip("/") handles "/workdir//foo" — without it, the leading slash on
# the remainder makes os.path.join discard workdir and escape the sandbox.
return os.path.join(workdir, file_path.removeprefix(WORKDIR_PREFIX).lstrip("/"))
if file_path == "/workdir":
return workdir
# os.path.join discards the workdir prefix if file_path is absolute (e.g.
# "/etc/hostname"), so absolute paths pass through to the host and FS
# permissions enforce access.
return os.path.join(workdir, file_path)
def _workdir_relative_path(resolved_path: str, workdir: str) -> str | None:
# realpath resolves "../" segments so paths that escape workdir don't get
# treated as inside it via a raw commonpath check.
canonical = os.path.realpath(resolved_path)
try:
if os.path.commonpath([canonical, workdir]) != workdir:
return None
except ValueError:
return None
rel_path = os.path.relpath(canonical, workdir)
return "" if rel_path == "." else rel_path
def _display_path(file_path: str, resolved_path: str, workdir: str) -> str:
rel_path = _workdir_relative_path(resolved_path, workdir)
if rel_path is not None:
return "/workdir" if rel_path == "" else f"/workdir/{rel_path}"
return file_path
def _resource_uri(display_path: str) -> AnyUrl:
return AnyUrl(f"file://{quote(display_path, safe='/')}")
def _has_alpha(img: Image.Image) -> bool:
if img.mode in ("RGBA", "LA", "PA"):
return True
return img.mode == "P" and "transparency" in img.info
def _encode_image(img: Image.Image, output_format: str, quality: int) -> bytes:
buf = io.BytesIO()
if output_format == "JPEG":
img.save(buf, format="JPEG", quality=quality)
else:
img.save(buf, format="PNG", optimize=True)
return buf.getvalue()
def _parse_page_range(pages: str) -> tuple[int, int] | None:
"""Parse a 1-indexed page range like "3" or "1-10". Returns None if malformed."""
match = re.fullmatch(r"(\d+)(?:-(\d+))?", pages.strip())
if not match:
return None
start = int(match.group(1))
end = int(match.group(2)) if match.group(2) else start
if start < 1 or end < start:
return None
return start, end
def _normalize_image(raw: bytes) -> tuple[bytes, str, list[str]] | str:
"""Make image bytes safe to send to a model provider.
Detects the real format from the bytes (a mime type that mismatches the
content is a fatal provider error), downscales anything over
MAX_IMAGE_DIMENSION_PX on the long edge, and re-encodes oversized payloads
under MAX_ENCODED_IMAGE_BYTES. Returns (bytes, mime_type, notes) on success
or an error message string.
"""
try:
img = Image.open(io.BytesIO(raw))
width, height = img.size
actual_format = img.format
# Force a full decode now: Image.open() is lazy, so a valid header over a
# truncated/corrupt body otherwise slips through the pass-through branch
# below and emits ImageContent the model later rejects. load() raises here
# on a bad body, turning it into a graceful parse error.
img.load()
except Exception:
return "file content is not a parseable image (corrupt file, or extension does not match content?)"
actual_mime = Image.MIME.get(actual_format or "", None)
notes: list[str] = []
# Phone photos carry an EXIF Orientation tag that providers honor only on the
# original bytes; once we re-encode, that metadata is dropped, so bake the
# rotation into the pixels. Values 2-8 mean a transform is needed (1/absent is
# a no-op); when one applies we must re-encode to emit the rotated result
# rather than passing the unrotated raw bytes through.
needs_orientation_fix = img.getexif().get(0x0112, 1) not in (0, 1)
if needs_orientation_fix:
img = ImageOps.exif_transpose(img)
width, height = img.size
notes.append("applied EXIF orientation")
needs_resize = max(width, height) > MAX_IMAGE_DIMENSION_PX
needs_reencode = (
needs_resize
or needs_orientation_fix
or actual_mime not in SUPPORTED_IMAGE_MIME_TYPES
or len(raw) > MAX_ENCODED_IMAGE_BYTES
)
if not needs_reencode:
assert actual_mime is not None
return raw, actual_mime, notes
try:
if getattr(img, "is_animated", False):
img.seek(0)
img.load()
notes.append("animated image flattened to its first frame")
if needs_resize:
img.thumbnail((MAX_IMAGE_DIMENSION_PX, MAX_IMAGE_DIMENSION_PX), Image.Resampling.LANCZOS)
notes.append(f"resized from {width}x{height} to {img.width}x{img.height} to fit model image limits")
# Lossless sources stay PNG when they fit the byte budget (keeps text
# crisp for transcription); photos and anything oversized go to JPEG.
lossless_source = actual_format in ("PNG", "GIF", "BMP", "TIFF")
encoded: bytes | None = None
output_mime = "image/png"
if lossless_source:
png_img = img
encoded = _encode_image(png_img, "PNG", 0)
if len(encoded) > MAX_ENCODED_IMAGE_BYTES:
encoded = None
if encoded is None:
jpeg_img = img
if _has_alpha(jpeg_img):
# JPEG has no alpha channel; composite onto white.
background = Image.new("RGB", jpeg_img.size, (255, 255, 255))
background.paste(jpeg_img.convert("RGBA"), mask=jpeg_img.convert("RGBA").split()[-1])
jpeg_img = background
elif jpeg_img.mode != "RGB":
jpeg_img = jpeg_img.convert("RGB")
for quality in JPEG_QUALITY_LADDER:
encoded = _encode_image(jpeg_img, "JPEG", quality)
if len(encoded) <= MAX_ENCODED_IMAGE_BYTES:
break
while encoded is not None and len(encoded) > MAX_ENCODED_IMAGE_BYTES and min(jpeg_img.size) > 200:
jpeg_img = jpeg_img.resize(
(max(1, int(jpeg_img.width * 0.75)), max(1, int(jpeg_img.height * 0.75))),
Image.Resampling.LANCZOS,
)
encoded = _encode_image(jpeg_img, "JPEG", JPEG_QUALITY_LADDER[-1])
output_mime = "image/jpeg"
if encoded is None or len(encoded) > MAX_ENCODED_IMAGE_BYTES:
return "could not compress image under the per-image size limit"
except Exception as e:
return f"failed to convert image for model consumption: {e}"
if output_mime != actual_mime:
notes.append(f"re-encoded from {actual_mime or 'unknown format'} to {output_mime}")
return encoded, output_mime, notes
async def readMedia(
file_path: Annotated[
str,
Field(
description=(
"Path to an image (gif/jpeg/png/webp), PDF, audio (wav/mp3/aiff/aac/ogg/flac), or video "
"(mp4/mpeg/mov/avi/flv/webm/wmv/3gpp) file. "
"Use /workdir/ prefix for sandbox files, or an absolute path within the sandbox. "
"Returns multimodal MCP content; images are automatically downscaled/re-encoded to fit "
"model limits (max 2000px on the long edge). Audio/video require a model with native "
"audio/video support (e.g. Gemini) — other models see only a text placeholder. "
"Use readFile for text and readPDF for PDF text extraction."
)
),
],
pages: Annotated[
str | None,
Field(
description=(
"PDF page range to read, 1-indexed, e.g. '3' or '1-10' (max 20 pages per read). "
"Required for PDFs over 20 pages or 10 MiB. Ignored for images."
)
),
] = None,
) -> ToolResult:
"""Read an image, PDF, audio, or video file and return it as multimodal MCP content (base64-encoded, capped at 20 MiB; images auto-resized to model limits, large PDFs read in page ranges)."""
# File read, base64 encode, and image/PDF re-encoding all block; run in a worker thread.
return await asyncio.to_thread(_read_media_sync, file_path, pages)
def _read_pdf_sync(*, file_path: str, display_path: str, raw: bytes, file_size: int, pages: str | None) -> ToolResult:
try:
reader = PdfReader(io.BytesIO(raw))
# is_encrypted is true even for owner-restricted PDFs that carry only
# permission flags (no-print etc.) and open silently in any viewer.
# decrypt("") succeeds for those; only a real user password fails it.
if reader.is_encrypted and not reader.decrypt(""):
return _error_result(
file_path=file_path,
mime_type=PDF_MIME_TYPE,
stderr="PDF is password-protected (requires a password to open)",
)
total_pages = len(reader.pages)
except Exception:
return _error_result(file_path=file_path, mime_type=PDF_MIME_TYPE, stderr="file content is not a valid PDF")
if pages is None:
if file_size > MAX_PDF_FILE_BYTES or total_pages > MAX_PDF_PAGES_PER_READ:
return _error_result(
file_path=file_path,
mime_type=PDF_MIME_TYPE,
stderr=(
f"PDF is too large to attach whole ({total_pages} pages, {file_size} bytes; "
f"limits: {MAX_PDF_PAGES_PER_READ} pages, {MAX_PDF_FILE_BYTES} bytes). "
f"Pass pages='1-{min(total_pages, MAX_PDF_PAGES_PER_READ)}' to read a page range, "
"or use readPDF to extract the text."
),
)
blob = raw
page_note = ""
else:
page_range = _parse_page_range(pages)
if page_range is None:
return _error_result(
file_path=file_path,
mime_type=PDF_MIME_TYPE,
stderr=f"Invalid pages value {pages!r}. Use a 1-indexed page or range, e.g. '3' or '1-10'.",
)
start, end = page_range
if start > total_pages:
return _error_result(
file_path=file_path,
mime_type=PDF_MIME_TYPE,
stderr=f"Page {start} is out of range: the PDF has {total_pages} pages.",
)
end = min(end, total_pages)
if end - start + 1 > MAX_PDF_PAGES_PER_READ:
return _error_result(
file_path=file_path,
mime_type=PDF_MIME_TYPE,
stderr=f"Page range {start}-{end} spans more than {MAX_PDF_PAGES_PER_READ} pages. Read it in smaller chunks.",
)
try:
writer = PdfWriter()
for index in range(start - 1, end):
writer.add_page(reader.pages[index])
buf = io.BytesIO()
writer.write(buf)
blob = buf.getvalue()
except Exception as e:
return _error_result(
file_path=file_path, mime_type=PDF_MIME_TYPE, stderr=f"failed to extract pages {start}-{end}: {e}"
)
if len(blob) > MAX_PDF_FILE_BYTES:
return _error_result(
file_path=file_path,
mime_type=PDF_MIME_TYPE,
stderr=(
f"Pages {start}-{end} are still {len(blob)} bytes (limit {MAX_PDF_FILE_BYTES}). "
"Read a smaller range, or use readPDF to extract the text."
),
)
page_note = f" (pages {start}-{end} of {total_pages})"
return ToolResult(
content=[
TextContent(
type="text",
text=f"Read {display_path} as {PDF_MIME_TYPE} ({len(blob)} bytes){page_note}.",
),
EmbeddedResource(
type="resource",
resource=BlobResourceContents(
uri=_resource_uri(display_path),
mimeType=PDF_MIME_TYPE,
blob=base64.b64encode(blob).decode("ascii"),
),
),
]
)
def _read_media_sync(file_path: str, pages: str | None = None) -> ToolResult:
workdir = os.path.realpath(sandbox.WORKDIR)
resolved_path = _resolve_read_path(file_path, workdir)
mime_type = _media_mime_type(file_path)
if mime_type is None:
return _error_result(
file_path=file_path,
mime_type=None,
stderr=f"Unsupported media type. Supported: {sorted(SUPPORTED_MEDIA_MIME_TYPES)}",
)
# Read the bytes through the unprivileged sandbox reader so the open() runs
# as the sandbox user (the core server itself is root). Filesystem
# permissions — not the server's uid — gate what the agent can reach, so an
# absolute path to /app, /opt/venv, /__modal, etc. fails with EACCES exactly
# as it would in the agent's own shell. The reader also rejects non-regular
# files (a FIFO/device under /workdir would otherwise block a worker until
# the timeout) via an O_NONBLOCK + S_ISREG check on the opened fd, and
# limit=MAX+1 bounds the read so an oversized file isn't slurped whole.
try:
file_size, _header, raw, _start = sandbox.agent_read_window(
resolved_path, offset=0, limit=MAX_MEDIA_FILE_BYTES + 1, sniff=0
)
except sandbox.AgentReadError as e:
return _error_result(file_path=file_path, mime_type=mime_type, stderr=str(e))
if file_size > MAX_MEDIA_FILE_BYTES:
return _error_result(
file_path=file_path,
mime_type=mime_type,
stderr=(
f"File too large for media read: {file_size} bytes exceeds {MAX_MEDIA_FILE_BYTES} bytes. "
"Shrink it first (e.g. downscale the image, or for a PDF use readPDF / render individual pages)."
),
)
display_path = _display_path(file_path, resolved_path, workdir)
if mime_type in SUPPORTED_IMAGE_MIME_TYPES:
normalized = _normalize_image(raw)
if isinstance(normalized, str):
return _error_result(file_path=file_path, mime_type=mime_type, stderr=normalized)
data, actual_mime, notes = normalized
suffix = f" ({'; '.join(notes)})" if notes else ""
text = TextContent(
type="text",
text=f"Read {display_path} as {actual_mime} ({len(data)} bytes).{suffix}",
)
return ToolResult(
content=[
text,
ImageContent(type="image", data=base64.b64encode(data).decode("ascii"), mimeType=actual_mime),
]
)
encoded = base64.b64encode(raw).decode("ascii")
text = TextContent(
type="text",
text=f"Read {display_path} as {mime_type} ({file_size} bytes).",
)
if mime_type in SUPPORTED_AUDIO_MIME_TYPES:
return ToolResult(
content=[
text,
AudioContent(type="audio", data=encoded, mimeType=mime_type),
]
)
if mime_type == PDF_MIME_TYPE:
return _read_pdf_sync(file_path=file_path, display_path=display_path, raw=raw, file_size=file_size, pages=pages)
# Video: MCP has no dedicated content type, so embed as a blob
# resource; runners forward supported mime types as native model content.
return ToolResult(
content=[
text,
EmbeddedResource(
type="resource",
resource=BlobResourceContents(
uri=_resource_uri(display_path),
mimeType=mime_type,
blob=encoded,
),
),
]
)
docker/packages/core/core/tools/read_pdf.py
+75
View File
@@ -0,0 +1,75 @@
import asyncio
import io
from typing import Any
from pypdf import PdfReader
from core.tools.sandbox import (
DEFAULT_TIMEOUT_SECONDS,
run_in_sandbox,
)
async def readPDF(file_path: str) -> dict[str, Any]:
"""Extract text from a PDF file inside the sandbox."""
# The subprocess read and pypdf parsing both block; run the whole body in a
# worker thread so the event loop stays free for other tool calls.
return await asyncio.to_thread(_read_pdf_sync, file_path)
def _read_pdf_sync(file_path: str) -> dict[str, Any]:
result = run_in_sandbox(["cat", "--", file_path], DEFAULT_TIMEOUT_SECONDS, text=False)
if result["returncode"] != 0:
stderr = result.get("stderr", b"")
if isinstance(stderr, bytes):
stderr = stderr.decode("utf-8", errors="replace")
return {
"pages": [],
"page_count": 0,
"stderr": stderr or result.get("error", ""),
"returncode": result["returncode"],
"file_path": file_path,
}
try:
reader = PdfReader(io.BytesIO(result["stdout"]))
if reader.is_encrypted:
try:
reader.decrypt("")
except Exception:
return {
"pages": [],
"page_count": 0,
"stderr": "Encrypted PDF: unable to decrypt",
"returncode": 1,
"file_path": file_path,
}
if reader.is_encrypted:
return {
"pages": [],
"page_count": 0,
"stderr": "Encrypted PDF: unable to decrypt",
"returncode": 1,
"file_path": file_path,
}
pages: list[str] = []
for page in reader.pages:
text = page.extract_text() or ""
pages.append(text)
return {
"pages": pages,
"page_count": len(pages),
"stderr": "",
"returncode": 0,
"file_path": file_path,
}
except Exception as e:
return {
"pages": [],
"page_count": 0,
"stderr": str(e),
"returncode": 1,
"file_path": file_path,
}
docker/packages/core/core/tools/sandbox.py
+381
View File
@@ -0,0 +1,381 @@
import os
import subprocess
from collections.abc import Iterator
from typing import Any, cast
DEFAULT_TIMEOUT_SECONDS = 120
MAX_TIMEOUT_SECONDS = 1800
# Agent's working directory. Created in docker/base/Dockerfile and scripts/start.sh,
# owned by the model user. Tests reassign this module attribute to point at a temp
# dir, so the annotation is `str` rather than the inferred `Literal["/workdir"]`.
WORKDIR: str = "/workdir"
# MCP_PROXY_TOKEN is injected directly into core's env by the proxy (see
# mcp_proxy/service.py:start_service) so core's own HTTP server can
# authenticate inbound requests on non-MCP routes. The credential-name filter
# in mcp_proxy does NOT catch this, so we strip it here to avoid leaking it to
# the agent's bash/python.
#
# FAKETIME_SHARED is exported by libfaketime when the proxy runs services under
# the `faketime` wrapper for --current-time. It names a /dev/shm semaphore +
# shared-memory segment that libfaketime created as root (mode 0600), used for
# cross-process monotonic-time coordination. The agent command runs as the
# model uid, so a child it spawns (bash) re-initializes libfaketime,
# can't open that root-owned semaphore, and *hangs* on the very first clock read
# (`date`, time.time(), ...). Stripping it makes libfaketime run per-process —
# the clock is still faked (LD_PRELOAD + FAKETIME are kept); we only lose the
# cross-process "time never goes backward" guarantee, which a sandbox doesn't
# need.
_PROXY_INTERNAL_ENV_KEYS = frozenset({"MCP_PROXY_TOKEN", "FAKETIME_SHARED"})
# Stripped from every agent-spawned subprocess so the server venv (/opt/venv)
# and repo (/app) can't leak into the model's shell.
_AGENT_DEFAULT_PATH = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
_AGENT_PATH_STRIP_PREFIXES = ("/opt/venv", "/app")
_AGENT_ENV_DROP = (
"VIRTUAL_ENV",
"UV_PROJECT_ENVIRONMENT",
"PYTHONPATH",
"PYTHONHOME",
"SETUID",
"SETGID",
)
# Harness-internal vars that point at the locked-down /app tree, name the task,
# carry the run seed, or expose internal service ports. The agent's shell has no
# legitimate use for them, and leaking them advertises the eval layout (paths
# like INPUTDIR=/app/setup_data/…), the seed, and the viewer/proxy ports even
# though those resources are themselves locked down. Matched by PREFIX so a
# newly-added WORLDBENCH_*/BUNDLE* var is dropped automatically rather than
# silently leaking until someone remembers to list it.
_AGENT_ENV_DROP_PREFIXES = ("WORLDBENCH_", "BUNDLE")
# Exact harness vars without a shared prefix.
_AGENT_ENV_DROP_EXACT = ("INPUTDIR", "OUTPUTDIR", "PORT", "VIEWER_PORT")
# Survives the prefix sweep: the fake clock must reach the agent's shell.
_AGENT_ENV_KEEP = frozenset({"WORLDBENCH_CURRENT_TIME"})
def _sanitize_path(path: str) -> str:
"""Drop PATH entries under /opt/venv or /app, preserving the rest in order
(e.g. a derived image's /opt/conda/bin). Falls back to a default if empty."""
def _stripped(entry: str) -> bool:
norm = os.path.normpath(entry)
return any(norm == p or norm.startswith(p + os.sep) for p in _AGENT_PATH_STRIP_PREFIXES)
kept = [entry for entry in path.split(os.pathsep) if entry and not _stripped(entry)]
return os.pathsep.join(kept) if kept else _AGENT_DEFAULT_PATH
def _agent_env() -> dict[str, str]:
"""Inherited env with server-side venv pointers, harness-internal paths, and
proxy-internal credentials stripped — the single chokepoint every
agent command spawn flows through. Copies os.environ so the shell
keeps a real /home/<user>."""
env = os.environ.copy()
for var in (*_AGENT_ENV_DROP, *_AGENT_ENV_DROP_EXACT, *_PROXY_INTERNAL_ENV_KEYS):
env.pop(var, None)
for key in list(env):
if key not in _AGENT_ENV_KEEP and key.startswith(_AGENT_ENV_DROP_PREFIXES):
del env[key]
env["PATH"] = _sanitize_path(env.get("PATH", ""))
return env
def _privilege_drop_kwargs() -> dict[str, Any]:
"""``subprocess`` kwargs that drop an agent command to the unprivileged
sandbox user, or ``{}`` when there's nothing to drop.
core's server process stays **root** in production so it can read the
locked-down ``/app`` tree and keep ``/opt/venv`` closed to uid 1000. Every
agent-facing command (``bash``/file tools) MUST
therefore drop to ``SETUID``/``SETGID`` itself — a command left running as
root is a sandbox escape (it could read ``/app``, the grading data, the
venv, ``/__modal``).
We use subprocess's ``user``/``group``/``extra_groups`` kwargs rather than a
``preexec_fn``: they perform the ``setgroups``/``setgid``/``setuid`` in C
between fork and exec, which is async-signal-safe and avoids the
multithreaded-fork deadlock ``preexec_fn`` is prone to under the async MCP
server.
* **Not root** (local dev, CI, tests): nothing to drop to — return ``{}``.
* **Root, both env vars set**: return the drop kwargs.
* **Root, env vars missing/partial**: raise. Fail closed rather than exec
the agent's command as root.
"""
if os.geteuid() != 0:
return {}
setuid = os.environ.get("SETUID")
setgid = os.environ.get("SETGID")
if setuid is None or setgid is None:
raise RuntimeError(
"core is running as root but SETUID/SETGID are not both set "
f"(SETUID={setuid!r}, SETGID={setgid!r}); refusing to run an agent command as root"
)
# extra_groups=[] clears supplementary groups so the dropped command can't
# inherit any group membership root happened to carry.
return {"user": int(setuid), "group": int(setgid), "extra_groups": []}
# Stdlib-only reader run AS THE SANDBOX USER (uid 1000) via run_in_sandbox. The
# open() happens in the dropped subprocess, so filesystem permissions — not the
# root server — gate access: a path under /app, /opt/venv, /__modal, etc. fails
# with EACCES exactly as in the agent's own shell, and a symlink swap can't
# escalate a root read (the resolve+open are one atomic op as uid 1000, so this
# is TOCTOU-safe). Emits one JSON line so the privileged caller can reuse its
# normal in-memory processing on bytes it never opened itself.
_AGENT_READ_WINDOW_SNIPPET = r"""
import sys, os, stat, base64, json
path, offset, limit, sniff = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]), int(sys.argv[4])
try:
# O_NONBLOCK + S_ISREG on the opened fd: refuse FIFOs/devices/sockets, whose
# open()/read() can block on a writer that never comes (TOCTOU-safe — the
# check is on the fd, not the path).
fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
st = os.fstat(fd)
if not stat.S_ISREG(st.st_mode):
os.close(fd)
raise OSError("not a regular file")
size = st.st_size
with os.fdopen(fd, "rb") as f:
header = f.read(min(sniff, size)) if sniff > 0 else b""
start = min(offset, size) if offset >= 0 else 0
f.seek(start)
raw = f.read() if limit < 0 else f.read(limit)
sys.stdout.write(json.dumps({
"ok": True, "size": size, "start": start,
"header": base64.b64encode(header).decode(),
"raw": base64.b64encode(raw).decode(),
}))
except Exception as e:
sys.stdout.write(json.dumps({"ok": False, "error": str(e)}))
"""
class AgentReadError(Exception):
"""A read performed as the sandbox user failed (missing file, EACCES, …)."""
def agent_read_window(
path: str,
*,
offset: int = 0,
limit: int | None = None,
sniff: int = 0,
) -> tuple[int, bytes, bytes, int]:
"""Read ``path`` AS THE SANDBOX USER and return ``(size, header, raw, start)``.
``raw`` is the ``limit`` bytes (all, if ``None``) at ``offset``; ``header`` is
the first ``sniff`` bytes (for binary detection), or empty when ``sniff`` is
0. Use this anywhere the (now root) server would otherwise ``open()`` an
agent-supplied path in-process. Raises :class:`AgentReadError` on failure.
"""
lim = -1 if limit is None else int(limit)
result = run_in_sandbox(
["python3", "-c", _AGENT_READ_WINDOW_SNIPPET, path, str(offset), str(lim), str(sniff)],
DEFAULT_TIMEOUT_SECONDS,
)
if result["returncode"] != 0:
raise AgentReadError(result.get("stderr") or result.get("error") or "agent read failed")
import json as _json
try:
data = _json.loads(result["stdout"])
except (ValueError, TypeError) as e:
raise AgentReadError(f"agent reader returned invalid output: {e}") from e
if not data.get("ok"):
raise AgentReadError(data.get("error", "agent read failed"))
import base64 as _b64
return data["size"], _b64.b64decode(data["header"]), _b64.b64decode(data["raw"]), data["start"]
# Raw byte-copier run AS THE SANDBOX USER. Unlike _AGENT_READ_WINDOW_SNIPPET it
# does NOT base64/JSON-wrap the payload — it copies the file straight to stdout
# so the parent can stream it without inflating or buffering the whole thing.
_AGENT_STREAM_SNIPPET = r"""
import sys, os, stat
path = sys.argv[1]
try:
# O_NONBLOCK so opening a FIFO/socket with no writer can't block, and reject
# any non-regular file (FIFO, device, socket, dir): reading one could hang
# forever waiting on a writer. The check is on the opened fd (fstat), so it's
# TOCTOU-safe against a path swapped after open.
fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
try:
if not stat.S_ISREG(os.fstat(fd).st_mode):
sys.stderr.write("not a regular file")
sys.exit(1)
while True:
chunk = os.read(fd, 1 << 16)
if not chunk:
break
sys.stdout.buffer.write(chunk)
finally:
os.close(fd)
except Exception as e:
sys.stderr.write(str(e))
sys.exit(1)
"""
def agent_stream_file(
path: str,
*,
chunk_size: int = 1 << 16,
timeout: int = DEFAULT_TIMEOUT_SECONDS,
) -> Iterator[bytes]:
"""Yield ``path``'s bytes in ``chunk_size`` blocks, read AS THE SANDBOX USER,
without ever holding the whole file in the root server.
Same isolation model as :func:`agent_read_window` — a uid-1000 subprocess
does the ``open()`` (filesystem perms gate access, TOCTOU-safe) — but the
bytes are streamed straight through instead of materialized. Use for
whole-file reads where buffering the entire payload would be wasteful (e.g.
viewer downloads of large ``/workdir`` files).
Raises :class:`AgentReadError` if the read fails, whether ``open()`` fails
before any bytes are produced or the subprocess dies mid-stream. An empty
file yields nothing and is not an error. The subprocess is always reaped
(killed if the consumer stops iterating early).
"""
os.makedirs(WORKDIR, exist_ok=True)
# python3 (not sys.executable): the agent's subprocess runs as uid 1000 and
# must use the system interpreter on the sanitized PATH — /opt/venv is locked
# to root. Same partial-path resolution as run_in_sandbox.
command = ["python3", "-c", _AGENT_STREAM_SNIPPET, path]
# cast: the **Any privilege-drop kwargs make the type checker pick Popen's
# text-mode overload, but we pass no text/encoding so this is binary — the
# pipes carry bytes.
proc = cast(
"subprocess.Popen[bytes]",
subprocess.Popen(
command,
cwd=WORKDIR,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env=_agent_env(),
**_privilege_drop_kwargs(),
),
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
while True:
chunk = proc.stdout.read(chunk_size)
if not chunk:
break
yield chunk
returncode = proc.wait(timeout=timeout)
if returncode != 0:
err = proc.stderr.read().decode("utf-8", errors="replace").strip()
raise AgentReadError(err or "agent read failed")
finally:
if proc.poll() is None:
proc.kill()
proc.wait()
_AGENT_LIST_DIR_SNIPPET = r"""
import sys, os, json
path = sys.argv[1]
try:
entries = []
with os.scandir(path) as it:
for e in it:
try:
is_dir = e.is_dir()
size = None if is_dir else e.stat().st_size
except OSError:
continue
entries.append({"name": e.name, "is_dir": is_dir, "size": size})
sys.stdout.write(json.dumps({"ok": True, "entries": entries}))
except Exception as e:
sys.stdout.write(json.dumps({"ok": False, "error": str(e)}))
"""
def agent_list_dir(path: str) -> list[dict[str, Any]]:
"""List ``path`` AS THE SANDBOX USER, returning ``[{name, is_dir, size}, …]``.
Companion to :func:`agent_read_window` for the (root) server's directory
listings — the ``scandir`` runs as uid 1000 so it can't enumerate /app etc.
Raises :class:`AgentReadError` on failure.
"""
result = run_in_sandbox(["python3", "-c", _AGENT_LIST_DIR_SNIPPET, path], DEFAULT_TIMEOUT_SECONDS)
if result["returncode"] != 0:
raise AgentReadError(result.get("stderr") or result.get("error") or "agent list failed")
import json as _json
try:
data = _json.loads(result["stdout"])
except (ValueError, TypeError) as e:
raise AgentReadError(f"agent lister returned invalid output: {e}") from e
if not data.get("ok"):
raise AgentReadError(data.get("error", "agent list failed"))
return data["entries"]
def run_in_sandbox(
command: list[str],
timeout: int,
input: str | bytes | None = None,
*,
text: bool = True,
) -> dict[str, Any]:
"""
Run a command in the sandbox environment.
Args:
command: The command to run as a list of arguments.
timeout: Timeout in seconds.
input: Optional string or bytes to pass to stdin. Must be bytes when
text=False.
text: If True (default), stdout/stderr are decoded as UTF-8 strings.
If False, they are returned as raw bytes — useful for binary file
payloads.
Returns:
Dict with stdout, stderr, and returncode.
"""
workdir = WORKDIR
os.makedirs(workdir, exist_ok=True)
env = _agent_env()
drop_kwargs = _privilege_drop_kwargs()
try:
result = subprocess.run(
command,
cwd=workdir,
capture_output=True,
text=text,
timeout=timeout,
input=input,
env=env,
**drop_kwargs,
)
except subprocess.TimeoutExpired as e:
if text:
stdout = (
(e.stdout or b"").decode("utf-8", errors="replace") if isinstance(e.stdout, bytes) else (e.stdout or "")
)
stderr = (
(e.stderr or b"").decode("utf-8", errors="replace") if isinstance(e.stderr, bytes) else (e.stderr or "")
)
else:
stdout = e.stdout or b""
stderr = e.stderr or b""
return {
"stdout": stdout,
"stderr": stderr,
"returncode": -1,
"error": f"Command timed out after {timeout} seconds",
}
return {
"stdout": result.stdout,
"stderr": result.stderr,
"returncode": result.returncode,
}
docker/packages/core/core/tools/state.py
+29
View File
@@ -0,0 +1,29 @@
"""Standard state tools: export_state and import_state.
Delegates to the shared state codec in ``core.state_codec`` so the MCP tools
and any future file-based loader read/write the same canonical format.
"""
from __future__ import annotations
from typing import Any
from core import state_codec
async def export_state() -> dict[str, list[dict[str, Any]]]:
"""Export the full core state as JSON.
Round-trips with import_state. State is keyed by snake_case entity-class
name (e.g. ``project``, ``employee``, ``slack_message``).
"""
return state_codec.state_to_json()
async def import_state(state: state_codec.SyntaraState) -> dict:
"""Replace the full core state with the provided JSON.
For synthetic-data injection and test setup. Round-trips with export_state.
"""
state_codec.state_from_json(state.model_dump(exclude_unset=True))
return {"ok": True}
docker/packages/core/core/tools/write_file.py
+26
View File
@@ -0,0 +1,26 @@
import asyncio
from typing import Any
from core.tools.sandbox import (
DEFAULT_TIMEOUT_SECONDS,
run_in_sandbox,
)
_WRITE_SCRIPT = 'd=$(dirname -- "$1"); mkdir -p -- "${d:-.}" && tee -- "$1" >/dev/null'
async def writeFile(file_path: str, content: str) -> dict[str, Any]:
"""Write content to a file in an isolated sandbox."""
# Offload the blocking subprocess write to a worker thread.
result = await asyncio.to_thread(
run_in_sandbox,
["bash", "-c", _WRITE_SCRIPT, "--", file_path],
DEFAULT_TIMEOUT_SECONDS,
input=content,
)
return {
"stdout": result.get("stdout", ""),
"stderr": result.get("stderr") or result.get("error", ""),
"returncode": result["returncode"],
"file_path": file_path,
}
docker/packages/core/core/viewer.py
+398
View File
@@ -0,0 +1,398 @@
"""Syntara viewer — read-only filesystem browser for the agent's sandbox workdir.
Serves:
GET /api/tree?path=<rel> — directory listing
GET /api/file?path=<rel> — file contents (text; binary files flagged)
GET / — viewer HTML (single-page app)
All paths are relative to ``sandbox.WORKDIR`` (what readFile/writeFile see).
Requests are rejected if the resolved path escapes that root.
All non-MCP routes require the X-Proxy-Token header.
"""
from __future__ import annotations
from collections.abc import Iterator
from pathlib import Path
import uvicorn
from starlette.applications import Starlette
from starlette.concurrency import run_in_threadpool
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response, StreamingResponse
from starlette.routing import Route
from core._token import get_proxy_token
from core.tools import sandbox
MAX_FILE_BYTES = 1_000_000
BINARY_SNIFF_BYTES = 4096
class ProxyTokenMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
if request.url.path.startswith("/mcp"):
return await call_next(request)
token = get_proxy_token()
if token and request.headers.get("x-proxy-token") != token:
return Response("Forbidden: invalid proxy token", status_code=403)
return await call_next(request)
def _get_root() -> Path:
root = Path(sandbox.WORKDIR).resolve()
root.mkdir(parents=True, exist_ok=True)
return root
def _resolve(rel_path: str) -> Path | None:
root = _get_root()
candidate = (root / rel_path.lstrip("/")).resolve()
try:
candidate.relative_to(root)
except ValueError:
return None
return candidate
def _looks_binary(data: bytes) -> bool:
if b"\x00" in data:
return True
try:
data.decode("utf-8")
return False
except UnicodeDecodeError:
return True
# NOTE: the core server runs as root, so every filesystem access below goes
# through the sandbox.agent_* helpers, which perform the read/scandir as the
# unprivileged sandbox user (uid 1000). _resolve() confines paths to WORKDIR,
# but reading as uid 1000 is what actually prevents this (token-gated, but
# proxied) endpoint from being a root read of /app via a symlink/TOCTOU race.
async def api_tree(request: Request) -> JSONResponse:
rel = request.query_params.get("path", "")
target = _resolve(rel)
if target is None:
return JSONResponse({"error": "path outside sandbox"}, status_code=400)
try:
raw_entries = sandbox.agent_list_dir(str(target))
except sandbox.AgentReadError as e:
return JSONResponse({"error": str(e)}, status_code=404)
entries = [
{"name": e["name"], "type": "dir" if e["is_dir"] else "file", "size": e["size"]}
for e in sorted(raw_entries, key=lambda e: (not e["is_dir"], e["name"].lower()))
]
root = _get_root()
return JSONResponse(
{
"path": str(target.relative_to(root)) if target != root else "",
"entries": entries,
}
)
async def api_file(request: Request) -> JSONResponse:
rel = request.query_params.get("path", "")
target = _resolve(rel)
if target is None:
return JSONResponse({"error": "path outside sandbox"}, status_code=400)
try:
size, header, raw, _start = sandbox.agent_read_window(
str(target), offset=0, limit=MAX_FILE_BYTES, sniff=BINARY_SNIFF_BYTES
)
except sandbox.AgentReadError as e:
return JSONResponse({"error": str(e)}, status_code=404)
if _looks_binary(header):
return JSONResponse({"path": rel, "size": size, "binary": True, "content": None})
return JSONResponse(
{
"path": rel,
"size": size,
"binary": False,
"truncated": size > MAX_FILE_BYTES,
"content": raw.decode("utf-8", errors="replace"),
}
)
async def api_download(request: Request) -> Response:
rel = request.query_params.get("path", "")
target = _resolve(rel)
if target is None:
return JSONResponse({"error": "path outside sandbox"}, status_code=400)
# Stream the file from a uid-1000 reader straight to the client so a large
# /workdir file is never buffered whole in the (root) server. Prime the
# first chunk up front (off the event loop) so a missing/denied file
# surfaces as a 404 rather than a streamed 200 that aborts mid-body.
chunks = sandbox.agent_stream_file(str(target))
try:
first = await run_in_threadpool(lambda: next(chunks, None))
except sandbox.AgentReadError as e:
return JSONResponse({"error": str(e)}, status_code=404)
def body() -> Iterator[bytes]:
if first is not None:
yield first
yield from chunks
return StreamingResponse(
body(),
media_type="application/octet-stream",
headers={"Content-Disposition": f'attachment; filename="{target.name}"'},
)
async def viewer_html(request: Request) -> HTMLResponse:
return HTMLResponse(VIEWER_HTML)
def create_core_viewer_app() -> Starlette:
routes = [
Route("/", viewer_html),
Route("/api/tree", api_tree),
Route("/api/file", api_file),
Route("/api/download", api_download),
]
return Starlette(
routes=routes,
middleware=[Middleware(ProxyTokenMiddleware)],
)
def run_http_server(mcp_app, port: int) -> None:
"""Run combined MCP + viewer HTTP server."""
fastmcp_asgi = mcp_app.http_app(
transport="streamable-http",
path="/mcp",
)
viewer = create_core_viewer_app()
async def combined_app(scope, receive, send):
if scope["type"] == "lifespan":
await fastmcp_asgi(scope, receive, send)
return
path = scope.get("path", "")
if path.startswith("/mcp"):
await fastmcp_asgi(scope, receive, send)
else:
await viewer(scope, receive, send)
uvicorn.run(
combined_app,
host="127.0.0.1",
port=port,
log_level="warning",
)
VIEWER_HTML = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Syntara — Sandbox</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #0f172a; color: #e2e8f0; display: flex; height: 100vh; overflow: hidden; }
.sidebar { width: 320px; min-width: 320px; background: #1e293b; border-right: 1px solid #334155; display: flex; flex-direction: column; overflow: hidden; }
.sidebar-header { padding: 16px; border-bottom: 1px solid #334155; }
.sidebar-header h1 { font-size: 14px; font-weight: 600; color: #f8fafc; letter-spacing: -0.2px; }
.sidebar-header .subtitle { font-size: 11px; color: #64748b; margin-top: 2px; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; word-break: break-all; }
.tree { flex: 1; padding: 8px 4px; overflow-y: auto; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12.5px; }
.tree-node { user-select: none; }
.tree-row { display: flex; align-items: center; gap: 4px; padding: 3px 6px; border-radius: 4px; cursor: pointer; color: #cbd5e1; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.tree-row:hover { background: #0f172a; }
.tree-row.active { background: #1d4ed8; color: #fff; }
.tree-row .chev { width: 12px; color: #64748b; flex-shrink: 0; font-size: 10px; }
.tree-row .icon { width: 14px; flex-shrink: 0; opacity: 0.8; }
.tree-row .name { overflow: hidden; text-overflow: ellipsis; }
.tree-row.active .chev { color: #cbd5e1; }
.tree-children { padding-left: 14px; display: none; }
.tree-children.open { display: block; }
.main { flex: 1; display: flex; flex-direction: column; overflow: hidden; }
.file-header { padding: 12px 20px; border-bottom: 1px solid #334155; background: #1e293b; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px; color: #94a3b8; display: flex; justify-content: space-between; align-items: center; }
.file-header .path { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex: 1; min-width: 0; }
.file-header .meta { color: #64748b; font-size: 11px; flex-shrink: 0; margin-left: 12px; }
.file-header .actions { flex-shrink: 0; margin-left: 12px; }
.download-btn { background: #334155; color: #e2e8f0; border: 1px solid #475569; border-radius: 4px; padding: 4px 10px; font-size: 11px; font-family: inherit; cursor: pointer; text-decoration: none; display: inline-block; }
.download-btn:hover { background: #475569; color: #fff; }
.download-btn[disabled], .download-btn.hidden { display: none; }
.file-body { flex: 1; overflow: auto; padding: 16px 20px; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12.5px; line-height: 1.55; color: #e2e8f0; white-space: pre; tab-size: 4; }
.placeholder { color: #475569; font-size: 13px; padding: 24px; font-family: -apple-system, BlinkMacSystemFont, sans-serif; white-space: normal; }
.warn { color: #fb923c; }
</style>
</head>
<body>
<div class="sidebar">
<div class="sidebar-header">
<h1>Sandbox</h1>
<div class="subtitle" id="root-path">/workdir</div>
</div>
<div class="tree" id="tree"><div class="placeholder">Loading…</div></div>
</div>
<div class="main">
<div class="file-header">
<div class="path" id="file-path">No file selected</div>
<div class="meta" id="file-meta"></div>
<div class="actions">
<a class="download-btn hidden" id="download-btn" href="#" download>⬇ Download</a>
</div>
</div>
<div class="file-body" id="file-body"><div class="placeholder">Select a file from the tree on the left.</div></div>
</div>
<script>
const base = window.location.pathname.replace(/\\/$/, '');
let activeFile = null;
async function fetchJSON(path) {
const r = await fetch(base + path);
if (!r.ok) throw new Error('HTTP ' + r.status);
return r.json();
}
function esc(s) {
if (s === null || s === undefined) return '';
const d = document.createElement('div');
d.textContent = s;
return d.innerHTML;
}
function formatSize(n) {
if (n === null || n === undefined) return '';
if (n < 1024) return n + ' B';
if (n < 1024 * 1024) return (n / 1024).toFixed(1) + ' KB';
return (n / (1024 * 1024)).toFixed(1) + ' MB';
}
function joinPath(dir, name) {
if (!dir) return name;
return dir.replace(/\\/$/, '') + '/' + name;
}
async function renderRoot() {
const treeEl = document.getElementById('tree');
try {
const data = await fetchJSON('/api/tree?path=');
treeEl.innerHTML = '';
const node = buildNode('', data.entries, 0);
treeEl.appendChild(node);
} catch (e) {
treeEl.innerHTML = '<div class="placeholder">Failed to load: ' + esc(e.message) + '</div>';
}
}
function buildNode(parentPath, entries, depth) {
const frag = document.createDocumentFragment();
if (!entries.length) {
const empty = document.createElement('div');
empty.className = 'placeholder';
empty.style.padding = '8px 12px';
empty.textContent = '(empty)';
frag.appendChild(empty);
return frag;
}
for (const entry of entries) {
const fullPath = joinPath(parentPath, entry.name);
const node = document.createElement('div');
node.className = 'tree-node';
const row = document.createElement('div');
row.className = 'tree-row';
row.dataset.path = fullPath;
row.dataset.type = entry.type;
const chev = document.createElement('span');
chev.className = 'chev';
chev.textContent = entry.type === 'dir' ? '▸' : '';
const icon = document.createElement('span');
icon.className = 'icon';
icon.textContent = entry.type === 'dir' ? '📁' : '📄';
const name = document.createElement('span');
name.className = 'name';
name.textContent = entry.name;
row.appendChild(chev);
row.appendChild(icon);
row.appendChild(name);
node.appendChild(row);
if (entry.type === 'dir') {
const children = document.createElement('div');
children.className = 'tree-children';
node.appendChild(children);
row.addEventListener('click', async () => {
const isOpen = children.classList.contains('open');
if (isOpen) {
children.classList.remove('open');
chev.textContent = '▸';
} else {
if (!children.dataset.loaded) {
children.innerHTML = '<div class="placeholder" style="padding:4px 12px;">…</div>';
try {
const data = await fetchJSON('/api/tree?path=' + encodeURIComponent(fullPath));
children.innerHTML = '';
children.appendChild(buildNode(fullPath, data.entries, depth + 1));
children.dataset.loaded = '1';
} catch (e) {
children.innerHTML = '<div class="placeholder" style="padding:4px 12px;">Error</div>';
}
}
children.classList.add('open');
chev.textContent = '▾';
}
});
} else {
row.addEventListener('click', () => openFile(fullPath, row));
}
frag.appendChild(node);
}
return frag;
}
async function openFile(path, rowEl) {
document.querySelectorAll('.tree-row.active').forEach(el => el.classList.remove('active'));
if (rowEl) rowEl.classList.add('active');
activeFile = path;
const pathEl = document.getElementById('file-path');
const metaEl = document.getElementById('file-meta');
const bodyEl = document.getElementById('file-body');
const dlBtn = document.getElementById('download-btn');
pathEl.textContent = path;
metaEl.textContent = '';
bodyEl.innerHTML = '<div class="placeholder">Loading…</div>';
dlBtn.href = base + '/api/download?path=' + encodeURIComponent(path);
dlBtn.classList.remove('hidden');
try {
const data = await fetchJSON('/api/file?path=' + encodeURIComponent(path));
metaEl.textContent = formatSize(data.size) + (data.truncated ? ' (truncated — use Download for full file)' : '');
if (data.binary) {
bodyEl.innerHTML = '<div class="placeholder warn">Binary file — not rendered. Use Download to fetch the raw bytes.</div>';
} else {
bodyEl.textContent = data.content;
}
} catch (e) {
bodyEl.innerHTML = '<div class="placeholder">Failed to load: ' + esc(e.message) + '</div>';
}
}
renderRoot();
</script>
</body>
</html>"""
docker/packages/core/mcp-tools.generated.json
+297
View File
@@ -0,0 +1,297 @@
{
"schema_version": "v1",
"tools": [
{
"name": "bash",
"description": "Execute a bash command in an isolated directory.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"command": {
"description": "The bash command to execute",
"type": "string"
},
"timeout_seconds": {
"anyOf": [
{
"maximum": 1800,
"minimum": 1,
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"description": "Timeout in seconds (default 120, max 1800)"
}
},
"required": [
"command"
],
"type": "object"
}
},
{
"name": "echo",
"description": "Echoes a message",
"inputSchema": {
"additionalProperties": false,
"properties": {
"message": {
"description": "The message to echo",
"type": "string"
}
},
"required": [
"message"
],
"type": "object"
}
},
{
"name": "export_state",
"description": "Export the full core state as JSON.\n\nRound-trips with import_state. State is keyed by snake_case entity-class\nname (e.g. ``project``, ``employee``, ``slack_message``).",
"inputSchema": {
"additionalProperties": false,
"properties": {},
"type": "object"
}
},
{
"name": "import_state",
"description": "Replace the full core state with the provided JSON.\n\nFor synthetic-data injection and test setup. Round-trips with export_state.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"state": {
"additionalProperties": true,
"description": "Empty state \u2014 core has no DB to snapshot.",
"properties": {},
"type": "object"
}
},
"required": [
"state"
],
"type": "object"
}
},
{
"name": "listFiles",
"description": "List files and subdirectories in a directory inside the sandbox.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"directory": {
"default": ".",
"description": "Directory to list, relative to the sandbox root. Use /workdir/ prefix or an absolute path within the sandbox to be explicit. Defaults to the sandbox root.",
"type": "string"
}
},
"type": "object"
}
},
{
"name": "prepareGradingContext",
"description": "Prepare the text_for_grading string for rubric evaluation.\n\nCollects file evidence from the sandbox, formats as XML,\nand prepends to the agent's final_output.\n\nArgs:\n final_output: The agent's final response text.\n directory: Root directory to search. Defaults to the sandbox directory.\n extensions: Only include files with these extensions (without dot).\n exclude_patterns: Directory/file name patterns to skip.\n max_files: Maximum number of files to include as evidence. Eligible files\n beyond this cap are omitted but reported (logged and listed in the\n XML under <files_omitted_due_to_max_files>) rather than dropped\n silently. Defaults to DEFAULT_MAX_FILES.\n max_content_bytes: Maximum bytes of content to read per file.\n\nReturns:\n The assembled text_for_grading string.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"final_output": {
"type": "string"
},
"directory": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null
},
"extensions": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null
},
"exclude_patterns": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null
},
"max_files": {
"default": 100,
"type": "integer"
},
"max_content_bytes": {
"default": 250000,
"type": "integer"
}
},
"required": [
"final_output"
],
"type": "object"
}
},
{
"name": "readFile",
"description": "Read a file from the sandbox, with optional offset/limit (bytes) for paginating large files.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"file_path": {
"description": "Path to the file. Use /workdir/ prefix for sandbox files, or an absolute path within the sandbox.",
"type": "string"
},
"offset": {
"default": 0,
"description": "Byte offset to start reading from. Use 0 for the beginning or a previous call's next_offset to continue.",
"minimum": 0,
"type": "integer"
},
"limit": {
"anyOf": [
{
"minimum": 1,
"type": "integer"
},
{
"type": "null"
}
],
"default": 200000,
"description": "Maximum number of bytes to read. Defaults to 200000. Pass null to read from offset to end of file."
}
},
"required": [
"file_path"
],
"type": "object"
}
},
{
"name": "readMedia",
"description": "Read an image, PDF, audio, or video file and return it as multimodal MCP content (base64-encoded, capped at 20 MiB; images auto-resized to model limits, large PDFs read in page ranges).",
"inputSchema": {
"additionalProperties": false,
"properties": {
"file_path": {
"description": "Path to an image (gif/jpeg/png/webp), PDF, audio (wav/mp3/aiff/aac/ogg/flac), or video (mp4/mpeg/mov/avi/flv/webm/wmv/3gpp) file. Use /workdir/ prefix for sandbox files, or an absolute path within the sandbox. Returns multimodal MCP content; images are automatically downscaled/re-encoded to fit model limits (max 2000px on the long edge). Audio/video require a model with native audio/video support (e.g. Gemini) \u2014 other models see only a text placeholder. Use readFile for text and readPDF for PDF text extraction.",
"type": "string"
},
"pages": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "PDF page range to read, 1-indexed, e.g. '3' or '1-10' (max 20 pages per read). Required for PDFs over 20 pages or 10 MiB. Ignored for images."
}
},
"required": [
"file_path"
],
"type": "object"
}
},
{
"name": "readPDF",
"description": "Extract text from a PDF file inside the sandbox.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"file_path": {
"type": "string"
}
},
"required": [
"file_path"
],
"type": "object"
}
},
{
"name": "writeFile",
"description": "Write content to a file in an isolated sandbox.",
"inputSchema": {
"additionalProperties": false,
"properties": {
"file_path": {
"type": "string"
},
"content": {
"type": "string"
}
},
"required": [
"file_path",
"content"
],
"type": "object"
}
}
],
"toolsets": {
"read": [
"echo",
"listFiles",
"readFile",
"readPDF"
],
"read_multimodal": [
"echo",
"listFiles",
"readFile",
"readMedia",
"readPDF"
],
"write": [
"bash",
"writeFile"
],
"debug": [
"bash",
"echo"
],
"ds_all": [
"bash",
"listFiles",
"readFile",
"readPDF",
"writeFile"
],
"state": [
"export_state",
"import_state"
],
"grading": [
"prepareGradingContext"
]
}
}
docker/packages/core/mcp.json
+28
View File
@@ -0,0 +1,28 @@
{
"run": {
"command": "python",
"args": ["-m", "core.server"],
"env": { "SETUID": "1000", "SETGID": "1000", "HOME": "/home/model", "USER": "model", "LOGNAME": "model" }
},
"setup": {
"command": "python",
"args": ["-m", "core.setup"],
"env": { "SETUID": "1000", "SETGID": "1000", "HOME": "/home/model", "USER": "model", "LOGNAME": "model" }
},
"namespaced": false,
"toolsets": {
"read": ["echo", "listFiles", "readFile", "readPDF"],
"read_multimodal": ["echo", "listFiles", "readFile", "readMedia", "readPDF"],
"write": ["bash", "writeFile"],
"debug": ["bash", "echo"],
"ds_all": [
"bash",
"listFiles",
"readFile",
"readPDF",
"writeFile"
],
"state": ["export_state", "import_state"],
"grading": ["prepareGradingContext"]
}
}
docker/packages/core/pyproject.toml
+38
View File
@@ -0,0 +1,38 @@
[build-system]
build-backend = "uv_build"
requires = [ "uv-build>=0.11,<0.12" ]
[project]
name = "core"
version = "0.0.1"
description = "MCP Server"
requires-python = ">=3.13"
classifiers = [
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
dependencies = [
"fastmcp>=3.2,<4",
"json5>=0.9",
"mcp[cli]>=1.19",
"openpyxl>=3.1",
"pillow>=10",
"pypdf>=4.2",
"python-dateutil>=2.8",
"python-docx>=1.1",
"python-pptx>=0.6",
"read-file-safe",
]
optional-dependencies.dev = [ "pytest>=8", "pytest-snapshot>=0.9" ]
scripts.core-mcp = "core.server:main"
[tool.uv]
environments = [
"sys_platform == 'darwin' and platform_machine == 'arm64'",
"sys_platform == 'linux' and platform_machine == 'x86_64'",
]
build-backend.module-root = ""
[tool.pytest]
ini_options.pythonpath = [ "." ]
docker/packages/core/tests/test_agent_env.py
+305
View File
@@ -0,0 +1,305 @@
"""Pin `sandbox._agent_env`: the single chokepoint that hides /opt/venv from
agent-spawned subprocesses (bash)."""
from __future__ import annotations
import os
from unittest import mock
import pytest
from core.tools import sandbox
def _polluted_environ() -> dict[str, str]:
return {
"HOME": "/home/model",
"USER": "model",
"LANG": "C.UTF-8",
"PATH": "/opt/venv/bin:/app/scripts:/usr/local/bin:/usr/bin:/bin",
"VIRTUAL_ENV": "/opt/venv",
"UV_PROJECT_ENVIRONMENT": "/opt/venv",
"PYTHONPATH": "/app:/opt/venv/lib/python3.13/site-packages",
"PYTHONHOME": "/opt/venv",
"SETUID": "1000",
"SETGID": "1000",
}
def test_strips_opt_venv_from_path():
with mock.patch.dict(os.environ, _polluted_environ(), clear=True):
env = sandbox._agent_env()
assert "/opt/venv" not in env["PATH"], env["PATH"]
def test_strips_app_from_path():
with mock.patch.dict(os.environ, _polluted_environ(), clear=True):
env = sandbox._agent_env()
assert "/app" not in env["PATH"], env["PATH"]
def test_drops_venv_pointers():
with mock.patch.dict(os.environ, _polluted_environ(), clear=True):
env = sandbox._agent_env()
for var in ("VIRTUAL_ENV", "UV_PROJECT_ENVIRONMENT", "PYTHONPATH", "PYTHONHOME"):
assert var not in env, f"{var!r} leaked: {env.get(var)!r}"
def test_drops_privilege_control_vars():
with mock.patch.dict(os.environ, _polluted_environ(), clear=True):
env = sandbox._agent_env()
assert "SETUID" not in env
assert "SETGID" not in env
def test_preserves_user_facing_vars():
with mock.patch.dict(os.environ, _polluted_environ(), clear=True):
env = sandbox._agent_env()
assert env["HOME"] == "/home/model"
assert env["USER"] == "model"
assert env["LANG"] == "C.UTF-8"
def test_path_is_a_concrete_default_not_absent():
with mock.patch.dict(os.environ, _polluted_environ(), clear=True):
env = sandbox._agent_env()
assert env.get("PATH"), "agent PATH unexpectedly empty"
assert "/usr/bin" in env["PATH"]
assert "/usr/local/bin" in env["PATH"]
# Regression guard for the P1: derived images (docker/stem-*) append dirs like
# /opt/conda/bin; sanitising must strip only /opt/venv and /app, never the rest.
def _derived_image_environ() -> dict[str, str]:
env = _polluted_environ()
env["PATH"] = "/opt/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/conda/bin"
return env
def test_preserves_derived_image_path_additions():
with mock.patch.dict(os.environ, _derived_image_environ(), clear=True):
env = sandbox._agent_env()
entries = env["PATH"].split(os.pathsep)
assert "/opt/conda/bin" in entries, env["PATH"]
assert "/opt/venv/bin" not in entries, env["PATH"]
def test_preserves_path_entry_order():
with mock.patch.dict(os.environ, _derived_image_environ(), clear=True):
env = sandbox._agent_env()
entries = env["PATH"].split(os.pathsep)
assert entries == ["/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", "/opt/conda/bin"]
def test_strips_only_venv_and_app_prefixes_not_lookalikes():
polluted = _polluted_environ()
polluted["PATH"] = "/opt/venv/bin:/opt/venvtools/bin:/application/bin:/app/scripts:/usr/bin"
with mock.patch.dict(os.environ, polluted, clear=True):
env = sandbox._agent_env()
entries = env["PATH"].split(os.pathsep)
assert entries == ["/opt/venvtools/bin", "/application/bin", "/usr/bin"], env["PATH"]
def test_falls_back_to_default_when_only_server_entries():
polluted = _polluted_environ()
polluted["PATH"] = "/opt/venv/bin:/app/scripts"
with mock.patch.dict(os.environ, polluted, clear=True):
env = sandbox._agent_env()
assert env["PATH"] == sandbox._AGENT_DEFAULT_PATH
def test_falls_back_to_default_when_path_unset():
polluted = _polluted_environ()
del polluted["PATH"]
with mock.patch.dict(os.environ, polluted, clear=True):
env = sandbox._agent_env()
assert env["PATH"] == sandbox._AGENT_DEFAULT_PATH
def test_parent_env_unchanged():
# Sanitiser must not mutate os.environ — core itself relies on those
# vars for its own subsequent imports.
polluted = _polluted_environ()
with mock.patch.dict(os.environ, polluted, clear=True):
sandbox._agent_env()
assert os.environ["PATH"] == polluted["PATH"]
assert os.environ["VIRTUAL_ENV"] == polluted["VIRTUAL_ENV"]
assert os.environ["UV_PROJECT_ENVIRONMENT"] == polluted["UV_PROJECT_ENVIRONMENT"]
assert os.environ["SETUID"] == polluted["SETUID"]
def test_drops_harness_internal_vars():
polluted = _polluted_environ()
polluted.update(
{
"WORLDBENCH_ROOT": "/app",
"WORLDBENCH_SEED": "deadbeef",
"WORLDBENCH_TOOL_SETS": "core_debug",
# A harness var nobody listed explicitly — the prefix sweep must catch it.
"WORLDBENCH_SOME_FUTURE_VAR": "/app/whatever",
"INPUTDIR": "/app/setup_data/entities/core",
"OUTPUTDIR": "/app/output_data/core",
"BUNDLE_OUTPUT_DIR": "/app/output_data/services",
"BUNDLE_INPUT_DIR": "/app/bundle/services/core",
"BUNDLEDIR": "/app/bundle",
"PORT": "41984",
"VIEWER_PORT": "8123",
}
)
with mock.patch.dict(os.environ, polluted, clear=True):
env = sandbox._agent_env()
for var in (
"WORLDBENCH_ROOT",
"WORLDBENCH_SEED",
"WORLDBENCH_TOOL_SETS",
"WORLDBENCH_SOME_FUTURE_VAR",
"INPUTDIR",
"OUTPUTDIR",
"BUNDLE_OUTPUT_DIR",
"BUNDLE_INPUT_DIR",
"BUNDLEDIR",
"PORT",
"VIEWER_PORT",
):
assert var not in env, f"{var!r} leaked to the agent: {env.get(var)!r}"
# /app must not survive anywhere in the agent env (paths or otherwise).
assert not any("/app" in v for v in env.values()), env
def test_preserves_clock_var():
# The fake clock must reach the agent's shell; it is NOT a harness leak.
polluted = _polluted_environ()
polluted["WORLDBENCH_CURRENT_TIME"] = "2026-01-02T03:04:05Z"
with mock.patch.dict(os.environ, polluted, clear=True):
env = sandbox._agent_env()
assert env["WORLDBENCH_CURRENT_TIME"] == "2026-01-02T03:04:05Z"
def test_privilege_drop_kwargs_noop_when_not_root():
# Tests/CI run unprivileged: nothing to drop to, so subprocess gets no
# user/group kwargs and runs as the invoking user.
with mock.patch("core.tools.sandbox.os.geteuid", return_value=1000):
assert sandbox._privilege_drop_kwargs() == {}
def test_privilege_drop_kwargs_drops_when_root():
polluted = _polluted_environ() # has SETUID/SETGID = 1000
with (
mock.patch("core.tools.sandbox.os.geteuid", return_value=0),
mock.patch.dict(os.environ, polluted, clear=True),
):
kwargs = sandbox._privilege_drop_kwargs()
assert kwargs == {"user": 1000, "group": 1000, "extra_groups": []}
def test_privilege_drop_kwargs_fails_closed_when_root_without_ids():
# Root but no SETUID/SETGID: refuse rather than run the agent's command as
# root (a sandbox escape).
env = {k: v for k, v in _polluted_environ().items() if k not in ("SETUID", "SETGID")}
with (
mock.patch("core.tools.sandbox.os.geteuid", return_value=0),
mock.patch.dict(os.environ, env, clear=True),
pytest.raises(RuntimeError, match="SETUID/SETGID are not both set"),
):
sandbox._privilege_drop_kwargs()
# ---------------------------------------------------------------------------
# agent_stream_file — the unprivileged whole-file reader the viewer streams from
# ---------------------------------------------------------------------------
def test_agent_stream_file_streams_whole_file(tmp_path, monkeypatch):
monkeypatch.setattr(sandbox, "WORKDIR", str(tmp_path))
target = tmp_path / "data.bin"
payload = bytes(range(256)) * 1000 # 256 KB, spans many read() calls
target.write_bytes(payload)
chunks = list(sandbox.agent_stream_file(str(target), chunk_size=4096))
assert b"".join(chunks) == payload
# Actually streamed in pieces rather than buffered as one blob.
assert len(chunks) > 1
def test_agent_stream_file_empty_file_yields_nothing(tmp_path, monkeypatch):
monkeypatch.setattr(sandbox, "WORKDIR", str(tmp_path))
target = tmp_path / "empty.bin"
target.write_bytes(b"")
assert list(sandbox.agent_stream_file(str(target))) == []
def test_agent_stream_file_missing_file_raises(tmp_path, monkeypatch):
monkeypatch.setattr(sandbox, "WORKDIR", str(tmp_path))
with pytest.raises(sandbox.AgentReadError):
list(sandbox.agent_stream_file(str(tmp_path / "nope.bin")))
def test_agent_stream_file_rejects_fifo(tmp_path, monkeypatch):
# A named pipe with no writer would block a reader forever; the helper must
# refuse non-regular files instead of hanging. (No writer is opened here, so
# this test would deadlock if the guard regressed.)
monkeypatch.setattr(sandbox, "WORKDIR", str(tmp_path))
fifo = tmp_path / "pipe"
os.mkfifo(fifo)
with pytest.raises(sandbox.AgentReadError):
list(sandbox.agent_stream_file(str(fifo)))
def test_agent_read_window_rejects_fifo(tmp_path, monkeypatch):
# Same guard on the windowed reader (backs readFile / viewer / grading).
monkeypatch.setattr(sandbox, "WORKDIR", str(tmp_path))
fifo = tmp_path / "pipe"
os.mkfifo(fifo)
with pytest.raises(sandbox.AgentReadError):
sandbox.agent_read_window(str(fifo))
def test_agent_read_window_roundtrips(tmp_path):
sandbox.WORKDIR = str(tmp_path)
p = tmp_path / "data.bin"
p.write_bytes(b"0123456789abcdef")
size, header, raw, start = sandbox.agent_read_window(str(p), offset=4, limit=5, sniff=3)
assert size == 16
assert header == b"012"
assert raw == b"45678"
assert start == 4
def test_agent_read_window_raises_on_missing(tmp_path):
sandbox.WORKDIR = str(tmp_path)
with pytest.raises(sandbox.AgentReadError):
sandbox.agent_read_window(str(tmp_path / "nope.bin"))
def test_agent_list_dir(tmp_path):
sandbox.WORKDIR = str(tmp_path)
(tmp_path / "a.txt").write_text("hi")
(tmp_path / "sub").mkdir()
entries = {e["name"]: e for e in sandbox.agent_list_dir(str(tmp_path))}
assert entries["a.txt"]["is_dir"] is False
assert entries["a.txt"]["size"] == 2
assert entries["sub"]["is_dir"] is True
def test_run_in_sandbox_passes_sanitised_env(tmp_path):
# End-to-end: catches the failure mode where _agent_env exists but
# run_in_sandbox forgot to call it.
sandbox.WORKDIR = str(tmp_path)
polluted = _polluted_environ()
with mock.patch.dict(os.environ, polluted, clear=True):
result = sandbox.run_in_sandbox(
[
"sh",
"-c",
"echo PATH=$PATH; echo VENV=${VIRTUAL_ENV:-}; echo UV=${UV_PROJECT_ENVIRONMENT:-}; echo SU=${SETUID:-}",
],
timeout=10,
)
assert result["returncode"] == 0, result
out = result["stdout"]
assert "/opt/venv" not in out, out
assert "/app" not in out, out
assert "VENV=\n" in out or out.rstrip().endswith("VENV="), out
assert "UV=\n" in out or ("UV=" in out and "/opt/venv" not in out), out
assert "SU=\n" in out or out.rstrip().endswith("SU="), out
docker/packages/core/tests/test_hidden_tool_filter.py
+52
View File
@@ -0,0 +1,52 @@
"""Verify HiddenToolFilter: hidden tools excluded from list_tools but still callable."""
import asyncio
from fastmcp import Client, FastMCP
from fastmcp.server.middleware import Middleware
from fastmcp.tools.function_tool import FunctionTool
class HiddenToolFilter(Middleware):
async def on_list_tools(self, context, call_next):
tool_list = await call_next(context)
return [t for t in tool_list if "hidden" not in (t.tags or set())]
def _build_app() -> FastMCP:
app = FastMCP("test")
def visible_tool() -> str:
"""A normal tool."""
return "visible"
def hidden_tool() -> str:
"""A hidden tool."""
return "hidden"
app.add_tool(FunctionTool.from_function(fn=visible_tool, name="visible_tool"))
app.add_tool(FunctionTool.from_function(fn=hidden_tool, name="hidden_tool", tags={"hidden"}))
app.add_middleware(HiddenToolFilter())
return app
async def main():
app = _build_app()
async with Client(app) as client:
# 1. Hidden tool should NOT appear in list_tools
tools = await client.list_tools()
tool_names = [t.name for t in tools]
assert "visible_tool" in tool_names, f"visible_tool missing from {tool_names}"
assert "hidden_tool" not in tool_names, f"hidden_tool should be hidden but found in {tool_names}"
print("PASS: hidden tool excluded from list_tools")
# 2. Hidden tool should still be callable
result = await client.call_tool("hidden_tool", {})
assert result.content[0].text == "hidden", f"unexpected result: {result}"
print("PASS: hidden tool still callable via call_tool")
print("\nAll checks passed.")
if __name__ == "__main__":
asyncio.run(main())
docker/packages/core/tests/test_prepare_grading_context.py
+524
View File
@@ -0,0 +1,524 @@
import os
import pathlib
import shutil
import tempfile
import unittest
from unittest import mock
import pytest
from core.tools import sandbox
from core.tools.prepare_grading_context import (
_extract_docx,
_extract_pptx,
_extract_text,
_extract_xlsx,
_format_evidence_as_xml,
_get_extension,
_is_excluded,
_walk_dir,
prepareGradingContext,
)
class TestHelpers(unittest.TestCase):
def test_get_extension(self):
assert _get_extension("/foo/bar.txt") == "txt"
assert _get_extension("/foo/bar.CSV") == "csv"
assert _get_extension("/foo/bar") == ""
# Dotfiles like .hidden have no extension per os.path.splitext
assert _get_extension("/foo/.hidden") == ""
def test_is_excluded(self):
excl = {"__pycache__", ".git"}
assert _is_excluded("__pycache__/foo.pyc", excl)
assert _is_excluded("src/__pycache__/foo.pyc", excl)
assert _is_excluded(".git/config", excl)
assert not _is_excluded("src/main.py", excl)
def test_walk_dir_sorted(self):
with tempfile.TemporaryDirectory() as d:
os.makedirs(os.path.join(d, "b"))
open(os.path.join(d, "b", "z.txt"), "w").close()
open(os.path.join(d, "b", "a.txt"), "w").close()
open(os.path.join(d, "c.txt"), "w").close()
results = _walk_dir(d)
names = [os.path.basename(r) for r in results]
assert names == ["a.txt", "z.txt", "c.txt"] or results == sorted(results)
class TestExtractText(unittest.TestCase):
# _extract_text now takes (raw_bytes, file_size, max_bytes) — the actual file
# read happens earlier, as the sandbox user, in prepareGradingContext.
def test_extract_text_normal(self):
content, method, truncated = _extract_text(b"hello world", 11, 50_000)
assert content == "hello world"
assert method == "read"
assert not truncated
def test_extract_text_truncated(self):
raw = b"a" * 1000
content, _method, truncated = _extract_text(raw, len(raw), 100)
assert truncated
assert content is not None
assert "[... truncated at 100 bytes" in content
assert "total size: 1,000 bytes]" in content
def test_extract_text_decodes_non_utf8_lossily(self):
# Invalid UTF-8 bytes are replaced, never raise.
content, _method, _truncated = _extract_text(b"\xff\xfe bad bytes", 12, 50_000)
assert content is not None
class TestFormatEvidenceAsXml(unittest.TestCase):
def test_empty_evidence(self):
assert _format_evidence_as_xml([]) == ""
def test_single_file_with_content(self):
evidence = [
{
"path": "/tmp/test.txt",
"extension": "txt",
"size_bytes": 11,
"content": "hello world",
"truncated": False,
}
]
xml = _format_evidence_as_xml(evidence, "/tmp")
assert '<workspace_files directory="/tmp">' in xml
assert 'path="/tmp/test.txt"' in xml
assert 'type="txt"' in xml
assert 'size_bytes="11"' in xml
assert "hello world" in xml
assert "</workspace_files>" in xml
def test_null_content_self_closing(self):
evidence = [
{
"path": "/tmp/test.bin",
"extension": "bin",
"size_bytes": 100,
"content": None,
"truncated": False,
}
]
xml = _format_evidence_as_xml(evidence)
assert "/>" in xml
assert "</file>" not in xml
def test_special_extension_extraction_method(self):
evidence = [
{
"path": "/tmp/test.pdf",
"extension": "pdf",
"size_bytes": 5000,
"content": "pdf text",
"extraction_method": "pypdf",
"truncated": False,
}
]
xml = _format_evidence_as_xml(evidence)
assert 'extraction_method="pypdf"' in xml
def test_truncated_attribute(self):
evidence = [
{
"path": "/tmp/big.txt",
"extension": "txt",
"size_bytes": 100000,
"content": "partial...",
"truncated": True,
}
]
xml = _format_evidence_as_xml(evidence)
assert 'truncated="true"' in xml
class TestPrepareGradingContext(unittest.IsolatedAsyncioTestCase):
def setUp(self):
self.temp_dir = tempfile.mkdtemp()
self.original_workdir = sandbox.WORKDIR
# prepareGradingContext now confines reads to the sandbox workdir, so
# point WORKDIR at the dir under test (mirrors production, where grading
# collects the agent's deliverables from WORKDIR).
sandbox.WORKDIR = self.temp_dir
def tearDown(self):
shutil.rmtree(self.temp_dir, ignore_errors=True)
sandbox.WORKDIR = self.original_workdir
def _write(self, rel_path: str, content: str = "test") -> str:
full = os.path.join(self.temp_dir, rel_path)
os.makedirs(os.path.dirname(full), exist_ok=True)
with open(full, "w") as f:
f.write(content)
return full
async def test_returns_xml_prepended_to_final_output(self):
self._write("hello.txt", "hello world")
result = await prepareGradingContext(final_output="agent response", directory=self.temp_dir)
assert result.startswith("<workspace_files")
assert result.endswith("agent response")
assert "\n\nagent response" in result
async def test_no_files_returns_just_final_output(self):
result = await prepareGradingContext(final_output="agent response", directory=self.temp_dir)
assert result == "agent response"
async def test_nonexistent_directory_returns_final_output(self):
result = await prepareGradingContext(final_output="agent response", directory="/nonexistent/path")
assert result == "agent response"
async def test_default_workdir(self):
workdir = tempfile.mkdtemp()
sandbox.WORKDIR = workdir
with open(os.path.join(workdir, "file.txt"), "w") as f:
f.write("test")
result = await prepareGradingContext(final_output="output")
assert "<workspace_files" in result
assert result.endswith("output")
shutil.rmtree(workdir)
async def test_extension_filter(self):
self._write("hello.txt", "text")
self._write("hello.py", "print('hi')")
self._write("hello.csv", "a,b")
result = await prepareGradingContext(
final_output="output",
directory=self.temp_dir,
extensions=["txt", "csv"],
)
assert 'type="txt"' in result
assert 'type="csv"' in result
assert 'type="py"' not in result
async def test_unsupported_extension_raises(self):
with pytest.raises(ValueError) as ctx:
await prepareGradingContext(
final_output="output",
directory=self.temp_dir,
extensions=["exe"],
)
assert "Unsupported" in str(ctx.value)
async def test_exclude_patterns(self):
self._write("src/main.py", "code")
self._write("__pycache__/cache.py", "cached")
self._write(".git/config", "git stuff")
result = await prepareGradingContext(final_output="output", directory=self.temp_dir)
assert "main.py" in result
assert "cache.py" not in result
assert "config" not in result
async def test_max_files_limit(self):
for i in range(10):
self._write(f"file{i:02d}.txt", f"content {i}")
result = await prepareGradingContext(final_output="output", directory=self.temp_dir, max_files=3)
assert result.count("<file ") == 3
async def test_max_files_overflow_is_reported_not_silent(self):
# The walk is flat + alphabetical, so a low cap can drop later files.
# Those must be surfaced (count + names), not silently dropped, so a
# rubric does not wrongly conclude an existing file is missing.
for i in range(5):
self._write(f"file{i:02d}.txt", f"content {i}")
result = await prepareGradingContext(final_output="output", directory=self.temp_dir, max_files=2)
assert result.count("<file ") == 2
assert 'truncated_to_max_files="true"' in result
assert 'dropped_file_count="3"' in result
assert "<files_omitted_due_to_max_files" in result
# The dropped files are the alphabetically-later ones.
assert "file04.txt" in result
assert result.count("<omitted_file ") == 3
async def test_no_overflow_has_no_truncation_marker(self):
for i in range(3):
self._write(f"file{i:02d}.txt", f"content {i}")
result = await prepareGradingContext(final_output="output", directory=self.temp_dir, max_files=10)
assert "truncated_to_max_files" not in result
assert "files_omitted_due_to_max_files" not in result
async def test_overflow_marker_emitted_even_when_no_files_fit(self):
# max_files=0: nothing is included, but the dir is non-empty. The grader
# must still see that files exist (and were omitted), not a bare output.
self._write("only.txt", "content")
result = await prepareGradingContext(final_output="output", directory=self.temp_dir, max_files=0)
assert result.startswith("<workspace_files")
assert 'dropped_file_count="1"' in result
assert "only.txt" in result
async def test_unsupported_files_skipped(self):
self._write("image.png", "binary")
self._write("doc.txt", "text")
result = await prepareGradingContext(final_output="output", directory=self.temp_dir)
assert 'type="txt"' in result
# Check for the unsupported file's marker tokens specifically — the
# tempdir path itself can contain "png" as a random substring.
assert "image.png" not in result
assert 'type="png"' not in result
async def test_oversized_special_file_recorded_not_extracted(self):
# A special file over the raw-read cap must be surfaced as evidence (so
# the rubric knows it exists) but NOT extracted — extracting truncated
# PDF/Office bytes would corrupt, and reading it whole could OOM grading.
self._write("big.pdf", "x" * 100)
with mock.patch("core.tools.prepare_grading_context.MAX_SPECIAL_FILE_BYTES", 10):
result = await prepareGradingContext(final_output="output", directory=self.temp_dir)
assert 'type="pdf"' in result
assert 'size_bytes="100"' in result
assert "exceeds" in result and "grading read cap" in result
assert 'truncated="true"' in result
# content is None -> self-closing tag, and the raw bytes never appear.
assert "<big.pdf" not in result
async def test_special_file_under_cap_is_extracted(self):
# A small docx under the cap still flows through normal extraction — the
# cap must not block legitimate files.
from docx import Document
path = os.path.join(self.temp_dir, "report.docx")
doc = Document()
doc.add_paragraph("under-cap evidence")
doc.save(path)
result = await prepareGradingContext(final_output="output", directory=self.temp_dir)
assert 'extraction_method="python-docx"' in result
assert "under-cap evidence" in result
async def test_returns_plain_string(self):
self._write("test.txt", "content")
result = await prepareGradingContext(final_output="my output", directory=self.temp_dir)
assert isinstance(result, str)
# Should NOT be JSON
assert not result.startswith("{")
assert not result.startswith("[")
async def test_empty_directory_returns_final_output(self):
result = await prepareGradingContext(final_output="my output", directory=self.temp_dir)
assert result == "my output"
class TestExtractDocx(unittest.TestCase):
def _make_docx(self, path: str) -> None:
from docx import Document
doc = Document()
doc.add_paragraph("Quarterly summary: revenue up 20%.")
table = doc.add_table(rows=2, cols=2)
table.cell(0, 0).text = "Metric"
table.cell(0, 1).text = "Value"
table.cell(1, 0).text = "Revenue"
table.cell(1, 1).text = "1000"
doc.save(path)
def test_extracts_paragraphs(self):
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "report.docx")
self._make_docx(path)
content, method, truncated = _extract_docx(pathlib.Path(path).read_bytes(), 50_000)
assert method == "python-docx"
assert not truncated
assert content is not None
assert "revenue up 20%" in content
def test_includes_table_cells(self):
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "report.docx")
self._make_docx(path)
content, _method, _truncated = _extract_docx(pathlib.Path(path).read_bytes(), 50_000)
# Regression: table cell text used to be dropped (paragraphs only).
assert content is not None
assert "Metric" in content
assert "Revenue" in content
assert "1000" in content
def test_includes_nested_table_cells(self):
from docx import Document
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "nested.docx")
doc = Document()
outer = doc.add_table(rows=1, cols=1)
cell = outer.cell(0, 0)
cell.paragraphs[0].text = "OuterCell"
inner = cell.add_table(rows=1, cols=2)
inner.cell(0, 0).text = "InnerA"
inner.cell(0, 1).text = "InnerB"
doc.save(path)
content, _method, _truncated = _extract_docx(pathlib.Path(path).read_bytes(), 50_000)
assert content is not None
# cell.text flattens only direct paragraphs; nested tables must recurse.
assert "OuterCell" in content
assert "InnerA" in content
assert "InnerB" in content
def test_missing_file_returns_none(self):
content, method, truncated = _extract_docx(b"not a real docx", 50_000)
assert content is None
assert method == "python-docx"
assert not truncated
class TestExtractXlsx(unittest.TestCase):
def _make_xlsx(self, path: str) -> None:
from openpyxl import Workbook
wb = Workbook()
ws = wb.active
ws.title = "Data"
ws.append(["Name", "Value"])
ws.append(["foo", 42])
wb.save(path)
def test_extracts_rows(self):
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "sheet.xlsx")
self._make_xlsx(path)
content, method, truncated = _extract_xlsx(pathlib.Path(path).read_bytes(), 50_000)
assert method == "openpyxl"
assert not truncated
assert content is not None
assert "--- Sheet: Data ---" in content
assert "Name\tValue" in content
assert "foo\t42" in content
def test_missing_file_returns_none(self):
content, method, truncated = _extract_xlsx(b"not a real xlsx", 50_000)
assert content is None
assert method == "openpyxl"
assert not truncated
def test_every_sheet_appears_even_when_earlier_sheet_truncated(self):
# Regression: a big first sheet used to drain the whole budget, so later
# tabs never appeared. The budget is now per-sheet.
from openpyxl import Workbook
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "big.xlsx")
wb = Workbook()
ws1 = wb.active
ws1.title = "TB Current"
for i in range(2000):
ws1.append([f"row{i}", "Trial Balance as of 12/31/2025", i, i * 2, "padding text"])
ws2 = wb.create_sheet("TB Prior")
ws2.append(["Trial Balance as of 11/30/2025"])
ws2.append(["Prior period data", 123])
wb.save(path)
content, _method, truncated = _extract_xlsx(pathlib.Path(path).read_bytes(), 2000)
assert truncated
assert content is not None
assert "--- Sheet: TB Current ---" in content
assert "--- Sheet: TB Prior ---" in content
# The starved later sheet's data survives, and truncation is attributed.
assert "11/30/2025" in content
assert "Prior period data\t123" in content
assert "'TB Current' truncated" in content
def test_empty_trailing_rows_do_not_consume_budget(self):
# openpyxl read-only dimensions can be inflated; fully-empty rows must be
# skipped so they don't burn the budget producing only blank tab output.
from openpyxl import Workbook
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "sparse.xlsx")
wb = Workbook()
ws = wb.active
ws.title = "Sparse"
ws.append(["header", "value"])
ws.append(["real", 1])
# Force a large inflated dimension with empty cells far below.
ws.cell(row=5000, column=1, value=None)
wb.save(path)
content, _method, truncated = _extract_xlsx(pathlib.Path(path).read_bytes(), 50_000)
assert content is not None
assert not truncated
assert "header\tvalue" in content
assert "real\t1" in content
# No run of empty tab-joined rows should appear.
assert "\t\t\t" not in content
class TestExtractPptx(unittest.TestCase):
def _make_pptx(self, path: str) -> None:
from pptx import Presentation
from pptx.util import Inches
prs = Presentation()
s1 = prs.slides.add_slide(prs.slide_layouts[5])
s1.shapes.title.text = "Helix Pricing SLT Deck"
# Slide 2: a textbox, a table, and speaker notes.
s2 = prs.slides.add_slide(prs.slide_layouts[6])
tb = s2.shapes.add_textbox(Inches(1), Inches(1), Inches(4), Inches(1))
tb.text_frame.text = "Recommended price: $499"
table = s2.shapes.add_table(2, 2, Inches(1), Inches(3), Inches(4), Inches(1)).table
table.cell(0, 0).text = "Tier"
table.cell(0, 1).text = "Price"
table.cell(1, 0).text = "Pro"
table.cell(1, 1).text = "499"
s2.notes_slide.notes_text_frame.text = "Emphasize ROI in the meeting"
prs.save(path)
def test_extracts_slides_tables_and_notes(self):
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "deck.pptx")
self._make_pptx(path)
content, method, truncated = _extract_pptx(pathlib.Path(path).read_bytes(), 50_000)
assert method == "python-pptx"
assert not truncated
assert content is not None
assert "--- Slide 1 ---" in content
assert "--- Slide 2 ---" in content
assert "Helix Pricing SLT Deck" in content
assert "Recommended price: $499" in content
assert "Tier\tPrice" in content
assert "Pro\t499" in content
assert "[Notes] Emphasize ROI in the meeting" in content
def test_missing_file_returns_none(self):
content, method, truncated = _extract_pptx(b"not a real pptx", 50_000)
assert content is None
assert method == "python-pptx"
assert not truncated
class TestPrepareGradingContextBinary(unittest.IsolatedAsyncioTestCase):
async def test_docx_and_xlsx_evidence_assembled(self):
from docx import Document
from openpyxl import Workbook
original_workdir = sandbox.WORKDIR
with tempfile.TemporaryDirectory() as d:
sandbox.WORKDIR = d # reads are confined to WORKDIR
self.addCleanup(setattr, sandbox, "WORKDIR", original_workdir)
doc = Document()
doc.add_paragraph("agent wrote this in a docx")
doc.save(os.path.join(d, "out.docx"))
wb = Workbook()
wb.active.append(["col", "val"])
wb.save(os.path.join(d, "out.xlsx"))
from pptx import Presentation
prs = Presentation()
slide = prs.slides.add_slide(prs.slide_layouts[5])
slide.shapes.title.text = "agent wrote this in a pptx"
prs.save(os.path.join(d, "out.pptx"))
result = await prepareGradingContext(final_output="FINAL", directory=d)
assert 'type="docx"' in result
assert 'extraction_method="python-docx"' in result
assert "agent wrote this in a docx" in result
assert 'type="xlsx"' in result
assert 'extraction_method="openpyxl"' in result
# Regression: a .pptx deliverable used to be silently skipped.
assert 'type="pptx"' in result
assert 'extraction_method="python-pptx"' in result
assert "agent wrote this in a pptx" in result
assert result.endswith("FINAL")
if __name__ == "__main__":
unittest.main()
docker/packages/core/tests/test_privilege.py
+136
View File
@@ -0,0 +1,136 @@
"""Tests for ``core.privilege`` — the workdir-ownership helpers
(``ensure_workdir``, ``chown_tree_to_target``) used by ``core.setup``.
The per-command privilege drop lives in ``core.tools.sandbox`` now (the
server stays root and drops each agent command instead of the whole process);
its tests are in ``test_agent_env.py``.
"""
from __future__ import annotations
from unittest import mock
from core import privilege
def _patch_root(is_root: bool):
return mock.patch("core.privilege.os.geteuid", return_value=0 if is_root else 1000)
# ---------------------------------------------------------------------------
# ensure_workdir
# ---------------------------------------------------------------------------
def test_ensure_workdir_creates_and_chowns_when_root(tmp_path, monkeypatch):
monkeypatch.setenv("SETUID", "1000")
monkeypatch.setenv("SETGID", "1000")
target = tmp_path / "workdir-new"
with (
_patch_root(True),
mock.patch("core.privilege.os.chown") as chown,
mock.patch("core.privilege.sandbox.WORKDIR", str(target)),
):
privilege.ensure_workdir()
assert target.is_dir()
chown.assert_called_once_with(str(target), 1000, 1000)
def test_ensure_workdir_noop_when_not_root(tmp_path):
"""Non-root can't mkdir at / and can't chown, so the whole function is a
no-op — WORKDIR must already exist (or the caller will fail loudly)."""
target = tmp_path / "workdir-missing"
with (
_patch_root(False),
mock.patch("core.privilege.os.chown") as chown,
mock.patch("core.privilege.sandbox.WORKDIR", str(target)),
):
privilege.ensure_workdir()
assert not target.exists()
chown.assert_not_called()
def test_ensure_workdir_skips_chown_when_env_vars_unset(tmp_path, monkeypatch):
monkeypatch.delenv("SETUID", raising=False)
monkeypatch.delenv("SETGID", raising=False)
target = tmp_path / "workdir"
with (
_patch_root(True),
mock.patch("core.privilege.os.chown") as chown,
mock.patch("core.privilege.sandbox.WORKDIR", str(target)),
):
privilege.ensure_workdir()
assert target.is_dir()
chown.assert_not_called()
# ---------------------------------------------------------------------------
# chown_tree_to_target
# ---------------------------------------------------------------------------
def _make_tree(root):
"""Build a small tree: root/a.txt, root/sub/b.txt, root/sub/c.txt."""
root.mkdir(parents=True, exist_ok=True)
(root / "a.txt").write_text("a")
(root / "sub").mkdir()
(root / "sub" / "b.txt").write_text("b")
(root / "sub" / "c.txt").write_text("c")
def test_chown_tree_recursively_when_root(tmp_path, monkeypatch):
monkeypatch.setenv("SETUID", "1000")
monkeypatch.setenv("SETGID", "1000")
target = tmp_path / "workdir"
_make_tree(target)
with _patch_root(True), mock.patch("core.privilege.os.chown") as chown:
privilege.chown_tree_to_target(target)
paths = sorted(call.args[0] for call in chown.call_args_list)
expected = sorted(
[
str(target),
str(target / "a.txt"),
str(target / "sub"),
str(target / "sub" / "b.txt"),
str(target / "sub" / "c.txt"),
]
)
assert paths == expected
for call in chown.call_args_list:
assert call.args[1:] == (1000, 1000)
assert call.kwargs == {"follow_symlinks": False}
def test_chown_tree_noop_when_not_root(tmp_path, monkeypatch):
"""Local dev / CI: same env vars, but no real privilege to chown — skip."""
monkeypatch.setenv("SETUID", "1000")
monkeypatch.setenv("SETGID", "1000")
target = tmp_path / "workdir"
_make_tree(target)
with _patch_root(False), mock.patch("core.privilege.os.chown") as chown:
privilege.chown_tree_to_target(target)
chown.assert_not_called()
def test_chown_tree_noop_when_env_vars_unset(tmp_path, monkeypatch):
monkeypatch.delenv("SETUID", raising=False)
monkeypatch.delenv("SETGID", raising=False)
target = tmp_path / "workdir"
_make_tree(target)
with _patch_root(True), mock.patch("core.privilege.os.chown") as chown:
privilege.chown_tree_to_target(target)
chown.assert_not_called()
def test_chown_tree_uses_minus_one_for_missing_uid_or_gid(tmp_path, monkeypatch):
"""When only one of SETUID/SETGID is set, the other side is left unchanged
via the chown ``-1`` sentinel."""
monkeypatch.setenv("SETUID", "1000")
monkeypatch.delenv("SETGID", raising=False)
target = tmp_path / "workdir"
target.mkdir()
(target / "f.txt").write_text("x")
with _patch_root(True), mock.patch("core.privilege.os.chown") as chown:
privilege.chown_tree_to_target(target)
for call in chown.call_args_list:
assert call.args[1:] == (1000, -1)
docker/packages/core/tests/test_sandbox_tools.py
+1021
View File
File diff suppressed because it is too large Load Diff
docker/packages/core/tests/test_setup.py
+190
View File
@@ -0,0 +1,190 @@
"""Tests for core.setup — copies uploaded context files into the workdir."""
import os
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from core import setup as setup_mod # aliased to avoid clash with pytest's xunit setup_module fixture
from core.tools import sandbox
class SetupTests(unittest.TestCase):
def setUp(self):
self.temp_dir = Path(tempfile.mkdtemp())
self.world_root = self.temp_dir / "world"
self.workdir = self.temp_dir / "workdir"
self.world_root.mkdir()
self._original_workdir = sandbox.WORKDIR
sandbox.WORKDIR = str(self.workdir)
def tearDown(self):
sandbox.WORKDIR = self._original_workdir
shutil.rmtree(self.temp_dir, ignore_errors=True)
def _run(self, task_id: str | None = None) -> int:
"""Invoke setup.main() in-process and return its exit code (0 on normal return)."""
env = {"WORLDBENCH_ROOT": str(self.world_root)}
if task_id is not None:
env["WORLDBENCH_TASK_ID"] = task_id
with mock.patch.dict(os.environ, env, clear=False):
os.environ.pop("BUNDLEDIR", None)
if task_id is None:
os.environ.pop("WORLDBENCH_TASK_ID", None)
try:
setup_mod.main()
return 0
except SystemExit as e:
return int(e.code or 0)
def test_copies_task_specific_files_into_workdir(self):
"""Task-specific setup_data/{task_id}/files/ is copied into WORKDIR."""
task_id = "task-abc"
files_dir = self.world_root / "tasks" / "setup_data" / task_id / "files"
files_dir.mkdir(parents=True)
(files_dir / "james.txt").write_text("james is the CEO")
(files_dir / "nested").mkdir()
(files_dir / "nested" / "notes.md").write_text("# notes")
assert self._run(task_id=task_id) == 0
assert (self.workdir / "james.txt").read_text() == "james is the CEO"
assert (self.workdir / "nested" / "notes.md").read_text() == "# notes"
def test_falls_back_to_generic_setup_data_when_task_dir_missing(self):
"""If no task-specific dir exists, copies from {world_root}/setup_data/files/."""
generic_files = self.world_root / "setup_data" / "files"
generic_files.mkdir(parents=True)
(generic_files / "shared.txt").write_text("shared context")
assert self._run(task_id="unknown-task") == 0
assert (self.workdir / "shared.txt").read_text() == "shared context"
def test_uses_generic_when_no_task_id(self):
"""With no WORLDBENCH_TASK_ID set, generic setup_data/files/ is used."""
generic_files = self.world_root / "setup_data" / "files"
generic_files.mkdir(parents=True)
(generic_files / "default.txt").write_text("default")
assert self._run(task_id=None) == 0
assert (self.workdir / "default.txt").read_text() == "default"
def test_merges_into_existing_workdir(self):
"""Existing workdir contents are preserved; new files are added alongside."""
self.workdir.mkdir(parents=True)
(self.workdir / "preexisting.txt").write_text("keep me")
generic_files = self.world_root / "setup_data" / "files"
generic_files.mkdir(parents=True)
(generic_files / "new.txt").write_text("new file")
assert self._run(task_id=None) == 0
assert (self.workdir / "preexisting.txt").read_text() == "keep me"
assert (self.workdir / "new.txt").read_text() == "new file"
def test_no_setup_data_exits_cleanly(self):
"""If neither task-specific nor generic setup_data exists, exit 0 without error."""
assert self._run(task_id="whatever") == 0
def test_setup_data_without_files_subdir_is_noop(self):
"""setup_data/ with no files/ subdirectory is a no-op (does not error).
Workdir IS created (ensure_workdir runs unconditionally so subsequent
tools have something to chdir into), but no files are copied.
"""
(self.world_root / "setup_data").mkdir()
assert self._run(task_id=None) == 0
def test_rejects_symlink_in_setup_files(self):
"""Symlinks in the source tree are refused — they'd let a bundle
materialize an arbitrary host path (e.g. /app/tasks) into /workdir
and then chown it to the model user."""
files_dir = self.world_root / "setup_data" / "files"
files_dir.mkdir(parents=True)
(files_dir / "real.txt").write_text("ok")
# Target doesn't need to exist for the check to fire.
os.symlink("/app/tasks", str(files_dir / "rubrics"))
assert self._run(task_id=None) == 1
assert not (self.workdir / "rubrics").exists()
assert not (self.workdir / "real.txt").exists() # copytree never ran
def test_rejects_symlinked_files_dir_root(self):
"""A symlinked files/ root is rejected too."""
real_dir = self.world_root / "setup_data" / "real_files"
real_dir.mkdir(parents=True)
(real_dir / "x.txt").write_text("x")
os.symlink(str(real_dir), str(self.world_root / "setup_data" / "files"))
assert self._run(task_id=None) == 1
assert not (self.workdir / "x.txt").exists()
def test_rejects_hardlink_in_setup_files(self):
"""Hardlinked files in the source tree are refused — they could
point at a locked-down inode (e.g. /app/packages/grading/*) and
slip past the symlink guard."""
files_dir = self.world_root / "setup_data" / "files"
files_dir.mkdir(parents=True)
# Two paths sharing one inode: the "target" simulates a protected
# file outside the bundle; the entry inside files/ is its hardlink.
protected = self.temp_dir / "fake-protected.py"
protected.write_text("rubric body")
os.link(str(protected), str(files_dir / "rubric.py"))
assert self._run(task_id=None) == 1
assert not (self.workdir / "rubric.py").exists()
def test_rejects_preexisting_workdir_symlink(self):
"""A symlink planted in /workdir before setup runs is refused —
otherwise copytree(dirs_exist_ok=True) would write through it as
root, into paths the model can't normally write."""
files_dir = self.world_root / "setup_data" / "files"
files_dir.mkdir(parents=True)
(files_dir / "context").mkdir()
(files_dir / "context" / "doc.txt").write_text("payload")
# Plant a model-controlled symlink in /workdir before setup.
self.workdir.mkdir(parents=True)
protected_target = self.temp_dir / "fake-protected-dir"
protected_target.mkdir()
os.symlink(str(protected_target), str(self.workdir / "context"))
assert self._run(task_id=None) == 1
# The bundle file must not have been written through the symlink.
assert not (protected_target / "doc.txt").exists()
def test_chowns_workdir_to_target_uid_gid_when_root(self):
"""Under (mocked) root, copied files are chowned back to SETUID:SETGID
so the unprivileged model user can modify them."""
generic_files = self.world_root / "setup_data" / "files"
generic_files.mkdir(parents=True)
(generic_files / "a.txt").write_text("hello")
(generic_files / "sub").mkdir()
(generic_files / "sub" / "b.txt").write_text("world")
env = {"WORLDBENCH_ROOT": str(self.world_root), "SETUID": "1000", "SETGID": "1000"}
with (
mock.patch.dict(os.environ, env, clear=False),
mock.patch("core.privilege.os.geteuid", return_value=0),
mock.patch("core.privilege.os.chown") as chown,
):
os.environ.pop("BUNDLEDIR", None)
os.environ.pop("WORLDBENCH_TASK_ID", None)
setup_mod.main()
chowned = sorted(call.args[0] for call in chown.call_args_list)
# ensure_workdir chowns WORKDIR; chown_tree_to_target chowns it again plus
# every entry under it. We only assert the post-copy entries are present.
assert str(self.workdir / "a.txt") in chowned
assert str(self.workdir / "sub") in chowned
assert str(self.workdir / "sub" / "b.txt") in chowned
for call in chown.call_args_list:
assert call.args[1:] == (1000, 1000)
if __name__ == "__main__":
unittest.main()
docker/packages/core/tests/test_setup_bundle.py
+120
View File
@@ -0,0 +1,120 @@
"""Tests for the unified-bundle layout in core.setup.
When ``$BUNDLEDIR/files/`` is mounted (the production harness unpacks the unified
trajectory bundle there), core.setup should copy from that path instead
of the legacy ``setup_data/files/`` location. We point ``BUNDLEDIR`` at a
temp path so the test doesn't depend on the production mount path.
"""
import os
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from core import setup as setup_mod # aliased to avoid clash with pytest's xunit setup_module fixture
from core.tools import sandbox
class BundleLayoutTests(unittest.TestCase):
def setUp(self):
self.temp_dir = Path(tempfile.mkdtemp())
self.world_root = self.temp_dir / "world"
self.workdir = self.temp_dir / "workdir"
self.bundle_dir = self.temp_dir / "bundle"
self.world_root.mkdir()
self._env_patch = mock.patch.dict(
os.environ,
{
"WORLDBENCH_ROOT": str(self.world_root),
"BUNDLEDIR": str(self.bundle_dir),
},
clear=False,
)
self._env_patch.start()
os.environ.pop("WORLDBENCH_TASK_ID", None)
self._original_workdir = sandbox.WORKDIR
sandbox.WORKDIR = str(self.workdir)
def tearDown(self):
sandbox.WORKDIR = self._original_workdir
self._env_patch.stop()
shutil.rmtree(self.temp_dir, ignore_errors=True)
def test_bundle_files_take_precedence_over_legacy_setup_data(self):
"""When $BUNDLEDIR/files/ exists, it's used instead of setup_data/files/."""
bundle_files = self.bundle_dir / "files"
bundle_files.mkdir(parents=True)
(bundle_files / "from_bundle.txt").write_text("from bundle")
legacy_files = self.world_root / "setup_data" / "files"
legacy_files.mkdir(parents=True)
(legacy_files / "from_legacy.txt").write_text("from legacy")
setup_mod.main()
assert (self.workdir / "from_bundle.txt").read_text() == "from bundle"
assert not (self.workdir / "from_legacy.txt").exists()
def test_falls_back_to_legacy_when_bundle_dir_absent(self):
"""When BUNDLEDIR points at a path that doesn't exist on disk, the
legacy setup_data path still works."""
# bundle_dir intentionally not created
legacy_files = self.world_root / "setup_data" / "files"
legacy_files.mkdir(parents=True)
(legacy_files / "from_legacy.txt").write_text("from legacy")
setup_mod.main()
assert (self.workdir / "from_legacy.txt").read_text() == "from legacy"
def test_falls_back_to_legacy_when_bundledir_unset(self):
"""When BUNDLEDIR isn't set at all (common local-dev path), the
legacy setup_data path is used."""
os.environ.pop("BUNDLEDIR", None)
legacy_files = self.world_root / "setup_data" / "files"
legacy_files.mkdir(parents=True)
(legacy_files / "from_legacy.txt").write_text("from legacy")
setup_mod.main()
assert (self.workdir / "from_legacy.txt").read_text() == "from legacy"
def test_bundle_and_legacy_produce_equivalent_workdir(self):
"""Same files served via either layout yield the same WORKDIR contents."""
contents = {"a.txt": "alpha", "nested/b.md": "# beta"}
# First run: bundle layout
bundle_files = self.bundle_dir / "files"
for rel, body in contents.items():
p = bundle_files / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(body)
setup_mod.main()
bundle_snapshot = {
str(p.relative_to(self.workdir)): p.read_text() for p in self.workdir.rglob("*") if p.is_file()
}
# Second run: same files via legacy layout, fresh workdir
shutil.rmtree(self.bundle_dir)
shutil.rmtree(self.workdir, ignore_errors=True)
legacy_files = self.world_root / "setup_data" / "files"
for rel, body in contents.items():
p = legacy_files / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(body)
setup_mod.main()
legacy_snapshot = {
str(p.relative_to(self.workdir)): p.read_text() for p in self.workdir.rglob("*") if p.is_file()
}
assert bundle_snapshot == legacy_snapshot
if __name__ == "__main__":
unittest.main()
docker/packages/core/tests/test_viewer.py
+55
View File
@@ -0,0 +1,55 @@
"""Tests for core's sandbox viewer app — focused on the download path, which
streams file bytes from a uid-1000 reader instead of buffering them whole."""
import os
import pytest
from starlette.testclient import TestClient
from core import viewer
from core.tools import sandbox
@pytest.fixture
def client(tmp_path, monkeypatch):
# Point WORKDIR at a temp dir so reads stay confined to it, and run the
# viewer with no proxy token so the middleware lets requests through.
monkeypatch.setattr(sandbox, "WORKDIR", str(tmp_path))
monkeypatch.setattr(viewer, "get_proxy_token", lambda: None)
return TestClient(viewer.create_core_viewer_app())
def test_download_streams_file_contents(client, tmp_path):
payload = bytes(range(256)) * 1000 # 256 KB — larger than one read() chunk
(tmp_path / "data.bin").write_bytes(payload)
resp = client.get("/api/download", params={"path": "data.bin"})
assert resp.status_code == 200
assert resp.content == payload
assert resp.headers["content-type"] == "application/octet-stream"
assert 'filename="data.bin"' in resp.headers["content-disposition"]
def test_download_empty_file(client, tmp_path):
(tmp_path / "empty.bin").write_bytes(b"")
resp = client.get("/api/download", params={"path": "empty.bin"})
assert resp.status_code == 200
assert resp.content == b""
def test_download_missing_file_returns_404(client):
resp = client.get("/api/download", params={"path": "nope.bin"})
assert resp.status_code == 404
def test_download_path_escape_rejected(client):
resp = client.get("/api/download", params={"path": "../../etc/passwd"})
assert resp.status_code == 400
def test_download_fifo_returns_404_not_hang(client, tmp_path):
# An agent-created named pipe must not block a request worker waiting for a
# writer — it's rejected as a non-regular file. (No writer is opened, so a
# regression here would hang the test.)
os.mkfifo(tmp_path / "pipe")
resp = client.get("/api/download", params={"path": "pipe"})
assert resp.status_code == 404
docker/packages/google_calendar/CAPABILITIES.md
+45
View File
@@ -0,0 +1,45 @@
# Google Calendar Capabilities
A mock calendar service supporting multiple calendars, recurring events, attendees with RSVP tracking, and availability checking.
## What the agent can do
**Manage events.** Create, read, update, and delete calendar events. Events can be timed (with specific start/end times) or all-day. Events support descriptions, locations, attendees, reminders, and recurrence rules.
**Recurring events.** Create events that repeat on a schedule — daily, weekly (with specific days like Tuesday/Thursday), or monthly. Recurrence supports end conditions (after N occurrences, or until a specific date). When listing events, recurring events are automatically expanded into individual instances within the query range.
**Multiple calendars.** Create named calendars (e.g., "Personal", "Work", "Team Meetings") and manage events across them. List all calendars with event counts. When listing events without specifying a calendar, results come from all calendars. Each calendar is isolated — events in one don't appear in another unless explicitly queried.
**Event search.** Search events across one or all calendars by keyword, quoted phrase, location, description, attendee email/name, creator, organizer, or calendar name. Search can be combined with time ranges, attendee filters, RSVP response-status filters, creator filters, and organizer filters. Recurring events are expanded within the requested range; without a range, search expands recurring events within a one-year default window from the series start.
**Attendees and RSVPs.** Invite people to events by email. Track RSVP responses (accepted, declined, tentative, needs action). Update individual attendee responses.
**Availability checking.** Check whether a time range is free or busy. Returns busy periods and available free slots of at least a specified duration. Can filter by specific attendees — for example, "find a free hour that works for both Alice and Bob." Declined events are excluded from busy calculations. Works across recurring events too.
**Reminders.** Set reminders on events (e.g., popup 15 minutes before, email 1 hour before). Reminders are stored but not actually triggered in the mock.
## Coverage gaps
- No "edit this and all future events" for recurring series (can only edit the template)
- No event colors or categories
- No shared calendar permissions (all calendars are fully accessible)
- No time zone conversion tools
- No meeting room or resource booking
- Search uses `query` rather than the real Google Calendar `q` parameter, and does not implement `eventTypes`
- Reminders are stored but don't fire
## Toolsets
10 tools total. Toolsets map to `WORLDBENCH_TOOL_SETS` values (prefixed form — e.g., `google_calendar_events`).
| Toolset | Tools | Description |
|---------|-------|-------------|
| `all` / `google_calendar_all` | 10 | Everything |
| `read` / `google_calendar_read` | 5 | Read-only: get, list/search events, list calendars, check availability |
| `write` / `google_calendar_write` | 5 | Write: create/update/delete event, create calendar, respond to event |
| `google_calendar_events` | 6 | Event CRUD plus list/search events |
| `google_calendar_calendars` | 2 | Calendar management: create, list calendars |
| `google_calendar_scheduling` | 2 | Availability + RSVP: check availability, respond to event |
| `google_calendar_core` | 6 | Baseline event management (legacy Toolathlon subset plus search) |
| `google_calendar_toolathlon_legacy` | 5 | Legacy Toolathlon tool subset (pre-integration) |
| `google_calendar_state` | 2 | `export_state`, `import_state` for fixture seeding and grading |
docker/packages/google_calendar/google_calendar/__init__.py
+37
View File
@@ -0,0 +1,37 @@
"""Google Calendar MCP server package."""
from __future__ import annotations
import argparse
import logging
import os
from .server import mcp
def main() -> None:
parser = argparse.ArgumentParser(description="Google Calendar MCP Server")
parser.add_argument("--debug", action="store_true", help="Enable debug logging")
parser.add_argument("--agent-workspace", help="Agent workspace path used to resolve persistent calendar state")
args = parser.parse_args()
logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)
from .async_tool_guard import assert_tools_async
assert_tools_async(mcp)
from .state import init_state
init_state(args.agent_workspace)
port = os.environ.get("PORT")
if port:
from .viewer import run_http_server
run_http_server(mcp, int(port))
else:
mcp.run()
__all__ = ["main", "mcp"]
docker/packages/google_calendar/google_calendar/__main__.py
+5
View File
@@ -0,0 +1,5 @@
"""Entry point for ``python -m google_calendar``."""
from google_calendar import main
main()
docker/packages/google_calendar/google_calendar/async_tool_guard.py
+59
View File
@@ -0,0 +1,59 @@
"""Boot-time guard: every registered MCP tool must be async.
Sync (`def`) tools make FastMCP run pydantic argument validation in an anyio
worker threadpool. Under concurrent calls the shared pydantic-core validator is
re-entered across threads and panics with ``pyo3_runtime.PanicException:
dictionary changed size during iteration``, which tears down the
StreamableHTTP task group and turns every later request into a 500. Async
(`async def`) tools validate on the single-threaded event loop and are safe.
This guard runs at server boot (and therefore during ``mcp-proxy gen``, which
boots every server): a non-conformant package fails fast instead of shipping a
server that can crash under load. The same check is intentionally duplicated in
each package because the bundled prod images install only that package's own
dependencies, so there is no shared runtime module to import.
"""
from __future__ import annotations
import asyncio
import inspect
from functools import partial
from typing import Any
def _unwrap(fn: Any) -> Any:
while isinstance(fn, partial):
fn = fn.func
return fn
def find_sync_tools(mcp: Any) -> list[str]:
"""Return the names of registered tools whose function is not a coroutine."""
# mcp.server.fastmcp.FastMCP exposes a synchronous tool manager.
manager = getattr(mcp, "_tool_manager", None)
if manager is not None and hasattr(manager, "list_tools"):
return sorted(t.name for t in manager.list_tools() if not inspect.iscoroutinefunction(_unwrap(t.fn)))
# fastmcp.FastMCP exposes an async API.
async def _collect() -> list[str]:
sync: list[str] = []
for descriptor in await mcp.list_tools():
tool = await mcp.get_tool(descriptor.name)
if not inspect.iscoroutinefunction(_unwrap(tool.fn)):
sync.append(descriptor.name)
return sorted(sync)
return asyncio.run(_collect())
def assert_tools_async(mcp: Any) -> None:
"""Raise if any registered tool is synchronous."""
sync = find_sync_tools(mcp)
if sync:
raise RuntimeError(
"MCP tools must be async (`async def`). Sync tools run pydantic argument "
"validation in a worker threadpool and can trigger a pydantic-core "
"'dictionary changed size during iteration' panic under concurrent calls, "
"which kills the server. Make these tools async: " + ", ".join(sync)
)
docker/packages/google_calendar/google_calendar/models.py
+608
View File
@@ -0,0 +1,608 @@
"""Pydantic models and typed aliases for the Google Calendar mock."""
import re
from datetime import UTC, date, datetime
from enum import StrEnum
from typing import Annotated, Any, TypedDict, cast
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from pydantic import BaseModel, EmailStr, Field, field_validator, model_validator
_RFC3339_DATETIME_RE = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})?$")
_RFC3339_OFFSET_DATETIME_RE = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})$")
_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
DEFAULT_RECURRING_SEARCH_DAYS = 365
DEFAULT_CALENDAR_TIME_ZONE = "UTC"
# ---------------------------------------------------------------------------
# Pydantic Types for Event Ingestion
# ---------------------------------------------------------------------------
NonEmptyStateString = Annotated[str, Field(min_length=1)]
Rfc3339DateTimeString = Annotated[str, Field(pattern=_RFC3339_DATETIME_RE.pattern)]
Rfc3339OffsetDateTimeString = Annotated[str, Field(pattern=_RFC3339_OFFSET_DATETIME_RE.pattern)]
CalendarDateString = Annotated[str, Field(pattern=_DATE_RE.pattern)]
EventId = NonEmptyStateString
CalendarId = NonEmptyStateString
CalendarAccountId = NonEmptyStateString
CalendarTimeZone = Annotated[str, Field(description="IANA time zone name, e.g. America/New_York")]
EventSummary = NonEmptyStateString
CalendarSummary = NonEmptyStateString
EventQuery = NonEmptyStateString
RecurrenceRuleString = Annotated[str, Field(pattern=r"^(?:RRULE|EXRULE|RDATE|EXDATE):")]
PeopleResourceName = Annotated[
str,
Field(
pattern=r"^people/c[0-9]+$",
description='Google People API resource name used by Calendar birthdays, e.g. "people/c12345".',
),
]
ListEventsMaxResults = Annotated[int, Field(ge=0)]
AvailabilityDurationMinutes = Annotated[int, Field(ge=1)]
def _validate_time_zone(value: str) -> ZoneInfo:
try:
return ZoneInfo(value)
except ZoneInfoNotFoundError as e:
raise ValueError("timeZone must be a valid IANA time zone") from e
def parse_rfc3339_datetime(value: str, time_zone: str | None = None) -> datetime:
if not _RFC3339_DATETIME_RE.fullmatch(value):
raise ValueError("dateTime must be an RFC 3339 date-time")
normalized = value.replace("Z", "+00:00")
try:
parsed = datetime.fromisoformat(normalized)
except ValueError as e:
raise ValueError("dateTime must be a valid RFC 3339 date-time") from e
if time_zone is not None:
zone = _validate_time_zone(time_zone)
if parsed.tzinfo is None:
return parsed.replace(tzinfo=zone)
expected_offset = zone.utcoffset(parsed.replace(tzinfo=None))
if expected_offset != parsed.utcoffset():
raise ValueError("dateTime offset must match timeZone")
return parsed.astimezone(zone)
return parsed
def _parse_calendar_date(value: str) -> date:
if not _DATE_RE.fullmatch(value):
raise ValueError("date must use YYYY-MM-DD format")
try:
return datetime.strptime(value, "%Y-%m-%d").date()
except ValueError as e:
raise ValueError("date must be a valid calendar date") from e
class EventStatus(StrEnum):
"""Event status — mirrors Google Calendar API values."""
CONFIRMED = "confirmed"
TENTATIVE = "tentative"
CANCELLED = "cancelled"
class EventVisibility(StrEnum):
"""Event visibility — mirrors Google Calendar API values."""
DEFAULT = "default"
PUBLIC = "public"
PRIVATE = "private"
CONFIDENTIAL = "confidential"
class EventTransparency(StrEnum):
"""Whether an event blocks availability in free/busy calculations."""
OPAQUE = "opaque"
TRANSPARENT = "transparent"
class EventType(StrEnum):
"""Event type values supported by Google Calendar."""
DEFAULT = "default"
OUT_OF_OFFICE = "outOfOffice"
FOCUS_TIME = "focusTime"
FROM_GMAIL = "fromGmail"
WORKING_LOCATION = "workingLocation"
BIRTHDAY = "birthday"
class EventClearField(StrEnum):
"""Optional event fields that update_event can remove explicitly."""
DESCRIPTION = "description"
LOCATION = "location"
STATUS = "status"
COLOR_ID = "colorId"
VISIBILITY = "visibility"
TRANSPARENCY = "transparency"
RECURRENCE = "recurrence"
REMINDERS = "reminders"
ATTENDEES = "attendees"
CREATOR = "creator"
ORGANIZER = "organizer"
EXTENDED_PROPERTIES = "extendedProperties"
SOURCE = "source"
OUT_OF_OFFICE_PROPERTIES = "outOfOfficeProperties"
FOCUS_TIME_PROPERTIES = "focusTimeProperties"
WORKING_LOCATION_PROPERTIES = "workingLocationProperties"
BIRTHDAY_PROPERTIES = "birthdayProperties"
HTML_LINK = "htmlLink"
HANGOUT_LINK = "hangoutLink"
EVENT_TYPE_PROPERTY_FIELDS = {
EventType.OUT_OF_OFFICE: "outOfOfficeProperties",
EventType.FOCUS_TIME: "focusTimeProperties",
EventType.WORKING_LOCATION: "workingLocationProperties",
EventType.BIRTHDAY: "birthdayProperties",
}
class AutoDeclineMode(StrEnum):
"""Auto-decline behavior for focus time and out-of-office events."""
DECLINE_NONE = "declineNone"
DECLINE_ALL_CONFLICTING_INVITATIONS = "declineAllConflictingInvitations"
DECLINE_ONLY_NEW_CONFLICTING_INVITATIONS = "declineOnlyNewConflictingInvitations"
class FocusTimeChatStatus(StrEnum):
"""Chat status values for focus time events."""
AVAILABLE = "available"
DO_NOT_DISTURB = "doNotDisturb"
class WorkingLocationType(StrEnum):
"""Working location type values supported by Google Calendar."""
HOME_OFFICE = "homeOffice"
OFFICE_LOCATION = "officeLocation"
CUSTOM_LOCATION = "customLocation"
class BirthdayType(StrEnum):
"""Birthday or special-date type values supported by Google Calendar."""
ANNIVERSARY = "anniversary"
BIRTHDAY = "birthday"
CUSTOM = "custom"
OTHER = "other"
SELF = "self"
class AttendeeResponseStatus(StrEnum):
"""Attendee RSVP state — mirrors Google Calendar API values."""
NEEDS_ACTION = "needsAction"
DECLINED = "declined"
TENTATIVE = "tentative"
ACCEPTED = "accepted"
class SearchEventsOrderBy(StrEnum):
"""Sort options for search_events results."""
START_TIME = "startTime"
UPDATED = "updated"
class ReminderMethod(StrEnum):
"""Event reminder delivery methods supported by Google Calendar."""
EMAIL = "email"
POPUP = "popup"
SearchEventsMaxResults = Annotated[int, Field(ge=0)]
ReminderMinutes = Annotated[int, Field(ge=0, le=40320)]
class EventDateTime(BaseModel):
"""Time specification for an event — mirrors the Google Calendar API shape."""
model_config = {"extra": "forbid"}
dateTime: Rfc3339DateTimeString | None = None # RFC 3339 format for timed events
date: CalendarDateString | None = None # YYYY-MM-DD for all-day events
timeZone: CalendarTimeZone | None = None
@model_validator(mode="after")
def validate_date_or_datetime(self) -> "EventDateTime":
has_datetime = bool(self.dateTime)
has_date = bool(self.date)
if has_datetime == has_date:
raise ValueError("Event time must include exactly one of dateTime or date")
if self.timeZone is not None:
_validate_time_zone(self.timeZone)
if self.dateTime is not None:
parsed = parse_rfc3339_datetime(self.dateTime, self.timeZone)
if parsed.tzinfo is None and not self.timeZone:
raise ValueError("dateTime without an offset requires timeZone")
if self.date is not None:
_parse_calendar_date(self.date)
return self
def _sort_value(self) -> Any:
if self.dateTime is not None:
return parse_rfc3339_datetime(self.dateTime, self.timeZone)
if self.date is not None:
return _parse_calendar_date(self.date)
raise ValueError("Event time must include exactly one of dateTime or date")
class EventAttendee(BaseModel):
"""Attendee on an event — mirrors the Google Calendar API shape."""
model_config = {"extra": "forbid"}
email: EmailStr
displayName: str | None = None
organizer: bool | None = None
self: bool | None = None
resource: bool | None = None
optional: bool | None = None
responseStatus: AttendeeResponseStatus | None = None
comment: str | None = None
additionalGuests: int | None = Field(default=None, ge=0)
class CalendarPerson(BaseModel):
"""Creator or organizer person object on an event."""
model_config = {"extra": "forbid"}
id: str | None = None
email: EmailStr | None = None
displayName: str | None = None
self: bool | None = None
class ExtendedProperties(BaseModel):
"""Arbitrary event metadata supported by Google Calendar."""
model_config = {"extra": "forbid"}
private: dict[str, str] | None = None
shared: dict[str, str] | None = None
class EventSource(BaseModel):
"""Source metadata for an event."""
model_config = {"extra": "forbid"}
title: NonEmptyStateString | None = None
# Add url once worlds have hosted or Drive-backed source links.
class EventReminder(BaseModel):
"""Event reminder override — mirrors Google Calendar API values."""
model_config = {"extra": "forbid"}
method: ReminderMethod
minutes: ReminderMinutes
class EventReminders(BaseModel):
"""Reminder settings for an event."""
model_config = {"extra": "forbid"}
useDefault: bool
overrides: list[EventReminder] | None = Field(default=None, min_length=1, max_length=5)
@model_validator(mode="after")
def validate_overrides_match_default_mode(self) -> "EventReminders":
if self.useDefault and self.overrides is not None:
raise ValueError("reminders.overrides cannot be set when useDefault is true")
if not self.useDefault and self.overrides is None:
raise ValueError("reminders.overrides is required when useDefault is false")
return self
class OutOfOfficeProperties(BaseModel):
"""Out-of-office event data."""
model_config = {"extra": "forbid"}
autoDeclineMode: AutoDeclineMode | None = None
declineMessage: str | None = None
class FocusTimeProperties(BaseModel):
"""Focus time event data."""
model_config = {"extra": "forbid"}
autoDeclineMode: AutoDeclineMode | None = None
declineMessage: str | None = None
chatStatus: FocusTimeChatStatus | None = None
class WorkingLocationCustomLocation(BaseModel):
"""Custom working location details."""
model_config = {"extra": "forbid"}
label: str | None = None
class WorkingLocationOfficeLocation(BaseModel):
"""Office working location details."""
model_config = {"extra": "forbid"}
buildingId: str | None = None
floorId: str | None = None
floorSectionId: str | None = None
deskId: str | None = None
label: str | None = None
class WorkingLocationHomeOffice(BaseModel):
"""Home office marker for working location events."""
model_config = {"extra": "forbid"}
class WorkingLocationProperties(BaseModel):
"""Working location event data."""
model_config = {"extra": "forbid"}
type: WorkingLocationType
homeOffice: WorkingLocationHomeOffice | None = None
customLocation: WorkingLocationCustomLocation | None = None
officeLocation: WorkingLocationOfficeLocation | None = None
@model_validator(mode="after")
def validate_location_matches_type(self) -> "WorkingLocationProperties":
if self.type == WorkingLocationType.CUSTOM_LOCATION and self.customLocation is None:
raise ValueError("customLocation is required when working location type is customLocation")
if self.type == WorkingLocationType.OFFICE_LOCATION and self.officeLocation is None:
raise ValueError("officeLocation is required when working location type is officeLocation")
if self.type != WorkingLocationType.CUSTOM_LOCATION and self.customLocation is not None:
raise ValueError("customLocation can only be set when type is customLocation")
if self.type != WorkingLocationType.OFFICE_LOCATION and self.officeLocation is not None:
raise ValueError("officeLocation can only be set when type is officeLocation")
if self.type != WorkingLocationType.HOME_OFFICE and self.homeOffice is not None:
raise ValueError("homeOffice can only be set when type is homeOffice")
return self
class BirthdayProperties(BaseModel):
"""Birthday or special-date event data."""
model_config = {"extra": "forbid"}
type: BirthdayType = BirthdayType.BIRTHDAY
contact: PeopleResourceName | None = None
customTypeName: str | None = None
@model_validator(mode="after")
def validate_birthday_shape(self) -> "BirthdayProperties":
if self.type == BirthdayType.SELF and self.contact is not None:
raise ValueError("self birthday events cannot have contact")
if self.type in {BirthdayType.ANNIVERSARY, BirthdayType.CUSTOM, BirthdayType.OTHER} and self.contact is None:
raise ValueError(f"{self.type.value} birthday events require contact")
if self.type == BirthdayType.CUSTOM and not self.customTypeName:
raise ValueError("custom birthday events require customTypeName")
if self.type != BirthdayType.CUSTOM and self.customTypeName is not None:
raise ValueError("customTypeName can only be set when birthday type is custom")
return self
class EventInput(BaseModel):
"""Schema for ingesting events from external systems (e.g., CSV, APIs)."""
model_config = {"extra": "forbid"}
summary: NonEmptyStateString
start: EventDateTime
end: EventDateTime
description: str | None = None
location: str | None = None
status: EventStatus | None = None
colorId: str | None = None
visibility: EventVisibility | None = None
transparency: EventTransparency | None = None
recurrence: list[RecurrenceRuleString] | None = None
reminders: EventReminders | None = None
attendees: list[EventAttendee] | None = None
creator: CalendarPerson | None = None
organizer: CalendarPerson | None = None
extendedProperties: ExtendedProperties | None = None
source: EventSource | None = None
eventType: EventType = EventType.DEFAULT
outOfOfficeProperties: OutOfOfficeProperties | None = None
focusTimeProperties: FocusTimeProperties | None = None
workingLocationProperties: WorkingLocationProperties | None = None
birthdayProperties: BirthdayProperties | None = None
htmlLink: str | None = None
hangoutLink: str | None = None
@field_validator("recurrence", mode="before")
@classmethod
def _coerce_recurrence(cls, value: object) -> object:
# Legacy fixtures sometimes store a single RRULE string rather than a list.
if isinstance(value, str):
return [value]
return value
@model_validator(mode="before")
@classmethod
def default_from_gmail_transparency(cls, value: object) -> object:
if isinstance(value, dict):
raw_value = cast(dict[str, object], value)
if raw_value.get("eventType") != EventType.FROM_GMAIL.value:
return value
value = dict(raw_value)
value.setdefault("transparency", EventTransparency.TRANSPARENT.value)
return value
@model_validator(mode="after")
def validate_start_end_shape_and_order(self) -> "EventInput":
start_is_datetime = self.start.dateTime is not None
end_is_datetime = self.end.dateTime is not None
if start_is_datetime != end_is_datetime:
raise ValueError("Event start and end must both use dateTime or both use date")
start_value = self.start._sort_value()
end_value = self.end._sort_value()
if isinstance(start_value, datetime) and isinstance(end_value, datetime):
if start_value.tzinfo is None:
start_value = start_value.replace(tzinfo=UTC)
if end_value.tzinfo is None:
end_value = end_value.replace(tzinfo=UTC)
if end_value <= start_value:
raise ValueError("Event end must be after start")
return self
@model_validator(mode="after")
def validate_unique_attendees(self) -> "EventInput":
if self.attendees is None:
return self
seen_emails = set()
for attendee in self.attendees:
email = attendee.email.lower()
if email in seen_emails:
raise ValueError(f"Duplicate attendee email: {attendee.email}")
seen_emails.add(email)
return self
@model_validator(mode="after")
def validate_event_type_properties(self) -> "EventInput":
property_by_type = {
EventType.OUT_OF_OFFICE: ("outOfOfficeProperties", self.outOfOfficeProperties),
EventType.FOCUS_TIME: ("focusTimeProperties", self.focusTimeProperties),
EventType.WORKING_LOCATION: ("workingLocationProperties", self.workingLocationProperties),
EventType.BIRTHDAY: ("birthdayProperties", self.birthdayProperties),
}
for event_type, (field_name, value) in property_by_type.items():
if self.eventType == event_type and value is None:
raise ValueError(f"{field_name} is required when eventType is {event_type.value}")
if self.eventType != event_type and value is not None:
raise ValueError(f"{field_name} can only be set when eventType is {event_type.value}")
return self
class Event(EventInput):
"""Full stored event with system-generated fields."""
id: NonEmptyStateString
# Optional so synthetic/legacy snapshots without audit timestamps still round-trip.
created: Rfc3339DateTimeString | None = None
updated: Rfc3339DateTimeString | None = None
@field_validator("created", "updated")
@classmethod
def _validate_audit_timestamp(cls, value: str | None) -> str | None:
if value is not None:
parsed = parse_rfc3339_datetime(value)
if parsed.tzinfo is None:
raise ValueError("audit timestamps must include an offset")
return value
def dump_model(value: Any, model_type: type[BaseModel]) -> dict[str, Any]:
if isinstance(value, model_type):
return value.model_dump(exclude_none=True)
return model_type.model_validate(value).model_dump(exclude_none=True)
def event_attendee_payload(attendee: EventAttendee | dict[str, Any]) -> dict[str, Any]:
if not isinstance(attendee, EventAttendee):
attendee = EventAttendee.model_validate(attendee)
payload = attendee.model_dump(exclude_none=True)
payload["displayName"] = payload.get("displayName") or attendee.email
payload["responseStatus"] = payload.get("responseStatus") or AttendeeResponseStatus.NEEDS_ACTION.value
payload["organizer"] = payload.get("organizer", False)
payload["self"] = payload.get("self", False)
return payload
class Calendar(BaseModel):
"""Stored secondary calendar metadata and events."""
model_config = {"extra": "forbid"}
summary: NonEmptyStateString
description: str = ""
timeZone: CalendarTimeZone = DEFAULT_CALENDAR_TIME_ZONE
events: dict[NonEmptyStateString, Event] = Field(default_factory=dict)
@field_validator("timeZone")
@classmethod
def _validate_time_zone(cls, value: str) -> str:
_validate_time_zone(value)
return value
@model_validator(mode="after")
def validate_events_keyed_by_id(self) -> "Calendar":
for key, event in self.events.items():
if key != event.id:
raise ValueError(f"events key {key!r} does not match event.id {event.id!r}")
return self
class CalendarState(BaseModel):
"""Full google_calendar state — round-trips with load_data()/save_data()."""
model_config = {"extra": "forbid"}
timeZone: CalendarTimeZone = DEFAULT_CALENDAR_TIME_ZONE
events: dict[NonEmptyStateString, Event] = Field(default_factory=dict)
calendars: dict[NonEmptyStateString, Calendar] = Field(default_factory=dict)
@field_validator("timeZone")
@classmethod
def _validate_time_zone(cls, value: str) -> str:
_validate_time_zone(value)
return value
@model_validator(mode="after")
def validate_primary_calendar_is_flat(self) -> "CalendarState":
if "primary" in self.calendars:
raise ValueError("Primary calendar events must be stored in top-level events, not calendars['primary']")
return self
@model_validator(mode="after")
def validate_events_keyed_by_id(self) -> "CalendarState":
for key, event in self.events.items():
if key != event.id:
raise ValueError(f"events key {key!r} does not match event.id {event.id!r}")
return self
class CalendarAccountsState(BaseModel):
"""Multi-account google_calendar state wrapper.
Each account is fully isolated: calendar IDs, event IDs, calendars, and
availability/search state are scoped to the selected account.
"""
model_config = {"extra": "forbid"}
accounts: dict[NonEmptyStateString, CalendarState] = Field(default_factory=dict)
GoogleCalendarState = CalendarState | CalendarAccountsState
class SearchEventsResponse(TypedDict, total=False):
status: str
message: str
events: list[dict[str, Any]]
count: int
warnings: list[str]
skipped_unparseable: list[str]
docker/packages/google_calendar/google_calendar/server.py
+492
View File
@@ -0,0 +1,492 @@
"""FastMCP tool surface for the Google Calendar mock."""
import functools
import inspect
from fastmcp import FastMCP
from pydantic import EmailStr
from .models import (
AttendeeResponseStatus,
AvailabilityDurationMinutes,
BirthdayProperties,
CalendarAccountId,
CalendarAccountsState,
CalendarId,
CalendarPerson,
CalendarState,
CalendarSummary,
CalendarTimeZone,
EventAttendee,
EventClearField,
EventDateTime,
EventId,
EventReminders,
EventSource,
EventSummary,
EventTransparency,
EventType,
ExtendedProperties,
FocusTimeProperties,
GoogleCalendarState,
ListEventsMaxResults,
OutOfOfficeProperties,
RecurrenceRuleString,
Rfc3339OffsetDateTimeString,
SearchEventsMaxResults,
SearchEventsOrderBy,
SearchEventsResponse,
WorkingLocationProperties,
)
from .state import list_accounts as list_calendar_accounts
from .state import set_active_account, state_from_json, state_to_json, write_snapshots
from .tools import availability, calendars, events, search
mcp = FastMCP("google_calendar")
def _snapshot_on_write(fn):
"""Write configured state snapshots after successful write tools.
The wrapper is async so the registered tool is a coroutine: FastMCP then
validates and runs it on the event loop instead of a worker threadpool,
avoiding the pydantic-core concurrency panic. See ``async_tool_guard``.
"""
@functools.wraps(fn)
async def wrapper(*args, **kwargs):
result = fn(*args, **kwargs)
if inspect.isawaitable(result):
result = await result
write_snapshots()
return result
return wrapper
def _with_account(fn):
"""Decorator: extract account_id from tool calls and select that account."""
@functools.wraps(fn)
async def wrapper(*args, **kwargs):
account_id = kwargs.pop("account_id", None)
if account_id is not None:
set_active_account(account_id)
result = fn(*args, **kwargs)
if inspect.isawaitable(result):
result = await result
return result
return wrapper
@mcp.tool()
async def export_state() -> GoogleCalendarState:
"""Export the full google_calendar state as JSON.
Round-trips with import_state.
"""
state = state_to_json()
if "accounts" in state:
return CalendarAccountsState.model_validate(state)
return CalendarState.model_validate(state)
@mcp.tool()
@_snapshot_on_write
def import_state(state: GoogleCalendarState) -> dict:
"""Replace the google_calendar state with the provided JSON.
For synthetic-data injection and test setup. Round-trips with export_state.
"""
state_from_json(state)
return {"status": "success"}
@mcp.tool()
async def list_accounts() -> dict:
"""List available isolated Google Calendar accounts.
Multi-account worlds store each account as a fully separate calendar state.
Use the returned ``account_id`` with other tools to select the account to
read or mutate. Single-account worlds expose their only configured account.
"""
accounts = list_calendar_accounts()
return {"status": "success", "accounts": accounts, "count": len(accounts)}
@mcp.tool()
@_with_account
@_snapshot_on_write
def create_event(
summary: EventSummary,
start: EventDateTime,
end: EventDateTime,
description: str | None = None,
location: str | None = None,
calendar_id: CalendarId = "primary",
recurrence: list[RecurrenceRuleString] | None = None,
reminders: EventReminders | None = None,
attendees: list[EventAttendee] | None = None,
creator: CalendarPerson | None = None,
organizer: CalendarPerson | None = None,
extendedProperties: ExtendedProperties | None = None,
source: EventSource | None = None,
transparency: EventTransparency | None = None,
eventType: EventType = EventType.DEFAULT,
outOfOfficeProperties: OutOfOfficeProperties | None = None,
focusTimeProperties: FocusTimeProperties | None = None,
workingLocationProperties: WorkingLocationProperties | None = None,
birthdayProperties: BirthdayProperties | None = None,
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Creates a new event in Google Calendar.
Args:
summary: Event title
start: Start time object with either 'dateTime' (ISO format, e.g. '2025-12-10T09:00:00-05:00')
for timed events, or 'date' (YYYY-MM-DD, e.g. '2025-12-10') for all-day events.
Optionally include 'timeZone' (e.g. 'America/New_York').
end: End time object with either 'dateTime' or 'date' (same format as start).
For all-day events, 'date' is exclusive (e.g. end '2025-12-11' means the event ends on 2025-12-10).
description: Event description
location: Event location
calendar_id: Calendar to create the event in (default 'primary')
recurrence: RRULE strings for recurring events (e.g., ['RRULE:FREQ=WEEKLY;BYDAY=TU;COUNT=10'])
reminders: Reminder overrides, e.g. {"useDefault": false, "overrides": [{"method": "popup", "minutes": 15}]}
attendees: List of attendees, e.g. [{"email": "alice@co.com", "displayName": "Alice"}]. responseStatus defaults to 'needsAction'.
transparency: Whether the event blocks availability. 'opaque' is busy; 'transparent' is free.
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
The created event object with its ID
"""
return events.create_event(
summary=summary,
start=start,
end=end,
description=description,
location=location,
calendar_id=calendar_id,
recurrence=recurrence,
reminders=reminders,
attendees=attendees,
creator=creator,
organizer=organizer,
extendedProperties=extendedProperties,
source=source,
transparency=transparency,
eventType=eventType,
outOfOfficeProperties=outOfOfficeProperties,
focusTimeProperties=focusTimeProperties,
workingLocationProperties=workingLocationProperties,
birthdayProperties=birthdayProperties,
)
@mcp.tool()
@_with_account
def get_event(
eventId: EventId,
calendar_id: CalendarId = "primary",
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Retrieves details of a specific event.
Args:
eventId: ID of the event to retrieve
calendar_id: Calendar to look in (default 'primary')
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
The event object if found
"""
return events.get_event(eventId=eventId, calendar_id=calendar_id)
@mcp.tool()
@_with_account
@_snapshot_on_write
def update_event(
eventId: EventId,
summary: EventSummary | None = None,
start: EventDateTime | None = None,
end: EventDateTime | None = None,
description: str | None = None,
location: str | None = None,
calendar_id: CalendarId = "primary",
recurrence: list[RecurrenceRuleString] | None = None,
reminders: EventReminders | None = None,
attendees: list[EventAttendee] | None = None,
creator: CalendarPerson | None = None,
organizer: CalendarPerson | None = None,
extendedProperties: ExtendedProperties | None = None,
source: EventSource | None = None,
transparency: EventTransparency | None = None,
eventType: EventType | None = None,
outOfOfficeProperties: OutOfOfficeProperties | None = None,
focusTimeProperties: FocusTimeProperties | None = None,
workingLocationProperties: WorkingLocationProperties | None = None,
birthdayProperties: BirthdayProperties | None = None,
clear_fields: list[EventClearField] | None = None,
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Updates an existing event.
Args:
eventId: ID of the event to update
summary: New event title
start: New start time object with either 'dateTime' (ISO format) for timed events,
or 'date' (YYYY-MM-DD) for all-day events. Optionally include 'timeZone'.
end: New end time object with either 'dateTime' or 'date' (same format as start).
description: New event description
location: New event location
calendar_id: Calendar containing the event (default 'primary')
recurrence: RRULE strings for recurring events (pass empty list to remove recurrence)
reminders: Reminder overrides (pass {"useDefault": true} to reset to defaults)
attendees: Replace attendee list (pass empty list to remove all attendees)
transparency: Whether the event blocks availability. 'opaque' is busy; 'transparent' is free.
eventType: Event type to set. Switching types clears properties that only belong to the old type.
clear_fields: Optional event fields to remove, e.g. ['source', 'transparency'].
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
The updated event object
"""
return events.update_event(
eventId=eventId,
summary=summary,
start=start,
end=end,
description=description,
location=location,
calendar_id=calendar_id,
recurrence=recurrence,
reminders=reminders,
attendees=attendees,
creator=creator,
organizer=organizer,
extendedProperties=extendedProperties,
source=source,
transparency=transparency,
eventType=eventType,
outOfOfficeProperties=outOfOfficeProperties,
focusTimeProperties=focusTimeProperties,
workingLocationProperties=workingLocationProperties,
birthdayProperties=birthdayProperties,
clear_fields=clear_fields,
)
@mcp.tool()
@_with_account
@_snapshot_on_write
def delete_event(
eventId: EventId,
calendar_id: CalendarId = "primary",
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Deletes an event from the calendar.
Args:
eventId: ID of the event to delete
calendar_id: Calendar containing the event (default 'primary')
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
Confirmation of deletion
"""
return events.delete_event(eventId=eventId, calendar_id=calendar_id)
@mcp.tool()
@_with_account
@_snapshot_on_write
def respond_to_event(
eventId: EventId,
email: EmailStr,
response: AttendeeResponseStatus,
calendar_id: CalendarId = "primary",
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Update an attendee's RSVP response on an event.
Args:
eventId: ID of the event
email: Email address of the attendee responding
response: Response status — 'accepted', 'declined', 'tentative', or 'needsAction'
calendar_id: Calendar containing the event (default 'primary')
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
The updated event with attendee responses
"""
return events.respond_to_event(eventId=eventId, email=email, response=response, calendar_id=calendar_id)
@mcp.tool()
@_with_account
@_snapshot_on_write
def create_calendar(
summary: CalendarSummary,
description: str | None = None,
timeZone: CalendarTimeZone | None = None,
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Creates a new calendar (e.g., 'Personal', 'Team Meetings', 'On-Call').
Args:
summary: Calendar name
description: Calendar description
timeZone: IANA time zone for the calendar. Defaults to the primary calendar time zone.
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
The created calendar object
"""
return calendars.create_calendar(summary=summary, description=description, timeZone=timeZone)
@mcp.tool()
@_with_account
def list_calendars(account_id: CalendarAccountId | None = None) -> dict:
"""
Lists all calendars. Always includes the 'primary' calendar plus any
additional calendars that have been created.
Returns:
List of calendars with their IDs, names, and event counts
"""
return calendars.list_calendars()
@mcp.tool()
@_with_account
def list_events(
timeMin: Rfc3339OffsetDateTimeString,
timeMax: Rfc3339OffsetDateTimeString,
maxResults: ListEventsMaxResults | None = None,
orderBy: SearchEventsOrderBy | None = None,
calendar_id: CalendarId | None = None,
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Lists events within a specified time range.
Args:
timeMin: Lower bound (exclusive) for an event's end time (ISO format).
Events that end after this time are included.
timeMax: Upper bound (exclusive) for an event's start time (ISO format).
Events that start before this time are included.
maxResults: Maximum number of events to return
orderBy: Sort order ('startTime' or 'updated')
calendar_id: Calendar to list events from. If omitted, lists events from all calendars.
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
List of events that overlap the given time range
"""
return search.list_events(
timeMin=timeMin,
timeMax=timeMax,
maxResults=maxResults,
orderBy=orderBy,
calendar_id=calendar_id,
)
@mcp.tool()
@_with_account
def search_events(
query: str = "",
timeMin: Rfc3339OffsetDateTimeString | None = None,
timeMax: Rfc3339OffsetDateTimeString | None = None,
calendar_id: CalendarId | None = None,
attendee_email: EmailStr | None = None,
response_status: AttendeeResponseStatus | None = None,
organizer_email: EmailStr | None = None,
creator_email: EmailStr | None = None,
maxResults: SearchEventsMaxResults | None = None,
orderBy: SearchEventsOrderBy | None = SearchEventsOrderBy.START_TIME,
account_id: CalendarAccountId | None = None,
) -> SearchEventsResponse:
"""
Search calendar events by keyword, phrase, attendee, location, or calendar.
Args:
query: Search query. Bare words are ANDed across summary, description,
location, attendees, and calendar names. Quoted phrases require
exact adjacency. May be empty when attendee_email is provided.
timeMin: Optional lower bound (exclusive) for an event's end time.
timeMax: Optional upper bound (exclusive) for an event's start time.
calendar_id: Calendar to search. If omitted, searches all calendars.
attendee_email: Optional attendee email or display-name substring.
response_status: Optional attendee RSVP filter ('accepted', 'declined',
'tentative', or 'needsAction'). When attendee_email is
provided, applies to that attendee; otherwise matches
any attendee with that status.
organizer_email: Optional organizer email substring.
creator_email: Optional creator email substring.
maxResults: Maximum number of matching events to return. 0 returns no results.
orderBy: Sort order ('startTime', 'updated', or None).
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
Matching events with calendar_id annotated on each result.
"""
return search.search_events(
query=query,
timeMin=timeMin,
timeMax=timeMax,
calendar_id=calendar_id,
attendee_email=attendee_email,
response_status=response_status,
organizer_email=organizer_email,
creator_email=creator_email,
maxResults=maxResults,
orderBy=orderBy,
)
@mcp.tool()
@_with_account
def check_availability(
timeMin: Rfc3339OffsetDateTimeString,
timeMax: Rfc3339OffsetDateTimeString,
duration_minutes: AvailabilityDurationMinutes = 30,
calendar_id: CalendarId | None = None,
attendee_emails: list[EmailStr] | None = None,
account_id: CalendarAccountId | None = None,
) -> dict:
"""
Check availability within a time range. Returns busy periods and free slots
of at least the requested duration.
When attendee_emails is provided, only events where at least one of those
people is an attendee (and hasn't declined) are considered busy. This lets
you find times that work for specific people.
Args:
timeMin: Start of the range to check (ISO format)
timeMax: End of the range to check (ISO format)
duration_minutes: Minimum free slot duration in minutes (default 30)
calendar_id: Calendar to check. If omitted, checks all calendars.
attendee_emails: Only consider events involving these attendees (optional)
account_id: Calendar account to use in multi-account worlds. Defaults to the active account.
Returns:
Busy periods and available free slots within the range
"""
return availability.check_availability(
timeMin=timeMin,
timeMax=timeMax,
duration_minutes=duration_minutes,
calendar_id=calendar_id,
attendee_emails=attendee_emails,
)
docker/packages/google_calendar/google_calendar/state.py
+478
View File
@@ -0,0 +1,478 @@
"""State loading, saving, and storage helpers for the Google Calendar mock."""
import json
import os
import uuid
from copy import deepcopy
from pathlib import Path
from typing import Any, cast
from pydantic import ValidationError
from .models import DEFAULT_CALENDAR_TIME_ZONE, CalendarAccountsState, CalendarState
from .utils import get_calendar_data_path
SERVICE_NAME = "google_calendar"
_UNSET = object()
# Will be set by argparse before server starts.
_AGENT_WORKSPACE_ARG: str | None = None
# Cached data file path (lazy initialization). ``None`` means in-memory only.
_DATA_FILE: Path | None = None
_final_path: Path | None = None
_bundle_state_path: Path | None = None
_active_account_id: str = "default"
_accounts: dict[str, dict[str, Any]] = {}
def resolve_bundle_state_paths() -> list[Path]:
"""Resolve the seed-state files inside this service's bundle subdir.
The folder ``<BUNDLEDIR>/services/<name>/`` is the unit: everything in it
is this service's seed. Prefer the canonical single-file ``state.json``
(the output round-trip shape); otherwise hand back ALL ``*.json`` in the
folder (the raw entities layout), coalesced by the loader.
"""
bundle_dir = os.environ.get("BUNDLEDIR")
if not bundle_dir:
return []
service_dir = Path(bundle_dir) / "services" / SERVICE_NAME
state_file = service_dir / "state.json"
if state_file.is_file():
return [state_file]
if service_dir.is_dir():
return sorted(service_dir.glob("*.json"))
return []
def resolve_bundle_state_path() -> Path | None:
"""Back-compat single-file view of :func:`resolve_bundle_state_paths`."""
paths = resolve_bundle_state_paths()
return paths[0] if paths else None
def _merge_flat_into(target: dict[str, Any], source: dict[str, Any]) -> None:
"""Merge a flat account seed into ``target``: dicts update, lists extend, scalars overwrite."""
for key, value in source.items():
if isinstance(value, dict) and isinstance(target.get(key), dict):
target[key].update(value)
elif isinstance(value, list) and isinstance(target.get(key), list):
target[key].extend(value)
else:
target[key] = value
def _coalesce_account_files(paths: list[Path]) -> dict[str, Any] | None:
"""Coalesce a folder of seed files into one seed dict.
A single file passes through unchanged (flat single-account or ``{accounts:
{...}}`` wrapper). With multiple files, flat (non-wrapper) files are merged
into ONE ``default`` account — the raw entities layout splits a single
account across per-entity files (``events.json`` + ``calendars.json``), and
those halves belong together, not in separate accounts. Files carrying an
explicit ``{accounts: {...}}`` wrapper contribute their named accounts.
"""
if not paths:
return None
if len(paths) == 1:
return cast(dict[str, Any], json.loads(paths[0].read_text()))
accounts: dict[str, Any] = {}
default_account: dict[str, Any] = {}
for path in paths:
data = json.loads(path.read_text())
if isinstance(data, dict) and "accounts" in data:
accounts.update(data["accounts"])
elif isinstance(data, dict):
_merge_flat_into(default_account, data)
if default_account:
# Flat files form the default account unless a wrapper already named one.
accounts.setdefault("default", default_account)
return {"accounts": accounts}
def resolve_bundle_output_path() -> Path | None:
output_dir = os.environ.get("BUNDLE_OUTPUT_DIR")
if not output_dir:
return None
return Path(output_dir) / "state.json"
def set_agent_workspace(workspace: str | None) -> None:
"""Set the agent workspace used to resolve the calendar data file."""
global _AGENT_WORKSPACE_ARG, _DATA_FILE, _active_account_id
_AGENT_WORKSPACE_ARG = workspace
_DATA_FILE = None
_active_account_id = "default"
_accounts.clear()
def get_data_file_path() -> Path | None:
"""
Get the data file path. Checks in order:
1. --agent-workspace CLI argument (computes path via shared function)
2. AGENT_WORKSPACE environment variable
3. None for in-memory local testing
"""
global _AGENT_WORKSPACE_ARG
# Check CLI arg first (set by argparse before mcp.run())
if _AGENT_WORKSPACE_ARG:
return get_calendar_data_path(_AGENT_WORKSPACE_ARG)
# Check env var
env_workspace = os.environ.get("AGENT_WORKSPACE")
if env_workspace:
return get_calendar_data_path(env_workspace)
return None
def get_data_file() -> Path | None:
"""Get cached data file path, initializing on first call."""
global _DATA_FILE
if _DATA_FILE is None:
_DATA_FILE = get_data_file_path()
return _DATA_FILE
def set_snapshot_paths(
*,
final_path: Path | str | None | object = _UNSET,
bundle_state_path: Path | str | None | object = _UNSET,
) -> None:
"""Set snapshot output paths.
Omitted keyword arguments leave the corresponding path unchanged. Passing
``None`` explicitly clears that path.
"""
global _final_path, _bundle_state_path
if final_path is not _UNSET:
_final_path = Path(cast(str | Path, final_path)) if final_path is not None else None
if bundle_state_path is not _UNSET:
_bundle_state_path = Path(cast(str | Path, bundle_state_path)) if bundle_state_path is not None else None
def get_final_path() -> Path | None:
return _final_path
def get_bundle_state_path() -> Path | None:
return _bundle_state_path
def dump_state(dest: Path, label: str) -> None:
"""Write a JSON snapshot of the current calendar state to *dest*."""
try:
data = state_to_json()
dest.parent.mkdir(parents=True, exist_ok=True)
with open(dest, "w") as f:
json.dump(data, f, indent=2)
print(f"Wrote {label} state to {dest}")
except Exception as e:
print(f"Error writing {label} state to {dest}: {e}")
def write_snapshots() -> None:
"""Write configured post-tool snapshots."""
if _bundle_state_path is not None:
dump_state(_bundle_state_path, "bundle")
if _final_path is not None:
dump_state(_final_path, "final")
def configure_snapshots_from_env(*, write_initial: bool = False) -> None:
"""Configure snapshot paths from OUTPUTDIR/BUNDLE_OUTPUT_DIR."""
outputdir = os.environ.get("OUTPUTDIR")
set_snapshot_paths(
bundle_state_path=resolve_bundle_output_path(),
final_path=(Path(outputdir) / "final.json") if outputdir else None,
)
if write_initial:
if _bundle_state_path is not None:
dump_state(_bundle_state_path, "bundle")
if outputdir:
dump_state(Path(outputdir) / "initial.json", "initial")
def load_seed_state_from_env() -> None:
"""Load startup seed state from BUNDLEDIR or INPUTDIR, if present."""
# The bundle folder is read in full and coalesced (a lone state.json is a
# one-element list); otherwise fall back to the first INPUTDIR/*.json.
bundle_paths = resolve_bundle_state_paths()
inputdir = os.environ.get("INPUTDIR")
if bundle_paths:
try:
seed = _coalesce_account_files(bundle_paths)
assert seed is not None
state_from_json(seed)
print(f"Loaded state from {[str(p) for p in bundle_paths]}")
except Exception as e:
print(f"Warning: failed to load {[str(p) for p in bundle_paths]}: {e}")
return
if inputdir and Path(inputdir).is_dir():
json_files = sorted(Path(inputdir).glob("*.json"))
if json_files:
try:
with open(json_files[0]) as f:
state_from_json(json.load(f))
print(f"Loaded state from {json_files[0]}")
except Exception as e:
print(f"Warning: failed to load {json_files[0]}: {e}")
def init_state(agent_workspace: str | None = None, *, write_initial_snapshot: bool = True) -> None:
"""Initialize calendar state paths, seed data, and snapshot outputs."""
if agent_workspace:
set_agent_workspace(agent_workspace)
print(f"BUNDLEDIR={os.environ.get('BUNDLEDIR', '<unset>')}")
print(f"INPUTDIR={os.environ.get('INPUTDIR', '<unset>')}")
print(f"OUTPUTDIR={os.environ.get('OUTPUTDIR', '<unset>')}")
load_seed_state_from_env()
configure_snapshots_from_env(write_initial=write_initial_snapshot)
def _read_storage_data() -> dict[str, Any]:
data_file = get_data_file()
if data_file is not None and data_file.exists():
with open(data_file) as f:
return json.load(f)
return {"events": {}}
def _accounts_from_storage(
data: dict[str, Any] | CalendarState | CalendarAccountsState,
*,
validate: bool = False,
) -> dict[str, dict[str, Any]]:
if isinstance(data, (CalendarAccountsState, CalendarState)):
data = data.model_dump(mode="json", exclude_none=True)
if "accounts" in data:
if validate:
accounts = CalendarAccountsState.model_validate(data).accounts
return {
account_id: account.model_dump(mode="json", exclude_none=True)
for account_id, account in accounts.items()
}
return {account_id: deepcopy(account) for account_id, account in data["accounts"].items()}
if validate:
account = CalendarState.model_validate(data)
return {"default": account.model_dump(mode="json", exclude_none=True)}
return {"default": deepcopy(data)}
def _storage_from_accounts(accounts: dict[str, dict[str, Any]]) -> dict[str, Any]:
if len(accounts) == 1 and "default" in accounts:
return accounts["default"]
return {"accounts": accounts}
def _write_storage(accounts: dict[str, dict[str, Any]]) -> None:
data_file = get_data_file()
if data_file is None:
return
with open(data_file, "w") as f:
json.dump(_storage_from_accounts(accounts), f, indent=2)
def _ensure_loaded() -> None:
global _active_account_id
if _accounts:
return
_accounts.update(_accounts_from_storage(_read_storage_data()))
_active_account_id = "default" if "default" in _accounts else next(iter(_accounts), "default")
def set_active_account(account_id: str) -> None:
"""Select the account used by subsequent tool handlers."""
global _active_account_id
_ensure_loaded()
if account_id not in _accounts:
available = ", ".join(sorted(_accounts.keys()))
raise ValueError(f"Calendar account '{account_id}' not found. Available: {available}")
_active_account_id = account_id
def get_active_account_id() -> str:
return _active_account_id
def get_default_account_id() -> str:
"""Return the deterministic default account for viewer reads."""
_ensure_loaded()
if "default" in _accounts:
return "default"
return next(iter(_accounts), _active_account_id)
def list_accounts() -> list[dict[str, Any]]:
"""List available isolated calendar accounts."""
_ensure_loaded()
result = []
for account_id, account in _accounts.items():
calendars = get_all_calendars(account)
result.append(
{
"account_id": account_id,
"calendar_count": len(calendars),
"event_count": sum(calendar["eventCount"] for calendar in calendars),
"timeZone": account.get("timeZone", DEFAULT_CALENDAR_TIME_ZONE),
}
)
return result
def load_data(account_id: str | None = None) -> dict:
"""Return the in-memory calendar data for the selected account.
INPUTDIR seeding happens once at startup via state_from_json() in
__main__; after that the in-memory account registry is the canonical
state and the data file is its persisted serialization.
"""
_ensure_loaded()
selected_account = account_id or _active_account_id
if selected_account not in _accounts:
available = ", ".join(sorted(_accounts.keys()))
raise ValueError(f"Calendar account '{selected_account}' not found. Available: {available}")
return _accounts[selected_account]
def save_data(data: dict, account_id: str | None = None) -> None:
"""Validate and save calendar data for the selected account to JSON file."""
_ensure_loaded()
selected_account = account_id or _active_account_id
validated = CalendarState.model_validate(data).model_dump(mode="json", exclude_none=True)
if selected_account not in _accounts:
available = ", ".join(sorted(_accounts.keys()))
raise ValueError(f"Calendar account '{selected_account}' not found. Available: {available}")
_accounts[selected_account] = validated
_write_storage(_accounts)
def save_data_or_error(data: dict) -> dict | None:
try:
save_data(data)
except ValidationError as e:
return {"status": "error", "message": str(e)}
return None
def generate_event_id() -> str:
"""Generate a unique event ID."""
return str(uuid.uuid4())[:8]
# ---------------------------------------------------------------------------
# Multi-Calendar Helpers
# ---------------------------------------------------------------------------
def get_calendar_events(data: dict, calendar_id: str) -> dict:
"""Get the events dict for a calendar. Handles backward compatibility:
- If data has 'calendars' dict, looks up by calendar_id
- If data only has flat 'events' dict, treats it as 'primary'
Returns the events dict (mutable reference into data).
"""
if "calendars" in data and calendar_id in data["calendars"]:
return data["calendars"][calendar_id].setdefault("events", {})
# Backward compat: flat 'events' dict is the primary calendar
if calendar_id == "primary":
return data.setdefault("events", {})
# Calendar doesn't exist
return {}
def set_calendar_event(data: dict, calendar_id: str, event_id: str, event: dict) -> bool:
"""Set an event in a calendar. Returns False if calendar doesn't exist."""
if "calendars" in data and calendar_id in data["calendars"]:
if "events" not in data["calendars"][calendar_id]:
data["calendars"][calendar_id]["events"] = {}
data["calendars"][calendar_id]["events"][event_id] = event
return True
if calendar_id == "primary":
if "events" not in data:
data["events"] = {}
data["events"][event_id] = event
return True
return False
def delete_calendar_event(data: dict, calendar_id: str, event_id: str) -> dict | None:
"""Delete an event from a calendar. Returns the deleted event or None."""
events = get_calendar_events(data, calendar_id)
if event_id in events:
return events.pop(event_id)
return None
def get_all_calendars(data: dict) -> list[dict]:
"""Get list of all calendars with metadata."""
calendars = []
# Always include primary
primary_events = data.get("events", {})
primary_time_zone = data.get("timeZone", DEFAULT_CALENDAR_TIME_ZONE)
calendars.append(
{
"id": "primary",
"summary": "Primary",
"description": "Default calendar",
"timeZone": primary_time_zone,
"primary": True,
"eventCount": len(primary_events),
}
)
# Add any explicit calendars (except primary which we already handled)
for cal_id, cal_data in data.get("calendars", {}).items():
if cal_id == "primary":
continue
calendars.append(
{
"id": cal_id,
"summary": cal_data.get("summary", cal_id),
"description": cal_data.get("description", ""),
"timeZone": cal_data.get("timeZone", primary_time_zone),
"primary": False,
"eventCount": len(cal_data.get("events", {})),
}
)
return calendars
# ---------------------------------------------------------------------------
# State Management Tools
# ---------------------------------------------------------------------------
def state_to_json() -> dict:
"""Return the full calendar state as a JSON-native dict.
Round-trips with state_from_json. Emits both legacy flat ``events`` shape
and the multi-calendar ``calendars`` dict when present, so multi-calendar
worlds survive an export/import cycle.
"""
_ensure_loaded()
return deepcopy(_storage_from_accounts(_accounts))
def state_from_json(data: dict[str, Any] | CalendarState | CalendarAccountsState) -> None:
"""Full-replace the calendar state from a JSON-native dict."""
global _active_account_id
accounts = _accounts_from_storage(data, validate=True)
_accounts.clear()
_accounts.update(accounts)
_write_storage(_accounts)
_active_account_id = "default" if "default" in _accounts else next(iter(_accounts), "default")
docker/packages/google_calendar/google_calendar/tools/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Tool implementation modules for the Google Calendar mock."""
docker/packages/google_calendar/google_calendar/tools/availability.py
+174
View File
@@ -0,0 +1,174 @@
"""Availability tool implementation for the Google Calendar mock."""
from datetime import datetime, timedelta
from pydantic import EmailStr
from google_calendar.models import (
AvailabilityDurationMinutes,
CalendarId,
EventTransparency,
Rfc3339OffsetDateTimeString,
)
from google_calendar.state import get_calendar_events, load_data
from google_calendar.tools.search import _expand_recurring_event, _parse_event_end, _parse_event_start
def check_availability(
timeMin: Rfc3339OffsetDateTimeString,
timeMax: Rfc3339OffsetDateTimeString,
duration_minutes: AvailabilityDurationMinutes = 30,
calendar_id: CalendarId | None = None,
attendee_emails: list[EmailStr] | None = None,
) -> dict:
"""
Check availability within a time range. Returns busy periods and free slots
of at least the requested duration.
When attendee_emails is provided, only events where at least one of those
people is an attendee (and hasn't declined) are considered busy. This lets
you find times that work for specific people.
Args:
timeMin: Start of the range to check (ISO format)
timeMax: End of the range to check (ISO format)
duration_minutes: Minimum free slot duration in minutes (default 30)
calendar_id: Calendar to check. If omitted, checks all calendars.
attendee_emails: Only consider events involving these attendees (optional)
Returns:
Busy periods and available free slots within the range
"""
data = load_data()
# Parse boundaries
try:
range_start = datetime.fromisoformat(timeMin)
range_end = datetime.fromisoformat(timeMax)
except ValueError as e:
return {"status": "error", "message": f"Invalid time format: {e}"}
# Normalize timezone awareness
if range_start.tzinfo is None and range_end.tzinfo is not None:
range_start = range_start.replace(tzinfo=range_end.tzinfo)
elif range_start.tzinfo is not None and range_end.tzinfo is None:
range_end = range_end.replace(tzinfo=range_start.tzinfo)
# Collect events from requested calendar(s). Event IDs are scoped to a
# calendar, so all-calendar availability must not merge by event ID.
all_events: list[dict] = []
if calendar_id:
all_events = list(get_calendar_events(data, calendar_id).values())
else:
all_events.extend(data.get("events", {}).values())
for cal_data in data.get("calendars", {}).values():
all_events.extend(cal_data.get("events", {}).values())
# Expand recurring events and collect all event instances
expanded_events: list[dict] = []
for event in all_events:
if event.get("recurrence"):
expanded_events.extend(_expand_recurring_event(event, range_start, range_end))
else:
expanded_events.append(event)
# Filter by attendee emails if specified
if attendee_emails:
emails_lower = {e.lower() for e in attendee_emails}
filtered = []
for event in expanded_events:
event_attendees = event.get("attendees", [])
if not event_attendees:
# Events without attendees are considered relevant (owner's events)
filtered.append(event)
continue
for a in event_attendees:
if a.get("email", "").lower() in emails_lower and a.get("responseStatus") != "declined":
filtered.append(event)
break
expanded_events = filtered
# Find busy periods within the range
busy_periods: list[tuple[datetime, datetime, str]] = []
for event in expanded_events:
if event.get("transparency") == EventTransparency.TRANSPARENT.value:
continue
event_start = _parse_event_start(event)
event_end = _parse_event_end(event)
if event_start is None or event_end is None:
continue
# Normalize timezone
if event_start.tzinfo is None and range_start.tzinfo is not None:
event_start = event_start.replace(tzinfo=range_start.tzinfo)
elif event_start.tzinfo is not None and range_start.tzinfo is None:
event_start = event_start.replace(tzinfo=None)
if event_end.tzinfo is None and range_start.tzinfo is not None:
event_end = event_end.replace(tzinfo=range_start.tzinfo)
elif event_end.tzinfo is not None and range_start.tzinfo is None:
event_end = event_end.replace(tzinfo=None)
# Check overlap with requested range
if event_end > range_start and event_start < range_end:
# Clamp to range
clamped_start = max(event_start, range_start)
clamped_end = min(event_end, range_end)
busy_periods.append((clamped_start, clamped_end, event.get("summary", "")))
# Sort busy periods by start time
busy_periods.sort(key=lambda x: x[0])
# Merge overlapping busy periods
merged: list[tuple[datetime, datetime, list[str]]] = []
for start, end, summary in busy_periods:
if merged and start <= merged[-1][1]:
# Overlapping — extend
merged[-1] = (merged[-1][0], max(merged[-1][1], end), merged[-1][2] + [summary])
else:
merged.append((start, end, [summary]))
# Find free slots
min_duration = timedelta(minutes=duration_minutes)
free_slots: list[dict] = []
cursor = range_start
for busy_start, busy_end, _summaries in merged:
gap = busy_start - cursor
if gap >= min_duration:
free_slots.append(
{
"start": cursor.isoformat(),
"end": busy_start.isoformat(),
"duration_minutes": int(gap.total_seconds() / 60),
}
)
cursor = max(cursor, busy_end)
# Check final gap
final_gap = range_end - cursor
if final_gap >= min_duration:
free_slots.append(
{
"start": cursor.isoformat(),
"end": range_end.isoformat(),
"duration_minutes": int(final_gap.total_seconds() / 60),
}
)
busy_formatted = [
{
"start": s.isoformat(),
"end": e.isoformat(),
"events": summaries,
}
for s, e, summaries in merged
]
return {
"status": "success",
"busy": busy_formatted,
"free_slots": free_slots,
"total_busy_minutes": sum(int((e - s).total_seconds() / 60) for s, e, _ in merged),
"total_free_minutes": sum(slot["duration_minutes"] for slot in free_slots),
}
docker/packages/google_calendar/google_calendar/tools/calendars.py
+78
View File
@@ -0,0 +1,78 @@
"""Calendar metadata tool implementations for the Google Calendar mock."""
from google_calendar.models import DEFAULT_CALENDAR_TIME_ZONE, Calendar, CalendarSummary, CalendarTimeZone
from google_calendar.state import get_all_calendars, load_data, save_data_or_error
def create_calendar(
summary: CalendarSummary,
description: str | None = None,
timeZone: CalendarTimeZone | None = None,
) -> dict:
"""
Creates a new calendar (e.g., 'Personal', 'Team Meetings', 'On-Call').
Args:
summary: Calendar name
description: Calendar description
timeZone: IANA time zone for the calendar. Defaults to the primary calendar time zone.
Returns:
The created calendar object
"""
data = load_data()
# Generate calendar ID from summary
cal_id = summary.lower().replace(" ", "-")
cal_id = "".join(c for c in cal_id if c.isalnum() or c == "-")
cal_id = cal_id.strip("-")
if "calendars" not in data:
data["calendars"] = {}
if not cal_id:
return {
"status": "error",
"message": "Calendar summary must contain at least one letter or number",
}
if cal_id in data["calendars"] or cal_id == "primary":
return {"status": "error", "message": f"Calendar '{cal_id}' already exists"}
calendar = Calendar.model_validate(
{
"summary": summary,
"description": description or "",
"timeZone": timeZone or data.get("timeZone", DEFAULT_CALENDAR_TIME_ZONE),
"events": {},
}
).model_dump(mode="json")
data["calendars"][cal_id] = calendar
if error := save_data_or_error(data):
return error
return {
"status": "success",
"calendar": {
"id": cal_id,
"summary": calendar["summary"],
"description": calendar["description"],
"timeZone": calendar["timeZone"],
"primary": False,
"eventCount": 0,
},
}
def list_calendars() -> dict:
"""
Lists all calendars. Always includes the 'primary' calendar plus any
additional calendars that have been created.
Returns:
List of calendars with their IDs, names, and event counts
"""
data = load_data()
calendars = get_all_calendars(data)
return {"status": "success", "calendars": calendars, "count": len(calendars)}
docker/packages/google_calendar/google_calendar/tools/events.py
+338
View File
@@ -0,0 +1,338 @@
"""Event tool implementations for the Google Calendar mock."""
from datetime import UTC, datetime
from pydantic import EmailStr, ValidationError
from google_calendar.models import (
EVENT_TYPE_PROPERTY_FIELDS,
AttendeeResponseStatus,
BirthdayProperties,
CalendarId,
CalendarPerson,
Event,
EventAttendee,
EventClearField,
EventDateTime,
EventId,
EventReminders,
EventSource,
EventSummary,
EventTransparency,
EventType,
ExtendedProperties,
FocusTimeProperties,
OutOfOfficeProperties,
RecurrenceRuleString,
WorkingLocationProperties,
dump_model,
event_attendee_payload,
)
from google_calendar.state import (
delete_calendar_event,
generate_event_id,
get_calendar_events,
load_data,
save_data_or_error,
set_calendar_event,
)
def create_event(
summary: EventSummary,
start: EventDateTime,
end: EventDateTime,
description: str | None = None,
location: str | None = None,
calendar_id: CalendarId = "primary",
recurrence: list[RecurrenceRuleString] | None = None,
reminders: EventReminders | None = None,
attendees: list[EventAttendee] | None = None,
creator: CalendarPerson | None = None,
organizer: CalendarPerson | None = None,
extendedProperties: ExtendedProperties | None = None,
source: EventSource | None = None,
transparency: EventTransparency | None = None,
eventType: EventType = EventType.DEFAULT,
outOfOfficeProperties: OutOfOfficeProperties | None = None,
focusTimeProperties: FocusTimeProperties | None = None,
workingLocationProperties: WorkingLocationProperties | None = None,
birthdayProperties: BirthdayProperties | None = None,
) -> dict:
"""
Creates a new event in Google Calendar.
Args:
summary: Event title
start: Start time object with either 'dateTime' (ISO format, e.g. '2025-12-10T09:00:00-05:00')
for timed events, or 'date' (YYYY-MM-DD, e.g. '2025-12-10') for all-day events.
Optionally include 'timeZone' (e.g. 'America/New_York').
end: End time object with either 'dateTime' or 'date' (same format as start).
For all-day events, 'date' is exclusive (e.g. end '2025-12-11' means the event ends on 2025-12-10).
description: Event description
location: Event location
calendar_id: Calendar to create the event in (default 'primary')
recurrence: RRULE strings for recurring events (e.g., ['RRULE:FREQ=WEEKLY;BYDAY=TU;COUNT=10'])
reminders: Reminder overrides, e.g. {"useDefault": false, "overrides": [{"method": "popup", "minutes": 15}]}
attendees: List of attendees, e.g. [{"email": "alice@co.com", "displayName": "Alice"}]. responseStatus defaults to 'needsAction'.
transparency: Whether the event blocks availability. 'opaque' is busy; 'transparent' is free.
Returns:
The created event object with its ID
"""
data = load_data()
event_id = generate_event_id()
now = datetime.now(UTC).isoformat().replace("+00:00", "Z")
event: dict = {
"id": event_id,
"summary": summary,
"start": dump_model(start, EventDateTime),
"end": dump_model(end, EventDateTime),
"created": now,
"updated": now,
"eventType": EventType(eventType).value,
}
if description:
event["description"] = description
if location:
event["location"] = location
if recurrence:
event["recurrence"] = recurrence
if reminders:
event["reminders"] = dump_model(reminders, EventReminders)
if attendees:
event["attendees"] = [event_attendee_payload(attendee) for attendee in attendees]
if creator is not None:
event["creator"] = dump_model(creator, CalendarPerson)
if organizer is not None:
event["organizer"] = dump_model(organizer, CalendarPerson)
if extendedProperties is not None:
event["extendedProperties"] = dump_model(extendedProperties, ExtendedProperties)
if source is not None:
event["source"] = dump_model(source, EventSource)
if transparency is not None:
event["transparency"] = EventTransparency(transparency).value
if outOfOfficeProperties is not None:
event["outOfOfficeProperties"] = dump_model(outOfOfficeProperties, OutOfOfficeProperties)
if focusTimeProperties is not None:
event["focusTimeProperties"] = dump_model(focusTimeProperties, FocusTimeProperties)
if workingLocationProperties is not None:
event["workingLocationProperties"] = dump_model(workingLocationProperties, WorkingLocationProperties)
if birthdayProperties is not None:
event["birthdayProperties"] = dump_model(birthdayProperties, BirthdayProperties)
try:
event = Event.model_validate(event).model_dump(exclude_none=True)
except ValidationError as e:
return {"status": "error", "message": str(e)}
if not set_calendar_event(data, calendar_id, event_id, event):
return {"status": "error", "message": f"Calendar '{calendar_id}' not found"}
if error := save_data_or_error(data):
return error
return {"status": "success", "event": event}
def get_event(eventId: EventId, calendar_id: CalendarId = "primary") -> dict:
"""
Retrieves details of a specific event.
Args:
eventId: ID of the event to retrieve
calendar_id: Calendar to look in (default 'primary')
Returns:
The event object if found
"""
data = load_data()
events = get_calendar_events(data, calendar_id)
if eventId not in events:
return {"status": "error", "message": f"Event with ID '{eventId}' not found in calendar '{calendar_id}'"}
return {"status": "success", "event": events[eventId]}
def update_event(
eventId: EventId,
summary: EventSummary | None = None,
start: EventDateTime | None = None,
end: EventDateTime | None = None,
description: str | None = None,
location: str | None = None,
calendar_id: CalendarId = "primary",
recurrence: list[RecurrenceRuleString] | None = None,
reminders: EventReminders | None = None,
attendees: list[EventAttendee] | None = None,
creator: CalendarPerson | None = None,
organizer: CalendarPerson | None = None,
extendedProperties: ExtendedProperties | None = None,
source: EventSource | None = None,
transparency: EventTransparency | None = None,
eventType: EventType | None = None,
outOfOfficeProperties: OutOfOfficeProperties | None = None,
focusTimeProperties: FocusTimeProperties | None = None,
workingLocationProperties: WorkingLocationProperties | None = None,
birthdayProperties: BirthdayProperties | None = None,
clear_fields: list[EventClearField] | None = None,
) -> dict:
"""
Updates an existing event.
Args:
eventId: ID of the event to update
summary: New event title
start: New start time object with either 'dateTime' (ISO format) for timed events,
or 'date' (YYYY-MM-DD) for all-day events. Optionally include 'timeZone'.
end: New end time object with either 'dateTime' or 'date' (same format as start).
description: New event description
location: New event location
calendar_id: Calendar containing the event (default 'primary')
recurrence: RRULE strings for recurring events (pass empty list to remove recurrence)
reminders: Reminder overrides (pass {"useDefault": true} to reset to defaults)
attendees: Replace attendee list (pass empty list to remove all attendees)
transparency: Whether the event blocks availability. 'opaque' is busy; 'transparent' is free.
eventType: Event type to set. Switching types clears properties that only belong to the old type.
clear_fields: Optional event fields to remove, e.g. ['source', 'transparency'].
Returns:
The updated event object
"""
data = load_data()
events = get_calendar_events(data, calendar_id)
if eventId not in events:
return {"status": "error", "message": f"Event with ID '{eventId}' not found in calendar '{calendar_id}'"}
event = dict(events[eventId])
if clear_fields:
fields_to_clear = {field.value for field in clear_fields}
for field in fields_to_clear:
event.pop(field, None)
if summary is not None:
event["summary"] = summary
if start is not None:
event["start"] = dump_model(start, EventDateTime)
if end is not None:
event["end"] = dump_model(end, EventDateTime)
if description is not None:
event["description"] = description
if location is not None:
event["location"] = location
if recurrence is not None:
if recurrence:
event["recurrence"] = recurrence
else:
event.pop("recurrence", None)
if reminders is not None:
event["reminders"] = dump_model(reminders, EventReminders)
if attendees is not None:
event["attendees"] = [event_attendee_payload(attendee) for attendee in attendees]
if creator is not None:
event["creator"] = dump_model(creator, CalendarPerson)
if organizer is not None:
event["organizer"] = dump_model(organizer, CalendarPerson)
if extendedProperties is not None:
event["extendedProperties"] = dump_model(extendedProperties, ExtendedProperties)
if source is not None:
event["source"] = dump_model(source, EventSource)
if transparency is not None:
event["transparency"] = EventTransparency(transparency).value
if eventType is not None:
event_type = EventType(eventType)
event["eventType"] = event_type.value
for property_event_type, property_field in EVENT_TYPE_PROPERTY_FIELDS.items():
if property_event_type != event_type:
event.pop(property_field, None)
if outOfOfficeProperties is not None:
event["outOfOfficeProperties"] = dump_model(outOfOfficeProperties, OutOfOfficeProperties)
if focusTimeProperties is not None:
event["focusTimeProperties"] = dump_model(focusTimeProperties, FocusTimeProperties)
if workingLocationProperties is not None:
event["workingLocationProperties"] = dump_model(workingLocationProperties, WorkingLocationProperties)
if birthdayProperties is not None:
event["birthdayProperties"] = dump_model(birthdayProperties, BirthdayProperties)
event["updated"] = datetime.now(UTC).isoformat().replace("+00:00", "Z")
try:
event = Event.model_validate(event).model_dump(exclude_none=True)
except ValidationError as e:
return {"status": "error", "message": str(e)}
events[eventId] = event
if error := save_data_or_error(data):
return error
return {"status": "success", "event": event}
def delete_event(eventId: EventId, calendar_id: CalendarId = "primary") -> dict:
"""
Deletes an event from the calendar.
Args:
eventId: ID of the event to delete
calendar_id: Calendar containing the event (default 'primary')
Returns:
Confirmation of deletion
"""
data = load_data()
deleted_event = delete_calendar_event(data, calendar_id, eventId)
if deleted_event is None:
return {"status": "error", "message": f"Event with ID '{eventId}' not found in calendar '{calendar_id}'"}
if error := save_data_or_error(data):
return error
return {"status": "success", "message": f"Event '{deleted_event['summary']}' deleted"}
def respond_to_event(
eventId: EventId,
email: EmailStr,
response: AttendeeResponseStatus,
calendar_id: CalendarId = "primary",
) -> dict:
"""
Update an attendee's RSVP response on an event.
Args:
eventId: ID of the event
email: Email address of the attendee responding
response: Response status — 'accepted', 'declined', 'tentative', or 'needsAction'
calendar_id: Calendar containing the event (default 'primary')
Returns:
The updated event with attendee responses
"""
data = load_data()
events = get_calendar_events(data, calendar_id)
if eventId not in events:
return {"status": "error", "message": f"Event with ID '{eventId}' not found in calendar '{calendar_id}'"}
event = events[eventId]
attendees = event.get("attendees", [])
found = False
for attendee in attendees:
if attendee.get("email", "").lower() == email.lower():
attendee["responseStatus"] = response.value
found = True
break
if not found:
return {"status": "error", "message": f"Attendee '{email}' not found on this event"}
event["updated"] = datetime.now(UTC).isoformat().replace("+00:00", "Z")
if error := save_data_or_error(data):
return error
return {"status": "success", "event": event}
docker/packages/google_calendar/google_calendar/tools/search.py
+618
View File
@@ -0,0 +1,618 @@
"""Search, listing, and recurrence helpers for the Google Calendar mock."""
import re
from datetime import UTC, datetime, timedelta
from typing import cast
from pydantic import EmailStr
from google_calendar.models import (
DEFAULT_RECURRING_SEARCH_DAYS,
AttendeeResponseStatus,
CalendarId,
ListEventsMaxResults,
Rfc3339OffsetDateTimeString,
SearchEventsMaxResults,
SearchEventsOrderBy,
SearchEventsResponse,
parse_rfc3339_datetime,
)
from google_calendar.state import get_calendar_events, load_data
_QUERY_TOKEN_RE = re.compile(r'"([^"]+)"|\S+')
_DAY_MAP = {"MO": 0, "TU": 1, "WE": 2, "TH": 3, "FR": 4, "SA": 5, "SU": 6}
def _parse_rrule(rrule_str: str) -> dict:
"""Parse an RRULE string into a dict of parameters."""
params: dict = {}
rule = rrule_str.replace("RRULE:", "")
for part in rule.split(";"):
if "=" not in part:
continue
key, value = part.split("=", 1)
params[key] = value
return params
def _expand_recurring_event(
event: dict,
range_start: datetime,
range_end: datetime,
) -> list[dict]:
"""Expand a recurring event into instances within a time range.
Supports: FREQ=DAILY|WEEKLY|MONTHLY, INTERVAL, COUNT, UNTIL, BYDAY.
Returns a list of synthetic event dicts (copies with adjusted start/end).
"""
recurrence = event.get("recurrence", [])
if not recurrence:
return []
# Find the RRULE
rrule_str = None
for r in recurrence:
if r.startswith("RRULE:"):
rrule_str = r
break
if not rrule_str:
return []
params = _parse_rrule(rrule_str)
freq = params.get("FREQ", "").upper()
interval = int(params.get("INTERVAL", "1"))
count = int(params.get("COUNT", "0")) or None
until_str = params.get("UNTIL")
byday = params.get("BYDAY", "").split(",") if params.get("BYDAY") else None
# Parse event start/end to get duration
event_start = _parse_event_start(event)
event_end = _parse_event_end(event)
if event_start is None or event_end is None:
return []
duration = event_end - event_start
# Parse UNTIL. Normalize to match event_start's tz-awareness so comparisons
# don't raise when RRULE UNTIL is specified without an offset.
until_dt = None
if until_str:
import contextlib
try:
until_dt = datetime.fromisoformat(until_str)
except ValueError:
with contextlib.suppress(ValueError):
until_dt = datetime.strptime(until_str, "%Y%m%dT%H%M%SZ").replace(tzinfo=event_start.tzinfo)
if until_dt is not None and until_dt.tzinfo is None and event_start.tzinfo is not None:
until_dt = until_dt.replace(tzinfo=event_start.tzinfo)
# Normalize timezone for comparison
if event_start.tzinfo is not None and range_start.tzinfo is None:
range_start = range_start.replace(tzinfo=event_start.tzinfo)
range_end = range_end.replace(tzinfo=event_start.tzinfo)
elif event_start.tzinfo is None and range_start.tzinfo is not None:
event_start = event_start.replace(tzinfo=range_start.tzinfo)
# Generate instances
instances = []
current = event_start
instance_count = 0
max_iterations = 1000 # safety limit
for _ in range(max_iterations):
if until_dt and current > until_dt:
break
if count and instance_count >= count:
break
if current >= range_end:
break
current_end = current + duration
# Check if this instance matches BYDAY (for WEEKLY)
day_match = True
if byday and freq == "WEEKLY":
day_abbr = ["MO", "TU", "WE", "TH", "FR", "SA", "SU"][current.weekday()]
day_match = day_abbr in byday
if day_match:
instance_count += 1
# Only include if it overlaps the query range
if current_end > range_start and current < range_end:
instance = dict(event)
instance["id"] = f"{event['id']}_{instance_count}"
instance["recurringEventId"] = event["id"]
if "dateTime" in event.get("start", {}):
instance["start"] = {"dateTime": current.isoformat()}
instance["end"] = {"dateTime": current_end.isoformat()}
if time_zone := event.get("start", {}).get("timeZone"):
instance["start"]["timeZone"] = time_zone
if time_zone := event.get("end", {}).get("timeZone"):
instance["end"]["timeZone"] = time_zone
else:
instance["start"] = {"date": current.strftime("%Y-%m-%d")}
instance["end"] = {"date": current_end.strftime("%Y-%m-%d")}
# Don't include recurrence on instances
instance.pop("recurrence", None)
instances.append(instance)
# Advance to next occurrence
if freq == "DAILY":
current += timedelta(days=interval)
elif freq == "WEEKLY":
if byday:
# Advance one day at a time for BYDAY matching
current += timedelta(days=1)
else:
current += timedelta(weeks=interval)
elif freq == "MONTHLY":
month = current.month + interval
year = current.year + (month - 1) // 12
month = ((month - 1) % 12) + 1
try:
current = current.replace(year=year, month=month)
except ValueError:
break # e.g., Jan 31 → Feb 31 doesn't exist
else:
break # Unsupported frequency
return instances
def _parse_dt(dt_obj: dict) -> datetime | None:
"""Parse a dateTime or date field from an event start/end object.
Always returns a timezone-aware datetime so comparisons against offset-aware
query bounds don't raise. Offset-less dateTime strings use the event's
timeZone when present. All-day events (date-only) are treated as UTC.
Tolerant of data where dateTime is explicitly null (common in all-day
events that also carry a date field).
"""
dt = dt_obj.get("dateTime")
if isinstance(dt, str):
time_zone = dt_obj.get("timeZone")
if not isinstance(time_zone, str):
time_zone = None
try:
parsed = parse_rfc3339_datetime(dt, time_zone)
except ValueError:
parsed = None
if parsed is not None:
return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
d = dt_obj.get("date")
if isinstance(d, str):
try:
return datetime.strptime(d, "%Y-%m-%d").replace(tzinfo=UTC)
except ValueError:
return None
return None
def _parse_event_start(event: dict) -> datetime | None:
"""Parse event start time from either dateTime or date field."""
return _parse_dt(event.get("start", {}))
def _parse_event_end(event: dict) -> datetime | None:
"""Parse event end time from either dateTime or date field."""
end = _parse_dt(event.get("end", {}))
start_obj = event.get("start", {})
end_obj = event.get("end", {})
if (
end is not None
and isinstance(start_obj, dict)
and isinstance(end_obj, dict)
and "dateTime" not in start_obj
and "dateTime" not in end_obj
and isinstance(start_obj.get("date"), str)
and isinstance(end_obj.get("date"), str)
):
start = _parse_dt(start_obj)
if start is not None and end <= start:
return start + timedelta(days=1)
return end
def _get_event_sort_key(event: dict) -> str:
"""Get a sortable string key for event start time.
dateTime takes precedence, then date. Tolerates events where dateTime is
explicitly null by falling through to date.
"""
start = event.get("start", {})
dt = start.get("dateTime")
if isinstance(dt, str):
return dt
d = start.get("date")
if isinstance(d, str):
return d
return ""
def _parse_search_tokens(query: str) -> list[str]:
"""Parse a query into lower-case word tokens and quoted phrase tokens."""
tokens = []
for match in _QUERY_TOKEN_RE.finditer(query):
token = (match.group(1) or match.group(0)).strip().lower()
if token:
tokens.append(token)
return tokens
def _search_query_warnings(query: str) -> list[str]:
"""Warn when users put unsupported operator syntax in the free-text query."""
warnings: list[str] = []
seen: set[str] = set()
for match in re.finditer(r"\b([A-Za-z_]+):\S+", query):
operator = match.group(1)
if operator.lower() in {"http", "https", "mailto"}:
continue
message = (
f"Unsupported Calendar query operator '{operator}:'; use search_events parameters "
"such as timeMin, timeMax, attendee_email, response_status, organizer_email, or creator_email."
)
if message not in seen:
warnings.append(message)
seen.add(message)
return warnings
def _event_search_haystack(event: dict, calendar_id: str, calendar_summary: str | None = None) -> str:
"""Build searchable event text, using newlines so phrases don't span fields."""
parts = [
event.get("summary", ""),
event.get("description", ""),
event.get("location", ""),
calendar_id,
calendar_summary or "",
]
for person_field in ("organizer", "creator"):
person = event.get(person_field)
if isinstance(person, dict):
parts.extend([person.get("email", ""), person.get("displayName", "")])
elif person:
parts.append(person)
for attendee in event.get("attendees", []) or []:
parts.extend(
[
attendee.get("email", ""),
attendee.get("displayName", ""),
attendee.get("comment", ""),
]
)
return "\n".join(str(part) for part in parts if part).lower()
def _extract_person_emails(value: object) -> list[str]:
if isinstance(value, dict):
person = cast("dict[str, object]", value)
email = person.get("email")
return [str(email).lower()] if email else []
if isinstance(value, str):
return [value.lower()]
return []
def _event_matches_attendee(event: dict, attendee_email: str | None) -> bool:
if not attendee_email:
return True
needle = attendee_email.lower()
for attendee in event.get("attendees", []) or []:
email = str(attendee.get("email", "")).lower()
display_name = str(attendee.get("displayName", "")).lower()
if needle in email or needle in display_name:
return True
return False
def _event_matches_response_status(
event: dict, response_status: AttendeeResponseStatus | None, attendee_email: str | None
) -> bool:
if response_status is None:
return True
needle = response_status.value.lower()
attendee_needle = attendee_email.lower() if attendee_email else None
for attendee in event.get("attendees", []) or []:
if attendee_needle:
email = str(attendee.get("email", "")).lower()
display_name = str(attendee.get("displayName", "")).lower()
if attendee_needle not in email and attendee_needle not in display_name:
continue
if str(attendee.get("responseStatus", "")).lower() == needle:
return True
return False
def _event_matches_creator(event: dict, creator_email: str | None) -> bool:
if not creator_email:
return True
needle = creator_email.lower()
return any(needle in email for email in _extract_person_emails(event.get("creator")))
def _event_matches_organizer(event: dict, organizer_email: str | None) -> bool:
if not organizer_email:
return True
needle = organizer_email.lower()
if any(needle in email for email in _extract_person_emails(event.get("organizer"))):
return True
for attendee in event.get("attendees", []) or []:
if attendee.get("organizer") and needle in str(attendee.get("email", "")).lower():
return True
return False
def _normalize_range_bound(value: str | None) -> datetime | None:
if value is None:
return None
parsed = datetime.fromisoformat(value)
return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
def _event_overlaps_range(event: dict, time_min: datetime | None, time_max: datetime | None) -> bool:
event_start = _parse_event_start(event)
event_end = _parse_event_end(event)
if event_start is None or event_end is None:
return False
if time_min is not None:
if event_end.tzinfo is None and time_min.tzinfo is not None:
event_end = event_end.replace(tzinfo=time_min.tzinfo)
elif event_end.tzinfo is not None and time_min.tzinfo is None:
event_end = event_end.replace(tzinfo=None)
if event_end <= time_min:
return False
if time_max is not None:
if event_start.tzinfo is None and time_max.tzinfo is not None:
event_start = event_start.replace(tzinfo=time_max.tzinfo)
elif event_start.tzinfo is not None and time_max.tzinfo is None:
event_start = event_start.replace(tzinfo=None)
if event_start >= time_max:
return False
return True
def _recurring_search_range(
event: dict, time_min: datetime | None, time_max: datetime | None
) -> tuple[datetime, datetime] | None:
"""Return bounded expansion range for recurring search results."""
event_start = _parse_event_start(event)
if event_start is None:
return None
range_start = time_min or event_start
range_end = time_max or event_start + timedelta(days=DEFAULT_RECURRING_SEARCH_DAYS)
if range_end <= range_start:
return None
return range_start, range_end
def _iter_calendar_events(data: dict, calendar_id: str | None = None) -> list[tuple[str, str, dict]]:
"""Return (calendar_id, calendar_summary, event) tuples for requested calendars."""
if calendar_id:
calendar_summary = (
"Primary"
if calendar_id == "primary"
else data.get("calendars", {}).get(calendar_id, {}).get("summary", calendar_id)
)
return [(calendar_id, calendar_summary, event) for event in get_calendar_events(data, calendar_id).values()]
events = [("primary", "Primary", event) for event in data.get("events", {}).values()]
for cid, cal_data in data.get("calendars", {}).items():
calendar_summary = cal_data.get("summary", cid)
events.extend((cid, calendar_summary, event) for event in cal_data.get("events", {}).values())
return events
def list_events(
timeMin: Rfc3339OffsetDateTimeString,
timeMax: Rfc3339OffsetDateTimeString,
maxResults: ListEventsMaxResults | None = None,
orderBy: SearchEventsOrderBy | None = None,
calendar_id: CalendarId | None = None,
) -> dict:
"""
Lists events within a specified time range.
Args:
timeMin: Lower bound (exclusive) for an event's end time (ISO format).
Events that end after this time are included.
timeMax: Upper bound (exclusive) for an event's start time (ISO format).
Events that start before this time are included.
maxResults: Maximum number of events to return
orderBy: Sort order ('startTime' or 'updated')
calendar_id: Calendar to list events from. If omitted, lists events from all calendars.
Returns:
List of events that overlap the given time range
"""
data = load_data()
# Parse time boundaries
try:
time_min = datetime.fromisoformat(timeMin)
time_max = datetime.fromisoformat(timeMax)
except ValueError as e:
return {"status": "error", "message": f"Invalid time format: {e}"}
# Ensure time_min and time_max have consistent timezone awareness
if time_min.tzinfo is None and time_max.tzinfo is not None:
time_min = time_min.replace(tzinfo=time_max.tzinfo)
elif time_min.tzinfo is not None and time_max.tzinfo is None:
time_max = time_max.replace(tzinfo=time_min.tzinfo)
# Collect events from requested calendar(s). Event IDs are scoped to a
# calendar, so all-calendar queries must not merge by event ID.
all_events: list[dict] = []
if calendar_id:
all_events = list(get_calendar_events(data, calendar_id).values())
else:
all_events.extend(data.get("events", {}).values())
for cal_data in data.get("calendars", {}).values():
all_events.extend(cal_data.get("events", {}).values())
# Expand recurring events and filter by time range
filtered_events = []
skipped_ids = []
for event in all_events:
# Expand recurring events into instances
if event.get("recurrence"):
instances = _expand_recurring_event(event, time_min, time_max)
filtered_events.extend(instances)
continue
event_start = _parse_event_start(event)
event_end = _parse_event_end(event)
if event_start is None or event_end is None:
skipped_ids.append(event.get("id", "unknown"))
continue
# Normalize timezone awareness for comparison
if event_start.tzinfo is None and time_min.tzinfo is not None:
event_start = event_start.replace(tzinfo=time_min.tzinfo)
elif event_start.tzinfo is not None and time_min.tzinfo is None:
event_start = event_start.replace(tzinfo=None)
if event_end.tzinfo is None and time_min.tzinfo is not None:
event_end = event_end.replace(tzinfo=time_min.tzinfo)
elif event_end.tzinfo is not None and time_min.tzinfo is None:
event_end = event_end.replace(tzinfo=None)
# Overlap: event ends after timeMin AND event starts before timeMax
if event_end > time_min and event_start < time_max:
filtered_events.append(event)
# Sort events
if orderBy == SearchEventsOrderBy.START_TIME:
filtered_events.sort(key=_get_event_sort_key)
elif orderBy == SearchEventsOrderBy.UPDATED:
filtered_events.sort(key=lambda e: e.get("updated", ""), reverse=True)
# Limit results
if maxResults is not None:
filtered_events = filtered_events[:maxResults]
result = {
"status": "success",
"events": filtered_events,
"count": len(filtered_events),
}
if skipped_ids:
result["skipped_unparseable"] = skipped_ids
return result
def search_events(
query: str = "",
timeMin: Rfc3339OffsetDateTimeString | None = None,
timeMax: Rfc3339OffsetDateTimeString | None = None,
calendar_id: CalendarId | None = None,
attendee_email: EmailStr | None = None,
response_status: AttendeeResponseStatus | None = None,
organizer_email: EmailStr | None = None,
creator_email: EmailStr | None = None,
maxResults: SearchEventsMaxResults | None = None,
orderBy: SearchEventsOrderBy | None = SearchEventsOrderBy.START_TIME,
) -> SearchEventsResponse:
"""
Search calendar events by keyword, phrase, attendee, location, or calendar.
Args:
query: Search query. Bare words are ANDed across summary, description,
location, attendees, and calendar names. Quoted phrases require
exact adjacency. May be empty when attendee_email is provided.
timeMin: Optional lower bound (exclusive) for an event's end time.
timeMax: Optional upper bound (exclusive) for an event's start time.
calendar_id: Calendar to search. If omitted, searches all calendars.
attendee_email: Optional attendee email or display-name substring.
response_status: Optional attendee RSVP filter ('accepted', 'declined',
'tentative', or 'needsAction'). When attendee_email is
provided, applies to that attendee; otherwise matches
any attendee with that status.
organizer_email: Optional organizer email substring.
creator_email: Optional creator email substring.
maxResults: Maximum number of matching events to return. 0 returns no results.
orderBy: Sort order ('startTime', 'updated', or None).
Returns:
Matching events with calendar_id annotated on each result.
"""
tokens = _parse_search_tokens(query)
warnings = _search_query_warnings(query)
if not any([tokens, attendee_email, response_status, organizer_email, creator_email]):
return {
"status": "error",
"message": "query or at least one filter is required",
"events": [],
"count": 0,
"warnings": warnings,
}
try:
time_min = _normalize_range_bound(timeMin)
time_max = _normalize_range_bound(timeMax)
except ValueError as e:
return {
"status": "error",
"message": f"Invalid time format: {e}",
"events": [],
"count": 0,
"warnings": warnings,
}
data = load_data()
matching_events = []
skipped_ids = []
for cid, calendar_summary, event in _iter_calendar_events(data, calendar_id):
candidates = [event]
if event.get("recurrence"):
recurring_range = _recurring_search_range(event, time_min, time_max)
if recurring_range is None:
skipped_ids.append(event.get("id", "unknown"))
continue
candidates = _expand_recurring_event(event, recurring_range[0], recurring_range[1])
for candidate in candidates:
if (time_min is not None or time_max is not None) and not _event_overlaps_range(
candidate, time_min, time_max
):
if _parse_event_start(candidate) is None or _parse_event_end(candidate) is None:
skipped_ids.append(candidate.get("id", "unknown"))
continue
if not _event_matches_attendee(candidate, attendee_email):
continue
if not _event_matches_response_status(candidate, response_status, attendee_email):
continue
if not _event_matches_organizer(candidate, organizer_email):
continue
if not _event_matches_creator(candidate, creator_email):
continue
haystack = _event_search_haystack(candidate, cid, calendar_summary)
if tokens and not all(token in haystack for token in tokens):
continue
result_event = dict(candidate)
result_event["calendar_id"] = cid
result_event["calendar_summary"] = calendar_summary
matching_events.append(result_event)
if orderBy == SearchEventsOrderBy.START_TIME:
matching_events.sort(key=_get_event_sort_key)
elif orderBy == SearchEventsOrderBy.UPDATED:
matching_events.sort(key=lambda e: e.get("updated", ""), reverse=True)
if maxResults is not None:
matching_events = matching_events[:maxResults]
result: SearchEventsResponse = {"status": "success", "events": matching_events, "count": len(matching_events)}
result["warnings"] = warnings
if skipped_ids:
result["skipped_unparseable"] = skipped_ids
result["warnings"].append(
"Some recurring or malformed events were skipped because they could not be expanded or parsed."
)
return result
docker/packages/google_calendar/google_calendar/utils.py
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env python3
"""
Utilities for setting up calendar data from JSON files.
Used by task preprocess scripts to initialize calendar state.
"""
from __future__ import annotations
import json
from datetime import UTC, datetime
from pathlib import Path
def get_calendar_data_path(agent_workspace: str | Path) -> Path:
"""
Get the calendar data file path for a given workspace.
Stores in external_services/ directory NEXT TO the workspace:
- If workspace is at /workspace/dumps/workspace, stores at /workspace/dumps/external_services/
- Outside the agent workspace so it can't be read directly via filesystem MCP
- Inside the dumps mount so it persists to the host
- Path is deterministic and can be computed by preprocess, MCP, and evaluation
"""
workspace_path = Path(agent_workspace)
# Go up one level from workspace and create external_services directory
external_services_dir = workspace_path.parent / "external_services"
return external_services_dir / "calendar_data.json"
def load_events_from_json(json_path: Path) -> dict:
"""
Load calendar data from a JSON file.
Expected format:
{
"events": {
"event-id": {
"id": "event-id",
"summary": "Event Title",
"start": { "dateTime": "2025-12-10T11:30:00-05:00", "timeZone": "America/New_York" },
"end": { "dateTime": "2025-12-10T12:30:00-05:00", "timeZone": "America/New_York" },
"description": "Optional description",
"location": "Optional location"
}
}
}
Args:
json_path: Path to the JSON file
Returns:
Calendar data dictionary with events
"""
with open(json_path) as f:
data = json.load(f)
# Ensure we have the events key
if "events" not in data:
data = {"events": data if isinstance(data, dict) else {}}
# Add created/updated timestamps if missing
now_utc = datetime.now(UTC).isoformat()
for event_id, event in data["events"].items():
if "id" not in event:
event["id"] = event_id
if "created" not in event:
event["created"] = now_utc
if "updated" not in event:
event["updated"] = now_utc
return data
def create_calendar_data(
agent_workspace: str,
json_path: Path,
verbose: bool = True,
) -> Path:
"""
Create calendar data from a JSON file and store in external_services.
Uses a deterministic file path based on the workspace, so the MCP
server can find it without needing a marker file in the agent workspace.
Args:
agent_workspace: Path to the agent's workspace directory
json_path: Path to the JSON file with events (google_calendar.json)
verbose: Print progress messages
Returns:
Path to the created data file
"""
if not json_path.exists():
if verbose:
print(f"⚠️ No {json_path.name} found at {json_path}, creating empty calendar")
calendar_data = {"events": {}}
else:
calendar_data = load_events_from_json(json_path)
if verbose:
print(f"✅ Loaded {len(calendar_data['events'])} events from {json_path}")
# Deterministic path based on workspace (allows parallel runs)
data_path = get_calendar_data_path(agent_workspace)
# Create parent directory if needed
data_path.parent.mkdir(parents=True, exist_ok=True)
# Write calendar data to external_services directory (outside agent workspace)
with open(data_path, "w") as f:
json.dump(calendar_data, f, indent=2)
if verbose:
print(f"✅ Created {data_path} with {len(calendar_data['events'])} events")
# Print summary
if verbose:
for event in calendar_data["events"].values():
start_time = event.get("start", {}).get("dateTime", "unknown")
print(f" - {event.get('summary', 'Untitled')}: {start_time}")
return data_path
docker/packages/google_calendar/google_calendar/viewer.py
+702
View File
@@ -0,0 +1,702 @@
"""Calendar viewer — read-only calendar UI and API endpoints.
Serves:
GET /api/events — event list (optional ?date=YYYY-MM-DD for week filter)
GET /api/events/:id — single event detail
GET /api/stats — calendar stats
GET / — viewer HTML (single-page app)
All non-MCP routes require the X-Proxy-Token header.
"""
from __future__ import annotations
import os
from datetime import UTC, datetime, timedelta
from typing import Any
import uvicorn
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response
from starlette.routing import Route
# ---------------------------------------------------------------------------
# Auth middleware
# ---------------------------------------------------------------------------
class ProxyTokenMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
# Skip auth for MCP endpoint
if request.url.path.startswith("/mcp"):
return await call_next(request)
token = os.environ.get("MCP_PROXY_TOKEN", "")
if token and request.headers.get("x-proxy-token") != token:
return Response("Forbidden: invalid proxy token", status_code=403)
return await call_next(request)
# ---------------------------------------------------------------------------
# Data access
# ---------------------------------------------------------------------------
def _get_events() -> dict[str, Any]:
"""Load and return all calendar events keyed uniquely for viewer lookups."""
from .state import get_default_account_id, load_data
data = load_data(account_id=get_default_account_id())
events: dict[str, Any] = {}
for event_id, event in data.get("events", {}).items():
events[event_id] = {**event, "calendar_id": "primary", "lookup_id": event_id}
for calendar_id, calendar in data.get("calendars", {}).items():
for event_id, event in calendar.get("events", {}).items():
lookup_id = f"{calendar_id}:{event_id}"
events[lookup_id] = {**event, "calendar_id": calendar_id, "lookup_id": lookup_id}
return events
def _parse_event_dt(dt_obj: dict) -> datetime | None:
"""Parse a dateTime or date field from an event start/end object. Always
returns a timezone-aware datetime; all-day dates are treated as UTC."""
if not dt_obj:
return None
dt = dt_obj.get("dateTime")
if isinstance(dt, str):
try:
parsed = datetime.fromisoformat(dt)
except ValueError:
parsed = None
if parsed is not None:
return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
d = dt_obj.get("date")
if isinstance(d, str):
try:
return datetime.strptime(d, "%Y-%m-%d").replace(tzinfo=UTC)
except ValueError:
return None
return None
def _event_sort_key(event: dict) -> str:
start = event.get("start", {})
dt = start.get("dateTime")
if isinstance(dt, str):
return dt
d = start.get("date")
if isinstance(d, str):
return d
return ""
def _format_event(event: dict) -> dict[str, Any]:
"""Normalize an event for the API response."""
start = event.get("start", {})
end = event.get("end", {})
return {
"id": event.get("id", ""),
"lookup_id": event.get("lookup_id", event.get("id", "")),
"summary": event.get("summary", "(No title)"),
"start": start,
"end": end,
"description": event.get("description"),
"location": event.get("location"),
"created": event.get("created"),
"updated": event.get("updated"),
"calendar_id": event.get("calendar_id", "primary"),
"all_day": "date" in start and "dateTime" not in start,
}
# ---------------------------------------------------------------------------
# API handlers
# ---------------------------------------------------------------------------
async def api_events(request: Request) -> JSONResponse:
events_dict = _get_events()
events = list(events_dict.values())
date_param = request.query_params.get("date")
if date_param:
# Filter to the week containing the given date
try:
anchor = datetime.strptime(date_param, "%Y-%m-%d")
except ValueError:
return JSONResponse({"error": "Invalid date format, use YYYY-MM-DD"}, status_code=400)
week_start = anchor - timedelta(days=anchor.weekday()) # Monday
week_end = week_start + timedelta(days=7)
filtered = []
for ev in events:
start_dt = _parse_event_dt(ev.get("start", {}))
if start_dt is None:
continue
# Strip timezone for naive comparison
start_naive = start_dt.replace(tzinfo=None)
if week_start <= start_naive < week_end:
filtered.append(ev)
events = filtered
events.sort(key=_event_sort_key)
return JSONResponse({"events": [_format_event(e) for e in events]})
async def api_event_detail(request: Request) -> JSONResponse:
event_id = request.path_params["event_id"]
events_dict = _get_events()
event = events_dict.get(event_id)
if event is None:
return JSONResponse({"error": "Event not found"}, status_code=404)
return JSONResponse({"event": _format_event(event)})
async def api_stats(request: Request) -> JSONResponse:
events_dict = _get_events()
total = len(events_dict)
now = datetime.now(UTC)
upcoming = sum(
1
for ev in events_dict.values()
if (_parse_event_dt(ev.get("start", {})) or datetime.min.replace(tzinfo=UTC)) >= now
)
return JSONResponse({"total_events": total, "upcoming_events": upcoming})
async def viewer_html(request: Request) -> HTMLResponse:
return HTMLResponse(VIEWER_HTML)
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
def create_calendar_viewer_app():
routes = [
Route("/", viewer_html),
Route("/api/events", api_events),
Route("/api/events/{event_id}", api_event_detail),
Route("/api/stats", api_stats),
]
return Starlette(
routes=routes,
middleware=[Middleware(ProxyTokenMiddleware)],
)
def run_http_server(mcp_app, port: int) -> None:
"""Run combined MCP + viewer HTTP server."""
# Get the ASGI app from FastMCP
fastmcp_asgi = mcp_app.http_app(
transport="streamable-http",
path="/mcp",
)
viewer = create_calendar_viewer_app()
# Combined app: route /mcp to FastMCP, everything else to viewer
async def combined_app(scope, receive, send):
if scope["type"] == "lifespan":
await fastmcp_asgi(scope, receive, send)
return
path = scope.get("path", "")
if path.startswith("/mcp"):
await fastmcp_asgi(scope, receive, send)
else:
await viewer(scope, receive, send)
uvicorn.run(
combined_app,
host="127.0.0.1",
port=port,
log_level="warning",
)
# ---------------------------------------------------------------------------
# Viewer HTML
# ---------------------------------------------------------------------------
VIEWER_HTML = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Calendar</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #f8fafc; color: #1e293b; height: 100vh; display: flex; flex-direction: column; overflow: hidden; }
/* Header / toolbar */
.toolbar { display: flex; align-items: center; gap: 16px; padding: 10px 20px; background: #fff; border-bottom: 1px solid #e2e8f0; flex-shrink: 0; }
.toolbar h1 { font-size: 20px; font-weight: 600; color: #1e293b; margin-right: auto; }
.nav-btn { background: none; border: 1px solid #d1d5db; border-radius: 6px; padding: 6px 12px; font-size: 13px; cursor: pointer; color: #374151; }
.nav-btn:hover { background: #f1f5f9; }
.today-btn { background: #2563eb; color: #fff; border-color: #2563eb; }
.today-btn:hover { background: #1d4ed8; }
.week-label { font-size: 14px; font-weight: 600; color: #374151; min-width: 200px; text-align: center; }
.view-toggle { display: flex; border: 1px solid #d1d5db; border-radius: 6px; overflow: hidden; }
.view-btn { background: none; border: none; padding: 6px 14px; font-size: 13px; cursor: pointer; color: #374151; }
.view-btn.active { background: #2563eb; color: #fff; }
/* Week grid */
.week-view { flex: 1; display: flex; flex-direction: column; overflow: hidden; }
.week-header { display: grid; grid-template-columns: 56px repeat(7, 1fr); border-bottom: 1px solid #e2e8f0; background: #fff; flex-shrink: 0; padding-right: var(--calendar-scrollbar-width, 0px); }
.week-header .time-col { }
.day-header { padding: 8px 4px; text-align: center; border-left: 1px solid #e2e8f0; }
.day-header .day-name { font-size: 11px; color: #64748b; text-transform: uppercase; font-weight: 600; }
.day-header .day-num { font-size: 22px; font-weight: 400; color: #374151; line-height: 1.2; margin-top: 2px; }
.day-header.today .day-num { background: #2563eb; color: #fff; border-radius: 50%; width: 36px; height: 36px; display: flex; align-items: center; justify-content: center; margin: 2px auto 0; }
.week-body { flex: 1; overflow-y: auto; position: relative; }
.week-scroll { display: grid; grid-template-columns: 56px repeat(7, 1fr); min-height: 1440px; /* 60px per hour * 24h */ }
.time-gutter { position: relative; }
.time-label { position: absolute; right: 8px; font-size: 10px; color: #94a3b8; transform: translateY(-50%); }
.day-col { border-left: 1px solid #e2e8f0; position: relative; }
.hour-line { position: absolute; left: 0; right: 0; border-top: 1px solid #f1f5f9; }
.hour-line.half { border-top-style: dotted; }
/* Events */
.cal-event { position: absolute; left: 2px; right: 2px; border-radius: 4px; padding: 2px 6px; font-size: 11px; font-weight: 500; cursor: pointer; overflow: hidden; z-index: 1; border-left: 3px solid transparent; }
.cal-event:hover { opacity: 0.85; }
.cal-event .evt-time { font-size: 10px; opacity: 0.8; }
.cal-event .evt-title { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.all-day-row { display: grid; grid-template-columns: 56px repeat(7, 1fr); background: #fff; border-bottom: 1px solid #e2e8f0; min-height: 28px; padding-right: var(--calendar-scrollbar-width, 0px); }
.all-day-col { border-left: 1px solid #e2e8f0; padding: 2px; }
.all-day-event { border-radius: 3px; padding: 1px 6px; font-size: 11px; font-weight: 500; cursor: pointer; margin-bottom: 1px; overflow: hidden; white-space: nowrap; text-overflow: ellipsis; }
/* Current time line */
.now-line { position: absolute; left: 0; right: 0; z-index: 2; pointer-events: none; }
.now-line::before { content: ''; display: block; height: 2px; background: #ef4444; }
.now-dot { position: absolute; left: -5px; top: -4px; width: 10px; height: 10px; border-radius: 50%; background: #ef4444; }
/* List view */
.list-view { flex: 1; overflow-y: auto; padding: 16px 24px; display: none; }
.list-day-group { margin-bottom: 24px; }
.list-day-label { font-size: 13px; font-weight: 600; color: #64748b; text-transform: uppercase; letter-spacing: 0.05em; margin-bottom: 8px; padding-bottom: 4px; border-bottom: 1px solid #e2e8f0; }
.list-event { display: flex; gap: 16px; padding: 10px 12px; background: #fff; border-radius: 8px; margin-bottom: 6px; cursor: pointer; border: 1px solid #e2e8f0; }
.list-event:hover { border-color: #93c5fd; background: #eff6ff; }
.list-event .evt-color-dot { width: 10px; height: 10px; border-radius: 50%; flex-shrink: 0; margin-top: 3px; }
.list-event .evt-info { flex: 1; min-width: 0; }
.list-event .evt-title { font-size: 14px; font-weight: 500; color: #1e293b; }
.list-event .evt-meta { font-size: 12px; color: #64748b; margin-top: 2px; }
.list-empty { text-align: center; padding: 60px; color: #94a3b8; font-size: 14px; }
/* Modal */
.modal-overlay { position: fixed; inset: 0; background: rgba(0,0,0,0.3); z-index: 100; display: none; align-items: center; justify-content: center; }
.modal-overlay.open { display: flex; }
.modal { background: #fff; border-radius: 12px; width: 420px; max-width: 95vw; box-shadow: 0 20px 60px rgba(0,0,0,0.15); overflow: hidden; }
.modal-header { padding: 20px 20px 16px; display: flex; align-items: flex-start; gap: 12px; }
.modal-color { width: 14px; height: 14px; border-radius: 3px; flex-shrink: 0; margin-top: 3px; }
.modal-title { font-size: 18px; font-weight: 600; flex: 1; }
.modal-close { background: none; border: none; font-size: 18px; cursor: pointer; color: #94a3b8; padding: 0 4px; }
.modal-body { padding: 0 20px 20px; }
.modal-row { display: flex; gap: 10px; font-size: 13px; margin-bottom: 10px; align-items: flex-start; }
.modal-icon { font-size: 15px; flex-shrink: 0; width: 20px; }
.modal-text { color: #374151; flex: 1; }
.modal-desc { color: #475569; line-height: 1.6; white-space: pre-wrap; margin-top: 4px; }
.no-events-msg { display: flex; align-items: center; justify-content: center; height: 200px; color: #94a3b8; font-size: 14px; }
</style>
</head>
<body>
<div class="toolbar">
<h1>Calendar</h1>
<button class="nav-btn" onclick="prevWeek()">&#8249;</button>
<div class="week-label" id="week-label"></div>
<button class="nav-btn" onclick="nextWeek()">&#8250;</button>
<button class="nav-btn today-btn" onclick="goToday()">Today</button>
<div class="view-toggle">
<button class="view-btn active" id="btn-week" onclick="setView('week')">Week</button>
<button class="view-btn" id="btn-list" onclick="setView('list')">List</button>
</div>
</div>
<div class="week-view" id="week-view">
<div class="all-day-row" id="all-day-row">
<div style="font-size:10px;color:#94a3b8;padding:8px 4px;text-align:right;align-self:center">all-day</div>
</div>
<div class="week-header" id="week-header"></div>
<div class="week-body">
<div class="week-scroll" id="week-scroll"></div>
</div>
</div>
<div class="list-view" id="list-view">
<div id="list-content"></div>
</div>
<div class="modal-overlay" id="modal-overlay" onclick="closeModal(event)">
<div class="modal" id="modal"></div>
</div>
<script>
const HOURS = 24;
const PX_PER_HOUR = 60;
const DAYS = ['Sun','Mon','Tue','Wed','Thu','Fri','Sat'];
const MONTHS = ['January','February','March','April','May','June','July','August','September','October','November','December'];
const COLORS = [
{ bg: '#dbeafe', fg: '#1d4ed8', border: '#3b82f6' },
{ bg: '#dcfce7', fg: '#166534', border: '#22c55e' },
{ bg: '#fef3c7', fg: '#92400e', border: '#f59e0b' },
{ bg: '#fce7f3', fg: '#9d174d', border: '#ec4899' },
{ bg: '#ede9fe', fg: '#5b21b6', border: '#8b5cf6' },
{ bg: '#ffedd5', fg: '#9a3412', border: '#f97316' },
{ bg: '#cffafe', fg: '#164e63', border: '#06b6d4' },
{ bg: '#f0fdf4', fg: '#14532d', border: '#16a34a' },
];
let currentWeekStart = getWeekStart(new Date());
let allEvents = [];
let colorMap = {};
let colorIdx = 0;
let currentView = 'week';
function syncScrollbarGutter() {
const body = document.querySelector('.week-body');
if (!body) return;
const width = Math.max(body.offsetWidth - body.clientWidth, 0);
document.documentElement.style.setProperty('--calendar-scrollbar-width', width + 'px');
}
function getWeekStart(date) {
const d = new Date(date);
const day = d.getDay(); // 0=Sun
// Use Monday as week start
const diff = (day === 0) ? -6 : 1 - day;
d.setDate(d.getDate() + diff);
d.setHours(0, 0, 0, 0);
return d;
}
function getColor(eventId) {
if (!colorMap[eventId]) {
colorMap[eventId] = COLORS[colorIdx % COLORS.length];
colorIdx++;
}
return colorMap[eventId];
}
function esc(s) {
if (!s) return '';
const d = document.createElement('div');
d.textContent = s;
return d.innerHTML;
}
function parseEventDt(dtObj) {
if (!dtObj) return null;
if (dtObj.dateTime) {
try { return new Date(dtObj.dateTime); } catch { return null; }
}
if (dtObj.date) {
// Parse as local date to avoid timezone shift
const [y, m, d] = dtObj.date.split('-').map(Number);
return new Date(y, m - 1, d);
}
return null;
}
function formatTime(date) {
if (!date) return '';
return date.toLocaleTimeString('en-US', { hour: 'numeric', minute: '2-digit', hour12: true });
}
function formatDateLong(date) {
return DAYS[date.getDay()] + ', ' + MONTHS[date.getMonth()] + ' ' + date.getDate() + ', ' + date.getFullYear();
}
function isSameDay(a, b) {
return a.getFullYear() === b.getFullYear() && a.getMonth() === b.getMonth() && a.getDate() === b.getDate();
}
const base = window.location.pathname.replace(/\\/$/, '');
async function fetchEvents() {
const dateStr = currentWeekStart.toISOString().slice(0, 10);
const r = await fetch(base + '/api/events?date=' + dateStr);
const data = await r.json();
allEvents = data.events || [];
}
function getWeekDays() {
const days = [];
for (let i = 0; i < 7; i++) {
const d = new Date(currentWeekStart);
d.setDate(d.getDate() + i);
days.push(d);
}
return days;
}
function updateWeekLabel() {
const days = getWeekDays();
const start = days[0];
const end = days[6];
let label;
if (start.getMonth() === end.getMonth()) {
label = MONTHS[start.getMonth()] + ' ' + start.getDate() + ' – ' + end.getDate() + ', ' + start.getFullYear();
} else {
label = MONTHS[start.getMonth()] + ' ' + start.getDate() + ' – ' + MONTHS[end.getMonth()] + ' ' + end.getDate() + ', ' + end.getFullYear();
}
document.getElementById('week-label').textContent = label;
}
function renderWeekHeader() {
const days = getWeekDays();
const today = new Date();
const header = document.getElementById('week-header');
let html = '<div class="time-col"></div>';
days.forEach(d => {
const isToday = isSameDay(d, today);
html += '<div class="day-header' + (isToday ? ' today' : '') + '">' +
'<div class="day-name">' + DAYS[d.getDay()] + '</div>' +
'<div class="day-num">' + d.getDate() + '</div>' +
'</div>';
});
header.innerHTML = html;
}
function renderWeekGrid() {
const days = getWeekDays();
const scroll = document.getElementById('week-scroll');
// Build time gutter
let gutterHtml = '<div class="time-gutter">';
for (let h = 0; h < HOURS; h++) {
const top = h * PX_PER_HOUR;
const label = h === 0 ? '' : (h < 12 ? h + ' AM' : h === 12 ? '12 PM' : (h - 12) + ' PM');
gutterHtml += '<div class="time-label" style="top:' + top + 'px">' + label + '</div>';
// hour line marker
gutterHtml += '<div class="hour-line" style="top:' + top + 'px;left:0;width:56px"></div>';
}
gutterHtml += '</div>';
// Build day columns
let daysHtml = '';
days.forEach((day, idx) => {
daysHtml += '<div class="day-col" id="day-col-' + idx + '">';
// Hour lines
for (let h = 0; h < HOURS; h++) {
daysHtml += '<div class="hour-line" style="top:' + (h * PX_PER_HOUR) + 'px"></div>';
daysHtml += '<div class="hour-line half" style="top:' + (h * PX_PER_HOUR + 30) + 'px"></div>';
}
// Events for this day
const dayEvents = allEvents.filter(ev => {
if (ev.all_day) return false;
const start = parseEventDt(ev.start);
return start && isSameDay(start, day);
});
dayEvents.forEach(ev => {
const start = parseEventDt(ev.start);
const end = parseEventDt(ev.end);
if (!start) return;
const startMins = start.getHours() * 60 + start.getMinutes();
const endMins = end ? (end.getHours() * 60 + end.getMinutes()) : startMins + 30;
const top = (startMins / 60) * PX_PER_HOUR;
const height = Math.max(((endMins - startMins) / 60) * PX_PER_HOUR, 20);
const lookupId = ev.lookup_id || ev.id;
const color = getColor(lookupId);
daysHtml += '<div class="cal-event" style="top:' + top + 'px;height:' + height + 'px;background:' + color.bg + ';color:' + color.fg + ';border-left-color:' + color.border + '" onclick="showEvent(\\'' + esc(lookupId) + '\\')">' +
'<div class="evt-time">' + formatTime(start) + '</div>' +
'<div class="evt-title">' + esc(ev.summary) + '</div>' +
'</div>';
});
daysHtml += '</div>';
});
scroll.innerHTML = gutterHtml + daysHtml;
// All-day events
renderAllDay(days);
// Current time indicator
renderNowLine(days);
}
function renderAllDay(days) {
const allDayRow = document.getElementById('all-day-row');
// reset to first placeholder
allDayRow.innerHTML = '<div style="font-size:10px;color:#94a3b8;padding:8px 4px;text-align:right;align-self:center">all-day</div>';
days.forEach((day, idx) => {
const col = document.createElement('div');
col.className = 'all-day-col';
const dayEvents = allEvents.filter(ev => {
if (!ev.all_day) return false;
const start = parseEventDt(ev.start);
return start && isSameDay(start, day);
});
dayEvents.forEach(ev => {
const lookupId = ev.lookup_id || ev.id;
const color = getColor(lookupId);
const el = document.createElement('div');
el.className = 'all-day-event';
el.style.cssText = 'background:' + color.bg + ';color:' + color.fg;
el.textContent = ev.summary;
el.onclick = () => showEvent(lookupId);
col.appendChild(el);
});
allDayRow.appendChild(col);
});
}
function renderNowLine(days) {
const now = new Date();
days.forEach((day, idx) => {
if (!isSameDay(day, now)) return;
const mins = now.getHours() * 60 + now.getMinutes();
const top = (mins / 60) * PX_PER_HOUR;
const col = document.getElementById('day-col-' + idx);
if (!col) return;
const line = document.createElement('div');
line.className = 'now-line';
line.style.top = top + 'px';
line.innerHTML = '<span class="now-dot"></span>';
col.appendChild(line);
});
}
function renderListView() {
const content = document.getElementById('list-content');
if (!allEvents.length) {
content.innerHTML = '<div class="list-empty">No events this week</div>';
return;
}
// Group by day
const groups = {};
allEvents.forEach(ev => {
const start = parseEventDt(ev.start);
if (!start) return;
const key = start.toDateString();
if (!groups[key]) groups[key] = { date: start, events: [] };
groups[key].events.push(ev);
});
const sortedKeys = Object.keys(groups).sort((a, b) => new Date(a) - new Date(b));
let html = '';
sortedKeys.forEach(key => {
const g = groups[key];
html += '<div class="list-day-group">';
html += '<div class="list-day-label">' + formatDateLong(g.date) + '</div>';
g.events.forEach(ev => {
const start = parseEventDt(ev.start);
const end = parseEventDt(ev.end);
const lookupId = ev.lookup_id || ev.id;
const color = getColor(lookupId);
const timeStr = ev.all_day ? 'All day' : formatTime(start) + (end ? ' – ' + formatTime(end) : '');
html += '<div class="list-event" onclick="showEvent(\\'' + esc(lookupId) + '\\')">' +
'<div class="evt-color-dot" style="background:' + color.border + '"></div>' +
'<div class="evt-info">' +
'<div class="evt-title">' + esc(ev.summary) + '</div>' +
'<div class="evt-meta">' + esc(timeStr) + (ev.location ? ' · ' + esc(ev.location) : '') + '</div>' +
'</div>' +
'</div>';
});
html += '</div>';
});
content.innerHTML = html || '<div class="list-empty">No events this week</div>';
}
async function showEvent(id) {
const r = await fetch(base + '/api/events/' + encodeURIComponent(id));
const data = await r.json();
const ev = data.event;
if (!ev) return;
const start = parseEventDt(ev.start);
const end = parseEventDt(ev.end);
const color = getColor(ev.lookup_id || ev.id);
let timeStr;
if (ev.all_day) {
timeStr = formatDateLong(start);
} else {
timeStr = formatDateLong(start) + '<br>' + formatTime(start) + (end ? ' – ' + formatTime(end) : '');
}
let html = '<div class="modal-header">' +
'<div class="modal-color" style="background:' + color.border + '"></div>' +
'<div class="modal-title">' + esc(ev.summary) + '</div>' +
'<button class="modal-close" onclick="closeModal()">&#x2715;</button>' +
'</div>';
html += '<div class="modal-body">';
html += '<div class="modal-row"><span class="modal-icon">&#128197;</span><span class="modal-text">' + timeStr + '</span></div>';
if (ev.location) {
html += '<div class="modal-row"><span class="modal-icon">&#128205;</span><span class="modal-text">' + esc(ev.location) + '</span></div>';
}
if (ev.description) {
html += '<div class="modal-row"><span class="modal-icon">&#128221;</span><span class="modal-text modal-desc">' + esc(ev.description) + '</span></div>';
}
html += '</div>';
document.getElementById('modal').innerHTML = html;
document.getElementById('modal-overlay').classList.add('open');
}
function closeModal(e) {
if (e && e.target !== document.getElementById('modal-overlay')) return;
document.getElementById('modal-overlay').classList.remove('open');
}
function setView(view) {
currentView = view;
document.getElementById('btn-week').classList.toggle('active', view === 'week');
document.getElementById('btn-list').classList.toggle('active', view === 'list');
document.getElementById('week-view').style.display = view === 'week' ? 'flex' : 'none';
document.getElementById('list-view').style.display = view === 'list' ? 'block' : 'none';
if (view === 'list') renderListView();
}
async function prevWeek() {
currentWeekStart.setDate(currentWeekStart.getDate() - 7);
await refresh();
}
async function nextWeek() {
currentWeekStart.setDate(currentWeekStart.getDate() + 7);
await refresh();
}
async function goToday() {
currentWeekStart = getWeekStart(new Date());
await refresh();
}
async function refresh() {
syncScrollbarGutter();
updateWeekLabel();
await fetchEvents();
renderWeekHeader();
renderWeekGrid();
if (currentView === 'list') renderListView();
// Scroll to 8am on week view
scrollToHour(8);
}
function scrollToHour(hour) {
const body = document.querySelector('.week-body');
if (body) body.scrollTop = hour * PX_PER_HOUR - 20;
}
// Keyboard nav
document.addEventListener('keydown', (e) => {
if (e.key === 'Escape') document.getElementById('modal-overlay').classList.remove('open');
if (e.key === 'ArrowLeft' && !e.target.closest('.modal')) prevWeek();
if (e.key === 'ArrowRight' && !e.target.closest('.modal')) nextWeek();
});
// Init
window.addEventListener('resize', syncScrollbarGutter);
refresh();
</script>
</body>
</html>"""
docker/packages/google_calendar/mcp-tools.generated.json
+10062
View File
File diff suppressed because it is too large Load Diff
docker/packages/google_calendar/mcp.json
+72
View File
@@ -0,0 +1,72 @@
{
"run": {
"command": "python",
"args": [
"-m",
"google_calendar"
]
},
"toolsets": {
"read": [
"check_availability",
"get_event",
"list_accounts",
"list_calendars",
"list_events",
"search_events"
],
"write": [
"create_calendar",
"create_event",
"delete_event",
"respond_to_event",
"update_event"
],
"events": [
"create_event",
"delete_event",
"get_event",
"list_events",
"search_events",
"respond_to_event",
"update_event"
],
"calendars": [
"create_calendar",
"list_calendars"
],
"accounts": [
"list_accounts"
],
"search": [
"list_events",
"search_events"
],
"availability": [
"check_availability"
],
"scheduling": [
"check_availability",
"respond_to_event"
],
"core": [
"create_event",
"delete_event",
"get_event",
"list_events",
"search_events",
"update_event"
],
"toolathlon_legacy": [
"create_event",
"delete_event",
"get_event",
"list_events",
"update_event"
],
"state": [
"export_state",
"import_state"
]
}
}
docker/packages/google_calendar/pyproject.toml
+25
View File
@@ -0,0 +1,25 @@
[build-system]
build-backend = "uv_build"
requires = [ "uv-build>=0.11,<0.12" ]
[project]
name = "google-calendar"
version = "0.1.0"
description = "Google Calendar mock MCP server"
requires-python = ">=3.13"
classifiers = [
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
dependencies = [
"fastmcp>=3,<4",
"pydantic[email]>=2",
]
[tool.uv]
build-backend.module-root = ""
[tool.pytest]
ini_options.testpaths = [ "tests" ]
ini_options.pythonpath = [ "." ]
docker/packages/google_calendar/tests/test_attendees.py
+528
View File
@@ -0,0 +1,528 @@
"""Tests for attendees and RSVP functionality."""
import importlib
import json
import pytest
from pydantic import EmailStr, TypeAdapter, ValidationError
from google_calendar.models import (
AttendeeResponseStatus,
CalendarPerson,
EventAttendee,
EventClearField,
EventSource,
EventTransparency,
EventType,
ExtendedProperties,
)
def _get_gc():
return importlib.import_module("google_calendar.server")
def _get_state():
return importlib.import_module("google_calendar.state")
@pytest.fixture
def calendar_data(tmp_path):
external_services = tmp_path / "external_services"
external_services.mkdir()
data_file = external_services / "calendar_data.json"
data_file.write_text(json.dumps({"events": {}}))
return data_file
@pytest.fixture
def outputdir(tmp_path):
out = tmp_path / "output" / "google_calendar"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_globals(calendar_data, outputdir):
state = _get_state()
workspace = calendar_data.parent.parent / "workspace"
workspace.mkdir()
state.set_agent_workspace(str(workspace))
state.set_snapshot_paths(final_path=outputdir / "final.json", bundle_state_path=None)
yield
state.set_agent_workspace(None)
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
class TestCreateEventWithAttendees:
def test_create_and_update_attendees_are_schema_models(self):
gc = _get_gc()
assert gc.create_event.__annotations__["attendees"] == list[EventAttendee] | None
assert gc.update_event.__annotations__["attendees"] == list[EventAttendee] | None
def test_create_and_update_people_are_schema_models(self):
gc = _get_gc()
assert gc.create_event.__annotations__["creator"] == CalendarPerson | None
assert gc.create_event.__annotations__["organizer"] == CalendarPerson | None
assert gc.update_event.__annotations__["creator"] == CalendarPerson | None
assert gc.update_event.__annotations__["organizer"] == CalendarPerson | None
def test_create_and_update_metadata_are_schema_models(self):
gc = _get_gc()
assert gc.create_event.__annotations__["extendedProperties"] == ExtendedProperties | None
assert gc.create_event.__annotations__["source"] == EventSource | None
assert gc.create_event.__annotations__["transparency"] == EventTransparency | None
assert gc.update_event.__annotations__["extendedProperties"] == ExtendedProperties | None
assert gc.update_event.__annotations__["source"] == EventSource | None
assert gc.update_event.__annotations__["transparency"] == EventTransparency | None
assert gc.update_event.__annotations__["eventType"] == EventType | None
assert gc.update_event.__annotations__["clear_fields"] == list[EventClearField] | None
@pytest.mark.asyncio
async def test_create_with_attendees(self):
gc = _get_gc()
result = await gc.create_event(
summary="Team Sync",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[
{"email": "alice@co.com", "displayName": "Alice"},
{"email": "bob@co.com"},
],
)
assert result["status"] == "success"
attendees = result["event"]["attendees"]
assert len(attendees) == 2
assert attendees[0]["email"] == "alice@co.com"
assert attendees[0]["responseStatus"] == "needsAction"
assert attendees[1]["displayName"] == "bob@co.com" # falls back to email
@pytest.mark.asyncio
async def test_create_preserves_full_attendee_payload(self):
gc = _get_gc()
result = await gc.create_event(
summary="Team Sync",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[
{
"email": "room@example.com",
"resource": True,
"optional": True,
"comment": "Projector needed",
"additionalGuests": 2,
}
],
)
attendee = result["event"]["attendees"][0]
assert attendee["resource"] is True
assert attendee["optional"] is True
assert attendee["comment"] == "Projector needed"
assert attendee["additionalGuests"] == 2
assert attendee["displayName"] == "room@example.com"
assert attendee["responseStatus"] == "needsAction"
assert attendee["organizer"] is False
assert attendee["self"] is False
@pytest.mark.asyncio
async def test_create_rejects_duplicate_attendees(self):
gc = _get_gc()
result = await gc.create_event(
summary="Team Sync",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[
{"email": "alice@co.com"},
{"email": "ALICE@co.com"},
],
)
assert result["status"] == "error"
assert "Duplicate attendee email" in result["message"]
@pytest.mark.asyncio
async def test_create_without_attendees(self):
gc = _get_gc()
result = await gc.create_event(
summary="Solo Event",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
assert "attendees" not in result["event"]
class TestUpdateEventAttendees:
@pytest.mark.asyncio
async def test_add_attendees_via_update(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(
eventId=event_id,
attendees=[{"email": "carol@co.com", "displayName": "Carol"}],
)
assert len(updated["event"]["attendees"]) == 1
@pytest.mark.asyncio
async def test_update_preserves_full_attendee_payload(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(
eventId=event_id,
attendees=[
{
"email": "room@example.com",
"resource": True,
"optional": True,
"comment": "Projector needed",
"additionalGuests": 2,
}
],
)
attendee = updated["event"]["attendees"][0]
assert attendee["resource"] is True
assert attendee["optional"] is True
assert attendee["comment"] == "Projector needed"
assert attendee["additionalGuests"] == 2
assert attendee["displayName"] == "room@example.com"
assert attendee["responseStatus"] == "needsAction"
assert attendee["organizer"] is False
assert attendee["self"] is False
@pytest.mark.asyncio
async def test_update_rejects_duplicate_attendees(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(
eventId=event_id,
attendees=[
{"email": "alice@co.com"},
{"email": "ALICE@co.com"},
],
)
assert updated["status"] == "error"
assert "Duplicate attendee email" in updated["message"]
@pytest.mark.asyncio
async def test_clear_attendees(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com"}],
)
event_id = result["event"]["id"]
updated = await gc.update_event(eventId=event_id, attendees=[])
assert updated["event"]["attendees"] == []
class TestUpdateEventMetadata:
@pytest.mark.asyncio
async def test_update_can_clear_optional_metadata_fields(self):
gc = _get_gc()
result = await gc.create_event(
summary="Annotated Event",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
creator={"email": "creator@example.com"},
organizer={"email": "organizer@example.com"},
extendedProperties={"private": {"task_id": "task-123"}},
source={"title": "Planning notes"},
transparency="transparent",
)
event_id = result["event"]["id"]
updated = await gc.update_event(
eventId=event_id,
clear_fields=[
EventClearField.CREATOR,
EventClearField.ORGANIZER,
EventClearField.EXTENDED_PROPERTIES,
EventClearField.SOURCE,
EventClearField.TRANSPARENCY,
],
)
assert updated["status"] == "success"
for field in ["creator", "organizer", "extendedProperties", "source", "transparency"]:
assert field not in updated["event"]
@pytest.mark.asyncio
async def test_update_can_switch_special_event_type_back_to_default(self):
gc = _get_gc()
result = await gc.create_event(
summary="Focus Block",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
eventType="focusTime",
focusTimeProperties={"chatStatus": "doNotDisturb"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(eventId=event_id, eventType="default")
assert updated["status"] == "success"
assert updated["event"]["eventType"] == "default"
assert "focusTimeProperties" not in updated["event"]
@pytest.mark.asyncio
async def test_create_returns_error_for_invalid_event_type_shape(self):
gc = _get_gc()
result = await gc.create_event(
summary="Focus Block",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
eventType="focusTime",
)
assert result["status"] == "error"
assert "focusTimeProperties is required" in result["message"]
@pytest.mark.asyncio
async def test_update_returns_error_for_invalid_merged_event(self):
gc = _get_gc()
result = await gc.create_event(
summary="All Day",
start={"date": "2025-06-01"},
end={"date": "2025-06-02"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(eventId=event_id, start={"dateTime": "2025-06-01T10:00:00Z"})
assert updated["status"] == "error"
assert "both use dateTime or both use date" in updated["message"]
@pytest.mark.asyncio
async def test_update_can_switch_default_event_to_special_type_with_properties(self):
gc = _get_gc()
result = await gc.create_event(
summary="Working location",
start={"date": "2025-06-01"},
end={"date": "2025-06-02"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(
eventId=event_id,
eventType=EventType.WORKING_LOCATION,
workingLocationProperties={"type": "customLocation", "customLocation": {"label": "Client office"}},
)
assert updated["status"] == "success"
assert updated["event"]["eventType"] == "workingLocation"
assert updated["event"]["workingLocationProperties"]["customLocation"]["label"] == "Client office"
class TestRespondToEvent:
@pytest.mark.asyncio
async def test_accept_invitation(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com"}, {"email": "bob@co.com"}],
)
event_id = result["event"]["id"]
rsvp = await gc.respond_to_event(
eventId=event_id, email="alice@co.com", response=AttendeeResponseStatus.ACCEPTED
)
assert rsvp["status"] == "success"
alice = next(a for a in rsvp["event"]["attendees"] if a["email"] == "alice@co.com")
assert alice["responseStatus"] == "accepted"
# Bob should still be needsAction
bob = next(a for a in rsvp["event"]["attendees"] if a["email"] == "bob@co.com")
assert bob["responseStatus"] == "needsAction"
@pytest.mark.asyncio
async def test_decline_invitation(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com"}],
)
event_id = result["event"]["id"]
rsvp = await gc.respond_to_event(
eventId=event_id, email="alice@co.com", response=AttendeeResponseStatus.DECLINED
)
alice = rsvp["event"]["attendees"][0]
assert alice["responseStatus"] == "declined"
@pytest.mark.asyncio
async def test_tentative_response(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com"}],
)
event_id = result["event"]["id"]
rsvp = await gc.respond_to_event(
eventId=event_id, email="alice@co.com", response=AttendeeResponseStatus.TENTATIVE
)
assert rsvp["event"]["attendees"][0]["responseStatus"] == "tentative"
def test_respond_response_is_schema_enum(self):
gc = _get_gc()
assert gc.respond_to_event.__annotations__["response"] == AttendeeResponseStatus
assert gc.respond_to_event.__annotations__["email"] == EmailStr
with pytest.raises(ValidationError):
TypeAdapter(AttendeeResponseStatus).validate_python("maybe")
@pytest.mark.asyncio
async def test_respond_attendee_not_found(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com"}],
)
event_id = result["event"]["id"]
rsvp = await gc.respond_to_event(
eventId=event_id,
email="unknown@co.com",
response=AttendeeResponseStatus.ACCEPTED,
)
assert rsvp["status"] == "error"
assert "not found" in rsvp["message"]
@pytest.mark.asyncio
async def test_respond_event_not_found(self):
gc = _get_gc()
rsvp = await gc.respond_to_event(
eventId="nonexistent",
email="alice@co.com",
response=AttendeeResponseStatus.ACCEPTED,
)
assert rsvp["status"] == "error"
@pytest.mark.asyncio
async def test_respond_case_insensitive_email(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "Alice@CO.com"}],
)
event_id = result["event"]["id"]
rsvp = await gc.respond_to_event(
eventId=event_id, email="alice@co.com", response=AttendeeResponseStatus.ACCEPTED
)
assert rsvp["status"] == "success"
class TestAvailabilityWithAttendees:
@pytest.mark.asyncio
async def test_filter_by_attendee(self):
gc = _get_gc()
# Create event with alice as attendee
await gc.create_event(
summary="Alice's Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com", "responseStatus": "accepted"}],
)
# Create event with only bob
await gc.create_event(
summary="Bob's Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
attendees=[{"email": "bob@co.com", "responseStatus": "accepted"}],
)
# Check alice's availability — should only see alice's meeting
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
attendee_emails=["alice@co.com"],
)
assert len(result["busy"]) == 1
assert "Alice's Meeting" in result["busy"][0]["events"]
@pytest.mark.asyncio
async def test_declined_events_not_busy(self):
gc = _get_gc()
result = await gc.create_event(
summary="Optional Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com"}],
)
event_id = result["event"]["id"]
# Alice declines
await gc.respond_to_event(eventId=event_id, email="alice@co.com", response=AttendeeResponseStatus.DECLINED)
# Check alice's availability — declined event should not block
avail = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
attendee_emails=["alice@co.com"],
)
assert len(avail["busy"]) == 0
@pytest.mark.asyncio
async def test_find_mutual_availability(self):
gc = _get_gc()
# Alice busy 10-11am
await gc.create_event(
summary="Alice Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
attendees=[{"email": "alice@co.com", "responseStatus": "accepted"}],
)
# Bob busy 2-3pm (non-adjacent so they don't merge)
await gc.create_event(
summary="Bob Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
attendees=[{"email": "bob@co.com", "responseStatus": "accepted"}],
)
# Check availability for both — should see both busy periods
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
attendee_emails=["alice@co.com", "bob@co.com"],
)
assert len(result["busy"]) == 2
assert result["total_busy_minutes"] == 120
docker/packages/google_calendar/tests/test_bundle_datadir.py
+239
View File
@@ -0,0 +1,239 @@
"""Tests for the nested bundle-input layout: <BUNDLEDIR>/services/<name>/state.json."""
import importlib
import json
import pytest
def _get_state():
return importlib.import_module("google_calendar.state")
# A minimal, valid CalendarState seed. CalendarState has all-default fields, so
# an empty events dict is valid; one event keeps the round-trip meaningful.
SEED = {
"events": {
"evt-1": {
"id": "evt-1",
"summary": "Seeded Event",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
}
}
@pytest.fixture(autouse=True)
def _reset_state():
"""Keep state in-memory and reset the account registry around each test."""
state = _get_state()
state.set_agent_workspace(None)
state._accounts.clear()
state._active_account_id = "default"
yield
state.set_agent_workspace(None)
state._accounts.clear()
state._active_account_id = "default"
def test_resolve_bundle_state_path_prefers_state_json(tmp_path, monkeypatch):
state = _get_state()
service_dir = tmp_path / "services" / "google_calendar"
service_dir.mkdir(parents=True)
state_json = service_dir / "state.json"
state_json.write_text(json.dumps(SEED))
(service_dir / "events.json").write_text(json.dumps(SEED))
monkeypatch.setenv("BUNDLEDIR", str(tmp_path))
assert state.resolve_bundle_state_path() == state_json
def test_resolve_bundle_state_path_globs_when_no_state_json(tmp_path, monkeypatch):
state = _get_state()
service_dir = tmp_path / "services" / "google_calendar"
service_dir.mkdir(parents=True)
a_json = service_dir / "a.json"
a_json.write_text(json.dumps(SEED))
(service_dir / "b.json").write_text(json.dumps(SEED))
monkeypatch.setenv("BUNDLEDIR", str(tmp_path))
assert state.resolve_bundle_state_path() == a_json
def test_resolve_bundle_state_path_missing_subdir(tmp_path, monkeypatch):
state = _get_state()
(tmp_path / "services").mkdir()
monkeypatch.setenv("BUNDLEDIR", str(tmp_path))
assert state.resolve_bundle_state_path() is None
monkeypatch.delenv("BUNDLEDIR", raising=False)
assert state.resolve_bundle_state_path() is None
def test_resolve_bundle_output_path(tmp_path, monkeypatch):
state = _get_state()
output_dir = tmp_path / "services" / "google_calendar"
monkeypatch.setenv("BUNDLE_OUTPUT_DIR", str(output_dir))
assert state.resolve_bundle_output_path() == output_dir / "state.json"
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
assert state.resolve_bundle_output_path() is None
def test_bundle_state_json_matches_inputdir(tmp_path, monkeypatch):
state = _get_state()
bundle_dir = tmp_path / "bundle"
service_dir = bundle_dir / "services" / "google_calendar"
service_dir.mkdir(parents=True)
(service_dir / "state.json").write_text(json.dumps(SEED))
inputdir = tmp_path / "input"
inputdir.mkdir()
(inputdir / "calendar.json").write_text(json.dumps(SEED))
# Load from the nested bundle layout.
state.set_agent_workspace(None)
state._accounts.clear()
state._active_account_id = "default"
monkeypatch.setenv("BUNDLEDIR", str(bundle_dir))
monkeypatch.delenv("INPUTDIR", raising=False)
state.load_seed_state_from_env()
bundle_state = state.state_to_json()
# Reset and load the same seed from the INPUTDIR fallback.
state._accounts.clear()
state._active_account_id = "default"
state.set_agent_workspace(None)
monkeypatch.delenv("BUNDLEDIR", raising=False)
monkeypatch.setenv("INPUTDIR", str(inputdir))
state.load_seed_state_from_env()
inputdir_state = state.state_to_json()
assert bundle_state == inputdir_state
# Two distinguishable accounts: different seeded event so a swap would be caught.
ACCT_A = {
"events": {
"evt-a": {
"id": "evt-a",
"summary": "Account A Event",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
}
}
ACCT_B = {
"events": {
"evt-b": {
"id": "evt-b",
"summary": "Account B Event",
"start": {"dateTime": "2025-07-02T14:00:00Z"},
"end": {"dateTime": "2025-07-02T15:00:00Z"},
}
}
}
def _load_from_bundle(state, monkeypatch, bundle_dir):
"""Reset the registry and load seed state from *bundle_dir* in isolation."""
state.set_agent_workspace(None)
state._accounts.clear()
state._active_account_id = "default"
monkeypatch.setenv("BUNDLEDIR", str(bundle_dir))
monkeypatch.delenv("INPUTDIR", raising=False)
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
state.load_seed_state_from_env()
return state.state_to_json()
def test_bundle_multifile_folder_matches_consolidated_state(tmp_path, monkeypatch):
"""A multi-file bundle folder coalesces to the same state as a single
consolidated state.json with the same accounts."""
state = _get_state()
# (a) Consolidated: one state.json with both accounts.
consolidated = tmp_path / "consolidated"
consolidated_dir = consolidated / "services" / "google_calendar"
consolidated_dir.mkdir(parents=True)
(consolidated_dir / "state.json").write_text(json.dumps({"accounts": {"default": ACCT_A, "work": ACCT_B}}))
# (b) Split: two wrapper files, no state.json.
split = tmp_path / "split"
split_dir = split / "services" / "google_calendar"
split_dir.mkdir(parents=True)
(split_dir / "a.json").write_text(json.dumps({"accounts": {"default": ACCT_A}}))
(split_dir / "b.json").write_text(json.dumps({"accounts": {"work": ACCT_B}}))
consolidated_state = _load_from_bundle(state, monkeypatch, consolidated)
split_state = _load_from_bundle(state, monkeypatch, split)
assert consolidated_state == split_state
assert set(consolidated_state["accounts"]) == {"default", "work"}
assert set(split_state["accounts"]) == {"default", "work"}
def test_resolve_bundle_state_paths_returns_whole_folder(tmp_path, monkeypatch):
"""The plural resolver returns ALL sorted *.json when there's no state.json,
and exactly [state.json] when one is present."""
state = _get_state()
service_dir = tmp_path / "services" / "google_calendar"
service_dir.mkdir(parents=True)
a_json = service_dir / "a.json"
b_json = service_dir / "b.json"
a_json.write_text(json.dumps({"accounts": {"default": ACCT_A}}))
b_json.write_text(json.dumps({"accounts": {"work": ACCT_B}}))
monkeypatch.setenv("BUNDLEDIR", str(tmp_path))
assert state.resolve_bundle_state_paths() == [a_json, b_json]
state_json = service_dir / "state.json"
state_json.write_text(json.dumps({"accounts": {"default": ACCT_A}}))
assert state.resolve_bundle_state_paths() == [state_json]
def test_bundle_flat_files_merge_into_one_account(tmp_path, monkeypatch):
"""the raw entities layout splits ONE account across per-entity files
(e.g. events.json + calendars.json, no {accounts} wrapper). Flat files must
merge into a single default account, not fragment into a phantom account
per file that the server never activates."""
state = _get_state()
service_dir = tmp_path / "bundle" / "services" / "google_calendar"
service_dir.mkdir(parents=True)
# One account split into its two collection halves.
(service_dir / "events.json").write_text(json.dumps({"events": ACCT_A["events"]}))
(service_dir / "calendars.json").write_text(
json.dumps(
{
"calendars": {
"team": {
"summary": "Team",
"events": {
"evt-team": {
"id": "evt-team",
"summary": "Team Event",
"start": {"dateTime": "2025-08-03T09:00:00Z"},
"end": {"dateTime": "2025-08-03T10:00:00Z"},
}
},
}
}
}
)
)
merged = _load_from_bundle(state, monkeypatch, tmp_path / "bundle")
# Single (default) account holding BOTH halves — events and the calendar.
assert "accounts" not in merged, "flat per-entity files must merge into ONE account"
assert "evt-a" in merged["events"]
assert "team" in merged["calendars"]
docker/packages/google_calendar/tests/test_calendars.py
+980
View File
@@ -0,0 +1,980 @@
"""Tests for multiple calendar support."""
import importlib
import json
import pytest
from pydantic import ValidationError
from starlette.testclient import TestClient
from google_calendar.models import EventTransparency, EventType
def _get_gc():
return importlib.import_module("google_calendar.server")
def _get_state():
return importlib.import_module("google_calendar.state")
@pytest.fixture
def calendar_data(tmp_path):
"""Seed with primary events (backward-compatible flat format)."""
external_services = tmp_path / "external_services"
external_services.mkdir()
data_file = external_services / "calendar_data.json"
data_file.write_text(
json.dumps(
{
"timeZone": "America/New_York",
"events": {
"evt-1": {
"id": "evt-1",
"summary": "Primary Event",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
"created": "2025-01-01T00:00:00Z",
"updated": "2025-01-01T00:00:00Z",
}
},
}
)
)
return data_file
@pytest.fixture
def outputdir(tmp_path):
out = tmp_path / "output" / "google_calendar"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_globals(calendar_data, outputdir):
state = _get_state()
workspace = calendar_data.parent.parent / "workspace"
workspace.mkdir()
state.set_agent_workspace(str(workspace))
state.set_snapshot_paths(final_path=outputdir / "final.json", bundle_state_path=None)
yield
state.set_agent_workspace(None)
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
class TestBackwardCompatibility:
"""Existing flat events dict works as 'primary' calendar."""
def test_get_calendar_events_attaches_missing_primary_events_dict(self):
state = _get_state()
data = {}
events = state.get_calendar_events(data, "primary")
events["evt-1"] = {"id": "evt-1"}
assert data["events"] is events
assert data["events"]["evt-1"]["id"] == "evt-1"
def test_get_calendar_events_attaches_missing_secondary_events_dict(self):
state = _get_state()
data = {"calendars": {"work": {"summary": "Work"}}}
events = state.get_calendar_events(data, "work")
events["evt-1"] = {"id": "evt-1"}
assert data["calendars"]["work"]["events"] is events
assert data["calendars"]["work"]["events"]["evt-1"]["id"] == "evt-1"
@pytest.mark.asyncio
async def test_get_event_from_primary(self):
gc = _get_gc()
result = await gc.get_event(eventId="evt-1")
assert result["status"] == "success"
assert result["event"]["summary"] == "Primary Event"
@pytest.mark.asyncio
async def test_get_event_with_explicit_primary(self):
gc = _get_gc()
result = await gc.get_event(eventId="evt-1", calendar_id="primary")
assert result["status"] == "success"
@pytest.mark.asyncio
async def test_create_event_defaults_to_primary(self):
gc = _get_gc()
result = await gc.create_event(
summary="New Event",
start={"dateTime": "2025-06-02T10:00:00Z"},
end={"dateTime": "2025-06-02T11:00:00Z"},
)
assert result["status"] == "success"
# Should be in the flat events dict
data = _get_state().load_data()
assert result["event"]["id"] in data["events"]
@pytest.mark.asyncio
async def test_list_events_default_includes_primary(self):
gc = _get_gc()
result = await gc.list_events(
timeMin="2025-01-01T00:00:00Z",
timeMax="2025-12-31T23:59:59Z",
)
assert result["count"] >= 1
summaries = [e["summary"] for e in result["events"]]
assert "Primary Event" in summaries
def test_save_data_rejects_invalid_calendar_state(self):
state = _get_state()
data = state.load_data()
data["calendars"] = {"primary": {"summary": "Duplicate Primary", "events": {}}}
with pytest.raises(ValidationError):
state.save_data(data)
@pytest.mark.asyncio
async def test_write_returns_error_when_existing_state_is_invalid(self, calendar_data):
gc = _get_gc()
invalid_state = _get_state().load_data()
invalid_state["calendars"] = {"primary": {"summary": "Duplicate Primary", "events": {}}}
calendar_data.write_text(json.dumps(invalid_state))
result = await gc.create_calendar(summary="Work")
assert result["status"] == "error"
assert "Primary calendar events must be stored in top-level events" in result["message"]
persisted = json.loads(calendar_data.read_text())
assert "work" not in persisted["calendars"]
class TestCreateCalendar:
@pytest.mark.asyncio
async def test_create_calendar(self):
gc = _get_gc()
result = await gc.create_calendar(summary="Personal")
assert result["status"] == "success"
assert result["calendar"]["id"] == "personal"
assert result["calendar"]["summary"] == "Personal"
assert result["calendar"]["primary"] is False
@pytest.mark.asyncio
async def test_create_calendar_with_description(self):
gc = _get_gc()
result = await gc.create_calendar(summary="Team Meetings", description="Shared team calendar")
assert result["calendar"]["description"] == "Shared team calendar"
@pytest.mark.asyncio
async def test_create_calendar_with_timezone(self):
gc = _get_gc()
result = await gc.create_calendar(summary="London Office", timeZone="Europe/London")
assert result["status"] == "success"
assert result["calendar"]["timeZone"] == "Europe/London"
data = _get_state().load_data()
assert data["calendars"]["london-office"]["timeZone"] == "Europe/London"
@pytest.mark.asyncio
async def test_create_duplicate_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Personal")
result = await gc.create_calendar(summary="Personal")
assert result["status"] == "error"
assert "already exists" in result["message"]
@pytest.mark.asyncio
async def test_cannot_create_primary(self):
gc = _get_gc()
result = await gc.create_calendar(summary="Primary")
assert result["status"] == "error"
@pytest.mark.asyncio
async def test_cannot_create_empty_sanitized_calendar_id(self):
gc = _get_gc()
result = await gc.create_calendar(summary="!!!")
assert result["status"] == "error"
assert "letter or number" in result["message"]
assert "" not in _get_state().load_data().get("calendars", {})
class TestListCalendars:
@pytest.mark.asyncio
async def test_list_includes_primary(self):
gc = _get_gc()
result = await gc.list_calendars()
assert result["status"] == "success"
assert result["count"] >= 1
ids = [c["id"] for c in result["calendars"]]
assert "primary" in ids
primary = next(c for c in result["calendars"] if c["id"] == "primary")
assert primary["timeZone"] == "America/New_York"
@pytest.mark.asyncio
async def test_list_includes_created_calendars(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
await gc.create_calendar(summary="Personal")
result = await gc.list_calendars()
ids = [c["id"] for c in result["calendars"]]
assert "work" in ids
assert "personal" in ids
assert result["count"] == 3 # primary + work + personal
@pytest.mark.asyncio
async def test_primary_shows_event_count(self):
gc = _get_gc()
result = await gc.list_calendars()
primary = next(c for c in result["calendars"] if c["id"] == "primary")
assert primary["eventCount"] == 1 # from fixture
class TestMultiCalendarEvents:
@pytest.mark.asyncio
async def test_create_event_in_secondary_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
result = await gc.create_event(
summary="Work Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
assert result["status"] == "success"
assert result["event"]["summary"] == "Work Meeting"
@pytest.mark.asyncio
async def test_get_event_from_secondary_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
create_result = await gc.create_event(
summary="Work Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
event_id = create_result["event"]["id"]
result = await gc.get_event(eventId=event_id, calendar_id="work")
assert result["status"] == "success"
assert result["event"]["summary"] == "Work Meeting"
@pytest.mark.asyncio
async def test_event_not_found_in_wrong_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
create_result = await gc.create_event(
summary="Work Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
event_id = create_result["event"]["id"]
result = await gc.get_event(eventId=event_id, calendar_id="primary")
assert result["status"] == "error"
@pytest.mark.asyncio
async def test_create_event_in_nonexistent_calendar(self):
gc = _get_gc()
result = await gc.create_event(
summary="Orphan",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="nonexistent",
)
assert result["status"] == "error"
@pytest.mark.asyncio
async def test_list_events_all_calendars(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
await gc.create_event(
summary="Work Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
# List without calendar_id = all calendars
result = await gc.list_events(
timeMin="2025-01-01T00:00:00Z",
timeMax="2025-12-31T23:59:59Z",
)
summaries = [e["summary"] for e in result["events"]]
assert "Primary Event" in summaries
assert "Work Meeting" in summaries
@pytest.mark.asyncio
async def test_list_events_preserves_duplicate_ids_across_calendars(self):
gc = _get_gc()
state = _get_state()
data = state.load_data()
data["calendars"] = {
"work": {
"summary": "Work",
"events": {
"evt-1": {
"id": "evt-1",
"summary": "Work Event With Same ID",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
"created": "2025-01-01T00:00:00Z",
"updated": "2025-01-01T00:00:00Z",
}
},
}
}
state.save_data(data)
result = await gc.list_events(
timeMin="2025-01-01T00:00:00Z",
timeMax="2025-12-31T23:59:59Z",
)
summaries = [e["summary"] for e in result["events"]]
assert "Primary Event" in summaries
assert "Work Event With Same ID" in summaries
assert result["count"] == 2
def test_viewer_events_include_secondary_calendars(self):
from google_calendar.viewer import _get_events
state = _get_state()
data = state.load_data()
data["calendars"] = {
"work": {
"summary": "Work",
"events": {
"evt-1": {
"id": "evt-1",
"summary": "Work Event With Same ID",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
"created": "2025-01-01T00:00:00Z",
"updated": "2025-01-01T00:00:00Z",
}
},
}
}
state.save_data(data)
events = _get_events()
assert events["evt-1"]["summary"] == "Primary Event"
assert events["evt-1"]["calendar_id"] == "primary"
assert events["evt-1"]["lookup_id"] == "evt-1"
assert events["work:evt-1"]["summary"] == "Work Event With Same ID"
assert events["work:evt-1"]["calendar_id"] == "work"
assert events["work:evt-1"]["lookup_id"] == "work:evt-1"
def test_viewer_events_are_pinned_to_default_account(self):
from google_calendar.viewer import _get_events
state = _get_state()
state.state_from_json(
{
"accounts": {
"default": {
"events": {
"default-event": {
"id": "default-event",
"summary": "Default Account Event",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
}
},
"work": {
"events": {
"work-event": {
"id": "work-event",
"summary": "Work Account Event",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
}
}
},
}
}
)
state.set_active_account("work")
events = _get_events()
assert "default-event" in events
assert "work-event" not in events
def test_viewer_events_use_first_account_when_default_is_absent(self):
from google_calendar.viewer import _get_events
state = _get_state()
state.state_from_json(
{
"accounts": {
"work": {
"events": {
"work-event": {
"id": "work-event",
"summary": "Work Account Event",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
}
}
}
}
}
)
events = _get_events()
assert list(events) == ["work-event"]
assert events["work-event"]["summary"] == "Work Account Event"
def test_viewer_event_detail_uses_unique_lookup_id(self):
from google_calendar.viewer import create_calendar_viewer_app
state = _get_state()
data = state.load_data()
data["calendars"] = {
"work": {
"summary": "Work",
"events": {
"evt-1": {
"id": "evt-1",
"summary": "Work Event With Same ID",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
"created": "2025-01-01T00:00:00Z",
"updated": "2025-01-01T00:00:00Z",
}
},
}
}
state.save_data(data)
client = TestClient(create_calendar_viewer_app())
primary = client.get("/api/events/evt-1")
work = client.get("/api/events/work%3Aevt-1")
assert primary.status_code == 200
assert primary.json()["event"]["summary"] == "Primary Event"
assert primary.json()["event"]["lookup_id"] == "evt-1"
assert work.status_code == 200
assert work.json()["event"]["summary"] == "Work Event With Same ID"
assert work.json()["event"]["lookup_id"] == "work:evt-1"
def test_viewer_stats_handles_timed_events(self):
from google_calendar.viewer import create_calendar_viewer_app
client = TestClient(create_calendar_viewer_app(), raise_server_exceptions=False)
response = client.get("/api/stats")
assert response.status_code == 200
assert response.json()["total_events"] >= 1
@pytest.mark.asyncio
async def test_list_events_single_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
await gc.create_event(
summary="Work Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
# List only work calendar
result = await gc.list_events(
timeMin="2025-01-01T00:00:00Z",
timeMax="2025-12-31T23:59:59Z",
calendar_id="work",
)
assert result["count"] == 1
assert result["events"][0]["summary"] == "Work Meeting"
@pytest.mark.asyncio
async def test_list_events_max_results_zero_returns_no_events(self):
gc = _get_gc()
result = await gc.list_events(
timeMin="2025-01-01T00:00:00Z",
timeMax="2025-12-31T23:59:59Z",
maxResults=0,
)
assert result["status"] == "success"
assert result["events"] == []
assert result["count"] == 0
@pytest.mark.asyncio
async def test_delete_event_from_secondary_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
create_result = await gc.create_event(
summary="To Delete",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
event_id = create_result["event"]["id"]
result = await gc.delete_event(eventId=event_id, calendar_id="work")
assert result["status"] == "success"
@pytest.mark.asyncio
async def test_update_event_in_secondary_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
create_result = await gc.create_event(
summary="Original",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
event_id = create_result["event"]["id"]
result = await gc.update_event(eventId=event_id, summary="Updated", calendar_id="work")
assert result["status"] == "success"
assert result["event"]["summary"] == "Updated"
class TestCheckAvailability:
"""Tests for free/busy availability checking."""
@pytest.mark.asyncio
async def test_fully_free_range(self):
gc = _get_gc()
# Primary has one event at 10-11am on June 1. Check June 2 = all free.
result = await gc.check_availability(
timeMin="2025-06-02T08:00:00Z",
timeMax="2025-06-02T18:00:00Z",
)
assert result["status"] == "success"
assert len(result["busy"]) == 0
assert len(result["free_slots"]) == 1
assert result["free_slots"][0]["duration_minutes"] == 600 # 10 hours
@pytest.mark.asyncio
async def test_busy_period_detected(self):
gc = _get_gc()
# Check range that includes the fixture event (10-11am June 1)
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
)
assert len(result["busy"]) == 1
assert "Primary Event" in result["busy"][0]["events"]
@pytest.mark.asyncio
async def test_transparent_events_do_not_block_availability(self):
gc = _get_gc()
result = await gc.create_event(
summary="FYI hold",
start={"dateTime": "2025-06-05T09:00:00Z"},
end={"dateTime": "2025-06-05T10:00:00Z"},
transparency="transparent",
)
assert result["event"]["transparency"] == "transparent"
availability = await gc.check_availability(
timeMin="2025-06-05T08:00:00Z",
timeMax="2025-06-05T12:00:00Z",
)
assert availability["busy"] == []
assert availability["total_free_minutes"] == 240
@pytest.mark.asyncio
async def test_from_gmail_events_default_transparent_and_do_not_block_availability(self):
gc = _get_gc()
result = await gc.create_event(
summary="Flight to Denver",
start={"dateTime": "2025-06-05T09:00:00Z"},
end={"dateTime": "2025-06-05T10:00:00Z"},
eventType=EventType.FROM_GMAIL,
)
assert result["event"]["transparency"] == "transparent"
availability = await gc.check_availability(
timeMin="2025-06-05T08:00:00Z",
timeMax="2025-06-05T12:00:00Z",
)
assert availability["busy"] == []
@pytest.mark.asyncio
async def test_update_event_can_mark_event_transparent(self):
gc = _get_gc()
result = await gc.create_event(
summary="Optional office hours",
start={"dateTime": "2025-06-05T10:00:00Z"},
end={"dateTime": "2025-06-05T11:00:00Z"},
)
updated = await gc.update_event(eventId=result["event"]["id"], transparency=EventTransparency.TRANSPARENT)
assert updated["event"]["transparency"] == "transparent"
availability = await gc.check_availability(
timeMin="2025-06-05T08:00:00Z",
timeMax="2025-06-05T12:00:00Z",
)
assert availability["busy"] == []
@pytest.mark.asyncio
async def test_free_slots_around_event(self):
gc = _get_gc()
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
duration_minutes=30,
)
# Should have free slot before (8-10am) and after (11am-6pm) the event
assert len(result["free_slots"]) == 2
assert result["free_slots"][0]["duration_minutes"] == 120 # 8am-10am
assert result["free_slots"][1]["duration_minutes"] == 420 # 11am-6pm
@pytest.mark.asyncio
async def test_duration_filter(self):
gc = _get_gc()
# Create two events close together, leaving only a 30min gap
await gc.create_event(
summary="Meeting 1",
start={"dateTime": "2025-06-03T09:00:00Z"},
end={"dateTime": "2025-06-03T10:00:00Z"},
)
await gc.create_event(
summary="Meeting 2",
start={"dateTime": "2025-06-03T10:30:00Z"},
end={"dateTime": "2025-06-03T11:30:00Z"},
)
# Ask for 60min slots — the 30min gap should be filtered out
result = await gc.check_availability(
timeMin="2025-06-03T08:00:00Z",
timeMax="2025-06-03T12:00:00Z",
duration_minutes=60,
)
free_durations = [s["duration_minutes"] for s in result["free_slots"]]
assert 30 not in free_durations # 30min gap filtered
assert 60 in free_durations # 8-9am slot
@pytest.mark.asyncio
async def test_check_specific_calendar(self):
gc = _get_gc()
await gc.create_calendar(summary="Work")
await gc.create_event(
summary="Work Meeting",
start={"dateTime": "2025-06-01T14:00:00Z"},
end={"dateTime": "2025-06-01T15:00:00Z"},
calendar_id="work",
)
# Check only work calendar — should not see primary event
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
calendar_id="work",
)
assert len(result["busy"]) == 1
assert "Work Meeting" in result["busy"][0]["events"]
@pytest.mark.asyncio
async def test_check_availability_preserves_duplicate_ids_across_calendars(self):
gc = _get_gc()
state = _get_state()
data = state.load_data()
data["calendars"] = {
"work": {
"summary": "Work",
"events": {
"evt-1": {
"id": "evt-1",
"summary": "Work Event With Same ID",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
"created": "2025-01-01T00:00:00Z",
"updated": "2025-01-01T00:00:00Z",
}
},
}
}
state.save_data(data)
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
)
busy_event_names = [name for period in result["busy"] for name in period["events"]]
assert "Primary Event" in busy_event_names
assert "Work Event With Same ID" in busy_event_names
@pytest.mark.asyncio
async def test_overlapping_events_merged(self):
gc = _get_gc()
await gc.create_event(
summary="A",
start={"dateTime": "2025-06-04T09:00:00Z"},
end={"dateTime": "2025-06-04T10:30:00Z"},
)
await gc.create_event(
summary="B",
start={"dateTime": "2025-06-04T10:00:00Z"},
end={"dateTime": "2025-06-04T11:00:00Z"},
)
result = await gc.check_availability(
timeMin="2025-06-04T08:00:00Z",
timeMax="2025-06-04T12:00:00Z",
)
# A and B overlap, so should merge into one busy period
assert len(result["busy"]) == 1
assert result["total_busy_minutes"] == 120 # 9am-11am
@pytest.mark.asyncio
async def test_totals_correct(self):
gc = _get_gc()
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T18:00:00Z",
)
assert result["total_busy_minutes"] + result["total_free_minutes"] == 600 # 10 hours
class TestMultiAccountSupport:
@pytest.mark.asyncio
async def test_list_accounts_reports_default_for_flat_state(self):
gc = _get_gc()
result = await gc.list_accounts()
assert result["status"] == "success"
assert result["count"] == 1
assert result["accounts"][0]["account_id"] == "default"
assert result["accounts"][0]["event_count"] == 1
@pytest.mark.asyncio
async def test_multi_account_state_routes_reads_and_writes_by_account(self):
gc = _get_gc()
state = _get_state()
state.state_from_json(
{
"accounts": {
"default": {
"events": {
"default-event": {
"id": "default-event",
"summary": "Default Account Event",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
}
},
"work": {
"events": {
"work-event": {
"id": "work-event",
"summary": "Work Account Event",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
}
}
},
}
}
)
accounts = await gc.list_accounts()
assert {account["account_id"] for account in accounts["accounts"]} == {"default", "work"}
default_result = await gc.get_event(eventId="default-event")
assert default_result["status"] == "success"
assert default_result["event"]["summary"] == "Default Account Event"
work_result = await gc.get_event(eventId="work-event", account_id="work")
assert work_result["status"] == "success"
assert work_result["event"]["summary"] == "Work Account Event"
missing_cross_account = await gc.get_event(eventId="work-event", account_id="default")
assert missing_cross_account["status"] == "error"
created = await gc.create_event(
summary="Work Follow-up",
start={"dateTime": "2025-06-02T10:00:00Z"},
end={"dateTime": "2025-06-02T11:00:00Z"},
account_id="work",
)
assert created["status"] == "success"
exported = state.state_to_json()
created_id = created["event"]["id"]
assert created_id in exported["accounts"]["work"]["events"]
assert created_id not in exported["accounts"]["default"]["events"]
@pytest.mark.asyncio
async def test_search_and_availability_respect_account_id(self):
gc = _get_gc()
state = _get_state()
state.state_from_json(
{
"accounts": {
"default": {
"events": {
"default-event": {
"id": "default-event",
"summary": "Shared Planning",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
}
},
"work": {
"events": {
"work-event": {
"id": "work-event",
"summary": "Shared Planning",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
}
}
},
}
}
)
default_search = await gc.search_events(query="shared planning")
work_search = await gc.search_events(query="shared planning", account_id="work")
assert [event["id"] for event in default_search["events"]] == ["default-event"]
assert [event["id"] for event in work_search["events"]] == ["work-event"]
default_availability = await gc.check_availability(
timeMin="2025-06-01T09:00:00Z",
timeMax="2025-06-01T14:00:00Z",
account_id="default",
)
work_availability = await gc.check_availability(
timeMin="2025-06-01T09:00:00Z",
timeMax="2025-06-01T14:00:00Z",
account_id="work",
)
assert default_availability["busy"][0]["events"] == ["Shared Planning"]
assert default_availability["busy"][0]["start"] == "2025-06-01T10:00:00+00:00"
assert work_availability["busy"][0]["events"] == ["Shared Planning"]
assert work_availability["busy"][0]["start"] == "2025-06-01T12:00:00+00:00"
@pytest.mark.asyncio
async def test_failed_write_in_one_account_leaves_other_accounts_untouched(self):
gc = _get_gc()
state = _get_state()
state.state_from_json(
{
"accounts": {
"default": {"events": {}},
"work": {
"events": {
"work-event": {
"id": "work-event",
"summary": "Work Event",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
}
},
}
}
)
result = await gc.create_event(
summary="Broken Event",
start={"dateTime": "2025-06-02T11:00:00Z"},
end={"dateTime": "2025-06-02T10:00:00Z"},
)
exported = state.state_to_json()
assert result["status"] == "error"
assert exported["accounts"]["default"]["events"] == {}
assert exported["accounts"]["work"]["events"]["work-event"]["summary"] == "Work Event"
@pytest.mark.asyncio
async def test_state_from_json_resets_active_account_to_default(self):
gc = _get_gc()
state = _get_state()
state.state_from_json({"accounts": {"default": {"events": {}}, "work": {"events": {}}}})
await gc.create_event(
summary="Work Event",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
account_id="work",
)
assert state.get_active_account_id() == "work"
state.state_from_json({"accounts": {"default": {"events": {}}, "personal": {"events": {}}}})
assert state.get_active_account_id() == "default"
result = await gc.create_event(
summary="Default Event",
start={"dateTime": "2025-06-02T10:00:00Z"},
end={"dateTime": "2025-06-02T11:00:00Z"},
)
exported = state.state_to_json()
assert result["status"] == "success"
assert result["event"]["id"] in exported["accounts"]["default"]["events"]
assert exported["accounts"]["personal"]["events"] == {}
@pytest.mark.asyncio
async def test_omitted_account_uses_first_loaded_account_when_default_is_absent(self):
gc = _get_gc()
state = _get_state()
state.state_from_json(
{
"accounts": {
"work": {
"events": {
"work-event": {
"id": "work-event",
"summary": "Work Account Event",
"start": {"dateTime": "2025-06-01T12:00:00Z"},
"end": {"dateTime": "2025-06-01T13:00:00Z"},
}
}
}
}
}
)
result = await gc.get_event(eventId="work-event")
assert state.get_active_account_id() == "work"
assert result["status"] == "success"
assert result["event"]["summary"] == "Work Account Event"
def test_ensure_loaded_uses_first_persisted_account_when_default_is_absent(self, calendar_data):
state = _get_state()
workspace = calendar_data.parent.parent / "workspace"
calendar_data.write_text(json.dumps({"accounts": {"work": {"events": {}}}}))
state.set_agent_workspace(str(workspace))
assert state.load_data() == {"events": {}}
assert state.get_active_account_id() == "work"
def test_no_workspace_mode_stays_in_memory(self, tmp_path, monkeypatch):
state = _get_state()
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("AGENT_WORKSPACE", raising=False)
state.set_agent_workspace(None)
state.state_from_json({"events": {}})
state.save_data({"events": {}})
assert not (tmp_path / "calendar_data.json").exists()
@pytest.mark.asyncio
async def test_multi_account_snapshot_preserves_wrapper(self, outputdir):
gc = _get_gc()
state = _get_state()
state.state_from_json({"accounts": {"default": {"events": {}}, "personal": {"events": {}}}})
result = await gc.create_event(
summary="Personal Event",
start={"dateTime": "2025-06-02T10:00:00Z"},
end={"dateTime": "2025-06-02T11:00:00Z"},
account_id="personal",
)
assert result["status"] == "success"
snapshot = json.loads((outputdir / "final.json").read_text())
assert set(snapshot["accounts"]) == {"default", "personal"}
assert result["event"]["id"] in snapshot["accounts"]["personal"]["events"]
docker/packages/google_calendar/tests/test_recurrence_and_reminders.py
+414
View File
@@ -0,0 +1,414 @@
"""Tests for recurring events and reminders."""
import importlib
import json
import pytest
from pydantic import TypeAdapter, ValidationError
from google_calendar.models import EventReminders
def _get_gc():
return importlib.import_module("google_calendar.server")
def _get_state():
return importlib.import_module("google_calendar.state")
@pytest.fixture
def calendar_data(tmp_path):
external_services = tmp_path / "external_services"
external_services.mkdir()
data_file = external_services / "calendar_data.json"
data_file.write_text(json.dumps({"events": {}}))
return data_file
@pytest.fixture
def outputdir(tmp_path):
out = tmp_path / "output" / "google_calendar"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_globals(calendar_data, outputdir):
state = _get_state()
workspace = calendar_data.parent.parent / "workspace"
workspace.mkdir()
state.set_agent_workspace(str(workspace))
state.set_snapshot_paths(final_path=outputdir / "final.json", bundle_state_path=None)
yield
state.set_agent_workspace(None)
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
# ---------------------------------------------------------------------------
# Reminders
# ---------------------------------------------------------------------------
class TestReminders:
@pytest.mark.asyncio
async def test_create_event_with_reminders(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
reminders={"useDefault": False, "overrides": [{"method": "popup", "minutes": 15}]},
)
assert result["event"]["reminders"]["overrides"][0]["minutes"] == 15
@pytest.mark.asyncio
async def test_update_event_reminders(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
event_id = result["event"]["id"]
updated = await gc.update_event(
eventId=event_id,
reminders={"useDefault": False, "overrides": [{"method": "email", "minutes": 30}]},
)
assert updated["event"]["reminders"]["overrides"][0]["method"] == "email"
@pytest.mark.asyncio
async def test_update_event_reverts_to_default_reminders(self):
gc = _get_gc()
result = await gc.create_event(
summary="Meeting",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
reminders={"useDefault": False, "overrides": [{"method": "email", "minutes": 30}]},
)
event_id = result["event"]["id"]
updated = await gc.update_event(eventId=event_id, reminders={"useDefault": True})
assert updated["event"]["reminders"] == {"useDefault": True}
@pytest.mark.asyncio
async def test_event_without_reminders(self):
gc = _get_gc()
result = await gc.create_event(
summary="No Reminders",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
assert "reminders" not in result["event"]
def test_reminders_are_schema_model(self):
gc = _get_gc()
assert gc.create_event.__annotations__["reminders"] == EventReminders | None
assert gc.update_event.__annotations__["reminders"] == EventReminders | None
@pytest.mark.parametrize(
"payload",
[
{"useDefault": False, "overrides": [{"method": "sms", "minutes": 10}]},
{"useDefault": False, "overrides": [{"method": "popup", "minutes": -1}]},
{"useDefault": False, "overrides": [{"method": "popup", "minutes": 40321}]},
{"useDefault": True, "overrides": [{"method": "popup", "minutes": 10}]},
{"useDefault": False},
{"useDefault": False, "overrides": []},
{
"useDefault": False,
"overrides": [
{"method": "popup", "minutes": 1},
{"method": "popup", "minutes": 2},
{"method": "popup", "minutes": 3},
{"method": "popup", "minutes": 4},
{"method": "popup", "minutes": 5},
{"method": "popup", "minutes": 6},
],
},
],
)
def test_invalid_reminders_are_rejected(self, payload):
with pytest.raises(ValidationError):
TypeAdapter(EventReminders).validate_python(payload)
# ---------------------------------------------------------------------------
# Recurring Events
# ---------------------------------------------------------------------------
class TestRecurringEvents:
@pytest.mark.asyncio
async def test_create_daily_recurring_event(self):
gc = _get_gc()
result = await gc.create_event(
summary="Daily Standup",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:15:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=5"],
)
assert result["event"]["recurrence"] == ["RRULE:FREQ=DAILY;COUNT=5"]
@pytest.mark.asyncio
async def test_create_rejects_bad_recurrence_prefix(self):
gc = _get_gc()
result = await gc.create_event(
summary="Bad recurrence",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:15:00Z"},
recurrence=["FREQ=DAILY;COUNT=5"],
)
assert result["status"] == "error"
assert "String should match pattern" in result["message"]
@pytest.mark.asyncio
async def test_create_accepts_google_calendar_recurrence_exception_prefixes(self):
gc = _get_gc()
result = await gc.create_event(
summary="Recurring With Exceptions",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:15:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=5", "EXDATE:20250603T090000Z", "RDATE:20250610T090000Z"],
)
assert result["status"] == "success"
assert result["event"]["recurrence"] == [
"RRULE:FREQ=DAILY;COUNT=5",
"EXDATE:20250603T090000Z",
"RDATE:20250610T090000Z",
]
@pytest.mark.asyncio
async def test_list_expands_daily_recurring(self):
gc = _get_gc()
await gc.create_event(
summary="Daily Standup",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:15:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=5"],
)
result = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-10T00:00:00Z",
)
# Should have 5 instances (June 1-5)
standups = [e for e in result["events"] if "Standup" in e["summary"]]
assert len(standups) == 5
@pytest.mark.asyncio
async def test_list_expands_weekly_recurring(self):
gc = _get_gc()
await gc.create_event(
summary="Weekly Sync",
start={"dateTime": "2025-06-02T10:00:00Z"}, # Monday
end={"dateTime": "2025-06-02T11:00:00Z"},
recurrence=["RRULE:FREQ=WEEKLY;COUNT=4"],
)
result = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-07-01T00:00:00Z",
)
syncs = [e for e in result["events"] if "Weekly Sync" in e["summary"]]
assert len(syncs) == 4
@pytest.mark.asyncio
async def test_list_expands_weekly_with_byday(self):
gc = _get_gc()
await gc.create_event(
summary="Tue/Thu Meeting",
start={"dateTime": "2025-06-03T14:00:00Z"}, # Tuesday
end={"dateTime": "2025-06-03T15:00:00Z"},
recurrence=["RRULE:FREQ=WEEKLY;BYDAY=TU,TH;COUNT=6"],
)
result = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-30T00:00:00Z",
)
meetings = [e for e in result["events"] if "Tue/Thu" in e["summary"]]
assert len(meetings) == 6
@pytest.mark.asyncio
async def test_recurring_instances_have_unique_ids(self):
gc = _get_gc()
result = await gc.create_event(
summary="Daily",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:30:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=3"],
)
parent_id = result["event"]["id"]
listed = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-05T00:00:00Z",
)
ids = [e["id"] for e in listed["events"] if "Daily" in e["summary"]]
# Each instance has a unique ID derived from parent
assert len(ids) == 3
assert len(set(ids)) == 3 # all unique
assert all(parent_id in eid for eid in ids)
@pytest.mark.asyncio
async def test_recurring_instances_have_recurringEventId(self):
gc = _get_gc()
result = await gc.create_event(
summary="Recurring",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:30:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=2"],
)
parent_id = result["event"]["id"]
listed = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-05T00:00:00Z",
)
instances = [e for e in listed["events"] if "Recurring" in e["summary"]]
for inst in instances:
assert inst["recurringEventId"] == parent_id
assert "recurrence" not in inst # instances don't carry the rule
@pytest.mark.asyncio
async def test_recurring_instances_preserve_parent_time_zone(self):
gc = _get_gc()
await gc.create_event(
summary="Recurring with time zone",
start={"dateTime": "2025-06-01T09:00:00", "timeZone": "America/New_York"},
end={"dateTime": "2025-06-01T09:30:00", "timeZone": "America/New_York"},
recurrence=["RRULE:FREQ=DAILY;COUNT=2"],
)
listed = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-05T00:00:00Z",
)
instances = [e for e in listed["events"] if "Recurring with time zone" in e["summary"]]
assert len(instances) == 2
for inst in instances:
assert inst["start"]["timeZone"] == "America/New_York"
assert inst["end"]["timeZone"] == "America/New_York"
@pytest.mark.asyncio
async def test_recurring_instances_use_named_time_zone_across_dst(self):
gc = _get_gc()
await gc.create_event(
summary="DST recurring",
start={"dateTime": "2025-03-08T09:00:00-05:00", "timeZone": "America/New_York"},
end={"dateTime": "2025-03-08T09:30:00-05:00", "timeZone": "America/New_York"},
recurrence=["RRULE:FREQ=DAILY;COUNT=3"],
)
listed = await gc.list_events(
timeMin="2025-03-08T00:00:00Z",
timeMax="2025-03-12T00:00:00Z",
)
instances = [e for e in listed["events"] if "DST recurring" in e["summary"]]
assert [inst["start"]["dateTime"] for inst in instances] == [
"2025-03-08T09:00:00-05:00",
"2025-03-09T09:00:00-04:00",
"2025-03-10T09:00:00-04:00",
]
assert {inst["start"]["timeZone"] for inst in instances} == {"America/New_York"}
@pytest.mark.asyncio
async def test_recurring_only_within_query_range(self):
gc = _get_gc()
await gc.create_event(
summary="Daily",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:30:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=30"],
)
# Query only June 5-7
result = await gc.list_events(
timeMin="2025-06-05T00:00:00Z",
timeMax="2025-06-08T00:00:00Z",
)
daily = [e for e in result["events"] if "Daily" in e["summary"]]
assert len(daily) == 3 # June 5, 6, 7
@pytest.mark.asyncio
async def test_monthly_recurring(self):
gc = _get_gc()
await gc.create_event(
summary="Monthly Review",
start={"dateTime": "2025-01-15T10:00:00Z"},
end={"dateTime": "2025-01-15T11:00:00Z"},
recurrence=["RRULE:FREQ=MONTHLY;COUNT=6"],
)
result = await gc.list_events(
timeMin="2025-01-01T00:00:00Z",
timeMax="2025-07-01T00:00:00Z",
)
reviews = [e for e in result["events"] if "Monthly Review" in e["summary"]]
assert len(reviews) == 6
@pytest.mark.asyncio
async def test_recurring_with_until(self):
gc = _get_gc()
await gc.create_event(
summary="Until Event",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:30:00Z"},
recurrence=["RRULE:FREQ=DAILY;UNTIL=2025-06-04T00:00:00Z"],
)
result = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-10T00:00:00Z",
)
events = [e for e in result["events"] if "Until Event" in e["summary"]]
assert len(events) == 3 # June 1, 2, 3 (UNTIL is exclusive-ish)
@pytest.mark.asyncio
async def test_remove_recurrence_via_update(self):
gc = _get_gc()
result = await gc.create_event(
summary="Was Recurring",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:30:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=5"],
)
event_id = result["event"]["id"]
# Remove recurrence by passing empty list
await gc.update_event(eventId=event_id, recurrence=[])
listed = await gc.list_events(
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-10T00:00:00Z",
)
matching = [e for e in listed["events"] if "Was Recurring" in e["summary"]]
assert len(matching) == 1 # Just the single event now
class TestRecurringAvailability:
@pytest.mark.asyncio
async def test_recurring_events_block_availability(self):
gc = _get_gc()
await gc.create_event(
summary="Daily Standup",
start={"dateTime": "2025-06-01T09:00:00Z"},
end={"dateTime": "2025-06-01T09:15:00Z"},
recurrence=["RRULE:FREQ=DAILY;COUNT=5"],
)
result = await gc.check_availability(
timeMin="2025-06-01T08:00:00Z",
timeMax="2025-06-01T10:00:00Z",
)
assert len(result["busy"]) == 1
assert "Daily Standup" in result["busy"][0]["events"]
assert result["total_busy_minutes"] == 15
docker/packages/google_calendar/tests/test_schema_loose.py
+369
View File
@@ -0,0 +1,369 @@
"""Event schema enforces the supported Google Calendar mock shape."""
from datetime import UTC, datetime
import pytest
from pydantic import ValidationError
from google_calendar.models import (
Calendar,
CalendarPerson,
CalendarState,
Event,
EventAttendee,
EventDateTime,
)
from google_calendar.tools.search import _parse_event_end, _parse_event_start
def _minimal_event(**overrides: object) -> dict[str, object]:
base: dict[str, object] = {
"id": "evt-1",
"summary": "s",
"start": {"dateTime": "2025-06-01T10:00:00Z"},
"end": {"dateTime": "2025-06-01T11:00:00Z"},
}
base.update(overrides)
return base
def test_event_accepts_payload_without_created_and_updated():
event = Event.model_validate(_minimal_event())
assert event.created is None
assert event.updated is None
def test_event_rejects_unmodeled_extra_fields():
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(conferenceData={"entryPoints": []}))
def test_event_coerces_single_recurrence_string_to_list():
event = Event.model_validate(_minimal_event(recurrence="RRULE:FREQ=WEEKLY"))
assert event.recurrence == ["RRULE:FREQ=WEEKLY"]
def test_event_recurrence_requires_supported_prefix():
for recurrence in ["RRULE:FREQ=WEEKLY", "EXRULE:FREQ=WEEKLY", "RDATE:20250601", "EXDATE:20250601"]:
assert Event.model_validate(_minimal_event(recurrence=[recurrence]))
for recurrence in ["FREQ=WEEKLY", "BAD:FREQ=WEEKLY"]:
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(recurrence=[recurrence]))
def test_calendar_state_round_trips_legacy_event():
state = CalendarState.model_validate({"events": {"evt-1": _minimal_event(recurrence="RRULE:FREQ=DAILY")}})
dumped = state.model_dump(mode="json", exclude_none=True)
reloaded = CalendarState.model_validate(dumped)
assert reloaded == state
def test_calendar_state_round_trips_multi_calendar_state():
state = CalendarState.model_validate(
{
"timeZone": "America/New_York",
"events": {},
"calendars": {
"work": {
"summary": "Work",
"timeZone": "Europe/London",
"events": {"evt-1": _minimal_event()},
}
},
}
)
dumped = state.model_dump(mode="json", exclude_none=True)
assert dumped["timeZone"] == "America/New_York"
assert dumped["calendars"]["work"]["timeZone"] == "Europe/London"
assert dumped["calendars"]["work"]["events"]["evt-1"]["id"] == "evt-1"
assert CalendarState.model_validate(dumped) == state
def test_calendar_state_rejects_explicit_primary_calendar():
with pytest.raises(ValidationError):
CalendarState.model_validate(
{
"events": {"evt-1": _minimal_event()},
"calendars": {
"primary": {
"summary": "Primary",
"events": {"evt-2": _minimal_event(id="evt-2")},
}
},
}
)
def test_calendar_events_must_be_keyed_by_event_id():
with pytest.raises(ValidationError):
Calendar.model_validate(
{
"summary": "Work",
"events": {"wrong-key": _minimal_event(id="evt-1")},
}
)
with pytest.raises(ValidationError):
CalendarState.model_validate({"events": {"wrong-key": _minimal_event(id="evt-1")}})
def test_calendar_time_zone_is_validated():
assert Calendar.model_validate({"summary": "Work", "timeZone": "America/Los_Angeles"})
with pytest.raises(ValidationError):
Calendar.model_validate({"summary": "Work", "timeZone": "Not/AZone"})
with pytest.raises(ValidationError):
CalendarState.model_validate({"timeZone": "Not/AZone"})
@pytest.mark.parametrize(
"payload",
[
{"dateTime": "2025-06-01T10:00:00Z"},
{"dateTime": "2025-06-01T10:00:00-04:00"},
{"dateTime": "2025-06-01T10:00:00.123Z"},
{"dateTime": "2025-06-01T10:00:00", "timeZone": "America/New_York"},
{"dateTime": "2025-06-01T10:00:00-04:00", "timeZone": "America/New_York"},
{"date": "2025-06-01"},
],
)
def test_event_datetime_accepts_google_calendar_shapes(payload):
assert EventDateTime.model_validate(payload)
@pytest.mark.parametrize(
"payload",
[
{},
{"dateTime": "2025-06-01T10:00:00Z", "date": "2025-06-01"},
{"dateTime": "2025-06-01 10:00:00Z"},
{"dateTime": "2025-06-01T10:00:00"},
{"date": "2025-6-1"},
{"date": "2025-02-31"},
{"dateTime": "2025-06-01T10:00:00", "timeZone": "Not/AZone"},
{"dateTime": "2025-06-01T10:00:00Z", "timeZone": "America/New_York"},
],
)
def test_event_datetime_rejects_invalid_shapes(payload):
with pytest.raises(ValidationError):
EventDateTime.model_validate(payload)
def test_offsetless_datetime_uses_event_timezone():
event = _minimal_event(
start={"dateTime": "2025-06-01T10:00:00", "timeZone": "America/New_York"},
end={"dateTime": "2025-06-01T11:00:00", "timeZone": "America/New_York"},
)
parsed_start = _parse_event_start(event)
assert parsed_start is not None
assert parsed_start.isoformat() == "2025-06-01T10:00:00-04:00"
def test_legacy_all_day_same_day_end_is_normalized():
event = _minimal_event(
start={"date": "2025-06-01"},
end={"date": "2025-06-01"},
)
assert _parse_event_end(event) == datetime(2025, 6, 2, tzinfo=UTC)
def test_event_rejects_mixed_start_end_shapes():
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(start={"dateTime": "2025-06-01T10:00:00Z"}, end={"date": "2025-06-02"}))
def test_event_rejects_end_before_start():
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T10:00:00Z"},
)
)
def test_event_rejects_empty_id():
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(id=""))
def test_event_rejects_invalid_audit_timestamp():
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(created="2025-06-01T10:00:00"))
def test_event_transparency_is_typed():
event = Event.model_validate(_minimal_event(transparency="transparent"))
assert event.transparency == "transparent"
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(transparency="busy-ish"))
def test_from_gmail_events_default_to_transparent():
event = Event.model_validate(_minimal_event(eventType="fromGmail"))
assert event.transparency == "transparent"
def test_attendee_email_and_guest_count_are_validated():
assert EventAttendee.model_validate({"email": "alice@example.com", "additionalGuests": 1})
with pytest.raises(ValidationError):
EventAttendee.model_validate({"email": "not-an-email"})
with pytest.raises(ValidationError):
EventAttendee.model_validate({"email": "alice@example.com", "additionalGuests": -1})
def test_event_rejects_duplicate_attendees_case_insensitively():
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(
attendees=[
{"email": "alice@example.com"},
{"email": "ALICE@example.com"},
]
)
)
def test_creator_and_organizer_are_typed_person_objects():
event = Event.model_validate(
_minimal_event(
creator={"email": "creator@example.com", "displayName": "Creator"},
organizer={"email": "organizer@example.com", "displayName": "Organizer", "self": True},
)
)
assert event.creator is not None
assert event.creator.email == "creator@example.com"
assert event.organizer is not None
assert event.organizer.self is True
with pytest.raises(ValidationError):
CalendarPerson.model_validate({"email": "not-an-email"})
def test_extended_properties_and_source_are_typed():
event = Event.model_validate(
_minimal_event(
extendedProperties={
"private": {"task_id": "task-123"},
"shared": {"project": "Migration"},
},
source={"title": "Sprint planning notes"},
)
)
assert event.extendedProperties is not None
assert event.extendedProperties.private == {"task_id": "task-123"}
assert event.source is not None
assert event.source.title == "Sprint planning notes"
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(extendedProperties={"private": {"task_id": {"nested": "no"}}}))
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(source={"title": ""}))
def test_extended_properties_reject_unmodeled_top_level_keys():
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(
extendedProperties={
"private": {"task_id": "task-123"},
"unexpected": {"not": "modeled"},
}
)
)
def test_event_type_defaults_to_default():
event = Event.model_validate(_minimal_event())
assert event.eventType == "default"
def test_focus_time_event_accepts_matching_properties():
event = Event.model_validate(
_minimal_event(
eventType="focusTime",
focusTimeProperties={
"autoDeclineMode": "declineOnlyNewConflictingInvitations",
"chatStatus": "doNotDisturb",
"declineMessage": "Focusing right now",
},
)
)
assert event.focusTimeProperties is not None
assert event.focusTimeProperties.chatStatus == "doNotDisturb"
def test_event_type_rejects_missing_or_mismatched_properties():
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(eventType="focusTime"))
with pytest.raises(ValidationError):
Event.model_validate(_minimal_event(eventType="default", focusTimeProperties={"chatStatus": "available"}))
def test_working_location_properties_match_location_type():
event = Event.model_validate(
_minimal_event(
eventType="workingLocation",
workingLocationProperties={"type": "customLocation", "customLocation": {"label": "Client office"}},
)
)
assert event.workingLocationProperties is not None
assert event.workingLocationProperties.type == "customLocation"
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(eventType="workingLocation", workingLocationProperties={"type": "customLocation"})
)
def test_working_location_home_office_is_empty_marker():
event = Event.model_validate(
_minimal_event(
eventType="workingLocation",
workingLocationProperties={"type": "homeOffice", "homeOffice": {}},
)
)
assert event.workingLocationProperties is not None
assert event.workingLocationProperties.type == "homeOffice"
assert event.workingLocationProperties.homeOffice is not None
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(
eventType="workingLocation",
workingLocationProperties={"type": "homeOffice", "homeOffice": {"label": "Home"}},
)
)
def test_birthday_properties_validate_special_date_shape():
Event.model_validate(_minimal_event(eventType="birthday", birthdayProperties={"type": "birthday"}))
Event.model_validate(
_minimal_event(eventType="birthday", birthdayProperties={"type": "anniversary", "contact": "people/c12345"})
)
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(eventType="birthday", birthdayProperties={"type": "self", "contact": "people/c12345"})
)
with pytest.raises(ValidationError):
Event.model_validate(
_minimal_event(eventType="birthday", birthdayProperties={"type": "anniversary", "contact": "people/a"})
)
docker/packages/google_calendar/tests/test_search_events.py
+444
View File
@@ -0,0 +1,444 @@
"""Tests for event search behavior."""
import importlib
import inspect
import json
import pytest
from pydantic import EmailStr, TypeAdapter, ValidationError
from google_calendar.models import (
AttendeeResponseStatus,
AvailabilityDurationMinutes,
ListEventsMaxResults,
Rfc3339OffsetDateTimeString,
SearchEventsMaxResults,
SearchEventsOrderBy,
)
def _get_gc():
return importlib.import_module("google_calendar.server")
def _get_state():
return importlib.import_module("google_calendar.state")
@pytest.fixture
def calendar_data(tmp_path):
external_services = tmp_path / "external_services"
external_services.mkdir()
data_file = external_services / "calendar_data.json"
data_file.write_text(
json.dumps(
{
"events": {
"evt-budget": {
"id": "evt-budget",
"summary": "Budget Review",
"description": "Q3 planning with the finance team",
"location": "Conference Room A",
"start": {"dateTime": "2025-06-03T14:00:00Z"},
"end": {"dateTime": "2025-06-03T15:00:00Z"},
"created": "2025-05-01T00:00:00Z",
"updated": "2025-05-02T00:00:00Z",
"creator": {"email": "manager@example.com", "displayName": "Budget Manager"},
"organizer": {"email": "finance-lead@example.com", "displayName": "Finance Lead"},
"attendees": [
{
"email": "alice@example.com",
"displayName": "Alice Chen",
"responseStatus": "accepted",
},
{"email": "bob@example.com", "displayName": "Bob Smith", "responseStatus": "declined"},
],
},
"evt-design": {
"id": "evt-design",
"summary": "Design Sync",
"description": "Review updated mobile mockups",
"location": "Studio",
"start": {"dateTime": "2025-06-05T16:00:00Z"},
"end": {"dateTime": "2025-06-05T17:00:00Z"},
"created": "2025-05-01T00:00:00Z",
"updated": "2025-05-04T00:00:00Z",
"attendees": [
{"email": "carol@example.com", "displayName": "Carol Lee", "responseStatus": "tentative"}
],
},
"evt-naive": {
"id": "evt-naive",
"summary": "Naive Time Review",
"description": "Stored without a timezone offset",
"start": {"dateTime": "2025-06-07T10:00:00"},
"end": {"dateTime": "2025-06-07T11:00:00"},
"created": "2025-05-01T00:00:00Z",
"updated": "2025-05-06T00:00:00Z",
},
"evt-broken": {
"id": "evt-broken",
"summary": "Broken Planning",
"description": "Missing start and end should be reported when range-filtered",
},
"evt-standup": {
"id": "evt-standup",
"summary": "Daily Standup",
"description": "Engineering status updates",
"start": {"dateTime": "2025-06-01T09:00:00Z"},
"end": {"dateTime": "2025-06-01T09:15:00Z"},
"created": "2025-05-01T00:00:00Z",
"updated": "2025-05-03T00:00:00Z",
"recurrence": ["RRULE:FREQ=DAILY;COUNT=5"],
},
},
"calendars": {
"work": {
"id": "work",
"summary": "Work Calendar",
"events": {
"evt-roadmap": {
"id": "evt-roadmap",
"summary": "Roadmap Planning",
"description": "Quarterly product planning",
"location": "War Room",
"start": {"dateTime": "2025-06-04T10:00:00Z"},
"end": {"dateTime": "2025-06-04T11:30:00Z"},
"created": "2025-05-01T00:00:00Z",
"updated": "2025-05-05T00:00:00Z",
"creator": {"email": "product@example.com", "displayName": "Product"},
"organizer": {"email": "dana@example.com", "displayName": "Dana Patel"},
"attendees": [{"email": "dana@example.com", "displayName": "Dana Patel"}],
}
},
}
},
}
)
)
return data_file
@pytest.fixture
def outputdir(tmp_path):
out = tmp_path / "output" / "google_calendar"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_globals(calendar_data, outputdir):
state = _get_state()
workspace = calendar_data.parent.parent / "workspace"
workspace.mkdir()
state.set_agent_workspace(str(workspace))
state.set_snapshot_paths(final_path=outputdir / "final.json", bundle_state_path=None)
yield
state.set_agent_workspace(None)
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
@pytest.mark.asyncio
async def test_search_events_matches_all_query_terms_case_insensitively():
gc = _get_gc()
result = await gc.search_events(query="budget finance")
assert result["status"] == "success"
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-budget"
assert result["events"][0]["calendar_id"] == "primary"
@pytest.mark.asyncio
async def test_search_events_supports_quoted_phrases():
gc = _get_gc()
result = await gc.search_events(query='"Conference Room A"')
assert result["count"] == 1
assert result["events"][0]["summary"] == "Budget Review"
@pytest.mark.asyncio
async def test_search_events_quoted_phrases_do_not_span_field_boundaries():
gc = _get_gc()
result = await gc.search_events(query='"review q3"')
assert result["events"] == []
assert result["count"] == 0
@pytest.mark.asyncio
async def test_search_events_matches_attendee_query_text():
gc = _get_gc()
result = await gc.search_events(query="carol@example.com")
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-design"
@pytest.mark.asyncio
async def test_search_events_attendee_filter_can_search_without_keywords():
gc = _get_gc()
result = await gc.search_events(query="", attendee_email="bob@example.com")
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-budget"
@pytest.mark.asyncio
async def test_search_events_filters_by_response_status_for_attendee():
gc = _get_gc()
accepted = await gc.search_events(
query="", attendee_email="alice@example.com", response_status=AttendeeResponseStatus.ACCEPTED
)
assert accepted["count"] == 1
assert accepted["events"][0]["id"] == "evt-budget"
declined = await gc.search_events(
query="", attendee_email="alice@example.com", response_status=AttendeeResponseStatus.DECLINED
)
assert declined["events"] == []
assert declined["count"] == 0
@pytest.mark.asyncio
async def test_search_events_filters_by_response_status_without_attendee():
gc = _get_gc()
result = await gc.search_events(query="", response_status=AttendeeResponseStatus.DECLINED)
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-budget"
def test_search_events_response_status_is_schema_enum():
gc = _get_gc()
assert gc.search_events.__annotations__["response_status"] == AttendeeResponseStatus | None
with pytest.raises(ValidationError):
TypeAdapter(AttendeeResponseStatus).validate_python("maybe")
@pytest.mark.asyncio
async def test_search_events_filters_by_creator_and_organizer():
gc = _get_gc()
by_creator = await gc.search_events(query="", creator_email="manager@example.com")
assert by_creator["count"] == 1
assert by_creator["events"][0]["id"] == "evt-budget"
by_organizer = await gc.search_events(query="", organizer_email="dana@example.com")
assert by_organizer["count"] == 1
assert by_organizer["events"][0]["id"] == "evt-roadmap"
@pytest.mark.asyncio
async def test_search_events_matches_creator_and_organizer_query_text():
gc = _get_gc()
result = await gc.search_events(query="finance-lead@example.com")
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-budget"
@pytest.mark.asyncio
async def test_search_events_time_range_filters_results():
gc = _get_gc()
result = await gc.search_events(
query="planning",
timeMin="2025-06-04T00:00:00Z",
timeMax="2025-06-06T00:00:00Z",
)
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-roadmap"
@pytest.mark.asyncio
async def test_search_events_handles_naive_event_times_with_aware_ranges():
gc = _get_gc()
result = await gc.search_events(
query="naive",
timeMin="2025-06-07T00:00:00Z",
timeMax="2025-06-08T00:00:00Z",
)
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-naive"
@pytest.mark.asyncio
async def test_search_events_calendar_id_scopes_results():
gc = _get_gc()
result = await gc.search_events(query="planning", calendar_id="work")
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-roadmap"
assert result["events"][0]["calendar_summary"] == "Work Calendar"
@pytest.mark.asyncio
async def test_search_events_nonexistent_calendar_returns_empty_results():
gc = _get_gc()
result = await gc.search_events(query="planning", calendar_id="nonexistent")
assert result["status"] == "success"
assert result["events"] == []
assert result["count"] == 0
@pytest.mark.asyncio
async def test_search_events_expands_recurring_events_with_time_range():
gc = _get_gc()
result = await gc.search_events(
query="standup",
timeMin="2025-06-03T00:00:00Z",
timeMax="2025-06-05T00:00:00Z",
)
assert result["count"] == 2
assert [event["start"]["dateTime"] for event in result["events"]] == [
"2025-06-03T09:00:00+00:00",
"2025-06-04T09:00:00+00:00",
]
assert {event["recurringEventId"] for event in result["events"]} == {"evt-standup"}
@pytest.mark.asyncio
async def test_search_events_expands_recurring_events_without_time_range():
gc = _get_gc()
result = await gc.search_events(query="standup")
assert result["count"] == 5
assert [event["start"]["dateTime"] for event in result["events"]] == [
"2025-06-01T09:00:00+00:00",
"2025-06-02T09:00:00+00:00",
"2025-06-03T09:00:00+00:00",
"2025-06-04T09:00:00+00:00",
"2025-06-05T09:00:00+00:00",
]
assert {event["recurringEventId"] for event in result["events"]} == {"evt-standup"}
@pytest.mark.asyncio
async def test_search_events_can_order_by_updated_and_limit_results():
gc = _get_gc()
result = await gc.search_events(query="planning", maxResults=1, orderBy=SearchEventsOrderBy.UPDATED)
assert result["count"] == 1
assert result["events"][0]["id"] == "evt-roadmap"
@pytest.mark.asyncio
async def test_search_events_max_results_zero_returns_no_results():
gc = _get_gc()
result = await gc.search_events(query="planning", maxResults=0)
assert result["status"] == "success"
assert result["events"] == []
assert result["count"] == 0
def test_search_events_max_results_is_schema_non_negative():
with pytest.raises(ValidationError):
TypeAdapter(SearchEventsMaxResults).validate_python(-1)
def test_range_bounds_require_rfc3339_offset_in_schema():
gc = _get_gc()
assert gc.list_events.__annotations__["timeMin"] == Rfc3339OffsetDateTimeString
assert gc.list_events.__annotations__["timeMax"] == Rfc3339OffsetDateTimeString
assert gc.search_events.__annotations__["timeMin"] == Rfc3339OffsetDateTimeString | None
assert gc.check_availability.__annotations__["timeMin"] == Rfc3339OffsetDateTimeString
assert gc.check_availability.__annotations__["attendee_emails"] == list[EmailStr] | None
assert gc.check_availability.__annotations__["duration_minutes"] == AvailabilityDurationMinutes
with pytest.raises(ValidationError):
TypeAdapter(Rfc3339OffsetDateTimeString).validate_python("2025-06-01T00:00:00")
with pytest.raises(ValidationError):
TypeAdapter(AvailabilityDurationMinutes).validate_python(0)
def test_search_events_order_by_is_schema_enum():
gc = _get_gc()
assert gc.search_events.__annotations__["orderBy"] == SearchEventsOrderBy | None
with pytest.raises(ValidationError):
TypeAdapter(SearchEventsOrderBy).validate_python("summary")
def test_list_events_order_by_is_schema_enum():
gc = _get_gc()
assert gc.list_events.__annotations__["orderBy"] == SearchEventsOrderBy | None
assert inspect.signature(gc.list_events).parameters["orderBy"].default is None
assert gc.list_events.__annotations__["maxResults"] == ListEventsMaxResults | None
with pytest.raises(ValidationError):
TypeAdapter(SearchEventsOrderBy).validate_python("summary")
with pytest.raises(ValidationError):
TypeAdapter(ListEventsMaxResults).validate_python(-1)
@pytest.mark.asyncio
async def test_search_events_reports_skipped_unparseable_events():
gc = _get_gc()
result = await gc.search_events(
query="planning",
timeMin="2025-06-01T00:00:00Z",
timeMax="2025-06-10T00:00:00Z",
)
assert result["status"] == "success"
assert result["skipped_unparseable"] == ["evt-broken"]
@pytest.mark.asyncio
async def test_search_events_no_match_returns_empty_results():
gc = _get_gc()
result = await gc.search_events(query="vendor escalation")
assert result["status"] == "success"
assert result["events"] == []
assert result["count"] == 0
@pytest.mark.asyncio
async def test_search_events_requires_query_or_attendee_filter():
gc = _get_gc()
result = await gc.search_events(query="")
assert result["status"] == "error"
assert result["events"] == []
assert result["count"] == 0
@pytest.mark.asyncio
async def test_search_events_reports_unsupported_query_operator():
gc = _get_gc()
result = await gc.search_events(query="before:2024-01-01 planning")
assert result["status"] == "success"
assert result["count"] == 0
assert result["warnings"] == [
"Unsupported Calendar query operator 'before:'; use search_events parameters such as timeMin, timeMax, "
"attendee_email, response_status, organizer_email, or creator_email."
]
docker/packages/google_calendar/tests/test_snapshot.py
+135
View File
@@ -0,0 +1,135 @@
"""Tests for the _snapshot_on_write decorator — final.json written after every write tool call."""
import importlib
import json
import pytest
def _get_gc():
"""Import the server module lazily so tests control its module globals."""
return importlib.import_module("google_calendar.server")
def _get_state():
return importlib.import_module("google_calendar.state")
@pytest.fixture
def calendar_data(tmp_path):
"""Seed a minimal calendar data file."""
external_services = tmp_path / "external_services"
external_services.mkdir()
data_file = external_services / "calendar_data.json"
data_file.write_text(json.dumps({"events": {}}))
return data_file
@pytest.fixture
def outputdir(tmp_path):
out = tmp_path / "output" / "google_calendar"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_globals(calendar_data, outputdir):
"""Patch google_calendar module globals for isolated testing."""
state = _get_state()
workspace = calendar_data.parent.parent / "workspace"
workspace.mkdir()
state.set_agent_workspace(str(workspace))
state.set_snapshot_paths(final_path=outputdir / "final.json", bundle_state_path=None)
yield
state.set_agent_workspace(None)
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
@pytest.mark.asyncio
async def test_create_event_writes_final_json(outputdir):
gc = _get_gc()
final = outputdir / "final.json"
assert not final.exists()
result = await gc.create_event(
summary="Test Event",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
assert result["event"]["summary"] == "Test Event"
assert final.exists(), "final.json must be written after create_event"
snapshot = json.loads(final.read_text())
assert len(snapshot["events"]) == 1
@pytest.mark.asyncio
async def test_update_event_writes_final_json(outputdir):
gc = _get_gc()
result = await gc.create_event(
summary="Original",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
event_id = result["event"]["id"]
final = outputdir / "final.json"
final.unlink() # isolate the update's snapshot
await gc.update_event(eventId=event_id, summary="Updated")
assert final.exists(), "final.json must be written after update_event"
snapshot = json.loads(final.read_text())
assert snapshot["events"][event_id]["summary"] == "Updated"
@pytest.mark.asyncio
async def test_delete_event_writes_final_json(outputdir):
gc = _get_gc()
result = await gc.create_event(
summary="To Delete",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
event_id = result["event"]["id"]
final = outputdir / "final.json"
final.unlink()
await gc.delete_event(eventId=event_id)
assert final.exists(), "final.json must be written after delete_event"
snapshot = json.loads(final.read_text())
assert event_id not in snapshot["events"]
@pytest.mark.asyncio
async def test_list_events_does_not_write_final_json(outputdir):
gc = _get_gc()
final = outputdir / "final.json"
await gc.list_events(timeMin="2025-01-01T00:00:00Z", timeMax="2025-12-31T23:59:59Z")
assert not final.exists(), "final.json must NOT be written after a read-only tool"
@pytest.mark.asyncio
async def test_search_events_does_not_write_final_json(outputdir):
gc = _get_gc()
final = outputdir / "final.json"
await gc.search_events(query="test")
assert not final.exists(), "final.json must NOT be written after a read-only tool"
@pytest.mark.asyncio
async def test_no_final_path_skips_snapshot():
gc = _get_gc()
state = _get_state()
state.set_snapshot_paths(final_path=None)
result = await gc.create_event(
summary="No Output",
start={"dateTime": "2025-06-01T10:00:00Z"},
end={"dateTime": "2025-06-01T11:00:00Z"},
)
assert result["event"]["summary"] == "No Output"
docker/packages/google_calendar/tests/test_utils.py
+47
View File
@@ -0,0 +1,47 @@
"""Tests for google_calendar utility functions."""
import json
from pathlib import Path
from google_calendar.utils import get_calendar_data_path, load_events_from_json
class TestGetCalendarDataPath:
def test_computes_external_services_path(self):
result = get_calendar_data_path("/workspace/dumps/workspace")
assert result == Path("/workspace/dumps/external_services/calendar_data.json")
def test_path_is_sibling_to_workspace(self):
result = get_calendar_data_path("/a/b/workspace")
assert result.parent == Path("/a/b/external_services")
class TestLoadEventsFromJson:
def test_loads_events_and_adds_timestamps(self, tmp_path):
data = {"events": {"evt1": {"summary": "Meeting"}}}
json_file = tmp_path / "calendar.json"
json_file.write_text(json.dumps(data))
result = load_events_from_json(json_file)
assert "evt1" in result["events"]
assert result["events"]["evt1"]["id"] == "evt1"
assert "created" in result["events"]["evt1"]
assert "updated" in result["events"]["evt1"]
def test_wraps_bare_dict_in_events(self, tmp_path):
data = {"evt1": {"summary": "Meeting"}}
json_file = tmp_path / "calendar.json"
json_file.write_text(json.dumps(data))
result = load_events_from_json(json_file)
assert "events" in result
assert "evt1" in result["events"]
def test_preserves_existing_timestamps(self, tmp_path):
data = {"events": {"evt1": {"summary": "Meeting", "created": "2024-01-01", "updated": "2024-01-01"}}}
json_file = tmp_path / "calendar.json"
json_file.write_text(json.dumps(data))
result = load_events_from_json(json_file)
assert result["events"]["evt1"]["created"] == "2024-01-01"
docker/packages/google_mail/CAPABILITIES.md
+47
View File
@@ -0,0 +1,47 @@
# Google Mail Capabilities
A mock email server with a realistic closed-world simulation. The agent has one or more mailboxes, each with contacts, folders, and emails. Only contacts in the address book can receive messages — sending to unknown addresses fails with a bounce. Supports multi-mailbox worlds where the agent can operate across multiple personas.
## What the agent can do
**Read and search email.** Browse by folder, read individual emails (which marks them as read), and search across all emails by keyword. Search supports Gmail-style operators: `from:`, `to:`, `cc:`, `bcc:`, `subject:`, `has:attachment`, `filename:`, `is:unread`, `is:read`, `is:important`, `in:`, `label:`, `before:`, `after:`, `newer_than:`, `older_than:`, `-term` (exclusion), and adjacent-term `OR`. Parenthesized boolean grouping is not supported. Check unread counts per folder and overall mailbox statistics.
**Send, reply, and forward.** Compose new emails to any contact, reply to existing threads (with proper Re: prefixes and quoted text), forward emails to other contacts. Replies maintain threading via message ID references. Sent mail automatically also gets an INBOX copy when the sender is also a recipient (self-send).
**Manage contacts.** List all contacts, search by name or email, add new contacts (individual or group), edit contact details, and delete contacts. Group contacts have member lists — sending to a group delivers to all members.
**Organize with folders.** View all folders with message counts, create custom folders, move emails between folders (including in bulk), and delete custom folders (moves their emails back to INBOX). System folders (INBOX, Sent, Drafts, Trash, Scheduled) cannot be deleted.
**Work with drafts.** Save draft emails, view all drafts, update draft content, and delete drafts.
**Schedule emails.** Schedule an email for future delivery with a specific date/time, view all scheduled emails, and cancel scheduled sends. Since this is a mock, scheduled emails are stored but not actually delivered at the scheduled time.
**Attachments and utilities.** Download attachments from emails, get technical email headers (message IDs, routing info).
**Multi-mailbox.** When a world defines multiple mailboxes, `list_mailboxes` returns all available mailbox IDs + email addresses, and every tool accepts an optional `mailbox_id` argument (defaults to `"default"`). Each mailbox's contacts, folders, and emails are isolated. State round-trips through a `{"mailboxes": {id: {...}}}` wrapper.
## Coverage gaps
- No email rules or filters (auto-sort, auto-reply)
- No rich text composition (HTML editor)
- No email templates
- Scheduled emails are stored but do not actually fire
## Toolsets
28 tools total. Toolsets map to `WORLDBENCH_TOOL_SETS` values (prefixed form — e.g., `google_mail_core`).
| Toolset | Tools | Description |
|---------|-------|-------------|
| `all` / `google_mail_all` | 28 | Everything |
| `read` / `google_mail_read` | 11 | All read-only tools |
| `write` / `google_mail_write` | 17 | All write tools |
| `google_mail_core` | 20 | Basic inbox plus legacy Toolathlon mail/folder/draft operations |
| `google_mail_contacts` | 5 | Contact management: get, search, add, edit, delete |
| `google_mail_folders` | 4 | Folder organization: get, create, delete, move emails |
| `google_mail_drafts` | 4 | Drafts: get, save, update, delete |
| `google_mail_scheduling` | 3 | Schedule, list scheduled, cancel scheduled |
| `google_mail_toolathlon_legacy` | 20 | Legacy Toolathlon tool subset (pre-integration) |
| `google_mail_state` | 2 | `export_state`, `import_state` for fixture seeding and grading |
**Multi-mailbox** worlds round-trip through `export_state` / `import_state` under a `{"mailboxes": {mailbox_id: {...}}}` wrapper; single-mailbox worlds use the flat `MailboxData` shape.
docker/packages/google_mail/README.md
+319
View File
@@ -0,0 +1,319 @@
# Google Mail MCP
A mock Gmail MCP (Model Context Protocol) server for testing and RL environment training. It simulates a complete email system without requiring any network connectivity - all state is loaded from and persisted to a JSON file.
## Overview
This MCP server provides 20 email tools that operate on a "closed-world" simulation:
- **Closed-world contacts**: Only predefined email addresses can receive messages
- **Bounce simulation**: Emails to invalid addresses generate realistic bounce notifications
- **Group support**: Sending to a group delivers copies to members (including yourself if you're a member)
- **State persistence**: All changes are saved back to the JSON file
- **No network required**: Everything runs locally from a JSON file
## Project Structure
```
packages/google_mail/
├── google_mail.py # Entry point for container environment
├── utils.py # Utilities for data path computation
├── __init__.py # Package exports
├── src/mail_mcp/ # Core MCP server implementation
│ ├── server.py # FastMCP server with tool definitions
│ ├── models/ # Pydantic schemas
│ └── services/ # Mailbox business logic
├── examples/
│ └── sample_mailbox.json
└── tests/
```
## Installation
### Local Development
```bash
cd packages/google_mail
uv sync
```
### Container Environment
The MCP server is pre-installed in the DAT container. Data is stored in `external_services/mailbox.json` next to the agent workspace (outside the agent's filesystem access).
## Usage
### Running Locally
```bash
# Using environment variable
MAIL_MCP_DATA_PATH=/path/to/mailbox.json uv run mail-mcp
# Using CLI argument
uv run mail-mcp --data-path /path/to/mailbox.json
# With debug logging
uv run mail-mcp --data-path /path/to/mailbox.json --debug
```
### Container Usage (via MCP Config)
The server is configured in `configs/mcp_servers/google_mail.yaml`:
```yaml
type: stdio
name: google_mail
params:
command: uv
args:
- "run"
- "--project"
- "/workspace/packages/google_mail"
- "python"
- "/workspace/packages/google_mail/google_mail.py"
- "--agent-workspace"
- "${agent_workspace}"
```
### Task Preprocessing
To set up mailbox data for a task, use the utility functions:
```python
from google_mail import create_mail_data
from pathlib import Path
# Copy mailbox.json to external_services location
create_mail_data(
agent_workspace="/workspace/dumps/workspace",
source_mailbox_path=Path("initial_workspace/mailbox.json"),
)
```
### MCP Client Configuration
For direct MCP client usage:
```json
{
"mcpServers": {
"mail": {
"command": "uv",
"args": ["run", "--project", "/path/to/google_mail", "mail-mcp"],
"env": {
"MAIL_MCP_DATA_PATH": "/path/to/mailbox.json"
}
}
}
}
```
## Tools
### Email Operations
| Tool | Description |
|------|-------------|
| `mail_get_emails` | Get paginated emails from a folder |
| `mail_read_email` | Read a single email (marks as read) |
| `mail_search_emails` | Search emails by query string, including Gmail-style operators such as `from:`, `subject:`, `has:attachment`, `filename:`, `is:unread`, and `in:` |
| `mail_send_email` | Send an email |
| `mail_reply_email` | Reply to an email |
| `mail_forward_email` | Forward an email with attachments |
| `mail_delete_emails` | Delete one or more emails (moves to Trash or permanent) |
| `mail_move_emails` | Move one or more emails to a different folder |
| `mail_mark_emails` | Mark emails as read/unread/important |
### Folder Operations
| Tool | Description |
|------|-------------|
| `mail_get_folders` | List all folders with message counts |
| `mail_create_folder` | Create a new folder |
| `mail_delete_folder` | Delete a folder (system folders protected) |
| `mail_get_unread_count` | Get unread counts per folder |
| `mail_get_mailbox_stats` | Get statistics for all folders |
### Draft Operations
| Tool | Description |
|------|-------------|
| `mail_save_draft` | Save a new draft |
| `mail_get_drafts` | Get paginated list of drafts |
| `mail_update_draft` | Update an existing draft |
| `mail_delete_draft` | Delete a draft |
### Other
| Tool | Description |
|------|-------------|
| `mail_get_contacts` | List all valid contacts (closed-world) |
| `mail_download_attachment` | Download an attachment as base64 |
## Threading
Emails are linked into threads via the `in_reply_to` field, which references the parent email's `message_id`.
When using `mail_reply_email`:
1. The reply's `in_reply_to` is automatically set to the original's `message_id`
2. The reply body includes a quoted copy of the original message:
```
Your reply here...
--- Original Message ---
From: alice@example.com
Date: 2024-01-15 10:30
Subject: Meeting Tomorrow
Original message content...
```
This provides both programmatic thread linking (via `in_reply_to`) and human-readable context (via quoted original).
## JSON Schema
The mailbox data file follows this schema:
### Root Structure
```json
{
"mailbox": { ... },
"contacts": [ ... ],
"folders": [ ... ],
"emails": [ ... ],
"drafts": [ ... ],
"next_email_id": 1
}
```
### Fields
#### mailbox (required)
The identity of the mailbox owner. This is the "From" address when sending emails.
```json
{
"email": "user@example.com",
"name": "Display Name"
}
```
#### contacts (required)
List of valid email recipients. Only these addresses (plus the mailbox owner) can receive emails. Sending to any other address will generate a bounce notification.
```json
[
{"email": "alice@example.com", "name": "Alice Smith"},
{"email": "team@example.com", "name": "Engineering Team", "members": [
"user@example.com",
"alice@example.com"
]}
]
```
Groups have a `members` array. When you send to a group and you're a member, you receive a copy in your INBOX.
#### folders (optional)
Custom folders beyond the system defaults. System folders (INBOX, Sent, Drafts, Trash) always exist implicitly.
```json
[
{"name": "Work"},
{"name": "Archive/2024"}
]
```
#### emails (optional)
List of emails in the mailbox.
```json
[
{
"email_id": "1",
"folder": "INBOX",
"subject": "Meeting Tomorrow",
"from_addr": "alice@example.com",
"to_addr": "user@example.com",
"cc_addr": null,
"bcc_addr": null,
"date": "2024-01-15T10:30:00Z",
"message_id": "<msg001@example.com>",
"in_reply_to": null,
"body_text": "Plain text content here",
"body_html": "<p>Optional HTML content</p>",
"is_read": false,
"is_important": false,
"attachments": [
{
"filename": "document.pdf",
"content_type": "application/pdf",
"content_base64": "JVBERi0xLjQK..."
}
]
}
]
```
The `in_reply_to` field links replies to their parent email via `message_id`. This enables thread tracking.
#### drafts (optional)
List of draft emails.
```json
[
{
"draft_id": "draft_1",
"subject": "Draft Subject",
"body": "Draft content",
"html_body": null,
"to": "alice@example.com",
"cc": null,
"bcc": null,
"created_at": "2024-01-15T09:00:00Z",
"updated_at": "2024-01-15T09:30:00Z"
}
]
```
#### next_email_id (optional)
Counter for generating new email IDs. Defaults to 1 if not specified.
## Examples
### Minimal Empty Mailbox
```json
{
"mailbox": {
"email": "user@example.com",
"name": "Test User"
},
"contacts": [],
"folders": [],
"emails": [],
"drafts": [],
"next_email_id": 1
}
```
### Populated Mailbox
See `examples/sample_mailbox.json` for a complete example with contacts, emails, groups, drafts, and attachments.
## Development
### Running Tests
```bash
cd packages/google_mail
uv run pytest
```
### Code Quality
```bash
uv run ruff check .
uv run mypy .
```
docker/packages/google_mail/mcp-tools.generated.json
+2978
View File
File diff suppressed because it is too large Load Diff
docker/packages/google_mail/mcp.json
+141
View File
@@ -0,0 +1,141 @@
{
"run": {
"command": "python",
"args": [
"-m",
"google_mail"
]
},
"toolsets": {
"read": [
"download_attachment",
"get_contacts",
"get_groups",
"get_drafts",
"get_emails",
"get_folders",
"get_mailbox_stats",
"get_scheduled_emails",
"get_unread_count",
"list_mailboxes",
"search_contacts",
"search_groups",
"search_emails"
],
"write": [
"add_contact",
"add_group",
"cancel_scheduled_email",
"create_folder",
"delete_contact",
"delete_group",
"delete_draft",
"delete_emails",
"delete_folder",
"edit_contact",
"edit_group",
"forward_email",
"mark_emails",
"move_emails",
"read_email",
"reply_email",
"save_draft",
"schedule_email",
"send_email",
"update_draft"
],
"core": [
"forward_email",
"get_emails",
"read_email",
"reply_email",
"search_emails",
"send_email",
"create_folder",
"delete_draft",
"delete_emails",
"delete_folder",
"download_attachment",
"get_contacts",
"get_drafts",
"get_folders",
"get_mailbox_stats",
"get_unread_count",
"mark_emails",
"move_emails",
"save_draft",
"update_draft"
],
"messages": [
"delete_emails",
"download_attachment",
"forward_email",
"get_emails",
"get_mailbox_stats",
"get_unread_count",
"list_mailboxes",
"mark_emails",
"move_emails",
"read_email",
"reply_email",
"search_emails",
"send_email"
],
"contacts": [
"add_contact",
"add_group",
"delete_contact",
"delete_group",
"edit_contact",
"edit_group",
"get_contacts",
"get_groups",
"search_contacts",
"search_groups"
],
"folders": [
"create_folder",
"delete_folder",
"get_folders",
"move_emails"
],
"drafts": [
"delete_draft",
"get_drafts",
"save_draft",
"update_draft"
],
"scheduling": [
"cancel_scheduled_email",
"get_scheduled_emails",
"schedule_email"
],
"toolathlon_legacy": [
"create_folder",
"delete_draft",
"delete_emails",
"delete_folder",
"download_attachment",
"forward_email",
"get_contacts",
"get_groups",
"get_drafts",
"get_emails",
"get_folders",
"get_mailbox_stats",
"get_unread_count",
"mark_emails",
"move_emails",
"read_email",
"reply_email",
"save_draft",
"search_emails",
"send_email",
"update_draft"
],
"state": [
"export_state",
"import_state"
]
}
}
docker/packages/google_mail/pyproject.toml
+45
View File
@@ -0,0 +1,45 @@
[build-system]
build-backend = "uv_build"
requires = [ "uv-build>=0.11,<0.12" ]
[project]
name = "google-mail"
version = "0.1.0"
description = "Mock email MCP server for RL environment training"
readme = "README.md"
requires-python = ">=3.13"
classifiers = [
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
dependencies = [
"fastmcp>=3,<4",
"pydantic[email]>=2.12.5",
]
scripts.google-mail = "google_mail:main"
[dependency-groups]
dev = [
"mypy>=1.19.1",
"pytest>=9.0.2",
"pytest-asyncio>=1.3",
"ruff>=0.14.14",
]
[tool.ruff]
target-version = "py313"
line-length = 120
lint.select = [ "B", "E", "F", "I", "PLW", "SIM", "UP" ]
lint.ignore = [ "E501", "PLW0603" ]
lint.isort.known-first-party = [ "google_mail" ]
[tool.mypy]
python_version = "3.13"
strict = true
warn_return_any = true
warn_unused_configs = true
[tool.pytest]
ini_options.testpaths = [ "tests" ]
ini_options.asyncio_mode = "auto"
docker/packages/google_mail/src/google_mail/__init__.py
+43
View File
@@ -0,0 +1,43 @@
"""Google Mail mock MCP server for RL environment training."""
from __future__ import annotations
import argparse
import logging
import os
from .server import mcp
__all__ = ["main", "mcp"]
def main():
parser = argparse.ArgumentParser(description="Google Mail MCP Server")
parser.add_argument(
"--debug",
action="store_true",
help="Enable debug logging",
)
args = parser.parse_args()
logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)
from .async_tool_guard import assert_tools_async
assert_tools_async(mcp)
from google_mail.server import init_state
init_state()
port = os.environ.get("PORT")
if port:
from google_mail.viewer import run_http_server
run_http_server(mcp, int(port))
else:
mcp.run()
if __name__ == "__main__":
main()
docker/packages/google_mail/src/google_mail/__main__.py
+5
View File
@@ -0,0 +1,5 @@
"""Entry point for ``python -m google_mail``."""
from google_mail import main
main()
docker/packages/google_mail/src/google_mail/async_tool_guard.py
+59
View File
@@ -0,0 +1,59 @@
"""Boot-time guard: every registered MCP tool must be async.
Sync (`def`) tools make FastMCP run pydantic argument validation in an anyio
worker threadpool. Under concurrent calls the shared pydantic-core validator is
re-entered across threads and panics with ``pyo3_runtime.PanicException:
dictionary changed size during iteration``, which tears down the
StreamableHTTP task group and turns every later request into a 500. Async
(`async def`) tools validate on the single-threaded event loop and are safe.
This guard runs at server boot (and therefore during ``mcp-proxy gen``, which
boots every server): a non-conformant package fails fast instead of shipping a
server that can crash under load. The same check is intentionally duplicated in
each package because the bundled prod images install only that package's own
dependencies, so there is no shared runtime module to import.
"""
from __future__ import annotations
import asyncio
import inspect
from functools import partial
from typing import Any
def _unwrap(fn: Any) -> Any:
while isinstance(fn, partial):
fn = fn.func
return fn
def find_sync_tools(mcp: Any) -> list[str]:
"""Return the names of registered tools whose function is not a coroutine."""
# mcp.server.fastmcp.FastMCP exposes a synchronous tool manager.
manager = getattr(mcp, "_tool_manager", None)
if manager is not None and hasattr(manager, "list_tools"):
return sorted(t.name for t in manager.list_tools() if not inspect.iscoroutinefunction(_unwrap(t.fn)))
# fastmcp.FastMCP exposes an async API.
async def _collect() -> list[str]:
sync: list[str] = []
for descriptor in await mcp.list_tools():
tool = await mcp.get_tool(descriptor.name)
if not inspect.iscoroutinefunction(_unwrap(tool.fn)):
sync.append(descriptor.name)
return sorted(sync)
return asyncio.run(_collect())
def assert_tools_async(mcp: Any) -> None:
"""Raise if any registered tool is synchronous."""
sync = find_sync_tools(mcp)
if sync:
raise RuntimeError(
"MCP tools must be async (`async def`). Sync tools run pydantic argument "
"validation in a worker threadpool and can trigger a pydantic-core "
"'dictionary changed size during iteration' panic under concurrent calls, "
"which kills the server. Make these tools async: " + ", ".join(sync)
)
docker/packages/google_mail/src/google_mail/models.py
+342
View File
@@ -0,0 +1,342 @@
"""Schema models for mock mailbox data."""
from __future__ import annotations
import base64
import binascii
from datetime import datetime
from typing import Annotated, Self
from pydantic import (
AliasChoices,
BaseModel,
ConfigDict,
EmailStr,
Field,
TypeAdapter,
ValidationInfo,
field_validator,
model_validator,
)
_EMAIL_ADAPTER: TypeAdapter[EmailStr] = TypeAdapter(EmailStr)
def _allow_list_of_strings(schema: dict) -> None:
"""Widen a string-typed JSON schema to also advertise ``array[string]``.
Pydantic emits the declared type only; our ``_coerce_addr_list`` validator
accepts lists at runtime, so the advertised schema must match or strict MCP
clients will reject list payloads before they reach the validator.
"""
array_variant = {"type": "array", "items": {"type": "string", "format": "email"}}
if "anyOf" in schema:
schema["anyOf"].append(array_variant)
else:
schema.pop("type", None)
schema["anyOf"] = [{"type": "string"}, array_variant]
def _validate_email_address(value: str) -> str:
"""Validate and normalize a single email address."""
return str(_EMAIL_ADAPTER.validate_python(value))
def _validate_address_list(value: str) -> str:
"""Validate and normalize a comma-separated email address list."""
addresses = [_validate_email_address(address.strip()) for address in value.split(",") if address.strip()]
if not addresses:
raise ValueError("At least one email address is required")
return ", ".join(addresses)
class StrictBaseStateModel(BaseModel):
"""Base model for canonical persisted Google Mail state.
Keep Pydantic's normal JSON coercions, such as parsing datetime strings,
but reject fields that are not part of the declared state contract.
"""
model_config = ConfigDict(extra="forbid")
NonEmptyStateString = Annotated[str, Field(min_length=1)]
INCOMPLETE_RECIPIENT_FOLDERS: frozenset[str] = frozenset({"Drafts", "Trash"})
class Mailbox(StrictBaseStateModel):
"""Identity of the mailbox owner."""
email: EmailStr = Field(..., description="Email address of the mailbox owner")
name: str = Field(..., description="Display name of the mailbox owner")
class Contact(StrictBaseStateModel):
"""A valid email contact in the closed-world simulation."""
email: EmailStr = Field(..., description="Email address of the contact")
name: str = Field(..., description="Display name of the contact")
class ContactGroup(StrictBaseStateModel):
"""An addressable email group in the closed-world simulation."""
email: EmailStr = Field(..., description="Email address of the group")
name: str = Field(..., description="Display name of the group")
members: list[EmailStr] = Field(..., min_length=1, description="Contact emails included in the group")
class Folder(StrictBaseStateModel):
"""A custom email folder."""
name: NonEmptyStateString = Field(..., description="Folder name")
class Attachment(StrictBaseStateModel):
"""An email attachment with base64-encoded content."""
filename: NonEmptyStateString = Field(..., description="Filename of the attachment")
content_type: str = Field(..., description="MIME type")
content_base64: str = Field(
...,
description="Base64-encoded file content",
json_schema_extra={"contentEncoding": "base64"},
)
@field_validator("content_base64")
@classmethod
def validate_content_base64(cls, value: str) -> str:
try:
base64.b64decode(value, validate=True)
except (binascii.Error, ValueError) as exc:
raise ValueError("Attachment content_base64 must be valid base64") from exc
return value
@property
def size(self) -> int:
"""Return the decoded size of the attachment in bytes."""
return len(base64.b64decode(self.content_base64))
class Email(StrictBaseStateModel):
"""An email message in the mailbox."""
email_id: NonEmptyStateString = Field(..., description="Unique email identifier")
folder: NonEmptyStateString = Field(..., description="Folder containing the email")
subject: str = Field(..., description="Email subject line")
from_addr: EmailStr = Field(..., description="Sender email address")
to_addr: str = Field(
...,
description="Recipient(s), comma-separated string or list of strings",
json_schema_extra=_allow_list_of_strings,
)
cc_addr: str | None = Field(
default=None,
description="CC recipients, comma-separated string or list of strings",
json_schema_extra=_allow_list_of_strings,
)
bcc_addr: str | None = Field(
default=None,
description="BCC recipients, comma-separated string or list of strings",
json_schema_extra=_allow_list_of_strings,
)
date: datetime = Field(..., description="Date/time the email was sent")
message_id: NonEmptyStateString = Field(..., description="RFC 2822 Message-ID")
in_reply_to: str | None = Field(default=None, description="Message-ID of the email this is replying to")
body_text: str = Field(..., description="Plain text body")
body_html: str | None = Field(default=None, description="HTML body")
is_read: bool = Field(default=False, description="Whether the email has been read")
is_important: bool = Field(default=False, description="Whether the email is marked important")
labels: list[str] = Field(
default_factory=list,
description="Search labels attached to the email",
validation_alias=AliasChoices("labels", "labelIds", "label_ids"),
)
attachments: list[Attachment] = Field(default_factory=list, description="Email attachments")
scheduled_time: datetime | None = Field(default=None, description="Scheduled send time (None if not scheduled)")
# Synthetic-data generators often emit ["a@x", "b@y"] for recipient fields
# even though the canonical form is "a@x, b@y"; accept both on import.
@field_validator("to_addr", "cc_addr", "bcc_addr", mode="before")
@classmethod
def _coerce_addr_list(cls, value: object) -> object:
if isinstance(value, list):
return ", ".join(str(a).strip() for a in value if a and str(a).strip())
return value
@field_validator("to_addr", "cc_addr", "bcc_addr")
@classmethod
def validate_addr_list(cls, value: str | None, info: ValidationInfo) -> str | None:
if value is None:
return None
if not value.strip():
if info.field_name == "to_addr":
return ""
return None
return _validate_address_list(value)
@field_validator("labels")
@classmethod
def normalize_labels(cls, value: list[str]) -> list[str]:
normalized: list[str] = []
seen: set[str] = set()
for label in value:
stripped = str(label).strip()
if not stripped or stripped in seen:
continue
normalized.append(stripped)
seen.add(stripped)
return normalized
@model_validator(mode="after")
def validate_attachment_filenames_are_unique(self) -> Self:
"""Ensure attachment filename lookups are unambiguous within an email."""
filenames = [attachment.filename for attachment in self.attachments]
if len(filenames) != len(set(filenames)):
raise ValueError("Duplicate attachment filenames")
return self
@model_validator(mode="after")
def validate_scheduled_time_matches_folder(self) -> Self:
"""Keep scheduled_time consistent with the Scheduled folder."""
if self.folder == "Scheduled" and self.scheduled_time is None:
raise ValueError("Scheduled emails must have scheduled_time")
if self.folder != "Scheduled" and self.scheduled_time is not None:
raise ValueError("Only Scheduled emails may have scheduled_time")
return self
@model_validator(mode="after")
def validate_recipient_presence_matches_folder(self) -> Self:
"""Allow incomplete drafts/trash, but require recipients for active messages."""
if self.folder not in INCOMPLETE_RECIPIENT_FOLDERS and not any(
[self.to_addr.strip(), self.cc_addr, self.bcc_addr]
):
raise ValueError("Active emails require at least one recipient")
return self
SYSTEM_FOLDERS: frozenset[str] = frozenset({"INBOX", "Sent", "Drafts", "Trash", "Scheduled"})
MailboxId = Annotated[str, Field(min_length=1, description="Mailbox identifier")]
class MailboxData(StrictBaseStateModel):
"""Root schema for mock mailbox data."""
mailbox: Mailbox = Field(..., description="Identity of the mailbox owner")
contacts: list[Contact] = Field(default_factory=list, description="Valid contacts")
groups: list[ContactGroup] = Field(default_factory=list, description="Addressable contact groups")
folders: list[Folder] = Field(default_factory=list, description="Custom folders")
emails: list[Email] = Field(default_factory=list, description="All emails (including drafts in Drafts folder)")
next_email_id: int = Field(default=1, ge=1, description="Next email ID counter")
@model_validator(mode="after")
def validate_contacts_and_groups_have_unique_emails(self) -> Self:
"""Ensure no duplicate address book entries across contacts and groups."""
emails = [c.email.lower() for c in self.contacts]
emails.extend(group.email.lower() for group in self.groups)
if len(emails) != len(set(emails)):
raise ValueError("Duplicate email addresses in contacts or groups")
return self
@model_validator(mode="after")
def validate_folders_have_unique_names(self) -> Self:
"""Ensure custom folder names are unique."""
folder_names = [f.name for f in self.folders]
if len(folder_names) != len(set(folder_names)):
raise ValueError("Duplicate folder names")
return self
@model_validator(mode="after")
def validate_folders_are_custom_only(self) -> Self:
"""Ensure fixed system folders are not persisted as custom folders."""
system_folders = sorted({folder.name for folder in self.folders if folder.name in SYSTEM_FOLDERS})
if system_folders:
raise ValueError(f"System folders must not be listed as custom folders: {', '.join(system_folders)}")
return self
@model_validator(mode="after")
def validate_emails_have_unique_ids(self) -> Self:
"""Ensure every email has a unique ID."""
email_ids = [e.email_id for e in self.emails]
if len(email_ids) != len(set(email_ids)):
raise ValueError("Duplicate email IDs")
return self
@model_validator(mode="after")
def validate_next_email_id_is_unused(self) -> Self:
"""Ensure the numeric ID counter will not collide with imported emails."""
numeric_email_ids = [int(email.email_id) for email in self.emails if email.email_id.isdecimal()]
if numeric_email_ids and self.next_email_id <= max(numeric_email_ids):
raise ValueError("next_email_id must be greater than all numeric email IDs")
return self
@model_validator(mode="after")
def validate_email_folders_exist(self) -> Self:
"""Ensure every email references a known system or custom folder."""
folder_names = self.get_all_folder_names()
unknown_folders = sorted({email.folder for email in self.emails if email.folder not in folder_names})
if unknown_folders:
raise ValueError(f"Email references unknown folders: {', '.join(unknown_folders)}")
return self
@model_validator(mode="after")
def validate_group_members(self) -> Self:
"""Ensure groups reference known person contacts or the mailbox owner."""
valid_member_emails = {self.mailbox.email.lower()} | {contact.email.lower() for contact in self.contacts}
for group in self.groups:
normalized_members = [member.strip().lower() for member in group.members]
if any(not member for member in normalized_members):
raise ValueError(f"Group has empty members: {group.email}")
if len(normalized_members) != len(set(normalized_members)):
raise ValueError(f"Group has duplicate members: {group.email}")
if group.email.lower() in normalized_members:
raise ValueError(f"Group cannot include itself as a member: {group.email}")
unknown_members = sorted(set(normalized_members) - valid_member_emails)
if unknown_members:
raise ValueError(f"Group references unknown members for {group.email}: {', '.join(unknown_members)}")
return self
def get_all_folder_names(self) -> set[str]:
"""Return all folder names (system + custom)."""
custom = {f.name for f in self.folders}
return set(SYSTEM_FOLDERS) | custom
def get_contact_by_email(self, email: str) -> Contact | None:
"""Find a contact by email address (case-insensitive)."""
for contact in self.contacts:
if contact.email.lower() == email.lower():
return contact
return None
def get_group_by_email(self, email: str) -> ContactGroup | None:
"""Find a group by email address (case-insensitive)."""
for group in self.groups:
if group.email.lower() == email.lower():
return group
return None
def is_valid_recipient(self, email: str) -> bool:
"""Check if an email address is a valid recipient (closed-world)."""
return (
email.lower() == self.mailbox.email.lower()
or self.get_contact_by_email(email) is not None
or self.get_group_by_email(email) is not None
)
def is_mailbox_member_of_group(self, group: ContactGroup) -> bool:
"""Check if the mailbox owner is a member of a group contact."""
mailbox_email = self.mailbox.email.lower()
return any(member.lower() == mailbox_email for member in group.members)
class MultiMailboxData(StrictBaseStateModel):
"""Root schema for multi-mailbox Google Mail state."""
mailboxes: dict[MailboxId, MailboxData] = Field(
...,
min_length=1,
description="Mailbox state keyed by mailbox identifier.",
)
type GoogleMailState = MailboxData | MultiMailboxData
docker/packages/google_mail/src/google_mail/server.py
+910
View File
@@ -0,0 +1,910 @@
"""Mock email MCP server."""
from __future__ import annotations
import functools
from fastmcp import FastMCP
from mcp.types import ToolAnnotations
from pydantic import EmailStr
from google_mail.models import GoogleMailState
from google_mail.state import init_state as init_mail_state
from google_mail.state import write_snapshots
from google_mail.tools import contacts, drafts, folders, messages, scheduling
from google_mail.tools import state as state_tools
from google_mail.tools.common import (
AttachmentFilename,
AttachmentPaths,
DraftId,
EmailId,
EmailIds,
FolderName,
GroupMembers,
MailboxIdArg,
PageNumber,
PageSize,
ScheduleTime,
)
def _snapshot_on_write(fn):
"""Decorator: dual-write the post-tool snapshot.
Writes ``<BUNDLE_OUTPUT_DIR>/state.json`` (per-service bundle subdir,
nested ``services/<name>/state.json`` layout) and the legacy
``final.json`` so consumers on either convention keep working.
"""
@functools.wraps(fn)
async def wrapper(*args, **kwargs):
result = await fn(*args, **kwargs)
write_snapshots()
return result
return wrapper
def init_state() -> None:
"""Eagerly initialize mailbox(es) and write the initial state snapshot."""
init_mail_state()
mcp = FastMCP("google_mail")
@mcp.tool(
name="list_mailboxes",
annotations=ToolAnnotations(readOnlyHint=True, idempotentHint=True),
)
async def mail_list_mailboxes() -> str:
"""List all available mailboxes.
Returns:
JSON with mailbox IDs, email addresses, and names.
"""
return await messages.list_mailboxes()
@mcp.tool(
name="get_emails",
annotations=ToolAnnotations(
title="Get Emails",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_get_emails(
folder: FolderName | None = None,
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Get emails from the mailbox, optionally filtered by folder.
Returns a paginated list of emails sorted by date (newest first).
Args:
folder: Folder to filter by (optional)
page: Page number (1-indexed)
page_size: Results per page (max 100)
Returns:
JSON with emails and pagination info.
"""
return await messages.get_emails(folder=folder, page=page, page_size=page_size, mailbox_id=mailbox_id)
@mcp.tool(
name="read_email",
annotations=ToolAnnotations(
title="Read Email",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_read_email(email_id: EmailId, mailbox_id: MailboxIdArg = "default") -> str:
"""Read an email and mark it as read.
Returns the full email content including body and attachments.
Args:
email_id: ID of the email to read
Returns:
JSON with full email details.
"""
return await messages.read_email(email_id=email_id, mailbox_id=mailbox_id)
@mcp.tool(
name="search_emails",
annotations=ToolAnnotations(
title="Search Emails",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_search_emails(
query: str,
folder: FolderName | None = None,
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Search emails by query string. Supports Gmail-style operators.
Bare words are ANDed together and matched across subject, body, from,
and to. Double-quoted segments require exact adjacency. Combine freely,
e.g. `from:alice after:2026/03/01 invoice`.
Operators:
from:alice match sender substring (alice@... hits this)
to:bob match to recipient substring (strict; use cc:/bcc: for other fields)
cc:alice / bcc:alice match cc/bcc recipient substring
subject:meeting match subject substring
has:attachment messages with one or more attachments
filename:agenda.pdf match attachment filename substring
is:unread state filters: unread, read, important
in:sent exact folder match
label:client exact label match when labels exist; otherwise folder-style match
"exact phrase" match a contiguous phrase
-term exclude messages containing term (works with operators, e.g. -from:bob)
invoice OR billing boolean OR (uppercase; lowercase 'or' is literal)
before:2026/04/01 messages strictly before date (YYYY/MM/DD or YYYY-MM-DD)
after:2026/04/01 messages on or after date
newer_than:7d messages within last N days (d), months (m = 30d), or years (y = 365d)
older_than:1y messages older than N d|m|y
Parenthesized boolean grouping is not supported; `OR` joins adjacent terms only.
Unsupported or malformed search syntax is reported in a `warnings` array
while preserving best-effort search behavior.
Args:
query: Search query (supports the operators above).
folder: Folder to limit search (optional).
page: Page number (1-indexed).
page_size: Results per page (max 100).
Returns:
JSON with matching emails and pagination info.
"""
return await messages.search_emails(
query=query, folder=folder, page=page, page_size=page_size, mailbox_id=mailbox_id
)
@mcp.tool(
name="send_email",
annotations=ToolAnnotations(
title="Send Email",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=False,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_send_email(
to: str,
subject: str,
body: str,
html_body: str | None = None,
cc: str | None = None,
bcc: str | None = None,
attachments: AttachmentPaths | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Send an email.
Recipients must be valid contacts in your address book.
If sending to a group you're a member of, you'll receive a copy.
Args:
to: Recipient(s), comma-separated
subject: Email subject
body: Plain text email body
html_body: HTML body (optional)
cc: CC recipients, comma-separated (optional)
bcc: BCC recipients, comma-separated (optional)
attachments: Paths to files to attach (optional)
Returns:
JSON with sent status and email summary.
"""
return await messages.send_email(
to=to,
subject=subject,
body=body,
html_body=html_body,
cc=cc,
bcc=bcc,
attachments=attachments,
mailbox_id=mailbox_id,
)
@mcp.tool(
name="reply_email",
annotations=ToolAnnotations(
title="Reply to Email",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=False,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_reply_email(
email_id: EmailId,
body: str,
html_body: str | None = None,
reply_all: bool = False,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Reply to an email.
Creates a reply with proper subject prefix and recipients.
Args:
email_id: ID of the email to reply to
body: Reply body text
html_body: HTML body (optional)
reply_all: Reply to all recipients
Returns:
JSON with sent status and reply email summary.
"""
return await messages.reply_email(
email_id=email_id, body=body, html_body=html_body, reply_all=reply_all, mailbox_id=mailbox_id
)
@mcp.tool(
name="forward_email",
annotations=ToolAnnotations(
title="Forward Email",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=False,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_forward_email(
email_id: EmailId,
to: str,
body: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Forward an email to one or more recipients.
Includes the original message content and any attachments.
Args:
email_id: ID of the email to forward
to: Recipient(s) to forward to, comma-separated
body: Additional message (optional)
Returns:
JSON with sent status and forwarded email summary.
"""
return await messages.forward_email(email_id=email_id, to=to, body=body, mailbox_id=mailbox_id)
@mcp.tool(
name="delete_emails",
annotations=ToolAnnotations(
title="Delete Emails",
readOnlyHint=False,
destructiveHint=True,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_delete_emails(
email_ids: EmailIds,
permanent: bool = False,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Delete one or more emails.
By default, moves them to Trash. Use permanent=True to skip Trash.
Deleting from Trash permanently deletes. Partial success is allowed —
emails that don't exist surface in the errors list but don't abort the batch.
Args:
email_ids: IDs of the emails to delete (pass a single-element list for one)
permanent: Permanently delete (skip Trash)
Returns:
JSON with per-id status and a deleted/failed count.
"""
return await messages.delete_emails(email_ids=email_ids, permanent=permanent, mailbox_id=mailbox_id)
@mcp.tool(
name="move_emails",
annotations=ToolAnnotations(
title="Move Emails",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_move_emails(
email_ids: EmailIds,
target_folder: FolderName,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Move one or more emails to a different folder.
Partial success is allowed — emails or folders that don't exist surface
in the errors list but don't abort the batch.
Args:
email_ids: IDs of the emails to move (pass a single-element list for one)
target_folder: Name of the target folder
Returns:
JSON with per-id status and a moved/failed count.
"""
return await messages.move_emails(email_ids=email_ids, target_folder=target_folder, mailbox_id=mailbox_id)
@mcp.tool(
name="mark_emails",
annotations=ToolAnnotations(
title="Mark Emails",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_mark_emails(
email_ids: EmailIds,
is_read: bool | None = None,
is_important: bool | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Mark emails as read/unread or important/not important.
Args:
email_ids: List of email IDs to mark
is_read: Set read status (optional)
is_important: Set important status (optional)
Returns:
JSON with number of emails updated.
"""
return await messages.mark_emails(
email_ids=email_ids, is_read=is_read, is_important=is_important, mailbox_id=mailbox_id
)
@mcp.tool(
name="get_folders",
annotations=ToolAnnotations(
title="Get Folders",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_get_folders(mailbox_id: MailboxIdArg = "default") -> str:
"""Get all folders with message counts.
Returns system folders (INBOX, Sent, Drafts, Trash) and custom folders.
Returns:
JSON with folder list including name, total, unread, is_system.
"""
return await folders.get_folders(mailbox_id=mailbox_id)
@mcp.tool(
name="create_folder",
annotations=ToolAnnotations(
title="Create Folder",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=False,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_create_folder(folder_name: FolderName, mailbox_id: MailboxIdArg = "default") -> str:
"""Create a new custom folder.
Args:
folder_name: Name of the folder to create
Returns:
JSON with creation status.
"""
return await folders.create_folder(folder_name=folder_name, mailbox_id=mailbox_id)
@mcp.tool(
name="delete_folder",
annotations=ToolAnnotations(
title="Delete Folder",
readOnlyHint=False,
destructiveHint=True,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_delete_folder(folder_name: FolderName, mailbox_id: MailboxIdArg = "default") -> str:
"""Delete a custom folder.
System folders (INBOX, Sent, Drafts, Trash) cannot be deleted.
Emails in the deleted folder are moved to INBOX.
Args:
folder_name: Name of the folder to delete
Returns:
JSON with deletion status.
"""
return await folders.delete_folder(folder_name=folder_name, mailbox_id=mailbox_id)
@mcp.tool(
name="get_unread_count",
annotations=ToolAnnotations(
title="Get Unread Count",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_get_unread_count(folder: FolderName | None = None, mailbox_id: MailboxIdArg = "default") -> str:
"""Get unread email count for folders.
Args:
folder: Specific folder (optional)
Returns:
JSON with unread counts per folder.
"""
return await messages.get_unread_count(folder=folder, mailbox_id=mailbox_id)
@mcp.tool(
name="get_mailbox_stats",
annotations=ToolAnnotations(
title="Get Mailbox Stats",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_get_mailbox_stats(mailbox_id: MailboxIdArg = "default") -> str:
"""Get overall mailbox statistics.
Returns owner info, total counts, and per-folder breakdown.
Returns:
JSON with comprehensive mailbox statistics.
"""
return await messages.get_mailbox_stats(mailbox_id=mailbox_id)
@mcp.tool(
name="save_draft",
annotations=ToolAnnotations(
title="Save Draft",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=False,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_save_draft(
subject: str = "",
body: str = "",
html_body: str | None = None,
to: str | None = None,
cc: str | None = None,
bcc: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Save a new email draft.
Args:
subject: Draft subject
body: Draft body
html_body: HTML body (optional)
to: Recipient(s), comma-separated (optional)
cc: CC recipient(s), comma-separated (optional)
bcc: BCC recipient(s), comma-separated (optional)
Returns:
JSON with saved draft details.
"""
return await drafts.save_draft(
subject=subject, body=body, html_body=html_body, to=to, cc=cc, bcc=bcc, mailbox_id=mailbox_id
)
@mcp.tool(
name="get_drafts",
annotations=ToolAnnotations(
title="Get Drafts",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_get_drafts(
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Get all drafts with pagination.
Args:
page: Page number (1-indexed)
page_size: Results per page (max 100)
Returns:
JSON with drafts and pagination info.
"""
return await drafts.get_drafts(page=page, page_size=page_size, mailbox_id=mailbox_id)
@mcp.tool(
name="update_draft",
annotations=ToolAnnotations(
title="Update Draft",
readOnlyHint=False,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_update_draft(
draft_id: DraftId,
subject: str | None = None,
body: str | None = None,
html_body: str | None = None,
to: str | None = None,
cc: str | None = None,
bcc: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Update an existing draft.
Only provided fields are updated; others remain unchanged.
Args:
draft_id: ID of the draft to update
subject: New subject (optional)
body: New body (optional)
html_body: New HTML body (optional)
to: New recipient(s), comma-separated (optional)
cc: New CC recipient(s), comma-separated (optional)
bcc: New BCC recipient(s), comma-separated (optional)
Returns:
JSON with updated draft details.
"""
return await drafts.update_draft(
draft_id=draft_id, subject=subject, body=body, html_body=html_body, to=to, cc=cc, bcc=bcc, mailbox_id=mailbox_id
)
@mcp.tool(
name="delete_draft",
annotations=ToolAnnotations(
title="Delete Draft",
readOnlyHint=False,
destructiveHint=True,
idempotentHint=True,
openWorldHint=False,
),
)
@_snapshot_on_write
async def mail_delete_draft(draft_id: DraftId, mailbox_id: MailboxIdArg = "default") -> str:
"""Delete a draft.
Args:
draft_id: ID of the draft to delete
Returns:
JSON with deletion status.
"""
return await drafts.delete_draft(draft_id=draft_id, mailbox_id=mailbox_id)
@mcp.tool(
name="get_contacts",
annotations=ToolAnnotations(
title="Get Contacts",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_get_contacts(mailbox_id: MailboxIdArg = "default") -> str:
"""Get all contacts.
Returns the person contacts in your address book. Groups are available
through get_groups.
Returns:
JSON with contact list.
"""
return await contacts.get_contacts(mailbox_id=mailbox_id)
@mcp.tool(
name="download_attachment",
annotations=ToolAnnotations(
title="Download Attachment",
readOnlyHint=True,
destructiveHint=False,
idempotentHint=True,
openWorldHint=False,
),
)
async def mail_download_attachment(
email_id: EmailId,
filename: AttachmentFilename,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Download an attachment from an email.
Returns the attachment content as base64-encoded data.
Args:
email_id: ID of the email
filename: Name of the attachment file
Returns:
JSON with attachment data.
"""
return await messages.download_attachment(email_id=email_id, filename=filename, mailbox_id=mailbox_id)
@mcp.tool(
name="search_contacts",
annotations=ToolAnnotations(readOnlyHint=True, idempotentHint=True),
)
async def mail_search_contacts(query: str, mailbox_id: MailboxIdArg = "default") -> str:
"""Search contacts by name or email address (case-insensitive).
Args:
query: Search string to match against contact name or email
"""
return await contacts.search_contacts(query=query, mailbox_id=mailbox_id)
@mcp.tool(
name="add_contact",
annotations=ToolAnnotations(readOnlyHint=False, idempotentHint=False),
)
@_snapshot_on_write
async def mail_add_contact(
email: EmailStr,
name: str,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Add a new contact to the address book.
Args:
email: Email address of the contact
name: Display name of the contact
"""
return await contacts.add_contact(email=email, name=name, mailbox_id=mailbox_id)
@mcp.tool(
name="edit_contact",
annotations=ToolAnnotations(readOnlyHint=False, idempotentHint=True),
)
@_snapshot_on_write
async def mail_edit_contact(
email: EmailStr,
name: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Update an existing contact's name.
Args:
email: Email address of the contact to update (lookup key)
name: New display name (optional, omit to keep current)
"""
return await contacts.edit_contact(email=email, name=name, mailbox_id=mailbox_id)
@mcp.tool(
name="delete_contact",
annotations=ToolAnnotations(readOnlyHint=False, destructiveHint=True, idempotentHint=True),
)
@_snapshot_on_write
async def mail_delete_contact(email: EmailStr, mailbox_id: MailboxIdArg = "default") -> str:
"""Remove a contact from the address book.
Args:
email: Email address of the contact to delete
"""
return await contacts.delete_contact(email=email, mailbox_id=mailbox_id)
@mcp.tool(
name="get_groups",
annotations=ToolAnnotations(readOnlyHint=True, idempotentHint=True),
)
async def mail_get_groups(mailbox_id: MailboxIdArg = "default") -> str:
"""Get all addressable contact groups."""
return await contacts.get_groups(mailbox_id=mailbox_id)
@mcp.tool(
name="search_groups",
annotations=ToolAnnotations(readOnlyHint=True, idempotentHint=True),
)
async def mail_search_groups(query: str, mailbox_id: MailboxIdArg = "default") -> str:
"""Search groups by name or email address (case-insensitive)."""
return await contacts.search_groups(query=query, mailbox_id=mailbox_id)
@mcp.tool(
name="add_group",
annotations=ToolAnnotations(readOnlyHint=False, idempotentHint=False),
)
@_snapshot_on_write
async def mail_add_group(
email: EmailStr,
name: str,
members: GroupMembers,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Add a new addressable contact group.
Args:
email: Email address of the group
name: Display name of the group
members: Contact emails included in the group
"""
return await contacts.add_group(email=email, name=name, members=members, mailbox_id=mailbox_id)
@mcp.tool(
name="edit_group",
annotations=ToolAnnotations(readOnlyHint=False, idempotentHint=True),
)
@_snapshot_on_write
async def mail_edit_group(
email: EmailStr,
name: str | None = None,
members: GroupMembers | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Update an existing contact group."""
return await contacts.edit_group(email=email, name=name, members=members, mailbox_id=mailbox_id)
@mcp.tool(
name="delete_group",
annotations=ToolAnnotations(readOnlyHint=False, destructiveHint=True, idempotentHint=True),
)
@_snapshot_on_write
async def mail_delete_group(email: EmailStr, mailbox_id: MailboxIdArg = "default") -> str:
"""Remove a contact group from the address book."""
return await contacts.delete_group(email=email, mailbox_id=mailbox_id)
@mcp.tool(
name="schedule_email",
annotations=ToolAnnotations(readOnlyHint=False, idempotentHint=False),
)
@_snapshot_on_write
async def mail_schedule_email(
to: str,
subject: str,
body: str,
scheduled_time: ScheduleTime,
html_body: str | None = None,
cc: str | None = None,
bcc: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Schedule an email for later delivery.
Args:
to: Recipient(s), comma-separated
subject: Email subject
body: Plain text email body
scheduled_time: ISO 8601 datetime for when to send (e.g., '2024-12-25T09:00:00Z')
html_body: HTML body (optional)
cc: CC recipient(s), comma-separated (optional)
bcc: BCC recipient(s), comma-separated (optional)
"""
return await scheduling.schedule_email(
to=to,
subject=subject,
body=body,
scheduled_time=scheduled_time,
html_body=html_body,
cc=cc,
bcc=bcc,
mailbox_id=mailbox_id,
)
@mcp.tool(
name="get_scheduled_emails",
annotations=ToolAnnotations(readOnlyHint=True, idempotentHint=True),
)
async def mail_get_scheduled_emails(
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
"""Get list of emails scheduled for later delivery."""
return await scheduling.get_scheduled_emails(page=page, page_size=page_size, mailbox_id=mailbox_id)
@mcp.tool(
name="cancel_scheduled_email",
annotations=ToolAnnotations(readOnlyHint=False, destructiveHint=True, idempotentHint=True),
)
@_snapshot_on_write
async def mail_cancel_scheduled_email(email_id: EmailId, mailbox_id: MailboxIdArg = "default") -> str:
"""Cancel a scheduled email. The email is permanently removed.
Args:
email_id: ID of the scheduled email to cancel
"""
return await scheduling.cancel_scheduled_email(email_id=email_id, mailbox_id=mailbox_id)
@mcp.tool(
name="export_state",
annotations=ToolAnnotations(readOnlyHint=True, idempotentHint=True),
)
async def export_state() -> GoogleMailState:
"""Export the full mailbox state as JSON.
Single-mailbox worlds emit ``MailboxData``; multi-mailbox worlds emit
``MultiMailboxData``. Round-trips with import_state.
"""
return await state_tools.export_state()
@mcp.tool(
name="import_state",
annotations=ToolAnnotations(readOnlyHint=False, idempotentHint=True),
)
@_snapshot_on_write
async def import_state(state: GoogleMailState) -> dict:
"""Replace the full mailbox state with the provided JSON.
Accepts either the flat MailboxData shape (loaded into the ``default``
mailbox) or the multi-mailbox wrapper (``{"mailboxes": {mailbox_id: ...}}``).
For synthetic-data injection and test setup.
"""
return await state_tools.import_state(state=state)
docker/packages/google_mail/src/google_mail/services/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Services for the mock email MCP."""
docker/packages/google_mail/src/google_mail/services/mailbox.py
+1602
View File
File diff suppressed because it is too large Load Diff
docker/packages/google_mail/src/google_mail/state.py
+292
View File
@@ -0,0 +1,292 @@
"""State registry, startup loading, and snapshots for the Google Mail mock."""
from __future__ import annotations
import json
import logging
import os
import tempfile
from pathlib import Path
from typing import Any, cast
from google_mail.models import GoogleMailState, MailboxData, MultiMailboxData
from google_mail.services.mailbox import MailboxService
_logger = logging.getLogger(__name__)
SERVICE_NAME = "google_mail"
_mailboxes: dict[str, MailboxService] = {}
_final_path: Path | None = None
_bundle_state_path: Path | None = None
_UNSET = object()
def resolve_bundle_state_paths() -> list[Path]:
"""Resolve the seed-state files inside this service's bundle subdir.
The folder ``<BUNDLEDIR>/services/<name>/`` is the unit: everything in it
is this service's seed. Prefer the canonical single-file ``state.json``
(the output round-trip shape); otherwise hand back ALL ``*.json`` in the
folder (the raw entities layout, e.g. one file per mailbox), read in
full the same way INPUTDIR files are.
"""
bundle_dir = os.environ.get("BUNDLEDIR")
if not bundle_dir:
return []
service_dir = Path(bundle_dir) / "services" / SERVICE_NAME
state_file = service_dir / "state.json"
if state_file.is_file():
return [state_file]
if service_dir.is_dir():
return sorted(service_dir.glob("*.json"))
return []
def resolve_bundle_state_path() -> Path | None:
"""Back-compat single-file view of :func:`resolve_bundle_state_paths`."""
paths = resolve_bundle_state_paths()
return paths[0] if paths else None
def resolve_bundle_output_path() -> Path | None:
output_dir = os.environ.get("BUNDLE_OUTPUT_DIR")
if not output_dir:
return None
return Path(output_dir) / "state.json"
def _merge_mailbox_into(target: dict[str, Any], source: dict[str, Any]) -> None:
"""Merge a flat mailbox seed into ``target``: lists extend, dicts update,
``next_email_id`` takes the max, remaining scalars overwrite."""
for key, value in source.items():
if key == "next_email_id":
target[key] = max(target.get(key, 0), value)
elif isinstance(value, list) and isinstance(target.get(key), list):
target[key].extend(value)
elif isinstance(value, dict) and isinstance(target.get(key), dict):
target[key].update(value)
else:
target[key] = value
def _temporary_json_path(suffix: str) -> Path:
"""Create and return a closed temporary JSON path."""
with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp:
return Path(tmp.name)
def get_mailboxes() -> dict[str, MailboxService]:
"""Return the live mailbox registry."""
return _mailboxes
def set_mailboxes(mailboxes: dict[str, MailboxService]) -> None:
"""Replace the live mailbox registry."""
_mailboxes.clear()
_mailboxes.update(mailboxes)
def get_mailbox(mailbox_id: str = "default") -> MailboxService:
"""Get a mailbox service instance by ID."""
if not _mailboxes:
raise RuntimeError("Mailbox service not initialized")
if mailbox_id not in _mailboxes:
available = ", ".join(sorted(_mailboxes.keys()))
raise ValueError(f"Mailbox '{mailbox_id}' not found. Available: {available}")
return _mailboxes[mailbox_id]
def set_snapshot_paths(
*,
final_path: Path | None | object = _UNSET,
bundle_state_path: Path | None | object = _UNSET,
) -> None:
"""Update post-write snapshot destinations.
Omitted arguments leave the existing path unchanged. Pass ``None``
explicitly to clear a path.
"""
global _final_path, _bundle_state_path
if final_path is not _UNSET:
_final_path = None if final_path is None else Path(cast(Path, final_path))
if bundle_state_path is not _UNSET:
_bundle_state_path = None if bundle_state_path is None else Path(cast(Path, bundle_state_path))
def get_final_path() -> Path | None:
"""Return the configured legacy final.json path."""
return _final_path
def get_bundle_state_path() -> Path | None:
"""Return the configured bundle snapshot path."""
return _bundle_state_path
def create_default_mailbox(data_path: Path) -> None:
"""Create a default empty mailbox file."""
default_email = os.environ.get("MAIL_MCP_DEFAULT_EMAIL", "agent@mail.com")
default_name = os.environ.get("MAIL_MCP_DEFAULT_NAME", "Agent")
default_data = {
"mailbox": {"email": default_email, "name": default_name},
"contacts": [],
"groups": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
data_path.parent.mkdir(parents=True, exist_ok=True)
with open(data_path, "w") as f:
json.dump(default_data, f, indent=2)
_logger.info("Created default empty mailbox at %s", data_path)
def dump_state(dest: Path, label: str) -> None:
"""Write a JSON snapshot of all mailboxes to *dest*."""
if not _mailboxes:
return
try:
dest.parent.mkdir(parents=True, exist_ok=True)
snapshot = state_to_json()
with open(dest, "w") as f:
json.dump(snapshot, f, indent=2, default=str)
_logger.info("Wrote %s state to %s", label, dest)
except Exception:
_logger.exception("Failed to write %s state to %s", label, dest)
def write_snapshots() -> None:
"""Write configured post-tool snapshots."""
if _bundle_state_path is not None:
dump_state(_bundle_state_path, "bundle")
if _final_path is not None:
dump_state(_final_path, "final")
def configure_snapshots_from_env(*, write_initial: bool = False) -> None:
"""Configure snapshot paths from OUTPUTDIR and BUNDLE_OUTPUT_DIR."""
outputdir = os.environ.get("OUTPUTDIR")
bundle_output_dir = os.environ.get("BUNDLE_OUTPUT_DIR")
_logger.info("OUTPUTDIR=%s BUNDLE_OUTPUT_DIR=%s", outputdir or "<unset>", bundle_output_dir or "<unset>")
bundle_state_path = resolve_bundle_output_path()
final_path = Path(outputdir) / "final.json" if outputdir else None
set_snapshot_paths(final_path=final_path, bundle_state_path=bundle_state_path)
if write_initial and bundle_state_path is not None:
dump_state(bundle_state_path, "bundle")
if write_initial and outputdir:
dump_state(Path(outputdir) / "initial.json", "initial")
def state_to_json() -> dict[str, Any]:
"""Return the full Google Mail state as a JSON-native dict."""
if len(_mailboxes) == 1 and "default" in _mailboxes:
return _mailboxes["default"].data.model_dump(mode="json")
return {"mailboxes": {mid: svc.data.model_dump(mode="json") for mid, svc in _mailboxes.items()}}
def export_state_model() -> GoogleMailState:
"""Return the full Google Mail state as a typed model."""
if len(_mailboxes) == 1 and "default" in _mailboxes:
return MailboxData.model_validate(_mailboxes["default"].to_json())
return MultiMailboxData(
mailboxes={mid: MailboxData.model_validate(svc.to_json()) for mid, svc in _mailboxes.items()}
)
def state_from_json(state: GoogleMailState | dict[str, Any]) -> None:
"""Full-replace the mailbox registry from a JSON-native dict or model."""
if isinstance(state, MailboxData | MultiMailboxData):
validated_state = state
elif "mailboxes" in state:
validated_state = MultiMailboxData.model_validate(state)
else:
validated_state = MailboxData.model_validate(state)
_mailboxes.clear()
if isinstance(validated_state, MultiMailboxData):
for mid, mdata in validated_state.mailboxes.items():
payload = mdata.model_dump(mode="json")
tmp = _temporary_json_path(suffix=f"_{mid}.json")
with open(tmp, "w") as f:
json.dump(payload, f, indent=2, default=str)
svc = MailboxService(tmp)
svc.from_json(payload, persist=True)
_mailboxes[mid] = svc
return
payload = validated_state.model_dump(mode="json")
tmp = _temporary_json_path(suffix="_default.json")
with open(tmp, "w") as f:
json.dump(payload, f, indent=2, default=str)
_mailboxes["default"] = MailboxService(tmp)
_mailboxes["default"].from_json(payload, persist=True)
def init_state() -> None:
"""Eagerly initialize mailbox(es) and write the initial state snapshot.
Supports two data formats:
- Single mailbox (backward compat): flat {mailbox, contacts, emails, ...}
- Multi-mailbox: {mailboxes: {id: {mailbox, contacts, emails, ...}, ...}}
"""
if _mailboxes:
configure_snapshots_from_env()
return
inputdir = os.environ.get("INPUTDIR")
# Bundle folder is coalesced (read in full); INPUTDIR keeps its legacy
# first-file-only contract (see scripts/validate_external_services.py).
bundle_files = resolve_bundle_state_paths()
inputdir_files: list[Path] = []
if not bundle_files and inputdir and Path(inputdir).is_dir():
inputdir_files = sorted(Path(inputdir).glob("*.json"))[:1]
json_files = bundle_files or inputdir_files
_logger.info(
"BUNDLEDIR=%s INPUTDIR=%s seed_files=%s",
os.environ.get("BUNDLEDIR") or "<unset>",
inputdir or "<unset>",
[str(p) for p in json_files] or "<none>",
)
if json_files:
# Coalesce the bundle folder: flat single-mailbox files merge into ONE
# default mailbox (the raw entities layout splits one mailbox
# across per-entity files, e.g. services/google_mail/inbox.json — those
# belong together, not in separate mailboxes). Files carrying a
# {mailboxes: {...}} wrapper contribute their named mailboxes. The
# legacy INPUTDIR path is a single file, so the loop is a no-op merge.
mailboxes_data: dict[str, dict[str, Any]] = {}
default_mailbox: dict[str, Any] = {}
for data_path in json_files:
_logger.info("Loading mailbox(es) from %s", data_path)
with open(data_path) as f:
raw = json.load(f)
if "mailboxes" in raw:
mailboxes_data.update(raw["mailboxes"])
else:
_merge_mailbox_into(default_mailbox, raw)
if default_mailbox:
mailboxes_data.setdefault("default", default_mailbox)
for mid, mdata in mailboxes_data.items():
tmp = _temporary_json_path(suffix=f"_{mid}.json")
with open(tmp, "w") as f:
json.dump(mdata, f, indent=2)
svc = MailboxService(tmp)
svc.load()
_mailboxes[mid] = svc
_logger.info("Loaded mailbox '%s' (%s)", mid, mdata.get("mailbox", {}).get("email", "?"))
else:
data_path = _temporary_json_path(suffix=".json")
create_default_mailbox(data_path)
if inputdir:
_logger.warning("INPUTDIR set but no .json files found in %s", inputdir)
svc = MailboxService(data_path)
svc.load()
_mailboxes["default"] = svc
_logger.info("Mail MCP server initialized with %d mailbox(es)", len(_mailboxes))
configure_snapshots_from_env(write_initial=True)
docker/packages/google_mail/src/google_mail/tools/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Domain handlers for Google Mail tools."""
docker/packages/google_mail/src/google_mail/tools/common.py
+145
View File
@@ -0,0 +1,145 @@
"""Shared helpers and public argument aliases for Google Mail tools."""
from __future__ import annotations
import json
from datetime import UTC, datetime
from typing import Annotated, Any
from pydantic import EmailStr, Field
from google_mail.models import Email
PageNumber = Annotated[int, Field(ge=1, description="Page number, starting at 1.")]
PageSize = Annotated[int, Field(ge=1, le=100, description="Number of results per page.")]
GroupMembers = Annotated[list[EmailStr], Field(min_length=1, description="Contact emails included in the group.")]
EmailId = Annotated[str, Field(min_length=1, description="Email ID.")]
EmailIds = Annotated[list[EmailId], Field(min_length=1, description="One or more email IDs.")]
DraftId = Annotated[str, Field(min_length=1, description="Draft ID.")]
FolderName = Annotated[str, Field(min_length=1, description="Folder name.")]
MailboxIdArg = Annotated[str, Field(min_length=1, description="Mailbox identifier.")]
AttachmentFilename = Annotated[str, Field(min_length=1, description="Attachment filename.")]
AttachmentPath = Annotated[str, Field(min_length=1, description="Path to a file to attach.")]
AttachmentPaths = Annotated[list[AttachmentPath], Field(min_length=1, description="Paths to files to attach.")]
ScheduleTime = Annotated[datetime, Field(description="ISO 8601 scheduled send time.")]
def error_response(error: str, **extra: Any) -> str:
"""Format a JSON error response."""
return json.dumps({"error": error, **extra})
def success_response(data: dict[str, Any]) -> str:
"""Format a JSON success response."""
return json.dumps(data, indent=2)
def batch_response(
*,
action: str,
requested_count: int,
succeeded_ids: list[str],
errors: list[dict[str, str]],
extra: dict[str, Any] | None = None,
) -> str:
"""Format a batch-action response with unambiguous aggregate status."""
succeeded_count = len(succeeded_ids)
failed_count = len(errors)
if succeeded_count == requested_count:
status = "all_succeeded"
summary = f"All {requested_count} attempts succeeded"
elif succeeded_count == 0:
status = "all_failed"
summary = f"All {requested_count} attempts failed"
else:
status = "partial_success"
summary = f"{succeeded_count} of {requested_count} attempts succeeded"
payload: dict[str, Any] = {
"status": status,
"summary": summary,
"action": action,
"requestedCount": requested_count,
"succeededCount": succeeded_count,
"failedCount": failed_count,
"succeededIds": succeeded_ids,
"errors": errors,
}
if extra:
payload.update(extra)
return success_response(payload)
def normalize_scheduled_time(value: datetime | str) -> datetime:
"""Normalize parsed or direct-call scheduled_time values."""
if isinstance(value, str):
value = datetime.fromisoformat(value)
if value.tzinfo is None:
value = value.replace(tzinfo=UTC)
return value
def format_email_summary(email: Email) -> dict[str, Any]:
"""Format an email for summary output."""
return {
"email_id": email.email_id,
"folder": email.folder,
"subject": email.subject,
"from": email.from_addr,
"to": email.to_addr,
"date": email.date.isoformat(),
"is_read": email.is_read,
"is_important": email.is_important,
"has_attachments": len(email.attachments) > 0,
}
def format_email_full(email: Email) -> dict[str, Any]:
"""Format an email for full output."""
result = format_email_summary(email)
result["cc"] = email.cc_addr
result["bcc"] = email.bcc_addr
result["message_id"] = email.message_id
result["in_reply_to"] = email.in_reply_to
result["body_text"] = email.body_text
result["body_html"] = email.body_html
result["attachments"] = [
{
"filename": a.filename,
"content_type": a.content_type,
"size": a.size,
}
for a in email.attachments
]
return result
def format_draft(draft: Email) -> dict[str, Any]:
"""Format a draft (email in Drafts folder) for output."""
return {
"draft_id": draft.email_id,
"subject": draft.subject,
"to": draft.to_addr,
"cc": draft.cc_addr,
"bcc": draft.bcc_addr,
"body": draft.body_text,
"html_body": draft.body_html,
"date": draft.date.isoformat(),
}
def format_contact(contact: Any) -> dict[str, Any]:
"""Format a contact for output."""
return {
"email": contact.email,
"name": contact.name,
}
def format_group(group: Any) -> dict[str, Any]:
"""Format a group for output."""
return {
"email": group.email,
"name": group.name,
"members": group.members,
}
docker/packages/google_mail/src/google_mail/tools/contacts.py
+144
View File
@@ -0,0 +1,144 @@
"""Contacts handlers for Google Mail tools."""
from __future__ import annotations
from pydantic import EmailStr
from google_mail.services.mailbox import (
ContactExistsError,
ContactInUseError,
ContactNotFoundError,
GroupExistsError,
GroupNotFoundError,
)
from google_mail.state import get_mailbox
from google_mail.tools.common import (
GroupMembers,
MailboxIdArg,
error_response,
format_contact,
format_group,
success_response,
)
async def get_contacts(mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
contacts = mailbox.get_contacts()
return success_response(
{
"contacts": [
{
"email": c.email,
"name": c.name,
}
for c in contacts
]
}
)
async def search_contacts(query: str, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
contacts = mailbox.search_contacts(query)
return success_response(
{
"contacts": [format_contact(c) for c in contacts],
"total": len(contacts),
}
)
async def add_contact(
email: EmailStr,
name: str,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
contact = mailbox.add_contact(email=email, name=name)
return success_response({"status": "created", "contact": format_contact(contact)})
except ContactExistsError as e:
return error_response(str(e))
async def edit_contact(
email: EmailStr,
name: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
contact = mailbox.edit_contact(email=email, name=name)
return success_response({"status": "updated", "contact": format_contact(contact)})
except ContactNotFoundError as e:
return error_response(str(e))
async def delete_contact(email: EmailStr, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
mailbox.delete_contact(email)
return success_response({"status": "deleted", "email": email})
except ContactNotFoundError as e:
return error_response(str(e))
except ContactInUseError as e:
return error_response(str(e))
async def get_groups(mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
groups = mailbox.get_groups()
return success_response({"groups": [format_group(group) for group in groups]})
async def search_groups(query: str, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
groups = mailbox.search_groups(query)
return success_response(
{
"groups": [format_group(group) for group in groups],
"total": len(groups),
}
)
async def add_group(
email: EmailStr,
name: str,
members: GroupMembers,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
group = mailbox.add_group(email=email, name=name, members=members)
return success_response({"status": "created", "group": format_group(group)})
except GroupExistsError as e:
return error_response(str(e))
except ValueError as e:
return error_response(str(e))
async def edit_group(
email: EmailStr,
name: str | None = None,
members: GroupMembers | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
group = mailbox.edit_group(email=email, name=name, members=members)
return success_response({"status": "updated", "group": format_group(group)})
except GroupNotFoundError as e:
return error_response(str(e))
except ValueError as e:
return error_response(str(e))
async def delete_group(email: EmailStr, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
mailbox.delete_group(email)
return success_response({"status": "deleted", "email": email})
except GroupNotFoundError as e:
return error_response(str(e))
docker/packages/google_mail/src/google_mail/tools/drafts.py
+97
View File
@@ -0,0 +1,97 @@
"""Drafts handlers for Google Mail tools."""
from __future__ import annotations
from pydantic import ValidationError
from google_mail.services.mailbox import (
DraftNotFoundError,
)
from google_mail.state import get_mailbox
from google_mail.tools.common import (
DraftId,
MailboxIdArg,
PageNumber,
PageSize,
error_response,
format_draft,
success_response,
)
async def save_draft(
subject: str = "",
body: str = "",
html_body: str | None = None,
to: str | None = None,
cc: str | None = None,
bcc: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
draft = mailbox.save_draft(
subject=subject,
body=body,
html_body=html_body,
to=to,
cc=cc,
bcc=bcc,
)
return success_response({"status": "saved", "draft": format_draft(draft)})
except ValidationError as e:
return error_response(str(e), status="failed")
async def get_drafts(
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
drafts, total = mailbox.get_drafts(page=page, page_size=page_size)
return success_response(
{
"drafts": [format_draft(d) for d in drafts],
"total": total,
"page": page,
"page_size": page_size,
}
)
async def update_draft(
draft_id: DraftId,
subject: str | None = None,
body: str | None = None,
html_body: str | None = None,
to: str | None = None,
cc: str | None = None,
bcc: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
draft = mailbox.update_draft(
draft_id=draft_id,
subject=subject,
body=body,
html_body=html_body,
to=to,
cc=cc,
bcc=bcc,
)
return success_response({"status": "updated", "draft": format_draft(draft)})
except DraftNotFoundError as e:
return error_response(str(e))
except ValidationError as e:
return error_response(str(e), status="failed")
async def delete_draft(draft_id: DraftId, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
mailbox.delete_draft(draft_id)
return success_response({"status": "deleted", "draft_id": draft_id})
except DraftNotFoundError as e:
return error_response(str(e))
docker/packages/google_mail/src/google_mail/tools/folders.py
+40
View File
@@ -0,0 +1,40 @@
"""Folders handlers for Google Mail tools."""
from __future__ import annotations
from google_mail.services.mailbox import (
FolderExistsError,
FolderNotFoundError,
SystemFolderError,
)
from google_mail.state import get_mailbox
from google_mail.tools.common import (
FolderName,
MailboxIdArg,
error_response,
success_response,
)
async def get_folders(mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
folders = mailbox.get_folders()
return success_response({"folders": folders})
async def create_folder(folder_name: FolderName, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
mailbox.create_folder(folder_name)
return success_response({"status": "created", "folder_name": folder_name})
except FolderExistsError as e:
return error_response(str(e))
async def delete_folder(folder_name: FolderName, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
mailbox.delete_folder(folder_name)
return success_response({"status": "deleted", "folder_name": folder_name})
except (SystemFolderError, FolderNotFoundError) as e:
return error_response(str(e))
docker/packages/google_mail/src/google_mail/tools/messages.py
+313
View File
@@ -0,0 +1,313 @@
"""Messages handlers for Google Mail tools."""
from __future__ import annotations
import base64
import mimetypes
from pathlib import Path
from google_mail.models import Attachment
from google_mail.services.mailbox import (
AttachmentNotFoundError,
EmailNotFoundError,
FolderNotFoundError,
RecipientNotFoundError,
RecipientRequiredError,
normalize_search_pagination,
search_query_warnings,
)
from google_mail.state import get_mailbox, get_mailboxes
from google_mail.tools.common import (
AttachmentFilename,
AttachmentPaths,
EmailId,
EmailIds,
FolderName,
MailboxIdArg,
PageNumber,
PageSize,
batch_response,
error_response,
format_email_full,
format_email_summary,
success_response,
)
async def list_mailboxes() -> str:
result = []
for mid, svc in get_mailboxes().items():
result.append(
{
"mailbox_id": mid,
"email": svc.data.mailbox.email,
"name": svc.data.mailbox.name,
"email_count": len(svc.data.emails),
"contact_count": len(svc.data.contacts),
}
)
return success_response({"mailboxes": result, "total": len(result)})
async def get_emails(
folder: FolderName | None = None,
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
emails, total = mailbox.get_emails(
folder=folder,
page=page,
page_size=page_size,
)
return success_response(
{
"emails": [format_email_summary(e) for e in emails],
"total": total,
"page": page,
"page_size": page_size,
}
)
except FolderNotFoundError as e:
return error_response(str(e))
async def read_email(email_id: EmailId, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
email = mailbox.read_email(email_id)
return success_response({"email": format_email_full(email)})
except EmailNotFoundError as e:
return error_response(str(e))
async def search_emails(
query: str,
folder: FolderName | None = None,
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
page, page_size, pagination_warnings = normalize_search_pagination(page, page_size)
warnings = search_query_warnings(query) + pagination_warnings
emails, total = mailbox.search_emails(
query=query,
folder=folder,
page=page,
page_size=page_size,
)
return success_response(
{
"emails": [format_email_summary(e) for e in emails],
"total": total,
"page": page,
"page_size": page_size,
"warnings": warnings,
}
)
async def send_email(
to: str,
subject: str,
body: str,
html_body: str | None = None,
cc: str | None = None,
bcc: str | None = None,
attachments: AttachmentPaths | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
# Process attachments from file paths
attachment_objs: list[Attachment] | None = None
if attachments:
attachment_objs = []
for file_path_str in attachments:
file_path = Path(file_path_str)
if not file_path.exists():
return error_response(f"Attachment file not found: {file_path_str}")
content = file_path.read_bytes()
content_base64 = base64.b64encode(content).decode("utf-8")
content_type, _ = mimetypes.guess_type(file_path.name)
if content_type is None:
content_type = "application/octet-stream"
attachment_objs.append(
Attachment(
filename=file_path.name,
content_type=content_type,
content_base64=content_base64,
)
)
try:
email = mailbox.send_email(
to=to,
subject=subject,
body=body,
html_body=html_body,
cc=cc,
bcc=bcc,
attachments=attachment_objs,
)
return success_response({"status": "sent", "email": format_email_summary(email)})
except RecipientNotFoundError as e:
return error_response(
f"Invalid recipient: {e.recipient}",
status="bounced",
message="Recipient not found in contacts. Check available contacts with mail_get_contacts.",
)
except RecipientRequiredError as e:
return error_response(str(e), status="bounced")
async def reply_email(
email_id: EmailId,
body: str,
html_body: str | None = None,
reply_all: bool = False,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
email = mailbox.reply_email(
email_id=email_id,
body=body,
html_body=html_body,
reply_all=reply_all,
)
return success_response({"status": "sent", "email": format_email_summary(email)})
except EmailNotFoundError as e:
return error_response(str(e))
except RecipientNotFoundError as e:
return error_response(f"Invalid recipient: {e.recipient}", status="bounced")
async def forward_email(
email_id: EmailId,
to: str,
body: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
email = mailbox.forward_email(
email_id=email_id,
to=to,
body=body,
)
return success_response({"status": "sent", "email": format_email_summary(email)})
except EmailNotFoundError as e:
return error_response(str(e))
except RecipientNotFoundError as e:
return error_response(f"Invalid recipient: {e.recipient}", status="bounced")
async def delete_emails(
email_ids: EmailIds,
permanent: bool = False,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
action = "permanently deleted" if permanent else "moved to Trash"
result = mailbox.delete_emails(email_ids, permanent=permanent)
deleted = result.succeeded_ids
return batch_response(
action=action,
requested_count=len(email_ids),
succeeded_ids=deleted,
errors=[error.to_dict() for error in result.errors],
extra={
"deletedCount": len(deleted),
"deletedIds": deleted,
},
)
async def move_emails(
email_ids: EmailIds,
target_folder: FolderName,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
result = mailbox.move_emails(email_ids, target_folder)
moved = result.succeeded_ids
return batch_response(
action="moved",
requested_count=len(email_ids),
succeeded_ids=moved,
errors=[error.to_dict() for error in result.errors],
extra={
"target_folder": target_folder,
"movedCount": len(moved),
"movedIds": moved,
},
)
async def mark_emails(
email_ids: EmailIds,
is_read: bool | None = None,
is_important: bool | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
if is_read is None and is_important is None:
return error_response("At least one of is_read or is_important must be provided", status="failed")
result = mailbox.mark_emails(
email_ids=email_ids,
is_read=is_read,
is_important=is_important,
)
marked = result.succeeded_ids
return batch_response(
action="marked",
requested_count=len(email_ids),
succeeded_ids=marked,
errors=[error.to_dict() for error in result.errors],
extra={
"markedCount": len(marked),
"markedIds": marked,
},
)
async def get_unread_count(folder: FolderName | None = None, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
counts = mailbox.get_unread_count(folder=folder)
return success_response({"unread_counts": counts})
except FolderNotFoundError as e:
return error_response(str(e))
async def get_mailbox_stats(mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
stats = mailbox.get_mailbox_stats()
return success_response(stats)
async def download_attachment(
email_id: EmailId,
filename: AttachmentFilename,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
attachment = mailbox.get_attachment(email_id, filename)
return success_response(
{
"filename": attachment.filename,
"content_type": attachment.content_type,
"size": attachment.size,
"content_base64": attachment.content_base64,
}
)
except EmailNotFoundError as e:
return error_response(str(e))
except AttachmentNotFoundError as e:
return error_response(str(e))
docker/packages/google_mail/src/google_mail/tools/scheduling.py
+87
View File
@@ -0,0 +1,87 @@
"""Scheduling handlers for Google Mail tools."""
from __future__ import annotations
from google_mail.services.mailbox import (
EmailNotFoundError,
RecipientNotFoundError,
RecipientRequiredError,
)
from google_mail.state import get_mailbox
from google_mail.tools.common import (
EmailId,
MailboxIdArg,
PageNumber,
PageSize,
ScheduleTime,
error_response,
format_email_summary,
normalize_scheduled_time,
success_response,
)
async def schedule_email(
to: str,
subject: str,
body: str,
scheduled_time: ScheduleTime,
html_body: str | None = None,
cc: str | None = None,
bcc: str | None = None,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
try:
send_at = normalize_scheduled_time(scheduled_time)
except ValueError:
return error_response(f"Invalid scheduled_time format: {scheduled_time}")
try:
email = mailbox.schedule_email(
to=to,
subject=subject,
body=body,
scheduled_time=send_at,
html_body=html_body,
cc=cc,
bcc=bcc,
)
result = format_email_summary(email)
result["scheduled_time"] = email.scheduled_time.isoformat() if email.scheduled_time else None
return success_response({"status": "scheduled", "email": result})
except RecipientNotFoundError as e:
return error_response(str(e), status="failed")
except RecipientRequiredError as e:
return error_response(str(e), status="failed")
async def get_scheduled_emails(
page: PageNumber = 1,
page_size: PageSize = 20,
mailbox_id: MailboxIdArg = "default",
) -> str:
mailbox = get_mailbox(mailbox_id)
emails, total = mailbox.get_scheduled_emails(page=page, page_size=page_size)
results = []
for e in emails:
r = format_email_summary(e)
r["scheduled_time"] = e.scheduled_time.isoformat() if e.scheduled_time else None
results.append(r)
return success_response(
{
"scheduled_emails": results,
"total": total,
"page": page,
"page_size": page_size,
}
)
async def cancel_scheduled_email(email_id: EmailId, mailbox_id: MailboxIdArg = "default") -> str:
mailbox = get_mailbox(mailbox_id)
try:
mailbox.cancel_scheduled_email(email_id)
return success_response({"status": "cancelled", "email_id": email_id})
except EmailNotFoundError as e:
return error_response(str(e))
docker/packages/google_mail/src/google_mail/tools/state.py
+15
View File
@@ -0,0 +1,15 @@
"""State handlers for Google Mail tools."""
from __future__ import annotations
from google_mail.models import GoogleMailState
from google_mail.state import export_state_model, state_from_json
async def export_state() -> GoogleMailState:
return export_state_model()
async def import_state(state: GoogleMailState) -> dict:
state_from_json(state)
return {"ok": True}
docker/packages/google_mail/src/google_mail/viewer.py
+454
View File
@@ -0,0 +1,454 @@
"""Mail viewer — read-only webmail UI and API endpoints.
Serves:
GET /api/folders — folder list with counts
GET /api/emails — email list (optional ?folder=X&search=X&page=N)
GET /api/emails/:id — single email detail
GET /api/contacts — contact list
GET /api/stats — mailbox stats
GET / — viewer HTML (single-page app)
All non-MCP routes require the X-Proxy-Token header.
"""
from __future__ import annotations
import os
from typing import Any
import uvicorn
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import HTMLResponse, JSONResponse, Response
from starlette.routing import Route
# ---------------------------------------------------------------------------
# Auth middleware
# ---------------------------------------------------------------------------
class ProxyTokenMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
# Skip auth for MCP endpoint
if request.url.path.startswith("/mcp"):
return await call_next(request)
token = os.environ.get("MCP_PROXY_TOKEN", "")
if token and request.headers.get("x-proxy-token") != token:
return Response("Forbidden: invalid proxy token", status_code=403)
return await call_next(request)
# ---------------------------------------------------------------------------
# Lazy access to the mailbox service
# ---------------------------------------------------------------------------
_mailbox_ref = None
def set_mailbox(mailbox) -> None:
"""Set the mailbox instance for the viewer to use."""
global _mailbox_ref
_mailbox_ref = mailbox
def _get_mailbox(request: Request | None = None):
"""Return the mailbox service, initializing state if needed.
If the MCP server has not initialized yet, initialize the mailbox eagerly
from the same state loader used by the server.
"""
if _mailbox_ref is not None:
return _mailbox_ref
from google_mail.state import get_mailboxes, init_state
init_state()
mailbox_id = request.query_params.get("mailbox_id") if request is not None else None
mailboxes = get_mailboxes()
if mailbox_id:
if mailbox_id not in mailboxes:
available = ", ".join(sorted(mailboxes.keys()))
raise KeyError(f"Mailbox '{mailbox_id}' not found. Available: {available}")
return mailboxes[mailbox_id]
if "default" in mailboxes:
return mailboxes["default"]
first_id = sorted(mailboxes.keys())[0]
return mailboxes[first_id]
# ---------------------------------------------------------------------------
# API handlers
# ---------------------------------------------------------------------------
async def api_folders(request: Request) -> JSONResponse:
try:
mailbox = _get_mailbox(request)
except KeyError as exc:
return JSONResponse({"error": str(exc)}, status_code=404)
folders = mailbox.get_folders()
return JSONResponse({"folders": folders})
async def api_emails(request: Request) -> JSONResponse:
try:
mailbox = _get_mailbox(request)
except KeyError as exc:
return JSONResponse({"error": str(exc)}, status_code=404)
folder = request.query_params.get("folder")
search = request.query_params.get("search")
page = int(request.query_params.get("page", "1"))
page_size = int(request.query_params.get("page_size", "50"))
if search:
emails, total = mailbox.search_emails(query=search, folder=folder, page=page, page_size=page_size)
else:
emails, total = mailbox.get_emails(folder=folder, page=page, page_size=page_size)
return JSONResponse(
{
"emails": [_format_summary(e) for e in emails],
"total": total,
"page": page,
"page_size": page_size,
}
)
async def api_email_detail(request: Request) -> JSONResponse:
try:
mailbox = _get_mailbox(request)
except KeyError as exc:
return JSONResponse({"error": str(exc)}, status_code=404)
email_id = request.path_params["email_id"]
try:
email = mailbox.read_email(email_id)
except Exception:
return JSONResponse({"error": "Email not found"}, status_code=404)
return JSONResponse({"email": _format_full(email)})
async def api_contacts(request: Request) -> JSONResponse:
try:
mailbox = _get_mailbox(request)
except KeyError as exc:
return JSONResponse({"error": str(exc)}, status_code=404)
contacts = mailbox.get_contacts()
groups = mailbox.get_groups()
return JSONResponse(
{
"contacts": [{"email": c.email, "name": c.name} for c in contacts],
"groups": [{"email": g.email, "name": g.name, "members": g.members} for g in groups],
}
)
async def api_stats(request: Request) -> JSONResponse:
try:
mailbox = _get_mailbox(request)
except KeyError as exc:
return JSONResponse({"error": str(exc)}, status_code=404)
stats = mailbox.get_mailbox_stats()
return JSONResponse(stats)
async def viewer_html(request: Request) -> HTMLResponse:
return HTMLResponse(VIEWER_HTML)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _format_summary(email) -> dict[str, Any]:
return {
"email_id": email.email_id,
"folder": email.folder,
"subject": email.subject,
"from_addr": email.from_addr,
"to_addr": email.to_addr,
"date": email.date.isoformat(),
"is_read": email.is_read,
"is_important": email.is_important,
"has_attachments": len(email.attachments) > 0,
}
def _format_full(email) -> dict[str, Any]:
d = _format_summary(email)
d["cc_addr"] = email.cc_addr
d["bcc_addr"] = email.bcc_addr
d["body_text"] = email.body_text
d["body_html"] = email.body_html
d["attachments"] = [
{"filename": a.filename, "content_type": a.content_type, "size": a.size} for a in email.attachments
]
return d
# ---------------------------------------------------------------------------
# App factory
# ---------------------------------------------------------------------------
def create_mail_viewer_app():
routes = [
Route("/", viewer_html),
Route("/api/folders", api_folders),
Route("/api/emails", api_emails),
Route("/api/emails/{email_id}", api_email_detail),
Route("/api/contacts", api_contacts),
Route("/api/stats", api_stats),
]
return Starlette(
routes=routes,
middleware=[Middleware(ProxyTokenMiddleware)],
)
def run_http_server(mcp_app, port: int) -> None:
"""Run combined MCP + viewer HTTP server."""
# Get the ASGI app from FastMCP
# mcp.server.fastmcp.FastMCP uses streamable_http_app(); fastmcp.FastMCP uses http_app()
if hasattr(mcp_app, "streamable_http_app"):
fastmcp_asgi = mcp_app.streamable_http_app()
else:
fastmcp_asgi = mcp_app.http_app(
transport="streamable-http",
path="/mcp",
)
viewer = create_mail_viewer_app()
# Combined app: route /mcp to FastMCP, everything else to viewer
async def combined_app(scope, receive, send):
if scope["type"] == "lifespan":
await fastmcp_asgi(scope, receive, send)
return
path = scope.get("path", "")
if path.startswith("/mcp"):
await fastmcp_asgi(scope, receive, send)
else:
await viewer(scope, receive, send)
uvicorn.run(
combined_app,
host="127.0.0.1",
port=port,
log_level="warning",
)
# ---------------------------------------------------------------------------
# Viewer HTML
# ---------------------------------------------------------------------------
VIEWER_HTML = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Mail</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #f8fafc; color: #1e293b; display: flex; height: 100vh; }
/* Folder sidebar */
.folders { width: 200px; min-width: 200px; background: #fff; border-right: 1px solid #e2e8f0; display: flex; flex-direction: column; }
.folders-header { padding: 16px; border-bottom: 1px solid #e2e8f0; }
.folders-header h2 { font-size: 15px; font-weight: 600; }
.folder-list { flex: 1; padding: 8px; overflow-y: auto; }
.folder-item { display: flex; align-items: center; justify-content: space-between; padding: 8px 12px; border-radius: 6px; cursor: pointer; font-size: 13px; color: #475569; margin-bottom: 2px; }
.folder-item:hover { background: #f1f5f9; }
.folder-item.active { background: #eff6ff; color: #2563eb; font-weight: 600; }
.folder-item .count { font-size: 11px; color: #94a3b8; }
.folder-item.active .count { color: #60a5fa; }
/* Email list */
.email-list-pane { width: 360px; min-width: 360px; border-right: 1px solid #e2e8f0; display: flex; flex-direction: column; background: #fff; }
.list-header { padding: 10px 16px; border-bottom: 1px solid #e2e8f0; }
.list-header input { width: 100%; padding: 8px 12px; border: 1px solid #d1d5db; border-radius: 8px; font-size: 13px; }
.list-header input:focus { outline: none; border-color: #3b82f6; box-shadow: 0 0 0 2px rgba(59,130,246,0.15); }
.email-list { flex: 1; overflow-y: auto; }
.email-item { padding: 12px 16px; border-bottom: 1px solid #f1f5f9; cursor: pointer; }
.email-item:hover { background: #f8fafc; }
.email-item.active { background: #eff6ff; }
.email-item.unread { border-left: 3px solid #3b82f6; }
.email-item .from { font-size: 13px; font-weight: 600; color: #1e293b; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.email-item.unread .from { color: #0f172a; }
.email-item .subject { font-size: 13px; color: #475569; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px; }
.email-item .preview { font-size: 12px; color: #94a3b8; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; margin-top: 2px; }
.email-item .meta { display: flex; justify-content: space-between; align-items: center; margin-top: 4px; }
.email-item .date { font-size: 11px; color: #94a3b8; }
.email-item .badges { display: flex; gap: 4px; }
.badge-important { color: #f59e0b; font-size: 12px; }
.badge-attachment { color: #64748b; font-size: 12px; }
/* Detail pane */
.detail-pane { flex: 1; display: flex; flex-direction: column; background: #fff; overflow-y: auto; }
.detail-header { padding: 20px 24px; border-bottom: 1px solid #e2e8f0; }
.detail-header h2 { font-size: 18px; font-weight: 600; line-height: 1.4; }
.detail-meta { margin-top: 12px; }
.detail-meta .row { display: flex; gap: 8px; font-size: 13px; margin-bottom: 4px; }
.detail-meta .label { color: #64748b; min-width: 40px; }
.detail-meta .value { color: #1e293b; }
.detail-body { padding: 24px; flex: 1; }
.detail-body .body-text { font-size: 14px; line-height: 1.7; white-space: pre-wrap; color: #334155; }
.detail-attachments { padding: 0 24px 24px; }
.detail-attachments h4 { font-size: 12px; font-weight: 600; color: #64748b; text-transform: uppercase; margin-bottom: 8px; }
.attachment-item { display: inline-flex; align-items: center; gap: 6px; padding: 6px 12px; background: #f1f5f9; border-radius: 6px; font-size: 12px; margin-right: 8px; margin-bottom: 4px; }
.empty { text-align: center; padding: 60px; color: #94a3b8; font-size: 14px; }
.detail-empty { display: flex; align-items: center; justify-content: center; flex: 1; color: #94a3b8; font-size: 14px; }
</style>
</head>
<body>
<div class="folders">
<div class="folders-header"><h2>Mail</h2></div>
<div id="folder-list" class="folder-list"></div>
</div>
<div class="email-list-pane">
<div class="list-header">
<input type="text" id="search-input" placeholder="Search emails..." oninput="onSearch()">
</div>
<div id="email-list" class="email-list">
<div class="empty">Loading...</div>
</div>
</div>
<div id="detail-pane" class="detail-pane">
<div class="detail-empty">Select an email to read</div>
</div>
<script>
let currentFolder = null;
let emails = [];
let searchTimeout = null;
const base = window.location.pathname.replace(/\\/$/, '');
async function fetchJSON(path) {
const r = await fetch(base + path);
return r.json();
}
function esc(s) { if (!s) return ''; const d = document.createElement('div'); d.textContent = s; return d.innerHTML; }
async function init() {
const data = await fetchJSON('/api/folders');
const el = document.getElementById('folder-list');
el.innerHTML = data.folders.map(f =>
'<div class="folder-item" data-folder="' + esc(f.name) + '" onclick="selectFolder(\\'' + esc(f.name) + '\\')">' +
'<span>' + folderIcon(f.name) + ' ' + esc(f.name) + '</span>' +
'<span class="count">' + (f.unread || '') + '</span>' +
'</div>'
).join('');
if (data.folders.length) selectFolder('INBOX');
}
function folderIcon(name) {
const icons = { INBOX: '📥', Sent: '📤', Drafts: '📝', Trash: '🗑️' };
return icons[name] || '📁';
}
async function selectFolder(folder) {
currentFolder = folder;
document.querySelectorAll('.folder-item').forEach(el => {
el.classList.toggle('active', el.dataset.folder === folder);
});
await loadEmails();
}
async function loadEmails() {
const search = document.getElementById('search-input').value;
let q = '/api/emails?page_size=100';
if (currentFolder) q += '&folder=' + encodeURIComponent(currentFolder);
if (search) q += '&search=' + encodeURIComponent(search);
const data = await fetchJSON(q);
emails = data.emails;
renderList();
}
function renderList() {
const el = document.getElementById('email-list');
if (!emails.length) {
el.innerHTML = '<div class="empty">No emails</div>';
return;
}
el.innerHTML = emails.map(e => {
const unread = !e.is_read ? ' unread' : '';
return '<div class="email-item' + unread + '" onclick="showEmail(\\'' + e.email_id + '\\')">' +
'<div class="from">' + esc(e.from_addr) + '</div>' +
'<div class="subject">' + esc(e.subject || '(no subject)') + '</div>' +
'<div class="meta">' +
'<span class="date">' + formatDate(e.date) + '</span>' +
'<span class="badges">' +
(e.is_important ? '<span class="badge-important">★</span>' : '') +
(e.has_attachments ? '<span class="badge-attachment">📎</span>' : '') +
'</span>' +
'</div>' +
'</div>';
}).join('');
}
async function showEmail(id) {
const data = await fetchJSON('/api/emails/' + id);
const e = data.email;
document.querySelectorAll('.email-item').forEach(el => el.classList.remove('active'));
// Mark active - find by looking at onclick
document.querySelectorAll('.email-item').forEach(el => {
if (el.getAttribute('onclick')?.includes(id)) el.classList.add('active');
});
let html = '<div class="detail-header"><h2>' + esc(e.subject || '(no subject)') + '</h2>';
html += '<div class="detail-meta">';
html += '<div class="row"><span class="label">From</span><span class="value">' + esc(e.from_addr) + '</span></div>';
html += '<div class="row"><span class="label">To</span><span class="value">' + esc(e.to_addr) + '</span></div>';
if (e.cc_addr) html += '<div class="row"><span class="label">CC</span><span class="value">' + esc(e.cc_addr) + '</span></div>';
html += '<div class="row"><span class="label">Date</span><span class="value">' + formatDate(e.date) + '</span></div>';
html += '</div></div>';
html += '<div class="detail-body"><div class="body-text">' + esc(e.body_text) + '</div></div>';
if (e.attachments && e.attachments.length) {
html += '<div class="detail-attachments"><h4>Attachments</h4>';
e.attachments.forEach(a => {
html += '<span class="attachment-item">📎 ' + esc(a.filename) + ' (' + formatSize(a.size) + ')</span>';
});
html += '</div>';
}
document.getElementById('detail-pane').innerHTML = html;
}
function onSearch() {
clearTimeout(searchTimeout);
searchTimeout = setTimeout(loadEmails, 300);
}
function formatDate(d) {
if (!d) return '';
try {
const dt = new Date(d);
const now = new Date();
if (dt.toDateString() === now.toDateString()) {
return dt.toLocaleTimeString('en-US', { hour: '2-digit', minute: '2-digit' });
}
return dt.toLocaleDateString('en-US', { month: 'short', day: 'numeric' });
} catch { return d; }
}
function formatSize(bytes) {
if (!bytes) return '0 B';
if (bytes < 1024) return bytes + ' B';
if (bytes < 1048576) return (bytes / 1024).toFixed(1) + ' KB';
return (bytes / 1048576).toFixed(1) + ' MB';
}
init();
</script>
</body>
</html>"""
docker/packages/google_mail/tests/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Tests for google_mail."""
docker/packages/google_mail/tests/test_datadir.py
+365
View File
@@ -0,0 +1,365 @@
import json
from pathlib import Path
import pytest
@pytest.fixture(autouse=True)
def _reset_mail_state():
import google_mail.state as state
state.set_mailboxes({})
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
yield
state.set_mailboxes({})
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
def _write_mailbox(path):
path.write_text(
json.dumps(
{
"mailbox": {"email": "t@t.com", "name": "T"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
def test_inputdir_loads_first_json(monkeypatch, tmp_path):
"""When INPUTDIR is set, the server loads the first .json in that dir."""
data_file = tmp_path / "google_mail.json"
_write_mailbox(data_file)
monkeypatch.setenv("INPUTDIR", str(tmp_path))
json_files = sorted(Path(str(tmp_path)).glob("*.json"))
assert json_files[0] == data_file
def test_inputdir_loads_only_first_file_not_merged(monkeypatch, tmp_path):
"""Legacy contract: INPUTDIR loads ONLY the first *.json (alphabetically),
not a merge of all of them — matching scripts/validate_external_services.py.
Folder coalescing is a bundle-only behavior."""
import google_mail.server as srv
import google_mail.state as state
inputdir = tmp_path / "in"
inputdir.mkdir()
(inputdir / "a.json").write_text(
json.dumps({"mailbox": {"email": "first@t.com", "name": "First"}, "emails": [], "next_email_id": 1})
)
(inputdir / "b.json").write_text(
json.dumps({"mailbox": {"email": "second@t.com", "name": "Second"}, "emails": [], "next_email_id": 1})
)
monkeypatch.delenv("BUNDLEDIR", raising=False)
monkeypatch.setenv("INPUTDIR", str(inputdir))
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
state.set_mailboxes({})
srv.init_state()
mailboxes = state.get_mailboxes()
assert set(mailboxes) == {"default"}
assert mailboxes["default"].data.mailbox.email == "first@t.com"
def test_inputdir_empty_uses_default(monkeypatch, tmp_path):
"""When INPUTDIR is set but empty, server starts with an empty default mailbox."""
monkeypatch.setenv("INPUTDIR", str(tmp_path))
json_files = sorted(Path(str(tmp_path)).glob("*.json"))
assert json_files == []
def test_inputdir_not_set_uses_default(monkeypatch):
"""When INPUTDIR is not set, server starts with an empty default mailbox (no error)."""
monkeypatch.delenv("INPUTDIR", raising=False)
import os
assert os.environ.get("INPUTDIR") is None
# Server should start cleanly — no error raised
def test_bundle_state_json_preferred_over_inputdir(monkeypatch, tmp_path):
"""When BUNDLEDIR is set and $BUNDLEDIR/services/google_mail/state.json
exists, init_state loads it instead of any legacy *.json files
in INPUTDIR."""
import google_mail.server as srv
import google_mail.state as state
bundle_root = tmp_path / "bundle"
service_dir = bundle_root / "services" / "google_mail"
service_dir.mkdir(parents=True)
(service_dir / "state.json").write_text(
json.dumps(
{
"mailbox": {"email": "bundle@t.com", "name": "Bundle"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
inputdir = tmp_path / "legacy"
inputdir.mkdir()
_write_mailbox(inputdir / "legacy.json")
monkeypatch.setenv("BUNDLEDIR", str(bundle_root))
monkeypatch.setenv("INPUTDIR", str(inputdir))
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
srv.init_state()
assert state.get_mailboxes()["default"].data.mailbox.email == "bundle@t.com"
def test_bundle_glob_when_state_json_absent(monkeypatch, tmp_path):
"""When BUNDLEDIR carries a per-service subdir with a named JSON (e.g.
the preserved entities-zip layout) but no state.json, the loader reads
it from that subdir."""
import google_mail.server as srv
import google_mail.state as state
bundle_root = tmp_path / "bundle"
service_dir = bundle_root / "services" / "google_mail"
service_dir.mkdir(parents=True)
(service_dir / "inbox.json").write_text(
json.dumps(
{
"mailbox": {"email": "preserved@t.com", "name": "Preserved"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
monkeypatch.setenv("BUNDLEDIR", str(bundle_root))
monkeypatch.delenv("INPUTDIR", raising=False)
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
srv.init_state()
assert state.get_mailboxes()["default"].data.mailbox.email == "preserved@t.com"
def test_falls_back_to_inputdir_when_bundle_service_dir_missing(monkeypatch, tmp_path):
"""When BUNDLEDIR is set but doesn't carry this service's slice,
init_state falls back to the legacy INPUTDIR/*.json glob — a partial
bundle doesn't strand a service."""
import google_mail.server as srv
import google_mail.state as state
# BUNDLEDIR points at a real dir, but no google_mail/ subdir under it.
bundle_root = tmp_path / "bundle"
(bundle_root / "services").mkdir(parents=True)
inputdir = tmp_path / "legacy"
inputdir.mkdir()
(inputdir / "legacy.json").write_text(
json.dumps(
{
"mailbox": {"email": "legacy@t.com", "name": "Legacy"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
monkeypatch.setenv("BUNDLEDIR", str(bundle_root))
monkeypatch.setenv("INPUTDIR", str(inputdir))
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
srv.init_state()
assert state.get_mailboxes()["default"].data.mailbox.email == "legacy@t.com"
def test_falls_back_to_inputdir_when_bundledir_unset(monkeypatch, tmp_path):
"""When BUNDLEDIR is unset entirely, init_state uses INPUTDIR (the
common local-dev path). The bundle code path is skipped."""
import google_mail.server as srv
import google_mail.state as state
inputdir = tmp_path / "legacy"
inputdir.mkdir()
(inputdir / "legacy.json").write_text(
json.dumps(
{
"mailbox": {"email": "legacy@t.com", "name": "Legacy"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
monkeypatch.delenv("BUNDLEDIR", raising=False)
monkeypatch.setenv("INPUTDIR", str(inputdir))
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
srv.init_state()
assert state.get_mailboxes()["default"].data.mailbox.email == "legacy@t.com"
def test_init_state_configures_snapshot_paths_when_registry_preloaded(monkeypatch, tmp_path):
"""Preloaded state should still pick up env-configured snapshot paths."""
import google_mail.server as srv
import google_mail.state as state
from google_mail.services.mailbox import MailboxService
data_file = tmp_path / "mailbox.json"
_write_mailbox(data_file)
svc = MailboxService(data_file)
svc.load()
outputdir = tmp_path / "output"
bundle_output_dir = tmp_path / "services" / "google_mail"
state.set_mailboxes({"default": svc})
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
monkeypatch.setenv("OUTPUTDIR", str(outputdir))
monkeypatch.setenv("BUNDLE_OUTPUT_DIR", str(bundle_output_dir))
srv.init_state()
assert state.get_final_path() == outputdir / "final.json"
assert state.get_bundle_state_path() == bundle_output_dir / "state.json"
def _write_mailbox_email(path, email):
path.write_text(
json.dumps(
{
"mailbox": {"email": email, "name": email.split("@")[0]},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
def _email_record(email_id, subject):
return {
"email_id": email_id,
"folder": "INBOX",
"subject": subject,
"from_addr": "sender@t.com",
"to_addr": "agent@t.com",
"date": "2026-01-01T00:00:00+00:00",
"message_id": f"<{email_id}@t.com>",
"body_text": subject,
"is_read": False,
}
def test_bundle_multifile_flat_files_merge_into_one_mailbox(monkeypatch, tmp_path):
"""the raw entities layout splits ONE mailbox across per-entity files
(e.g. inbox.json + sent.json). Flat files without a {mailboxes} wrapper
coalesce into a single default mailbox — entities combined, not fragmented
into separate mailboxes."""
import google_mail.server as srv
import google_mail.state as state
service_dir = tmp_path / "bundle" / "services" / "google_mail"
service_dir.mkdir(parents=True)
# Two halves of the SAME mailbox: identity + first email; second email.
(service_dir / "a.json").write_text(
json.dumps(
{
"mailbox": {"email": "agent@t.com", "name": "Agent"},
"contacts": [],
"folders": [],
"emails": [_email_record("1", "first")],
"next_email_id": 2,
}
)
)
(service_dir / "b.json").write_text(json.dumps({"emails": [_email_record("2", "second")], "next_email_id": 3}))
monkeypatch.setenv("BUNDLEDIR", str(tmp_path / "bundle"))
monkeypatch.delenv("INPUTDIR", raising=False)
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
state.set_mailboxes({})
srv.init_state()
mailboxes = state.get_mailboxes()
assert set(mailboxes) == {"default"}, "flat per-entity files must merge into ONE mailbox"
data = mailboxes["default"].data
assert data.mailbox.email == "agent@t.com"
subjects = {e.subject for e in data.emails}
assert subjects == {"first", "second"}, "both per-entity files' emails must be present"
assert data.next_email_id == 3
def test_bundle_mailboxes_wrapper_yields_named_mailboxes(monkeypatch, tmp_path):
"""An explicit {mailboxes: {...}} wrapper still produces multiple named
mailboxes — multi-tenant worlds opt in via the wrapper, not via separate
flat files."""
import google_mail.server as srv
import google_mail.state as state
service_dir = tmp_path / "bundle" / "services" / "google_mail"
service_dir.mkdir(parents=True)
(service_dir / "state.json").write_text(
json.dumps(
{
"mailboxes": {
"alice": {"mailbox": {"email": "a@t.com", "name": "A"}, "emails": [], "next_email_id": 1},
"bob": {"mailbox": {"email": "b@t.com", "name": "B"}, "emails": [], "next_email_id": 1},
}
}
)
)
monkeypatch.setenv("BUNDLEDIR", str(tmp_path / "bundle"))
monkeypatch.delenv("INPUTDIR", raising=False)
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
state.set_mailboxes({})
srv.init_state()
mailboxes = state.get_mailboxes()
assert set(mailboxes) == {"alice", "bob"}
assert {svc.data.mailbox.email for svc in mailboxes.values()} == {"a@t.com", "b@t.com"}
def test_resolve_bundle_state_paths_returns_whole_folder(monkeypatch, tmp_path):
"""The plural resolver returns ALL sorted *.json when there's no state.json,
and exactly [state.json] when one is present."""
import google_mail.state as state
service_dir = tmp_path / "services" / "google_mail"
service_dir.mkdir(parents=True)
a_json = service_dir / "a.json"
b_json = service_dir / "b.json"
_write_mailbox_email(a_json, "a@t.com")
_write_mailbox_email(b_json, "b@t.com")
monkeypatch.setenv("BUNDLEDIR", str(tmp_path))
assert state.resolve_bundle_state_paths() == [a_json, b_json]
state_json = service_dir / "state.json"
_write_mailbox_email(state_json, "state@t.com")
assert state.resolve_bundle_state_paths() == [state_json]
docker/packages/google_mail/tests/test_google_mail_snapshot.py
+350
View File
@@ -0,0 +1,350 @@
"""Tests for the _snapshot_on_write decorator — final.json is written after every write tool call."""
import asyncio
import json
import pytest
from google_mail.services.mailbox import MailboxService
@pytest.fixture
def mailbox_service(tmp_path):
"""Create a MailboxService seeded with minimal data."""
data_path = tmp_path / "mailbox.json"
data_path.write_text(
json.dumps(
{
"mailbox": {"email": "test@test.com", "name": "Test User"},
"contacts": [{"email": "bob@test.com", "name": "Bob"}],
"folders": [],
"emails": [],
"next_email_id": 1,
}
)
)
svc = MailboxService(data_path)
svc.load()
return svc
@pytest.fixture
def outputdir(tmp_path):
"""Return a temp directory for OUTPUTDIR, with a pre-configured final.json path."""
out = tmp_path / "output" / "google_mail"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_server_globals(mailbox_service, outputdir):
"""Wire up state globals so tools and the decorator can run."""
import google_mail.state as state
state.set_mailboxes({"default": mailbox_service})
state.set_snapshot_paths(final_path=outputdir / "final.json")
yield
state.set_mailboxes({})
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
def test_send_email_writes_final_json(outputdir):
"""Calling mail_send_email produces a final.json snapshot immediately."""
from google_mail.server import mail_send_email
final = outputdir / "final.json"
assert not final.exists(), "final.json should not exist before any write"
result = asyncio.run(
mail_send_email(
to="bob@test.com",
subject="snapshot test",
body="hello",
)
)
data = json.loads(result)
assert data.get("status") == "sent"
assert final.exists(), "final.json must be written after mail_send_email"
snapshot = json.loads(final.read_text())
subjects = [e["subject"] for e in snapshot.get("emails", [])]
assert "snapshot test" in subjects
def test_send_email_without_recipients_returns_error_without_consuming_id(mailbox_service):
"""Empty recipient sends return a structured error and leave counters untouched."""
from google_mail.server import mail_send_email
next_email_id = mailbox_service.data.next_email_id
result = asyncio.run(mail_send_email(to="", subject="empty recipient", body="hello"))
data = json.loads(result)
assert data.get("status") == "bounced"
assert data.get("error")
assert mailbox_service.data.next_email_id == next_email_id
def test_save_draft_invalid_recipient_returns_error_without_consuming_id(mailbox_service):
"""Invalid draft recipients return structured errors instead of raising."""
from google_mail.server import mail_save_draft
next_email_id = mailbox_service.data.next_email_id
result = asyncio.run(mail_save_draft(to="not an email", subject="bad draft", body="hello"))
data = json.loads(result)
assert data.get("status") == "failed"
assert data.get("error")
assert mailbox_service.data.next_email_id == next_email_id
def test_update_draft_invalid_recipient_returns_error_without_corrupting_draft(mailbox_service):
"""Invalid draft updates return structured errors and preserve the existing draft."""
from google_mail.server import mail_save_draft, mail_update_draft
created = json.loads(asyncio.run(mail_save_draft(to="bob@test.com", subject="draft", body="hello")))
draft_id = created["draft"]["draft_id"]
original = mailbox_service.get_draft(draft_id).model_dump(mode="json")
result = asyncio.run(mail_update_draft(draft_id=draft_id, to="not an email", subject="bad update"))
data = json.loads(result)
assert data.get("status") == "failed"
assert data.get("error")
assert mailbox_service.get_draft(draft_id).model_dump(mode="json") == original
def test_delete_email_writes_final_json(outputdir):
"""Calling mail_delete_emails produces a final.json snapshot."""
from google_mail.server import mail_delete_emails, mail_send_email
# First, create an email to delete
result = asyncio.run(mail_send_email(to="bob@test.com", subject="to-delete", body="bye"))
sent = json.loads(result)
email_id = sent["email"]["email_id"]
# Remove existing final.json to isolate the delete's snapshot
final = outputdir / "final.json"
final.unlink()
asyncio.run(mail_delete_emails([email_id]))
assert final.exists(), "final.json must be written after mail_delete_emails"
def test_delete_emails_reports_partial_batch_status():
from google_mail.server import mail_delete_emails, mail_send_email
result = asyncio.run(mail_send_email(to="bob@test.com", subject="to-delete", body="bye"))
email_id = json.loads(result)["email"]["email_id"]
data = json.loads(asyncio.run(mail_delete_emails([email_id, "missing"])))
assert data["status"] == "partial_success"
assert data["summary"] == "1 of 2 attempts succeeded"
assert data["requestedCount"] == 2
assert data["succeededCount"] == 1
assert data["failedCount"] == 1
assert data["deletedIds"] == [email_id]
assert data["errors"][0]["email_id"] == "missing"
def test_delete_emails_reports_all_failed_batch_status():
from google_mail.server import mail_delete_emails
data = json.loads(asyncio.run(mail_delete_emails(["missing-1", "missing-2"])))
assert data["status"] == "all_failed"
assert data["summary"] == "All 2 attempts failed"
assert data["requestedCount"] == 2
assert data["succeededCount"] == 0
assert data["failedCount"] == 2
def test_mark_emails_requires_an_action_flag():
from google_mail.server import mail_mark_emails, mail_send_email
result = asyncio.run(mail_send_email(to="bob@test.com", subject="to-mark", body="hi"))
email_id = json.loads(result)["email"]["email_id"]
data = json.loads(asyncio.run(mail_mark_emails([email_id])))
assert data["status"] == "failed"
assert "At least one" in data["error"]
def test_mark_emails_reports_partial_batch_status():
from google_mail.server import mail_mark_emails, mail_send_email
result = asyncio.run(mail_send_email(to="bob@test.com", subject="to-mark", body="hi"))
email_id = json.loads(result)["email"]["email_id"]
data = json.loads(asyncio.run(mail_mark_emails([email_id, "missing"], is_read=False)))
assert data["status"] == "partial_success"
assert data["summary"] == "1 of 2 attempts succeeded"
assert data["markedCount"] == 1
assert data["markedIds"] == [email_id]
assert data["errors"][0]["email_id"] == "missing"
def test_create_folder_writes_final_json(outputdir):
"""Calling mail_create_folder produces a final.json snapshot."""
from google_mail.server import mail_create_folder
final = outputdir / "final.json"
asyncio.run(mail_create_folder("TestFolder"))
assert final.exists(), "final.json must be written after mail_create_folder"
snapshot = json.loads(final.read_text())
folder_names = [f["name"] for f in snapshot.get("folders", [])]
assert "TestFolder" in folder_names
def test_read_only_tool_does_not_write_final_json(outputdir):
"""Calling a read-only tool (mail_get_emails) does NOT write final.json."""
from google_mail.server import mail_get_emails
final = outputdir / "final.json"
asyncio.run(mail_get_emails(folder="INBOX"))
assert not final.exists(), "final.json must NOT be written after a read-only tool"
def test_final_json_updates_incrementally(outputdir):
"""Each write tool call overwrites final.json with the latest state."""
from google_mail.server import mail_send_email
final = outputdir / "final.json"
asyncio.run(mail_send_email(to="bob@test.com", subject="first", body="1"))
snap1 = json.loads(final.read_text())
count1 = len(snap1.get("emails", []))
asyncio.run(mail_send_email(to="bob@test.com", subject="second", body="2"))
snap2 = json.loads(final.read_text())
count2 = len(snap2.get("emails", []))
assert count2 > count1, "final.json should reflect each incremental mutation"
def test_add_contact_writes_final_json(outputdir):
"""Calling mail_add_contact produces a final.json snapshot."""
from google_mail.server import mail_add_contact
final = outputdir / "final.json"
assert not final.exists()
result = asyncio.run(mail_add_contact(email="alice@test.com", name="Alice"))
data = json.loads(result)
assert data.get("status") == "created"
assert final.exists(), "final.json must be written after mail_add_contact"
def test_schedule_email_writes_final_json(outputdir):
"""Calling mail_schedule_email produces a final.json snapshot."""
from google_mail.server import mail_schedule_email
final = outputdir / "final.json"
assert not final.exists()
result = asyncio.run(
mail_schedule_email(
to="bob@test.com",
subject="scheduled test",
body="hello",
scheduled_time="2025-06-01T09:00:00Z",
)
)
data = json.loads(result)
assert data.get("status") == "scheduled"
assert final.exists(), "final.json must be written after mail_schedule_email"
def test_schedule_email_without_recipients_returns_error_without_consuming_id(mailbox_service):
"""Empty scheduled sends return a structured error and leave counters untouched."""
from google_mail.server import mail_schedule_email
next_email_id = mailbox_service.data.next_email_id
result = asyncio.run(
mail_schedule_email(
to="",
subject="empty recipient",
body="hello",
scheduled_time="2025-06-01T09:00:00Z",
)
)
data = json.loads(result)
assert data.get("status") == "failed"
assert data.get("error")
assert mailbox_service.data.next_email_id == next_email_id
def test_get_contacts_does_not_write_final_json(outputdir):
"""Calling a read-only contact tool does NOT write final.json."""
from google_mail.server import mail_get_contacts
final = outputdir / "final.json"
asyncio.run(mail_get_contacts())
assert not final.exists(), "final.json must NOT be written after a read-only tool"
def test_no_final_path_skips_snapshot():
"""When _final_path is None (no OUTPUTDIR), write tools still work without error."""
import google_mail.state as state
from google_mail.server import mail_send_email
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
result = asyncio.run(mail_send_email(to="bob@test.com", subject="no-outputdir", body="ok"))
data = json.loads(result)
assert data.get("status") == "sent"
def test_dual_writes_bundle_path_and_final_json(outputdir, tmp_path):
"""Bundle migration: writable tools dual-write the bundle path (nested
services/<name>/state.json layout) AND legacy final.json."""
import google_mail.state as state
from google_mail.server import mail_send_email
bundle_output_dir = tmp_path / "services" / "google_mail"
bundle_output_dir.mkdir(parents=True)
bundle_path = bundle_output_dir / "state.json"
final_path = outputdir / "final.json"
state.set_snapshot_paths(final_path=final_path, bundle_state_path=bundle_path)
asyncio.run(mail_send_email(to="bob@test.com", subject="dual-write", body="hi"))
assert bundle_path.exists(), "<BUNDLE_OUTPUT_DIR>/state.json must be written"
assert final_path.exists(), "legacy final.json must still be written"
assert bundle_path.read_text() == final_path.read_text()
def test_set_snapshot_paths_partial_update_preserves_existing_path(tmp_path):
import google_mail.state as state
final_path = tmp_path / "final.json"
bundle_path = tmp_path / "bundle.json"
updated_final_path = tmp_path / "updated-final.json"
state.set_snapshot_paths(final_path=final_path, bundle_state_path=bundle_path)
state.set_snapshot_paths(final_path=updated_final_path)
assert state.get_final_path() == updated_final_path
assert state.get_bundle_state_path() == bundle_path
def test_set_snapshot_paths_can_clear_individual_path(tmp_path):
import google_mail.state as state
final_path = tmp_path / "final.json"
bundle_path = tmp_path / "bundle.json"
state.set_snapshot_paths(final_path=final_path, bundle_state_path=bundle_path)
state.set_snapshot_paths(bundle_state_path=None)
assert state.get_final_path() == final_path
assert state.get_bundle_state_path() is None
docker/packages/google_mail/tests/test_mailbox_service.py
+1453
View File
File diff suppressed because it is too large Load Diff
docker/packages/google_mail/tests/test_multi_mailbox.py
+267
View File
@@ -0,0 +1,267 @@
"""Tests for multi-mailbox support."""
import asyncio
import json
import pytest
from google_mail.models import MultiMailboxData
from google_mail.services.mailbox import MailboxService
@pytest.fixture
def multi_mailbox_services(tmp_path):
"""Create two mailbox services with different data."""
work_path = tmp_path / "work.json"
work_path.write_text(
json.dumps(
{
"mailbox": {"email": "alex@acmecorp.com", "name": "Alex Morgan"},
"contacts": [{"email": "pam@acmecorp.com", "name": "Pam Chen"}],
"folders": [],
"emails": [
{
"email_id": "1",
"folder": "INBOX",
"subject": "Work email",
"from_addr": "pam@acmecorp.com",
"to_addr": "alex@acmecorp.com",
"date": "2026-04-14T09:00:00Z",
"message_id": "<work1@acmecorp.com>",
"body_text": "This is a work email.",
"is_read": False,
"is_important": False,
}
],
"next_email_id": 10,
}
)
)
personal_path = tmp_path / "personal.json"
personal_path.write_text(
json.dumps(
{
"mailbox": {"email": "alex.m@gmail.com", "name": "Alex"},
"contacts": [{"email": "friend@gmail.com", "name": "Best Friend"}],
"folders": [],
"emails": [
{
"email_id": "1",
"folder": "INBOX",
"subject": "Personal email",
"from_addr": "friend@gmail.com",
"to_addr": "alex.m@gmail.com",
"date": "2026-04-14T10:00:00Z",
"message_id": "<personal1@gmail.com>",
"body_text": "Hey Alex! Want to grab lunch?",
"is_read": False,
"is_important": False,
}
],
"next_email_id": 10,
}
)
)
work_svc = MailboxService(work_path)
work_svc.load()
personal_svc = MailboxService(personal_path)
personal_svc.load()
return {"work": work_svc, "personal": personal_svc}
@pytest.fixture
def outputdir(tmp_path):
out = tmp_path / "output" / "google_mail"
out.mkdir(parents=True)
return out
@pytest.fixture(autouse=True)
def _patch_server_globals(multi_mailbox_services, outputdir):
import google_mail.state as state
state.set_mailboxes(multi_mailbox_services)
state.set_snapshot_paths(final_path=outputdir / "final.json")
yield
state.set_mailboxes({})
state.set_snapshot_paths(final_path=None, bundle_state_path=None)
class TestListMailboxes:
def test_lists_all_mailboxes(self):
from google_mail.server import mail_list_mailboxes
result = json.loads(asyncio.run(mail_list_mailboxes()))
assert result["total"] == 2
ids = [m["mailbox_id"] for m in result["mailboxes"]]
assert "work" in ids
assert "personal" in ids
def test_mailbox_info(self):
from google_mail.server import mail_list_mailboxes
result = json.loads(asyncio.run(mail_list_mailboxes()))
work = next(m for m in result["mailboxes"] if m["mailbox_id"] == "work")
assert work["email"] == "alex@acmecorp.com"
assert work["email_count"] == 1
assert work["contact_count"] == 1
class TestMailboxIsolation:
def test_get_emails_from_work(self):
from google_mail.server import mail_get_emails
result = json.loads(asyncio.run(mail_get_emails(mailbox_id="work")))
assert result["total"] == 1
assert result["emails"][0]["subject"] == "Work email"
def test_get_emails_from_personal(self):
from google_mail.server import mail_get_emails
result = json.loads(asyncio.run(mail_get_emails(mailbox_id="personal")))
assert result["total"] == 1
assert result["emails"][0]["subject"] == "Personal email"
def test_contacts_are_isolated(self):
from google_mail.server import mail_get_contacts
work_result = json.loads(asyncio.run(mail_get_contacts(mailbox_id="work")))
assert len(work_result["contacts"]) == 1
assert work_result["contacts"][0]["name"] == "Pam Chen"
personal_result = json.loads(asyncio.run(mail_get_contacts(mailbox_id="personal")))
assert len(personal_result["contacts"]) == 1
assert personal_result["contacts"][0]["name"] == "Best Friend"
def test_send_email_in_correct_mailbox(self):
from google_mail.server import mail_send_email
result = json.loads(
asyncio.run(
mail_send_email(
to="pam@acmecorp.com",
subject="Test from work",
body="Sent from work mailbox",
mailbox_id="work",
)
)
)
assert result["status"] == "sent"
assert result["email"]["from"] == "alex@acmecorp.com"
def test_send_email_wrong_mailbox_contact(self):
from google_mail.server import mail_send_email
# pam@acmecorp.com is a work contact, not personal
result = json.loads(
asyncio.run(
mail_send_email(
to="pam@acmecorp.com",
subject="Should fail",
body="Wrong mailbox",
mailbox_id="personal",
)
)
)
assert "error" in result # recipient not found in personal contacts
def test_invalid_mailbox_id(self):
from google_mail.server import mail_get_emails
with pytest.raises(ValueError, match="not found"):
asyncio.run(mail_get_emails(mailbox_id="nonexistent"))
def test_write_to_one_doesnt_affect_other(self):
from google_mail.server import mail_create_folder, mail_get_folders
# Create folder in work
asyncio.run(mail_create_folder(folder_name="Projects", mailbox_id="work"))
# Verify it's in work
work_folders = json.loads(asyncio.run(mail_get_folders(mailbox_id="work")))
folder_names = [f["name"] for f in work_folders["folders"]]
assert "Projects" in folder_names
# Verify it's NOT in personal
personal_folders = json.loads(asyncio.run(mail_get_folders(mailbox_id="personal")))
personal_names = [f["name"] for f in personal_folders["folders"]]
assert "Projects" not in personal_names
class TestMultiMailboxSnapshot:
def test_snapshot_includes_all_mailboxes(self, outputdir):
from google_mail.server import mail_send_email
final = outputdir / "final.json"
asyncio.run(
mail_send_email(
to="pam@acmecorp.com",
subject="Trigger snapshot",
body="test",
mailbox_id="work",
)
)
assert final.exists()
snapshot = json.loads(final.read_text())
# Multi-mailbox format
assert "mailboxes" in snapshot
assert "work" in snapshot["mailboxes"]
assert "personal" in snapshot["mailboxes"]
class TestMultiMailboxStateTools:
def test_export_state_returns_typed_multi_mailbox_state(self):
from google_mail.server import export_state
state = asyncio.run(export_state())
assert isinstance(state, MultiMailboxData)
assert set(state.mailboxes) == {"work", "personal"}
def test_import_state_accepts_multi_mailbox_dict(self):
import google_mail.state as mail_state
from google_mail.server import import_state, mail_list_mailboxes
state = {
"mailboxes": {
"replacement": {
"mailbox": {"email": "replacement@example.com", "name": "Replacement"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
}
}
result = asyncio.run(import_state(state))
assert result == {"ok": True}
assert set(mail_state.get_mailboxes()) == {"replacement"}
listed = json.loads(asyncio.run(mail_list_mailboxes()))
assert listed["mailboxes"][0]["mailbox_id"] == "replacement"
def test_import_state_flat_payload_replaces_multi_mailbox_registry(self):
import google_mail.state as mail_state
from google_mail.server import import_state, mail_list_mailboxes
state = {
"mailbox": {"email": "replacement@example.com", "name": "Replacement"},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
result = asyncio.run(import_state(state))
assert result == {"ok": True}
assert set(mail_state.get_mailboxes()) == {"default"}
listed = json.loads(asyncio.run(mail_list_mailboxes()))
assert listed["total"] == 1
assert listed["mailboxes"][0]["mailbox_id"] == "default"
assert listed["mailboxes"][0]["email"] == "replacement@example.com"
docker/packages/google_mail/tests/test_schema.py
+573
View File
@@ -0,0 +1,573 @@
"""Tests for schema validators on MailboxData / Email."""
import pytest
from pydantic import ValidationError
from google_mail.models import Email, MailboxData, MultiMailboxData
def _sample_email(**overrides: object) -> dict[str, object]:
base: dict[str, object] = {
"email_id": "1",
"folder": "INBOX",
"subject": "s",
"from_addr": "bob@example.com",
"to_addr": "alice@example.com",
"date": "2024-01-15T10:00:00Z",
"message_id": "<m@x>",
"body_text": "hi",
}
base.update(overrides)
return base
def test_email_accepts_list_for_addr_fields():
email = Email.model_validate(
_sample_email(
to_addr=["alice@example.com", "bob@example.com"],
cc_addr=["carol@example.com"],
bcc_addr=["dan@example.com", "eve@example.com"],
)
)
assert email.to_addr == "alice@example.com, bob@example.com"
assert email.cc_addr == "carol@example.com"
assert email.bcc_addr == "dan@example.com, eve@example.com"
def test_email_list_validator_trims_and_drops_empties():
email = Email.model_validate(_sample_email(to_addr=[" alice@example.com ", "", "bob@example.com"]))
assert email.to_addr == "alice@example.com, bob@example.com"
def test_email_addr_string_passthrough_unchanged():
email = Email.model_validate(_sample_email(to_addr="alice@example.com, bob@example.com"))
assert email.to_addr == "alice@example.com, bob@example.com"
@pytest.mark.parametrize("alias", ["labelIds", "label_ids"])
def test_email_accepts_label_aliases(alias):
email = Email.model_validate(_sample_email(**{alias: ["INBOX", "Client"]}))
assert email.labels == ["INBOX", "Client"]
assert "labelIds" not in email.model_dump(mode="json")
assert "label_ids" not in email.model_dump(mode="json")
def test_email_normalizes_labels():
email = Email.model_validate(_sample_email(labels=[" client ", "", "client", "INBOX", " "]))
assert email.labels == ["client", "INBOX"]
@pytest.mark.parametrize("field", ["email_id", "folder", "message_id"])
def test_email_rejects_empty_identity_fields(field):
with pytest.raises(ValidationError):
Email.model_validate(_sample_email(**{field: ""}))
def test_email_allows_empty_subject_and_body_for_sparse_messages():
email = Email.model_validate(_sample_email(subject="", body_text="", body_html=""))
assert email.subject == ""
assert email.body_text == ""
assert email.body_html == ""
def test_email_accepts_empty_to_addr_for_drafts():
email = Email.model_validate(_sample_email(folder="Drafts", to_addr=""))
assert email.to_addr == ""
def test_email_accepts_empty_to_addr_for_trash():
email = Email.model_validate(_sample_email(folder="Trash", to_addr=""))
assert email.to_addr == ""
def test_email_rejects_empty_to_addr_outside_drafts():
with pytest.raises(ValidationError):
Email.model_validate(_sample_email(to_addr=""))
def test_email_allows_active_message_with_only_cc_recipient():
email = Email.model_validate(_sample_email(to_addr="", cc_addr="bob@example.com"))
assert email.to_addr == ""
assert email.cc_addr == "bob@example.com"
def test_email_normalizes_empty_optional_recipient_fields_to_none():
email = Email.model_validate(_sample_email(cc_addr="", bcc_addr=" "))
assert email.cc_addr is None
assert email.bcc_addr is None
@pytest.mark.parametrize(
"field,value",
[
("from_addr", "not-an-email"),
("to_addr", "not-an-email"),
("cc_addr", "alice@example.com, not-an-email"),
("bcc_addr", ["dan@example.com", "not-an-email"]),
],
)
def test_email_rejects_invalid_addr_fields(field, value):
with pytest.raises(ValidationError):
Email.model_validate(_sample_email(**{field: value}))
def test_mailbox_data_import_coerces_email_addr_lists():
"""import_state feeds JSON through MailboxData; list-form addrs should round-trip to strings."""
data = MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [],
"folders": [],
"emails": [
_sample_email(to_addr=["alice@example.com", "bob@example.com"]),
],
"next_email_id": 2,
}
)
assert data.emails[0].to_addr == "alice@example.com, bob@example.com"
@pytest.mark.parametrize(
"payload",
[
{"mailbox": {"email": "not-an-email", "name": "Alice"}},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [{"email": "not-an-email", "name": "Bob"}],
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"groups": [{"email": "not-an-email", "name": "Team", "members": ["alice@example.com"]}],
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"groups": [{"email": "team@example.com", "name": "Team", "members": ["not-an-email"]}],
},
],
)
def test_mailbox_data_rejects_invalid_email_addresses(payload):
with pytest.raises(ValidationError):
MailboxData.model_validate(payload)
@pytest.mark.parametrize(
"payload",
[
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"unexpected": True,
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice", "unexpected": True},
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [{"email": "bob@example.com", "name": "Bob", "unexpected": True}],
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"folders": [{"name": "Projects", "unexpected": True}],
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(unexpected=True)],
},
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(
attachments=[
{
"filename": "notes.txt",
"content_type": "text/plain",
"content_base64": "aGk=",
"unexpected": True,
}
]
)
],
},
],
)
def test_mailbox_data_rejects_unknown_state_fields(payload):
with pytest.raises(ValidationError):
MailboxData.model_validate(payload)
def test_mailbox_data_still_parses_json_datetime_strings():
data = MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(date="2024-01-15T10:00:00Z")],
"next_email_id": 2,
}
)
assert data.emails[0].date.isoformat() == "2024-01-15T10:00:00+00:00"
def test_mailbox_data_rejects_non_positive_next_email_id():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"next_email_id": 0,
}
)
def test_mailbox_data_rejects_duplicate_folder_names():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"folders": [{"name": "Projects"}, {"name": "Projects"}],
}
)
def test_mailbox_data_rejects_empty_folder_name():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"folders": [{"name": ""}],
}
)
@pytest.mark.parametrize("folder_name", ["INBOX", "Sent", "Drafts", "Trash", "Scheduled"])
def test_mailbox_data_rejects_system_folders_as_custom_folders(folder_name):
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"folders": [{"name": folder_name}],
}
)
def test_mailbox_data_rejects_case_insensitive_duplicate_contact_emails():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [
{"email": "bob@example.com", "name": "Bob"},
{"email": "BOB@example.com", "name": "Other Bob"},
],
}
)
def test_mailbox_data_rejects_duplicate_email_ids():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(email_id="1", subject="First"),
_sample_email(email_id="1", subject="Second"),
],
}
)
def test_mailbox_data_rejects_next_email_id_that_collides_with_numeric_email_id():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(email_id="5")],
"next_email_id": 5,
}
)
def test_mailbox_data_rejects_next_email_id_below_existing_numeric_email_id():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(email_id="7")],
"next_email_id": 6,
}
)
def test_mailbox_data_allows_next_email_id_with_non_numeric_email_ids():
data = MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(email_id="draft-1")],
"next_email_id": 1,
}
)
assert data.next_email_id == 1
def test_mailbox_data_rejects_email_with_unknown_folder():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(folder="Projects")],
}
)
def test_mailbox_data_accepts_empty_attachment_content():
data = MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(
attachments=[
{
"filename": "empty.txt",
"content_type": "text/plain",
"content_base64": "",
}
]
)
],
"next_email_id": 2,
}
)
assert data.emails[0].attachments[0].size == 0
def test_mailbox_data_rejects_invalid_attachment_base64():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(
attachments=[
{
"filename": "broken.txt",
"content_type": "text/plain",
"content_base64": "not base64!",
}
]
)
],
}
)
def test_mailbox_data_rejects_empty_attachment_filename():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(
attachments=[
{
"filename": "",
"content_type": "text/plain",
"content_base64": "aGk=",
}
]
)
],
}
)
def test_mailbox_data_rejects_duplicate_attachment_filenames_per_email():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(
attachments=[
{
"filename": "notes.txt",
"content_type": "text/plain",
"content_base64": "aGk=",
},
{
"filename": "notes.txt",
"content_type": "text/plain",
"content_base64": "Ynll",
},
]
)
],
}
)
def test_mailbox_data_accepts_scheduled_email_with_scheduled_time():
data = MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [
_sample_email(
folder="Scheduled",
scheduled_time="2024-01-16T10:00:00Z",
)
],
"next_email_id": 2,
}
)
scheduled_time = data.emails[0].scheduled_time
assert scheduled_time is not None
assert scheduled_time.isoformat() == "2024-01-16T10:00:00+00:00"
def test_mailbox_data_rejects_scheduled_folder_without_scheduled_time():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(folder="Scheduled")],
}
)
def test_mailbox_data_rejects_scheduled_time_outside_scheduled_folder():
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email(folder="INBOX", scheduled_time="2024-01-16T10:00:00Z")],
}
)
def test_mailbox_data_accepts_group_members_from_contacts_and_mailbox_owner():
data = MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [
{"email": "bob@example.com", "name": "Bob"},
],
"groups": [
{"email": "team@example.com", "name": "Team", "members": ["alice@example.com", "bob@example.com"]},
],
}
)
group = data.get_group_by_email("team@example.com")
assert group is not None
assert group.members == ["alice@example.com", "bob@example.com"]
@pytest.mark.parametrize(
"members",
[
[],
[""],
["bob@example.com", "bob@example.com"],
["team@example.com"],
["unknown@example.com"],
],
)
def test_mailbox_data_rejects_invalid_group_members(members):
with pytest.raises(ValidationError):
MailboxData.model_validate(
{
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [
{"email": "bob@example.com", "name": "Bob"},
],
"groups": [
{"email": "team@example.com", "name": "Team", "members": members},
],
}
)
def test_multi_mailbox_data_validates_nested_mailboxes():
state = MultiMailboxData.model_validate(
{
"mailboxes": {
"work": {
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"emails": [_sample_email()],
"next_email_id": 2,
}
}
}
)
assert state.mailboxes["work"].mailbox.email == "alice@example.com"
def test_multi_mailbox_data_rejects_unknown_wrapper_fields():
with pytest.raises(ValidationError):
MultiMailboxData.model_validate(
{
"mailboxes": {
"work": {
"mailbox": {"email": "alice@example.com", "name": "Alice"},
}
},
"unexpected": True,
}
)
def test_multi_mailbox_data_rejects_empty_mailbox_map():
with pytest.raises(ValidationError):
MultiMailboxData.model_validate({"mailboxes": {}})
def test_multi_mailbox_data_rejects_empty_mailbox_id():
with pytest.raises(ValidationError):
MultiMailboxData.model_validate(
{
"mailboxes": {
"": {
"mailbox": {"email": "alice@example.com", "name": "Alice"},
}
}
}
)
def test_addr_fields_advertise_array_variant_in_json_schema():
"""MCP clients validate payloads against inputSchema; the list form must be advertised."""
email_schema = MailboxData.model_json_schema()["$defs"]["Email"]["properties"]
array_variant = {"type": "array", "items": {"type": "string", "format": "email"}}
to_variants = email_schema["to_addr"]["anyOf"]
assert {"type": "string"} in to_variants
assert array_variant in to_variants
cc_variants = email_schema["cc_addr"]["anyOf"]
assert {"type": "string"} in cc_variants
assert {"type": "null"} in cc_variants
assert array_variant in cc_variants
def test_address_book_fields_advertise_email_format_in_json_schema():
schema = MailboxData.model_json_schema()
assert schema["$defs"]["Mailbox"]["properties"]["email"]["format"] == "email"
assert schema["$defs"]["Contact"]["properties"]["email"]["format"] == "email"
assert schema["$defs"]["ContactGroup"]["properties"]["email"]["format"] == "email"
assert schema["$defs"]["ContactGroup"]["properties"]["members"]["items"]["format"] == "email"
assert schema["$defs"]["Email"]["properties"]["from_addr"]["format"] == "email"
def test_attachment_content_advertises_base64_encoding_in_json_schema():
attachment_schema = MailboxData.model_json_schema()["$defs"]["Attachment"]["properties"]
assert attachment_schema["content_base64"]["contentEncoding"] == "base64"
def test_non_empty_state_fields_advertise_min_length_in_json_schema():
schema = MailboxData.model_json_schema()["$defs"]
assert schema["Folder"]["properties"]["name"]["minLength"] == 1
assert schema["Attachment"]["properties"]["filename"]["minLength"] == 1
assert schema["Email"]["properties"]["email_id"]["minLength"] == 1
assert schema["Email"]["properties"]["folder"]["minLength"] == 1
assert schema["Email"]["properties"]["message_id"]["minLength"] == 1
def test_multi_mailbox_data_advertises_non_empty_mailboxes_in_json_schema():
mailboxes_schema = MultiMailboxData.model_json_schema()["properties"]["mailboxes"]
assert mailboxes_schema["minProperties"] == 1
assert mailboxes_schema["propertyNames"]["minLength"] == 1
docker/packages/google_mail/tests/test_viewer.py
+254
View File
@@ -0,0 +1,254 @@
"""Regression tests for the google_mail viewer HTTP app.
These tests exercise the viewer routes against a real ``MailboxService`` loaded
from fixture data. The viewer reads from the shared ``google_mail.state``
mailbox registry so it uses the same state surface as the MCP server.
"""
from __future__ import annotations
import json
import tempfile
from pathlib import Path
import pytest
from starlette.testclient import TestClient
import google_mail.state as state_mod
from google_mail.services.mailbox import MailboxService
from google_mail.viewer import create_mail_viewer_app
SAMPLE_DATA = {
"mailbox": {"email": "alice@example.com", "name": "Alice"},
"contacts": [
{"email": "bob@example.com", "name": "Bob"},
],
"groups": [
{
"email": "team@example.com",
"name": "Team",
"members": ["alice@example.com", "bob@example.com"],
},
],
"folders": [],
"emails": [
{
"email_id": "1",
"folder": "INBOX",
"subject": "Hello Alice",
"from_addr": "bob@example.com",
"to_addr": "alice@example.com",
"cc_addr": "carol@example.com",
"date": "2024-01-15T10:00:00Z",
"message_id": "<msg1@example.com>",
"body_text": "Hi Alice.",
"is_read": False,
"is_important": True,
"attachments": [
{
"filename": "doc.pdf",
"content_type": "application/pdf",
"content_base64": "SGVsbG8=",
}
],
},
{
"email_id": "2",
"folder": "INBOX",
"subject": "Re: meeting",
"from_addr": "carol@example.com",
"to_addr": "alice@example.com",
"date": "2024-01-14T09:00:00Z",
"message_id": "<msg2@example.com>",
"body_text": "See you Monday.",
"is_read": True,
"is_important": False,
},
],
"next_email_id": 3,
}
@pytest.fixture
def populated_mailbox(monkeypatch):
"""Install a ``default`` MailboxService into the state registry."""
monkeypatch.delenv("MCP_PROXY_TOKEN", raising=False)
with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f:
json.dump(SAMPLE_DATA, f)
path = Path(f.name)
service = MailboxService(path)
service.load()
original = dict(state_mod.get_mailboxes())
state_mod.set_mailboxes({"default": service})
try:
yield service
finally:
state_mod.set_mailboxes(original)
path.unlink(missing_ok=True)
@pytest.fixture
def client(populated_mailbox) -> TestClient:
return TestClient(create_mail_viewer_app(), raise_server_exceptions=False)
class TestViewerRoutes:
def test_root_serves_html(self, client: TestClient) -> None:
resp = client.get("/")
assert resp.status_code == 200
assert "<title>Mail</title>" in resp.text
def test_folders_returns_counts(self, client: TestClient) -> None:
resp = client.get("/api/folders")
assert resp.status_code == 200
folders = {f["name"]: f for f in resp.json()["folders"]}
assert "INBOX" in folders
assert folders["INBOX"]["total"] == 2
assert folders["INBOX"]["unread"] == 1
def test_emails_list_returns_summary(self, client: TestClient) -> None:
resp = client.get("/api/emails?folder=INBOX")
assert resp.status_code == 200
body = resp.json()
assert body["total"] == 2
ids = {e["email_id"] for e in body["emails"]}
assert ids == {"1", "2"}
first = next(e for e in body["emails"] if e["email_id"] == "1")
assert first["has_attachments"] is True
assert first["is_important"] is True
def test_emails_search_filters(self, client: TestClient) -> None:
resp = client.get("/api/emails?search=meeting")
assert resp.status_code == 200
body = resp.json()
assert body["total"] == 1
assert body["emails"][0]["email_id"] == "2"
def test_email_detail_returns_full(self, client: TestClient) -> None:
resp = client.get("/api/emails/1")
assert resp.status_code == 200
email = resp.json()["email"]
assert email["body_text"] == "Hi Alice."
assert email["cc_addr"] == "carol@example.com"
assert email["attachments"][0]["filename"] == "doc.pdf"
def test_email_detail_marks_read(self, client: TestClient, populated_mailbox) -> None:
# The unread email becomes read after viewing.
client.get("/api/emails/1")
email = populated_mailbox.get_email("1")
assert email.is_read is True
def test_email_detail_missing_returns_404(self, client: TestClient) -> None:
resp = client.get("/api/emails/does-not-exist")
assert resp.status_code == 404
assert resp.json()["error"]
def test_contacts_route(self, client: TestClient) -> None:
resp = client.get("/api/contacts")
assert resp.status_code == 200
contacts = resp.json()["contacts"]
assert {"email": "bob@example.com", "name": "Bob"} in contacts
groups = resp.json()["groups"]
team = next(g for g in groups if g["email"] == "team@example.com")
assert team["members"] == ["alice@example.com", "bob@example.com"]
def test_stats_route(self, client: TestClient) -> None:
resp = client.get("/api/stats")
assert resp.status_code == 200
stats = resp.json()
assert stats["total_emails"] == 2
assert stats["total_unread"] == 1
assert stats["mailbox"]["email"] == "alice@example.com"
class TestAuth:
def test_proxy_token_required_when_set(self, populated_mailbox, monkeypatch) -> None:
monkeypatch.setenv("MCP_PROXY_TOKEN", "secret")
client = TestClient(create_mail_viewer_app(), raise_server_exceptions=False)
unauth = client.get("/api/folders")
assert unauth.status_code == 403
ok = client.get("/api/folders", headers={"x-proxy-token": "secret"})
assert ok.status_code == 200
class TestMultiMailbox:
@pytest.fixture
def multi_mailbox(self, monkeypatch):
monkeypatch.delenv("MCP_PROXY_TOKEN", raising=False)
def _load(email: str) -> MailboxService:
data = {
"mailbox": {"email": email, "name": email},
"contacts": [],
"folders": [],
"emails": [],
"next_email_id": 1,
}
with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f:
json.dump(data, f)
p = Path(f.name)
svc = MailboxService(p)
svc.load()
return svc
original = dict(state_mod.get_mailboxes())
state_mod.set_mailboxes(
{
"alice": _load("alice@example.com"),
"bob": _load("bob@example.com"),
}
)
try:
yield
finally:
state_mod.set_mailboxes(original)
def test_defaults_to_first_mailbox_when_no_default(self, multi_mailbox) -> None:
client = TestClient(create_mail_viewer_app(), raise_server_exceptions=False)
resp = client.get("/api/stats")
assert resp.status_code == 200
# Sorted ids → "alice" comes first.
assert resp.json()["mailbox"]["email"] == "alice@example.com"
def test_mailbox_id_query_param_selects(self, multi_mailbox) -> None:
client = TestClient(create_mail_viewer_app(), raise_server_exceptions=False)
resp = client.get("/api/stats?mailbox_id=bob")
assert resp.status_code == 200
assert resp.json()["mailbox"]["email"] == "bob@example.com"
def test_unknown_mailbox_id_returns_404(self, multi_mailbox) -> None:
client = TestClient(create_mail_viewer_app(), raise_server_exceptions=False)
resp = client.get("/api/folders?mailbox_id=nope")
assert resp.status_code == 404
class TestLazyInit:
"""Regression: when the registry is empty, hitting the viewer must not 500.
The viewer used to reference a non-existent server mailbox attribute.
With the registry empty, ``init_state()`` should be invoked and a default
mailbox materialized.
"""
def test_empty_registry_initializes_default(self, monkeypatch) -> None:
monkeypatch.delenv("MCP_PROXY_TOKEN", raising=False)
monkeypatch.delenv("BUNDLEDIR", raising=False)
monkeypatch.delenv("INPUTDIR", raising=False)
monkeypatch.delenv("OUTPUTDIR", raising=False)
monkeypatch.delenv("BUNDLE_OUTPUT_DIR", raising=False)
original = dict(state_mod.get_mailboxes())
state_mod.set_mailboxes({})
try:
client = TestClient(create_mail_viewer_app(), raise_server_exceptions=False)
for path in ("/api/folders", "/api/emails", "/api/contacts", "/api/stats"):
resp = client.get(path)
assert resp.status_code == 200, f"{path} → {resp.status_code}: {resp.text}"
assert "default" in state_mod.get_mailboxes()
finally:
state_mod.set_mailboxes(original)
docker/packages/jira/CAPABILITIES.md
+48
View File
@@ -0,0 +1,48 @@
# Jira Capabilities
A mock Jira project tracker with issues, sprints, workflow transitions, time tracking, watchers, comments, attachments, and issue linking.
## What the agent can do
**Create and manage issues.** Create issues (Stories, Tasks, Bugs, Epics, Sub-tasks), update fields (summary, description, priority, assignee, labels), delete, and search by JQL-style queries (with `~` fuzzy-match operator support). Search supports common equality, `IN`, `!=`, `NOT IN`, `IS EMPTY`, and date comparison filters for core fields including assignee, priority, status, statusCategory, resolution, parent, fixVersion, due, sprint, labels, and components. Browse issues by project or by epic. Link issues together (blocks, relates to, duplicates, clones).
**Workflow transitions.** Move issues through configured workflows. The default mock workflow is To Do → In Progress → In Review → Done (reopen also supported), and admin tools can add custom statuses or workflow transitions. The agent must use proper transitions — cannot jump directly from "To Do" to "Done" unless that transition exists. Query available transitions from the current status. Tests whether the agent understands process flow.
**Sprint management.** Discover seeded Scrum/Kanban boards, create admin-managed boards, create sprints on Scrum boards, view sprints and their issues, update sprint details (name, dates, state). Browse issues assigned to a specific sprint.
**Time tracking.** Log time spent on issues using Jira notation (e.g., "2h 30m", "1d 4h"). Set and update original and remaining time estimates. View all worklogs for an issue with a summary comparing estimated vs. actual time.
**Collaboration.** Add comments, attach files (with automatic MIME type detection), and manage watchers (add, remove, list who is watching an issue).
**User management.** Discover Jira users, inspect the currently authenticated user, and let admins create additional users for seeded worlds. Issue assignees, reporters, creators, watchers, comments, worklogs, and attachments reference state-level users by `accountId`. Tool-authored records use `currentUserAccountId`.
## Coverage gaps
- No board configuration tools beyond admin board creation; boards can also be seeded/imported
- No project creation or configuration
- No permissions or role-based access
- No notifications or @mentions
- No dashboard or reporting tools
- No bulk operations
- Sprint search uses configured fields named `Sprint`, with `customfield_10002` kept as the default mock convention
- No full workflow-screen or workflow-scheme modeling; custom status/transition tools cover the lightweight workflow cases
## Toolsets
34 tools total, including state import/export. `all` / `jira_all` contains the 32 non-state tools. Toolsets map to `WORLDBENCH_TOOL_SETS` values (prefixed form — e.g., `jira_issues`).
| Toolset | Tools | Description |
|---------|-------|-------------|
| `all` / `jira_all` | 32 | Everything |
| `read` / `jira_read` | 15 | All read-only tools |
| `write` / `jira_write` | 17 | All write tools |
| `jira_users` | 3 | Current-user lookup, user discovery, and admin user creation |
| `jira_issues` | 10 | Core issue CRUD: create, get, update, delete, search, project/epic issues, links, search fields |
| `jira_workflow` | 4 | Status transitions and workflow configuration: `create_status`, `get_transitions`, `transition_issue`, `upsert_workflow_transition` |
| `jira_sprints` | 6 | Sprint management: discover/create boards, create/update sprints, get sprints, get sprint issues |
| `jira_time` | 3 | Time tracking: add worklog, get worklogs, update estimate |
| `jira_collaboration` | 6 | Comments, watchers, attachments: add/get attachments, add comment, add/remove/get watchers |
| `jira_admin` | 7 | Admin-only operations: create users/boards, create/update sprint, delete issue, create statuses, configure workflow transitions |
| `jira_core` | 10 | Baseline work tracking: search, get/update issue, add comment, project issues, transitions, create issue, current-user/user lookup |
| `jira_toolathlon_legacy` | 15 | Legacy Toolathlon tool subset (pre-integration) |
| `jira_state` | 2 | `export_state`, `import_state` for fixture seeding and grading |
docker/packages/jira/README.md
+36
View File
@@ -0,0 +1,36 @@
# Jira Mock MCP Server
Python mock Jira MCP server for offline Syntara tasks. It stores all service state in JSON and validates state with Pydantic models before writes.
## Run
```bash
uv run --package jira-mock python -m jira_mock
```
## Test
```bash
uv run pytest packages/jira/tests
```
## Available Tool Areas
- Issues: search, get, create, update, delete, project/epic issue listing, and issue linking.
- Workflow: list transitions, transition issues, create custom statuses, and define workflow transitions.
- Sprints: discover/create boards, create/update sprints on Scrum boards, list sprints, and inspect sprint issues.
- Time tracking: add worklogs, list worklogs, and update estimates.
- Collaboration: comments, watchers, and attachments.
- Users: inspect the current user, discover users, and admin-create additional users for assignment/watchers.
- State: `export_state` and `import_state` for fixture seeding and grading.
## Environment Variables
| Variable | Description |
|----------|-------------|
| `PORT` | HTTP server port. |
| `MCP_PROXY_TOKEN` | Optional proxy auth token for non-MCP viewer routes. |
| `AGENT_WORKSPACE` | Workspace root used for attachment path resolution. |
| `BUNDLEDIR` | Bundle directory used to locate seeded service state. |
| `BUNDLE_OUTPUT_DIR` | Snapshot output directory. |
| `OUTPUTDIR` | Legacy snapshot output directory fallback. |

Some files were not shown because too many files have changed in this diff Show More