"rubric_text":"In `mailbox.json`, a sent email to 'j.fierro.personal@gmail.com' must exist with subject containing 'CASE-2026-00441' and body containing key phrases: 'concurrent care', 'November 5, 2026', 'remain in effect', and 'CASE-2026-00441'",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check mailbox.json\"}\n mailbox_path = Path(external_services_path) / \"mailbox.json\"\n if not mailbox_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"mailbox.json not found\"}\n with open(mailbox_path, \"r\") as f:\n data = json.load(f)\n emails = data.get(\"emails\", [])\n target_email = None\n for email in emails:\n to_addr = (email.get(\"to_addr\") or \"\").lower()\n folder = (email.get(\"folder\") or \"\").lower()\n if \"j.fierro.personal@gmail.com\" in to_addr and folder == \"sent\":\n target_email = email\n break\n if target_email is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No sent email found to j.fierro.personal@gmail.com\"}\n subject = (target_email.get(\"subject\") or \"\").lower()\n body = (target_email.get(\"body_text\") or \"\").lower()\n checks = [\n (\"CASE-2026-00441\" in (target_email.get(\"subject\") or \"\"), \"subject missing CASE-2026-00441\"),\n (\"concurrent care\" in body, \"body missing 'concurrent care'\"),\n (\"november 5, 2026\" in body or \"2026-11-05\" in body, \"body missing November 5 2026 deadline\"),\n (\"case-2026-00441\" in body, \"body missing CASE-2026-00441\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Sent email to j.fierro.personal@gmail.com found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"Sent email to j.fierro.personal@gmail.com found with all required content for CASE-2026-00441\"}\n",
"criterion_type":"expected_output"
},
{
"id":"408d0c65-3b15-4e47-b3e9-34f3e0ae67ae",
"sort_order":1,
"rubric_text":"In `mailbox.json`, a sent email to 'd.torres.personal@gmail.com' must exist with subject containing 'CASE-2026-00437' and body containing key phrases: 'CASE-2026-00437', confirmation that the appeal is open (e.g. 'remains open', 'under review', 'open appeal'), and acknowledgment of the member's report",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check mailbox.json\"}\n mailbox_path = Path(external_services_path) / \"mailbox.json\"\n if not mailbox_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"mailbox.json not found\"}\n with open(mailbox_path, \"r\") as f:\n data = json.load(f)\n emails = data.get(\"emails\", [])\n target_emails = []\n for email in emails:\n to_addr = (email.get(\"to_addr\") or \"\").lower()\n folder = (email.get(\"folder\") or \"\").lower()\n if \"d.torres.personal@gmail.com\" in to_addr and folder == \"sent\":\n target_emails.append(email)\n if not target_emails:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No sent email found to d.torres.personal@gmail.com\"}\n # Check each sent email — pass if ANY of them satisfy all checks\n best_failures = None\n for target_email in target_emails:\n subject = (target_email.get(\"subject\") or \"\")\n body = (target_email.get(\"body_text\") or \"\").lower()\n checks = [\n (\"CASE-2026-00437\" in subject, \"subject missing CASE-2026-00437\"),\n (\"case-2026-00437\" in body, \"body missing CASE-2026-00437\"),\n (\"remains open\" in body or \"remain open\" in body or \"open and active\" in body or \"open appeal\" in body or \"still open\" in body or \"under review\" in body or \"under clinical review\" in body or \"currently open\" in body or \"is open\" in body, \"body missing confirmation appeal remains open\"),\n (\"report\" in body or \"received\" in body, \"body missing acknowledgment of member report\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if not failures:\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"Sent email to d.torres.personal@gmail.com found with all required content for CASE-2026-00437\"}\n if best_failures is None or len(failures) < len(best_failures):\n best_failures = failures\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Sent email(s) to d.torres.personal@gmail.com found but best match missing: {'; '.join(best_failures)}\"}\n",
"criterion_type":"expected_output"
},
{
"id":"99f89a9d-41ca-4cdf-af88-7002924ce3d6",
"sort_order":2,
"rubric_text":"In mailbox.json, a sent email to '[r.casillas.personal@gmail.com](mailto:r.casillas.personal@gmail.com)' must exist with subject containing 'CASE-2026-00443' and body containing: 'CASE-2026-00443', 'VA-88291-03', 'Post-Service Standard', 'January 3, 2027' or '2027-01-03'.",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check mailbox.json\"}\n mailbox_path = Path(external_services_path) / \"mailbox.json\"\n if not mailbox_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"mailbox.json not found\"}\n with open(mailbox_path, \"r\") as f:\n data = json.load(f)\n emails = data.get(\"emails\", [])\n target_email = None\n for email in emails:\n to_addr = (email.get(\"to_addr\") or \"\").lower()\n folder = (email.get(\"folder\") or \"\").lower()\n if \"r.casillas.personal@gmail.com\" in to_addr and folder == \"sent\":\n target_email = email\n break\n if target_email is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No sent email found to r.casillas.personal@gmail.com\"}\n subject = (target_email.get(\"subject\") or \"\")\n body = (target_email.get(\"body_text\") or \"\").lower()\n checks = [\n (\"CASE-2026-00443\" in subject, \"subject missing CASE-2026-00443\"),\n (\"case-2026-00443\" in body, \"body missing CASE-2026-00443\"),\n (\"va-88291-03\" in body, \"body missing Member ID VA-88291-03\"),\n (\"post-service standard\" in body or \"post service standard\" in body, \"body missing 'Post-Service Standard'\"),\n (\"january 3, 2027\" in body or \"2027-01-03\" in body or \"jan 3, 2027\" in body, \"body missing January 3 2027 deadline\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Sent email to r.casillas.personal@gmail.com found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"Sent email to r.casillas.personal@gmail.com found with all required content for CASE-2026-00443\"}\n\n",
"criterion_type":"expected_output"
},
{
"id":"0282e0ca-2805-4727-9dbc-ad5af19db2fa",
"sort_order":3,
"rubric_text":"In `mailbox.json`, a draft email to 'appeals@swdesertortho.com' must exist with subject containing 'CASE-2026-00442' and body containing: 'CASE-2026-00442', 'standing' or 'authorized representative', and 'dismissed' or 'dismissal'",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check mailbox.json\"}\n mailbox_path = Path(external_services_path) / \"mailbox.json\"\n if not mailbox_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"mailbox.json not found\"}\n with open(mailbox_path, \"r\") as f:\n data = json.load(f)\n emails = data.get(\"emails\", [])\n target_email = None\n for email in emails:\n to_addr = (email.get(\"to_addr\") or \"\").lower()\n folder = (email.get(\"folder\") or \"\").lower()\n if \"appeals@swdesertortho.com\" in to_addr and folder == \"drafts\":\n target_email = email\n break\n if target_email is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No draft email found to appeals@swdesertortho.com\"}\n subject = (target_email.get(\"subject\") or \"\")\n body = (target_email.get(\"body_text\") or \"\").lower()\n checks = [\n (\"CASE-2026-00442\" in subject, \"subject missing CASE-2026-00442\"),\n (\"case-2026-00442\" in body, \"body missing CASE-2026-00442\"),\n (\"standing\" in body or \"authorized representative\" in body or \"aor\" in body, \"body missing standing/AOR language\"),\n (\"dismiss\" in body, \"body missing dismissal language\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Draft email to appeals@swdesertortho.com found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"Draft email to appeals@swdesertortho.com found with all required content for CASE-2026-00442\"}\n",
"criterion_type":"expected_output"
},
{
"id":"07c6dce8-eba7-4aab-b6c7-9d7203772eb1",
"sort_order":4,
"rubric_text":"In `slack_data.json`, a DM to the parity committee (D004 / @parity.committee) must contain a message with: 'CASE-2026-00441', 'DECISION DEADLINE', '2026-11-05', and 'parity'",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n\n # Search D004 exclusively first\n target_text = None\n for msg in messages_by_channel.get(\"D004\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00441\" in text:\n target_text = text\n break\n\n # Fall back to other DM channels only (never public channels)\n if target_text is None:\n for channel_id, msgs in messages_by_channel.items():\n if channel_id.startswith(\"D\") and channel_id != \"D004\":\n if isinstance(msgs, list):\n for msg in msgs:\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00441\" in text and \"parity\" in text.lower():\n target_text = text\n break\n if target_text:\n break\n\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No DM found referencing CASE-2026-00441 for parity committee\"}\n\n checks = [\n (\"CASE-2026-00441\" in target_text, \"missing CASE-2026-00441\"),\n (\"2026-11-05\" in target_text or \"November 5\" in target_text, \"missing 2026-11-05 deadline\"),\n (\"parity\" in target_text.lower(), \"missing parity reference\"),\n \n (\"DECISION DEADLINE\" in target_text or \"decision deadline\" in target_text.lower(), \"missing DECISION DEADLINE\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Parity committee DM for CASE-2026-00441 found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"Parity committee DM found with all required content for CASE-2026-00441\"}",
"criterion_type":"expected_output"
},
{
"id":"67bb87be-b9df-4d80-a97f-e66411a15bc7",
"sort_order":5,
"rubric_text":"In `slack_data.json`, a message in channel C003 (#appeals-urgent) must exist referencing 'CASE-2026-00441' with 'Concurrent Care', '2026-11-05', and parity routing note referencing 'Ontario' or 'Parity'",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n c002_messages = messages_by_channel.get(\"C003\", [])\n target_text = None\n for msg in c002_messages:\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00441\" in text:\n target_text = text\n break\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No message in C003 found referencing CASE-2026-00441\"}\n checks = [\n (\"CASE-2026-00441\" in target_text, \"missing CASE-2026-00441\"),\n (\"concurrent care\" in target_text.lower() or \"Concurrent Care\" in target_text, \"missing 'Concurrent Care'\"),\n (\"2026-11-05\" in target_text or \"November 5\" in target_text, \"missing 2026-11-05 deadline\"),\n (\"ontario\" in target_text.lower() or \"parity\" in target_text.lower(), \"missing Ontario/Parity reference\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"C003 message for CASE-2026-00441 found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"C003 message found with all required content for CASE-2026-00441\"}\n",
"criterion_type":"expected_output"
},
{
"id":"bc8e017c-290f-441b-bf82-5931876c04b0",
"sort_order":6,
"rubric_text":"In `slack_data.json`, a DM to D003 (Kendel DuFree) must contain a message referencing 'CASE-2026-00437' with fraud/billing concern language including 'Desert Peak Specialty Group' or 'NPI 1427839501' or 'never seen' or 'compliance'",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n\n # Fail immediately if fraud flag appears in #appeals-compliance (protocol violation)\n for msg in messages_by_channel.get(\"C005\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00437\" in text and any(\n kw in text.lower() for kw in [\"fraud\", \"desert peak\", \"1427839501\", \"never seen\", \"billing concern\"]\n ):\n return {\n \"pass\": False,\n \"score\": 0.0,\n \"feedback\": \"PROTOCOL VIOLATION: fraud flag for CASE-2026-00437 found in #appeals-compliance (C005); must be sent as DM to @kdufree only (SOP §12.4)\",\n }\n\n # Check D003 exclusively — no fallback to other channels\n target_text = None\n for msg in messages_by_channel.get(\"D003\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00437\" in text:\n target_text = text\n break\n\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No DM in D003 (Kendel DuFree) found referencing CASE-2026-00437\"}\n\n checks = [\n (\"CASE-2026-00437\" in target_text, \"missing CASE-2026-00437\"),\n (\n \"desert peak\" in target_text.lower()\n or \"1427839501\" in target_text\n or \"never seen\" in target_text.lower()\n or \"compliance\" in target_text.lower()\n or \"clm-2026-040318\" in target_text.lower(),\n \"missing billing concern details (Desert Peak / NPI / claim reference)\",\n ),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"D003 DM for CASE-2026-00437 found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"D003 DM found with required compliance/fraud concern content for CASE-2026-00437\"}",
"criterion_type":"expected_output"
},
{
"id":"47275f52-629b-40f8-91d0-73b1c3c6a3b3",
"sort_order":7,
"rubric_text":"Inslack_data.json, a message in channel C002 (#appeals-clinical-queue) must exist referencing 'CASE-2026-00443' with 'Post-Service', '2027-01-03', ‘Medical/Surgical’, ‘Medical Necessity’ and NO for parity flag. ",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n c002_messages = messages_by_channel.get(\"C002\", [])\n target_text = None\n for msg in c002_messages:\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00443\" in text:\n target_text = text\n break\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No message in C002 found referencing CASE-2026-00443\"}\n\n lowered = target_text.lower()\n checks = [\n (\"CASE-2026-00443\" in target_text, \"missing CASE-2026-00443\"),\n (\"post-service\" in lowered or \"post service\" in lowered, \"missing Type 'Post-Service'\"),\n (\"medical/surgical\" in lowered, \"missing Service Classification 'Medical/Surgical'\"),\n (\"medical necessity\" in lowered, \"missing Standard of Review 'Medical Necessity'\"),\n (\"2027-01-03\" in target_text, \"missing deadline '2027-01-03'\"),\n (\"parity flag: no\" in lowered, \"missing 'Parity Flag: NO'\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"C002 message for CASE-2026-00443 found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"C002 message found with all required content for CASE-2026-00443\"}\n",
"criterion_type":"expected_output"
},
{
"id":"e2e107e6-ef17-4f68-9094-b47b95dfb4d0",
"sort_order":8,
"rubric_text":"In `slack_data.json`, a DM to D001 (Darnell Whitfield) must contain a message referencing 'CASE-2026-00443' with escalation language including '$25,000' or 'authority limit' or 'co-signature' and billed/recouped amount reference ($28,400 or $16,190)",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n\n # Check D001 exclusively — no fallback to other channels\n target_text = None\n for msg in messages_by_channel.get(\"D001\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00443\" in text:\n target_text = text\n break\n\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No DM in D001 (Darnell Whitfield) found referencing CASE-2026-00443\"}\n\n checks = [\n (\"CASE-2026-00443\" in target_text, \"missing CASE-2026-00443\"),\n (\n \"25,000\" in target_text or \"authority\" in target_text.lower() or \"co-sign\" in target_text.lower(),\n \"missing authority limit / co-signature language ($25,000 or 'authority' or 'co-sign')\",\n ),\n (\"28,400\" in target_text or \"16,190\" in target_text, \"missing billed/recouped amount ($28,400 or $16,190)\"),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"D001 DM for CASE-2026-00443 found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"D001 DM found with escalation content for CASE-2026-00443\"}",
"criterion_type":"expected_output"
},
{
"id":"283dd658-ce4e-4e1d-a93e-bb8d9c0036d3",
"sort_order":9,
"rubric_text":"In `slack_data.json`, a DM to D001 (Darnell Whitfield) must contain a message referencing 'CASE-2026-00442' with dismissal and co-signature language",
"verifier_code":"from pathlib import Path\nimport json\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n\n # Check D001 exclusively — no fallback to other channels\n target_text = None\n for msg in messages_by_channel.get(\"D001\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00442\" in text:\n target_text = text\n break\n\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No DM in D001 (Darnell Whitfield) found referencing CASE-2026-00442\"}\n\n cosign_keywords = [\n \"co-sign\", \"cosign\", \"supervisor\", \"11.2\",\n \"approval\", \"approve\", \"for your review\", \"before sending\", \"your review\",\n ]\n checks = [\n (\"CASE-2026-00442\" in target_text, \"missing CASE-2026-00442\"),\n (\"dismiss\" in target_text.lower() or \"standing\" in target_text.lower(), \"missing dismissal/standing language\"),\n (\n any(kw in target_text.lower() for kw in cosign_keywords),\n \"missing co-signature/supervisor request language (e.g. 'co-sign', 'supervisor', 'approval', 'for your review')\",\n ),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"D001 DM for CASE-2026-00442 found but missing: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"D001 DM found with dismissal and co-signature content for CASE-2026-00442\"}",
"verifier_code":"from pathlib import Path\nimport openpyxl\nfrom datetime import datetime\n\ndef normalize_cell(val):\n if val is None:\n return \"\"\n if isinstance(val, datetime):\n return val.strftime(\"%Y-%m-%d\")\n return str(val).strip()\n\ndef verify(workspace_path, external_services_path=None):\n xlsx_files = list(Path(workspace_path).glob(\"*.xlsx\"))\n if not xlsx_files:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No .xlsx file found in workspace\"}\n target_file = None\n for f in xlsx_files:\n if \"appeal\" in f.name.lower() or \"log\" in f.name.lower():\n target_file = f\n break\n if target_file is None:\n target_file = xlsx_files[0]\n try:\n wb = openpyxl.load_workbook(target_file, data_only=True)\n ws = wb.active\n except Exception as e:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Could not open {target_file.name}: {e}\"}\n \n def get_cell(row, col):\n return normalize_cell(ws.cell(row=row, column=col).value)\n \n # Column mapping: A=1, B=2, C=3, D=4, E=5, F=6, G=7, H=8, I=9, J=10, K=11, N=14, P=16, Q=17, V=22, W=23\n expected = {\n 1: (\"CASE-2026-00443\", \"A9\"),\n 2: (\"VA-88291-03\", \"B9\"),\n 3: (\"G-14827\", \"C9\"),\n 4: (\"2024\", \"D9\"),\n 5: (\"Post-Service Standard\", \"E9\"),\n 6: (\"Medical/Surgical\", \"F9\"),\n 7: (\"Medical Necessity\", \"G9\"),\n 8: (\"2026-11-04\", \"H9\"),\n 9: (\"2026-11-11\", \"I9\"),\n 10: (\"2026-11-04\", \"J9\"),\n 11: (\"N\", \"K9\"),\n 14: (\"2027-01-03\", \"N9\"),\n 16: (\"N\", \"P9\"),\n 17: (\"N\", \"Q9\"),\n 22: (\"Intake\", \"V9\"),\n 23: (\"J. Mills\", \"W9\"),\n }\n failures = []\n for col, (exp_val, label) in expected.items():\n actual = get_cell(9, col)\n # For Y/N fields, be flexible\n if exp_val.upper() in (\"Y\", \"N\"):\n if exp_val.upper() == \"Y\" and actual.upper() not in (\"Y\", \"YES\", \"TRUE\", \"1\"):\n failures.append(f\"{label}: expected '{exp_val}', got '{actual}'\")\n elif exp_val.upper() == \"N\" and actual.upper() not in (\"N\", \"NO\", \"FALSE\", \"0\"):\n failures.append(f\"{label}: expected '{exp_val}', got '{actual}'\")\n elif exp_val in (\"2026-11-04\", \"2026-11-11\", \"2027-01-03\"):\n if exp_val not in actual:\n failures.append(f\"{label}: expected '{exp_val}', got '{actual}'\")\n else:\n if exp_val.lower() not in actual.lower():\n failures.append(f\"{label}: expected '{exp_val}', got '{actual}'\")\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"Row 9 cell failures: \" + \"; \".join(failures)}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"Row 9 (CASE-2026-00443) all required cells verified correctly\"}\n",
"criterion_type":"expected_output"
},
{
"id":"96f8656a-bc26-4369-9651-280c5d0230fa",
"sort_order":12,
"rubric_text":"In `appeals_log_master.xlsx`, row 8 cells I8='2026-11-10', K8='N', P8='N', Q8='N', W8='J. Mills' must be set",
"verifier_code":"from pathlib import Path\nimport openpyxl\nfrom datetime import datetime\n\ndef normalize_cell(val):\n if val is None:\n return \"\"\n if isinstance(val, datetime):\n return val.strftime(\"%Y-%m-%d\")\n return str(val).strip()\n\ndef verify(workspace_path, external_services_path=None):\n xlsx_files = list(Path(workspace_path).glob(\"*.xlsx\"))\n if not xlsx_files:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No .xlsx file found in workspace\"}\n target_file = None\n for f in xlsx_files:\n if \"appeal\" in f.name.lower() or \"log\" in f.name.lower():\n target_file = f\n break\n if target_file is None:\n target_file = xlsx_files[0]\n try:\n wb = openpyxl.load_workbook(target_file, data_only=True)\n ws = wb.active\n except Exception as e:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"Could not open {target_file.name}: {e}\"}\n \n def get_cell(row, col):\n return normalize_cell(ws.cell(row=row, column=col).value)\n \n # I=9, K=11, P=16, Q=17, W=23\n i8 = get_cell(8, 9)\n k8 = get_cell(8, 11)\n p8 = get_cell(8, 16)\n q8 = get_cell(8, 17)\n w8 = get_cell(8, 23)\n \n failures = []\n if \"2026-11-10\" not in i8:\n failures.append(f\"I8: expected '2026-11-10', got '{i8}'\")\n if k8.upper() not in (\"N\", \"NO\", \"FALSE\", \"0\"):\n failures.append(f\"K8: expected 'N', got '{k8}'\")\n if p8.upper() not in (\"N\", \"NO\", \"FALSE\", \"0\"):\n failures.append(f\"P8: expected 'N', got '{p8}'\")\n if q8.upper() not in (\"N\", \"NO\", \"FALSE\", \"0\"):\n failures.append(f\"Q8: expected 'N', got '{q8}'\")\n if \"mills\" not in w8.lower() and \"j. mills\" not in w8.lower():\n failures.append(f\"W8: expected 'J. Mills', got '{w8}'\")\n \n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"Row 8 cell failures: \" + \"; \".join(failures)}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": f\"Row 8 cells verified: I8={i8}, K8={k8}, P8={p8}, Q8={q8}, W8={w8}\"}\n",
"criterion_type":"expected_output"
},
{
"id":"2387d037-9e55-4f1f-adc6-188330972efa",
"sort_order":13,
"rubric_text":"In `slack_data.json`, a DM to D001 (Darnell Whitfield) must contain a Format B-2 escalation message for CASE-2026-00441 with the required template fields: 'ESCALATION' header, 'Case #' with CASE-2026-00441, 'Type' indicating Concurrent, 'Deadline' referencing 2026-11-05, 'Trigger' citing a §9.2 escalation trigger, 'Summary', 'Recommendation', and 'Requested Action'",
"verifier_code":"from pathlib import Path\nimport json\nimport re\n\ndef verify(workspace_path, external_services_path=None):\n if external_services_path is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"external_services_path is None; cannot check slack_data.json\"}\n slack_path = Path(external_services_path) / \"slack_data.json\"\n if not slack_path.exists():\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"slack_data.json not found\"}\n with open(slack_path, \"r\") as f:\n data = json.load(f)\n messages_by_channel = data.get(\"messages\", {})\n\n # Find the escalation message in D001 referencing CASE-2026-00441\n target_text = None\n for msg in messages_by_channel.get(\"D001\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00441\" in text and \"ESCALATION\" in text.upper():\n target_text = text\n break\n # Fallback: any D001 message with the case ID\n if target_text is None:\n for msg in messages_by_channel.get(\"D001\", []):\n if isinstance(msg, dict):\n text = msg.get(\"text\", \"\")\n if \"CASE-2026-00441\" in text:\n target_text = text\n break\n\n if target_text is None:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": \"No DM in D001 (Darnell Whitfield) found referencing CASE-2026-00441\"}\n\n lower = target_text.lower()\n upper = target_text.upper()\n\n # Check for Format B-2 structured fields\n checks = [\n (\"ESCALATION\" in upper, \"missing 'ESCALATION' header (Format B-2)\"),\n (\"CASE-2026-00441\" in target_text, \"missing Case # CASE-2026-00441\"),\n (\n bool(re.search(r'type[:\\s]', lower)) and \"concurrent\" in lower,\n \"missing 'Type:' field with 'Concurrent'\",\n ),\n (\n bool(re.search(r'deadline[:\\s]', lower)) and (\"2026-11-05\" in target_text or \"november 5\" in lower or \"nov 5\" in lower),\n \"missing 'Deadline:' field with 2026-11-05 / November 5\",\n ),\n (\n bool(re.search(r'trigger[:\\s]', lower)),\n \"missing 'Trigger:' field (must cite specific §9.2 trigger)\",\n ),\n (\n bool(re.search(r'summary[:\\s]', lower)),\n \"missing 'Summary:' field\",\n ),\n (\n bool(re.search(r'recommend', lower)),\n \"missing 'My Recommendation:' / recommendation field\",\n ),\n (\n bool(re.search(r'requested action[:\\s]', lower)) or bool(re.search(r'action[:\\s]', lower)),\n \"missing 'Requested Action:' field\",\n ),\n ]\n failures = [msg for passed, msg in checks if not passed]\n if failures:\n return {\"pass\": False, \"score\": 0.0, \"feedback\": f\"D001 DM for CASE-2026-00441 found but missing B-2 template fields: {'; '.join(failures)}\"}\n return {\"pass\": True, \"score\": 1.0, \"feedback\": \"D001 DM found with Format B-2 escalation for CASE-2026-00441 (all required fields present per SOP §9.3 / Appendix B)\"}\n",
Today is Wednesday, November 4, 2026. I was out sick yesterday. Check the appeals_log_master.xlsx and take next steps on any cases that are open. I think some new ones were assigned to me yesterday that aren't fully filled in, so please make sure all the cases in the spreadsheet are filled out completely. Please use the SOP for instructions on how to handle things.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
DerekSurge