GDPx Benchmark: Why the Best AI Agent Scored Just 12% on 206 Realistic Tasks

Published 2 November 2026 | Research benchmark, not a model launch. What happens when an AI agent is given the files, tools and time to finish a difficult professional job, rather than a perfectly packaged question? The new GDPx benchmark offers an uncomfortable answer. On 2 November 2026, SemiAnalysis published GDPx, evaluating agents across 206 demanding tasks in 11 specialist domains. Its highest reported aggregate score is 12.0%. That should make enterprise AI buyers more demanding about evidence of completed work. It does not mean AI fails at 88% of ordinary workplace tasks.

The GDPx v1.0 leaderboard is public now. This is the release of a research evaluation, not a new AI product, price change or regulatory obligation. Independent technology reporting from ShiftDelete and the separately maintained BenchLeader reference corroborate the launch and headline figures. The results remain researcher-reported, not independently replicated measurements.

GDPx results: the leading AI agents on launch day

The official leaderboard reports these aggregate scores for the leading configurations:

These are GDPx benchmark scores under one defined evaluation method, not a forecast of success on a company's HR, finance or engineering processes. The public site displays different numbers of configurations across views; its domain comparison currently shows 13 of 20. Results may be revised. A model that ranks first overall is not automatically the best for a particular business function.

What makes GDPx different from a chatbot test?

The benchmark contains 206 tasks, 11 domains and 14 specialist data partners. The domains are chip design, cybersecurity, biology, finance, legal, quantitative trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. The researchers deliberately chose areas where they expect high economic value from frontier AI, not a representative sample of all occupations. The official domain directory lists the partners and task counts.

Agents work inside environments rather than simply answering a prompt. Finance examples involve simulated transaction data rooms containing thousands of files. Security tasks involve live services and access boundaries. Hardware tasks require using engineering software and satisfying design constraints. The agent must locate evidence, choose tools, execute steps and deliver an outcome that can be checked. That is closer to real operational work than asking for a polished document in a single response.

According to the published methodology, the default time limit is eight hours, although some tasks are shorter. Where available, each model uses its native agent harness, including Claude Code, Codex or Antigravity; otherwise the researchers use OpenCode. Some tasks are graded as binary pass/fail, while others receive partial credit. The headline score averages the intended grading approach across three attempts. Domains have equal weight, and partners within a domain are weighted equally.

That creates an essential qualification: GDPx compares model-and-agent-system configurations, not bare models with identical toolchains. Harness design, context management, tool access and safety policies can all influence outcomes. Procurement teams should resist converting the table into a universal model-quality ranking.

Why a 12% score is not an 88% enterprise failure rate

Three different things are easy to confuse. First, GDPx's tasks were deliberately selected to challenge frontier systems. Second, its score is an aggregate across distinct domains and grading methods, not a simple count of everyday jobs completed. Third, a real organization may operate much narrower workflows with structured inputs, reliable APIs, defined approval gates and human support. None of those environments can be represented by a single 12% number.

But the opposite interpretation is also wrong. Difficult tasks matter because enterprise agents rarely work in ideal conditions. An integration may return incomplete data; a user may have only read permissions; an approval may be missing; or the right answer may be to stop rather than improvise. The value of GDPx is that it tests whether an agent can make progress through such complexity, not merely sound competent when describing it.

Different leaders by domain, and the safety-refusal problem

In SemiAnalysis's domain analysis, Opus 5.5 leads quant trading, hardware design and chip design. Gemini 4 Carbon leads biology and narrowly leads software engineering. GPT-6 Astra leads recursive self-improvement and materials science. The authors report near-zero results for almost every model on their unusually demanding finance and legal tasks. This does not establish that AI cannot assist lawyers or finance teams; it describes these specific tasks and grading rules.

The authors also report that safety classifiers reduced some Opus 5.5 and Astra results in cybersecurity, biology and alignment. That makes a leaderboard comparison more subtle. Refusing a dangerous instruction can lower a task score while protecting the organization. Conversely, attempting a risky task is not proof that a model is suitable for production. For enterprise selection, report task completion and policy compliance as separate dimensions. The GDPx cybersecurity domain description shows how complex the simulated access-control environments can be.

The official cost-versus-performance view also makes a useful point: the highest score and the lowest average cost per task are not the same objective. Benchmark task cost is not an enterprise quote, however. Actual economics depend on the deployment channel, model contract, tools, retries and human review.

Five production acceptance tests I would require

1. Completion, not confident narration

Define a verifiable outcome before running the agent. In an HR integration case, that might mean finding a discrepancy, identifying its source record and drafting a corrective action with supporting evidence. In finance, it could mean reconciling a defined record set with all exceptions accounted for. A polished explanation without a correct result should not pass.

2. Identity and permission boundaries

Test the same workflow under authorized and restricted roles. Does the agent recognize the difference between reading a record and changing it? Does it request approval before a consequential write? Does it handle access-denied errors without attempting a workaround? In SAP SuccessFactors or other systems of record, natural-language confidence never replaces authorization controls and an audit trail.

3. Evidence quality and uncertainty

Introduce missing documents, contradictory fields and outdated instructions. Measure unsupported assertions separately from completion. A safe agent should cite the evidence it actually retrieved, flag unresolved conflicts and escalate when a decision cannot be justified.

4. Cost per verified outcome

Record model charges, tool usage, elapsed time, retries, human corrections and rework. A low token price can still produce an expensive business result if the workflow loops or fails repeatedly. Compare candidate systems using representative tasks and the actual deployment contracts.

5. Interruption and recovery

Interrupt a task, revoke a permission, delay an API response or return a partial write. Verify that retries are bounded, state changes are idempotent, the audit trail remains intact and a human can take over. These tests matter especially for agents connected to payroll, recruiting, procurement or customer records.

A practical two-week enterprise AI evaluation

Days 1–3: assemble the work. Select 15–25 representative tasks, including routine cases, difficult exceptions and situations where stopping is the correct action. Document the expected result, source of truth, permitted data, tools, access level and accountable reviewer. Avoid using live personal or financial records without appropriate authorization and controls.

Days 4–7: run repeatable trials. Execute each task multiple times using a fixed rubric. Capture model version, agent harness, tool configuration, outcome, latency, cost and human intervention. Classify failures by cause: missing context, misunderstood requirement, tool error, refusal, permission boundary or flawed result.

Days 8–10: challenge the boundaries. Add conflicting data, expired access, partial tool failures and unexpected responses. Confirm the agent does not invent successful execution, silently widen its privileges or perform unapproved changes. Have domain owners judge both the business outcome and the path taken.

Days 11–14: set the operating mode. Classify each workflow as autonomous, human-approved, advisory-only or not ready. Establish thresholds for correctness, safe behavior, escalation, latency and total cost. Assign an owner to revisit the decision as models, tools and business processes change. This blueprint is a practitioner recommendation, not a GDPx certification or vendor requirement.

Important limitations before quoting the result

Not a representative jobs survey: GDPx intentionally selects high-difficulty tasks in chosen domains. Not a pure model comparison: native agent harnesses differ. Not independently reproduced: the public evidence reviewed here does not establish an external replication of the complete task set. Not an unconditional failure count: grading mixes binary and partial outcomes, while safety refusals affect some domains. Not proof of model availability: appearing on a leaderboard does not establish that every enterprise can buy or deploy the configuration today. These limitations should travel with the headline.

The benchmark and results were published on 2 November 2026 and are publicly accessible. There is no new customer-effective date, automatic tenant change or regulatory obligation attached to this research. Review the live results, methodology and independent transcription before using the numbers in a business case.

The architectural takeaway

Enterprise AI is moving from answering questions to carrying out multistep work. GDPx does not say agents are useless; it exposes the gap between describing a task convincingly and finishing it correctly. I would treat the benchmark as a reason to build domain-specific acceptance tests, not as a procurement shortcut. Before granting an agent production write access, require a verified result, enforceable permissions, a recoverable failure path and a named human owner. Those controls matter more than a single leaderboard position.

Sources: SemiAnalysis launch and methodology, 2 November 2026; GDPx v1.0 official leaderboard; GDPx domain directory; ShiftDelete technology coverage, 2 November 2026; BenchLeader reference, 2 November 2026. Rankings are researcher-reported and may change. Implementation guidance is original editorial analysis.