A few weeks ago I covered NIST AI 800-4, the first federal-level report on the gaps in post-deployment AI monitoring. Here’s what I concluded: the value of that document wasn’t what it prescribed. It was what it admitted the field doesn’t know yet. That kind of honest accounting is rare. That said, NIST 800-4 only covered one stage in the AI risk management lifecycle
A new paper just dropped that maps the other four. What they found is not reassuring. The paper is called Open Problems in Frontier AI Risk Management. It’s dense and academic but the information is valuable for practitioners. Here’s what actually matters for enterprise leaders and builders making AI decisions right now.
The Common Story (and Why It’s Wrong)
The common story in enterprise AI governance right now is something like this: we have NIST AI RMF, we have ISO 42001, we have the EU AI Act. We have a framework. The job is implementation. The reality is more uncomfortable.
Those frameworks were mostly designed for narrow AI, the kind of AI that does one specific thing in a bounded context. A fraud detection model. A recommendation engine. A document classifier. The standards that govern those systems were written before frontier AI existed, before models that can write code, conduct research, draft legal briefs, and operate autonomously across multi-agent pipelines were anywhere near production.
Frontier AI doesn’t fit in those boxes. The 30 researchers who wrote this paper are saying, clearly and systematically and with citations, that the mismatch isn’t a minor calibration issue. It’s structural.
They catalogued unresolved problems across five stages of the AI risk management lifecycle. I counted. It’s more than two dozen. And they classified each one by type: problems where there’s no scientific consensus yet, problems where frontier AI actively breaks the established framework, and problems where there’s theoretical agreement but no one actually knows how to implement it.
That third category is the one that should make enterprise leaders uncomfortable. “Everyone agrees this matters but no one knows how to do it” describes a lot of what you’re probably already dealing with.
The Five-Stage Breakdown. And Where Each One Breaks.
Here’s the lifecycle the paper uses, adapted from ISO 31000, the international risk management standard. This is the standard your compliance team references when they talk about AI risk management. Walk through each stage and see where the problems live.
Stage 1: Risk Planning. You Can’t Scope What You Can’t Define.
Risk planning is supposed to answer: what system are we talking about, who does it affect, and what does acceptable risk mean?
For narrow AI, this is manageable. You define the system, the use case, the users. Done.
For frontier AI, the paper identifies four open problems at this stage, and the most fundamental one is this: no one has a reliable way to define what the system actually is.
Frontier AI systems are general-purpose. They’re modular. They’re reused across contexts. They get fine-tuned by downstream deployers who didn’t build the base model. The boundary between developer and deployer collapses, and when it does, so does accountability. Who owns the risk when a foundation model gets fine-tuned and deployed in a medical context by a company that had nothing to do with training it?
The paper puts it plainly: there’s no standardized way to enumerate dependencies or interface responsibilities when scoping frontier AI systems. That creates what they call blind spots at integration points, which is a polite academic way of saying nobody’s watching the seams.
The second problem at this stage is risk acceptance criteria. Traditional safety-critical industries like aviation and nuclear define acceptable risk in concrete terms. The aviation industry says the probability of a catastrophic failure should not exceed 1 in a billion per flight hour. That’s a number. You can measure against it.
Frontier AI developers mostly use capability thresholds as proxies for risk: if the model can do X, trigger mitigation Y. The paper’s critique is precise. Capability thresholds measure what a model can do, not the actual probability and severity of harm. Those are not the same thing. Building your risk acceptance framework on a proxy instead of the real measure means your mitigation decisions are always one step removed from what actually matters.
Stage 2: Risk Identification. You Can’t Find What You Don’t Know to Look For.
This stage is about systematically finding risk sources before they find you.
The paper identifies two open problems here, and the second one is the one I keep coming back to: the techniques we use to identify risks were designed for bounded, deterministic systems.
Hazard and Operability Study (HAZOP). Failure Mode and Effects Analysis (FMEA). Fishbone analysis. These are powerful tools for systems that behave predictably within defined parameters. Frontier AI is not that. Its risks emerge from non-linear interactions, from deployment context, from how humans use it over time, from multi-agent dynamics nobody fully understands yet.
The researchers are honest about this: we don’t have good methods for identifying risks that emerge from complexity, adversarial use, and sociotechnical diffusion. We’re applying 20th-century tools to 21st-century systems and then acting surprised when we keep missing things.
This connects directly to what NIST 800-4 documented about behavioral drift in production. The monitoring report found that tracking how human behavior changes through sustained AI interaction is the least mature monitoring category in the field. But the reason that monitoring is so hard is partly because risk identification upstream never built a model for it. If you don’t conceptualize a risk category before deployment, you won’t build the infrastructure to watch for it afterward.
Stage 3: Risk Analysis. The Data You Have Isn’t the Data You Need.
This is the stage where NIST 800-4 lives, and where you already know the picture is bad. The paper identifies eight open problems at this stage, more than any other stage. That’s not a coincidence. Risk analysis is where theory meets practice, and the gap is widest.
A few that matter most for enterprise decision-makers:
Capability assessments measure the wrong thing. The evaluations frontier AI developers use, the benchmarks, the red-team results, the safety evals, measure what a model can do in controlled conditions. They don’t measure real-world risk. They don’t capture how the model behaves under adversarial pressure in a production environment over months. They don’t account for the difference between a model evaluated with limited compute and scaffolding and that same model deployed with full production resources. You’re making deployment decisions based on data that systematically understates the model’s actual capability, which means you may also be understating the risk.
External assessments have a structural independence problem. The paper is blunt on this one: many assessments described as external are actually hybrid arrangements where the developer selects and finances the assessor, defines the scope, and may be able to veto publication of negative results. That’s not independence.
Post-deployment monitoring is fragmented. The data you need to understand real-world risk comes from three places: model integration and usage data, application-level usage data, and impact and incident data. Repositories like the AI Incident Database and the OECD AI Incidents Monitor exist precisely because this data isn’t flowing through any centralized channel. Right now, each stream is collected separately, incompletely, and without standardization. You can’t build a coherent risk picture from three separate silos with no agreed methodology for combining them. NIST 800-4 documented the monitoring gaps in detail. This paper documents the upstream reason those gaps exist.
Stage 4: Risk Evaluation. Accepting Risk You Haven’t Measured.
Risk evaluation is where you take your analysis and decide: acceptable or not?
The problem the paper identifies here is that frontier AI developers are making that judgment with inconsistent criteria, applied inconsistently, without safety margins, and without any agreed method for rolling up individual risk decisions into an overall deployment readiness conclusion.
In aviation, the FAA sets the acceptable failure rate, not individual airlines. In frontier AI, each developer sets their own thresholds. Compare Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and Google DeepMind’s Frontier Safety Framework side by side. Each is serious work. None uses the same criteria. That’s not necessarily wrong. Reasonable people disagree on acceptable risk levels. But it means external stakeholders, your legal team, your board, your regulators, have no basis for comparison. When every developer uses different criteria, “we evaluated this model and found it acceptable” carries almost no signal.
The aggregate risk problem is the most underappreciated piece of this stage. Even if you correctly evaluate each individual risk a system poses, the paper points out there’s no agreed methodology for combining those evaluations into a judgment about the system as a whole. Does one unacceptable risk make the entire system unacceptable? How do you weigh a low-probability catastrophic risk against a high-probability moderate harm? There’s no standard. The field is improvising.
Stage 5: Risk Mitigation. The Controls You’re Relying On Are Fragile.
The paper organizes mitigations into four levels: data, model, system, and ecosystem. The open problems at each level share a common theme: we don’t know how durable any of these controls actually are under adversarial conditions.
Data-level mitigations, which filter training data to prevent models from learning harmful capabilities, are appealing because they’re upstream. The problem is that the relationship between what you filter out of training data and what capabilities the model ends up with is poorly understood. The research suggests you might successfully filter out complex dangerous capabilities in specialized domains. You’re much less likely to successfully filter for simpler behavioral tendencies like toxicity.
Model-level mitigations, including RLHF, fine-tuning, and machine unlearning, are where most enterprise AI safety investment goes. The paper’s critique here is pointed: existing unlearning techniques suppress harmful capabilities in normal conditions, but adversarial users can reliably surface those capabilities through prompting, fine-tuning, or targeted attacks. The mitigations are real. The durability of those mitigations under sustained adversarial pressure is not demonstrated.
System-level guardrails face a version of the same problem. They work in the deployment context they were designed for. As usage patterns evolve, as users learn to probe edge cases, as the models themselves get updated, the alignment between the guardrail logic and the actual risk landscape degrades. Nobody has a good answer for how fast that degradation happens or how to detect it before it matters.
What This Means for Your Organization Right Now
If you are an enterprise leader buying or building AI governance infrastructure in 2026, you are operating in a field where the foundational questions are still open. The frameworks you’re using were designed for different technology. The evaluations you’re relying on measure proxies. The controls you’re deploying haven’t been tested for durability.
That’s not a reason to stop. AI is moving whether you govern it or not. The organizations that win are the ones who build governance infrastructure that can adapt as the answers to these open questions emerge. The ones who lose are the ones who hardcode their programs to a framework that’s already misaligned with the technology it’s supposed to govern.
Three things you should do differently based on this paper:
1. Stop treating compliance as a point-in-time certification. The paper’s taxonomy of open problems, covering lack of consensus, framework misalignment, and implementation gaps, tells you that this field is actively moving. Your governance program needs to be built on adaptable infrastructure, not static checklists. If your AI governance vendor can’t update their framework faster than the regulatory landscape changes, that’s an exposure, not a feature.
2. Ask harder questions about your evaluations. When a model vendor shows you capability assessment results, the right question isn’t what did the model score. It’s what does this score tell us about real-world risk, and what specifically does it not tell us. Those are different questions. The paper is clear that current evaluations systematically understate deployment-context risk. Build that caveat into how you interpret and act on results.
3. Weight your post-deployment data more heavily than your pre-deployment data. NIST 800-4 documented how broken monitoring infrastructure is today. This paper documents why the pre-deployment governance stack has known gaps that current methods can’t fully close. The practical conclusion from reading both together: the signal from actual deployment, how users interact with the system, what incidents occur, how behavior shifts over time, is often more reliable than the pre-deployment evaluation record. Your governance posture should reflect that.
The Honest Bottom Line
The value of the paper isn’t in what it solves. It’s in what it names. Because you can’t fix what you haven’t acknowledged is broken. NIST 800-4 named the monitoring gap. This paper named the other 27.
The organizations that take this seriously, not as a compliance exercise but as a strategic signal about where the field is heading, are the ones that will build governance infrastructure that actually holds up. The ones that don’t will spend the next three years patching the gaps this paper already mapped.
Focus on active governance, not static governance.
Primary Sources: Go Read These Yourself
The two papers this article is built on:
Open Problems in Frontier AI Risk Management — Ziosi et al., Oxford/MIT/Stanford, 2026. The full paper. Start with the introduction and the open problems boxes at the end of each section.
NIST AI 800-4: Challenges to the Monitoring of Deployed AI Systems — NIST CAISI, March 2026. Read the executive summary and Table 3 on category-specific challenges.
The underlying governance frameworks:
NIST AI Risk Management Framework — The voluntary framework most enterprise AI governance programs are built on. The Generative AI Profile companion (NIST AI 600-1) is worth reading alongside it.
ISO 31000:2018 Risk Management Guidelines — Paywalled, but the overview is free. This is the foundation the Oxford paper’s five-stage structure is built on.
Developer safety frameworks — compare these side by side:
Anthropic Responsible Scaling Policy — Regularly updated. The most publicly detailed of the major developer frameworks.
OpenAI Preparedness Framework — Their capability threshold and risk evaluation approach.
Google DeepMind Frontier Safety Framework — Worth comparing against the Anthropic and OpenAI approaches directly.
Incident and risk data sources:
AI Incident Database — The closest thing the field has to a shared incident record. Useful for understanding what post-deployment failures actually look like.
OECD AI Incidents Monitor — The international complement to the AIID.
MITRE ATLAS Matrix — AI-specific attack tactics catalogued across the system lifecycle. Useful for risk identification work.
The three-stream fragmentation is the sharpest problem you've named. You can't govern a system whose failure modes only become visible post-deployment if your monitoring infrastructure was designed to measure pre-deployment capability. The missing piece isn't more evaluations; it's a feedback loop from production back to the governance layer in real time. That's an infrastructure problem before it's a policy problem. An authorization layer that doesn't know what the deployed system is actually doing has no real control surface.
Been doing testing and prototyping around all sorts of scenarios when you think on the risks, but at a scale and velocity that nothing in existence was designed to handle.