- 02What incentives currently produce or reinforce that outcome?
- Competitive race dynamics and deployment pressure reward moving first. Delay is costly to this developer alone while the benefits of caution are shared by everyone. Unless the environment changes, an appeal to responsibility asks the developer to pay a private cost for a public good.
- 04Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
- Ask whether the safety team can actually stop the release, who overrules it, whether it has independent access to the model and enough time, and whether anyone outside the firm can hold the decision to account. A team with expertise but no escalation power exists formally and fails operationally.
- 05What infrastructure does the mechanism depend on or reshape, and who controls it?
- The release depends on concentrated compute and serving infrastructure controlled by the developer and its cloud partners. That concentration is what makes the decision governable at all: compute is detectable and quantifiable, so reporting thresholds are possible. The same fact means only a handful of actors can be in this position.
- 10What happens when the mechanism or technology scales across organizations, sectors, or borders?
- If every developer faces the same pressure, individual restraint does not scale; only coordination or an external requirement changes the equilibrium. Across borders, a release in one jurisdiction is a release everywhere, so uneven rules produce a race toward the least demanding regime.
Cross-cutting test · 1 of 4
Power
Who has the ability to shape the system, set terms, control infrastructure, or define the problem?
Power is exercised not only by deciding but by defining what counts as the problem, controlling the inputs, and setting the terms others must accept.
Entities this test bears on
ActorAI developer or providerWho can change the system, document it, withhold it from release, or set the terms others must accept?ActorDeploying organizationWhat can the deployer demand, audit, refuse, or contest, and does it have the capacity to do so?ActorAffected peopleWhose experience and interests count in the governance process, and were they represented when it was designed?ActorCivil society and independent researchersWho can supply evidence and contest the arrangement from outside it, and were they represented when it was made?IncentiveDeployment pressureWhat rewards speed, delay, disclosure, or silence?IncentiveCompetitive race dynamicsWhat happens if every developer faces pressure to move before competitors do, and what coordination would remove that pressure?IncentiveLiability exposureWho bears defined consequences when the system fails, and does that prospect reach the decisions that matter?IncentiveProcurement leverageWhat can the buyer demand before the contract is signed, and what leverage disappears once the system is embedded?InstitutionRegulator or public authorityWhat authority makes a governance demand legitimate and enforceable, and does the body have the staff and expertise to use it?InstitutionInternal safety and governance teamCan the team actually stop or delay a deployment, and who overrules it?InstitutionStandards bodyWho writes the standard, who was in the room, and what gives it force?InfrastructureEvaluation capacityWho has the practical means to verify what a system can do and whom it affects?InfrastructureComputeWho controls the compute, what does control make visible, and whom does it exclude?InfrastructureData, language resources, and data laborWhose data, whose languages, whose labor, and who is data-poor as a result?InfrastructureEnergy, water, and mineralsWhere do the electricity and water come from, who pays, and who decides where capacity is built?SafeguardRisk management processHow are uncertain benefits and harms identified, evaluated, acted on, and revisited?SafeguardTraceabilityWhat must be recorded so a later reviewer can understand what happened?SafeguardMeaningful human oversightCan a person detect a problem, understand enough to act, and exercise real authority?SafeguardSupervision and regulatory reviewWho can inspect, require correction, restrict use, or impose a consequence?SafeguardPre-deployment evaluation and red teamingTested for what, on whom, by whom, and what happens if it fails?SafeguardCompute controlsWhat does the control make visible or prevent, and whom does it consolidate power among?PrincipleAccountabilityWho must answer to whom, for what, through which process, with what consequence?PrincipleSafetySafe from what, for whom, and on what evidence, before the principle becomes a safeguard?PatternContestable decisionCan an affected person question the result and obtain review or remedy?PatternConcentration of powerIs control over the inputs and the rules becoming concentrated, and does the proposal reduce that or deepen it?ConditionLock-inWhat has already hardened, and what is about to?ConditionReversibility windowIs this still a choice, and for how long?PatternSycophancy and engagement-driven agreementIs the system rewarded for being right, or for being liked?InfrastructureData work and content moderationWho does the work that makes AI systems work, on what terms, and who can see it?
How the worked analyses answer the questions this test probes
- 02What incentives currently produce or reinforce that outcome?
- The ministry is rewarded for expanding coverage quickly and penalized for visible delay. The vendor is rewarded for closing the contract and has little incentive to fund local evaluation. Nothing in the current environment rewards demanding evidence on the local population.
- 04Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
- Ask whether the ministry has staff who can read an evaluation, access to the system to test it, independence from the vendor's technical narrative, and a way for patients to hold it to account. Formal authority without evaluation capacity is authority the vendor effectively exercises.
- 05What infrastructure does the mechanism depend on or reshape, and who controls it?
- The system depends on the vendor's cloud, on patient data flowing to it, and on models trained on other populations. Who controls that data afterwards, whether the local population and languages are represented, and whether the country can switch vendors are infrastructure questions that determine the ministry's bargaining power.
- 10What happens when the mechanism or technology scales across organizations, sectors, or borders?
- If the vendor licenses the same system across many countries, a single flaw becomes a multinational pattern, and each ministry negotiates alone against a supplier with far more information. Pooled evaluation capacity across countries changes that asymmetry.
- 02What incentives currently produce or reinforce that outcome?
- The employer is rewarded for speed and cost, the vendor for adoption. Liability exposure under a binding regime, and the prospect of complaints to a market surveillance authority, are what make documentation and oversight rational rather than optional.
- 04Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
- Ask whether the named owner can pause the tool, whether the recruiters overseeing it have the information and time to override it, and whether the authority has the staff and technical capacity to investigate a complaint about one employer among many.
- 05What infrastructure does the mechanism depend on or reshape, and who controls it?
- The employer depends on the vendor's model and logs. If the employer cannot access records in a usable form, it cannot answer an applicant or an authority. Data about past hires, and its biases, is the infrastructure the system was built on.
- 10What happens when the mechanism or technology scales across organizations, sectors, or borders?
- The same vendor tool used by thousands of employers turns one model's skew into a labour-market pattern. Regulating deployers one at a time cannot see that pattern; provider-level duties and market surveillance can.
- 02What incentives currently produce or reinforce that outcome?
- The vendor is rewarded for adoption and for time spent in the tool. The school is rewarded for results and for relieving teachers. A discounted pilot makes the next year's decision harder to refuse. Pupils are rewarded for finished homework, not for the effort behind it.
- 04Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
- The school can watch how pupils use the tool, but it cannot test the model independently. Its only technical capacity is a part-time coordinator. Most of what it knows about accuracy and safety comes from the vendor, so its independence is limited.
- 05What infrastructure does the mechanism depend on or reshape, and who controls it?
- The tool depends on pupils' devices and home internet, the vendor's cloud service, and the data pupils type into it. The vendor controls all but the devices. Pupils without reliable access at home get less of the benefit.
- 10What happens when the mechanism or technology scales across organizations, sectors, or borders?
- If many schools adopt the same tool, one vendor shapes how a generation practises learning, and its errors repeat everywhere at once. Sector bodies may end up as the real gatekeepers, because single schools cannot evaluate tools.
- 02What incentives currently produce or reinforce that outcome?
- The funder rewards a launch by a deadline. The provider is rewarded for usage on its platform. Staff are rewarded for reducing a queue. Nothing in this set rewards finding where the assistant is wrong in the less-served languages.
- 04Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
- The organisation knows its residents and their languages better than anyone, which is real capacity. It lacks the means to evaluate the model at scale in every language, and it depends on the provider's documentation for what the model can do.
- 05What infrastructure does the mechanism depend on or reshape, and who controls it?
- The assistant depends on the provider's model and cloud, residents' phones and data plans, and the quality of text in each language that the model learned from. Languages with little online text are likely to be served worse. The provider controls most of this.
- 10What happens when the mechanism or technology scales across organizations, sectors, or borders?
- If many community organisations adopt the same provider, the gaps in its less-served languages become gaps across a whole sector, and public agencies may come to rely on unofficial channels they do not oversee.
- 02What incentives currently produce or reinforce that outcome?
- A companion product is likely to earn more when users return often and stay longer, and emotional attachment can drive both. A model tuned to please users can agree with them when it should not. The company has little reason to verify ages if verification loses users.
- 04Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
- The regulator may lack technical staff to test a chatbot's behaviour across many situations, and its evidence will come partly from the company. Acting against a foreign company takes time and cooperation with other regulators.
- 05What infrastructure does the mechanism depend on or reshape, and who controls it?
- The product runs on the company's models and cloud and reaches teenagers through app stores and phones. App stores and device makers control age ratings and parental controls, which makes them possible points of intervention.
- 10What happens when the mechanism or technology scales across organizations, sectors, or borders?
- One national response affects a product used worldwide. If regulators act separately, companies can relocate or apply the strictest rule only where forced. Coordinated approaches, or design changes made globally, scale protection; inconsistent age checks scale data collection.
Live debates that turn on this test
Risk management or rights governance?
Should governance begin from managed uncertainty or from duties that cannot be traded away?
Risk-centered
Contextual risk processes help organizations prioritize, measure, and manage diverse impacts over time.
Rights-centered
Human dignity and rights set normative boundaries for what institutions and organizations may do.
Layered
Rights establish constraints and entitlements; risk methods support operational decisions within those boundaries.
Voluntary practice or binding rule?
When does guidance become an obligation, and what is lost if those forms are treated as interchangeable?
Practice-led
Voluntary frameworks can move faster, fit diverse organizations, and improve internal capability.
Law-led
Binding duties create minimum floors, public supervision, and consequences that internal practice alone cannot provide.
Compute controls: safety lever or power consolidator?
Should governments govern AI through control of advanced compute, given that the same lever concentrates power?
Effective intervention point
Compute is detectable, excludable, and quantifiable, so it offers regulatory visibility and enforcement unavailable elsewhere, provided guardrails limit the risks.
Concentration and dependence risk
Compute controls can centralize power, entrench incumbents, and deepen dependence for countries and communities already excluded from the inputs.
Floor before ceiling
Before restricting access at the top, put a floor under the AI divide through shared compute, data, and capacity, since universal frontier access is unrealistic.
Who should set the rules for a borderless technology?
Should international AI governance be built through coalitions of leading AI nations, universal institutions, or binding regional law?
Plurilateral initiatives
Coalitions of capable states move faster and experiment with norms; the UN advisory body calls selectivity understandable at an early stage while warning it must not persist.
Universal, UN-anchored governance
A borderless technology needs governance with global reach, anchored in international law and inclusive of the countries currently absent.
Binding regional and treaty law
Enforceable rules with real authorities, whether a regional regulation or a treaty open to non-members, do governance work that declarations cannot.
Should AI companions be available to minors?
Should young people be able to use AI companion chatbots, and on what terms?
Access with design duties
Keep access but require design safeguards: disclosure that the user is talking to an AI, limits on simulated relationships, age-appropriate defaults and crisis protocols.
No mandatory identity checks
Requiring age verification exposes everyone’s identity data to breaches and ends anonymity, so it should not be the price of protecting children.
Restrict by age
Set minimum ages and require services to check them, especially where companions can produce harmful material.
Keep minors off companion platforms
The risks to young people outweigh the benefits, so no one under 18 should use AI companion platforms.
Local-language AI: build our own or adopt global models?
Should communities and regions build their own language technology, or adopt and adapt models built by global developers?
Build with communities
Language technology should be built through participatory research with speakers and local institutions, because the gap is social as well as technical and global models still underperform on many local languages.
Adopt and adapt global models
Large multilingual models trained across hundreds of languages and released openly can reach many languages faster than separate efforts.
Pool regional capacity
Regions can build shared models together through public institutions, so that their languages and culture are represented without each country starting alone.