A community organisation deploys a multilingual AI assistant
A community organisation must decide whether to launch an AI chat assistant that answers residents' questions about public services in several languages.
Setting. A non-profit that helps residents with housing, benefits and health services in a diverse city district. Residents differ in language, literacy and internet access, and children sometimes ask questions for their families. A funder offers a grant if the assistant launches within six months. The organisation would build on a general-purpose model from a large provider. No real organisation, provider, funder or city is assumed.
The outcome is whether residents who struggle most to reach public services get accurate help, or whether a fluent but unreliable channel replaces the staff they used to talk to. A pattern of confident wrong answers about eligibility or deadlines would fall hardest on people with the fewest alternatives.
Unknown in this scenario
Whether staff contact will remain available, or the assistant is meant to replace it.
What incentives currently produce or reinforce that outcome?
The funder rewards a launch by a deadline. The provider is rewarded for usage on its platform. Staff are rewarded for reducing a queue. Nothing in this set rewards finding where the assistant is wrong in the less-served languages.
Unknown in this scenario
Whether the grant measures resident outcomes or only launch and usage.
The organisation's director and board can launch or stop the assistant. The model provider controls the model, its updates and its terms. Public agencies own the rules the assistant describes but have no say over the assistant itself.
Unknown in this scenario
Whether any public agency must approve, or even knows about, a third-party channel describing its services.
Who a resident can complain to about a wrong answer.
Does that institution have the capacity, independence, expertise, legitimacy, and accountability to act?
The organisation knows its residents and their languages better than anyone, which is real capacity. It lacks the means to evaluate the model at scale in every language, and it depends on the provider's documentation for what the model can do.
Unknown in this scenario
How well the model performs in each of the languages residents use.
Whether bilingual staff have time to review answers.
Large language models tested on a benchmark for 17 African languages performed significantly worse than on high-resource languages such as English.
Our reading: One benchmark of 2024 models on African languages; gaps for other languages and later models may differ, but the direction matches other evaluations.
Inspect evidence · 1 passage
Passage summaries are paraphrases, not quotations. Open the source to read the full passage.
On a human-translated benchmark for 17 African languages, 16 large language models showed a significant performance gap between high-resource languages and African languages, and open models lagged proprietary ones.
Scope note: Preprint version cited; a peer-reviewed venue was not verified. Some co-authors work at industry labs.
What infrastructure does the mechanism depend on or reshape, and who controls it?
The assistant depends on the provider's model and cloud, residents' phones and data plans, and the quality of text in each language that the model learned from. Languages with little online text are likely to be served worse. The provider controls most of this.
Unknown in this scenario
Whether residents' questions, which may reveal immigration, health or money matters, are kept by the provider.
Large language models tested on a benchmark for 17 African languages performed significantly worse than on high-resource languages such as English.
Our reading: One benchmark of 2024 models on African languages; gaps for other languages and later models may differ, but the direction matches other evaluations.
Inspect evidence · 1 passage
Passage summaries are paraphrases, not quotations. Open the source to read the full passage.
On a human-translated benchmark for 17 African languages, 16 large language models showed a significant performance gap between high-resource languages and African languages, and open models lagged proprietary ones.
Scope note: Preprint version cited; a peer-reviewed venue was not verified. Some co-authors work at industry labs.
What safeguards translate the objective into practice?
Possible safeguards include a clear statement that this is an AI assistant, links to the official source for every answer about eligibility or deadlines, and an easy way to reach a person. Others are logging corrections and a test set reviewed by speakers of each language before launch.
Where in the AI lifecycle does the intervention operate?
The decision is at procurement and deployment, but the riskiest stage is monitoring: rules about services change, and the model's answers must change with them. Incident response matters when a wrong answer has already caused a missed deadline.
Unknown in this scenario
Who notices when a public service changes its rules.
Who decided, on what authority, and who was not represented?
The organisation and funder decide. Residents who will use the assistant, including speakers of the least-served languages and young people who ask on behalf of their families, are not part of the decision unless the organisation brings them in.
Unknown in this scenario
Whether residents tested the assistant in their own languages before launch.
Who bears the costs and risks, and who captures the benefits?
Residents in well-served languages gain the most; residents in poorly served languages, with low literacy or without data plans carry more risk of wrong answers and less benefit. Children answering for parents may carry adult decisions. The provider gains usage; the funder gains a visible launch.
Unknown in this scenario
How answer quality differs by language in practice.
What happens when the mechanism or technology scales across organizations, sectors, or borders?
If many community organisations adopt the same provider, the gaps in its less-served languages become gaps across a whole sector, and public agencies may come to rely on unofficial channels they do not oversee.
Unknown in this scenario
Whether public agencies would coordinate with organisations running such assistants.
What becomes difficult to reverse if governance arrives too late, and how will the governance mechanism itself be revised if it turns out to be wrong?
If the assistant replaces staff time, rebuilding human help later is hard. Keeping the choice reversible means keeping a person-to-person route open, setting review points tied to error reports by language, and agreeing with the funder that pausing is acceptable.
Unknown in this scenario
Whether the funder would accept a pause or a narrower launch.
The technical question is whether the assistant passes a test set. The governance questions show an organisation with deep local knowledge but little power over the model, pushed by a deadline. It serves residents whose languages the model may handle worst, and describes public services whose owners are not in the room. The safeguards that matter most keep a human route open.