Designing an explainability layer for a critical infrastructure tool — so grid operators can understand not just what to change, but why.
Grid operators who are responsible for keeping the electricity grid stable. Our users work in day-ahead congestion management — fixing tomorrow's grid problems today.
Germany has five national preventive redispatch (pRD) processes. These consist of adjusting which power plants and energy assets will generate more or less the day after, so transmission lines are not overloaded when electricity actually flows and the grid is always balanced.
The pRD2 tool provides the operator an optimization result telling them which power plants, wind turbines, and other energy assets need to adjust their output to prevent overloads in the transmission lines.
The tool shows what to change — but never why.
Operators verify and adjust these results under time pressure, with no tool support for the reasoning behind the results. They change the results using their experience and intuition built up over years.
"...manchmal fehlt die Information, warum eine Lösung als optimal ausgewählt wurde."
The operator explicitly talks about the gap: no explanation on why the result was considered optimal.
Respondent 1 — Q19"In einer durchschnittlichen Schicht hat man etwa 20 Minuten Zeit für die Plausibilisierung der Ergebnisse."
The operator talks about the time pressure to check result plausibility: roughly 20 minutes — already less time than the redispatch tools are allocated.
Respondent 3 — Q13We ran a survey of 23 questions with 7 participants, focused on understanding what tools they use, how confident they feel using them, how much they adjust results and why, and where they lose time.
The underlying thread across all questions: when do operators trust a result enough to accept it — and what would need to change for them to trust it faster and more reliably?
We also conducted observational research, sitting with one of the users for nearly 5 hours — though we weren't able to observe the pRD2 process directly due to access constraints.
Operators consistently flagged the absence of explanations. They know what to change, but not why the optimization chose a specific result — leaving them to rely entirely on personal experience.
Operators have roughly 20 minutes per shift to verify results. This window is already shorter than the time the tools themselves are allocated — making every second of cognitive overhead count.
Two respondents scored AI trust at 0 and 1 out of 10. A direct quote: "AI that cannot admit it doesn't know something is dangerous." Every response must state its limits.
Based on user feedback, the solution needed to do two things:
The core design challenge was information design: reasoning logic, confidence scores, and sources needed to be presented in a way that is immediately understandable under time pressure. An LLM-based chat interface lets operators ask follow-up questions without leaving the tool.
Scope constraint: users cannot activate, modify, or submit results through this tool — it is purely an explainability layer.