Towards Trustworthy LLMs

The Data-Resistant Mind-The Psychology Every Data Scientist Needs to Make Their Work Matter by Sebastian Wernicke, Oxera Consult

• Sebastian Wernicke
• Oxera Consulting LLP

NoteNotes

• Main claim
  • Data alone rarely changes minds; in many organizational settings, it can even harden existing decisions.
  • Data scientists should treat their role not only as analysis, modeling, and validation, but as participation in decision-making.
• Opening example
  • The speaker describes a logistics project in Southeast Asia where route optimization could save up to 20% in fuel.
  • Even though the result was valuable and technically sound, stakeholders returned to their old behavior after the presentation.
  • This illustrates the central frustration: excellent analysis often fails to create organizational change.
• Why data does not change minds by itself
  • Decisions are often already forming before the data science team presents its results.
  • The speaker uses a neuroscience example: decision signals accumulate over time, and late contradictory information has less influence.
  • In organizations, stakeholders may already have informal commitments, expectations, fears, and incentives before the analysis arrives.
  • By the time the data is presented, the decision may already be “in motion.”
• Mismatch between analysis and decision needs
  • Data scientists are comfortable with uncertainty, ambiguity, multiple interpretations, and probability distributions.
  • Decision-makers often expect clear answers: yes/no, proceed/stop, invest/do not invest.
  • A statement like “65% chance of success” may sound informative to a data scientist but unusable to a decision-maker.
  • The speaker’s metaphor: decision-makers want a flashlight, but data scientists often bring an MRI.
• Data scientists’ own bias
  • Data scientists are biased toward data and machine-learning solutions because that is their craft and identity.
  • The speaker gives an example of a metals manufacturer that wanted an algorithm to infer which production batch a part came from.
  • A simpler physical marking system may have been better than a complex predictive model.
  • The lesson: the diagnostic moment is often at project inception, not at the final presentation.
• Reframing the data scientist’s job
  • The job is not merely to provide analysis.
  • The job is to participate in the architecture of decisions.
  • Technical architecture includes ingestion, transformation, modeling, validation, and deployment.
  • Decision architecture asks how decisions form in human organizations and how analysis can enter that process effectively.
• Practical recommendation 1: start earlier
  • Data scientists should “push left” and get involved before the problem is fully framed.
  • They should interrogate the problem, the stakeholders, the assumptions, and the desired change.
  • A key question is: “What concrete change has to happen for this project to count as resolved?”
  • This can prevent technically successful but organizationally useless projects.
• Practical recommendation 2: map the real decision room
  • The “room” includes all stakeholders who influence or block the decision, not only those in the kickoff meeting.
  • Data scientists should ask:
    • Who is involved?
    • What are they measured on?
    • What are they afraid of?
    • Who has implementation capacity?
  • Stated concerns about “methodology” may actually reflect incentives, workload, distrust, risk, or fear of losing control.
• Practical recommendation 3: translate findings into decision structure
  • Do not merely report probabilities or charts.
  • Translate findings into conditions, actions, risks, and monitoring signals.
  • Instead of saying “there is a 65% chance this works,” say:
    • it works in about two-thirds of market conditions;
    • it fails under a specific condition;
    • here is the indicator to monitor.
  • This preserves the statistical truth while making it actionable.
• Q&A: best question to predict project success
  • The speaker says the best question is: “What do you want to change at the end of this?”
  • Good answers reveal intended action, stakeholders, and implementation path.
  • Bad answers include “we just want results” or “we need to prove another department wrong.”
• Q&A: career progression
  • Junior data scientists are often judged on technical correctness: cleaning, modeling, statistics.
  • Senior data scientists are judged more by business impact.
  • The speaker’s shift came from frustration: good analysis was not producing enough real-world effect.
• Q&A: multiple departments
  • When departments have conflicting goals, the data scientist must identify which stakeholder they are effectively serving.
  • Sometimes that is the person funding the work, but not always.
  • The important thing is to make a decision transparently rather than drift between incompatible stakeholder agendas.
• Q&A: making results survive translation
  • Put the analysis into the language of the business.
  • Translate numbers into decision options, effects on business metrics, career incentives, customer behavior, churn, demand, or operational risk.
  • The point is not to dilute the analysis, but to make it usable in the decision-maker’s frame.
• Q&A: how to practice
  • The speaker recommends mentorship from experienced decision-makers.
  • Ask them how a finding will land, how to frame it, and what a stakeholder is likely to hear.
  • Practicing with people who actually make decisions is presented as one of the best ways to develop this skill.
• Overall takeaway
  • Data science impact depends on joining the decision process early, understanding stakeholder incentives, and communicating results in a form that can guide action.
  • These are often dismissed as “soft skills,” but the speaker argues they are part of the real job of data science.

Reflection