Summary
New research suggests that trust in AI doesn’t just influence decisions, it can quietly degrade our confidence in making them. For cyber security leaders, this raises a critical question: as AI becomes embedded in workflows, are we strengthening human judgement or slowly displacing it?
A study led by Dr Sophie Nightingale at Lancaster University, published in Scientific Reports, explored how people judge whether faces are real or AI generated. Participants received guidance alongside each image, sometimes framed as coming from human experts, sometimes from an AI system, and it was only accurate half the time. The outcome showed something more subtle than people simply following incorrect advice. Those who already felt positively about AI became worse at telling real from synthetic when the guidance came from AI. Demonstrating how their confidence in their own judgement quietly weakened. This is particularly relevant in a cyber security landscape where everyday employees are often the most frequent targets.
There’s nothing new about people seeking input to validate thinking. We’ve always tested ideas with people we trust and the speed, availability and perceived authority of AI changes how that input is used. As Tim Ward, Redflags CEO and Co-founder, puts it, “the biggest shift wasn’t the technology itself, it was how quickly people began outsourcing critical thinking to it.” This means an accumulation of small, repeated moments where individuals defer and don’t question. In cyber security, where attackers exploit urgency and cognitive overload, that erosion of confidence can become a meaningful vulnerability. Tools like Redflags approach this challenge by introducing real-time behavioural nudges into the flow of work, prompting individuals to pause and re-engage their own judgement at the moment it matters.
The longer-term implication is about accuracy and the habits being formed. “The habits people form around questioning its answers, the degree to which they preserve or surrender confidence in their own thinking, may be the real legacy of this moment,” says Ward. For security leaders, that lands as a practical concern. If teams begin to default to AI outputs without scrutiny, the organisation’s human layer becomes less resilient over time. Reinforcing critical thinking at the point of decision, especially for those on the front line of daily threats, helps maintain that resilience. AI will continue to shape workflows and accelerate productivity. However, the organisations that benefit most will be the ones that keep people actively engaged in their own judgement and don’t allow that responsibility to drift.
Check out the Redflags’ AI Hub for more interesting thought leadership pieces and resources.
