Based on what we see across council environments, here are the patterns that explain why AI doesn’t behave the way you expect. And what to do about it.
TL;DR
- AI surfaces what’s already in SharePoint - good and bad
- Most councils have:
- Outdated and duplicated content
- Permissions that no longer reflect reality
- Weak metadata and structure
- These issues lead to:
- Incorrect or low-confidence AI answers
- Oversharing risks
- Low adoption and wasted licences
- You don’t need to fix everything. You need a clear plan
- WebVine helps councils assess, prioritise and clean up what matters most
AI Isn’t Breaking Your SharePoint. It’s Revealing It
Across the councils we work with, one pattern shows up every time we assess SharePoint for AI:
AI is doing exactly what it’s supposed to do.
It’s pulling from content people already have access to, summarising it, and presenting it back clearly.
The problem is what it finds.
What used to be hidden in folders or buried in search results is now front and centre. In a single answer.
That’s why many councils tell us:
- “It sounds confident, but it’s not always right”
- “It’s pulling the wrong version”
- “We didn’t realise people had access to that”
This was never an AI problem. It’s an information problem.
What We Consistently See Across Councils
Every environment is different, but the patterns are remarkably consistent. In most SharePoint environments we assess, the same issues show up over again, just in slightly different shapes.
This isn’t based on theory. It’s based on what we see across council environments every week.
1. Duplicate and Outdated Content Everywhere
We often see:
- Multiple versions
- Drafts sitting alongside final documents
- Content that hasn’t been touched in years
AI doesn’t know which one is “right.” It just sees what’s available.
So, it summarises outdated or conflicting information. And presents it as if it’s current.
This is one of the most common causes of low trust in AI outputs.
2. Permissions That Made Sense 5 Years Ago (But Not Today)
This is where things get uncomfortable.
Permissions are often:
- Too broad (“Everyone except external users”)
- Left over from restructures
- Still including people who’ve moved on
AI respects those permissions.
Which means if someone can technically access a document, AI can surface it, even if that access no longer makes sense.
For councils, its more than inconvenient. It’s a governance, compliance and risk issue.
3. Structure That Grew Organically (Not Intentionally)
Most council SharePoint environments weren’t designed all at once.
They grew:
- Department by department
- Project by project
- With different naming conventions and approaches
The result is often:
- Multiple places to store similar content
- Deep or inconsistent site structures
- Confusion about where “the source of truth” lives
For people, this is frustrating.
For AI, it’s information it needs to make a definitive decision on.
4. Metadata That Doesn’t Do Enough (or Doesn’t Exist at All)
Metadata is one of the biggest gaps we see.
Either:
- It’s not being used consistently
- It’s too complex to maintain
- Or it was never properly defined
Without it, AI has less context.
That leads to:
- Vague or generic responses
- Difficulty prioritising the “right” document
- Lower confidence in outputs
Without it, AI has less context. (You can read more about why this matters in our guide to metadata and SharePoint search)
5. Examples of What This Looks Like in Practice
We’ve seen this play out in very practical ways across councils:
- A planning officer asks a question and gets a response based on a superseded LEP document from 2021. Not the current approved version
- A staff member sees HR-related content surfaced in an AI response because their access was never removed after a role change
- AI pulls from three different policy documents across separate sites and blends them into a single, confident but incorrect answer
- A compliance response changes depending on which version of a document AI happens to prioritise that day
None of these are dramatic failures. But they’re exactly the kind of quiet inconsistencies that erode trust over time.
What Most Councils Miss
The biggest misconception we see is this:
“We need to fix everything before AI will work.”
You don’t.
Trying to clean up your entire SharePoint environment at once usually leads to:
- Analysis paralysis
- Slow progress
- Frustration from teams
The better approach is:
- Identify high-risk or high-impact areas
- Focus on the content AI is using
- Fix what will make the biggest difference first
This is how you move from:
“We’re not sure we trust this” to “This is actually useful”
How We Can Help (And Where We Start)
When we work with councils, we start by making the problem visible.
Through our Discovery and AI Readiness work, we:
- Start by looking at the quality of your information environment across your SharePoint and Microsoft 365
- Identify where inaccurate or low-confidence answers are coming from (not just that they exist)
- Surface oversharing and permissions risks that weren’t obvious before AI
- Pinpoint high-impact content areas, where improving quality will make the biggest difference quickly
From there, we prioritise what to fix first.
Not everything. Just what matters.
That might mean:
- Focusing on one high-risk content area
- Cleaning up duplicated or outdated documents
- Tightening access in areas where AI is exposing risk
- Improving structure or metadata where it directly impacts AI outputs
The result is a clear, practical roadmap.
Not a long list of theoretical recommendations, but a plan your team can act on and see results from quickly.
Why This Matters More Now
Before AI, many of these issues stayed hidden.
People:
- Asked colleagues
- Searched manually
- Avoided content they didn’t trust
AI changes that. It:
- Surfaces content faster
- Brings multiple sources together
- Removes the “friction” that used to hide problems
That’s why organisations across industries are seeing the same thing:
AI amplifies whatever is already there. Good or bad.
This is the part most organisations underestimate. AI doesn’t just need good technology. It needs well-managed information underneath it. And that’s where most of the work (and value) really sits.
About the Author
Chloe Dervin
Managing Director, WebVine
Chloe leads WebVine’s work with councils and organisations across Australia, helping teams make Microsoft 365, and now AI, genuinely useful in day-to-day work. With deep experience in SharePoint, governance and digital workplaces, Chloe focuses on practical, sustainable improvements that balance usability, compliance and long-term value.
Frequently Asked Questions
Why is AI giving incorrect answers from SharePoint?
Because it’s grounding responses on duplicated, outdated or poorly structured content. AI is summarising what exists , not validating accuracy.
Can AI expose sensitive information?
AI doesn’t grant new access, but it can surface content users already have permission to see. Even if that permission was never intentionally maintained.
Do we need to clean up everything before using AI?
Not necessarily. Focus on high-impact areas first, especially where accuracy, risk or adoption is being affected.
What’s the quickest way to improve AI accuracy?
Start with:
- Cleaning up duplicated/outdated content
- Fixing permissions in key areas
- Improving structure and metadata
What does a WebVine AI assessment involve?
A focused review of your document management environment across M365 (including SharePoint, Teams and Onedrive) to show:
- What AI can see
- Where it can go wrong
- What to fix first
Sources
- Microsoft Copilot governance and how it uses existing permissions:
Microsoft Inside Track – Copilot governance [microsoft.com] - SharePoint data governance and AI readiness:
Horizons – AI readiness and data governance - Why governance drives AI value:
SoftwareOne – SharePoint governance for Copilot [softwareone.com]