ChatGPT got it wrong — how to catch bad AI advice before it costs you

If you've used ChatGPT or Claude for anything related to Irish farming, you've probably been impressed by the output — and then caught something that was completely wrong.

That's not a bug. It's how these tools work. They generate plausible-sounding text based on patterns in their training data. When the answer exists clearly in that data, they're often right. When it doesn't — or when the information has changed since they were trained — they make things up. Confidently.

This isn't a reason to stop using AI. It's a reason to learn how to check it.

Real examples of AI getting Irish farming facts wrong

These are things AI tools have actually told Irish farmers in our testing:

Wrong scheme dates

ChatGPT stated that BISS applications open on 1 February and close on 15 May. The actual dates vary each year, and DAFM announces them via circular — the AI just guessed based on historical patterns.

Invented regulations

When asked about nitrates derogation limits, Claude once cited a "2025 NAP6 amendment" that doesn't exist. Ireland is still operating under NAP5 (as of early 2026). The AI created a plausible-sounding regulation from nothing.

Outdated stocking rates

ChatGPT quoted the organic nitrogen limit for derogation farms as 250 kg N/ha — which was correct historically but doesn't account for the conditional reductions that applied in recent years. The actual figure depends on your water quality zone.

Fictional organisations

Asked about farm safety training, ChatGPT referenced the "Irish Agricultural Safety Board" — which doesn't exist. It likely merged elements of the HSA (Health and Safety Authority) and Teagasc.

Wrong payment amounts

Claude estimated ACRES General payments at "up to €10,000 per year" for a typical farm. The actual payment depends entirely on your chosen actions and land area — there is no standard figure.

Why it happens

Three reasons, all related:

1. Training data cutoff — AI models are trained on data up to a certain date. They don't know about scheme changes, budget announcements, or regulation updates that happened after that date.

2. No access to official sources — ChatGPT can't check gov.ie in real time (unless you use the browsing feature). It's working from memory, not from live data.

3. Hallucination — when the AI doesn't have the answer, it generates one that sounds right. It doesn't say "I don't know." It says something plausible — and often wrong.

The 3-source rule

Before acting on any factual claim from AI — especially anything involving money, regulations, or deadlines — verify it with this simple method:

1. Check gov.ie — search for the scheme or regulation by name. If it exists, the official details will be there.
2. Check citizensinformation.ie — the plain-English version. Often easier to understand than the official circular.
3. Check with your advisor — your Teagasc advisor or accountant can confirm in 30 seconds whether something is current and correct.

If the AI's claim doesn't appear in any of these three places, treat it as wrong until proven otherwise.

Prompts that reduce errors

You can't eliminate hallucination, but you can reduce it:

"Are you sure this applies in the Republic of Ireland, not Northern Ireland or the UK?"

AI frequently mixes up Irish and UK regulations. This prompt forces it to reconsider.

"What date is your training data current to? Could this information be outdated?"

This makes the AI flag its own uncertainty — though it won't always do so accurately.

"Cite the specific official source for this information."

If it can't name a real document, regulation number, or website, the information is likely generated rather than retrieved.

"I'm going to verify this on gov.ie. Is there anything in your answer you're less confident about?"

This sometimes surfaces caveats the AI would otherwise omit.

When AI is reliable vs when it's risky

Generally reliable:

• Drafting letters, emails, and documents (the structure and language)
• Explaining concepts in plain English (what cross-compliance means, how EBI works)
• Brainstorming ideas (prompts for business plans, farm diversification options)
• Summarising long documents you've uploaded

High risk of error:

• Specific payment amounts and rates
• Scheme eligibility criteria
• Application deadlines and opening dates
• Legal requirements and regulations
• Veterinary advice and dosing rates
• Tax calculations

The bottom line

AI is a brilliant drafting tool and a dangerous reference tool. Use it to write. Use it to brainstorm. Use it to prepare.

But when it tells you something specific about a scheme, a regulation, or a payment — check it yourself. Every time. The 3-source rule takes 2 minutes. Getting it wrong can cost you thousands.