AI now sits on both sides of threat intelligence. Adversaries are using it in live operations, and analysts are using it every day, mostly without rules. The principles of intelligence still hold. What most teams lack is the tradecraft to apply them when a machine is collecting, translating and drafting. Treat the model as a source rather than an analyst, gate its output by the cost of being wrong, and control what data goes into which model. Teams that do will produce faster intelligence. Teams that don't will produce faster mistakes.
The situation
Two things have changed at once, and most threat intelligence functions have only planned for one of them.
On the adversary side, AI misuse has moved from speculation to documented operations. In July 2025 CERT-UA reported LameHug, malware attributed with moderate confidence to APT28, which queried an open-source language model to generate commands on infected machines. In November 2025 Google described further APT28 malware doing the same in live operations. That same month Anthropic reported a campaign it attributed with high confidence to a Chinese state-sponsored group, in which AI carried out an estimated 80 to 90% of the tactical work against around 30 targets. Fraud got there earlier: Arup lost about US$25 million in 2024 after a finance employee joined a video call on which the CFO and colleagues were deepfakes.
On the analyst side, the change is quieter. CTI teams already use large language models to triage feeds, translate forum posts, extract indicators and write first drafts. In our experience, few have written down what those models may touch, who checks their output, or how a judgement that began as a model's guess is kept from reaching a client as fact.
Most CTI training and doctrine predates both shifts. Where AI appears at all, it is treated as a technology topic. It is a tradecraft problem.
What is actually happening
The uplift is real, but uneven and often overstated
AI rarely changes an adversary's intent. It changes capability and tempo. Most observed use still sits at the productivity end: better lures, translation, research, faster scripting. Malware that calls a model at runtime and agents that run parts of an intrusion now have documented examples, but they are few, and the best reports are candid about the limits. Anthropic noted that its model often overstated findings and fabricated results, forcing the operators to check everything.
The hype tax is real too. PROMPTFLUX, a dropper that asks Gemini to rewrite its own code, was assessed by Google as experimental and not attack-ready. PromptLock, widely reported in 2025 as the first AI-powered ransomware, turned out to be an academic proof of concept. Before any AI claim enters an assessment, ask whether the AI use was observed or inferred, operational or experimental, and whether the actor could already do this without it. There is also a collection bias to account for: AI providers can see misuse of their own platforms, but not of open-weight models run elsewhere. LameHug used exactly that kind of model.
A model is a source, not an analyst
The most common error we see is treating model output as a colleague's work to be reviewed, when it should be graded as a source. A model generates plausible text. It cannot tell you how reliable it is, and two models agreeing is not corroboration, because they share much of the same training data.
The discipline already exists. Trace every claim to a primary source and grade that source. Anything you can't trace is, in Admiralty terms, F6: reliability and credibility cannot be judged. It is a lead, not intelligence. The legal profession has learned this the hard way. In June 2025 the High Court considered cases in which 23 cited authorities did not exist, and warned that professionals remain responsible for checking AI-assisted work whether they use the tools themselves or supervise people who do. Swap "case law" for "threat reports" and the warning applies to every CTI manager.
The quiet failures: laundering and homogenisation
Laundering is when a model's inference passes through enough hands to come out looking like a fact. A summary that drops "low confidence" is saved to the threat intelligence platform, quoted in a weekly brief and pushed to a blocklist. By the end, nobody knows where it started. The defence is provenance: tag AI-generated content wherever it is stored and keep the original source attached to every extracted fact.
Homogenisation is a sector-level risk. When many teams ask similar models similar questions about the same vendor reports, their assessments converge, and so do their blind spots. Diversity of analysis is a defence, and AI erodes it unless someone manages it deliberately.
The query is intelligence
A prompt such as "summarise threats to [client] focusing on their payment platform" reveals a client, a requirement and a focus, and it now sits in someone else's logs. Sending TLP:RED material to a third-party model shares it beyond its named recipients, whatever the provider's contract says. In threat-led testing under TIBER-EU or DORA, where only a small control team knows the test is happening, putting a target's staff list into an unapproved tool can compromise the engagement. CTI work is also more likely than most to trip a provider's abuse monitoring, because malware and exploit code are exactly what it is designed to flag.
- It is almost certain that AI-enabled social engineering, including deepfake impersonation of executives, is being used against UK and EU financial services firms now.High confidence
- It is likely that state-linked actors will make greater use of AI to speed up live intrusion operations over the next 12 months. The evidence comes from a small number of provider reports, each of which sees only its own platform.Moderate confidence
- It is a realistic possibility that a significant intelligence failure in a UK or EU regulated firm, such as a published judgement built on fabricated sources or a leak from a threat-led test, will be traced to unmanaged AI use within the next 12 months.Low confidence
What this means in practice
- Gate by consequence, not by habit. Light-touch checks for internal summaries; analyst verification for anything entering a product or a platform; manager sign-off for anything that names an actor, triggers action or goes to a client or regulator. The head of CTI owns the gates.
- Apply structured techniques to AI drafts. Analysts set out their hypotheses before they see the model's draft. Run a key assumptions check on every AI-assisted assessment. AI is at its most useful as a devil's advocate: ask it for the strongest case against your judgement, then decide for yourself whether it holds.
- Treat AI providers as third parties. Under DORA, an AI service supporting a critical or important function brings the full third-party requirements: the register of information, contract terms, exit plans and concentration risk. Many AI tools run on the same handful of cloud providers, 19 of which the European Supervisory Authorities designated as critical in November 2025.
- Declare AI use in threat-led testing. Threat intelligence and red team providers should name their AI tools, the data those tools will touch and the controls around them in the engagement plan, and get the control team's agreement before any processing.
- Brief the board on both sides. How AI changes the threat to the firm, where the firm itself uses AI and with what risk, what is being done, and what decision is needed. The UK's AI Cyber Security Code of Practice and the EU AI Act's deepfake disclosure duty, in force from 2 August 2026, give a credible external frame.
How ThreatInsights works with AI
We use AI where it is strong and keep it away from where it is dangerous. Our working rules are short: AI drafts, analysts decide. Every claim in a product is traced to a graded primary source. Content generated by AI is labelled wherever it is stored. No client or engagement data goes into a third-party model without the client's agreement, and in threat-led tests, without the control team's written agreement. Every key judgement has a named analyst who can defend it without the model in the room.
None of this is exotic. It is the discipline intelligence professionals already know, applied to a new kind of source. But it has to be taught, because most analysts picked up AI tools before anyone gave them rules.
Building it into CTI-CRAFT
That is why we are adding an AI and Intelligence track to CTI-CRAFT, our advanced practitioner programme. It is eight new modules, written for analysts and managers who sign off intelligence, not for data scientists. Each module includes worked case studies, a practical exercise with a model answer, and a knowledge check. The track closes with a marked capstone scenario.
- AI-enabled threat actor capabilityAssessing how AI changes intent, capability and opportunity, and separating demonstrated uplift from hype.
- AI systems as targetsPrompt injection, poisoning, model theft and agent abuse, mapped with MITRE ATLAS, the OWASP Top 10 for LLM Applications and ATT&CK.
- AI in the intelligence cycleWhere AI helps and fails at each stage, and how to design review gates in proportion to the cost of being wrong.
- Model reliability and source gradingGrading model output with Admiralty and 3x5x2, catching hallucination and fabricated citations, and the confidence rule.
- Synthetic content and provenanceFake personas, padded leaks and deepfakes in collection, and what C2PA can and cannot prove.
- AI and analytic biasAutomation bias, anchoring and homogenisation, and ACH and key assumptions checks for AI-assisted work.
- AI governance, handling and OPSECMatching data to models, TLP, regulated testing and query OPSEC, with a ready-to-adapt AI usage policy.
- AI law, regulation and assuranceThe EU AI Act, UK guidance and DORA third-party risk, and how to brief a board or regulator.
We are opening a founding cohort for the track. If you lead a CTI team, or are working towards a senior intelligence role, register your interest. If you would rather have the material applied to your own team's AI use, we run it as a facilitated workshop too.
Sources and confidence
- NCSC: Impact of AI on cyber threat from now to 2027 (May 2025). Authoritative UK government assessment; underpins key judgements 1 and 2.
- Anthropic: Disrupting the first reported AI-orchestrated cyber espionage campaign (November 2025). Primary source, but single-provider visibility and attribution evidence not independently published. Reliable for what it describes; treat scale estimates with care.
- The Register on Google Threat Intelligence Group's AI Threat Tracker (November 2025). Secondary reporting of a primary vendor report: PROMPTFLUX, PROMPTSTEAL, and PromptLock's academic origin.
- Security Affairs on CERT-UA's LameHug report (July 2025). Secondary; CERT-UA's attribution is moderate confidence.
- CNN, via ABC17, on the Arup deepfake fraud (May 2024). Confirmed by the victim company.
- BIICL on Ayinde v Haringey and Al-Haroun v Qatar National Bank (2025). Summary of a published judgment.
- Morgan Lewis on the ESAs' designation of critical ICT third-party providers (November 2025); Baker McKenzie on EU AI Act transparency duties; DSIT: Code of Practice for the Cyber Security of AI (January 2025). Regulatory facts; check current status before relying on dates.
- Observations on how CTI teams use AI, and key judgement 3, rest on ThreatInsights engagement and practitioner experience, not published data.