When AI Is Used Against You

AI-generated voices, video calls, and messages can now imitate a real boss, colleague, or family member well enough to fool a careful, experienced person. This module covers the three attacker-facing risks every organization faces today — even one that has never touched an AI tool itself — and teaches the single habit that defeats nearly all of them: never approve an urgent, money- or data-related request based on how it looks or sounds alone. Always confirm it on a second, already-trusted channel first.

In one line

You do not need to adopt any AI tool to be a target of one. Criminals already use AI to clone voices from a few seconds of audio, generate live video of a face that isn't real, and write phishing emails with none of the spelling mistakes or awkward phrasing that used to give scams away. This is the risk category that applies to every organization on day one, regardless of how much or how little AI the firm itself uses — which is exactly why it is the right place to start. The goal of this module is not to make you paranoid or to turn you into a technical detector of fakes. Detection is a losing game — the fakes are already good enough to beat a trained eye and ear. The goal is to give you one simple, reliable habit that makes the quality of the fake irrelevant.

The short version — what to remember

These risks exist for every organization today, even one that has never used an AI tool itself — the attacker's use of AI is what matters, not yours.
Deepfake video and cloned voices can now convincingly imitate a real, known person live, on a call or video meeting — this is no longer rare or futuristic.
AI-written phishing messages can be grammatically perfect and personally researched, so 'spot the typo' is no longer a reliable defense.
The one habit that works regardless of how good the fake is: verify any urgent money- or data-related request on a second channel you already trust, before acting.
Manufactured urgency and pressure to skip verification ('do it now', 'don't tell anyone') is itself a warning sign — agree on a no-exceptions callback rule as a team, in advance.

A real (anonymized) example

In early 2024, an employee at the engineering firm Arup's Hong Kong office joined what appeared to be a routine video conference call with the company's UK-based CFO and several other colleagues, all of whom looked and sounded exactly right. Every person on the call except the employee was, in fact, an AI-generated deepfake — recreated from publicly available video and audio of real Arup staff. Believing the instructions were genuine, the employee authorized 15 transfers totaling roughly HK$200 million (about US$25 million) to accounts controlled by the attackers. The company only realized what had happened after the fact, once the funds were already gone. The lesson organizations have since drawn from this case is not 'we should have spotted the fake' — by the account of those involved, the deepfakes were convincing enough that spotting them live was not realistic. The lesson is that a callback to a separately verified number, or a simple pre-agreed second-channel check before any large transfer, would have stopped the fraud regardless of how good the video and voice were.

Reflect

A few open questions — nothing to trip on

These are to think through, not a test — there's no score and no wrong answer. Jot a line if it helps, then open the note to see what a careful answer usually considers. Nothing is saved or shown to your admin.

Question 1 of 4

Your firm doesn't use any AI tools. Are you still at risk from deepfakes and voice cloning?

See what a careful answer considers

A careful answer usually lands on: Yes — these risks come from attackers using AI against you, not from your own AI use. These are attacker-facing risks. Criminals use AI tools regardless of what your organization does, so any firm with money, data, phones, or email is already a potential target.

Question 2 of 4

You get an urgent call from someone who sounds exactly like your managing director, asking you to send money immediately and to keep it confidential. What should you do?

See what a careful answer considers

A careful answer usually lands on: Hang up and call them back on their usual, already-saved number before doing anything. Voice cloning can now sound convincingly real, and asking questions on the same call doesn't help if the whole call is fake or the attacker has done their research. The only reliable check is contacting the person through a separate channel you already trust and control.

Question 3 of 4

How can you still recognize an AI-written phishing email, since spelling mistakes are no longer a reliable clue?

See what a careful answer considers

A careful answer usually lands on: You generally can't rely on how the message reads anymore — instead, verify unusual or urgent requests through a separate, known channel. Modern AI-written phishing can be flawless and personally researched. The defense isn't better spotting — it's verifying anything unusual or urgent through a channel the attacker doesn't control.

Question 4 of 4

In the real-world case discussed in this module, what ultimately would have stopped the fraud?

See what a careful answer considers

A careful answer usually lands on: A callback or second-channel verification before authorizing the large transfer, regardless of how convincing the call looked. By most accounts the deepfakes involved were realistic enough that catching them live wasn't practical. A pre-agreed rule to verify large or urgent transfers on a separate, trusted channel would have worked regardless of the fake's quality.

Read the full written lesson (~8 min)

You are already a target, whether or not you use AI

This is the most important idea in this module: your exposure to these risks does not depend on whether your organization uses AI. If your business has a phone, an email inbox, a bank account, or a public website, you are already a target. Attackers use AI to impersonate people you know and trust — your boss, a supplier, a client, even a family member — in order to trick you into sending money, sharing a password, or handing over confidential information.

Three capabilities have changed the game in the last few years, and they now cost an attacker almost nothing to use:

Deepfake video: a realistic moving video of a person's face, saying and doing things they never said or did — usable live, in a video call, not just in a pre-recorded clip.
Voice cloning: a synthetic copy of someone's real voice, built from as little as 3–10 seconds of audio pulled from a voicemail greeting, a podcast, a conference talk, or a social media video — good enough to use in a live phone call.
Flawless phishing and impersonation: emails, text messages, and chat messages written by AI that read exactly like a real, fluent, professional message — no typos, no odd grammar, no giveaway phrasing — and can be personalized in seconds using information scraped from your company's own website or social media.

None of these require your firm to have adopted AI. They only require the attacker to have a target — and every organization with money, data, or access is a target.

What deepfakes and voice cloning can actually do today

It helps to be concrete and honest about current capability, without exaggerating it into science fiction.

Voice cloning can now recreate a convincing version of someone's voice — including their accent, tone, and speech patterns — from a very short recorded sample. Attackers gather that sample from things people already post publicly: a voicemail greeting, a recorded webinar, a video posted on LinkedIn, a local news interview. The cloned voice is used in a live phone call, often to impersonate a company owner, a finance director, or a family member, and to create urgency: "I need you to move this payment right now, I'm in a meeting and can't talk long."

Deepfake video takes this further: a real-time video call where the face and voice both appear to belong to a real, known colleague or executive, but are being generated live by software controlled by the attacker. This is no longer rare or exotic — it has already been used in real corporate fraud (see the case study below), including in live video conference calls with multiple "attendees" who were all fake.

The honest, reassuring truth: you are not expected to be able to spot these by eye or ear, and increasingly, you can't reliably. Audio and video quality, lighting glitches, and lip-sync errors used to be tell-tale signs — they are disappearing as the technology improves. This is precisely why the defense in this module does not rely on detection at all.

Flawless phishing: the scam email that has no red flags left

For years, the advice for spotting a phishing email was "look for bad spelling, awkward grammar, or a generic greeting." That advice is now out of date. AI writing tools let attackers produce messages that are grammatically perfect, correctly branded, personally addressed, and written in a tone that matches how your real suppliers, bank, or colleagues actually write.

Attackers also use AI to research their target first: scraping your company website, LinkedIn profiles, and press mentions to build a message that references real names, real projects, or real recent events at your firm — making it feel authentic. A message might reference a real supplier's real invoice number, a real colleague's real job title, or a real recent company announcement, then attach a fake but very convincing invoice, login page, or "urgent document requiring your signature."

This means the old checklist of "look for typos and weird phrasing" no longer protects you. The message can be perfect and still be a scam. The one thing an attacker cannot fake is a channel of communication you already trust and control — which is exactly where the defense in this module is built.

The one defense that works: verify on a second, known channel

This is the core skill of this module, and it is deliberately simple enough to use under pressure, by anyone, regardless of technical knowledge:

Never approve an urgent request involving money, passwords, or sensitive data based only on how it looks or sounds. Always confirm it through a second communication channel that you already know is genuine — one you initiated yourself.

In practice, this means:

If you get a call, video call, email, or text asking you to send money, change payment details, share a password, or send confidential files — stop before acting.
Contact the person back using a phone number, email address, or app you already had saved for them before this request arrived — never a number or link provided in the suspicious message itself.
Ask a question only the real person would know the answer to, or simply say: "I'll call you back on your usual number to confirm" — and then actually do it, even if the person on the call objects, rushes you, or claims to be too busy.
Treat urgency and pressure to skip the check ("I don't have time," "this is confidential, don't tell anyone," "do it now or we lose the deal") as a warning sign in itself — real emergencies rarely require you to skip verification, and attackers manufacture urgency specifically to stop you from checking.

This single habit works regardless of how convincing the fake is, because it does not depend on spotting anything wrong with the message — it depends on a step the attacker cannot intercept: your own, independently-initiated contact with the real person.

Making it a habit, not a one-time lesson

Knowing the rule is not the same as using it in the moment — especially when a request feels urgent and comes from someone who sounds exactly like your boss. Two small structural habits make this easy to apply without having to think hard under pressure:

Agree on a callback rule in advance, as a team. Decide now, calmly, that any request to move money, change bank details, or share credentials — however it arrives — gets a callback on a known number before action, no exceptions, no matter who is asking or how urgent it seems. Having agreed this in advance removes the awkwardness of "checking on the boss" in the moment.
Keep your own list of trusted contact details — saved phone numbers and email addresses for your bank, key suppliers, and colleagues who might ever ask you to move money or data — so you always have a channel to verify with that didn't come from the suspicious message itself.

This is the entire lesson. It costs nothing, requires no technical skill, and works whether the attacker used a cheap script or the most advanced deepfake available.

This lesson is written and reviewed by named humans. Content current as of 21 July 2026. See the Trust Center for our review process and AI-assistance disclosure.