Every security operations centre eventually notices the same uncomfortable gap. The alert queue is full, the tooling is working as designed, and yet the assumption underneath it all — that an attacker will trip something — is unproven. Threat hunting exists to test that assumption deliberately rather than waiting to find out during an incident.

The discipline is often described badly. Hunting is not browsing dashboards until something looks odd, and it is not triaging the alerts that already fired. It is the structured pursuit of a specific idea about how an adversary could be operating in your environment without generating an alert.

Why hunting exists at all

Detection engineering builds rules for known behaviour. Those rules are necessarily written against what defenders already understand, which means they encode yesterday’s tradecraft. The space of attacker behaviour that is possible is always larger than the space that is alerted on, and the difference between the two is where intrusions live quietly.

Hunting attacks that difference from the other end. Instead of asking what fired, it asks a harder question: if an adversary were doing X right now, would I be able to see it, and can I prove they are not? Both possible answers are valuable. Finding evil justifies the exercise immediately. Finding nothing, if the hunt was properly scoped, converts an assumption into evidence and usually exposes a visibility gap worth fixing.

Behaviour beats indicators

The single most useful mental model here remains David Bianco’s Pyramid of Pain, which ranks indicator types by how much it costs an adversary when you detect them. Hash values sit at the bottom: trivially changed, so blocking one costs an attacker a recompile. IP addresses and domain names are only slightly better, because infrastructure is cheap and rotated constantly.

At the top sit tactics, techniques and procedures. Detecting the fact that an adversary dumps credentials from process memory, or establishes persistence through a scheduled task, forces them to change how they operate rather than what they use. That is expensive, and it is why hunts should be framed around behaviour.

A hunt for a specific file hash is not really a hunt; it is a search. A hunt for the technique that hash was one implementation of will still work when the binary changes.

Writing a hypothesis that can actually be tested

A good hypothesis is specific enough to be falsified with data you can obtain. Vague ambitions like “look for lateral movement” produce sprawling, inconclusive work. Sharpen it until it names a behaviour, a data source and a population.

Three reliable sources of hypotheses:

Compare a weak and a strong framing of the same idea. Weak: “hunt for malicious PowerShell”. Strong: “PowerShell launched by an Office application, on workstations, with an encoded command argument, in the last thirty days”. The second names the parent process, the population, the distinguishing artefact and the window. It can be answered, and the answer means something.

The hunt loop

A repeatable hunt moves through four stages, and skipping the last is the most common way hunting fails to compound.

  1. Hypothesise. State the behaviour, the data source that would reveal it, and what a true positive would look like.
  2. Collect. Confirm the telemetry actually exists at the required fidelity. Discovering mid-hunt that command-line arguments are not logged is itself a finding.
  3. Analyse. Reduce the population until what remains is small enough to examine individually. Aggregation, stack counting and outlier analysis do most of the work.
  4. Act. Convert the result into something durable — a detection rule, a logging change, a documented gap, or a tuned baseline.

Stack counting deserves particular mention because it is the workhorse technique of practical hunting. Take a field, count occurrences across the estate, and sort ascending. Malicious activity is usually rare by definition. A parent-child process pair seen on one host out of nine thousand is worth a look; the same pair on every host is a software deployment.

A worked example

Take the hypothesis that an adversary has achieved execution through a malicious document. The behaviour that follows is well understood: an Office process spawns a scripting or shell interpreter.

The data source is process creation telemetry, which on Windows means Security event 4688 with command-line auditing enabled, or Sysmon event ID 1. Expressed as pseudo-query logic:

process_create
WHERE parent_image IN (winword.exe, excel.exe, powerpnt.exe, outlook.exe)
  AND child_image IN (powershell.exe, pwsh.exe, cmd.exe, wscript.exe,
                      cscript.exe, mshta.exe, rundll32.exe, regsvr32.exe)
GROUP BY parent_image, child_image, command_line
SORT BY count ASC

Now analyse rather than alert. Some results will be legitimate: a finance macro that has run monthly for six years, a vendor add-in that shells out on install. Those form your baseline and should be documented, not deleted. What remains — a single host where Excel spawned powershell.exe with a base64 argument at 02:14 — is the hunt result.

Decode any encoded command before drawing conclusions, and do it safely. Encoded PowerShell arguments are ordinarily Base64 of UTF-16LE text, which decodes to readable script without executing anything.

// try it live

Decode a suspicious encoded command safely in your browser, with nothing sent to a server.

Base64 Decoder

Turning a hunt into a detection

A hunt that finds nothing and produces nothing was still a cost. The output of every hunt should be one of four things: a confirmed incident handed to response; a new detection rule; a logging or visibility change; or a documented, dated statement that the behaviour was searched for and not found.

The third outcome is chronically undervalued. Hunts routinely reveal that a data source is missing fields, retained for too short a window, or not collected from an entire class of asset. Fixing that improves every future hunt and every existing detection simultaneously.

When a hunt does become a rule, expect to trade precision for recall. Hunting tolerates a wide net because a human is reviewing the output; a production alert cannot. The tuning work of turning a hunt query into a rule — excluding the documented baseline, narrowing to the highest-signal conditions — is the actual engineering, and it is where a mature hunting program compounds into better detection coverage over time.

Measuring the programme honestly

Resist counting hunts run or hours spent. Neither says anything about security outcomes, and both are easy to inflate. More meaningful measures include technique coverage, expressed as which parts of ATT&CK relevant to your environment have been hunted and when; visibility gaps identified and closed; and detections shipped as a direct result of hunting.

Time is a legitimate metric too, but the useful one is how long it takes to answer a question, not how long a hunt lasts. If confirming whether a technique occurred anywhere in the estate takes three days of data wrangling, that is a tooling problem masquerading as a hunting problem.

Failure modes worth naming

Starting without a dedicated team

Hunting does not require headcount or a new platform. It requires process-creation telemetry with command lines, authentication logs, and a few uninterrupted hours. Pick one technique your organisation would plausibly face, write a hypothesis narrow enough to answer, and see whether your data can answer it. Whichever way it resolves, you will know something concrete that you only assumed the day before.

That is the entire value proposition. Hunting converts assumptions about your defences into evidence, and every honest answer — including the uncomfortable ones — makes the next intrusion more likely to be caught by a rule instead of a customer.

The telemetry a hunt actually needs

Hunting is bounded entirely by what you can see. Before planning a programme, it is worth being honest about which data sources exist, at what fidelity, and for how long. Four matter more than the rest.

Retention deserves as much attention as coverage. Dwell time is routinely measured in weeks, so thirty days of logs means a hunt can confirm recent activity but cannot establish when access began. If budget forces a choice, retaining fewer sources for longer generally beats retaining everything for a fortnight.

A second example: persistence you never provisioned

Execution hunts get the attention, but persistence hunts are often more productive, because persistence is rare, deliberate and comparatively easy to baseline. Legitimate persistence is provisioned by known processes and rarely changes; adversary persistence is added out of band.

The hypothesis: an adversary has established persistence via a scheduled task that survives reboot. The data source is task registration events, or Sysmon registry telemetry. The analysis is a stack count of task names and the commands they execute across the estate.

scheduled_task_created
WHERE task_command NOT IN (known_baseline)
GROUP BY task_name, task_command, creating_user
SORT BY count ASC

What surfaces is usually a short list. Software deployment tools create tasks with recognisable names and consistent paths. What does not fit — a task executing from a user profile directory, one whose name imitates a Microsoft component with a subtle spelling difference, or one created by an account that has no business creating tasks — is the finding. The same logic applies to services, run keys, WMI event subscriptions and startup folders.

Hunting identity rather than endpoints

A growing share of intrusions never touch an endpoint in a way EDR would notice. The adversary phishes credentials or a session token, authenticates as a legitimate user, and operates entirely through sanctioned interfaces. There is no malware, no unusual process, nothing for endpoint telemetry to reveal.

Identity hunting asks different questions. Which accounts authenticated from an unusual location or an anonymising service? Where has multi-factor authentication been satisfied in ways that suggest fatigue or interception rather than a genuine prompt? Which accounts registered a new authentication method shortly after a password reset — a common way to make access durable? Which mailbox rules were created to auto-forward or auto-delete, a hallmark of business email compromise? Which OAuth applications received consent, and who granted it?

These hunts rely on the audit logs of the identity provider, and they reward the same discipline as endpoint hunting: state the behaviour, find the population, stack count, and investigate the rare.

Recording a hunt so it compounds

Undocumented hunts are repeated from scratch by whoever comes next. A short, consistent record turns individual effort into institutional capability. Six fields are enough:

  1. Hypothesis as stated, before the work began.
  2. Data sources queried, with the exact time window.
  3. Method — the query or logic used, in a form someone can rerun.
  4. Baseline findings — benign activity identified and why it is benign. This is the section future hunters will thank you for.
  5. Outcome — incident, detection, visibility gap, or not found.
  6. Follow-up — what changed as a result, and who owns it.

Write the fourth field even when the hunt finds nothing, particularly then. The catalogue of explained-benign activity is what makes the next hunt take two hours instead of two days.

Maturing the practice

Programmes tend to progress along a predictable path. Early on, hunting is ad hoc and driven by whatever a threat report described last week. It becomes useful when hypotheses are drawn systematically from a framework, so coverage is deliberate rather than reactive. It becomes efficient when the analysis techniques that keep working — stack counting particular fields, comparing a host against its peer group — are turned into reusable notebooks or saved queries.

The end state is not automation of hunting itself, which would be a contradiction. It is that everything repeatable has been converted into a detection, freeing human attention for the genuinely novel questions that only a person can frame. A mature programme should be steadily working itself out of its own backlog, with each hunt either finding something or permanently removing the need to ask that question manually again.

References

  1. MITRE ATT&CK — adversary tactics and techniques knowledge base
  2. David Bianco — The Pyramid of Pain
  3. Microsoft Sysinternals — Sysmon
  4. Microsoft — Command line process auditing
  5. SigmaHQ — generic signature format for SIEM systems
« Living Off the Land: Detecting… Phishing-Resistant MFA: Why Passkeys Beat… »