Detection Engineering 101: Writing Detections That Survive

Most detections die young. A SOC ships a rule that fires on a specific file hash from last week’s incident report, and it works right up until the adversary recompiles their loader and the hash changes. Thirty seconds of the attacker’s time defeats a detection that took you an afternoon to build and will now sit in your ruleset forever, catching nothing. This is the central problem of detection engineering: it is easy to write detections that are technically correct and operationally worthless. The detections that survive are the ones that key on behavior the adversary cannot easily abandon, that are mapped to a shared taxonomy of adversary technique, and that are maintained like software rather than accumulated like sediment.

Two frameworks organize the entire discipline, and we will lean on both throughout. The first is David Bianco’s Pyramid of Pain, which explains why behavior-based detection hurts adversaries more than atomic indicators do. The second is MITRE’s Summiting the Pyramid, which gives you a way to score how durable a given detection actually is. Together they turn “write better detections” from a slogan into a repeatable engineering process.

The Pyramid of Pain

David J. Bianco published the Pyramid of Pain on his Enterprise Detection & Response blog in March 2013, and updated it on January 17, 2014 to add Hash Values at the base (the 2013 original started at IP Addresses). The pyramid ranks indicator types by how much pain it causes an adversary when you detect and deny them. From bottom to top:

Hash Values — trivial for the adversary to change. Flip a single byte, recompile, and the hash is new.
IP Addresses — easy. Spin up a new host, rotate through a VPN or bulletproof provider, and the old indicator is dead.
Domain Names — simple. Registering a new domain costs a few dollars and minutes.
Network Artifacts — annoying. A distinctive User-Agent string, a URI pattern, or a C2 beacon interval forces the adversary to reconfigure their tooling.
Host Artifacts — annoying. Registry keys, service names, mutexes, or dropped file paths that a tool leaves behind.
Tools — challenging. Detecting the tool itself (not one build of it) forces the adversary to find or write a replacement.
Tactics, Techniques & Procedures (TTPs) — tough. This is behavior: how they operate, not what they use.

Bianco’s argument lands at the top: when you detect and respond at the TTP level, “you’re operating directly on adversary behaviors, not against their tools.” That forces them to “learn new behaviors” and choose to “Give up” or “Reinvent themselves from scratch.” Recompiling a binary is free. Learning a fundamentally different way to dump credentials or move laterally is expensive, error-prone, and slow.

Hashes and IPs are cheap wins with a short shelf life — useful for blocking known-bad in bulk, but they decay by the hour. Behavior-based detections cost more to build and impose lasting cost on the adversary. Spend your engineering time near the top of the pyramid.

That does not mean atomic indicators are worthless. Blocking a known-malicious hash or IP is fast and cheap, and it stops the lazy and the automated. The mistake is treating the base of the pyramid as your detection strategy rather than as disposable ammunition.

Map to a Common Language: MITRE ATT&CK

If TTPs are what you want to detect, you need a shared vocabulary for them. That is MITRE ATT&CK. As of v17 (released April 2025), the Enterprise domain catalogs 14 Tactics, 211 Techniques, and 468 Sub-Techniques, and that release also added the ESXi platform. Every detection you write should be tagged to a specific technique or sub-technique ID — for example, T1059.001 (Command and Scripting Interpreter: PowerShell) or T1003.001 (OS Credential Dumping: LSASS Memory).

Tagging is not bureaucracy. It lets you answer “what can we actually catch?” using the free ATT&CK Navigator, a web-based tool for annotating the ATT&CK matrix. Color a technique green when you have a tested detection, red when you have a known gap, and you produce a coverage heatmap you can reason about, share with leadership, and use to prioritize the next quarter of work.

Coverage percentage is a vanity metric

Here is the trap. It is tempting to chase a big number — “we cover 78% of ATT&CK!” — and put it on a slide. That number is close to meaningless. As the Cloud Security Alliance put it in 2026, the effective programs “won’t be the ones with the highest coverage percentage on a slide — they’ll be the ones that map the right techniques, score detections honestly, layer in exposure context, correlate alerts into investigations, and refresh the map every quarter.” Covering an obscure technique no threat actor in your model uses, while missing the credential-dumping technique that shows up in every intrusion report for your sector, is worse than useless because it flatters you.

Summiting the Pyramid: scoring durability

ATT&CK tells you which behavior a detection targets. It does not tell you how hard that detection is to evade. MITRE’s Center for Threat-Informed Defense closed that gap with Summiting the Pyramid (STP), a methodology that scores how robust an analytic is — how difficult it is for an adversary to slip past it. STP’s core insight is to build detections on observables that trigger no matter how a technique is implemented — what its authors call a “spanning set” — actions the adversary must take regardless of the specific tool or observable they choose, tied to OS internals, authentication flows, or persistence mechanisms.

As the CTID team framed it, a robust analytic “will remain accurate, even as adversaries implement new and sneakier versions of ATT&CK techniques” precisely because it keys on invariants rather than on a particular tool’s fingerprint. That is the concrete backbone of “detections that survive”: climb the pyramid by anchoring your logic to what the technique fundamentally requires, not to how one implementation happens to look today.

The Detection Lifecycle

Detection engineering is a loop, not a one-shot. A workable lifecycle:

Hypothesis. Where does the idea come from? Threat intelligence about actors targeting your sector, gaps surfaced in your ATT&CK Navigator, and — best of all — the results of purple-team exercises where a red operator ran a technique and you found out whether you saw it.
Verify data source and log availability. This is the gating step, and it comes before you write a line of detection logic. Is the telemetry the detection needs actually being collected, parsed, and normalized into your SIEM? You cannot detect what you do not collect. Skipping this is the most common way a promising detection idea dies quietly.
Write the rule. Express the hypothesis as logic against the fields you confirmed exist.
Test. Fire the actual technique against a lab (and, carefully, production) using realistic adversary tooling and confirm the rule triggers — and, just as important, that benign activity does not.
Tune. Add filter clauses for the known-benign edge cases you discover.
Document and version. What does it detect, what are the known false positives, what is the response playbook, what ATT&CK ID does it map to.
Maintain or retire. Environments change; some rules stop earning their keep. Investigation output from real alerts feeds straight back into the next cycle.

The data source: Sysmon

On Windows, the workhorse that makes step 2 possible for most behavioral detections is Sysmon, part of the Sysinternals suite. It logs process creation with the full command line and parent/child relationships, image loads, network connections attributed to the originating process, file creation, registry changes, and — critically for credential-dumping detection — Process Access (Event ID 10). The two example rules below depend directly on Sysmon: process creation telemetry for the PowerShell rule, and Event ID 10 for the LSASS rule.

A relevant note on log availability: Microsoft now ships Sysmon as a Windows Optional Feature in Windows 11 and Windows Server 2025, enabled from the OS and serviced through Windows Update (it is disabled by default and cannot coexist with a standalone Sysmon install). That lowers the deployment friction that has historically kept teams from having the telemetry their detections assume.

Sigma: The Portable Detection Format

You have a hypothesis and confirmed telemetry. Now write the rule — ideally in a format you are not locked into. Sigma is the vendor-neutral, open, YAML-based generic signature format for SIEM systems — “for log files what Snort is for network traffic and YARA is for files.” One Sigma rule can be compiled to roughly 40 target backends — Splunk, Elastic, Microsoft Sentinel, QRadar, Chronicle, and more — via tooling like sigma-cli and pySigma. Write once, deploy everywhere, and share detections across the community without rewriting them for each SIEM’s query language.

A Sigma rule has three structural pieces. The logsource section (fields category, product, service) declares which log data to search. The detection section contains one or more named selections — field/value matches, optionally with modifiers like contains or endswith. And a boolean condition ties the selections together. Here is a (lightly abridged) SigmaHQ rule for base64-encoded PowerShell, mapped to T1059.001:

title: Suspicious Encoded PowerShell Command Line
id: ca2092a1-c273-4878-9b4b-0d60115bf5ea
status: test
description: Detects suspicious powershell process starts with base64 encoded commands (e.g. Emotet)
tags:
    - attack.execution
    - attack.t1059.001
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        - Image|endswith:
            - '\powershell.exe'
            - '\pwsh.exe'
        - OriginalFileName:
            - 'PowerShell.EXE'
            - 'pwsh.dll'
    selection_cli_enc:
        CommandLine|contains: ' -e'
    selection_cli_content:
        CommandLine|contains:
            - ' JAB'
            - ' SEVYI'
            - ' SQBFAFgA'
            - ' aQBlAHgA'
            - ' aWV4I'
            - ' IAA'
            - ' IAB'
            - ' UwB'
            - ' cwB'
    filter_optional_remote_signed:
        CommandLine|contains: ' -ExecutionPolicy remotesigned '
condition: selection_img and all of selection_cli_* and not 1 of filter_optional_*
level: high

Read it top to bottom. It fires when the image is PowerShell (selection_img) and the command line looks like a truncated -EncodedCommand flag (selection_cli_enc, matching -e, -en, -enc…) and the argument starts with one of the tell-tale base64 prefixes that PowerShell produces when it UTF-16LE-encodes common script openings (JAB is a leading $; the others decode toward strings like IEX and iex). The filter_optional_* clause carves out a common benign pattern. The real SigmaHQ rule carries a couple of additional selection and filter branches, but the structure is exactly this. Notice the rule already carries its ATT&CK tags — the mapping is baked into the artifact, not bolted on in a spreadsheet later.

Where does this sit on the pyramid? It targets the behavior of running obfuscated encoded commands rather than any single payload. An attacker can change the encoded script infinitely, but if their technique is “run base64 PowerShell,” they keep tripping this. That is a detection climbing toward TTPs.

Here is a second real SigmaHQ rule, for LSASS credential access (T1003.001), riding on Sysmon Event ID 10:

title: Potential Credential Dumping Activity Via LSASS
id: 5ef9853e-4d0e-4a70-846f-a9ca37d876da
status: test
description: Detects process access requests to the LSASS process with specific call
    trace calls and access masks. This behaviour is expressed by many credential
    dumping tools such as Mimikatz, NanoDump, Invoke-Mimikatz, Procdump and even
    the Taskmgr dumping feature.
logsource:
    category: process_access
    product: windows
detection:
    selection:
        TargetImage|endswith: '\lsass.exe'
        GrantedAccess|contains:
            - '0x1038'
            - '0x1438'
            - '0x143a'
            - '0x1fffff'
        CallTrace|contains:
            - 'dbgcore.dll'
            - 'dbghelp.dll'
            - 'kernel32.dll'
            - 'kernelbase.dll'
            - 'ntdll.dll'
    filter_main_system_user:
        SourceUser|contains:
            - 'AUTHORI'
            - 'AUTORI'
    # (rule also carries filter_optional_* clauses for known-benign tools)
condition: selection and not 1 of filter_main_* and not 1 of filter_optional_*
level: medium
tags:
    - attack.credential-access
    - attack.t1003.001

This rule is a small masterclass in behavioral invariance. To read the secrets in LSASS memory, a tool must open a handle to lsass.exe with sufficient rights — that is what the GrantedAccess flags capture. Sysmon logs that mask as a literal string, so rules match on exact values rather than bitwise, and this rule targets the masks associated with several dumping paths: 0x1438 and 0x143a map to Mimikatz’s lsadump operations, 0x1038 is another commonly abused pattern, and 0x1FFFFF is PROCESS_ALL_ACCESS. (For reference, the classic Mimikatz sekurlsa default when run from the command line is 0x1010 — PROCESS_VM_READ | PROCESS_QUERY_INFORMATION — which this particular rule does not include; a complete LSASS-access program layers in rules that cover it.) The rule further constrains on the CallTrace passing through the debug/dump DLLs these tools rely on, and filters out the SYSTEM account’s legitimate access. An attacker can rename Mimikatz, repack it, or write their own dumper — but if they use the standard Win32 path to touch LSASS memory, they still request those access masks. The detection targets the technique’s requirement, not the tool’s name.

Sigma vs YARA: Behavior vs Content

Sigma matches log events — behavior over time. YARA matches file and memory content — the bytes and strings of an artifact. Created by Victor M. Alvarez and maintained under VirusTotal, YARA is the standard for malware classification, file scanning, memory forensics (scanning process dumps produced by Volatility or Rekall), and retro-hunting across corpora like VirusTotal Intelligence and MalwareBazaar. A YARA rule has three sections: meta, strings (plaintext, hex byte sequences, or regex), and a required boolean condition. The canonical shape from the official docs:

rule MetadataExample
{
    meta:
        my_identifier_1 = "Some string data"
        my_identifier_2 = 24

    strings:
        $a = "text here"
        $b = { E2 34 A1 C8 23 FB }
        $c = "another string"

    condition:
        $a or ($b and $c)
}

The power of YARA on memory is that it catches what disk scanning misses: an attacker’s payload might be encrypted or packed on disk (defeating a hash or a naive string match) but must be unpacked in memory to run. A YARA rule scanning a memory image can hit the decoded strings and injected code that never appear cleanly on disk. (Note that the original YARA is now in maintenance mode in favor of a Rust rewrite, YARA-X, which reached its first stable release in 2025; the rule syntax above carries over.)

Place both tools on the pyramid honestly. A YARA rule matching a specific tool’s byte patterns usually sits at the Tools or Host Artifacts level — powerful, but evadable by repacking or rewriting the tool. A Sigma rule keyed on behavioral invariants reaches toward TTPs. The lesson is not “Sigma good, YARA bad” — it is to know which pain level each detection buys you, and to deploy both deliberately: YARA to shred known tooling on sight, Sigma to catch the behavior that outlives any one tool.

Alert Fatigue, False Positives, and the Precision/Recall Tradeoff

Every detection you deploy is a bet against your analysts’ attention. The SANS 2024 Detection & Response Survey found that 73% of respondents struggle to craft quality and reliable detection rules, and 64% cite false positives as a major problem in threat detection. That pain compounds because SOC staffing is chronically thin — resource and staffing constraints are repeatedly cited as top barriers. A noisy rule in a small shop is not a nuisance; it is a denial-of-service against the humans who are supposed to catch the real thing.

This is the precision/recall tradeoff, and it is unavoidable:

Precision — of the alerts that fire, how many are true positives. Low precision means false-positive floods, alert fatigue, and — the real danger — genuine alerts buried and ignored.
Recall — of the real attacks, how many you catch. A rule tuned so narrowly that it never false-positives often has terrible recall: it misses the variant the attacker actually used, driving up Mean Time to Detect.

You cannot maximize both at once. The lever that lets you buy precision without sacrificing much recall is tuning — the filter_* clauses you saw in both real rules above. Those filters are not decoration; they are the difference between a rule that survives contact with production and one that gets muted by an exhausted analyst in week one. Write the behavioral core broadly enough to catch variants, then carve out your specific environment’s known-benign patterns surgically.

Detection-as-Code and Metrics

If detections are logic, they deserve the same discipline as any other production code. Detection-as-Code (DaC) stores rules as code — Sigma YAML in a Git repository — versioned and reviewed through pull requests, and validated in CI/CD before deployment. A realistic pipeline:

An engineer writes or edits a Sigma rule and opens a pull request.
CI runs sigma-cli/pySigma to validate schema and syntax, unit-tests the rule against labeled known-bad and known-good log samples, and checks that ATT&CK tags are present.
A peer reviews the logic and the documented false-positive profile.
On merge, the rule is automatically converted and deployed to the SIEM (Splunk, Sentinel, Elastic).
Git history gives you versioning, auditability, and one-command rollback when a rule misbehaves in production.

This is the antidote to the “sediment” problem — rules that nobody remembers writing, nobody tests, and nobody dares delete. In a DaC repo, every rule has an author, a review, a test, and a change history.

Measure honestly

Two metrics matter most: detection coverage vs ATT&CK (mapped, as discussed, to the right techniques rather than a headline percentage) and Mean Time to Detect (MTTD). But be careful with MTTD. It is one of the most-tracked and least precisely defined SOC metrics — different teams clock it from the earliest telemetry signal, from alert generation, or from analyst confirmation, and those produce wildly different numbers. If you are going to report it, define the start point consistently, keep the measurement window fixed, and exclude false positives, or you are comparing noise to noise.

Key Takeaways

Aim high on the Pyramid of Pain. Hashes and IPs cost the adversary seconds to change; TTP-level behavioral detections force them to give up or reinvent themselves. Spend your engineering effort near the top.
Anchor detections to behavioral invariants. Key on what a technique fundamentally requires (the LSASS access mask, the encoded-command pattern), not on one tool’s name or hash. That is what makes a detection survive infrastructure and tooling churn.
Verify the log source before writing logic. You cannot detect what you do not collect. On Windows, Sysmon — now shipping natively in Windows 11 and Server 2025 — is the practical foundation.
Map to ATT&CK, but treat coverage percentage as a vanity metric. Cover the right techniques for your threat model, score robustness with Summiting the Pyramid, and refresh quarterly.
Tune for precision to fight alert fatigue. With a majority of SOC teams citing false positives as a major problem, surgical filter clauses are what keep a behavioral rule alive in production.
Ship detections as code. Git, peer-reviewed pull requests, CI tests against known-good/known-bad samples, automated deploy, and honest metrics turn a pile of rules into a maintainable detection program.

References

« The Windows Event Logs That… Identity Is the New Perimeter:… »