Abstract. Cyber threat attribution, associating observed intrusion activity with a responsible actor or campaign cluster, remains among the least automated functions in security operations, despite the maturity of automation elsewhere in the detection pipeline. This paper argues that the obstacle is not primarily algorithmic but representational and epistemic. We examine why indicator-centric representations fail to support generalisation, survey the feature representations and inference methods applicable to machine-assisted attribution, and identify the evaluation problems (label scarcity, class imbalance, temporal leakage and deliberate deception) that make reported accuracy figures in this domain difficult to interpret. We propose that attribution systems be designed and evaluated as instruments of calibrated inference rather than classification, producing ranked hypotheses with explicit uncertainty and traceable evidence, and we set out the properties such a system must exhibit to be operationally useful.

1. Introduction

Security operations have automated substantially over the past decade. Detection engineering has moved from static signatures toward behavioural analytics; triage is increasingly assisted by enrichment and correlation; response actions are orchestrated. Attribution has largely resisted this trend. In most organisations it remains a manual, expert-driven activity conducted after the fact, if it is conducted at all.

This is not because attribution is unimportant. Knowing which activity cluster one is facing carries direct operational consequences: it constrains the space of likely subsequent techniques, indicates which assets are plausible targets, and informs which detections are worth building first. The value is highest early in an incident and decays rapidly: an attribution judgement delivered after remediation is complete is of historical interest.

The mismatch between the value of early attribution and the latency of manual attribution is the motivation for this work. The question is not whether attribution can be automated in the sense of removing human judgement, which we argue is neither achievable nor desirable, but whether the evidence-marshalling that precedes judgement can be performed at machine speed and scale.

2. The attribution problem

Attribution is used to mean several distinguishable tasks, and conflating them is a persistent source of confusion. Following the distinction drawn in the strategic-studies literature, it is useful to separate:

Only the first is a plausible target for automation. Operational and strategic attribution depend on evidence classes (human intelligence, legal process, geopolitical context) that are neither available to nor interpretable by a detection pipeline. This paper concerns technical attribution exclusively, and we treat the output as a cluster assignment with confidence rather than a name.

Even so constrained, the problem has properties that distinguish it sharply from ordinary classification.

2.1 The ground truth is weak

Labels in this domain derive largely from vendor reporting, which is inconsistent in naming, variable in rigour, and produced by organisations with differing visibility. The same activity cluster carries several names across vendors; conversely, a single vendor label may span activity that later analysis separates. A supervised model trained on such labels learns, in part, the labelling conventions of its sources.

2.2 The classes are open and unbalanced

New clusters appear continuously, so the label space is open rather than closed. Any deployed system must be able to answer “none of the known clusters”: a requirement that ordinary closed-set classifiers handle poorly. Meanwhile the distribution over known clusters is extremely skewed: a small number of heavily reported groups dominate the training data, which biases models toward them for reasons unrelated to the evidence.

2.3 The adversary is aware of the method

This is the property with no analogue in most machine-learning applications. Threat actors read the reporting that describes how they are tracked, share tooling that dissolves distinctiveness, and in some documented cases plant artefacts intended to induce misattribution. Attribution therefore takes place under adversarial conditions in which the features themselves are partly under the adversary’s control.

3. Representation: what should the features be?

The representational choice dominates system performance, and it is where most naive approaches fail.

3.1 Why indicators do not generalise

The intuitive features (file hashes, IP addresses, domain names) are precisely the wrong ones. Bianco’s Pyramid of Pain formalises the reason: these artefacts are cheap for an adversary to change, so their presence is highly specific but their absence is uninformative. A model keyed on them memorises campaigns rather than learning actors, and degrades to chance the moment infrastructure rotates.

The formal statement of the problem is that indicators have high specificity and near-zero recall across time. They are excellent for confirming a match and useless for generalising beyond one.

3.2 Behaviour as the feature space

The durable signal lies in tactics, techniques and procedures: how the adversary achieves objectives rather than which artefacts they leave. Encoding an incident as a set of observed ATT&CK techniques yields a representation that is stable across campaigns because changing it requires the adversary to change how they operate, which carries real cost in retraining, tooling and operational risk.

A minimal encoding represents an incident as a binary vector over the technique space. This is workable but discards information that matters:

The last point deserves emphasis. ATT&CK technique identifiers are a coarse quantisation of behaviour. Much of the signal an analyst uses lives below that resolution, and a representation that stops at the technique label discards it.

3.3 Structured intelligence formats

Where structured threat intelligence is available in a standard representation, it provides relationship structure (between actors, campaigns, tooling and infrastructure) that a flat feature vector cannot express. Graph representations built on such data support inference over relationships (shared tooling, shared infrastructure, temporal proximity) rather than over attributes alone, which better matches how attribution reasoning actually proceeds.

4. Methods

Four families of approach are applicable, with different assumptions and failure modes.

4.1 Similarity-based retrieval

The simplest useful formulation treats attribution as nearest-neighbour retrieval: encode the incident, compare against encoded profiles of known clusters using a similarity measure appropriate to sparse binary data, and return a ranked list. This approach has three practical virtues that outweigh its simplicity. It handles the open-set problem naturally, since a low maximum similarity is interpretable as “no known match”. It requires no retraining as new clusters are added. And it is inherently explainable: the specific techniques driving the similarity can be enumerated.

4.2 Unsupervised clustering

Rather than assigning to known labels, clustering discovers structure in the activity itself. This sidesteps the weak-label problem entirely and can surface previously unrecognised relationships between incidents. Its output is not attribution in the naming sense but campaign grouping, which is frequently the more operationally useful product and is closer to what the evidence can actually support.

4.3 Supervised classification

Standard discriminative models can be trained where labels exist. They typically outperform similarity methods on well-represented classes and fail in the ways predicted by the problem structure: poor performance on rare clusters, overconfidence on out-of-distribution inputs, and sensitivity to the labelling conventions of the training corpus. Where used, calibration is essential: an uncalibrated softmax output is not a probability and should never be presented to an analyst as one.

4.4 Natural language processing over reporting

The majority of attribution-relevant knowledge exists as unstructured prose in vendor reports, incident write-ups and research publications. Extracting technique and tooling mentions from this corpus, and mapping them to a structured representation, is the mechanism by which cluster profiles are constructed at a scale no analyst can match by reading.

This introduces its own hazards. Extraction errors propagate silently into profiles. Reporting volume correlates with vendor interest rather than actor activity, so profiles built from reporting inherit that bias. And prose frequently describes techniques hypothetically or by way of comparison, which naive extraction records as observation.

5. Evaluation

Reported accuracy figures in this literature should be read with more than usual scepticism, for reasons intrinsic to the domain.

5.1 Temporal leakage

Random train/test splits are invalid here. Incidents from the same campaign share infrastructure and tooling, so a random split places near-duplicates on both sides and inflates measured performance dramatically. Evaluation must split temporally (train on activity before a cutoff, test after) which both reflects deployment reality and typically produces markedly worse numbers.

5.2 The open-set requirement

A system evaluated only on incidents belonging to known clusters has not been evaluated on the case that matters most operationally. Test sets must include activity from clusters absent in training, and the metric must reward correct abstention. A model that never says “unknown” is not usable regardless of its closed-set accuracy.

5.3 Calibration over accuracy

Top-one accuracy is the wrong headline metric. An attribution system informs a decision made by a human under uncertainty, so what matters is whether its stated confidence is meaningful: among assessments made at 70% confidence, roughly 70% should be correct. Calibration error and ranked metrics such as mean reciprocal rank are more informative than accuracy, and abstention behaviour should be reported explicitly.

5.4 Confidence must be expressible in analytic language

Output intended for intelligence consumers should map onto established probability-of-estimate conventions rather than presenting bare numeric scores, so that automated assessments compose with human ones rather than sitting awkwardly beside them.

6. Adversarial considerations

Because the adversary can observe and react to attribution methodology, three failure modes must be treated as design constraints rather than edge cases.

False flags. Artefacts characteristic of another actor may be deliberately introduced. A system that weights any single feature heavily is straightforwardly manipulable; robustness argues for requiring corroboration across independent evidence classes before confidence rises materially.

Convergence. Widespread adoption of the same commodity tooling and living-off-the-land techniques compresses the distance between actors in behavioural feature space. This is a genuine and worsening limit on discriminative power, not a modelling defect, and it argues for representations that reach below the technique label to procedural detail.

Feedback. Published methodology becomes an optimisation target. Any system whose logic is documented publicly should be assumed to be evadable by a motivated actor, which is an argument for ensembles over single decision rules and for treating attribution confidence as perishable.

7. Limitations

Several constraints bound what any system of this kind can deliver, and stating them plainly is more useful than implying they can be engineered away.

Attribution is underdetermined by technical evidence alone. Distinct actors can produce identical technical footprints, so the evidence supports a likelihood ratio rather than a determination. No improvement in modelling changes this.

Ground truth is unavailable in the strict sense. Reference labels are themselves analytic judgements, so a system agreeing with them has demonstrated agreement with prior analysis rather than correctness.

Visibility bounds inference. Technique profiles are constructed from what telemetry captured, so absence of a technique conflates “did not occur” with “not instrumented”, and instrumentation varies systematically across organisations.

Finally, the consequences of error are asymmetric and external. Misattribution can inform decisions with legal, commercial or diplomatic weight. A system that is right more often than a human but confidently wrong in unfamiliar cases may nonetheless be worse in practice than one that abstains more readily.

8. Design implications

Taken together, these considerations suggest a system with the following properties.

  1. Ranked hypotheses, never a single name. The output should be an ordered set of candidate clusters with calibrated confidence and an explicit “insufficient evidence” option.
  2. Behaviour-first representation, weighted by discriminative power, retaining sequence and reaching to procedural detail where telemetry permits.
  3. Traceable evidence. Every assessment must enumerate the observations driving it, so an analyst can evaluate the reasoning rather than the score. An unexplainable attribution is unusable in any context where the judgement will be challenged.
  4. Corroboration requirements before confidence rises, as a structural defence against planted evidence.
  5. Temporal validity. Assessments should carry an explicit basis date, since both actor behaviour and the reference corpus change.
  6. Human adjudication retained. The system’s role is to marshal and rank evidence at machine speed; the determination remains an analytic judgement.

9. Conclusion

The barrier to automated attribution is less a shortage of suitable algorithms than a set of representational and epistemic problems that algorithmic sophistication does not address. Indicator-centric features cannot generalise; labels are weak and inconsistent; the class space is open; and the adversary is a participant in the measurement.

Framing the task as calibrated inference rather than classification resolves much of the resulting confusion. Under that framing the objective is not to name an actor but to compress the analyst’s hypothesis space quickly, quantify the residual uncertainty honestly, and expose the evidence that a human can then adjudicate. That is a considerably more modest goal than the term “automated attribution” usually implies, and it is achievable with existing methods, which is precisely why it is the goal worth pursuing.

// related work

The detection-engineering counterpart to this research, and the tooling referenced throughout.

Research project Hash Analyzer

References

  1. MITRE ATT&CK, Adversary Tactics and Techniques Knowledge Base
  2. D. Bianco, The Pyramid of Pain (2013)
  3. S. Caltagirone, A. Pendergast, C. Betz, The Diamond Model of Intrusion Analysis (2013)
  4. OASIS, STIX / TAXII Structured Threat Information Documentation
  5. MITRE ATLAS, Adversarial Threat Landscape for AI Systems
  6. T. Rid and B. Buchanan, “Attributing Cyber Attacks,” Journal of Strategic Studies, vol. 38, no. 1–2, 2015.
  7. E. Hutchins, M. Cloppert, R. Amin, “Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains,” Lockheed Martin, 2011.
« Container and Kubernetes Security: The… Adversarial Machine Learning Against Security… »