Dual

Module 5: Agentic SecOps & Automation

Agentic SecOps & Automation

From CTI → detections, playbooks, and reliable human-in-the-loop agents

Outcome: Automate high-volume tasks safely while keeping analysts in control.

Topic Map (5)

• Agent tool-use & approvals
• CTI → ATT&CK/TTPs → detections
• Playbooks/orchestration budgets
• BAS/purple teaming with LLMs
• Auditability & connectors

Topic Map — Deep Dives (5)

• Agent tool-use & approvals
  • Capabilities list per agent; least-privilege scopes and expiring tokens.
  • Dry-run and ‘explain plan’ modes before execution.
  • Dual-control approvals for containment/deletion; justifications required.
  • Guardrails: allow/deny lists, regex/scope checks, environment sandboxes.
  • Structured action logs: who/what/when/why/inputs/outputs.
• CTI → ATT&CK/TTPs → detections
  • IE/NER over reports to extract entities, TTPs, and IOCs.
  • Mapping to ATT&CK technique/sub-technique IDs with confidence scores.
  • Rule drafting: Sigma/YARA templates with source citations and assumptions.
  • Replay/backtest harness using historical telemetry; unit tests per rule.
  • Change management: versioned rules, canary deploys, auto-rollback.
• Playbooks/orchestration budgets
  • Budgets: max actions/time/cost per incident; circuit breakers.
  • Idempotency keys and compensating actions for side-effectful steps.
  • Retry strategies with backoff and jitter; partial failure handling.
  • Data contracts for tools (schemas, timeouts, error surfaces).
  • Safe state machines: explicit terminals (success/fail/handoff).
• BAS/purple teaming with LLMs
  • Generate emulation steps from CTI; translate to atomic tests.
  • Constrain to lab/staging; forbidden-operation filters.
  • Measure detection coverage, alert quality, and MTTR deltas.
  • Iterate: failure cases feed prompt/rule improvements.
• Auditability & connectors
  • Event-sourced logs for every agent action; immutable storage.
  • Connector hardening: OAuth/device-code flows, token rotation, vault.
  • Provenance stamps on generated detections, notes, and tickets.
  • Privacy: PII minimization, redaction pipelines, retention policies.

Key Shifts Powered by AI (3)

• NLP/LLMs unlock CTI → detections Information extraction maps unstructured reports to ATT&CK/TTPs and detection rules. ^{[sok_ttp] [llmcloudhunter] [ttpxhunter]}
  Why it matters: Shorter time-to-detection content; less manual rule writing.
• From alerts to stories Graph learning/provenance + summarization reduce alert fatigue and speed IR. ^{[nodoze] [holmes]}
  Why it matters: Lower MTTR; better triage consistency.
• Human-over-the-loop agents LLM aids in drafting investigations and summaries while gated by approvals. ^[soups25_ir]
  Why it matters: Scale routine response without losing accountability.

Still Hard (5)

• Reliability, determinism, and cost control for agents at scale.
• Ground-truth evaluation of LLM-generated detections and false positive risk.
• Secure tool access: least-privilege scopes, short-lived creds, exhaustive logging.
• Measuring real analyst productivity gains beyond toy tasks.
• Vendor/API drift and brittle integrations over time.

References

1. Büchel et al. “SoK: Automated TTP Extraction from CTI Reports – Are We There Yet?” USENIX Security 2025.
2. Zhang et al. “Harnessing LLMs for Automated Extraction of Detection Rules from CTI (LLMCloudHunter).” 2024.
3. Rani et al. “TTPXHunter: Actionable Threat Intelligence Extraction as Sequence Labeling.” ACM TOPS 2024.
4. Hassan et al. “NoDoze: Combatting Threat Alert Fatigue with Automated Provenance Triage.” NDSS 2019.
5. Milajerdi et al. “HOLMES: Real-Time APT Detection through Correlation of Suspicious Information Flows.” IEEE S&P 2019.
6. Kramer et al. “Integrating Large Language Models into Security Incident Response.” USENIX SOUPS 2025.