AI→Sec

Module 2: Runtime Monitoring

Runtime Monitoring in an Encrypted World: Logs, Telemetry & Network Flows

Logs as language; learning from flows when payloads are opaque

Outcome: Detect and explain sophisticated attacks using logs-as-language and payload-blind flow analysis.

Learning Objectives (3)
  • Model logs as sequences to detect deviations beyond rules
  • Infer behaviors from encrypted flow metadata (what’s possible vs. not)
  • Correlate sources into a single, explainable narrative
Topic Map (5)
  • Log anomaly (sequence models)
  • Cross-source correlation
  • Encrypted traffic inference
  • Flow/timing fingerprinting
  • DFIR timelines & reporting
Topic Map — Deep Dives (3)
  • Log anomaly (seq models)
    • Template mining → stable event IDs; tokenization (e.g., Drain/Drain3)
    • Sequence baselines vs. deep models; SOC-tunable thresholds
  • Encrypted traffic inference
    • Timing/size/burst features; learned fingerprints (beyond JA3/JA4 hints)
    • QUIC/ECH limits; adversarial padding/morphing
  • Cross-source correlation
    • Entity resolution; time-window joins; causal chains
Key Shifts Powered by AI (3)
  • Logs → Sequences → Models Deep sequence models learn normal log semantics and flag deviations without brittle rules. [deeplog] [logbert] [loganomaly] [loggpt] [logllm]
    Why it matters: Lower rule debt; better zero-day sensitivity.
  • Payload-Blind Detection Flow/timing features enable useful classification/correlation even under TLS/QUIC/ECH. [flowprint] [deep_packet] [deepfinger] [deepcorr] [rimmer18]
    Why it matters: Retain visibility while respecting encryption; quantify limits.
  • Narratives, not Noise Embeddings and summarization condense alerts into human-readable timelines and causal stories.
    Why it matters: Faster IR handoffs and executive reporting.
Still Hard (5)
  • Concept drift and seasonality across tenants
  • Label scarcity & noisy ground truth
  • Cross-system generalization for log models
  • Privacy budgets for derived features
  • QUIC/ECH evolution breaking fragile fingerprints

References

  1. Du et al. “DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning.” CCS 2017.
  2. Guo et al. “LogBERT: Log Anomaly Detection via BERT.” 2021.
  3. Meng et al. “LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs.” IJCAI 2019.
  4. Han et al. “LogGPT: Log Anomaly Detection via GPT.” 2023.
  5. Guan et al. “LogLLM: Log-based Anomaly Detection Using Large Language Models.” 2024.
  6. Mirsky et al. “Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection.” NDSS 2018.
  7. van Ede et al. “FlowPrint: Semi-Supervised Mobile-App Fingerprinting on Encrypted Network Traffic.” NDSS 2020.
  8. Nasr et al. “DeepCorr: Strong Flow Correlation Attacks on Tor Using Deep Learning.” CCS 2018.
  9. Sirinam et al. “Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning.” CCS 2018.
  10. Rimmer et al. “Automated Website Fingerprinting Through Deep Learning.” NDSS 2018.
  11. Lotfollahi et al. “Deep Packet: A Novel Approach for Encrypted Traffic Classification Using Deep Learning.” 2017.
  12. He et al. “Drain: An Online Log Parsing Approach with Fixed Depth Tree.” ICWS 2017.