D6 — Security Metrics, Log Review & Continuous Monitoring

Lesson 9 covered how you test for vulnerabilities. This lesson covers how you know, on an ongoing basis, whether your security posture is actually improving, holding steady, or degrading — through metrics, log review, and continuous monitoring.

Metrics: measurable isn't the same as actionable

A security metric is only useful if it drives a decision. The common failure mode is the vanity metric — a number that's easy to collect and looks impressive, but doesn't tell anyone what to do differently:

• "Number of firewall rules configured" — measurable, but tells you nothing about whether those rules are correct, necessary, or effective. More rules isn't inherently better.
• "Total security spend" — measurable, but says nothing about risk reduction achieved per dollar.

An actionable metric ties directly to a decision or a trend that demands a response:

• "Mean time to detect (MTTD)" and "mean time to respond (MTTR)" — trending upward signals a real capability gap requiring investigation, and both connect directly to Lesson 7's incident-response lifecycle.
• "Percentage of critical vulnerabilities remediated within SLA" — a declining trend signals a specific, addressable process problem (patching cadence, resourcing, prioritization).

Two related terms worth distinguishing precisely: a KPI (Key Performance Indicator) measures how well a process is performing against its own target (e.g., "% of employees completing security awareness training on time"); a KRI (Key Risk Indicator) signals rising risk exposure before it becomes an incident (e.g., "number of unpatched critical vulnerabilities older than 30 days"). A KPI tells you if the process is working; a KRI warns you the risk is growing — an organization can have a healthy-looking KPI while its KRI is quietly climbing toward a real problem.

Log review: a distinct capability from log collection

Collecting logs and reviewing them are not the same capability, and an organization can have extensive logging infrastructure while genuinely lacking the second one. Log collection means events are captured and stored, typically centralized via a SIEM (Security Information and Event Management) platform. Log review means someone or something is actually analyzing that collected data for meaningful signal — correlating events across sources, tuning detection rules, and triaging alerts.

This distinction matters because "we have a SIEM" is frequently treated as evidence of security maturity when it only demonstrates the collection half. An organization drowning in unreviewed alerts (alert fatigue, where the volume of low-fidelity alerts causes real ones to be missed or ignored) has strong log collection and weak log review — the exact gap that most post-incident reviews find: the evidence was captured, sitting in the SIEM, and never actually looked at before the breach was discovered some other way.

Continuous monitoring vs. point-in-time assessment

Lesson 9's testing types (vulnerability assessment, penetration test, audit) are all, by nature, point-in-time: they describe the security posture at the moment the test ran, and say nothing certain about the moment after. A system that passed a penetration test on Monday can be misconfigured, patched incorrectly, or newly exploited on Tuesday, and the Monday test provides zero visibility into that.

Continuous monitoring closes this specific gap: automated, ongoing observation of security-relevant state (configuration drift, new vulnerabilities as they're disclosed, anomalous behavior) rather than periodic snapshots. It doesn't replace point-in-time testing — a penetration test still proves exploitability in a way passive monitoring can't — but it addresses the structural blind spot every point-in-time assessment necessarily has: the time between assessments.

The CISSP mindset: manager vs technician

A security team reports to leadership that "we blocked 2 million malicious login attempts this quarter," presented as evidence the security program is working well.

The technician answer: the number is accurate and demonstrates the defenses are actively stopping attacks — a strong headline figure for a leadership report.

The manager (CISSP) answer: ask what decision this number actually supports. 2 million blocked attempts, in isolation, is a vanity metric — it says nothing about whether that volume is trending up or down, whether the rate of successful compromises is changing, or whether the attempts represent a genuinely rising threat the organization is barely keeping pace with. The CISSP-correct reframe replaces it with an actionable pairing: the trend over time (rising, falling, stable) alongside a KRI like the number of accounts with weak or reused credentials still exposed to those attempts — because "we blocked a lot of attacks" and "our actual risk exposure is decreasing" are different claims, and only the second one tells leadership whether to invest further or not.