Security Operations: SIEM, SOC & Threat Intelligence
Understand how Security Operations Centers work — SIEM log aggregation, alert triage, threat intelligence integration, and the daily reality of defending an organization.
Learning Objectives
- → Describe the SOC team structure and analyst tiers
- → Explain how SIEMs ingest, correlate, and alert on log data
- → Write SIEM correlation rules to detect common attack patterns
- → Integrate threat intelligence feeds (IOCs) into security operations
- → Apply the MITRE ATT&CK framework to map and hunt for threats
Firewalls, patches and access controls reduce the chance of a breach, but none of them reduce it to zero. Security operations is the discipline that accepts this and asks the next question: when something gets through, how fast do we notice, and what do we do about it? The answer is a combination of people (the SOC), a platform that collects and correlates logs (the SIEM), and knowledge about attackers (threat intelligence).
This lesson explains how those three pieces fit together, how an alert travels from a raw log line to a closed ticket, and the queries and rules analysts write every day. It is also the most common entry point into a cybersecurity career: many security professionals start as Tier 1 SOC analysts.
Related: the hands-on side of log analysis is covered in Regular Expressions & Log Parsing. What happens after an alert becomes an incident is covered in Incident Response.
The Security Operations Center (SOC)
A SOC is the team and facility responsible for monitoring, detecting, and responding to security incidents 24/7.
Larger organisations run their own SOC; smaller ones often outsource it to a managed security service provider (MSSP) or a managed detection and response (MDR) service. Either way, the work is organised in tiers, so that the high volume of routine alerts doesn't consume the time of the most experienced people.
SOC Analyst Tiers
Tier 1 – Alert Analyst
Monitor dashboard, triage alerts, escalate
Tools: SIEM, EDR, ticketing system
Goal: decide real vs false positive in <15 min
Tier 2 – Incident Responder
Investigate escalated incidents, contain threats
Tools: forensic tools, sandbox, SIEM deep dive
Goal: root cause analysis, containment
Tier 3 – Threat Hunter / Expert
Proactively hunt for undetected threats
Tools: threat intel, MITRE ATT&CK, custom scripts
Goal: find what automated detection missed
Supporting Roles:
Security Engineer – builds/maintains tools
Threat Intel Analyst – tracks actors and TTPs
SOC Manager – metrics, escalation, reporting
The biggest day-to-day challenge for Tier 1 is alert fatigue: thousands of alerts a day, most of them benign. When analysts are flooded, real attacks get closed as noise. That's why tuning rules (removing chronic false positives) is as important as writing new ones.
SIEM (Security Information and Event Management)
A single log source rarely proves an attack. One failed login is nothing; 500 failed logins from one IP followed by a success, then a new admin account on the same server, is an intrusion. A SIEM aggregates logs from all sources and correlates events across them to detect attacks that no single system would notice:
Sources:
Firewalls → connection logs
IDS/IPS → alert logs
Web servers → access logs
Auth systems → login events
Endpoints → process/file/network events (EDR)
DNS → query logs
DHCP → IP assignment logs
SIEM Pipeline:
Log collection → Normalization → Indexing → Correlation → Alert → Ticket
DNS and DHCP logs look boring but are essential during investigations. DHCP tells you which machine had a given IP at a given time, and DNS shows which domains it contacted, often the first sign of malware talking to its command server.
Common SIEM platforms include Splunk, Microsoft Sentinel, Elastic Security, IBM QRadar and the open-source Wazuh. The concepts in this lesson apply to all of them; only the query language changes.
SIEM Log Normalization
Every product logs in its own format. A firewall calls the attacker SRC, sshd writes from, a Windows event uses IpAddress. To correlate them, the SIEM maps every source onto a common schema, so that one query on source.ip covers all of them:
Firewall log:
Jan 15 10:23:01 fw1 kernel: DROPPED IN=eth0 SRC=185.x.x.x DST=10.0.0.5 DPORT=22
Auth log:
Jan 15 10:23:02 server sshd[1234]: Failed password for root from 185.x.x.x port 54321
Normalized (CEF or ECS format):
timestamp: 2025-01-15T10:23:01Z
source.ip: 185.x.x.x
destination.ip: 10.0.0.5
destination.port: 22
event.action: blocked / failed-auth
event.category: network / authentication
Normalisation is where detections quietly fail. If a parser breaks after a software update and stops filling source.ip, every rule that depends on that field goes silent, without any error. Mature SOCs monitor their own log sources for this: a source that suddenly stops sending data is itself an alert.
Splunk SPL (Search Processing Language)
SPL queries are pipelines: each | passes results to the next command, much like a Unix shell. The three searches below cover classic detections.
Failed SSH logins per source IP. rex extracts the IP with a regular expression, stats counts per IP, and where keeps the noisy ones:
index=linux sourcetype=syslog "Failed password"
| rex "from (?<src_ip>\d+\.\d+\.\d+\.\d+)"
| stats count as failures by src_ip
| where failures > 10
| sort -failures
Brute force followed by a success. Failures alone are routine on any internet-facing server. A failure-then-success pair from the same IP for the same user means a password may have been guessed:
index=linux sourcetype=syslog ("Failed password" OR "Accepted password")
| rex "(?<status>Failed|Accepted) password for (?<user>\S+) from (?<src_ip>[\d.]+)"
| stats dc(status) as outcomes, count by src_ip, user
| where outcomes = 2
| rename src_ip as "Attacker IP"
dc(status) counts the distinct outcomes per IP and user. A value of 2 means both Failed and Accepted were seen.
Large outbound transfers, a possible sign of data exfiltration:
index=network
| stats sum(bytes_out) as total_out by src_ip, dst_ip
| where total_out > 100000000
| sort -total_out
A fixed 100 MB threshold is a starting point, not a final rule. Backup servers and software updates will trigger it constantly. In practice you exclude known destinations, or compare each host against its own normal volume.
Elastic / ECS Queries
Elastic uses JSON queries over ECS-normalised fields. This one finds authentication failures in the last hour from outside the internal 10.0.0.0/8 range, grouped by source IP:
{
"query": {
"bool": {
"must": [
{"match": {"event.action": "authentication_failure"}},
{"range": {"@timestamp": {"gte": "now-1h"}}}
],
"must_not": [
{"term": {"source.ip": "10.0.0.0/8"}}
]
}
},
"aggs": {
"by_ip": {
"terms": {"field": "source.ip", "size": 10}
}
}
}
The term query accepts CIDR notation because ECS defines source.ip as an ip field type. On a plain text field, the same filter would silently match nothing, which is one more reason normalisation matters.
Correlation Rules
Searches answer questions you ask. Correlation rules run continuously and fire alerts when specific patterns occur across multiple events. The pseudo-code below shows the logic of five common rules; every SIEM has its own syntax for them:
# Pseudo-code for common correlation rules
# Rule 1: Brute Force Detection
rule BruteForce:
if count(event.type == "auth_failure", src_ip, 5 minutes) > 10:
alert("Brute Force", severity="HIGH", src_ip=src_ip)
# Rule 2: Impossible Travel
rule ImpossibleTravel:
if user logs in from country_A, then from country_B within 2 hours
and distance(country_A, country_B) / 2 hours > 1000 km/h:
alert("Impossible Travel", severity="HIGH", user=user)
# Rule 3: Lateral Movement (SMB from workstation to workstation)
rule LateralMovement:
if SMB connection from workstation to workstation (not server):
alert("Possible Lateral Movement", severity="MEDIUM")
# Rule 4: Data Exfiltration
rule DataExfil:
if bytes_out > 100MB from internal host to external IP in 1 hour:
alert("Possible Data Exfiltration", severity="HIGH")
# Rule 5: PowerShell encoded command (evasion technique)
rule PowerShellEncoded:
if process.name == "powershell.exe"
and process.args contains "-EncodedCommand":
alert("PowerShell Encoded Command", severity="MEDIUM")
Each rule trades coverage against noise:
- Impossible travel fires on VPN users and cloud proxies that make a user appear in two countries at once. Most deployments exclude known VPN egress IPs.
- Workstation-to-workstation SMB is rare in a well-run network, which makes it a strong signal: workstations normally talk to servers, not to each other.
- Encoded PowerShell is used legitimately by some management tools, so it usually starts at medium severity and is tuned per environment.
A good rule documents why it exists, which ATT&CK technique it covers and which known false positives were excluded, so the next analyst can maintain it.
Alert Triage Process
Triage is the Tier 1 core skill: deciding quickly and consistently whether an alert is real. A repeatable process matters more than intuition, because it makes decisions auditable and lets a team hand work over between shifts:
1. Receive alert in ticketing system (Jira, ServiceNow, TheHive)
2. Initial context: what triggered it? What asset? What user?
3. Enrich: lookup source IP in threat intel (VirusTotal, MISP, AbuseIPDB)
4. Pivot: look for related events (same IP, same user, same host)
5. Classify:
- True Positive → escalate or respond
- Benign True Positive → the rule worked, but the activity is authorised
(e.g. a scheduled vulnerability scan) → close and document
- False Positive → the rule fired on something it shouldn't have
→ close with tuning note
6. Document findings in ticket
7. Update SIEM rule if false positive is chronic (tune threshold)
Asset context changes everything. The same alert means very different things on a developer's test VM and on the domain controller. Many SOCs weight alert severity by asset criticality for this reason.
Document as if someone else will read it, because someone will: the next shift, the incident responder, or an auditor. A ticket that says "checked, FP" is useless; one that says "source IP is our Qualys scanner, confirmed with the vulnerability-management team, scan window 02:00–04:00" lets the next analyst close the same alert in seconds.
Threat Intelligence
Threat intelligence is knowledge about adversaries that helps you defend: who targets organisations like yours, how they operate, and which concrete indicators reveal them. It comes in three levels, each serving a different audience:
Types of Threat Intel:
Strategic – high-level trends for executives
Operational – campaigns, actor TTPs, playbooks
Tactical – IOCs: IPs, hashes, domains, URLs
Sources:
Commercial: Recorded Future, CrowdStrike Falcon X
Open Source: MISP, AlienVault OTX, Shodan, VirusTotal
Feeds: AbuseIPDB, Emerging Threats, URLhaus
IOC Integration:
IP blocklist → firewall/proxy
Domain blocklist → DNS sinkhole
File hash → EDR blocking rules
YARA rules → SIEM / AV
Tactical indicators are easy to use but short-lived. Changing an IP address or recompiling malware to get a new hash costs an attacker almost nothing. This idea is known as the Pyramid of Pain: detections based on attacker behaviour (TTPs) are far harder to evade than detections based on indicators. Use IOCs for quick wins, and invest in behavioural detections for lasting coverage.
MITRE ATT&CK Framework
MITRE ATT&CK is a public knowledge base of real-world attacker behaviour. It maps adversary techniques to tactics, and it has become the shared vocabulary for detection, hunting and gap analysis. Tactics are the attacker's goals ("why"); techniques are how they achieve them ("how"), each with an ID such as T1566 for Phishing:
Tactics (columns) → Techniques (rows)
┌─────────────┬──────────────────────────────────┐
│ Tactic │ Example Techniques │
├─────────────┼──────────────────────────────────┤
│ Initial │ Phishing (T1566), Exploit (T1190) │
│ Access │ │
│ Execution │ PowerShell (T1059.001), Cron │
│ Persistence │ Registry Run Key, Cron Job │
│ Priv Escal │ SUID/SGID (T1548.001) │
│ Defense Eva │ Obfuscation, Process Injection │
│ Credential │ Keylogging, /etc/shadow dump │
│ Discovery │ Network Scan, Account Discovery │
│ Lateral Mov │ SSH, SMB/PsExec, Pass-the-Hash │
│ Collection │ Screen Capture, Clipboard │
│ Exfiltration│ Over HTTP/DNS, Compressed │
│ Impact │ Ransomware, Data Destruction │
└─────────────┴──────────────────────────────────┘
Because every vendor, threat report and detection rule references the same IDs, ATT&CK lets you answer concrete questions: "this threat group uses T1059.001; do we detect it?"
ATT&CK Navigator for Gap Analysis
The ATT&CK Navigator is a free web tool for colouring the matrix. SOCs use it to visualise which techniques they can detect and where the blind spots are:
1. Map your current detection coverage to ATT&CK cells
2. Identify uncovered techniques (gaps)
3. Prioritize new detection rules for most-used techniques
4. Track improvement over time
Example: if Lateral Movement via SMB is uncovered →
Write Splunk rule for SMB connections between workstations
Test against attack simulation (Atomic Red Team)
The last step matters most: a rule that has never been tested against the behaviour it claims to detect is an assumption, not coverage. Atomic Red Team provides small, safe test cases mapped to ATT&CK IDs for exactly this purpose.
Threat Hunting
Detection rules only catch what someone thought to write a rule for. Threat hunting starts from the assumption that an attacker may already be inside, undetected, and searches proactively for evidence:
Hunt Process:
1. Hypothesis: "Attacker may have used WMI for persistence"
2. Data: query SIEM for wmiprvse.exe spawning child processes
3. Investigate: is this normal in our environment?
4. Find/Clear: confirm or rule out the threat
Example hunting queries, written in a generic syntax:
# Processes spawned by Office apps (macro execution)
process.parent.name in ("WINWORD.EXE", "EXCEL.EXE")
AND process.name not in ("splwow64.exe")
# DNS queries to newly registered domains (< 30 days old)
dns.question.name in [new_domains_feed]
# Large PowerShell scripts (> 1000 chars in command line)
process.name == "powershell.exe"
AND length(process.args) > 1000
A hunt that finds nothing is not wasted. It either confirms the environment is clean for that hypothesis, or reveals that the data needed to answer it isn't being collected, which is itself a finding. Successful hunts should end as new automated detection rules, so the same question never needs to be asked by hand again.
Metrics and KPIs
SOC performance is measured mainly in time: how long attackers go unnoticed, and how long it takes to stop them once they are:
Mean Time to Detect (MTTD) – how long before we notice?
Mean Time to Respond (MTTR) – how long to contain?
False Positive Rate – % of alerts that are FP
Alert Volume – total daily alerts
Dwell Time – how long attacker was in before detection
Goals:
MTTD < 1 hour for P1
MTTR < 4 hours for P1
FP rate < 20%
Dwell time < 24 hours
Metrics can be gamed. A team judged only on MTTR may close tickets too early; a team judged on alert volume may disable noisy but useful rules. Good SOC managers track several metrics together and review a sample of closed tickets for quality.
Where to Go Next
- Write detections professionally, with testing and version control: Detection Engineering.
- Respond once an alert is confirmed: Incident Response, then Incident Response & Digital Forensics.
- Collect logs on Linux hosts: Linux Networking Tools & Log Management.
- See how auditors view monitoring: CISSP D6 — Security Metrics, Log Review & Continuous Monitoring.
FAQ
What is the difference between a SIEM and an EDR?
An EDR (Endpoint Detection and Response) watches individual machines in depth: processes, files and network connections, with the ability to isolate a host. A SIEM collects logs from everything (endpoints, network, cloud, applications) and correlates them. EDR data is usually one of the most valuable sources a SIEM ingests.
What does a Tier 1 SOC analyst do all day?
Mostly triage: reading alerts, gathering context, deciding whether each one is real, and either closing it with documentation or escalating it. Good Tier 1 analysts also feed back tuning suggestions that reduce noise for the whole team.
What is the difference between a false positive and a benign true positive?
A false positive means the rule fired on activity that doesn't match what it was meant to detect, so the rule needs fixing. A benign true positive means the rule worked correctly, but the activity was authorised, such as a penetration test or an admin script. It's usually handled with a documented exclusion rather than by changing the rule's logic.
Do I need to know a specific SIEM to get a SOC job?
It helps, but the concepts transfer. Employers mostly look for an understanding of logs, attacks and triage. Free options to practise with include Splunk's free training and data sets, and Elastic or Wazuh in a home lab.
Write Splunk SPL (or pseudo-code) detection rules for: (1) brute force: >10 failed logins in 5 min from same IP, (2) brute force success: IP in (1) later has successful login, (3) suspicious PowerShell: -EncodedCommand in process args, (4) data exfiltration: >50MB outbound to single external IP in 1 hour, (5) off-hours admin login: admin login between 10pm-6am. Test each rule against sample log data.
What is a correlation rule?
What is alert fatigue?
Build a Python script that: (1) downloads the AbuseIPDB blocklist (or a CSV of malicious IPs), (2) compares against your server's auth.log for any matches, (3) for each match, enriches with VirusTotal (using their free API tier), (4) outputs: IP, confidence score, number of reports, last seen, matched log lines. This simulates SIEM threat intel enrichment.
What is a threat intelligence feed?
Given this attack sequence: (1) phishing email with malicious macro → (2) PowerShell downloads payload → (3) scheduled task persistence → (4) credential dump from LSASS → (5) SMB lateral movement → (6) data compressed and exfiltrated over HTTPS. Map each step to MITRE ATT&CK tactic and technique ID. Then write one detection rule for each step.
What MITRE ATT&CK tactic does 'scheduled task persistence' belong to?
What ATT&CK technique ID covers credential dumping from LSASS?