Incident Response & Digital Forensics

Learn the IR lifecycle — preparation, detection, containment, eradication, recovery, and lessons learned — plus digital forensics techniques for collecting and analyzing evidence.

Hard 70m 3 tasks

Prerequisites: Malware Analysis Introduction

Learning Objectives

→ Apply the NIST IR lifecycle to a real incident
→ Collect volatile evidence in correct order of volatility
→ Acquire and analyze disk images with forensic integrity
→ Investigate log files and timelines to reconstruct an attack
→ Write a professional incident report

The IR Lifecycle (NIST SP 800-61)

1. Preparation
   └── IR plan, playbooks, trained team, tools ready

2. Detection & Analysis
   └── Alert → triage → confirm incident → classify severity

3. Containment
   └── Short-term (isolate host) → Long-term (patch, block)

4. Eradication
   └── Remove malware, close attack vector, audit for persistence

5. Recovery
   └── Restore systems, monitor closely, return to production

6. Post-Incident Activity
   └── Lessons learned, update playbooks, brief stakeholders

Incident Severity Classification

Severity	Description	Response SLA
P1 Critical	Active breach, data exfiltration, ransomware	< 1 hour
P2 High	Compromised account, spreading malware	< 4 hours
P3 Medium	Phishing success, policy violation	< 24 hours
P4 Low	Anomaly, failed attack, suspicious scan	< 72 hours

Step 1: Detection & Triage

# Who triggered the alert?
grep "FAILED\|ACCEPTED\|Invalid" /var/log/auth.log | tail -50

# What processes are running?
ps aux --sort=-%cpu | head -20
ps aux | grep -E "(nc|ncat|python|perl|ruby)" | grep -v grep

# Active network connections
ss -tunp            # all TCP/UDP with process names
netstat -tunp       # alternative

# Who is logged in?
who; w; last | head -20

# Scheduled tasks
crontab -l -u root
ls -la /etc/cron*

# Suspicious files
find /tmp /var/tmp -type f -newer /etc/passwd 2>/dev/null
find / -name "*.sh" -newer /etc/passwd -not -path "/proc/*" 2>/dev/null

Step 2: Order of Volatility (Collect First → Last)

Evidence disappears when power is cut — collect in this order:

1. CPU registers and cache       (lost on process kill)
2. Routing/ARP/process table    (lost on reboot)
3. RAM / memory dump             (lost on shutdown)
4. Temporary files               (may be cleared)
5. Disk image                    (stable — but may be modified)
6. Remote logging / SIEM data    (most stable)
7. Physical media / backup       (most stable)

# Capture memory (Linux — requires LiME module or avml)
avml /media/usb/memory.lime

# Capture RAM with dd (less reliable)
dd if=/proc/mem of=/media/usb/ram.img bs=1M 2>/dev/null

# Capture network state before touching anything
ss -tunpa > /media/usb/network_state.txt
ps auxwf  > /media/usb/process_list.txt
last       > /media/usb/login_history.txt

Step 3: Disk Imaging

# Write-block the disk first (hardware or software)
# Then image with dd or dcfldd
dd if=/dev/sda of=/media/usb/disk.img bs=4M status=progress
# or
dcfldd if=/dev/sda of=/media/usb/disk.img bs=4M hash=sha256 hashlog=/media/usb/hash.log

# Verify integrity
sha256sum /dev/sda
sha256sum /media/usb/disk.img
# Both must match — proves image was not tampered with

# Mount read-only for analysis
mount -o ro,loop /media/usb/disk.img /mnt/evidence

Step 4: Log Analysis

# Authentication logs
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn
# → Count failed logins per IP

# Successful logins after failures (may indicate brute-force success)
grep "Accepted password" /var/log/auth.log

# Web server logs — look for attacks
grep " 404 " /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn
grep -E "(union|select|script|etc/passwd)" /var/log/nginx/access.log -i

# System calls with auditd
ausearch -ts today -k passwd_changes
ausearch -ts today --comm python3

# Journal logs (systemd)
journalctl --since "2025-01-15 10:00" --until "2025-01-15 12:00" -p warning

Timeline Analysis

# Create a unified timeline with mactime (The Sleuth Kit)
fls -r -m "/" /media/usb/disk.img > body.txt
mactime -b body.txt -d 2025-01-15 > timeline_2025-01-15.csv

# Sort events by time and look for attack pattern:
# 10:23 — SSH login from 185.x.x.x
# 10:24 — /tmp/linpeas.sh created
# 10:24 — linpeas.sh executed (x permission added)
# 10:26 — /etc/cron.d/backdoor written
# 10:30 — outbound connection to 192.168.x.x:4444

# Bash history (attacker may have deleted it — check anyway)
cat /home/user/.bash_history
cat /root/.bash_history

# Check for history deletion
grep "history -c\|HISTFILE=/dev/null" /var/log/auth.log

Containment Strategies

# IMMEDIATE: Isolate the host (network level)
# Option A: firewall rule to block all traffic except forensic access
iptables -I INPUT  1 -s YOUR_IR_IP -j ACCEPT
iptables -I OUTPUT 1 -d YOUR_IR_IP -j ACCEPT
iptables -A INPUT  -j DROP
iptables -A OUTPUT -j DROP

# Option B: pull the network cable (simplest, loses volatile data)

# Block attacker IP at perimeter
iptables -I INPUT -s 185.220.101.x -j DROP

# Revoke compromised credentials
passwd -l compromised_user
# Or in Django
User.objects.filter(username='compromised').update(is_active=False)

Eradication

# Remove malware
rm /tmp/backdoor.sh
rm /etc/cron.d/malicious_cron

# Find all attacker persistence
find / -newer /tmp/backdoor.sh -not -path "/proc/*" 2>/dev/null
grep -r "185.220.101" /etc/cron* /var/spool/cron 2>/dev/null
diff /etc/passwd /etc/passwd.backup    # check for added users
diff /etc/sudoers /etc/sudoers.backup

# Change all credentials that may be compromised
# Patch the vulnerability used for initial access
# Reset all SSH host keys if key material was exposed

Chain of Custody

Every piece of evidence needs a chain of custody record:

Evidence Label:
  Case Number: IR-2025-001
  Collected by: Alice Smith (Security Analyst)
  Date/Time: 2025-01-15 11:30 UTC
  Item: Disk image of web-server-01 (/dev/sda)
  SHA-256: 3b4c8f...
  Storage: Encrypted USB drive in locked cabinet

Transfer Record:
  From: Alice Smith → To: Bob Jones (Forensics Lab)
  Date: 2025-01-15 14:00 UTC
  Purpose: Forensic analysis
  Signature: ___________

Incident Report Structure

Executive Summary    (1 page, non-technical)
Timeline of Events   (chronological attack reconstruction)
Technical Analysis   (IOCs, tools used, attack path)
Impact Assessment    (data affected, systems compromised)
Root Cause           (how the attacker got in)
Containment Actions  (what was done immediately)
Eradication Actions  (how it was fully removed)
Recovery Actions     (how systems were restored)
Recommendations      (what to fix to prevent recurrence)
Appendices           (evidence, logs, screenshots)

Given a compromised Linux server (use a deliberately misconfigured VM): (1) run triage commands: who, ps aux, ss -tunp, crontab -l -u root, find /tmp -type f, (2) identify suspicious processes or connections, (3) classify severity (P1-P4), (4) document your initial findings in triage notes with timestamps.

✦ Answer the questions to complete this task

What is the first thing you should do when you suspect an active compromise?

What does 'order of volatility' mean in IR?

Analyze provided auth.log and nginx access.log files to: (1) find IPs with >10 failed SSH logins in 1 hour, (2) identify if any of those IPs later had a successful login, (3) find SQL injection attempts in the web log, (4) build a 5-event timeline from first attack attempt to successful login. Write up findings as an incident timeline.

✦ Answer the questions to complete this task

What log file records SSH authentication events on Ubuntu/Debian?

Based on your triage and log analysis, write a complete incident report with: Executive Summary, Timeline, Technical Analysis (IOCs found), Root Cause, Containment Actions taken, Recommendations to prevent recurrence. Include at least 3 specific technical recommendations with implementation details.

✦ Answer the questions to complete this task

What is a lessons learned session?