Malware Analysis Introduction
Learn malware types, infection chains, and basic static and dynamic analysis techniques — the skills to understand what a suspicious file actually does.
Learning Objectives
- → Classify malware types: virus, worm, trojan, ransomware, rootkit, RAT
- → Describe modern infection chains and persistence mechanisms
- → Perform basic static analysis: strings, file type, hashes, YARA
- → Perform basic dynamic analysis: process monitoring, network capture, sandbox
- → Safely handle malware samples in isolated environments
Malware Types
| Type | How it spreads | What it does |
|---|---|---|
| Virus | Infects files | Self-replicates, payload |
| Worm | Network, no host file | Self-propagates automatically |
| Trojan | Disguised as legit | Backdoor, data theft |
| Ransomware | Email, exploit | Encrypts files, demands ransom |
| Rootkit | Exploit, dropper | Hides presence, persists |
| RAT | Phishing, exploit | Remote Access Trojan — full control |
| Keylogger | Bundled, exploit | Records keystrokes |
| Spyware | Drive-by, bundled | Surveillance, credential theft |
| Botnet agent | Worm, exploit | DDoS, spam, proxy |
| Cryptominer | Drive-by, supply chain | Uses CPU/GPU for mining |
Modern Infection Chain
1. Initial Access
Phishing email → malicious attachment (macro, PDF exploit)
Drive-by download → browser/plugin vulnerability
Supply chain → compromised package/update
2. Execution
User opens doc → macro runs PowerShell
Exploit triggers → shellcode executes
3. Persistence
Registry: HKCU\Software\Microsoft\Windows\CurrentVersion\Run
Scheduled task, startup folder, service installation
Cron job (/etc/cron.d/backdoor), ~/.bashrc, systemd unit
4. Defense Evasion
Process injection, DLL hijacking, AMSI bypass
Encrypted payloads, packing (UPX), code obfuscation
5. C2 Communication
HTTPS beacon to attacker server
DNS tunneling, domain fronting
6. Actions on Objectives
Data exfiltration, ransomware encryption, lateral movement
Safe Malware Handling
NEVER run malware on a host machine.
Setup:
1. Isolated VM (no shared folders, no network or host-only network)
2. Snapshot BEFORE analysis — restore after
3. Disable VM guest additions (clipboard sharing is a risk)
4. Use a dedicated analysis OS: REMnux, FlareVM, or Cuckoo Sandbox
File handling:
- Keep samples in password-protected ZIP (password: "infected")
- Use sha256sum to verify you have the right sample
- Submit hashes to VirusTotal — not the file itself if sensitive
Static Analysis
Static analysis examines the file WITHOUT executing it.
File Identification
# File type (don't trust extension)
file suspicious.bin
# → suspicious.bin: PE32 executable (GUI) Intel 80386
# Hash the sample
md5sum suspicious.bin
sha256sum suspicious.bin
# → search on VirusTotal
# Check if packed (UPX)
upx -t suspicious.bin
strings suspicious.bin | head -20
# Very few strings + "UPX0" section = likely packed
Strings Analysis
# Extract printable strings
strings suspicious.bin | grep -E "(http|ftp|cmd|powershell|reg|\\)"
# Common IOCs in strings:
# - URLs / IPs (C2 servers)
# - Registry keys (persistence)
# - File paths
# - Windows API calls (CreateRemoteThread, WriteProcessMemory)
# - Encoded data (base64)
strings suspicious.bin | grep -E "[A-Za-z0-9+/]{40,}={0,2}" # base64
strings suspicious.bin | grep -E "^[0-9]{1,3}\.[0-9]{1,3}" # IPs
PE Header Analysis
# Install: pip install pefile
python3 -c "
import pefile
pe = pefile.PE('suspicious.bin')
print('Sections:', [s.Name.decode().strip() for s in pe.sections])
print('Imports:')
for imp in pe.DIRECTORY_ENTRY_IMPORT:
print(' ', imp.dll.decode())
for fn in imp.imports[:3]:
print(' ', fn.name)
print('Timestamp:', pe.FILE_HEADER.TimeDateStamp)
"
Suspicious imports:
- CreateRemoteThread, VirtualAllocEx → process injection
- WriteProcessMemory → code injection
- RegSetValueEx → registry persistence
- WinSock + connect → network communication
- CryptEncrypt → ransomware
YARA Rules
YARA matches patterns in files:
rule RansomwareDetect {
meta:
description = "Detects common ransomware strings"
author = "CyberLearn"
strings:
$ransom1 = "Your files have been encrypted" nocase
$ransom2 = "Bitcoin" nocase
$ransom3 = "decrypt" nocase
$ext1 = ".locked"
$ext2 = ".encrypted"
$api1 = "CryptEncrypt"
condition:
(2 of ($ransom*)) and (1 of ($ext*)) or $api1
}
# Run YARA
yara ransomware.yar suspicious.bin
yara -r suspicious_rules/ /samples/ # scan directory
Dynamic Analysis
Dynamic analysis observes the malware WHILE it runs (in a sandbox).
Process Monitoring
# Linux: strace — trace system calls
strace -e trace=open,write,connect ./suspicious
# Linux: monitor new processes
watch -n 1 "ps aux | grep -v grep | grep -v watch"
# Check new files created
inotifywait -m -r /tmp /etc /home &
# Then run the sample
Network Analysis
# Capture all traffic during execution
tcpdump -i any -w malware_traffic.pcap &
# Run sample...
# Kill tcpdump, analyze with Wireshark
# Look for:
# - DNS queries to unknown domains (DGA domains, C2)
# - HTTP/HTTPS beacons (regular intervals = C2 checkin)
# - Large outbound data transfers (exfiltration)
# - IRC or non-standard protocol on unusual ports
Automated Sandbox
Submit to:
- any.run (interactive online sandbox)
- hybrid-analysis.com
- VirusTotal (file scan + behavior)
- Cuckoo Sandbox (self-hosted)
Report includes:
- Process tree
- File system changes
- Registry changes
- Network connections
- Screenshots
- Extracted strings and IOCs
Indicators of Compromise (IOCs)
File IOCs:
- SHA-256 hash of malware binary
- File names and paths created
Network IOCs:
- C2 IP addresses and domains
- User-Agent strings
- URI patterns
Host IOCs:
- Registry keys created
- Scheduled tasks added
- Services installed
- Mutexes (prevent double-execution)
# Extract IOCs from strings output
import re
def extract_iocs(strings_output: str) -> dict:
return {
"ips": re.findall(r'(?:\d{1,3}\.){3}\d{1,3}', strings_output),
"urls": re.findall(r'https?://[^\s"'<>]+', strings_output),
"domains": re.findall(r'[a-zA-Z0-9\-]+\.[a-zA-Z]{2,}', strings_output),
"emails": re.findall(r'[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}', strings_output),
"registry": re.findall(r'HKEY_[A-Z_\]+', strings_output),
}
Download a safe malware sample from MalwareBazaar (or use provided sample). Perform: (1) file type identification with file, (2) SHA-256 hash + VirusTotal lookup, (3) strings extraction — list top 10 suspicious strings, (4) base64-encoded strings — decode them, (5) pefile analysis — list imported DLLs and suspicious APIs, (6) write a YARA rule that detects this sample.
Why should you hash a sample and search VirusTotal before deeper analysis?
What does a very high entropy section in a PE file suggest?
In a VM: (1) take a snapshot, (2) run a suspicious script while capturing with tcpdump and inotifywait, (3) document: new files created, registry-equivalent changes (/etc/cron.d, ~/.bashrc), network connections made, (4) extract C2 IP from pcap, (5) restore snapshot. Document all IOCs found.
What is a sandbox in malware analysis?
Write YARA rules to detect: (1) ransomware based on encryption API imports + ransom note strings, (2) a reverse shell based on /bin/sh string + network connection setup, (3) credential harvesting based on common browser profile paths + network upload. Test each rule against sample files.
What is a YARA rule condition?