Linux Command Line Mastery

Pipes, redirection, text processing with grep/awk/sed, and building powerful one-liners.

Easy 45m 3 tasks

Prerequisites: The Linux Command Line

Learning Objectives

→ Chain commands with pipes and use stdin/stdout/stderr redirection
→ Extract and transform text with grep, awk, sed, and cut
→ Manage processes with ps, kill, and top
→ Build forensic one-liners for log analysis

The Power of the CLI

Pipes and Redirection

# Pipe: pass output of one command to another
ls -la | grep ".conf"
cat /etc/passwd | grep bash
ps aux | sort -k3 -rn | head -10   # top CPU processes

# Redirect stdout to file
echo "hello" > file.txt        # overwrite
echo "world" >> file.txt       # append

# Redirect stderr
find / -name "*.log" 2>/dev/null    # discard errors
find / -name "*.log" 2>&1 | less   # errors to stdout

# Redirect stdin
sort < names.txt

grep — Search Text

grep "error" /var/log/syslog           # basic search
grep -i "error" logfile                # case-insensitive
grep -r "password" /etc/               # recursive
grep -v "^#" /etc/ssh/sshd_config     # invert (exclude comments)
grep -n "failed" auth.log              # show line numbers
grep -E "fail|error|warn" syslog       # extended regex
grep -c "GET" access.log               # count matches

cut & awk — Field Extraction

# cut: extract columns
cat /etc/passwd | cut -d: -f1          # usernames only
cat /etc/passwd | cut -d: -f1,3        # username and UID

# awk: powerful field processing
awk -F: '{print $1}' /etc/passwd        # same as cut above
awk -F: '$3 > 1000 {print $1}' /etc/passwd   # users with UID > 1000
awk '{print NR, $0}' file.txt           # add line numbers
ps aux | awk '{print $1, $2, $11}'      # user, PID, command

# Sum a column
ps aux | awk '{sum += $4} END {print "Total CPU:", sum"%"}'

sed — Stream Editor

sed 's/old/new/' file.txt              # replace first occurrence per line
sed 's/old/new/g' file.txt            # replace all occurrences
sed '/^#/d' config.txt                # delete comment lines
sed -n '10,20p' file.txt              # print lines 10-20
sed -i 's/foo/bar/g' file.txt         # edit file in-place

Useful One-Liners

# Count unique IPs in access log
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

# Find files modified in last 24h
find /var/log -newer /var/log -type f

# Extract emails from a file
grep -oE '[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}' file.txt

# Show only non-empty, non-comment lines
grep -v '^#' /etc/ssh/sshd_config | grep -v '^$'

# Disk usage sorted
du -sh /var/log/* 2>/dev/null | sort -h | tail -10

The Unix pipe (|) connects the stdout of one command to the stdin of the next. This is the superpower of the Linux CLI.

Pipeline — data flows through each command

Redirection operators

Operator	Redirects	Example
>	stdout → file (overwrite)	ls > files.txt
>>	stdout → file (append)	echo 'new' >> log.txt
<	file → stdin	wc -l < /etc/passwd
2>	stderr → file	find / -name '*.key' 2>/dev/null
2>&1	stderr → same as stdout	cmd > out.txt 2>&1
\|	stdout → next stdin	ps aux \| grep ssh
tee	stdout → file AND next cmd	cmd \| tee output.txt \| wc -l

# Real-world one-liners cat /var/log/auth.log | grep 'Failed' | awk '{print $11}' | sort | uniq -c | sort -rn | head -10 # → Top 10 IPs with failed SSH logins ss -tlnp | grep LISTEN | awk '{print $4,$6}' # → All listening ports with process names ps aux | sort -k3 -rn | head -5 # → Top 5 CPU consumers

✦ Answer the questions to complete this task

What does 2>/dev/null do?

The tee command:

Three tools that every security professional uses daily for log parsing, config auditing, and data extraction.

grep — search by pattern

grep 'sshd' /var/log/auth.log # simple string grep -i 'root' /etc/passwd # case-insensitive grep -r 'api_key' /var/www/ 2>/dev/null # recursive grep -E '\b([0-9]{1,3}\.){3}[0-9]{1,3}\b' access.log # regex IPs grep -v '#' /etc/ssh/sshd_config # exclude comments

awk — extract and process columns

# Print specific columns from /etc/passwd awk -F: '{print $1,$3}' /etc/passwd # username uid # Show users with UID=0 (root-equivalent) awk -F: '$3==0 {print $1}' /etc/passwd # Sum bytes from access log (column 10) awk '{sum+=$10} END {print sum}' access.log

sed — stream editor, find and replace

# Replace http with https sed 's/http:/https:/g' urls.txt # Delete blank lines sed '/^$/d' config.txt # In-place edit (backup first!) sed -i.bak 's/old/new/g' file.txt # Print lines 10-20 of a large file sed -n '10,20p' hugefile.log

✦ Answer the questions to complete this task

Which awk field separator flag lets you parse /etc/passwd (colon-delimited)?

What does sed 's/old/new/g' do?

Processes are running instances of programs. Managing them — finding suspicious ones, killing malware, understanding signals — is a daily operation.

Command	Purpose	Security use
ps aux	List all processes with user and CPU	Spot unknown processes running as root
top / htop	Real-time process monitor	See crypto-miners consuming CPU
pgrep sshd	Find PID by name	Verify SSH daemon is running
kill -9	Force-kill a process (SIGKILL)	Terminate malware process
lsof -p	Files opened by a process	See what a suspicious process is touching
strace -p	Trace syscalls of a process	Understand what malware is doing
netstat -anp	Network connections with PIDs	Identify process behind suspicious connection

⚠ Security: Malware often masquerades as kworker, apache2, or other legitimate names. Always verify with lsof and strace.

✦ Answer the questions to complete this task

What signal does kill -9 send?

Which file under /proc/<pid>/ shows all open file descriptors for a process?

💪 Exercises & Challenges

⚙️ Practical Easy +25 XP

Log Analysis One-Liners

## Task: Analyse System Logs ```bash # 1. View recent auth events sudo tail -50 /var/log/auth.log # 2. Find failed login attempts sudo grep "Failed password" /var/log/auth.log | head -20 # 3. Count

Start →

🚩 Challenge Easy +50 XP

Extract the Flag

Run this command to create a test file: ```bash printf 'user1:x:1001:1001\nFLAG{cli_master}:x:0:0\nuser2:x:1002:1002\n' > /tmp/flagfile.txt ``` Extract only the second field (flag) from the line wher

Start →

🔗 Related Lessons

Know this first

The Linux Command Line bash fundamentals

Leads to

Users, Groups & Permissions Linux filesystem

Users, Groups & Permissions