Lab: Bash Variables and Operations

5.5. Lab: Bash Variables and Operations#

This lab combines all concepts from Chapter 5: command syntax, quoting, variables, parameter substitution, and arithmetic. You’ll build progressively from basic scripts to more complex data processing.

5.5.1. Reflection Questions#

  1. When do you need quotes? When does shell treat separate arguments as one?

  2. Parameter substitution vs arithmetic: Why can’t you use $() for arithmetic? What’s the difference between $() and $(())?

  3. Real-world usage: In what real scripts would you need to extract filename components? When would you process CSV-like data?

  4. String vs numbers: How does bash know when to treat something as a number vs. a string? What problems can this cause?

  5. Error handling: Which exercises could fail? How would you detect and handle errors?

5.5.2. Challenge Exercises#

5.5.2.1. Challenge 1: Filename Batch Renaming#

Create a script that renames files with a pattern. Example: convert IMG_001.jpg to photo_001.jpg:

#!/bin/bash

# For all .jpg files
for file in *.jpg; do
    # Extract number from original
    # Rename with new pattern
    # Hint: use parameter substitution and mv command
    
    newname=$(echo "$file" | sed 's/IMG_/photo_/g')
    echo "Would rename: $file -> $newname"
done

5.5.2.2. Challenge 2: System Resource Monitor#

Create a script that monitors disk usage and warns if it exceeds a threshold:

#!/bin/bash

# Get current disk usage
# Parse the percentage
# Compare with threshold
# Alert if exceeded

threshold=80

# Get root filesystem usage (macOS/Linux)
usage=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')

if [ "$usage" -gt "$threshold" ]; then
    echo "⚠ Warning: Disk usage at ${usage}%"
else
    echo "✓ Disk usage at ${usage}%"
fi

5.5.2.3. Challenge 3: Parse and Validate Configuration#

Create a script that reads a simple config file and validates all required fields are present:

# config.txt
hostname=myserver
port=8080
username=admin

# Script should:
# 1. Read the config file
# 2. Extract values (name=value format)
# 3. Validate required fields exist
# 4. Report missing or invalid values

5.5.3. Part 5: Comprehensive Data Processing Script#

5.5.3.1. Exercise 5.1: Build a CSV Processor#

Create process_csv.sh:

#!/bin/bash

# Process a simple CSV file with name,age,city format
csv_file="${1:?CSV file required}"
output_file="${2:-output.txt}"

if [ ! -f "$csv_file" ]; then
    echo "Error: File not found: $csv_file"
    exit 1
fi

# Create header for output
output_header="Processing report for: $csv_file"
echo "$output_header" > "$output_file"
echo "$(printf '=%.0s' $(seq 1 ${#output_header}))" >> "$output_file"
echo "" >> "$output_file"

# Initialize counters
line_count=0
total_age=0
city_counts=()

# Process each line
while IFS=',' read -r name age city; do
    # Skip header or empty lines
    [ -z "$name" ] && continue
    
    # Increment counters
    ((line_count++))
    ((total_age += age))
    
    # Display and log
    entry="$line_count. $name is $age years old from $city"
    echo "$entry"
    echo "$entry" >> "$output_file"
done < "$csv_file"

# Calculate and report statistics
echo "" >> "$output_file"
echo "Statistics:" >> "$output_file"
echo "  Total entries: $line_count" >> "$output_file"

if [ $line_count -gt 0 ]; then
    average_age=$((total_age / line_count))
    echo "  Average age: $average_age years" >> "$output_file"
fi

echo ""
echo "Report written to: $output_file"

Create sample data:

$ cat > data.csv << 'EOF'
name,age,city
Alice,28,Portland
Bob,35,Seattle
Carol,24,Denver
David,40,Boston
EOF

Test it:

$ bash process_csv.sh data.csv
1. Alice is 28 years old from Portland
2. Bob is 35 years old from Seattle
3. Carol is 24 years old from Denver
4. David is 40 years old from Boston

Report written to: output.txt

$ cat output.txt
Processing report for: data.csv
================================

1. Alice is 28 years old from Portland
2. Bob is 35 years old from Seattle
3. Carol is 24 years old from Denver
4. David is 40 years old from Boston

Statistics:
  Total entries: 4
  Average age: 31 years

5.5.4. Part 4: Arithmetic and Calculations#

5.5.4.1. Exercise 4.1: Simple Calculations#

Create math.sh:

#!/bin/bash

# Get numbers from arguments
num1="${1:?First number required}"
num2="${2:?Second number required}"

# Validate they are numbers (simple check)
if ! [[ "$num1" =~ ^[0-9]+$ ]] || ! [[ "$num2" =~ ^[0-9]+$ ]]; then
    echo "Error: Both arguments must be numbers"
    exit 1
fi

# Perform calculations
sum=$((num1 + num2))
diff=$((num1 - num2))
product=$((num1 * num2))
quotient=$((num1 / num2))
remainder=$((num1 % num2))

echo "Calculating with $num1 and $num2:"
echo "  Sum: $sum"
echo "  Difference: $diff"
echo "  Product: $product"
echo "  Quotient: $quotient"
echo "  Remainder: $remainder"

Test it:

$ bash math.sh 20 7
Calculating with 20 and 7:
  Sum: 27
  Difference: 13
  Product: 140
  Quotient: 2
  Remainder: 6

5.5.4.2. Exercise 4.2: Process Monitoring#

Create cpu_load.sh:

#!/bin/bash

# Simulate getting system metrics
# (In real scripts, you'd read from /proc or use system commands)

# Get approximate CPU load (simple example)
seconds_elapsed=0
start_time=$SECONDS

echo "Monitoring for 5 seconds..."

while [ $((SECONDS - start_time)) -lt 5 ]; do
    ((seconds_elapsed++))
    sleep 1
    echo -n "."
done

echo ""
echo "Elapsed: $seconds_elapsed seconds"

# Calculate if within threshold
max_seconds=10
if [ $seconds_elapsed -lt $max_seconds ]; then
    echo "✓ Completed within threshold ($max_seconds seconds)"
else
    echo "⚠ Exceeded threshold ($max_seconds seconds)"
fi

5.5.4.3. Exercise 4.3: Generate Report with Statistics#

Create report.sh:

#!/bin/bash

# Array of sales figures
sales=(1500 2200 1800 2500 3100 2800)

# Calculate statistics
total=0
count=0
for amount in "${sales[@]}"; do
    ((total += amount))
    ((count++))
done

# Calculate average
average=$((total / count))

# Find min/max (simple approach)
min=${sales[0]}
max=${sales[0]}

for amount in "${sales[@]}"; do
    if [ $amount -lt $min ]; then
        min=$amount
    fi
    if [ $amount -gt $max ]; then
        max=$amount
    fi
done

# Generate report
echo "Sales Report"
echo "============"
echo "Total sales: \$$total"
echo "Number of entries: $count"
echo "Average: \$$average"
echo "Min: \$$min"
echo "Max: \$$max"
echo "Range: \$$((max - min))"

Output:

Sales Report
============
Total sales: $14000
Number of entries: 6
Average: $2333
Min: $1500
Max: $3100
Range: $1600

5.5.5. Part 3: String Manipulation and Extraction#

5.5.5.1. Exercise 3.1: Extract File Parts#

Create parse_filename.sh:

#!/bin/bash

# Get filename from argument
filepath="${1:?Filepath required}"

# Extract components
filename="${filepath##*/}"         # Remove everything up to last /
directory="${filepath%/*}"         # Remove everything after last /
extension="${filename##*.}"        # Remove everything up to last .
basename="${filename%.*}"          # Remove extension

echo "Full path: $filepath"
echo "Directory: $directory"
echo "Filename: $filename"
echo "Basename: $basename"
echo "Extension: $extension"

# Example: backup filename
backup="${basename}.backup.${extension}"
echo "Backup name: $backup"

Test it:

$ bash parse_filename.sh /home/alice/documents/report.2025.pdf
Full path: /home/alice/documents/report.2025.pdf
Directory: /home/alice/documents
Filename: report.2025.pdf
Basename: report.2025
Extension: pdf
Backup name: report.2025.backup.pdf

5.5.5.2. Exercise 3.2: Find and Replace#

Create search_replace.sh:

#!/bin/bash

# Original string
text="The quick brown fox jumps over the lazy dog"

echo "Original: $text"

# Replace first occurrence
result1="${text/brown/orange}"
echo "First match: $result1"

# Replace all occurrences
result2="${text//o/O}"
echo "All 'o' -> 'O': $result2"

# Replace at beginning
result3="${text/#The/A}"
echo "Start replacement: $result3"

# Replace at end
result4="${text/%dog/cat}"
echo "End replacement: $result4"

# Count characters before/after
original_length=${#text}
no_spaces="${text// /}"
space_count=$((original_length - ${#no_spaces}))
echo "Number of spaces: $space_count"

Output:

Original: The quick brown fox jumps over the lazy dog
First match: The quick orange fox jumps over the lazy dog
All 'o' -> 'O': The quick brOwn fOx jumps Over the lazy dOg
Start replacement: A quick brown fox jumps over the lazy dog
End replacement: The quick brown fox jumps over the lazy cat
Number of spaces: 8

5.5.5.3. Exercise 3.3: Extract URL Components#

Create parse_url.sh:

#!/bin/bash

url="${1:?URL required}"

# Remove protocol (everything before ://)
host="${url#*://}"

# Extract domain (everything before /)
domain="${host%%/*}"

# Extract path (everything after domain)
path="/${host#*/}"
[ "$path" = "/" ] && path=""

# Extract query parameters (everything after ?)
query="${url##*\?}"
[ "$query" = "$url" ] && query=""

echo "Original: $url"
echo "Domain: $domain"
echo "Path: $path"
echo "Query: $query"

Test it:

$ bash parse_url.sh "https://example.com/users/alice?id=123"
Original: https://example.com/users/alice?id=123
Domain: example.com
Path: /users/alice
Query: id=123

5.5.6. Part 2: Parameter Substitution and Defaults#

5.5.6.1. Exercise 2.1: Default Values#

Create defaults.sh:

#!/bin/bash

# Parameters from command line or defaults
input_file="${1:-input.txt}"
output_file="${2:-output.txt}"
log_file="${3:-.process.log}"

echo "Input: $input_file"
echo "Output: $output_file"
echo "Log: $log_file"

# Create a dummy input file
echo "Sample data" > "$input_file"

# Process (just copy for this example)
cat "$input_file" > "$output_file"

# Log what happened
echo "Processed at $(date)" >> "$log_file"

# Show what was created
echo ""
echo "Files created:"
ls -la "$input_file" "$output_file" "$log_file"

Test it different ways:

# With no arguments (use all defaults)
$ bash defaults.sh

# With one argument
$ bash defaults.sh mydata.txt

# With all arguments
$ bash defaults.sh mydata.txt results.txt debug.log

What to observe:

  • How defaults work when arguments aren’t provided

  • Different default patterns: empty, filename, hidden file

5.5.6.2. Exercise 2.2: Validate with Defaults#

Create validate.sh:

#!/bin/bash

# Get username or error if not provided
username="${1:?Username required}"

# Get password with default 'changeme'
password="${2:-changeme}"

# Get email or generate default
email="${3:-$username@example.com}"

echo "User: $username"
echo "Password length: ${#password}"
echo "Email: $email"

Test it:

# No arguments (error)
$ bash validate.sh
validate.sh: line 5: 1: Username required

# With username only
$ bash validate.sh alice
User: alice
Password length: 8
Email: alice@example.com

# With all arguments
$ bash validate.sh bob mypass bob@company.com
User: bob
Password length: 6
Email: bob@company.com

What to observe:

  • :? causes error if variable is empty

  • := sets a default

  • Combined with other expansions

5.5.7. Part 1: Variable Basics and Quoting#

5.5.7.1. Exercise 1.1: Create and Use Variables#

Create a script called variables.sh:

#!/bin/bash

# Create variables for a user profile
first_name="Alice"
last_name="Johnson"
age=28
city="Portland"

# Display using echo
echo "Name: $first_name $last_name"
echo "Age: $age"
echo "City: $city"

# Combine into full output (quoting matters!)
profile="Name: $first_name $last_name, Age: $age, City: $city"
echo "$profile"

Run it:

$ bash variables.sh
Name: Alice Johnson
Age: 28
City: Portland
Name: Alice Johnson, Age: 28, City: Portland

What to observe:

  • Variables work within double quotes

  • Without quotes, spacing changes

  • Single quotes prevent expansion

5.5.7.2. Exercise 1.2: Quoting in File Operations#

Create file_ops.sh:

#!/bin/bash

# Create a filename with spaces
filename="my document.txt"

# Create the file
touch "$filename"

# List it (watch the quoting!)
ls -la "$filename"

# Without quotes, bash splits the name
# This would create TWO files:
# touch $filename  # Creates "my" and "document.txt"

# Clean up
rm "$filename"

Run it and observe:

  • With "$filename": treats entire string as one filename

  • Without quotes: splits into separate arguments

5.5.7.3. Exercise 1.3: Special Variables#

Create special_vars.sh and save it to some directory:

#!/bin/bash

echo "Script name: $0"
echo "First argument: $1"
echo "All arguments: $@"
echo "Number of arguments: $#"
echo "Exit status of last command: $?"
echo "Process ID: $$"

# Get a random number
echo "Random number: $RANDOM"

Run it with arguments:

$ bash special_vars.sh alice bob charlie
Script name: special_vars.sh
First argument: alice
All arguments: alice bob charlie
Number of arguments: 3
Exit status of last command: 0
Process ID: 12345
Random number: 28472

What to observe:

  • $0 is the script name

  • $1, $2, etc. are arguments

  • $@ expands all arguments as separate values

  • $# counts arguments

  • $? shows exit status

  • $RANDOM generates random numbers

Contents