File Management

2.2. File Management#

You learned the basics in Chapter 1. Now let’s explore advanced techniques for managing files in bulk, working with complex directory structures, and using powerful tools to organize your filesystem.

2.2.1. Bulk File Operations#

When you need to work with many files, repeating commands on each one is inefficient. Learn to leverage shell features for bulk operations.

2.2.1.1. Glob Patterns (Wildcards)#

Glob patterns match multiple files:

# Match all files with .txt extension
$ ls *.txt

# Match files starting with "test"
$ rm test*

# Match files with any single character
$ ls file?.txt        # matches file1.txt, fileA.txt, etc.

# Match files with character ranges
$ ls [a-c]*.txt       # matches a*.txt, b*.txt, c*.txt

# Match files with multiple patterns
$ ls *.{txt,md,py}    # matches all .txt, .md, and .py files

2.2.1.2. Copy Multiple Files#

# Copy all text files to backup directory
$ cp *.txt ~/backup/

# Copy with pattern and create directories
$ cp -r src/{module1,module2,module3} dest/

# Copy preserving attributes
$ cp -p important_file.txt backup_copy.txt

2.2.1.3. Rename Multiple Files#

# Using mv with patterns
$ for file in *.txt; do mv "$file" "${file%.txt}.md"; done

# More powerful: use rename command
$ rename 's/\.txt$/.md/' *.txt       # Rename all .txt to .md

# Batch rename with numbering
$ for i in *.jpg; do mv "$i" "photo_$(date +%s).jpg"; done

2.2.1.4. Delete Multiple Files Safely#

# Preview what would be deleted
$ ls *.tmp

# Delete them
$ rm *.tmp

# Delete everything older than 30 days (safer)
$ find . -type f -mtime +30 -name "*.tmp" -delete

# Delete with confirmation
$ rm -i *.tmp            # Ask for each file
$ rm -I *.tmp            # Ask once if more than 3 files

2.2.1.5. Viewing Multiple Files#

# Concatenate multiple files
$ cat chapter1.txt chapter2.txt chapter3.txt > book.txt

# View with line numbers
$ cat -n document.txt

# Show file sizes of multiple files
$ wc -l *.py             # Count lines in all Python files
$ du -sh *               # Size of each item in current directory

2.2.2. Building Complex Directory Structures#

Create nested project structures efficiently:

# Create multiple levels at once
$ mkdir -p project/{src,docs,tests,build}

# Create a template structure
$ mkdir -p ~/new_project/{src/{lib,utils},docs,tests,scripts,data}

# Verify structure with tree
$ tree ~/new_project

# Or with ls -R
$ ls -R ~/new_project

2.2.2.1. Project Initialization Script#

Create a script to set up project directories consistently:

# Create new_project.sh
$ cat > ~/bin/new_project.sh << 'EOF'
#!/bin/bash
PROJECT_NAME=$1
mkdir -p "$PROJECT_NAME"/{src,docs,tests,build,data}
cd "$PROJECT_NAME"
git init
touch README.md
echo "Project $PROJECT_NAME created"
EOF

$ chmod +x ~/bin/new_project.sh
$ new_project.sh my_app

2.2.3. Synchronizing and Backup#

For more reliable file operations, use specialized tools:

2.2.3.1. rsync — Efficient Copying and Backup#

Much more powerful than cp for large datasets:

# Copy with progress
$ rsync -av ~/Documents/ ~/backup/Documents/

# Dry run (preview what would be copied)
$ rsync -avn ~/Documents/ ~/backup/Documents/

# Sync (delete files in destination not in source)
$ rsync -av --delete ~/Documents/ ~/backup/Documents/

# Copy over SSH to remote server
$ rsync -avz ~/Documents/ user@remote:/backup/

2.2.3.2. tar — Create Backups#

Combine multiple files into one archive:

# Create archive
$ tar -czf backup.tar.gz ~/Documents/

# List contents
$ tar -tzf backup.tar.gz

# Extract
$ tar -xzf backup.tar.gz

2.2.4. Comparing Files and Directories#

See what’s different:

# Compare two files
$ diff file1.txt file2.txt

# Compare directories
$ diff -r dir1/ dir2/

# Side-by-side comparison
$ diff -y file1.txt file2.txt

# Show only differences (brief)
$ diff -q file1.txt file2.txt
Contents