Remote Execution with SSH

13.3. Remote Execution with SSH#

13.3.1. Common Pitfalls#

1. Hardcoding passwords or credentials

# Bad: Password in script (security risk)
sshpass -p 'password' ssh user@remote 'ls'

# Good: Use SSH keys
ssh -i ~/.ssh/key user@remote 'ls'

2. Not handling remote command failures

# Bad: Continues even if remote command fails
ssh user@remote 'systemctl restart nginx'
echo "Restart complete"

# Good: Check exit code
if ssh user@remote 'systemctl restart nginx'; then
  echo "✓ Restart successful"
else
  echo "✗ Restart failed" >&2
  exit 1
fi

3. Local command expansion breaking remote execution

# Bad: Variables expanded locally, not remotely
ssh user@remote 'echo $HOSTNAME'  # Expands $HOSTNAME locally

# Good: Escape or use single quotes
ssh user@remote 'echo $HOSTNAME'  # Expands remotely
ssh user@remote "echo \$HOSTNAME"  # Escapes variable

4. Timeout issues with slow operations

# Bad: SSH times out during long operation
ssh user@remote 'long_running_task'

# Better: Use timeout and nohup
ssh -o ConnectTimeout=10 user@remote 'nohup long_task > /tmp/output.log 2>&1 &'

# Check result later:
ssh user@remote 'cat /tmp/output.log'

5. Not using SSH multiplexing for efficiency

# Bad: Each command creates new SSH connection
for i in {1..10}; do
  ssh user@remote 'echo "task $i"'
done
# Creates 10 connections

# Good: Reuse connection
cat > ~/.ssh/config << 'EOF'
Host *
  ControlMaster auto
  ControlPath ~/.ssh/control-%C
  ControlPersist 600
EOF

# Now all commands use same connection

13.3.2. SSH Tunneling and Advanced Features#

13.3.2.1. Port Forwarding and Tunneling#

#!/bin/bash

# Local port forward: access remote service locally
ssh -L 3306:localhost:3306 user@remote.com -N
# Local port 3306 → remote localhost:3306
# -N: don't execute command (just tunnel)

# Background tunnel
ssh -L 5432:db-server:5432 user@gateway.com -N &
PID=$!
# Use local connection
psql -h localhost -U postgres
# Later: kill $PID

# Remote port forward: expose local service remotely
ssh -R 8080:localhost:8080 user@remote.com -N
# Remote can access http://remote.com:8080 → local:8080

# SOCKS proxy (dynamic port forward)
ssh -D 1080 user@jump-host.com -N
# Use remote as SOCKS proxy:
curl --socks5 localhost:1080 https://internal.example.com

# Reverse tunnel (for home system behind NAT)
ssh -R 2222:localhost:22 user@public-server.com -N
# From public-server: ssh -p 2222 localhost connects to home system

13.3.2.2. Agent Forwarding and Jump Hosts#

#!/bin/bash

# SSH agent forwarding (use local keys on remote)
ssh -A user@jump-host.com
# Now can ssh from jump-host using your local keys

# Jump host (bastion) configuration
ssh -J user@bastion.com user@internal-server.com

# Multiple hops
ssh -J user1@bastion1.com,user2@internal-bastion.com user@target-server.com

# In SSH config
cat > ~/.ssh/config << 'EOF'
Host internal-server
  ProxyJump bastion.com
  User deploy
  HostName 10.0.1.50
EOF

# Usage:
ssh internal-server 'uptime'

13.3.3. SSH Keys and Passwordless Authentication#

SSH keys enable automated, secure authentication without passwords.

13.3.3.1. Generate and Configure SSH Keys#

#!/bin/bash

# Generate SSH key pair
ssh-keygen -t rsa -b 4096 -f ~/.ssh/backup_key -N ""
# -t rsa: key type
# -b 4096: key size
# -f: output file
# -N "": no passphrase

# Generate with passphrase (more secure)
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -C "my@email.com"

# Copy public key to remote host
ssh-copy-id -i ~/.ssh/backup_key.pub user@remote.com

# Manual key installation
cat ~/.ssh/backup_key.pub | ssh user@remote.com 'mkdir -p .ssh && cat >> .ssh/authorized_keys'

# Restrict key permissions (required for security)
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 600 ~/.ssh/authorized_keys

13.3.3.2. SSH Configuration for Automation#

#!/bin/bash

# Create SSH config file for easier connections
cat > ~/.ssh/config << 'EOF'
Host backup-server
  HostName backup.example.com
  User backup
  IdentityFile ~/.ssh/backup_key
  Port 2222
  ConnectTimeout 5

Host prod-*
  HostName %h.example.com
  User deploy
  IdentityFile ~/.ssh/deploy_key
  StrictHostKeyChecking no
  UserKnownHostsFile=/dev/null
EOF

# Usage:
ssh backup-server 'df -h'
ssh prod-web1 'systemctl status nginx'

# Automated deployment with keys
deploy_to_server() {
  local server=$1
  local app_path=$2
  
  ssh "$server" "cd $app_path && git pull && npm install && pm2 restart app"
}

deploy_to_server prod-api /home/deploy/api

13.3.4. Running Commands on Remote Hosts#

SSH allows executing commands on remote systems securely, essential for administration and automation.

13.3.4.1. Basic SSH Command Execution#

#!/bin/bash

# Execute single command
ssh user@remote.com 'ls -la /home'

# Execute command with arguments
ssh user@remote.com 'df -h'

# Multiple commands
ssh user@remote.com 'pwd; whoami; date'

# Command with pipes (use quotes to prevent local interpretation)
ssh user@remote.com 'cat /var/log/syslog | grep error | wc -l'

# Use specific port
ssh -p 2222 user@remote.com 'systemctl status nginx'

# Use specific SSH key
ssh -i ~/.ssh/id_rsa_backup user@remote.com 'uptime'

# Verbose output for debugging
ssh -v user@remote.com 'echo "test"'

# Suppress warnings
ssh -q user@remote.com 'whoami'

13.3.4.2. SSH with Input/Output#

#!/bin/bash

# Capture remote output
remote_uptime=$(ssh user@remote.com 'uptime')
echo "Remote system uptime: $remote_uptime"

# Pipe local data to remote command
cat local_file.txt | ssh user@remote.com 'cat > remote_file.txt'

# Remote to local piping
ssh user@remote.com 'cat /var/log/syslog' | grep 'error' | wc -l

# Execute script stored locally on remote
ssh user@remote.com 'bash -s' < local_script.sh

# Pass arguments to remote script
ssh user@remote.com 'bash -s arg1 arg2' < local_script.sh

# Interactive shell (for interactive programs)
ssh -t user@remote.com 'sudo bash'

13.3.4.3. Practical Command Execution#

#!/bin/bash

# Check multiple servers' disk usage
for server in server1 server2 server3; do
  echo "=== $server ==="
  ssh user@$server 'df -h | grep -v loop'
done

# Execute with error handling
remote_command() {
  local host=$1
  local cmd=$2
  
  if ssh -o ConnectTimeout=5 "user@$host" "$cmd"; then
    echo "✓ Success on $host"
  else
    echo "✗ Failed on $host" >&2
    return 1
  fi
}

remote_command "webserver1" 'systemctl status nginx'

# Parallel execution on multiple hosts
hosts=("server1" "server2" "server3")
for host in "${hosts[@]}"; do
  ssh "user@$host" 'sudo systemctl restart app' &
done
wait
echo "Restart complete on all servers"
Contents