SSH and Remote Administration¶

Overview¶

SSH (Secure Shell) is the standard transport for remote Linux administration, file transfer, Git operations, and tunneling application ports. Password authentication is brittle and brute-forceable; key-based authentication with a hardened sshd_config is the production baseline.

This tutorial covers generating Ed25519 keys, organizing multi-host workflows with ~/.ssh/config, transferring files with scp and sftp, hardening the SSH server, and understanding why agent forwarding is powerful but dangerous.

This is Module 5, Tutorial 11 in the REBASH Academy Linux series.

Prerequisites¶

• Shell Scripting Fundamentals — environment variables and file permissions
• User and Group Management — users, home directories, sudo
• Two Linux systems (or one system connecting to localhost) for lab exercises
• OpenSSH client and server (openssh-client, openssh-server)

Learning Objectives¶

By the end of this tutorial, you will be able to:

• Generate and deploy Ed25519 SSH key pairs with correct file permissions
• Configure ~/.ssh/config for host aliases, identity files, and jump hosts
• Transfer files securely with scp and interactive sftp
• Harden sshd_config (disable root login, disable passwords, limit users)
• Diagnose connection failures with ssh -v and log files
• Explain the security trade-offs of SSH agent forwarding

Architecture Diagram¶

flowchart LR
    subgraph Client
        A[ssh client]
        B["~/.ssh/config"]
        C[Private Key]
        D[ssh-agent]
    end
    subgraph Network
        E["Port 22 / TCP"]
    end
    subgraph Server
        F[sshd]
        G[authorized_keys]
        H["Shell / sftp-server"]
    end
    A --> B
    A --> C
    C -.-> D
    A -->|encrypted session| E
    E --> F
    F --> G
    F --> H```

## Theory

### How SSH authentication works

1. Client initiates TCP connection to port 22 (or custom port).
2. Server presents host key; client verifies against `~/.ssh/known_hosts`.
3. Client and server negotiate encryption (modern defaults: chacha20-poly1305, AES-GCM).
4. Client authenticates via:
   - **Public key** — server checks `~/.ssh/authorized_keys` on the server
   - **Password** — discouraged in production
   - **Keyboard-interactive / MFA** — often layered on top of keys

The private key never leaves the client. Only the public key is copied to the server.

### Key types and generation

| Algorithm | Key size | Recommendation |
|-----------|----------|----------------|
| Ed25519 | 256-bit | **Preferred** — fast, secure, short keys |
| RSA | 4096-bit | Legacy compatibility |
| ECDSA | 256-bit | Acceptable; prefer Ed25519 for new keys |

Never reuse personal keys for production servers. Use separate keys per environment (lab, staging, prod) with descriptive comments.

### File permissions — non-negotiable

SSH refuses keys with loose permissions:

| Path | Permission |
|------|------------|
| `~/.ssh/` | `700` |
| `~/.ssh/id_ed25519` (private) | `600` |
| `~/.ssh/id_ed25519.pub` (public) | `644` |
| `~/.ssh/config` | `600` |
| `~/.ssh/authorized_keys` | `600` |

### Client configuration (`~/.ssh/config`)

Host blocks simplify daily workflows:

Supports **ProxyJump** for bastion access: `ProxyJump bastion.example.com`.

### scp vs sftp

Both run over SSH:

- **scp** — fast one-off copies; syntax mirrors `cp`. Being deprecated in favor of `sftp` in some OpenSSH versions; use `scp -O` for legacy protocol if needed.
- **sftp** — interactive session, resumable transfers, scripting with `sftp -b batchfile`.

For automation, consider `rsync -avz -e ssh` for incremental sync.

### Server hardening essentials

Production `sshd_config` baseline:

After changes: `sudo sshd -t && sudo systemctl reload sshd` (or `sshd` on RHEL).

### SSH agent and agent forwarding

**ssh-agent** holds decrypted private keys in memory so you enter the passphrase once.

**Agent forwarding** (`ssh -A` or `ForwardAgent yes`) lets a remote server use your local agent to authenticate further SSH hops (e.g., laptop → bastion → internal server).

!!! danger "Agent forwarding warning"
    If an attacker gains code execution on the intermediate server, they can use your forwarded agent to authenticate as **you** to any server that trusts your key — for the lifetime of the connection. Only enable forwarding to hosts you fully trust, prefer `ProxyJump` over `-A`, and use `AllowAgentForwarding no` on bastions when possible. Consider `ProxyCommand` with `ssh -W` instead.

## Hands-on Lab

### Step 1 – Generate an Ed25519 key pair

```bash
mkdir -p ~/.ssh && chmod 700 ~/.ssh
ssh-keygen -t ed25519 -C "rebash-lab-$(whoami)@$(hostname -s)" \
  -f ~/.ssh/rebash_ed25519 -N ""
ls -la ~/.ssh/rebash_ed25519*

Expected output: -rw------- private key and -rw-r--r-- public key.

View the public key:

cat ~/.ssh/rebash_ed25519.pub

Expected output: single line starting with ssh-ed25519 AAAA....

Step 2 – Install public key for local login¶

cat ~/.ssh/rebash_ed25519.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Test key-based login to localhost:

ssh -i ~/.ssh/rebash_ed25519 -o BatchMode=yes localhost 'echo SSH key auth OK'

Expected output: SSH key auth OK

Step 3 – Configure SSH client¶

cat >> ~/.ssh/config << 'EOF'

Host lab-local
    HostName localhost
    User YOUR_USER
    IdentityFile ~/.ssh/rebash_ed25519
    StrictHostKeyChecking accept-new
EOF
chmod 600 ~/.ssh/config

Replace YOUR_USER with your username, then:

ssh lab-local 'hostname && whoami'

Expected output: localhost hostname and your username.

Step 4 – Verbose debugging¶

Simulate a failed connection to see diagnostics:

ssh -vvv -i ~/.ssh/nonexistent_key localhost 2>&1 | tail -15

Expected output: debug lines showing key load failure and authentication methods tried.

Successful verbose tail:

ssh -v lab-local true 2>&1 | grep -E 'Authenticated|Offering'

Expected output: lines mentioning Offering public key and Authenticated.

Step 5 – scp file transfer¶

echo "deploy artifact v1" > /tmp/rebash_deploy.txt
scp -i ~/.ssh/rebash_ed25519 /tmp/rebash_deploy.txt lab-local:/tmp/
ssh lab-local 'cat /tmp/rebash_deploy.txt'

Expected output: deploy artifact v1 printed from remote /tmp/.

Copy directory recursively:

mkdir -p /tmp/rebash_dir && echo "nested" > /tmp/rebash_dir/nested.txt
scp -r -i ~/.ssh/rebash_ed25519 /tmp/rebash_dir lab-local:/tmp/
ssh lab-local 'find /tmp/rebash_dir -type f'

Expected output: /tmp/rebash_dir/nested.txt

Step 6 – sftp interactive session¶

sftp -i ~/.ssh/rebash_ed25519 lab-local << 'EOF'
pwd
ls /tmp/rebash_deploy.txt
get /tmp/rebash_deploy.txt /tmp/downloaded_deploy.txt
bye
EOF
cat /tmp/downloaded_deploy.txt

Expected output: remote working directory, file listing, then deploy artifact v1 from downloaded copy.

Step 7 – Inspect and harden sshd (requires sudo)¶

Review current effective settings:

sudo sshd -T | grep -E 'permitrootlogin|passwordauthentication|pubkeyauthentication|allowusers'

Expected output: current server values (varies by distro).

Recommended hardening snippet (review before applying):

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak.$(date +%F)
sudo sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sshd -t && sudo systemctl reload ssh 2>/dev/null || sudo systemctl reload sshd

Verify syntax test passes (no output = success).

Step 8 – Agent forwarding demonstration and risk¶

Start agent and add key:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/rebash_ed25519 2>/dev/null
ssh-add -l

Expected output: key fingerprint listed.

Connect with forwarding and check remote socket:

ssh -A lab-local 'echo AGENT=$SSH_AUTH_SOCK; ssh-add -l 2>&1 | head -1'

Expected output: non-empty AGENT= path and your key fingerprint visible on the remote side — demonstrating why a compromised remote host can abuse your agent.

Disable forwarding in config for untrusted hosts:

Host untrusted-*
    ForwardAgent no
    IdentityFile ~/.ssh/rebash_ed25519

Commands¶

Code¶

SSH config template for multi-environment ops¶

# ~/.ssh/config

Host *
    AddKeysToAgent yes
    IdentitiesOnly yes
    ServerAliveInterval 60

Host bastion
    HostName bastion.example.com
    User jump
    IdentityFile ~/.ssh/corp_ed25519

Host prod-*
    User deploy
    IdentityFile ~/.ssh/prod_ed25519
    ProxyJump bastion
    ForwardAgent no

Host staging-*
    User deploy
    IdentityFile ~/.ssh/staging_ed25519
    ForwardAgent no

Batch sftp upload script¶

#!/bin/bash
set -euo pipefail

HOST="${1:?usage: sftp_upload.sh HOST LOCAL_FILE REMOTE_PATH}"
LOCAL="${2:?}"
REMOTE="${3:?}"

sftp -b - "$HOST" <<EOF
put $LOCAL $REMOTE
ls -l $REMOTE
EOF

Common Mistakes¶

Best Practices¶

Troubleshooting¶

Summary¶

• SSH encrypts remote administration and file transfer; Ed25519 key pairs replace password auth in production.
• Correct permissions on ~/.ssh and keys are enforced by OpenSSH — not optional.
• ~/.ssh/config scales multi-host workflows with aliases, identity files, and ProxyJump for bastions.
• scp and sftp move files over the same encrypted channel; choose based on interactive vs scripted needs.
• Harden sshd: no root login, no passwords, limit AllowUsers, test with sshd -t before reload.
• Agent forwarding (-A) is convenient but dangerous on untrusted hosts — an attacker can pivot using your credentials.

Interview Questions¶

1. Why is Ed25519 preferred over RSA 2048 for new SSH keys?
2. What permissions must ~/.ssh/authorized_keys and the private key file have?
3. Explain the difference between scp and sftp. When would you use each?
4. What does PermitRootLogin no accomplish, and what is the safer alternative for admin tasks?
5. What is ProxyJump, and how does it differ from agent forwarding?
6. Why is agent forwarding considered a security risk? Describe an attack scenario.
7. How would you debug a Permission denied (publickey) error?
8. What happens when a server's host key changes and you reconnect?
9. What is the purpose of IdentitiesOnly yes in SSH config?
10. How do you validate sshd_config changes before applying them to avoid lockout?

Related Tutorials¶

• Linux – Category Overview
• Shell Scripting Fundamentals (previous)
• Remote systemd Service Control (next)
• Linux Security Hardening Basics
• Learning Paths – DevOps Engineer

References¶

• OpenSSH manual pages
• OpenSSH sshd_config documentation
• NIST SP 800-63B — authentication guidance
• Mozilla OpenSSH security guidelines
• REBASH Academy – Linux Overview