Enabling SSH on Ubuntu takes one command. Enabling it so that only the right people get in takes about 10 minutes more, and that's the part most guides skip. Out of the box, sshd accepts passwords from anywhere on the internet and lets root log in with a key. Every server you stand up inherits those defaults until you change them.
This guide covers both installing and hardening OpenSSH on an Ubuntu server, then connecting from Windows, macOS or Linux. It ends where per-server hardening stops scaling.
1. Install OpenSSH server
sudo apt update && sudo apt install -y openssh-server
2. Start the service and confirm it's running
sudo systemctl enable --now ssh
systemctl status ssh
You want Active: active (running) in the output.
3. Open the firewall
Ubuntu ships with Uncomplicated Firewall (UFW) installed but inactive. Allow SSH before you enable it, or you'll lock yourself out:
sudo ufw allow OpenSSH
sudo ufw enable
sudo ufw status
4. Make one connection
From the client, replace user and server-ip with your account and address:
ssh user@server-ip
The first connection shows the server's host key fingerprint and asks you to confirm it. Type yes and the client stores it in ~/.ssh/known_hosts. If a later connection warns that the fingerprint changed, stop: either the server was rebuilt or something is sitting between you and it.
Five directives close the gaps the defaults leave open. Put them in a drop-in file rather than editing /etc/ssh/sshd_config directly:
sudo nano /etc/ssh/sshd_config.d/00-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
Why a drop-in, and why 00: Ubuntu's sshd_config includes /etc/ssh/sshd_config.d/*.conf before anything else, and sshd keeps the first value it reads for each directive. A file named 00- outranks the 50-cloud-init.conf that cloud images drop in, which often sets PasswordAuthentication for you.
Set PasswordAuthentication no only after key-based login works (next section). Keep your current session open, test the config, then restart:
sudo sshd -t && sudo systemctl restart ssh
Open a second terminal and log in again before you close the first one.
Optional: Move off port 22
A non-default port cuts the noise in your logs. It doesn't cut risk, so treat it as housekeeping, not hardening. Add Port 2222 to the drop-in file, then open the firewall and restart the socket:
sudo ufw allow 2222/tcp
sudo systemctl daemon-reload
sudo systemctl restart ssh.socket
Ubuntu 22.10 and later start sshd through systemd socket activation, so restarting ssh.service alone leaves sshd listening on 22. The daemon-reload is what picks up the new port. Confirm it with sudo ss -tlnp | grep sshd, and remove the old rule with sudo ufw delete allow OpenSSH once you've logged in on the new port.
Optional: Ban repeated failures
sudo apt install -y fail2ban
sudo fail2ban-client status sshd
On Ubuntu 24.04, the package reads the systemd journal and enables the sshd jail with no extra configuration. Repeated failed logins from one address earn a temporary ban.
Optional: Script it for the next server
Everything above fits in a short script. Run it after your public key is already in ~/.ssh/authorized_keys (cloud-init puts it there on most images), or the last line will lock you out:
#!/bin/bash
set -e
sudo apt update && sudo apt install -y openssh-server fail2ban
sudo systemctl enable --now ssh
sudo ufw allow OpenSSH && sudo ufw --force enable
sudo tee /etc/ssh/sshd_config.d/00-hardening.conf >/dev/null <<'EOF'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
EOF
sudo sshd -t && sudo systemctl restart ssh
When researchers probed the 20 million SSH servers reachable on the public internet, more than 65% accepted passwords. Yours doesn't need to. On the client:
ssh-keygen -t ed25519
ssh-copy-id user@server-ip
ssh user@server-ip
Spell out -t ed25519: Ubuntu 22.04 ships OpenSSH 8.9, which still generates RSA keys by default. Once the third command logs you in without a password prompt, the PasswordAuthentication no line above is safe to apply.
The full walkthrough, including the Windows commands, agent setup and which key type to choose, is in How to Set Up SSH Passwordless Login.
Windows 10 and 11. The OpenSSH client is a built-in Windows feature. Open PowerShell or Windows Terminal and run ssh user@server-ip. If ssh isn't recognized, add it under Settings > Apps > Optional features > OpenSSH Client. Prefer a window? PuTTY still works: enter the hostname, leave the port at 22 and select Open.
macOS. Open Terminal and run ssh user@server-ip. The client is preinstalled.
Linux. Most desktop distributions include the client. If not: sudo apt install openssh-client on Debian-based systems.
On every platform, a custom port goes on the command line: ssh -p 2222 user@server-ip.
Five commands you'll use every week |
|
| Task | Command |
| Save a server as an alias | In ~/.ssh/config: Host web1, HostName server-ip, User user, Port 2222. Then ssh web1 |
| Run one command and exit | ssh web1 "df -h" |
| Copy a file to the server | scp report.csv web1:/tmp/ |
| Browse and transfer interactively | sftp web1 |
| Reach a service that only listens on the server | ssh -L 8080:localhost:80 web1, then open localhost:8080 |
Connection refused. sshd isn't running, the firewall is blocking the port, or you changed the port without the daemon-reload. Check in that order:
systemctl status ssh
sudo ufw status
sudo ss -tlnp | grep ssh
Permission denied (publickey). The server refused every key the client offered. Either the public key isn't in ~/.ssh/authorized_keys for that account, or the permissions are too open. On the server:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $USER:$USER ~/.ssh
For the server's side of the story, read the journal: sudo journalctl -u ssh -n 50. On Ubuntu 24.04, /var/log/auth.log exists only if rsyslog is installed.
Everything above hardens one server. Multiply it by 50 servers and 20 admins and the questions change: who holds a key to which box, who approved it, what did they run, and who removes the key when they leave. A drop-in file controls the point of access. It can't answer any of those. Privileged Access Management (PAM) can.
Privileged Remote Access gives admins and third parties a browser-based SSH session without a virtual private network (VPN) and without handing them a credential. The key never lands on their laptop, so there's no key file to copy, share or forget to revoke.
Secret Server vaults the SSH keys and passwords your servers already depend on, rotates them, and proxies and records every SSH session that uses them. Point your firewall at it and each server can reject any connection that doesn't arrive through the proxy.
For your admins, that means ssh web1 becomes a click in a browser with the same shell on the other end. For whoever owns your next audit, it means every privileged session has a named user, an approval and a recording behind it.
Schedule a demo and watch an SSH session run through Privileged Remote Access with no key on the admin's laptop.