How to Set Up SSH Key-Based Authentication on Linux

How SSH key-based authentication works

SSH public-key authentication uses a matching key pair. The private key stays on the client computer and should never be shared. The corresponding public key is installed on the Linux server in the target account’s ~/.ssh/authorized_keys file. During login, SSH verifies that the client can use the private key corresponding to an authorized public key.

A key pair is not the same as copying a password into a file. The server receives only the public key; the private key remains under your control. A passphrase can protect the private key if the client device is lost or accessed by someone else.

Prerequisites

• A Linux client computer or another system with an OpenSSH client.
• A Linux server with OpenSSH Server installed and running.
• An account on the server and an existing way to log in initially, usually with a password or console access.
• The server’s hostname or IP address and the correct SSH username.
• Permission to administer the server if you intend to change SSH daemon settings.

Install OpenSSH if needed

On Debian or Ubuntu server systems:

sudo apt update
sudo apt install openssh-server

On RHEL, Rocky Linux, AlmaLinux, or Fedora systems, package and firewall details can vary by release. A common installation command is:

sudo dnf install openssh-server
sudo systemctl enable --now sshd

On Debian/Ubuntu, the service is commonly named ssh; on many RHEL-family systems it is named sshd. Check the service name used by your distribution.

Step 1: Generate an SSH key pair on the client

Run the following command on the computer you will connect from, not on the remote server:

ssh-keygen -t ed25519 -C "your_name@your_client"

Ed25519 is a good default for modern OpenSSH deployments. If your environment has compatibility requirements, follow your organization’s approved key policy.

When prompted for a file location, press Enter to accept the default path, usually ~/.ssh/id_ed25519. Set a strong passphrase when prompted. This creates two files:

To confirm the public key exists:

ls -l ~/.ssh/id_ed25519*

Step 2: Copy the public key to the Linux server

If ssh-copy-id is installed on your client and password login is currently available, run:

ssh-copy-id username@server_ip_or_hostname

Replace username with the account on the server and server_ip_or_hostname with the server’s address. Enter the account password when prompted. The tool adds the public key to the account’s ~/.ssh/authorized_keys file.

To specify the key explicitly:

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@server_ip_or_hostname

If ssh-copy-id is unavailable

You can transfer the public key by another trusted method, then append it to the target account’s authorized_keys file. Run these commands on the server while logged in as the target user:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Append the complete single-line contents of id_ed25519.pub to ~/.ssh/authorized_keys. Do not overwrite existing entries. Ensure the file and directory are owned by the target user. If you use sudo to create them, correct the ownership, for example:

sudo chown -R "$USER":"$(id -gn)" ~/.ssh

Run the ownership command as the target user with care; if you are operating as root or editing another user’s home, specify that user and group explicitly instead.

Step 3: Test SSH key-based authentication

From the client, connect to the server:

ssh username@server_ip_or_hostname

If the key is at a non-default path, specify it:

ssh -i ~/.ssh/my_server_key username@server_ip_or_hostname

If the key has a passphrase, SSH may prompt you for it. That is normal: it unlocks your private key locally and is not the same as the remote account password.

For verbose diagnostics, run:

ssh -v username@server_ip_or_hostname

Look for messages indicating that the client offers a public key and the server accepts it. Do not share verbose output publicly without checking it for usernames, hostnames, IP addresses, and other sensitive details.

Step 4: Optional hardening — disable password authentication

After key login works reliably, administrators may choose to disable SSH password authentication. This is optional and should be tested carefully, especially on remote cloud servers or production systems.

Back up the SSH daemon configuration first:

sudo cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config.backup

Edit the server’s SSH daemon configuration, commonly /etc/ssh/sshd_config or a distribution-managed snippet under /etc/ssh/sshd_config.d/. Ensure the effective settings include:

PubkeyAuthentication yes
PasswordAuthentication no

Be aware that included configuration files and distribution defaults can affect the effective values. Validate the configuration before applying changes:

sudo sshd -t

If the command returns no error, reload or restart the correct service for your distribution. Common examples:

# Debian / Ubuntu
sudo systemctl reload ssh

# RHEL / Rocky Linux / AlmaLinux / Fedora
sudo systemctl reload sshd

Keep the original session open and test a new connection from a separate terminal. If the new login fails, restore the previous configuration using your still-open session or console access. Do not disable all password or interactive authentication methods blindly if your environment depends on them for other access workflows.

Permissions and troubleshooting

Useful server-side checks include:

# Check service state (use the service name for your distro)
sudo systemctl status ssh
sudo systemctl status sshd

# Inspect effective configuration (where supported)
sudo sshd -T | grep -E 'pubkeyauthentication|passwordauthentication|authorizedkeysfile'

# Recent authentication logs vary by distribution
sudo journalctl -u ssh --since "15 minutes ago"
sudo journalctl -u sshd --since "15 minutes ago"

Some commands above will report that a service or unit does not exist if your distribution uses the other service name; run the relevant command for your system.

SSH key security best practices

• Protect private keys. Use a strong passphrase and restrictive local permissions. Never paste a private key into tickets, chat, email, or a web form.
• Use separate keys where appropriate. Separate keys for personal access, automation, and different trust boundaries make revocation easier.
• Remove old access. When a user or device no longer needs access, remove its public-key line from the relevant authorized_keys file.
• Keep OpenSSH updated. Apply supported security updates for your distribution.
• Use least privilege. Log in as a named user and use sudo for administrative tasks rather than enabling direct root login unnecessarily.
• Preserve recovery access. Maintain console or out-of-band access before tightening authentication on remote systems.
• Review logs and access. Monitor successful and failed authentication attempts and restrict SSH access with firewalls or network controls where practical.

Frequently asked questions

Official documentation and further reading

• OpenSSH manual pages — client, server, key generation, and agent documentation.
• Ubuntu Server: OpenSSH server guide.
• Ubuntu man page: sshd and authorized_keys.
• Ubuntu Community Help: SSH keys.

Share this tutorial

VMoreCloud is an independent technology publication. Commands and service names can differ by distribution and release. Review your distribution’s documentation and test changes safely before applying them to production systems.