This article applies to NoMachine v. 4 or later.

Key-based authentication is an alternative to password-based authentication (default). Instead of requesting the user's password, the authentication relies on a SSH key-pair, consisting of a public key installed on the server host and on a private key owned by the user.

Support for key-based authentication with the NX protocol requires the public SSH key be added on the server side, in the user's home, to grant access.

When you connect to the server you will have to insert the private key in the NoMachine User Interface (UI) of whichever client device you are using.

Key-based authentication with NX protocol is available for all the NoMachine server products, including NoMachine Free Edition.

Generate the SSH key-pair

If you haven't done it already, generate an SSH key pair. Be sure that the keys are in OpenSSH  format.

On Linux and macOS you can use the ssh-keygen tool. Run in a terminal:

ssh-keygen -t rsa -b 4096

Note
For NoMachine version older than v. 6.9.2 and openssh version 7.8p1-1 (which introduces a new OpenSSH format) or later, specify to generate the key in the old format:
ssh-keygen -m PEM -t rsa -b 4096

On Windows you can use for example PuTTYgen (https://www.puttygen.com).
You need to ensure that the key-pair is in OpenSSH format or convert it.
To do that choose the key file in the PuTTYgen main window. Then go to Conversions->Export OpenSSH key to
export your private key and save it.
 

Add the public SSH key on the server

1. Navigate to the <user's home>/.nx/config directory.

Create this directory if it doesn't exist.
  
On Linux and macOS, run in a terminal:

mkdir $HOME/.nx/config

On Windows, create that (C:\Users\username\.nx\config) by using the system tools.

2. If the config directory already exists, it should contain the authorized.crt file. Create this file if it doesn't exist and set the appropriate permissions.

On Linux and macOS:

touch $HOME/.nx/config/authorized.crt
chmod 0600 $HOME/.nx/config/authorized.crt

On Windows permissions should be set to 0644.
 

3. Append your SSH public key at the end of the authorized.crt file.

Copy the public key to the server host. For example if you created it on Linux or macOS, you can use scp from a terminal on that computer:

scp ~/.ssh/id_rsa.pub username@serverHost:~/.ssh/id_rsa_nomachine.pub

where serverHost can be either hostname or IP of the machine where NoMachine server is installed.

Then, on the server host, append the public key to the authorized.crt file:

cat  ~/.ssh/id_rsa_nomachine.pub >> ~/.nx/config/authorized.crt

As an alternative, also on Windows, you can use a text editor to append the key to the authorized.crt file. In this case,  be sure to not save the file in a different format.
 

This is an example of how the public key added to the authorized.crt file looks like:

4. Disable password authentication on the NoMachine server (optional)

You can optionally configure the NoMachine server to force users to use key-based authentication only. With the following setting, users will be no longer able to login via NoMachine by using their system password.

Edit the server configuration file:

/usr/NX/etc/server.cfg on Linux

/Applications/NoMachine.app/Contents/Frameworks/etc/server.cfg on macOS

C:\Program files (x86)/NoMachine/etc/server.cfg on Windows

and set:

AcceptedAuthenticationMethods NX-private-key

Then restart the NoMachine server. You can do that via UI, open NoMachine -> Settings -> Server -> Status -> Restart the server, or via command line.

On Linux and macOS, run in a terminal:

$ sudo /etc/NX/nxserver --restart

On Windows open a CMD console as administrator and execute:

> cd C:\Program files (x86)\NoMachine\bin\
> nxserver --restart
 

Use the private SSH key when connecting with your NoMachine client

On your computer:

1. Launch the NoMachine User Interface from Programs menu or Applications and right click on a connection. Choose 'Edit connection'  to access connection settings.

2. Be sure that NX is the selected protocol in the drop-down menu.

3. Go to Configuration panel and choose 'Use key-based authentication with a key you provide' and click on Modify.

4. Select the private key and check 'Import the private key to connection file' if you want to save it for that session.

5. Back to the previous panel and click on Connect

The client will prompt you for username and passphrase. If your SSH key doesn't have a passhprase, leave this field empty.

Note
For client UI v.6 or v.5:
3. Click on the Advanced button.
4. Choose 'Private key'  authentication and click on the 'Settings' button.
5. Click  the '...' button navigate directories and provide path to your private SSH key.
6. Click on 'OK' button and connect.

The client will prompt you for username and passphrase. If your SSH key doesn't have a passhprase, leave this field empty.

Other references

- How to set up key based authentication with SSH protocol (not available with NoMachine Free Edition)
  https://www.nomachine.com/AR06O00932

- How to use a SSH private key when connecting from NoMachine client
  https://www.nomachine.com/AR11O00948