How to Install RustDesk Remote Desktop Server on Ubuntu 22.04

RustDesk is a free, open-source remote desktop application that supports multiple platforms like Windows, macOS, Linux, iOS, and Android. It offers high-performance remote control, secure end-to-end encryption, and easy setup without complex configurations. RustDesk also includes features like file transfer and a user-friendly interface, making it ideal for both personal and professional use.

Prerequisites

Before you begin, make sure you have the following:

  • A deployed Ubuntu server on Vultr.
  • A subdomain A record pointing to your server.
  • Docker Compose installed.
  • SSH access to the server as a non-root sudo user.

Install Docker

First, ensure Docker is installed on your server. If not, you can install it using the following commands,

apt update
apt install -y docker.io
systemctl start docker
systemctl enable docker

Next, install Docker Compose:

curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version

Install RustDesk

Create Directory for RustDesk,

Create a new directory to store RustDesk Docker files. This directory will hold the configuration files and any data needed by the RustDesk server.

mkdir -p /opt/rustdesk

Navigate to the newly created directory.

cd /opt/rustdesk

Create Docker Compose Configuration,

Create a new Docker Compose file named rustdesk.yml in the /opt/rustdesk directory. This file will define the services required to run the RustDesk server.

nano rustdesk.yml

Add the following configurations to the rustdesk.yml file. This configuration sets up two services: hbbs and hbbr, which are essential for RustDesk's server functionality.

version: '3'

networks:
  rustdesk-net:
    external: false

services:
  hbbs:
    container_name: hbbs
    ports:
      - 21115:21115
      - 21116:21116
      - 21116:21116/udp
      - 21118:21118
    image: rustdesk/rustdesk-server:latest
    command: hbbs -r 127.0.0.1:21117 -k _
    volumes:
      - ./hbbs:/root
    networks:
      - rustdesk-net
    depends_on:
      - hbbr
    restart: unless-stopped

  hbbr:
    container_name: hbbr
    ports:
      - 21117:21117
      - 21119:21119
    image: rustdesk/rustdesk-server:latest
    command: hbbr -k _
    volumes:
      - ./hbbr:/root
    networks:
      - rustdesk-net
    restart: unless-stopped

Save and close the file (Ctrl+O, Enter, Ctrl+X).

Start RustDesk Server Containers Use Docker Compose to start the RustDesk server containers. The -d flag runs the containers in detached mode, meaning they will run in the background.

docker-compose -f rustdesk.yml up -d

Output:

root@vps:/opt/rustdesk# docker-compose -f rustdesk.yml up -d
Creating network "rustdesk_rustdesk-net" with the default driver
Pulling hbbr (rustdesk/rustdesk-server:latest)...
latest: Pulling from rustdesk/rustdesk-server
81489fa8ccb8: Pull complete
7f26b1e949e2: Pull complete
fc7448e079c8: Pull complete
Digest: sha256:373f4fbeb259bf8621b26f3ed66876328ad523c0cc0c4e7625c4862a016e98ba
Status: Downloaded newer image for rustdesk/rustdesk-server:latest
Creating hbbr ... done
Creating hbbs ... done

Verify that the RustDesk hbbs and hbbr containers are up and running. The following command lists all running Docker containers.

docker ps

Output:

root@vps:/opt/rustdesk# docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED              STATUS              PORTS                                                                                                                                                                 NAMES
9ed90fccf890   rustdesk/rustdesk-server:latest   "hbbs -r 127.0.0.1:2…"   About a minute ago   Up About a minute   0.0.0.0:21115-21116->21115-21116/tcp, :::21115-21116->21115-21116/tcp, 0.0.0.0:21118->21118/tcp, :::21118->21118/tcp, 0.0.0.0:21116->21116/udp, :::21116->21116/udp   hbbs
c8d4e1b6b93d   rustdesk/rustdesk-server:latest   "hbbr -k _"              About a minute ago   Up About a minute   0.0.0.0:21117->21117/tcp, :::21117->21117/tcp, 0.0.0.0:21119->21119/tcp, :::21119->21119/tcp

Secure the Server with Nginx

Install Nginx

If Nginx is not already installed on your server, install it using the following command. Nginx will be used as a reverse proxy to route traffic to the RustDesk server.

apt install -y nginx

Configure Firewall,

Allow Nginx to communicate on HTTP port 80 by updating the firewall rules.

ufw allow 80/tcp
ufw enable
ufw reload

Create Nginx Configuration for RustDesk Create a new Nginx configuration file for RustDesk. This file will configure Nginx to forward requests from your subdomain to the RustDesk server.

touch /etc/nginx/conf.d/rust.domainhere.info
nano /etc/nginx/conf.d/rust.domainhere.info

Note: Replace the rust.domainhere.info with actual domain

Add the following configuration to the file. Replace rustdesk.example.com with your actual subdomain.

server {
    listen 80;
    listen [::]:80;

    server_name rust.domainhere.info;

    location / {
        proxy_pass http://127.0.0.1:21117;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Note: Replace the rust.domainhere.info with actual domain name

Save and close the file.

Test and Restart Nginx

Test the Nginx configuration for syntax errors.

nginx -t

Output:

root@vps:/opt/rustdesk# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
root@vps:/opt/rustdesk#

If no errors are found, restart Nginx to apply the new configuration.

systemctl restart nginx

View RustDesk Server Encryption Keys

The RustDesk server uses encryption keys for secure communication. View and copy your server's public key, which is needed to configure the RustDesk client.

cat /opt/rustdesk/hbbs/id_ed25519.pub

The output should look similar to:

WfgvfV002mcFRevj205Fd+3vEEEWpeHlcL3xAXnAxJ4=

Securing Nginx with SSL Using Certbot

To ensure secure communication with your RustDesk server, you need to secure your Nginx reverse proxy with SSL. This can be easily achieved using Certbot, which provides free SSL certificates from Let's Encrypt.

Install Certbot

First, install Certbot and the Nginx plugin to automate the certificate installation and configuration.

apt update
apt install -y certbot python3-certbot-nginx

Obtain and Install SSL Certificate Use Certbot to obtain and install an SSL certificate for your domain. Replace rust.domainhere.info with your actual subdomain.

certbot --nginx -d rust.domainhere.info

Description This command will:

  • Obtain an SSL certificate from Let's Encrypt for your domain.
  • Automatically configure Nginx to use the obtained SSL certificate.
  • Set up automatic certificate renewal.

Follow the prompts to complete the process. You will need to provide an email address for important renewal and security notices and agree to the terms of service.

Verify SSL Configuration

After Certbot completes the installation, verify that Nginx is correctly configured and running with SSL.

nginx -t
systemctl reload nginx

Setting Up the RustDesk Client

  1. Download the Client: Head to the RustDesk website and download the client application compatible with your system.
  2. Install RustDesk: Follow the installation process to get RustDesk up and running on your client machine.
  3. Launch the Application: Open RustDesk once the installation is complete.
  4. Open Settings: Click the settings icon located next to your connection ID.
  5. Configure the Server:
    • Select "ID/Relay Server" from the settings menu.
    • In the "ID Server" field, enter your server's domain name.
    • Paste the server’s public key into the "Key" field.
  6. Save and Connect: Click "OK" to save your settings and connect to your RustDesk server.

Testing Your RustDesk Server

  1. Install RustDesk Locally: Download and install the RustDesk client on your local computer.
  2. Open and Configure RustDesk:
    • Navigate to the "ID/Relay Server" section in the app.
    • Enter the domain name of your RustDesk server in the "ID Server" field.
    • Input the server's public key in the "Key" field.
  3. Apply Settings: Click "OK" to save your configuration.
  4. Establish a Connection:
    • Enter the remote machine’s ID in the "Enter Remote ID" field.
    • Click "Connect" to establish a secure and efficient connection through your private RustDesk server.

By following these steps, you will ensure a secure and swift connection to your remote desktop via your dedicated RustDesk server, providing a reliable and encrypted remote access solution.