RustDesk Remote Access Server Deployment

1. Create the VM in Proxmox

Create VM 102 in Proxmox. Install Debian Linux, assign CPU, RAM, disk, and a network interface, and configure a static IP address appropriate for the management or internal VLAN. After installation, update the system:

apt update && apt upgrade -y

2. Install Docker

Install the Docker engine from the Debian repositories:

apt install docker.io -y

Enable and start Docker:

systemctl enable docker
systemctl start docker

3. Create RustDesk directory

Create a dedicated directory for RustDesk data and keys so containers can persist configuration and keys across restarts:

mkdir -p /opt/rustdesk
cd /opt/rustdesk

4. Run RustDesk server containers

Start the ID server (hbbs) first. It will generate keys in the mounted volume; the relay server (hbbr) uses the same key material. Replace the host IP or ensure DNS resolves to VM 102 if clients use a hostname.

docker run -d \
  --name hbbs \
  -p 21115:21115 \
  -p 21116:21116 \
  -p 21116:21116/udp \
  -v /opt/rustdesk:/root \
  --restart unless-stopped \
  rustdesk/rustdesk-server hbbs

docker run -d \
  --name hbbr \
  -p 21117:21117 \
  -v /opt/rustdesk:/root \
  --restart unless-stopped \
  rustdesk/rustdesk-server hbbr

After the first run, /opt/rustdesk will contain the server's public key (e.g. key.pub). Clients must be configured with this key, the ID server address (VM 102 IP), and the relay server address (same IP).

5. Verify containers

Confirm both containers are running:

docker ps

You should see hbbs and hbbr with the expected ports mapped.

6. Configure RustDesk clients

On each RustDesk client (Windows, macOS, or Linux), open the RustDesk settings and set:

• ID Server: The IP or hostname of VM 102 (e.g. 192.168.x.x)
• Relay Server: Same IP or hostname as the ID server
• Key: The public key from /opt/rustdesk/key.pub on the server

Without the correct key and server addresses, clients will attempt to use the default RustDesk public servers. After applying these settings, restart the RustDesk client and verify that IDs resolve and connections relay through your server.

7. Firewall considerations

Ensure the following ports are open on VM 102 (and on any upstream firewall or security group) so RustDesk clients can reach the ID and relay servers:

• 21115 — hbbs (ID server TCP)
• 21116 — hbbs (TCP and UDP)
• 21117 — hbbr (relay server)

If clients are only used on the internal network, restrict these ports to the appropriate VLAN or subnet. For access from outside the homelab, route traffic through a VPN or a dedicated DMZ and keep the RustDesk VM updated and firewalled.

8. Service management

Common operational commands for the RustDesk server:

docker ps
docker logs hbbs
docker logs hbbr
docker restart hbbs
docker restart hbbr

Use docker logs when diagnosing connection or startup issues. Keys and config under /opt/rustdesk persist across container restarts.

RustDesk client cannot connect

Confirm the client is configured with the correct ID server and relay server IP (or hostname) and the server public key. Check that required ports (21115, 21116 TCP/UDP, 21117) are open on the VM and any firewall in between. Run docker ps to ensure both hbbs and hbbr are running.

ID server not reachable

Verify the hbbs container is running with docker ps and inspect logs with docker logs hbbs. Ensure port 21115 and 21116 (TCP and UDP) are not blocked by the host firewall (e.g. ufw or iptables).

Relay connection issues

Confirm the hbbr container is running and port 21117 is open. Check docker logs hbbr for errors. If clients can resolve IDs but sessions fail to establish, relay connectivity or NAT may be blocking UDP/TCP on 21117; verify routing and firewall rules for the relay port.

Container startup failures

Run docker logs hbbs and docker logs hbbr for startup errors. Common causes include port conflicts (another process using 21115–21117), permission issues on /opt/rustdesk, or an outdated image. Ensure the volume mount is correct and pull the latest rustdesk/rustdesk-server image if needed.