Maybe have a look at https://nginxproxymanager.com as well. I don’t know how difficult it is to install since I never used it, but I heard it has a relatively straight-forward graphical interface.

Configuring good old plain nginx isn’t super complicated. It depends a bit on your specific setup, though. Generally, you’d put config files into /etc/nginx/sites-available/servicexyz (or put it in the default)

server {  
    listen 80;  
    server_name jellyfin.yourdomain.com;  
    return 301 https://$server_name$request_uri;  
}  

server {  
    listen 443 ssl;  
    server_name jellyfin.yourdomain.com;  

    ssl_certificate /etc/ssl/certs/your_ssl_certificate.crt;  
    ssl_certificate_key /etc/ssl/private/your_private_key.key;  
    ssl_protocols TLSv1.2 TLSv1.3;  
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384';  
    ssl_prefer_server_ciphers on;  
    ssl_session_cache shared:SSL:10m;  

    location / {  
        proxy_pass http://127.0.0.1:8096;  
        proxy_http_version 1.1;  
        proxy_set_header Upgrade $http_upgrade;  
        proxy_set_header Connection 'upgrade';  
        proxy_set_header Host $host;  
        proxy_cache_bypass $http_upgrade;  
    }  

    access_log /var/log/nginx/jellyfin.yourdomain_access.log;  
    error_log /var/log/nginx/jellyfin.yourdomain_error.log;  
}  

It’s a bit tricky to search for tutorials these days… I got that from: https://linuxconfig.org/setting-up-nginx-reverse-proxy-server-on-debian-linux

Jellyfin would then take all requests addressed at jellyfin.yourdomain.com and forward that to your Jellyfin which hopefully runs on port 8096. You’d use a similar file like this for each service, just adapt them to the internal port and domain.

You can also have all of this on a single domain (and not sub-domains). That’d be the difference between “jellyfin.yourdomain.com” and “yourdomain.com/jellyfin”. That’s accomplished with one file with a single “server” block in it, but make it several “location” blocks within, like location /jellyfin

Alright, now that I wrote it down, it certainly requires some knowledge. If that’s too much and all the other people here recommend Caddy, maybe have a look at that as well. It seems to be packaged in Debian, too.

Edit: Oh yes, and you probably want to set up Letsencrypt so you connect securely to your services. The reverse proxy would be responsible for encryption.

Edit2: And many projects have descriptions in their documentation. Jellyfin has documentation on some major reverse proxies: https://jellyfin.org/docs/general/post-install/networking/advanced/nginx