I have a setup very similar to what you described in terms of having several web geegaws hiding behind a proxy, with Docker to manage them all. Foundry is even part of my setup as well (speaking of which, join https://lemmy.ml/c/FoundryVTT it's still pretty dead in there but one of the reddit mods came over and while they're not promoting things they're keeping an eye on them).
The proxy server of choice is very much matter of taste.
- I use Caddy, which is very modern, simple to configure, and automatically handles letsencrypt cert setup out of the box.
- Nginx is almost as modern, more flexible/powerful, IMO less beginner-friendly to setup, and has letsencrypt automation but not out of the box. Nginx is probably most common these days.
- I know less about traefik, I feel like that's most often employed as a kubernetes ingress... but maybe people use it for other stuff and I just haven't encountered it. I do feel like outside of k8s, it's MUCH less common than nginx.
- Good ole apache is great if you know how to use it. It's a bit archaic to configure compared to nginx or caddyz and though it is supremely capable, I do not recommend it unless you're already steeped in its ways.
There's a hundred other options as well, but it's really just preference for a toy setup like ours. I've used many of these things and prefer Caddy for my homelab stuff cause it's dreamily simple to configure. If I wasn't using Caddy, I'd be using nginx just because it's so commonly used that you can find help/instructions for anything you can imagine.