Exactly, Docker is great - until you leave the beaten path. At a previous workplace I set up a VPN solution around Wireguard and Docker. We wanted to start off by integrating one physical box into our network, where containers would be spawned with a client and a customized MAC addresses to sort them into individual VLANs, which would then be accessible through the VPN.
It took a number of tries to set up correctly, especially since documentation on these areas mostly consists of reading through various issues on the Docker issue tracker. Some necessary features weren't even supported in the current Docker-Compose file versions. And best of all - 3 clients worked without a problem in parallel, any further clients were not visible in the network. No error logs or anything, just no network.
Of course this wasn't using everyday features, but it would have been nice to have a bit more of an introductory guide into the subsystem. This way it felt like I was fighting against Docker more than it was helping me.
Docker networking complicates a lot of stuff if you're using bridges and overlays, especially WireGuard, which wholly depends on UDP and is insanely unfriendly once NATs come into play. Using --net=host or configuring a macvlan network to give the containers real IPs usually helps.
Also doesn't help that linuxserver/wireguard's docs are basically like "Oh, yeah, if you use this for connecting to your VPN from outside of your house (which, like, 99.95% of people installing WireGuard are trying to do), routing might be an issue and is left as an exercise for the reader."
(Funny enough, Tailscale is to WireGuard what Docker is to containers. They recognized that wg is amazing but amazingly complicated to get going with, especially through NAT/CGNAT, so they drastically simplified that, added amazing UX on top of it, and are raking in that VC cash. Can't wait to read "hurr durr tailscale dumb, wg-quick amirite" in like five years after Tailscale is a multi-billion dollar networking juggernaut)
> personally i would skip straight to kubernetes if you start messing with swarm
I don't know. Docker swarm mode is basically Docker spread over multiple nodes. Consequently it's a very spartan Kubernetes-light, but extremely easy to operate and maintain.
I love microk8s but I still find Docker swarm mode to be the ideal first step for on prem.
the problem with using swarm IMO is forming dependencies around it. i've found that it is easy enough to get started with, but scaling it to any degree is asking for trouble. it also doesn't help that swarm seems all but abandoned now that Docker spends a lot of time bolstering the built-in Kubernetes experience.
It took a number of tries to set up correctly, especially since documentation on these areas mostly consists of reading through various issues on the Docker issue tracker. Some necessary features weren't even supported in the current Docker-Compose file versions. And best of all - 3 clients worked without a problem in parallel, any further clients were not visible in the network. No error logs or anything, just no network.
Of course this wasn't using everyday features, but it would have been nice to have a bit more of an introductory guide into the subsystem. This way it felt like I was fighting against Docker more than it was helping me.