I had started working on migrating off of cloud services to self hosting everything a couple of months ago by getting some new gear to set up my server on (shoutout to Ryan for the hookup). I started off by setting up a stub webserver and general environment for me to deploy games servers for my friends which is pretty much the exact same process as if you were to do it on an external hosting service, except I was a little bit more careful to not open myself up to security vulnerabilities since I’m hosting this in my own home.
After I had set up a Minecraft server, I added a record to my domain provider pointing to my home network and tried to connect to it using that domain. No connection. That was a bit strange but maybe I forgot some configuration so I went through a checklist:
- The port was open
- The DNS record pointed to my public IP address
- Firewalls were fine
- Minecraft server was running
Well, what the hell was the issue then??
Turns out, it’s a much broader networking issue that’s usually already resolved by someone somewhere.
This answer to a stackexchange question explains a similar problem much better than I can, but I want to dumb it down and over-explain for myself in the future and for those who may stumble upon this and may find it helpful.
When your computer wants to send a message to someone, it attaches an address to whom the message is sent to (the destination) and one for whom the message was sent from (the source). After your computer sends the message, it expects to get a response from the receiver with the source and destination addresses swapped.
Your computer may sometimes know exactly where to send a message to, like in the case of sending a message to localhost (which should be set up for you automatically in almost all scenarios), but most of the time, it may need some help. This is done through a gateway. This is a device (likely your modem/router) that, figures out where to send messages among other things. So, when you connect to a local IP address, your computer doesn’t know where to send the message to, so it forwards it to the gateway. The gateway knows where to send the message, so it relays it to the receiver.
Okay, but what if you try to connect to a non-local IP address? Well, it’s still a similar process. Your computer forwards the message to your gateway and your gateway realizes that this IP address belongs to none of the locally connected addresses in your network. It wants to send it to the internet, but it can’t quite do so yet. This is because the local IP address of the source is not unique among the internet IP addresses, and so your gateway rewrites the source IP address to your public IP address, which is unique, and sends it off. When your gateway eventually receives a message back, the destination IP address is your public IP address, as per the protocol described earlier, and to finally send the message back to the original computer, it rewrites the destination IP address as the local IP address that originally sent the message. This process is called network address translation, or NAT for short.
Sending messages through a domain like www.public.com just adds one more simple step to the process (simple for our sake). When your gateway receives the message, it doesn’t know who www.public.com is. It only knows how to send a message to IP addresses. The extra step is to contact a DNS server to find out the IP address of www.public.com. Once it does that, the rest is the same.
Now that we understand this, we’ll work through what happened when I tried to connect to my server through the public domain.
Suppose my public domain is www.public.com, my public IP address is 1.2.3.4, and the local IP address of the computer my Minecraft server on is 192.168.1.2.
- My computer tries to send a message from 192.168.1.3 to www.public.com
- www.public.com is not an address that is recognized in my computer’s routing table (unlike localhost for example)
- My computer hands it off to the gateway on my router
- The gateway resolves the DNS lookup to be 1.2.3.4
- The gateway doesn’t know where to forward the message to! We know that it should be the local IP address 192.168.1.2, but the gateway doesn’t know that
- Sending the message fails, and your gateway doesn’t give any response.
Now that we know how it fails, how can we solve this?
The method I used is called DNS splitting. Really, all it is is telling your computer that when a message is supposed to be sent to X, send it to Y instead. So when we want to connect to www.public.com, we just tell it, “We know where www.public.com is already. It’s at 192.168.1.2″. How this can be accomplished is by modifying your /etc/hosts file to include a line like
192.168.1.2 www.public.comWith that, it was fixed!
This should work on all *nix systems (I’ve tried with Linux and macOS at least). If you’re on Windows, apparently doing something with the system drivers(?) might work. See this article by yunohost for more details.
I think there are other ways of solving this problem like NAT reflection as mentioned in the stackexchange thread earlier, but this solution is dead simple and I’m happy with it.
I think for some routers, this is configured for you by default in the gateway itself, or at least it’s configurable in the DNS settings in your gateway, so you don’t have to DNS split on every single machine that needs it, but my router, provided by Shaw, sucks and so I have to do this workaround instead.
Hopefully this helps someone out and hopefully you learned a thing or two reading this and I wish you luck in your networking adventures!