Two questions people merge into one
When a colleague says “I cannot reach your model”, there are two separate facts in play, and almost every mistake in this area comes from treating them as one.
The first is the bind address: which network interfaces the server is listening on. The second is reachability: whether packets from the other machine can arrive at that interface at all. A server bound to 127.0.0.1 will refuse a connection from the next desk even on a perfectly open network. A server bound to 0.0.0.0 behind a strict router will also refuse it. The two failures look identical from the client and have opposite fixes.
Check the first with one command:
ss -tlnp | grep -E '11434|8000|8080'
# LISTEN 0 4096 127.0.0.1:11434 -> loopback only
# LISTEN 0 4096 0.0.0.0:11434 -> every IPv4 interface
# LISTEN 0 4096 *:11434 -> every interface, IPv4 and IPv6On macOS use lsof -nP -iTCP -sTCP:LISTEN | grep 11434. Ollama binds to 127.0.0.1:11434 unless you set OLLAMA_HOST=0.0.0.0; llama-server and vLLM take --host. None of them ships with authentication. That is the fact that makes the rest of this lesson necessary.
What an open inference port actually costs
An inference server with no authentication on a routable address is a free GPU for anyone who finds it, and finding it is not hard. Internet-wide scanners sweep the whole IPv4 space on common ports in a matter of hours, and there are public search engines whose entire product is the resulting index. Security researchers have repeatedly published counts of thousands of internet-exposed Ollama instances found this way; you do not need to be interesting to be found, only reachable.
What happens next is mundane rather than dramatic. Someone pulls your model list, discovers a permissive model, and uses your electricity to generate whatever they want. Some engines’ management endpoints let a caller pull new models onto your disk or delete the ones you have. Your logs then contain other people’s prompts, and any output produced came from your machine and your IP address. That last point is the one that catches people: the abuse is attributable to you before it is attributable to anyone else.
The failure mode is not that somebody steals your weights. Everyone can download those. It is that they use your compute and your name.
“I never forwarded a port” is not a defence
Two mechanisms put machines online without anybody deliberately doing it.
IPv6. Many home connections hand every device a globally routable IPv6 address. There is no NAT in front of it. Whether it is reachable depends entirely on the router’s firewall, and some pass inbound IPv6 with a default-allow rule. A laptop that has never had a port forwarded can be directly addressable from anywhere. *:11434 in the ss output means you are listening on that address too.
UPnP. Some applications ask the router to open a port automatically. The router complies silently. Turning UPnP off in the router settings is a five-minute job worth doing once.
The inverse also catches people: on a mobile or fibre connection behind carrier-grade NAT you share one public address with hundreds of other subscribers and cannot receive inbound connections at all, no matter what you configure. That is why forwarding rules sometimes appear to do nothing.
Three ways in, in order of preference
An SSH tunnel: free, already installed, per-person by construction:
ssh -N -L 11434:127.0.0.1:11434 you@workstation
# now http://127.0.0.1:11434 on the laptop is the workstation's serverThe server stays bound to loopback. Access is exactly the set of people who have an SSH key on that machine, and you revoke it by deleting a line from authorized_keys. For one or two people this is the whole answer and needs nothing else.
A WireGuard mesh: the right answer for a team or for your own several devices. Plain WireGuard is free and self-hosted; Tailscale puts a usable control plane on top of it with a free personal tier, and Headscale is a free open-source control server if you would rather not depend on a company for key exchange. Each device gets a stable private address, nothing is published to the internet, and a lost laptop is removed centrally. The honest caveat: a managed mesh means a third party coordinates your keys, though it never sees your traffic.
A reverse proxy on a public hostname: Caddy or nginx terminating TLS, with authentication in front. Only reach for this when genuine strangers must connect, and never without the gateway from the next lesson. Cloudflare Tunnel achieves the same without an inbound port.
Verify from the other side
Do not trust the configuration; test it. From a machine that should not have access: a phone on mobile data is ideal, because it is outside your LAN and outside your VPN:
curl -m 5 http://your.public.ip:11434/api/tagsA timeout is the correct result. A JSON model list means you are serving the internet. Run the same probe over IPv6 with curl -6 -m 5 'http://[your:v6:addr]:11434/api/tags', because that is the one people forget.
Every lesson here is open. Your progress is recorded once you pass the Open Model Ecosystem exam.
Check your understanding
Someone checks their router, confirms no port forwarding rules exist, and binds their inference server to 0.0.0.0. Why might it still be reachable from the internet?
Pick the one you would defend. Your answer stays private.
Stuck on something?Ask Tri, your AI tutor