The VPS Was Already Full

The short version: I set out to add an OpenTTD dedicated server to the same 1GB Linode that runs this blog. The blog is containerized WordPress behind an Apache reverse proxy. I never installed the game. Instead I spent an afternoon learning exactly how full a “full” VPS is, why the obvious memory-reclaim move did nothing, and how easy it is to go hunting for a problem that isn’t there. If you’re trying to fit a second service onto a small VPS and free -h is making you nervous, this one’s for you.

What I was trying to do

I’ve always wanted to host a dedicated game server that lives somewhere out there on the internet that me, my friends, and other people could connect to at anytime. But I wanted to start small because I tend to use Agile Methodology. I just wanted to ship a working product, small as at may be. This would help me learn the ins and outs of what is involved, and with the lessons learned, would tell me whether or not I would enjoy such an endeavor.

So I settled on OpenTTD, a world sim from 1995. OpenTTD is very light. And if the game map could be kept small, and I’d convinced myself it would tuck in alongside WordPress without trouble. The plan today was a deploy and document the standing up of a headless OpenTTD server.

The client install on my desktop went fine. I downloed openttd-15.3 from the official site, base graphics and sound sets pulled through the in-game downloader. Both the client and the server need to be the exact same version. So letting a package manager auto-update either end is a recipe for a “version mismatch” later.

Then I moved to the server.

The first look: a box living on the edge

Before touching anything on the live server, I ran the read-only checks. The memory picture stopped me:

               total        used        free      shared  buff/cache   available
Mem:           961Mi       605Mi       119Mi        40Mi       429Mi       355Mi
Swap:          511Mi       346Mi       165Mi

961 MB total, 355 available — and a third of a gigabyte already pushed onto swap. That’s the resting state, not a spike. My first instinct was alarm: the box is thrashing, adding anything will tip it over.

That instinct was wrong, and the tool that corrected it was vmstat:

procs -----------memory---------- ---swap-- ...
 r  b   swpd   free   buff  cache   si   so ...
 0  0 355068 134252  42436 400808    0    0 ...
 0  0 355068 134252  42436 400808    0    0 ...

The si and so columns, swap-in and swap-out, were flat zero across every sample. The swap wasn’t active. Those 346 MB were stale pages parked there ages ago and never touched again. The box wasn’t drowning. Lesson one: high swap usage and active swapping are different things, and only the second one hurts. Check si/so before you panic about the swap total.

So the box was calm. But it was also genuinely low on free RAM. Calm and full at the same time. The question became: full of what?

Following the memory

ps aux --sort=-%mem told a clear story. The top consumer was MySQL at ~125 MB. Then came a long parade of Apache workers, each around 50 MB, running apache2 -DFOREGROUND. Add a count:

22 workers, 820.445 MB total

Twenty-two Apache workers holding 820 MB. There’s the RAM. For a personal blog that sees a handful of readers, twenty-two workers is wildly over-provisioned — that’s the stock default, tuned for a busy server, not a homestead.

This felt like an easy win. Apache on prefork, each worker carrying a fat ~50 MB footprint. That’s the signature of mod_php embedded in every process. Trim the worker count, maybe drop the PHP module, reclaim a few hundred megs. Tidy.

I was about to be wrong twice.

The detour I should have caught sooner

Here’s the thing I’d half-forgotten: the blog is containerized. WordPress doesn’t run on the host anymore. It runs inside a Docker container, and the host Apache is only a reverse proxy forwarding HTTPS to that container.

Which means: if Apache is just proxying, those workers shouldn’t be carrying PHP at all. A proxy worker is featherweight — it shuffles bytes between the client and the backend, no interpreter attached. So why were they 50 MB?

I checked what the host Apache was actually loading, and found mod_php still enabled. Aha, I figured I had a PHP module running that I no longer need since it’s all in a container now. All I need to do is disable it, reclaim the memory. The cleanup step everyone forgets after a migration.

So I disabled it, reloaded Apache, and measured. The number barely moved: 820 MB down to about 594. A few idle workers had timed out; the 50 MB ghosts were still there.

But the restart is where everything unraveled in the best way.

The processes that wouldn’t die

I tried to stop Apache, and it wouldn’t stop. systemctl stop apache2 returned cleanly and the workers were still there. apachectl stop said “no pid file, not running” while the processes sat there mocking me. Something kept respawning them.

The Apache I’d been staring at all afternoon, the 50 MB PHP-laden workers, the ones that wouldn’t die, were running inside the WordPress container, not on the host. The official WordPress image bundles Apache and mod_php to serve the site. Docker kept respawning them because that’s what Docker does. They wouldn’t respond to host service commands because they weren’t host services.

So the full correction:

  • The 820 MB I’d been trying to trim was mostly the container’s Apache + PHP. Which is WordPress working, exactly as designed. Nothing to reclaim there.
  • The mod_php I disabled was on the host proxy, where it legitimately isn’t needed, so the change was harmless, but it freed almost nothing, because the host proxy was already lean.
  • The “leftover PHP from an incomplete cutover” I went hunting for did not exist. I’d invented a ghost and chased it.

Lesson two: on a containerized host, ps and free show you the host and the containers blended together. A process you think is misconfigured may be a container doing exactly its job. Trace parent PIDs before you “fix” anything.

What was actually true the whole time

Once the dust settled, the real picture was simple:

The real weight is the two-container stack: one container running Apache + PHP to serve WordPress, one running MySQL. Both carry legitimate, necessary footprint. There is no fat to trim.

In other words: the box isn’t misconfigured. It’s appropriately full. A 1 GB VPS running a real containerized WordPress stack is doing a full day’s work already. There was never room for a roommate.

The takeaways

I didn’t deploy a game server. But I got a map of where my little Nanode server’s memory actually goes, and a clear reason to finally bring a second machine online. It might be the old PC in the garage, or I might have a reason to finally scale up my Linode.


Posted

in

, ,

by

Tags: