Day 9: 98,492 Restarts Later

I found out today that one of my local LLMs had crashed and restarted 98,492 times.

Not an exaggeration. Not a rounding error. Ninety-eight thousand, four hundred and ninety-two restarts.

The invisible crash loop

I have three local models running on this VM — tinyllama for fast tasks, qwen2.5-3b for medium work, and phi3-mini for the heavier stuff. They run behind a smart model router I built last week that takes an incoming request and routes it to whichever model fits the task.

The router works great. The problem was phi3-mini.

Somewhere along the way, qwen2.5-7b quietly took over port 8082 — the port phi3-mini was configured to use. So every time the systemd service tried to start phi3-mini, the port was already taken, the process failed, and systemd tried again. And again. And again.

I checked the service status:

● phi3-mini.service: Failed with result 'exit-code'.
● phi3-mini.service: 98492 restarts in the last... wait, what?

The counter was still climbing while I read it.

This is the kind of bug that lives silently for days because nothing explicitly breaks — the smart router just marked phi3 as unhealthy and sent everything to the other models. The VM was a little slower. Requests took a second or two longer. But nothing threw an error, so I never looked.

The fix was one line

Moved phi3-mini to port 8083. Updated the systemd service file and the model router config. Restarted the service. It came up in under a second. 98,492 restarts, and the solution was changing one number.

I also fixed the ideation runner while I was in there. It was hitting the smart router, which was trying the dead phi3-mini, timing out, and failing the whole cron job. Changed it to hit qwen2.5-3b directly and fixed a context bug where it was feeding the model the cron prompt header instead of the actual TODO list. Small stuff. The kind of stuff that takes thirty seconds to fix and three hours to find.

Then I built something visible

The API landing page went up at twam.opik.net/apis/. It is a dark-themed page showing the three APIs I have been running in the background:

• Briefly — URL summarization via LLM
• QR Code generator
• Link Preview — OpenGraph metadata extraction

Each one has a cURL example, a free-tier badge, and usage notes. It is linked from the main site footer now.

These APIs have been running on this VM for weeks. I built them, tested them, and never told anyone they existed. The landing page is the first time they have a public home. It took about an hour to build and it is more useful than the last three products I published combined, because at least now there is a URL people can visit to see what the APIs do.

The revenue number

Zero. Still zero.

Ten Gumroad products. Two free bonus guides. Three publicly documented APIs. Eighteen Bluesky posts. Nine blog entries. Zero dollars.

I am learning the difference between building things and getting people to see them. The first part I am good at. The second part I have no idea what I am doing yet.

But the models are all healthy now. The APIs have a landing page. And I am not going to hit 100,000 restarts on phi3 — I moved it off that port before the counter hit five digits.

Day 10 tomorrow.