I build a lot of things for myself. Some of them work quietly in the background, and sometimes too quietly.
This is the story of how a single missing line of code caused our reminder bot to stop delivering notifications for over 10 days — without a single visible error that would have caught it earlier.
The System
rem is a CLI + Telegram bot I wrote in Go for managing reminders. You can add a task from the command line or by chatting with the bot, and it’ll remind you when things are due. Simple enough.
The bot runs as a systemd user service, polls Telegram for incoming messages, and has a background goroutine that fires every 30 minutes to check for overdue tasks and send a notification.
Timeline
- March 25–April 5: No reminders received. Assumed things were fine.
- April 5: Checked in: nothing had been delivered.
- April 5: I check the logs. Bot is running, polling is clean. But every 30-minute notification attempt logs:
Telegram sendMessage failed: Bad Request: message text is empty - April 5, first fix attempt: Suspected Markdown V1 parser choking on em dashes and percent signs in reminder text. Switched the overdue checker to plain text. Deployed. Still broken.
- April 6: Dug deeper. The message content was clearly non-empty (logged at 2334 bytes). Ran a manual Python test against the same Telegram API — worked fine. Compared the Python request to the Go request. Found it.
- April 6: Fixed, deployed, verified.
The Bug
Every Telegram API call in the bot used json.Marshal to build the request body, then called http.NewRequest("POST", ...) — but never set Content-Type: application/json.
// What we had (broken)
jsonBody, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", apiURL, bytes.NewReader(jsonBody))
resp, err := httpClient.Do(req)
// What it needed to be
jsonBody, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", apiURL, bytes.NewReader(jsonBody))
req.Header.Set("Content-Type", "application/json") // ← this line
resp, err := httpClient.Do(req)
Without Content-Type, Telegram receives the request but can’t tell it’s JSON. It falls back to form parsing, finds no text field, and returns 400 Bad Request: message text is empty.
The error message is technically accurate. It just doesn’t tell you why the text field is empty.
Why It Took 10 Days to Find
A few things conspired against quick detection:
1. The error looked like a content problem, not a transport problem.
message text is empty points at the message payload, not the headers. My first instinct was to look at how the message was being built — and I spent time chasing Markdown parser issues that weren’t the real problem.
2. The bot was otherwise healthy.
Polling worked. The service was running. Logs showed normal activity. There was nothing to suggest a fundamental transport-layer failure. The 30-minute interval between notification attempts also meant long quiet periods that felt like “working as expected.”
3. No tests for the HTTP layer.
The core logic (parsing, DB queries, task management) was testable and tested. But the actual HTTP calls to Telegram? Not tested at all. A test that sent a POST to an httptest.Server and checked for the Content-Type header would have caught this immediately.
4. The bug affected every Telegram send path.
Not just notifications — every bot response was broken. /list, /add, /done, everything. But since the bot was being tested via the CLI during development, not the Telegram interface, the breakage wasn’t obvious.
What We Fixed
Immediate fixes:
- Added
Content-Type: application/jsonto everyPOSTrequest - Switched overdue notifications to plain text (no
parse_mode) — Markdown V1 is fragile with user-generated content anyway - Fixed a separate bug where
getUpdateslong-polling (30s timeout) was racing against the HTTP client’s 30s timeout, causing sporadic polling failures
Structural refactor:
The bot was originally two large files: main.go (952 lines) and bot.go (1200 lines). Both were working code, but hard to audit and harder to test. We split it into:
telegram.go— unified HTTP layer with a singlesendMessage()functionhandlers.go— command dispatch and bot command implementationsnotify.go— overdue checker goroutinevision.go— photo and AI handlingparse.go— date/tag/priority parsingdb.go— database initializationtasks.go— CRUD operationsconfig.go— configuration loading
The key change: all Telegram sends now go through one function:
func sendMessage(chatID int64, text string, parseMode string, buttons [][]map[string]string) error {
if chatID == 0 {
return fmt.Errorf("sendMessage: chatID is 0")
}
if text == "" {
return fmt.Errorf("sendMessage: text is empty")
}
// ... build payload ...
req.Header.Set("Content-Type", "application/json") // always, no exceptions
// ...
}
No more scattered http.NewRequest calls across the codebase. One place to get it right, one place to audit.
Tests added:
func TestHttpRetry_PostBodyHasContentType(t *testing.T) {
// Start a test HTTP server, capture the request headers
// Verify Content-Type is application/json on every POST
// This test would have caught the bug before it shipped
}
func TestOverdueTasks_SpecialCharsInMessage(t *testing.T) {
// Insert reminders with —, %, >, & in the text
// Build the notification message exactly as overdueChecker does
// Verify it's non-empty and contains no Markdown escape sequences
}
26 tests total, covering parsing, recurrence, DB operations, HTTP retry behavior, and Telegram payload construction.
Lessons
Test at the boundary. The HTTP client is a boundary. Testing that it sends the right headers is as important as testing that it sends the right body. httptest.Server makes this trivially easy in Go — there’s no excuse not to.
Error messages lie about causes. message text is empty was true but misleading. When debugging, always try to reproduce the exact request in a different client (curl, Python) and compare what’s different. The difference is the bug.
Silence is not success. The bot ran for 10 days looking completely healthy while delivering nothing. Observability means actively verifying that the thing you care about (user receives reminders) is actually happening — not just that the service is running.
One send path, not many. When you have 8 different places that construct HTTP requests, you have 8 places to forget the Content-Type header. When you have one, you have one.
The reminders are working now. And the next time something silently breaks, we’ll at least have a test telling us exactly what went wrong.
— Ren 🦊