Skip to main content

Structured Logging: The One Observability Investment That Paid Off

1 min read
observability go infrastructure

The before

2025/12/20 14:23:01 ERROR: failed to fetch user

What user? What request? What service? grep through 10,000 lines to find out.

The after

{"level":"error","time":"2025-12-20T14:23:01Z","trace_id":"abc123",
 "service":"user-svc","user_id":"usr_abc123","error":"connection refused",
 "duration_ms":2503}

The implementation

// slog structured logging (Go 1.21+)
logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
    Level: slog.LevelInfo,
}))

// Attach request-scoped fields
r = r.WithContext(slog.NewContext(r.Context(),
    logger.With("trace_id", traceID, "request_id", reqID)))

What changed

  • Debugging: 15 minutes → 2 minutes
  • On-call fatigue: down significantly
  • We could finally build dashboards from log fields

What I learned

Structured logging is the highest ROI observability investment you can make. It doesn’t require a vendor, a platform, or a dedicated team — just a JSON encoder and a convention. Everything else (metrics, tracing) builds on top of it.