The Night You Let Go of console.log: Five Questions to Answer Beforehand

The night you let go of console.log usually lands on the first weekend after deploy. Swapping out the library itself takes about an hour. The problem is what comes next. You've adopted structured logging, yet the log schema is split in two, the levels are by nobody's standard, and sensitive data sits whole somewhere in the collector. The library isn't at fault. It's just that there were that many decisions you never answered before the switch.

A migration isn't a tools problem; it's a chain of decisions. When a line of text becomes JSON, you have to choose what to put in it, what criteria to store by, and who gets to see it. If you don't answer those five things now, you'll pay the price of tearing apart your log schema later. These aren't things to decide on the night of the migration; they're the answers you should collect beforehand.

Levels start with definitions and end with alerts

Who sets log levels? Most teams skip this question and just use the library defaults. What's left is only a vague sense that debug should be used sparingly and error should be used emphatically. With no standard, three developers classify the same event three different ways, and the level quickly becomes a meaningless string. When that happens, alert thresholds die with it. When an alert that fires on every error piles up thousands of times a day, people start turning off alerts entirely the next day.

Standards live longest when they're short. info is where the flow changes, warn is where it auto-recovers but deserves a closer look, error is where users have already been affected. If the whole team settles those three lines together, half the job is done. One caution: don't draft these definitions alone. Log levels should be defined by the standard of the systems that consume them, not by the people who write them. Once defined, verify. Pick a couple of events and have two people classify them, then check the agreement rate; that's the test for the day before the migration. In the first week after the switch, add to your routine a check that alerts are alive and haven't gone quiet.

Mismatched field names leave bruises on your dashboard

The second question is which fields to put in, and under what names. The most common mistake in log field design is running the same meaning under two names. orderId and order_id, createdAt and created_at. It started as one, but at some point both exist, and the aggregation query only picks up one. If your dashboard graph suddenly breaks in the middle, it pays to suspect field names first.

For fields you'll pull in production, it's better to add them sparingly at the start and scrutinize additions. request_id, trace_id, service name, version. Four fields are enough to start. The moment you put a value whose cardinality grows without limit — a timestamp, an instance name, an individual identifier — into a label, storage and indexing get more expensive by that label count. Rather than adding ten fields on adoption day, adding four today and one at a time when needed keeps the schema alive longer. But your field naming rules have to be documented. A naming convention can't be enforced by code review.

Masking only works right before transport

Of the five questions, masking is the only one where being late is irreversible. Field names can be fixed; level standards can be reset. Logs already shipped to the collector can't. The moment the original text lands in storage, backups, or somewhere in the pipeline, it's over. So the migration spec must contain one line: masking is applied at the point where the logger emits the log, immediately before it goes out.

const safe = maskFields(record, ["authorization", "cardNumber", "token"]);
logger.info("order.created", safe);

maskFields runs right before the record passes through the logger. If that line sits behind the transport library, or in some other pipeline, it's already too late. Late masking is useless anyway.

Failure modes mostly come from pattern-based masking. Well-formed values like emails or card numbers are easy for regex to catch. The danger in structured logging isn't well-formed values; it's tokens hidden inside custom payloads. If you log an entire HTTP response body, it can contain a token that's neither a JWT nor a standard header format. No pattern can catch that. That's why masking needs both patterns and a field blacklist. The blacklist works by name; patterns are a safety net for values with no format. Make it a passing condition of the migration PR to run a regression test that applies the masking rules to a set of copied production logs and scans for surviving sensitive values. Just because a regex passes on staging data doesn't mean it passes in production. Production data is far messier than staging.

Buying without knowing what a day of logs costs

The fourth question: how much cost and storage does a day of logs eat? Few teams do the math before migrating. The estimate is simple. Multiply daily call volume by line count and average number of fields. At 500 bytes per log and 10 million calls a day, that's 5GB. On a monthly basis, spending stacks up across four directions: transport, storage, indexing, and querying. Log cost is ultimately the product of volume, cardinality, and retention period.

Retention depends on regulations and query frequency. There's no single right answer here. But this holds regardless of context: the moment you put a value that grows without limit into a field, you reach a point where even shortening retention won't keep budget in check. If you want to cut volume, start with sampling. error at 100%, debug at 1%, everything else in between. Sampling is easier to manage when it happens at the point of emission rather than being reduced in the collector after transport. The reduction rules then come up in review alongside the code.

Who is the person reading this log?

The last question is the consumer. Is the reader a person, or a system firing alerts? The fields a developer wants to see locally aren't the same fields operations uses for alerts. Mix both into one stream, and you reach a point where level filters alone can't sort it out. The incident of mistaking a high-cardinality debug field for an operations field and feeding it into aggregation happens right there. The separation rule is one line: does a system depend on this log? If an alert hinges on it, it's the operations stream. If it's just a human digging around out of frustration, it's the debug stream.

From an observability perspective, logs are best designed backward from their destination. First map out which alerts this event will trigger, which dashboards it will fill, which queries it will answer — then pick the fields. Skip that process, and you accumulate logs that only raise the query count without answering anything. That loops right back to question one, the chaos of levels.

In the end, touching logs in dev-ops isn't a tool replacement; it's a decision. You just attach rules to the migration PR itself. A structured logging adoption PR should carry the answers to the five questions as review comments: the level definitions table, the field spec and the reasoning behind the names, the masking regression test results, the daily volume estimate, and the stream separation criteria. Let go of console.log only on the night when all five answers are in place. If even one answer is unusable, that night hasn't come yet. You'll regret it two months from now anyway.

Comments

Loading comments.