What Is Observability? – Definition and How It Works
Learn what observability means, how logs, metrics, and traces differ, and which signals reveal a silently failing crawler.
Observability is the property of a system whose internal state can be understood from its external outputs. The practical test is whether you can answer a question you did not anticipate, without deploying new code.
1. Observability Versus Monitoring
Monitoring watches known conditions: is the process up, is the error rate above a threshold. It answers questions you thought of in advance.
Observability lets you ask new ones: why did extraction quality drop for one source, on one day, at one concurrency level, from one region?
The difference is the cardinality of the data you keep. Monitoring aggregates; observability retains enough dimension to slice.
2. The Three Signals
| Signal | Answers | Cost |
|---|---|---|
| Logs | What happened, in detail | High volume |
| Metrics | How much, how often, aggregated | Cheap, low cardinality |
| Traces | Where time went across a request path | Moderate, sampled |
Metrics tell you something changed. Logs tell you what. Traces tell you where. Any one alone leaves a gap.
3. The Signals That Matter for Collection
Standard service metrics — uptime, latency, error rate — miss the failure mode that matters most, which is a pipeline that succeeds while collecting nothing.
The signals that catch it:
| Metric | Why it matters |
|---|---|
| Records extracted per page | The direct measure of extraction health |
| Pages with zero records | The silent-failure indicator |
| Field fill rate, per field | Catches one selector breaking while others hold |
| Response size distribution | A shifted distribution means changed pages |
| Status code mix, per domain | Reveals blocking before it becomes total |
| Challenge rate | Rising means detection |
| Duplicate rate | Rising means pagination broken |
| Freshness of the newest record | Catches a pipeline stopped without erroring |
The second row is the important one. A crawl returning 200s and extracting nothing looks perfect on every conventional dashboard.
4. Structured Logs, Not Prose
import json, logging
def log_event(**fields):
logging.info(json.dumps(fields, default=str))
log_event(event="page_parsed", url=url, status=200,
bytes=len(html), records=len(records),
duration_ms=elapsed, proxy_region=region, run_id=run_id)
A structured line is queryable: records-equals-zero grouped by domain is a search rather than a grep of English sentences. Free-text logs cannot answer questions nobody anticipated, which is the entire point.
5. Cardinality Is the Constraint
Metrics with high-cardinality labels — a URL, a session ID, a proxy address — explode the number of series and become expensive or get dropped.
The workable split: low-cardinality labels on metrics (domain, status class, region), high-cardinality detail in logs and traces where it can be searched rather than aggregated.
6. Traces for Multi-Stage Pipelines
A trace follows one unit of work across stages: queue wait, DNS, connect, TLS, response, parse, store.
Its value is showing where time actually goes, which is routinely surprising. A pipeline believed to be network-bound often turns out to be spending most of its time waiting for a queue slot or parsing — and that is not visible in any per-stage metric.
7. Alerting on the Right Things
Alert on outcomes, not mechanisms:
- Zero records extracted for a source that normally produces some.
- Field fill rate dropping below a baseline.
- Freshness exceeding its expected interval.
- Challenge rate crossing a threshold.
- Dead-letter depth growing.
Do not alert on individual request failures. Some proportion always fails, and an alert that fires constantly is one nobody reads.
8. Summary
Observability means being able to answer unanticipated questions from a system's outputs, using metrics to detect change, logs to explain it, and traces to locate it.
For collection, the conventional signals miss the characteristic failure — success with no data — so instrument records per page, zero-record pages, per-field fill rates, and record freshness, keep structured logs for high-cardinality detail, and alert on outcomes rather than on individual request failures.