What Is Latency? – Definition and How It Works

Learn what latency is, how it differs from throughput, where a web request spends its time, and why percentiles matter.

Latency is the time between making a request and receiving a response. Throughput is how many requests complete per unit time. They are different quantities, and confusing them causes most capacity mistakes.

1. Latency Versus Throughput

A single request taking two seconds has high latency. A system completing 500 requests per second has high throughput. Both can be true at once — with enough concurrency, slow requests still produce high throughput.

The relationship is roughly throughput ≈ concurrency / latency. Which means: to raise throughput, either reduce latency or raise concurrency. For collection work, latency is largely set by the target, so throughput comes from concurrency — bounded by what the target tolerates.

2. Where a Request's Time Goes

Measure the phases separately. "The site is slow" and "our DNS resolver is slow" produce identical end-to-end numbers and need entirely different fixes.

curl -o /dev/null -s -w 'dns=%{time_namelookup} conn=%{time_connect} tls=%{time_appconnect} ttfb=%{time_starttransfer} total=%{time_total}\n' https://example.com

3. Connection Reuse Is the Biggest Easy Win

DNS, TCP, and TLS together cost several round trips, paid once per connection. A client that opens a fresh connection for every request pays them every time.

import requests
session = requests.Session()      # keep-alive, connection pooling
for url in urls:
    session.get(url, timeout=(10, 60))

Reusing a session frequently halves per-request latency on a short response, and it is a one-line change. It is the first thing to check in any client that seems slower than it should be.

4. Averages Hide the Problem

An average latency of 200 ms is compatible with 95% of requests at 100 ms and 5% at 2 seconds. Those 5% are the ones that time out, hold workers, and stall batches.

Report percentiles:

• p50 — the typical experience.
• p95 — where the slow tail begins.
• p99 — the pathological cases.
• max — the worst observed, which is what your timeout must accommodate.

Set timeouts from the p99, not from the mean. A timeout near the average fails a large share of legitimate requests.

5. Physical Limits

Latency has a floor set by distance: light in fibre covers roughly 200 km per millisecond, and a round trip doubles the path. Intercontinental round trips of 100–200 ms are physics plus routing, not inefficiency.

For collection this means proxy geography has a direct latency cost. Routing through a distant residential exit adds real time per request, and the added latency is often larger than the target's own processing time.

6. What Raises Latency in Collection Work

• Proxy hops. Every intermediary adds a round trip, and residential exits add more than datacenter ones.
• Fresh connections per request — the case above.
• Browser rendering, which turns a fast fetch into a slow page load.
• Waiting for network idle rather than for the specific element needed.
• Retry backoff, which is latency you chose but must still count.
• Queue wait, which is often the largest component and is invisible unless measured.

That last one is worth watching: measured end-to-end from enqueue rather than from request start, "latency" is frequently dominated by time spent waiting for a worker.

7. Reducing It

1. Reuse connections. Sessions and pooling.
2. Cache DNS, or use a local resolver.
3. Choose proxy locations near the target, not merely in a permitted country.
4. Request only what you need — a JSON endpoint instead of a rendered page.
5. Wait for the element, not for idle, in browser automation.
6. Raise concurrency where latency is irreducible and the target tolerates it.
7. Measure phases so effort goes where the time actually is.

8. Summary

Latency is per-request delay, throughput is completion rate, and with enough concurrency slow requests still yield high throughput.

Measure the phases separately, report percentiles rather than averages and set timeouts from the p99, reuse connections to avoid repaying DNS, TCP, and TLS costs — and remember that proxy distance and queue wait are frequently larger contributors than the target's own processing.

Potential Related Glossary

• Concurrency vs Parallelism
• Residential Proxies
• IP Rotation
• Web Crawling