Scrapeless Wiki

HTTP 404 Not Found – What It Means

Troubleshooting P1 http 404 meaning

Learn what HTTP 404 Not Found means, why a 404 does not always mean the page is missing, how soft 404s differ, and how to diagnose one properly.

404 is the best-known status code on the web and one of the most misread. It does not mean the page never existed, and it does not always mean the page is gone. It means the server found nothing to return for this URI right now β€” a narrower statement than most people hear.

1. What Does HTTP 404 Mean?

404 Not Found indicates the server could not find a current representation for the requested resource.

  • Key idea: the request was well-formed and the server reachable β€” there is simply nothing at that address.
  • Mechanism: routing found no match, or the handler determined the underlying record does not exist.
  • Notably: the specification does not require the server to say whether the absence is temporary or permanent. 410 Gone exists for deliberate permanence, and is rarely used.

The server is also permitted to return 404 to hide a resource it would rather not confirm exists. A 404 is therefore weak evidence of absence.

2. Common Causes

The URL is wrong. A typo, a stale link, a changed slug. The ordinary case.

The resource was deleted or moved without a redirect. The page existed, the URL was published, and nothing maps the old address to the new one.

Routing or rewrite misconfiguration. A single-page application deployed without a catch-all rewrite returns 404 for every route except the root, because the server looks for files that were never meant to exist.

Case sensitivity. /About and /about are different resources on most Linux servers and the same one on Windows and macOS. This is why a link works locally and 404s in production.

Trailing-slash mismatch. /docs and /docs/ can be distinct depending on configuration.

Authorisation disguised as absence. A resource you may not see, returned as 404 rather than 403 to avoid confirming it exists. Common in APIs handling private records, and entirely deliberate.

Content not yet published. A CMS entry in draft state has a URL that resolves for editors and 404s for everyone else.

Geographic or regional routing. The path exists in one locale and not another.

3. How to Diagnose It

Compare against a known-good URL on the same host. If everything 404s, it is routing or deployment. If only one URL does, it is that resource.

Try the URL unauthenticated and authenticated. A 404 that becomes a 200 with credentials was an authorisation decision wearing a 404's clothing.

Check the sitemap. A URL present in sitemap.xml but returning 404 signals a publishing or deployment problem, not a bad link.

Look at the response body and size. A short, generic 404 page is usually the server's. A full, branded page suggests the application handled the route and decided the record was missing β€” different layers, different fixes.

Watch for soft 404s. Some sites return 200 OK with a "page not found" body. Automated clients treat that as success and ingest the error page as content. Checking the status code alone is insufficient; a length or content check matters too.

Check redirects. curl -IL reveals whether the URL redirects to something that then 404s, which points at the redirect target rather than the original address.

4. How to Fix It

If the resource moved: issue a 301 to the new location. Permanent redirects preserve links and search ranking; deleting without redirecting discards both.

If it is genuinely gone forever: 410 Gone states that explicitly and stops crawlers retrying indefinitely.

If it is a single-page application: configure the server to serve index.html for unmatched routes so the client-side router can handle them.

If it is case or slash mismatch: normalise with a redirect rule rather than hoping every link is written correctly.

If it is a broken internal link: fix the link. Redirecting to paper over bad links accumulates rules nobody can later untangle.

Serve a useful 404 page with navigation and search. It will be seen, and a dead end is a lost visitor.

For automated clients: treat 404 as terminal for that URL. Do not retry β€” unlike 429 or 503, nothing is going to change by asking again.

Code Meaning Permanent?
404 Not Found No representation found Unstated
410 Gone Deliberately removed Yes
403 Forbidden Exists, but refused β€”
301 Moved Permanently Now lives elsewhere Yes
302 Found Temporarily elsewhere No
Soft 404 200 OK with error content A bug, not a code

6. Real-World Examples

  • Every route 404s except the homepage after deploying an SPA β€” the rewrite rule is missing.
  • A crawl returning thousands of 404s usually means URL construction is wrong, not that the site is broken. Check one manually before assuming the target changed.
  • 404 in the API, 200 in the browser. The browser is authenticated and you are not, and the API hides private records behind 404.
  • Search console reporting 404s for URLs nobody published β€” often from malformed links elsewhere, or a crawler assembling relative URLs incorrectly.
  • A scraper storing "Page Not Found" as a product title is the soft-404 failure, and it is silent because the status code said everything was fine.

7. Summary

404 means no representation was found for this URI, which is not the same as "this never existed" or "this is gone for good". It may be a typo, a missing redirect, a routing gap, or an authorisation decision deliberately disguised.

Diagnose by narrowing scope: one URL or all of them, authenticated or not, listed in the sitemap or not. Fix by redirecting what moved, using 410 for what is genuinely gone, and configuring rewrites for client-routed applications. For automated work the important defence is the soft 404 β€” always verify that a 200 actually contains what you expected, because the status line will not warn you.