Scrapeless Wiki

What Is Competitive Intelligence? – Definition and Practice

Informational P2 what is competitive intelligence

Learn what competitive intelligence covers, which public sources support it, how to keep it lawful, and why analysis matters more than collection.

Competitive intelligence is the systematic collection and analysis of public information about competitors and markets, to inform decisions. Its defining constraint is that it is built from public sources.

1. What It Is Built From

Source Yields
Public websites Positioning, pricing, product range, claims
Job postings Hiring direction, technology choices, expansion
Regulatory filings Financials and strategy, where required
Patents and trademarks Development direction
Reviews Product strengths and recurring complaints
Search results Which competitors dominate which topics
App stores Release cadence and feature changes
Public social content Messaging and campaign timing

Job postings are the most consistently underused: they describe intentions before products exist.

2. Collection Is the Easy Part

Assembling a large volume of competitor pages is straightforward. Turning it into something a decision rests on is not, and that gap is where most programmes fail.

The discipline is to start from the decision β€” pricing, positioning, roadmap, market entry β€” and collect only what informs it.

3. Track Change, Not State

A snapshot of a competitor's pricing is mildly interesting. A record of how it changed over six months, and what changed alongside it, is a finding.

That means storing timestamped observations rather than overwriting current values, which is a data-modelling decision made at the outset or not at all.

4. Keep It Lawful and Ethical

  • Public sources only. Content behind authentication is not public.
  • No misrepresentation. Do not pose as a customer, partner, or applicant to obtain information.
  • No confidential information, however it arrives.
  • Respect site terms and robots.txt.
  • Handle personal data under applicable privacy law, including employee information from job postings and profiles.

Competition law is the one people underestimate: observing competitor pricing is ordinary market intelligence, while using it to coordinate is not. Keep the use on the observation side.

5. Normalise Before Comparing

Competitors describe the same things differently β€” different plan names, bundles, units, and currencies. Comparison requires normalising into shared dimensions first.

Skipping this produces comparisons that look rigorous and compare unlike things, which is worse than no comparison at all.

6. Record Provenance

Store the source URL, the collection timestamp, and where content varies by region, the location. A finding that cannot be traced to its source cannot be checked when someone disputes it.

Screenshots or stored raw pages are worth keeping for claims that will be quoted internally.

7. Report Honestly

State what was observed, when, from where, and what was not observed. A competitor's public pricing is not their actual pricing for negotiated deals, and a report that does not say so invites a wrong conclusion.

8. Summary

Competitive intelligence assembles public information β€” websites, job postings, filings, reviews, search results β€” into analysis that informs a specific decision.

Start from the decision, track change rather than state, normalise before comparing, record provenance, stay on public sources, and be explicit about the limits of what public data can show.