Objectives of the competitive analysis
The objectives of a competitive analysis vary from company to company. The objectives determine which information must be collected and which methods of information acquisition and processing are necessary

Possible objectives of the strategic competition analysis

Recognize how far competitors are in the technical development
Identifying the market share of competitors
Recognition of potential on the market (e.g. new fields of application, technological developments, approaches)
Identifying the strengths and weaknesses of competitors
Avoid that competitors overtake your company unrecognized. Aligning one’s own actions (positioning, adjustments,) with the information gained.
Knowledge of prices and offers of competitors in order to be able to align oneself with them.

Competition analysis
Strategic and operative competition analysis

The operative competitive analysis

Gain information for operational adjustments, e.g. prices, advertising channels
Sales arguments

Information that can be obtained with the help of a crawler in the market analysis

In principle, a crawler can read out all the information that a website provides. This includes not only the information that the normal user sees but also information from the structure of the page.

Which information is actually collected depends on the purpose. Frequently, new information can be derived from the collected data, which is explicitly not found on the website.

Information to be collected

The following information is only ideas. In fact, more information may be collected depending on the user’s interest:

  • Price
  • Product designation
  • Size/ weight of the product
  • Quantity (If specified)
  • Product number
  • Evaluation (amount, number of evaluations given)
  • Number of products/ forum entries/ categories
  • Number of subpages
  • If available ISBN or EAN
  • Picture
  • Category
  • What recommendations are displayed for this product
  • Poslt. whether special designation as special offer
  • Address, Managing Director
  • Length of texts e.g. the data protection declaration or general terms and conditions (changes indicate changes in content.)
  • Page structure (changes indicate changes in the offer)
  • Metadata
  • Number and provider of the included scripts
  • Loading speed

To enable an automatic comparison with own products, the product designations must be compared if no EAN is available.

Preparation of the data

Since designations for identical products can differ in spelling, similarity algorithms are necessary here. Data preparation is necessary to make them comparable and usable.


AEG Lavamat L 87484 EFL Eco or AEG Lavamat L 87484EFL Eco

Here only one space is missing between the digits and EFL. However, if the strings are compared automatically, they are recognized as different.

One possibility is to use a similarity algorithm. E.g. Levensthein algorithm

And to set a tolerance threshold up to which a product is considered to be alike.

Prior to this, various sources of error can be eliminated, for example, by making no distinction between upper and lower case letters. In addition, spaces could not be counted, or certain characters that differ, such as letters or numbers, could be weighted more heavily.

The exact assignment is necessary so that the algorithm for dynamic price adjustment does not draw the wrong conclusions.

Even if the data is not used for dynamic pricing, but for determining the price strategy or pricing policy, you must be able to rely on its reliability, especially with large amounts of data.

Example 2:

(A) AEG Lavamat L 87484 EFL Öko To be compared with:

(B) AEG Lavamat L87484EFL Eco Two spaces have been left out here.

(C) AEG Lavamat L 97484 EFL Öko Here a 9 was used instead of the 8 in the numerical part. This can therefore either be a completely different product or a spelling mistake by the website operator.

(D) AEG Öko Lavamat L 87484 EFL Here the position of the word Öko has been changed.

In a simple comparison, B and C would not be recognized as being the same as product A.

In the Levenshtein algorithm B would be more dissimilar to A than C, because there is only a deviation from the comparison string. This means that despite the fact that a difference in one digit has a greater meaningful value than space, the product name of C is considered more similar to product B.

The product name D would have the greatest distance to the name A if the Levensthein algorithm were applied, although only the position of the word Öko was changed. However, this is not an indication of different products.

It is therefore recommended to perform a permutation of different components. FOR EXAMPLE

(P0) AEG Lavamat L 87484 EFL Eco

(P1) Eco