Managing Google s bot crawling

Transcription

1 Managing Google s bot crawling USE CASE 1001pneus Industry: E-commerce Date: November

2 TABLE OF CONTENT 03 About 1001pneus 05 How to manage Google s bot crawling? 06 Using log analysis to monitor Google s behavior 07 Identifying crawl gaps 08 Understanding the SEO impact 09 About OnCrawl 2

3 ABOUT 1001PNEUS The heart of 1001pneus.fr s activity is the tire. 1001pneus is a pure player present on the web for more than 7 years. Today, they are the No. 2 online tyre retailer in France. Online sales of tyres now account for 17% of market sales in France. This low share of the online market is mainly due to a reflex still marked among French consumers. Most of whom continue to buy their tyres in physical shops. However, the trend seems to be gradually changing. The reason is simple: buying tires is rarely associated with a purchase called «pleasure», so the willingness to go online to find the right deal seems natural. The online tyre market is very competitive: between the «pure players» with a high reputation, the new entrants - often very aggressive - and the click & mortar such as Norauto or Feu Vert, the competition is fierce. Pierre Bruat, SEO Project Manager at 1001pneus, is a 27-year-old Bordeaux wine lover (South West obliges) and passionate about the web, new technologies or rugby. He has been working for 1001 tires since His job is to improve the natural visibility of the site on search engines. He was contaminated by SEO a few years ago, working for another e-commerce site. He has done his homework there and since then he has never left this field. This passion was built around the idea that SEO is like an endurance race. The goal is to pass over the other sites and stay ahead. And as in sports, there are opponents who play by the rules and those who circumvent them. Google being the referee, the one who distributes «penalties» to stay in the vocabulary of rugby (and Google). 3

4 KEY FACT N 2 online tyre retailer in France product references Born in M turnover in

5 HOW TO MANAGE GOOGLE S BOT CRAWLING? In the SEO sphere, it is often common to refer to offpage and on-page optimizations. For 1001pneus, before working on off-page aspects, they devote a large part of their time to optimize the site s strategic pages. Apart from optimizations related to content, meta tags and other tags, they ensure that their strategic pages are highlighted for Google when it visits the website. To favour Google s crawl, it is necessary to work on internal links for two reasons: It helps the transmission of the link juice between pages It links these pages between them. Thanks to this lever, it is possible to increase the popularity of certain pages of a website. Why is it necessary to ensure the crawl s quality? It s no longer a secret, Google s bot has a crawl budget to explore each site. It is therefore advisable to optimize the crawl to favour strategic pages to the detriment of certain pages called «useless». Working on internal links is now at the heart of strategies for those who work in SEO. Pierre Bruat has noticed this with the Topical Mesh (a concept created by Laurent Bourrelly) based on the implementation of a «surgical» internal linking between pages of a site. To sum up, the work done on internal links allows to guide Google s bot towards strategic pages. Therefore, the quality of the crawl must be monitored. To monitor and control Google s bot crawl, it is necessary to use a log analysis tool. 5

6 USING LOG ANALYSIS TO MONITOR GOOGLE S BEHAVIOR OnCrawl allows you to analyze Google s bot behavior on a site by transforming log files data into relevant statistics. It is a tool that Pierre first tried and finally adopted to accompany him in his decision-making. 1001pneus and OnCrawl both being based in Bordeaux, France, it seemed natural for Pierre to meet us. It seemed all natural that I should meet François Goube so that he could introduce me OnCrawl and the log analysis tool. A SEO log analysis tool allows you to follow, on a daily basis, the behaviour of Google s bot on a site. Through OnCrawl s SEO log analyzer, a lot of useful informations are revealed, such as: The number of times Google s bot has passed on a site (bots hits); The number of unique pages viewed by Google (unique crawled pages); New crawled pages (newly crawled pages); The crawl frequency. Moreover, it is possible to isolate data according to determined pages categories. Crawl behavior by group 6

7 IDENTIFYING CRAWL GAPS Using log analysis tool, 1001pneus was able to identify numerous optimizations such as, for example, an imbalance in the crawl. Indeed, a group of «useless» pages for SEO purposes captured more than a third of Google s crawl... What a waste! By correcting this, in particular by blocking the crawl on these pages and working on the internal linking, They freed up some «crawl time» for the more strategic pages. In orange, the volume of Google s crawl on «useless» pages and in pink, the volume of Google s crawl on a group of strategic pages are represented. The moment when they blocked the crawl of «useless» pages can be observed. This manipulation gradually increased the crawl on strategic pages (in pink). Other strategic pages have also benefited from a larger volume of crawl following their intervention. Before and after optimizations crawl budget being waste on orange parts 7

8 UNDERSTANDING THE SEO IMPACT OnCrawl s log analyzer also has an SEO impact section that provides tables and other graphs to give an overview of the current status of active pages, crawl rate per active page or active pages not crawled by Google. With OnCrawl, we were able to identify active pages that were not crawled by Google, in order to link them to the site. Active pages generating organic traffic 8

9 ABOUT ONCRAWL Today, along with some of the most important SEO tools, OnCrawl has become a «must have» of everyday life. The efforts of the OnCrawl team in presenting the data should be recognized. Often neglected, the presentation of tables and graphs with OnCrawl offers a real ease of reading. Pierre Bruat, SEO Project Manager at 1001pneus 9

10 Give a go to OnCrawl with a 14 days free trial Start your free trial today 10