Yandexbot IP address segment

YandexBot  yes Yandex Search engine crawler .Yandex Is a Russian Internet company , The largest search engine in Russia , It has about... In the country 60% Market share . By 2012 year 4 month ,Yandex Ranking the fifth largest search engine in the world , More than 1.5 Billion searches , More than 2550 ten thousand .

Netizens often ask me how to block yandexbot, Or how to query yandexbot ip range（yandex ip Address segment ）？

In this article, I will introduce these two problems in detail .

How to shield yandexbot

We can go through robots.txt shielding yandexbot The reptiles of , Here is an example ：

User-agent: Yandex

Disallow: /

So we can shield yandexbot Grab all pages .

If we just want yandexbot Grab some pages , The following is an example of writing ：

User-agent: Yandex

Allow: /public

Disallow: /

Above except  /public  Page allowed to be yandexbot Outside of the visit , Everything else is forbidden yandexbot  Reptiles Grab .

adopt IP Address segment shielding yandexbot

yandexbot Of ASN yes  AS13238、AS208722  these two items. , We can go through  ASN Inquire about obtain ASN Of IP Address segment .

Will search for IP Add the address to the blacklist to block yandexbot The purpose of .

At the same time, we can find Crawler query This tool , Direct input IP Address , You can judge IP Whether to belong to yandexbot Of .

summary

This article mainly introduces shielding yandexbot Two methods of , One is ： adopt robots.txt agreement , One is through yandexbot ip range To shield

Of course, because yandexbot It belongs to search engine crawler , It can bring us traffic , So in yandexbot Without affecting the performance of your server , Try not to block it .