Owners of Russian-language Internet resources and webmasters should already be aware of the news from Yandex regarding the robots.txt file, which is related to the site indexing.
Previously, it was imperative that the host directive be correctly entered in the robots.txt file, which we recommended to our customers in case of its absence or incorrect indication. But since it does not matter for Google, and since 03/20/2018 Yandex itself has announced in its blog that it is refusing this directive, while the main important condition is that there should be 301 page redirects on all existing NOT main mirrors of yours site.
If you decide, for whatever convictions, leave the host in robots.txt in the old manner or you are just afraid to take such a step, there will be nothing terrible in this, the point is that the robot will only ignore it, in this case it’s just no role will not play. Therefore, we simply recommend that you correctly prescribe what should be and not add anything extra.
Despite this, some webmasters even manage to show their creativity and make it truly unique even in such a simple but necessary robots file. And look at the file of the well-known company Nike …
Not that we set it as an example to follow, just get inspired and be the best in your business.