User-agent: * Disallow: /admin Disallow: /to/* User-agent: YandexNews Allow: /feed/yandex Crawl-delay: 0.1