К основному контенту

Robots.txt Template

Minimized template (no comments). Robots.txt file templates are whitelisted.

The robots.txt template files contain an alphabetically ordered white list of legitimate web robots.

----------------------- ROBOTS.TXT -------------------------

User-agent: 360Spider

User-agent: 360Spider-Image

User-agent: 360Spider-Video

User-agent: AdsBot-Google

User-agent: AdsBot-Google-Mobile

User-agent: AdsBot-Google-Mobile-Apps

User-agent: adidxbot

User-agent: Applebot

User-agent: AppleNewsBot

User-agent: Baiduspider

User-agent: Baiduspider-image

User-agent: Baiduspider-news

User-agent: Baiduspider-video

User-agent: bingbot

User-agent: BingPreview

User-agent: BublupBot

User-agent: CCBot

User-agent: Cliqzbot

User-agent: coccoc

User-agent: coccocbot-image

User-agent: coccocbot-web

User-agent: Daumoa

User-agent: Dazoobot

User-agent: DeuSu

User-agent: DuckDuckBot

User-agent: DuckDuckGo-Favicons-Bot

User-agent: EuripBot

User-agent: Exploratodo

User-agent: Facebot

User-agent: Feedly

User-agent: Findxbot

User-agent: Googlebot

User-agent: Googlebot-Image

User-agent: Googlebot-Mobile

User-agent: Googlebot-News

User-agent: Googlebot-Video

User-agent: HaoSouSpider

User-agent: ichiro

User-agent: istellabot

User-agent: JikeSpider

User-agent: Lycos

User-agent: Mail.Ru

User-agent: Mediapartners-Google

User-agent: MojeekBot

User-agent: msnbot

User-agent: msnbot-media

User-agent: OrangeBot

User-agent: Pinterest

User-agent: Plukkie

User-agent: Qwantify

User-agent: Rambler

User-agent: SeznamBot

User-agent: Sosospider

User-agent: Slurp

User-agent: Sogou blog

User-agent: Sogou inst spider

User-agent: Sogou News Spider

User-agent: Sogou Orion spider

User-agent: Sogou spider2

User-agent: Sogou web spider

User-agent: SputnikBot

User-agent: Teoma

User-agent: Twitterbot

User-agent: wotbox

User-agent: yacybot

User-agent: Yandex

User-agent: YandexMobileBot

User-agent: Yeti

User-agent: YioopBot

User-agent: yoozBot

User-agent: YoudaoBot

Disallow:

User-agent: *

Disallow: /

Комментарии

Отправить комментарий

Популярные сообщения из этого блога

Site re-linking

click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here click here

Website promotion in several languages

 How can multilingualism be implemented ? Through different domains. This method is considered the most effective, but at the same time the most complex and costly. In fact, we need to make a new site in a specific language. Implementation example:  https://150sitemaps.blogspot.com/ru https://150sitemaps.blogspot.com/en What should be on a multilingual site. Hreflang attribute: <link rel="alternate" hreflang="language code" href="URL pages" />

Page creation AMP HTML

 Required markup Start with document type <!doctype html>. Contain top level tag <html ⚡> (also allowed to use <html amp>). Contain <head> and <body> tags (optional in HTML markup). Include a <link rel="canonical" href="$SOME_URL"> tag within the header that points to the regular HTML version of the AMP HTML document, or to the original document itself if no such version exists. Contain the <meta charset="utf-8"> tag as the first child of the header tag. Contain the <meta name="viewport" content="width=device-width"> tag in the header tag with the value initial-scale=1. Contain as the last element of the header a <script async src="https://cdn.ampproject.org/v0.js"></script> tag that specifies and loads the library AMP JS. Contain the following code in the <head> tag: <style amp-boilerplate>body{-webkit-animation:-amp-start 8s steps(1,end) 0s 1 normal both