FrancyneGilligan187

2015-03-26T03:26:24Z

FrancyneGilligan187：以“Sometimes, we might want search engines never to index certain elements of the site, as well as ban other SE in the site all together. Identify supplementary informa...”为内容创建页面

Sometimes, we might want search engines never to index certain elements of the site, as well as ban other SE in the site all together. Identify supplementary information about [http://www.imwwy.com/youtube-why-its-therefore-popular/ YouTube Why It’s Therefore Popular Imwwy] by browsing our refreshing website. If you are interested in shopping, you will likely hate to learn about [http://www.ykszxxz.com/video-sharing-websites/ Keep Smilling Video-sharing Websites]. That is the place where a simple, little 2-line text file called robots.txt is available in. Once we have a website up and running, we need to make sure that all visiting search-engines can access all the pages we want them to look at. Browse here at the link [http://www.flowerscanadaonline.com/how-to-download-and-play-free-game-activities/ click here for] to learn when to provide for this enterprise. Sometimes, we might want search engines to not catalog certain parts of the site, or even ban other SE in the site completely. This is the place where a simple, little 2-line text file called robots.txt is available in. Click this link [http://www.lawofficewoburn.com/the-fundamentals-of-adding-video-on-your-site-6/ www.youtube.com/user/1orangecountyseo] to compare the meaning behind this thing. Robots.txt lives within your sites main directory (o-n LINUX systems this can be your /public_html/ directory), and looks something just like the following User-agent * Disallow The first line controls the bot that will be visiting your site, the 2nd line controls if they're allowed in, or which elements of the site they're maybe not allowed to see Then easy repeat the above mentioned lines, If you'd like to take care of multiple spiders. So an example User-agent googlebot Disallow User-agent askjeeves Disallow / This can enable Goggle (user-agent name GoogleBot) to go to every page and index, while in the sam-e time banning Ask Jeeves from the site fully. To discover a fairly current set of robot consumer names this visit http://www.robotstxt.org/wc/active/html/index.html Even though you need to allow every software to index every page of your site, its still very advisable to put a robots.txt report in your site. It will end your problem logs filling with records from search-engines attempting to access your robots.txt file that doesnt exist. To find out more on robots.txt see, the full list of resources about robots.txt at http://www.websitesecrets101.com/robotstxt-further-reading-resources.

NoteExpress知识库 - 用户贡献 [zh-cn]

FrancyneGilligan187