Hedir Review : Robots exclusion

DeySuman · Joined: 14 Dec 2005 Posts: 2 131 Hedir Points

I want to know what is meant by and it's purpose-

User-agent: Mediapartners-Google
Disallow: /

Any help from community members?

susan · Posted: Fri Dec 23, 2005 7:23 am Post subject:

This is also a robot used by Google as a part of its Adsense programme. As a bot Mediapartners-Google will frequently visit your site if your site using Google Adsense. This robot also supports the standard robots.txt exclusion protocols.

Example: To disallow this robot to visit cgi-bin directory:

User-agent: mediapartners-google
Disallow: /cgi-bin