# All robots will spider the domain User-agent: * Disallow: /images/ Disallow: /media/ # Just for Google User-agent: googlebot Disallow: *.jpg$ Disallow: *.gif$ Disallow: *.pdf$