[collideous]

I changed my user agent info and was surprised to see a blank page when browsing to the forum. Do you keep bots out on purpose?

+++GET 11369+++
GET / HTTP/1.1
Host: forums.clubtouareg.com
User-Agent: Googlebot/2.1 (+http://www.google.com/bot.html)
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Cookie: ...
Cache-Control: max-age=0
Connection: keep-alive
Browser reload detected...

+++RESP 11369+++
HTTP/1.1 200 OK
Date: Fri, 23 Jul 2004 14:43:43 GMT
Server: Apache/1.3.31 (Unix) PHP/4.3.8 mod_ssl/2.8.18 OpenSSL/0.9.7c-p1
X-Powered-By: PHP/4.3.8
Content-Encoding: gzip
Pragma: no-cache
Transfer-Encoding: chunked
Content-Type: text/html
<start> 11369: Kill pop-up windows
<start> 11369: Force pop-ups to have browser controls
<start> 11369: Suppress all JavaScript errors
<start> 11369: Stop browser window resizing
<start> 11369: Stop status bar scrollers
<end> 11369: Restore pop-ups after a page loads
+++CLOSE 11369+++

 

[Mike]

Nope - very odd - googlebot has heavily spidered the site - i'll research this

 

[collideous]

Nope - very odd - googlebot has heavily spidered the site - i'll research this

Google often visits in disguise to prevent cloaking.

 

[Mike]

I'm well aware. Thanks =)

 

[Mike]

The real googlebot:
64.68.82.33 - - [23/Jul/2004:05:07:17 +0000] "GET /ftopic1379.html HTTP/1.0" 200 35499 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
64.68.82.159 - - [23/Jul/2004:05:21:15 +0000] "GET /ftopic1369.html HTTP/1.0" 200 48893 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
64.68.82.181 - - [23/Jul/2004:05:21:20 +0000] "GET /ftopic1277.html HTTP/1.0" 200 42083 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
64.68.82.142 - - [23/Jul/2004:05:21:26 +0000] "GET /album_comment.php?pic_id=347 HTTP/1.0" 200 17842 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
64.68.82.136 - - [23/Jul/2004:06:02:03 +0000] "GET /ftopic1307.html HTTP/1.0" 200 54733 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
[07/23/2004 12:50]

My Firefox-spoofed Googlebot attempt:
67.117.xxx.xxx - - [23/Jul/2004:19:54:38 +0000] "GET /ftopic1307.html HTTP/1.1" 200 0 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

Using wget spoofing attempt:
216.149.213.2 - - [23/Jul/2004:19:56:28 +0000] "GET /ftopic1307.html HTTP/1.0" 200 54735 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"


Looks like a problem with the way Firefox spoofs the useragent =)

 

[collideous]

Looks like a problem with the way Firefox spoofs the useragent =)

Same happens in Internet Explorer if I spoof the user agent through my proxy.

Space Bison/0.02 [fu] (Win67; X; Special Edition)