Hedir Review : Feedback wanted on article

Señor COOL · Posted: Mon Mar 13, 2006 5:51 am Post subject: Feedback wanted on article

http://www.searchenginefriendlylayouts.com/assertivenet.asp

It's an explanation of the alleged "Assertivenet" bot and what it actually is.

If you guys could provide me with some feedback on the article--if it makes sense, if it's clear, if there's anything I may have missed (although I doubt the latter, since the proof is fairly obvious)--I'd appreciate it muchly.

samlewis · Joined: 28 Feb 2006 Posts: 56 1677 Hedir Points

I noticed a couple of spelling mistakes, otherwise the article is a good read into your new find of another search engine which you originally thought was malicious indexing of your sites in question.

Google Junky · SuperMOD Joined: 01 Jan 2006 Posts: 3868 286 Hedir Points

That was a very interesting article. I know of one solution for investigating a malicious crawl.

On the websites I design I add a custom 404 page. When the person, bot, etc cant find the file it wants, I then get an email that shows the path and file trying to be accessed, the ip address, and the time the occurrence happen.

Señor COOL · Posted: Thu Mar 16, 2006 1:57 am Post subject:

lakhya · Joined: 01 Mar 2005 Posts: 473 15238 Hedir Points

Just finished reading the article. Good one Adam Smile

The reason many people block unknown bots is the increasing activity of malicious programs. Gigabot indexes a large portion of the unknown web and hence blocking this spider can only lead to reduced traffic. Many webmasters have been doing this unknowingly and hopefully this article can help dispell the doubts about this bot.

Señor COOL · Posted: Sun Mar 19, 2006 6:58 am Post subject:

Thank you, kind sir. Smile

And yes, this is true. I tend to do the same myself. But I always try to look up what something is first...raw logs are a wonderful thing!

kappixteam · Joined: 17 Apr 2006 Posts: 265 10351 Hedir Points

Hi Adam,

Good article! My belated gloss of your article led to the following suggestions. I tried to be as lenient as possible. Apart from the typo, it is intelligible as is, mind you, but.....

From this information, Mark and I gathered that the owner of the spider in question appears to be a company called Assertive Networks, and is hosted through a company called "BC Hosting."

More information wass not immediately available.

Gigabot only crawled pages and files as other search engines have (do)

reason for 'do' vs. have = have refers to a pre-present verb, e.g. have done, while do refers to a habitual generally ocurring present.

While it is not on the same level in terms of popularity as the Big 3 (Yahoo!, MSN, and Google) , it has indexed a significantly large portion of the web, and can be useful for some searches.

Nancy

Brett · Posted: Fri Jun 02, 2006 10:36 am Post subject:

Interesting read