You're right that all our defenses are down. We were blocking some at one point, but that file was lost along the way. What do you think? Scooter is the alta-vista bot. I remember we found that it wasn't as well behaved as the google bot (it hit a lot harder.)
But bandwidth isn't really a problem any more. I guess it might be slowing things down, but I sort of doubt it. Or at least not very much.
I'm all for reblocking the turn-it-in and mail syphon bots. I'll try to get a handle back on those.
But should we block alta-vista? I never use it, but there is an argument for letting it index, since we don't want google to be the only one with an index, right? In that same vein I'd think we'd want to keep inktomi as well (yahoo bot.)
I don't really feel too strongly either way. Tell me what you think. Or anyone else?
|
But bandwidth isn't really a problem any more. I guess it might be slowing things down, but I sort of doubt it. Or at least not very much.
I'm all for reblocking the turn-it-in and mail syphon bots. I'll try to get a handle back on those.
But should we block alta-vista? I never use it, but there is an argument for letting it index, since we don't want google to be the only one with an index, right? In that same vein I'd think we'd want to keep inktomi as well (yahoo bot.)
I don't really feel too strongly either way. Tell me what you think. Or anyone else?
- jim 4-25-2003 7:17 pm