Holy cow we are being demolished by the googlebot! It is going crazy and absolutely pegging our bandwidth with bizarre page requests like:
/treehouse/date/2000/systemnews/index.php3/mrwilson/rachael/ mrwilson/arboretum.php3/dave/systemnews/index.php3/
WTF?
Unfortunately, instead of generating a 404 (since that obviously is not a real page) this requests generates the entire year 2000 archive for the /treehouse page (a huge transfer!) The system just sees /treehouse/date/2000/ and can't make anything of the rest of it, so serves the whole year.
Ouch.
I have temporarily disabled entire year archives. We'll see if that helps. Damn. I should have caught this a few days ago, but I wasn't paying close enough attention. Might cost me. Literally.
Ouch, indeed. Keep us posted.
On closer inspection, all of the bizarre page requests seem to contain /index.php3/ somewhere in them, and since there are no pages with that in the path (or any .php3 files at all,) I am just 404ing any request that contains index.php3. Hopefully google will eventually get the message.
Does the robot learn from its mistakes?
It does, but in my experience not so well.
That did it. Tragedy averted. Phew.
good work Jim
Thanks for your effforts. Do you think you could do anything about Katrina now?
like get it a different hooker name?
Please, please, "tantric specialist."
sorry to be flip, i just found out katrina is gathering with a direct n.o. trajectory that could trigger the ponchartrain effect. this isnt the first time they used the superdome for emergency shelter. next time they build a stadium they should build it up strong and high and with real consideration of its dual purpose as sportatorium and an emergency shelter for the huge number of people too poor to beat it out of town.
I used the exact same expression, "flip," in my apology to a New Orleanian I had yesterday emailed to jokingly about the storm, then today I decided yesterday's comments were too flip. Things often change last minute with these big hurricanes but it sure does look bad right now. Tomorrow at this time we'll know how bad, or not.
|
/treehouse/date/2000/systemnews/index.php3/mrwilson/rachael/ mrwilson/arboretum.php3/dave/systemnews/index.php3/
WTF?
Unfortunately, instead of generating a 404 (since that obviously is not a real page) this requests generates the entire year 2000 archive for the /treehouse page (a huge transfer!) The system just sees /treehouse/date/2000/ and can't make anything of the rest of it, so serves the whole year.
Ouch.
I have temporarily disabled entire year archives. We'll see if that helps. Damn. I should have caught this a few days ago, but I wasn't paying close enough attention. Might cost me. Literally.
- jim 8-25-2005 9:14 pm
Ouch, indeed. Keep us posted.
- tom moody 8-25-2005 9:28 pm [add a comment]
On closer inspection, all of the bizarre page requests seem to contain /index.php3/ somewhere in them, and since there are no pages with that in the path (or any .php3 files at all,) I am just 404ing any request that contains index.php3. Hopefully google will eventually get the message.
- jim 8-25-2005 9:37 pm [add a comment]
Does the robot learn from its mistakes?
- tom moody 8-25-2005 9:43 pm [add a comment]
It does, but in my experience not so well.
- jim 8-25-2005 9:52 pm [add a comment]
That did it. Tragedy averted. Phew.
- jim 8-26-2005 7:00 pm [add a comment]
good work Jim
- steve 8-27-2005 1:52 pm [add a comment]
Thanks for your effforts. Do you think you could do anything about Katrina now?
- jimlouis 8-28-2005 3:59 pm [add a comment]
like get it a different hooker name?
- bill 8-28-2005 4:03 pm [add a comment]
Please, please, "tantric specialist."
- tom moody 8-28-2005 4:19 pm [add a comment]
sorry to be flip, i just found out katrina is gathering with a direct n.o. trajectory that could trigger the ponchartrain effect. this isnt the first time they used the superdome for emergency shelter. next time they build a stadium they should build it up strong and high and with real consideration of its dual purpose as sportatorium and an emergency shelter for the huge number of people too poor to beat it out of town.
- bill 8-28-2005 6:15 pm [add a comment]
I used the exact same expression, "flip," in my apology to a New Orleanian I had yesterday emailed to jokingly about the storm, then today I decided yesterday's comments were too flip. Things often change last minute with these big hurricanes but it sure does look bad right now. Tomorrow at this time we'll know how bad, or not.
- jimlouis 8-28-2005 8:01 pm [add a comment]