...more recent posts
Had a little drama today. I needed to reboot the server so that a few changes I made would take effect. But it never came back up. D'oh. So I ran down to the datacenter to see what was up. Yuck. Bad Superbock on /dev/VolGroup01/LogVol00. WFTBBQ?
In a little over my head here. Not the best feeling. Nobody there could help me either, but they were nice and at least set me up at a work station so I could begin googling.
It's really important to keep your head in this situation. It's easy to feel like it is hopeless when you have no idea what is going on, but the key is just to keep looking. In this case (and really in every case with computer problems like this,) looking just means plugging the exact error messages you get into google.
Of course most of what you get back is not helpful. But you keep looking. Inevitably you learn things along the way. I kept running out to my machine, trying something, then running back into the office and googling something else.
This went on for a few hours. During this time I cursed myself pretty thoroughly. My recent decision to keep all my clients on this box instead of moving them to ServerBeach was not seeming too wise. And my other plan to have that old PIII 1U server setup as a backup so I could just swap the ethernet cord and at least get a recent backup of everything on line was seeming very wise, except I hadn't gotten around to setting that up yet.
But no giving up. There is no one to give up too. No one to call. So just keep going. Eventually google helped me determine that it was just my /tmp partition that was giving me trouble on boot. So I erased that line from fstab and rebooted again, although truthfully not expecting this to help. But it booted. A little more hassle getting MySQL to start up (more googling... aha!)
And now everything is okay. I think I was down for a little under 3 hours. Not so smart, but even in failure I am somewhat proud to have found my way back to a working system. I really had to fight with my brain so that it wouldn't just turn off (like "Forget it - I'll never figure it out!") Read, look, try, repeat. Read, look, try, repeat. Eventually you get there.
Need to get started on the more robust contingency plan for sure.