...more recent posts
You learn something new everyday. It's not always what you want to learn though.
So I have the NTFS drive mounted on the Mac. And my idea was that if I can just get the Mac and the Linux box to connect directly to each other (over ethernet, not over 802.11b to the router) that everything would go much faster. After all their are gigabit ports on both ends.
I thought maybe I couldn't get it to work this way before because I need a crossover ethernet cable. I'm not sure that is true, since the Mac ports are supposed to be auto-sensing, but maybe that's only when connected to other Macs. Anyway I bought the crossover cable and got it to work.
Hallelujah I thought.
Except the copy still only went at 300 KB/sec. WTF? I guess the network connection was not the limiting factor. I wonder what is? I guess it must be the external drive that is the bottleneck. I'm surprised it can't do better than that though. This is a 2 drive RAID-0 connected over 400 Mb/sec Firewire to the Mac (and then over gigabit ethernet to the server.)
In any case, I just went back to doing it wirelessly because that way I can still be on the internet with the Mac while it is copying.
Not sure how this is going to work though since at this rate it will take a *long* time to make the copy. While I figure out what to do I'm just going to let it run (or crawl) in the background. Almost to 1 GB! Only 369 more to go. :-)
Well, HFS+ is supported in the CentOS 2.6 kernel, which was surprising to me, but NTFS (Windows partitions) are not! Crazy. What a pain in the butt. My system can see it, but it can't mount it unless I make unsupported changes to my kernel. Yuck. Maybe I'll just mount it on my Mac and copy it over, but that is going to be even slower. Damn.
Seems like Sun is really kicking butt lately. A year ago I wouldn't have believed it, but that just shows again how little I know. First they released a sub $1000 SunFire X2100 1U server that looks really sweet. And now plans have been revealed for a massive storage monster called Thumper:
The 4U high system will hold two dual-core Opterons and support up to 16GB of memory. A more unique part of the server will be Sun's use of 48 SATA drives.Holy cow. And the key to utilizing all that storage is a new filesystem, ZFS, that will be included in Solaris 10. ZFS sounds *really* amazing. The sort of thing that might make someone consider some really expensive Sun gear. Only now their gear isn't expensive any more. Lookin' good Sun.
Turns out you can just plug an HFS+ formatted drive into the server and it will recognize it. That's with CentOS 4.2 on the standard 2.6 kernel. I wouldn't have thought that would work. I should always remember to try the easy way first. You know, just in case.
Anyway, transferring 100 GBs over USB is not so much fun. Not sure if it's USB 1 or 2. Might have to sleep with the amazingly loud server fans on tonight which will be interesting to say the least.
But it does feel good to be loading it up at last.
Server just arrived. It is rather bad ass looking.
Man I have so many posts brewing but I'm just not sure when they will come out. I feel like everything has been on hold for so long, and now it is all starting to happen at once.
The new server is our way of playing a little part in this next phase of the internet. Disk space has plunged in cost, bandwidth - both fixed and especially mobile - have soared in speed, and new protocols - especially BitTorrent and RSS/ATOM - are allowing for new applications that will create a significantly richer media experience.
Here's one example. If you're a Mac user (Windows and Linux clients are on their way,) go download DTV. It's an RSS reader that pulls down feeds that contain video clips. You subscribe to different "channels" (which are like different websites,) and your DTV client polls them at a selective interval and downloads any new content. It's a program you just keep running in the background all the time, and then whenever you want you can open it up and there will be tons of new video clips that you watch right in the DTV program. And the downloads can themselves be torrents, in which case the DTV client is also a BitTorrent client - except the user doesn't see any of this, nor needs to know what any of these protocols are or how they work. Very very slick.
The DTV client looks and works much like iTunes. Click on 'Channel Guide' at the top of the left hand column (like clicking on your 'library' in iTunes) and you get a listing of different channels they think you might like to subscribe to. But the channels are just RSS feeds, so anyone can make a feed that can be shown (this is an open system - not a walled garden.) Just click 'Add Channel' at the bottom of the left hand column and you can enter any URL. Here are two to start with:
http://del.icio.us/popular/system:media:video
http://www.commonbits.org/rss/tag/daily-show
The first is a feed of popular video clips from the group site del.icio.us. The second are daily short clips from the Daily Show.
Now nothing here is completely new. We've been reading RSS feeds for years now. And downloading with BitTorrent as well. But it's never all been easy enough to reach critical mass. Now it is. And the way they have baked BitTorrent into this thing is just mind blowing. It solves the bandwidth bottle neck of being a content producer. And that means it levels the playing field significantly in terms of who can distribute the media.
What blogs have done for the written word, and what is generally called "news", which is much like what P2P networks did to the music industry, BitTorrent and RSS combos like DTV will now do to television and movies. It won't destroy them, but it will completely change everything, creating a lot of opportunities for new comers and a lot of disappointment for entrenched players who don't see it coming.
Server shipped. Should arrive on Wednesday.
Like most things I try to do this project is taking much longer, at every stage, than I ever anticipated. But today I finally ordered the server. Should ship early next week.
Dual Opteron server with 2 GB ECC RAM, and 12 400 GB SATA hard drives in a 3U Supermicro case.
One good thing about not being an expert is that when you get something to work it feels like a blessed event has occurred, rather than something routine just happening the way you expect.
I took the server in just before noon this morning. Slid it into my little cage (looking enviously at the beautiful Apple Xserve someone else had in the cabinet above mine.) Hooked it up and booted not really knowing what to expect. Theoretically the machine was all set up and should just work. Realistically I knew this might not be the case.
And of course it wasn't. The server booted, of course, and everything *seemed* okay, but I couldn't get out onto the net. And I figured if I couldn't get out, certainly no one else could get in. Ugh. What a horrible feeling. I didn't even really know where to begin.
So I left and came back here and started googling. Wrote some frantic messages on some discussion boards. Even fired off an email to an old college friend who is a professional geek asking if he knew of anyone I could hire for a day.
I didn't get too far with my research, but I did figure out a few things to look for. And I knew in order to get anyone else's help I would have to have some detailed info that I didn't have (not having any access to the server from here.)
So I headed back to the colo. Turns out to be ridiculously convenient from my place - 4 stops on the J M Z. Anyway I set up again. They have carts with some monitors and keyboards on them for everyone to use, and you just wheel one over in front of your cabinet and hook it up. I decided, what the hell, I'll just wipe the hard drive and install from scratch. I didn't really think this would work, but I didn't really have too much to go on.
After a short spell in a strange place called boot loader hell - and a second complete wipe and install - I finally got back to where I was before. Broke. So I started taking the careful notes I would need (like the outputs of ifconfig and route -n as well as the contents of /etc/hosts, /etc/sysconfig/network, /etc/sysconfig/network-scripts/ifcfg-eth0, and /etc/resolv.conf) and started packing up, more than a little depressed at my failure right out of the gate.
But by pure luck one of the guys who works there walked by and sort of asked off handed how I was doing. More out of politeness than from actually wanting to know how I was doing. But I answered anyway, saying I was having some trouble, but also that I was just learning, and I sort of expected to run into trouble. I really didn't think it could possibly be their fault. He made a few suggestions which made sense, the best one being to hook my laptop to the ethernet connection and see if I could get that to work.
Of course. Good idea since I am very comfortable networking on the powerbook. This would help isolate where the problem really was. And the exact same thing happened. Everything configured like it was okay, but then I couldn't get anywhere. Pings just timed out.
The guy had gone around the corner where the big switches were for a minute, and when he came back he had a real sheepish look on his face. "How much time did you waste on this?" he asked. Turns out he had plugged me into the wrong switch. "Try now" he said, and I did, and it worked. Just like it was supposed to.
Hey, this stuff is easy!
Just got my IP addresses. Server goes in tomorrow. My goal is just to get it up and running and to be able to SSH in using an IP address. Then over the weekend I will hope to get DNS working and to make Apache respond to a domain name. If that all works I will pull the trigger on the new server next week. Not sure how long that will take build and arrive. I'm hoping about a week.