...more recent posts
Down the home stretch.
The music system now defaults to sorting by album (then artist, track number, track name.) You can click the column headings to get it to sort by 'track name' or by 'album'.
And the search engine was not constraining it's results properly in some cases, most notably when searching while in a 'by artist' view. Now you should only get results from the restricted collection of music defined by whatever view you are in. (If I am looking at /music/jim/artist/beatles and I do a search, I will only get results back from hits on just the set comprised of my beatles songs.)
Assuming everything is working correctly now, the only thing left is to add the ability to order the tracks inside a user created playlist (this will happen when you click through 'edit this playlist'.)
So my idea is to use the new /music scripts as the basis for all the media management on the site. Right now that means updating /image to the new format, and also creating /video. Basically all the heavy work is done, I just have to tweak the script to look in different tables in the database (say, the image table instead of music) and then change how the output is displayed.
First, does anybody see any big problems I am missing?
And next, what information do people want displayed in /image? Obviously the description of the picture (this will be like trackname is in /music.) Do we want the filename displayed to, or just link the description to the filename like /music does it? Of course the size of the image file. Maybe the upload date? Do we need to see the image owner listed?
I'd like to keep it as simple as possible. Additional information for each picture could be displayed (as it is now) on the image edit screen. But what info can't be left off the main /image lists?
Not sure if anyone other than Tom and Dave have looked at the new /music stuff yet. If you already have, here are some changes from today:
You can now (in most situations) click on the artist and album column headings to sort alphabetically by artist or album instead of track name.
Search now works better.
If you /sync/, and all the files you sync are music files, you are automatically redirected to /music/yourname/recent where the new files will be right at the top.
The checkboxes let you edit (click 'edit' at the bottom - although you can only edit your own songs of course.) Note that you can edit multiple songs at once. This is especially useful if you are adding album and artist info (yeah, sorry, I can't read the id3 tags from the mp3 files yet.) On the edit page you can fill in the artist and album info one time, and have it apply to all the tracks you have selected for a multiple track edit.
So I'm imagining: FTP a whole album worth of songs. Go to /sync and they will be sucked into the database and you will be taken right to /music/username/recent where what you just uploaded will be at the top of the page. Click to select them all and on the edit page add the artist and album info in one place to assign them to all the tracks.
Also note that there is a checkbox at the very bottom that will select all the checkboxes above it (or unselect them if any are selected.)
Also to note, 'trackname' is distinct from the actual filename. So trackname can have any special characters or spaces or whatever you want. When you /sync the system just uses the filename as the trackname (probably I can make it smarter about this), but you're going to have to do a little editing no matter what. I leave the filename as whatever it was when uploaded, say, 01_ella_and_louis-I_wont_dance.mp3 and change the trackname to the more readable 'I won't dance', while making the artist 'Louis Armstrong and Ella Fitzgerald' and the album 'Louis and Ella Again'.
Since the system sorts by album and artist it would be nice if we could be careful with spelling. When entering artist names (again on the edit page) I also supply a pull down menu of artists already in the system. Take a look in there first, since using that for repeat artists will assure that all spellings are consistent.
I've been building something for the site, and for the first time ever I didn't put it up live well before it was done. I've actually been, like, testing it. And shit.
It's really hard not to let it loose (not that it is so cool or anything, but I get attached to these things.) I think maybe tomorrow. So yes, this post basically amounts to nothing but I had to at least let out a squeak.
My stupidity really pays off in the sense that I end up working for a long time on simple problems, and then when I finally solve them it feels like I am some kind of genius because, well, the problem must really have been hard for me to work on it for so long!
Follow that?
Anyway, I believe I have the OS X client ftp uploading script now working. At least I slightly impressed Dave with it. The script is an icon that sits on my desktop. Any file (or files) I drag onto the icon are FTP'd to the dmtmusic upload server (which can now handle all media types, not just music.) No connecting. No signing in. No hassles. Just drop 'em and they upload.
I think I could just email this script to other OS X users if anybody wants it. I'll be curious to see if that works. So let me know.
I ended up doing it completely in Applescript so it should be very portable across OS X machines (doesn't rely on Apache and PHP like my first attempt.)
The music system is getting the same makeover the image system got. We are still stuck with the FTP upload system for now, although it looks like I'll be able to automate it at least for OS X people. But as long as you are comfortable with an FTP program, it's extremely easy. And really more powerful than [upload] since you can set your ftp program to upload directories of files ([upload] still works the same as before, just to be clear.)
Once you have files in your directory on the ftp server - (dmtmusic@tulip.he.net/dmtuser_n) where n is your user id - you now go to:
http://www.digitalmediatree.com/sync/
instead of /music_sync/.
Basically it does the same thing, except now it is much smarter, and it puts your files in the new /library/music/n/ location. The /sync/ script can handle most files provided they have the proper extension. Feel free to FTP .jpg, .gif, .mpeg, .ogg, .swf, .png, .wma, .avi, etc... files into your directory. /sync/ will grab them and put them in the right place. If it's an image it will add it to your 'complete list' of images. If it's a music file it will add it to your 'complete list' of music. If it's a movie file it will at least put it in the right place, although the /video/ system isn't built yet (but since it's almost exactly the same as /image/ and /music/ it should be easy.)
Fiddled with the client side uploading script today. It was timing out on large uploads, but now seems to be working after a tweak to the httpd.conf file. This is a very exciting feature I think. Watch what happens to the music library now.
I think Tom was onto this a long time ago, but the Userland weblogs.com update notification system is now not only not working for us, but actually causing posts to time out (if your page is set to notify userland when you post.) So I have removed that function (even though it's still listed as an option on [editpage].)
When I'm done with the media library projects I have open now I will look into this. I know a lot has happened in this space since I last looked. If we want to be doing this sort of thing then technorati is maybe a better place to be pinging. And there are others too.
One thing at a time though.
/editpic now gives you a fully formed img tag so you can cut and paste. This is helpful if you are replacing old instances of /getpic/ with the new img tags. Just [edit] the post. Copy the /getpic/xxxx number, and put it onto the end of /editpic/?pid=xxxx. Then the editpic screen will give you the full img tag you can copy and paste back into the post to replace the img src=/getpic/ tag.
Note, there is no need to do this to old /getpic tags. They will still work. But the new way is slightly (noticeable though) faster. So it is probably a good idea to replace often loaded images (like images that load every time your page loads) with the new form. At least if you want your page to load a little faster.
I've done this for a few pages that are (relatively for us) high traffic. Hopefully nobody minds me taking that liberty.
Mark was onto this potential problem already, with his previous suggestion that I include the picture ID numbers as a custom argument to the img tag. The problem is that given an image in a post, there is no way to get to the editpic page for that image (without looking up the picture in your complete list of [images] to get the ID number to pass to editpic/?pid=xxxx.)
But instead of doing that I made it so editpic understands image paths. So now you can pass editpic the image location in the URL, like this:
http://www.digitalmediatree.com/editpic/library/image/5/example.jpg
You can also still pass the ID number as ?pid=xxxx.
A little esoteric, but this situation will probably come up.
Sometimes it all goes wrong.
Well, actually, I've seen worse. I mean, nothing big came unravelled. But I've spent all of today so far working on one problem with thumbnails that turns out to be unsolvable under my installation of PHP and GD.
And the answer is merely to use image imagejpeg() even when it really really seems like you should use imagegif(). Okay, fair enough, I just wish it didn't take me 5 1/2 hours to figure that out.
Scripting quagmire; I want my cakewalk.
But I believe my client side uploading scripts are now on their way to being finished. At least 90% done, which means I only have about 300% of the time so far invested remaining until they are done. Go figure. Must be that new math.
The [image] pages are now a little better. And thumbnail pages are working again.
I still need to map the old album and thumbnail addresses so they redirect to the new pages. And possibly getpic might be screwed up again for recent pics (remember, it sort of isn't supported anymore) but I'm going to still make sure it's all right tomorrow.
Damn. I just screwed up all the image album information. D'oh.
I'm working on fixing this now, so bear with me if some of your albums are screwed up. I think it probably only screwed up album info for recently uploaded pictures.
I have a full back up of the site from 3/6/2004 so I'm downloading that now to compare, and then I guess I'll fix whatever was uploaded after that by hand. I think I should be able to do this and get everything back.
Sorry sorry sorry.
Just thinking out loud here.
What to do about mp3s? I'm going to use a lot of the code from the image system, but I think we need some security. I'm not really interested in serving up our mp3 collections to the general public (for bandwidth as well as legal reasons) but at the same time there will of course be legitimate mp3 files that need to be served to the general public.
So I guess there will be some option while uploading to put sound files into either your public or private directory. I'm imagining that the private directory will be available to everyone who has a non anonymous user account. I mean everyone who has an account that was not merely made by clicking 'remember me' on a comment page.
[For the record there is a group_level number associated with each account: 0 is a random surfer without a cookie, 50 is someone who clicked 'remember me', 100 gets you image upload ability, 300 lets you add other users, and 900 makes you a super user (I'd tell you what that means but then I'd have to....)]
This means that private mp3s will be accessed through PHP (so I can check cookies.) I am really not sure what kind of a hit this makes on the server (just using fopen($filename) and then passthru()) but my guess is that we have power to spare. Of course this is the sort of assumption that can come back to bite you.
One thing that will alleviate this problem (if it even is a problem) is that my OS X (unix) uploading client can also be a downloading client - thus moving the CPU hit to the client. God I love that idea. I wish I could make that stuff available for windows too. (Theoretically I could, since it only relies on Apache and PHP, both of which can run on Windows, but I would be completely unable to support getting those to run on anyone's windows machine.)
If I do the public/private directory thing for mp3s I could reuse that code to make private image directories as well. But I can't think of why anyone would need that. And I don't want to add even a single checkbox that isn't necessary (despite the look of the [settings] page.)
Also, although I keep saying 'mp3' the new music system will allow for any encoding type (.wav, .ogg, .whatever...)
Going to finish the /image pages first though. Those are still in very rough form. If anyone has any design input for how those image and album listings should look I am all ears. Mock it up in photoshop if you want. Design is obviously not my strong point, but if you don't mind messy table laden HTML I can probably make a fair representation of most designs.
I'm still praying for the day we recruit an HTML/CSS design person to work with us.
The image system continues to change. The [upload] screen is now much different. All the album options are gone from there now. I like this more simple interface.
When an image upload is complete you are taken to the new /editpic/ page where you see the picture and all vital information. Below that you can delete the picture, and below that you can control in which, if any, albums the image belongs to. I think this is more straightforward as well. Check an album and the image goes into the album. Uncheck an album and the image comes out of that album.
You can also make a new album here. When you erase the final picture from an album the album itself disappears.
I am keeping it so that the old album URL scheme still works. So /image/album22/ still gets you my Paul Laffoley album, even though that URL is deprecated and the official URL is now /image/album/5/Paul_Laffoley/
I still have to put back thumbnail view when accessed from the new URL scheme. And some of the other /image/ listing pages are not complete design-wise.
Okay, I put off changing the /album/ file structure for the moment. I need to think more about this. But the new editing has been partially activated. Clicking on the picture id link from any of the image pages (or clicking on an image thumbnail from a thumbnail view image page) now takes you to the new /editpic/ page. This shows the picture along with meta information about the picture. And if the picture is yours it allows you to make some changes (edit name and size) as well as deleting. This script will also allow you to add or edit how the picture appears in different albums, but that is not yet working.
Deleting from this new new /editpic/ script will solve the problem Tom ran into yesterday.
I have a lot more work done on the image system (uploading, editing, and categorization) but nothing ready to show yet.
I'm curious if anyone is wedded to the /image/album0/ naming convention for image albums? At the moment I am planning on changing this so that, say, my Austria 2002 album would move from http://www.digitalmediatree.com/image/album32/ to http://www.digitalmediatree.com/image/album/5/austria_2002/ (or maybe I'll put /image inside /library, so /library/image/5/album/austria_2002/)
Any thoughts on this? Any idea if there are external links going to any albums at the old locations which might break? (I don't really think so, but I could be wrong.)
Another long day today. And a lot more progress.
This will be exciting for Mac OS X users. I have the functioning skeleton of a new program that uploads files to the website. Right now it understands a bunch of different image formats and the mp3 music format. But I can add other types very easily (certainly other music formats are dead simple to add, but movie formats will be only slightly more difficult.)
The cool thing is that half the program runs on the client machine. Since OS X is unix this makes everything really easy - and I'll be able to install it on any Mac OS X machine. This program makes it so that there is a web upload folder on my machine. I can drag media files into this folder, and then I just load a particular web page in my browser. That starts a chain of events that uploads every file in that folder to the server, and then synchs all those files (putting them into /library/image/x/ or /library/music/x/ (or /library/video/x/...) and adding the appropriate information to the database.
The big win here is multiple file upload capability. You could just set it going on a big folder and go out for a bite. But the other win is speed. Unlike the present photo upload system, which sends the file over HTTP, this program (running on the client) makes an FTP connection to the server. This means that big files will not be a problem. It's basically a replacement for the complex music uploading system we have now (that requires a standalone FTP program on the client) and it handles photos (and soon movies) as well.
This should run on any unix based machine that can run PHP.
Of course I will continue to do my best to support all platforms, so there will always be ways to do these things from windows - I just can't guarantee it will be as easy to use.
Probably take me a couple more days to get it all polished up.
Holy cow what a long day. Very frustrating. Nothing was going right.
I was just going to finish up my work on the upload scripts this morning, but it ended up taking all day. I *think* things are back in working order now though.
If you were around today you might have noticed some errors since I was uploading lots of different files to the server (if you request a page right as I'm changing it you would get an error.)
I changed the new library structure a bit. Before (I mean yesterday) a picture (say test.jpg) that I uploaded would be in /library/5/test.jpg. Now it is in /library/image/5/test.jpg. I added the extra layer in the hierarchy because soon we will also have directories like /library/music/ and /library/video/. Nothing should have broken because of this move. But I'll go back and change any links anyone made yesterday while the filesystem was set up the other way.
Otherwise everything else is the same. But under the hood there are a lot of changes - and everything is much more sensible down there now.
Yesterday I slightly broke the comment system in the case where you are leaving a comment from a /new_comment page. In those cases line breaks were not being converted to html break tags.
This is now fixed.