...more recent posts
Yesterday I slightly broke the comment system in the case where you are leaving a comment from a /new_comment page. In those cases line breaks were not being converted to html break tags.
This is now fixed.
The active page owners on the site can now create top level pages. Let me know if you want this and I've overlooked your account.
I've been putting it off all week, but this morning I finally dove in and made changes to the picture uploading system. I'm 90% done, and it has gone fairly well so far.
Don't panic too much, all photos still exist in their old location, and I'm leaving the /getpic/ scripts in place for the foreseeable future. So everything will continue to work as it always has.
But I'm bringing the new system up in parallel, so the new way will start to work alongside the old way, and maybe eventually we'll get rid of the old way (but we'll always leave /getpic/ in place so no external links will ever break.)
The goal here is to get PHP completely out of the equation when an HTTP request is for an image file.
There is a new directory /library. Inside /library are folders for each user id (well, for each user id that has uploaded pictures.) A copy of every picture you have uploaded is now inside that directory. In the old folder (kept outside the webserver domain) the files had unique numeric filenames corresponding to the id for that picture in the database. All file information (name,date,user,etc...) was in the database. In the new system the info is still in the database, but the filename is kept the same as when you uploaded it (except in the case of duplicates where the system appends _1, _2, _3 as necessary, as well as replacing spaces with underscores.)
So where img src="/getpic/123" would request picture 123 (which had, say, the real filename 'test.jpg' when uploaded,) the new tag would be img src="/library/5/test.jpg".
The /5/ in the above example is my user id number.
The thumbnail is still at img src="/getthumb/123", but now it is also at img src="/library/5/thumbnail/test.jpg".
But remember /getpic/123 will continue to work, so nothing is going to break. When you upload new pictures they go to both the old directory and the new directory (same for thumbnails.)
Next I am going to update the post script to do some substitution. I am thinking of using this shorthand for inserting pictures: <[xxx]> where xxx is either the picture number or the exact text of the description you gave the picture when uploading. When you post an entry with that code in it, the post script will replace it with the appropriate img src="/library/n/xxx" format.
If that seems to work I'll write something to go back through the database and change all the /getpic/ statements to the new form.
This will be much more efficient. And it also will solve the weird problems we sometimes encounter because the file extension is not visible in the html. Some browsers get confused and download the file to disk rather than displaying it in the browsers. This will become even more of a problem as we move to dealing with many more mime types (moving images and such.)
I've revamped the subscription script. The changes can be seen when clicking [subscriptions] from the front page. This is the master subscription page that used to take *forever* to load. Everything you can do on the master page, you can also do on each individual subscription page, but on the master page you can change many pages at once.
(For instance, you can go to /systemnews/subscription/ and add or remove this page from your home page. But you can also go to /subscription/ and add or remove this page, as well as adding or removing any other pages.)
It now loads much faster because the 'Add any page' pull down menu no longer lists every freaking page on the site. I have changed this to 'Pages you might want to add' and that pull down now lists every page you have permission for which isn't on your homepage, but is on at least one other persons homepage. In addition, there is now a text field where you can type in the path of any page you want to add (in case you want to add one no one else has on their homepage.)
The rest works the same. This is the best page to use to organize the ordering of pages on your homepage. The system is pretty smart when you change the numeric order of any of your pages. You can change, say, number 15 to number 2 without manually reordering all the pages in between. Just change 15 to 2 and leave everything else the same (so you have 2 number 2's and no number 15.) The system will sort out the conflicts and reorder everything nicely (the old number 2 becomes 3, 3 becomes 4, etc....)
Does anybody else feel like they lost all their unread posts (not comments) around noon today?
The [create] scripts have been slightly updated. Gone are the confusing options for recommending different posting and home page values (which didn't really work as you might imagine anyway.)
Now you either choose to make the new page like an existing page (cloning), or you choose the template, page style, and public/private.
If you are cloning a page the new page will have the identical permanent page text entries as the source page (if the page you are using as a template for the new page has some permanent text on the left, then the new page will have that same text - useful if you have certain links you want on all your pages.)
And also, again when cloning, the subscriptions of all users will be the same for the new page as for the source page - *except* for position on each users home page. So, if you create a new page through cloning an old one, all the new subscriptions will be identical for each user *except* home page position will always equal 0 (the new page won't be on any users home page.)
Cloning is especially useful if you have private pages (and doubly so if your private pages have a large group of people with permission to post.) You could just make a private page (without cloning,) but then you'd have to add each poster in [admin] - one at a time! By cloning a page that already has the group of posters you want you get the correct permissions in one step.
This sort of worked before, except that the new page would mimic the source page in terms of appearing on people's home page. I'm trying to make it a rule that no one can add a page to an existing users home page. Only the user themselves can modify the pages on their home page. Any other behavior is now considered a bug.
If none of this makes sense, don't worry. [create] is now actually easier to use, and should work more intuitively. So go ahead and [create] new pages to your hearts content. If you try to [create] a page where you do not have permission the system will let you know.
Probably there will be some bugs until we use it some, but I have completed initial work on the [add user] scripts. All members should now see that option in their menu bar (let me know if you don't).
It's pretty straight forward. Just give them a name and (optionally) a url for their page, and the system does the rest. The subscriptions are made to mirror a dummy account I have set up. This sets up the persons home page with what I believe are the main pages on the site, as well as granting posting permissions for the common pages (treehouse, sustenance, artifax, cinefiles, regenerate.)
If, in the future, people make pages that they feel should be added to this starter set of front page pages, let me know and I will add your page to the dummy account ('generic friend') home page. Then, from that point forward, all new accounts will include this page on their home pages.
So feel free to add people and give out pages. What's the worst that could happen?
Oh, so you mean if I just use indexes correctly in mysql everything gets much faster? Who knew?
Am I crazy, or can anyone else confirm a speed up when loading the front page?
Remember back in the day when the 'new post' and 'new comment' counters on the front page would often be wrong? Those were the good old days! Since the last update (god when was that? over a year ago...) this seems to have been ironed out (has anyone had this problem in recent memory?)
Anyway, I am thinking of moving back to the old way of doing it. This would have the advantage of speeding up front page load times, which is becoming a big problem (any disagreements?) Hopefully my implementation this time around will not be error prone.
Basically, the system used to calculate the 'new' values for each subscriber at post time, and then store this number in the database so the system was ready to build your front page without doing much calculating. In the present system these values are calculated fresh each time you load the front page. The present way has the advantage of not getting out of sync with reality, but the disadvantage of not scaling. I didn't think we'd hit the ceiling so fast though (no doubt coding style and language choice also play big roles here.)
I may start posting over here again. I am going to try to devote a block of serious time to the system software. But I am not rewriting the system (don't worry Alex!) and any changes to the database will be very minor. If I ever do a full rewrite again (I guess that's happened twice? I can't remember, I'd have to look back) it would be done in Python instead of PHP, but I'm not nearly ready to do that.
Still, there are lots of things that I would like to fix and a few things I would like to add. Writing about it here is a way to get myself to think more formally about how to tackle all this.
First is speed. I'm starting to see some problems. Both here and a little more at datamantic.com. The bottleneck seems to be associated with having lots of users. I'm guessing especially lots of inactive users (resulting in unread information piling up in the database.) I think I have a fairly easy to implement solution here involving putting subscriptions "to sleep" after a certain period of inactivity. This sleep state will be recoverable, in the sense that if the person ever returns they can choose to wait for a moment and I can restore the "new" information about posts and comments.
There is also a deeper issue about speed that I sometimes think about, but I'm not sure whether I will address this or not at the moment. Ideally I agree with the notion that the ideal CMS (or weblog system, or whatever) has a database back end but uses that to generate static .html pages. Right now we assemble each page out of the database for every request. This gives us some things (like the personalized [new comment] counters which obviously can't be static,) but comes at a very high price. Especially as you try to scale. A good solution for us here is to at least have static pages for non users.
I guess I want all members to be able to create top level pages. Pretty easy change. But creating a page itself has some problems now. Especially in terms of who gets subscribed to the page, and whether the new pages automatically goes on other people's home pages.
I like the idea that members have total control over their own home pages (the system never adds or subtracts without being told by that user) but then how do you find out about new pages? I guess it could just be a social convention to announce them in /treehouse?
Adding users has to be completely redone. Right now I have to do a lot of tweaking in the database, mostly around the issue of home page listings, and posting permissions on group pages. I think I see how to fix this using a model account which has a base subscription set which is copied onto each new user as a starting point when their accounts are created. Maybe this model account could be somehow accessible to other members so that when you create a new page you also set the subscription for the model account, and then this is the subscription settings any new users will start with for that page.
And then there is the upload features. Obviously music is completely unworkable. Movies haven't even been addressed, but will follow easily on the heels of a solution to the music problem (I mean, it's the same problem.) I think when I move the server in house, and to OS X where I am more comfortable, I will be able to solve this issue. Possibly this will be a script that will have to be written in Python (I can mix languages like this without problem,) since I think the upload speed issue for big files is a PHP problem.
Those seem like the top of the list concerns at the moment, but no doubt I will add to this.