...more recent posts
Remember back in the day when the 'new post' and 'new comment' counters on the front page would often be wrong? Those were the good old days! Since the last update (god when was that? over a year ago...) this seems to have been ironed out (has anyone had this problem in recent memory?)
Anyway, I am thinking of moving back to the old way of doing it. This would have the advantage of speeding up front page load times, which is becoming a big problem (any disagreements?) Hopefully my implementation this time around will not be error prone.
Basically, the system used to calculate the 'new' values for each subscriber at post time, and then store this number in the database so the system was ready to build your front page without doing much calculating. In the present system these values are calculated fresh each time you load the front page. The present way has the advantage of not getting out of sync with reality, but the disadvantage of not scaling. I didn't think we'd hit the ceiling so fast though (no doubt coding style and language choice also play big roles here.)
I may start posting over here again. I am going to try to devote a block of serious time to the system software. But I am not rewriting the system (don't worry Alex!) and any changes to the database will be very minor. If I ever do a full rewrite again (I guess that's happened twice? I can't remember, I'd have to look back) it would be done in Python instead of PHP, but I'm not nearly ready to do that.
Still, there are lots of things that I would like to fix and a few things I would like to add. Writing about it here is a way to get myself to think more formally about how to tackle all this.
First is speed. I'm starting to see some problems. Both here and a little more at datamantic.com. The bottleneck seems to be associated with having lots of users. I'm guessing especially lots of inactive users (resulting in unread information piling up in the database.) I think I have a fairly easy to implement solution here involving putting subscriptions "to sleep" after a certain period of inactivity. This sleep state will be recoverable, in the sense that if the person ever returns they can choose to wait for a moment and I can restore the "new" information about posts and comments.
There is also a deeper issue about speed that I sometimes think about, but I'm not sure whether I will address this or not at the moment. Ideally I agree with the notion that the ideal CMS (or weblog system, or whatever) has a database back end but uses that to generate static .html pages. Right now we assemble each page out of the database for every request. This gives us some things (like the personalized [new comment] counters which obviously can't be static,) but comes at a very high price. Especially as you try to scale. A good solution for us here is to at least have static pages for non users.
I guess I want all members to be able to create top level pages. Pretty easy change. But creating a page itself has some problems now. Especially in terms of who gets subscribed to the page, and whether the new pages automatically goes on other people's home pages.
I like the idea that members have total control over their own home pages (the system never adds or subtracts without being told by that user) but then how do you find out about new pages? I guess it could just be a social convention to announce them in /treehouse?
Adding users has to be completely redone. Right now I have to do a lot of tweaking in the database, mostly around the issue of home page listings, and posting permissions on group pages. I think I see how to fix this using a model account which has a base subscription set which is copied onto each new user as a starting point when their accounts are created. Maybe this model account could be somehow accessible to other members so that when you create a new page you also set the subscription for the model account, and then this is the subscription settings any new users will start with for that page.
And then there is the upload features. Obviously music is completely unworkable. Movies haven't even been addressed, but will follow easily on the heels of a solution to the music problem (I mean, it's the same problem.) I think when I move the server in house, and to OS X where I am more comfortable, I will be able to solve this issue. Possibly this will be a script that will have to be written in Python (I can mix languages like this without problem,) since I think the upload speed issue for big files is a PHP problem.
Those seem like the top of the list concerns at the moment, but no doubt I will add to this.
I've made some changes so that when guests cllick 'remember me' when commenting everything works better. Yeah, I know that's pretty descriptive, huh?
I'm going to start deleting references to unread comments that are more than one month old. (Right now if you don't read comments on a page, and that page is listed on your homepage, the new comment count just continues to grow without end.)
Any objections or problems with this? Should it be longer than a month?
Well, I really screwed up. Everyone's home page information was lost. I have restored a basic list of pages to everyone's home page, but you'll have to tweak it from there. Click [subscriptions] at the bottom of the front page, and you can add, remove, or reorder from there. Unfortunately, the order tweaking still seems to suffer from some strange bugs. But I'm too tired now to work more on it. I'll be on the case in the morning, and I'll appreciate any feedback. So let me know what you're seeing...
And again, sorry.
Lots of work on the main /subscription page today. There are still some weird problems with reordering. And in all my experimenting today I think I might have reordered some of your pages as well. Sorry. That, at least, shouldn't happen again. Still have more to do though. Quitting for today.
The subscription system has been pretty seriously overhauled. This is connected to the recent overhaul of the add a new user function.
In any case, I know there was always some confusion about pages listed on the front page (and how to control this.) I think it is much more straightforward now. Click [subscriptions] from the front page (or from this link) and you can check it out. It's pretty slow, but that's because there are so many pages.
In the pursuit of simplicity you no longer choose to track posts and/or comments. Now it simply works that any page on your home page will have both posts and comments tracked. Any page not on your home page will have no tracking.
Feedback is appreciated.
Thanks to Bruno I think I founder a pretty major bug where posts made as previews and then later edited and turned into live posts weren't registering as new posts for subscribers to that page. This might have been introduced fairly recently as I made some changes to how that system works. Anyway, should be fixed now.
I might have just introduced some weirdness into the new post counters in some specific situations. If you suddenly have a really big number of unread posts on a page, just click through once to the page (don't click on the [4888 new posts] link, just click through on the page link itself.) When you reload the front page the weirdness will be gone.
Sorry. This was unavoidable as I make some backend changes. Might not effect everyone.
Oh yeah, you can change your password in /settings now.