Been wanting to post some more here without too much luck. Giving it another try. Warning: no proof reading or anything, just unedited ramblings ahead.
I am just finishing up my second big project for my new business. It has been a very slow process to get the whole thing (the business) off the ground, but that is mostly because I am having to learn so many new things. None of it is particularly hard, per se, it's just having to pay attention to everything at the same time. I'm not used to that. Being the only employee is great, and maybe the only way I can work happily, but the downside is just that I end up doing everything, and to put it mildly, I'm not good at everything.
It would probably be more efficient for me to work for another company that is run by people who, you know, actually know how to run a company. But that doesn't seem as fun. So I'm struggling along by myself, probably making a lot of stupid mistakes, and probably spending too much time reinventing wheels. But at least they are my mistakes and my wheels.
Building websites is pretty easy. Building websites that can be maintained and extended through time is more difficult. And building many websites that can be maintained and extended by just one person is even more difficult. This is where almost all my thinking goes.
Standardization is the key. I have spent a lot of time building the tools that will let me build websites. I could probably just code each site by hand in less time than it is taking me to develop an automated approach, but the problem is that once you have dozens of sites built, if they are all running on different code you have hacked together it is a nightmare to maintain. So what I've been aiming at is having just one very flexible code base that runs all the sites, and then as upgrades and bug fixes are made to that code they are automatically rolled out to all the sites.
And I've done pretty well with this. The setup is actually pretty extreme. Here's the quick technical rundown:
All the domains are Apache virtual hosts. To add a new site I edit httpd.conf, and point the new domain at the same directory as all the other sites. Huh? I know, bear with me.
All the code is PHP, and everything non binary is stored in MySQL. Mod_Rewrite redirects every request (with a few exceptions I'll get to later if I go into more depth) to a single PHP script - dj.php.
The dj script does all the ground work. It checks for a user cookie and validates the user against the database. It parses the request and determines the host that is being called. It checks if the page requested is an "actual" page, or whether the request contains what I call a verb - that is a request to do something rather than just return a web page (verbs are things like /post/, /upload/, /edit/, ect...)
If it's a page request dj fetches the info out of the database and hands it off to the appropriate script for rendering that particular style of page. If it's a verb request dj hands it off to the appropriate script for that verb.
When those other scripts are done they hand control back to dj which finishes up by doing any logging and closing all our database connections.
It's a pretty good system. If I need to add a new verb to the mix I just write the script in PHP, name it the same as the name I want for the verb, and put it in the special verb directory on the server. That's it. The dj script scans this directory every time it runs, so the new verb and it's abilities are just magically available as soon as I drop it in that directory.
And since all sites are running out of the same directory, if I fix a bug in, say, post.php, the fix immediately goes into effect for all sites. That is sort of the holy grail of maintainability I have been after. This should allow for a one person shop like mine to manage a lot of websites.
So far so good. The only problem is that it already isn't working out this way. My software is very flexible. But people always need it to do something else. There is a constant tension between the desire to add features, and the desire to not create a bloated mess. This is a very fine line to walk, and my job basically consists of walking this line.
And I can't complain. It is a fun challenge. The hard part is not just building a particular thing. As a one off project it is pretty easy to build almost anything. The hard part is maintaining it. If I just build some one off project, then don't look at it for a year or two, and then have to come back to fix something or add something it is almost impossible. This is exactly the result of me not being a professional programmer. Or, from the other side, this is exactly what is professional about a real programmer. Not the ability to solve the problem - almost anyone can do that - but to solve the problem in a way that is clean and self documenting and easy for a future version of yourself or, more difficultly, someone else, to step in and pick up where you left off.
That is really hard if you are always building each solution from scratch. Instead you have to build standardized tools, and then use those tools to solve the problems. Then when you need to go back to some old site you know exactly what is going on because it is built the same way as every other site.
But like I was saying, this is harder to work out in practice than in theory. On this last project I abandoned the "everything in the same directory running off the exact same code" idea, and copied the code into it's own directory so I could make some rather extensive modifications to it. This gives me more flexibility for this project, which I needed, but now I have two different (although largely similar) code bases to maintain.
And that's the constant battle. This is what I think about all the time. Trying to find the sweet spot between solving the local problem (how do I build the site the way the client wants?) and the global problem (how do I maintain the code over time on many many different sites.) There is no right answer to this. There is no perfect way to do it. There are only choices and trade offs and compromises. But I have to say I enjoy that kind of thinking a lot. I'm sure I'm making mistakes, but they are my mistakes and that makes them easier to live with.
There is a constant tension between the desire to add features, and the desire to not create a bloated mess. This is a very fine line to walk, and my job basically consists of walking this line.
Well, who doesn't think that way...
|
I am just finishing up my second big project for my new business. It has been a very slow process to get the whole thing (the business) off the ground, but that is mostly because I am having to learn so many new things. None of it is particularly hard, per se, it's just having to pay attention to everything at the same time. I'm not used to that. Being the only employee is great, and maybe the only way I can work happily, but the downside is just that I end up doing everything, and to put it mildly, I'm not good at everything.
It would probably be more efficient for me to work for another company that is run by people who, you know, actually know how to run a company. But that doesn't seem as fun. So I'm struggling along by myself, probably making a lot of stupid mistakes, and probably spending too much time reinventing wheels. But at least they are my mistakes and my wheels.
Building websites is pretty easy. Building websites that can be maintained and extended through time is more difficult. And building many websites that can be maintained and extended by just one person is even more difficult. This is where almost all my thinking goes.
Standardization is the key. I have spent a lot of time building the tools that will let me build websites. I could probably just code each site by hand in less time than it is taking me to develop an automated approach, but the problem is that once you have dozens of sites built, if they are all running on different code you have hacked together it is a nightmare to maintain. So what I've been aiming at is having just one very flexible code base that runs all the sites, and then as upgrades and bug fixes are made to that code they are automatically rolled out to all the sites.
And I've done pretty well with this. The setup is actually pretty extreme. Here's the quick technical rundown:
All the domains are Apache virtual hosts. To add a new site I edit httpd.conf, and point the new domain at the same directory as all the other sites. Huh? I know, bear with me.
All the code is PHP, and everything non binary is stored in MySQL. Mod_Rewrite redirects every request (with a few exceptions I'll get to later if I go into more depth) to a single PHP script - dj.php.
The dj script does all the ground work. It checks for a user cookie and validates the user against the database. It parses the request and determines the host that is being called. It checks if the page requested is an "actual" page, or whether the request contains what I call a verb - that is a request to do something rather than just return a web page (verbs are things like /post/, /upload/, /edit/, ect...)
If it's a page request dj fetches the info out of the database and hands it off to the appropriate script for rendering that particular style of page. If it's a verb request dj hands it off to the appropriate script for that verb.
When those other scripts are done they hand control back to dj which finishes up by doing any logging and closing all our database connections.
It's a pretty good system. If I need to add a new verb to the mix I just write the script in PHP, name it the same as the name I want for the verb, and put it in the special verb directory on the server. That's it. The dj script scans this directory every time it runs, so the new verb and it's abilities are just magically available as soon as I drop it in that directory.
And since all sites are running out of the same directory, if I fix a bug in, say, post.php, the fix immediately goes into effect for all sites. That is sort of the holy grail of maintainability I have been after. This should allow for a one person shop like mine to manage a lot of websites.
So far so good. The only problem is that it already isn't working out this way. My software is very flexible. But people always need it to do something else. There is a constant tension between the desire to add features, and the desire to not create a bloated mess. This is a very fine line to walk, and my job basically consists of walking this line.
And I can't complain. It is a fun challenge. The hard part is not just building a particular thing. As a one off project it is pretty easy to build almost anything. The hard part is maintaining it. If I just build some one off project, then don't look at it for a year or two, and then have to come back to fix something or add something it is almost impossible. This is exactly the result of me not being a professional programmer. Or, from the other side, this is exactly what is professional about a real programmer. Not the ability to solve the problem - almost anyone can do that - but to solve the problem in a way that is clean and self documenting and easy for a future version of yourself or, more difficultly, someone else, to step in and pick up where you left off.
That is really hard if you are always building each solution from scratch. Instead you have to build standardized tools, and then use those tools to solve the problems. Then when you need to go back to some old site you know exactly what is going on because it is built the same way as every other site.
But like I was saying, this is harder to work out in practice than in theory. On this last project I abandoned the "everything in the same directory running off the exact same code" idea, and copied the code into it's own directory so I could make some rather extensive modifications to it. This gives me more flexibility for this project, which I needed, but now I have two different (although largely similar) code bases to maintain.
And that's the constant battle. This is what I think about all the time. Trying to find the sweet spot between solving the local problem (how do I build the site the way the client wants?) and the global problem (how do I maintain the code over time on many many different sites.) There is no right answer to this. There is no perfect way to do it. There are only choices and trade offs and compromises. But I have to say I enjoy that kind of thinking a lot. I'm sure I'm making mistakes, but they are my mistakes and that makes them easier to live with.
- jim 12-01-2006 7:31 pm
There is a constant tension between the desire to add features, and the desire to not create a bloated mess. This is a very fine line to walk, and my job basically consists of walking this line.
Well, who doesn't think that way...
- Liner (guest) 2-21-2007 8:14 pm