Daring Fireball attempts to correct the story with some facts about the AAC audio format. I recently drove a thread over at Tom's a bit off topic with similar concerns, although I didn't have all the numbers that John Gruber has put together.
- jim 4-10-2007 8:07 pm

Nice write-up.

I have a few quibbles. I wouldn't give Dolby the bulk of the credit for AAC. They were just one of several contributors. He is right about VIA, which is an arm's length subsidiary of Dolby, being the agent for licensing the pool.

Also, Thomson doesn't hold all of the IP on MP3. Fraunhofer claims to be the key contributor, but there are several other contributors. At the moment, Lucent/Alcatel are trying to parlay their contribution into piles of cash. There's a $1.5 billion judgment against MS that's the subject on ongoing appeals.

So arguably, AAC is "cleaner" than MP3 with respect to license fees.

In the MPEG-4 AVC/H.264 standards process (a joint activity by ISO/MPEG and ITU/VCEQ (the h.xxx people)), getting patent licensing sorted out in parallel with the technical issues was a priority, to avoid the pissing matches that occur once something gets popular. The standards guys have learned a lot since the days of MP3.

- mark 4-10-2007 9:19 pm



"The ideal scenario would be for a genuinely open and free file format such as Ogg Vorbis to supplant MP3 as the de facto world standard. No patents, no licensing fees, a documented file format, open source libraries for encoding and decoding. That doesn’t seem to be in the cards, however. In the real world, major corporations only seem comfortable with multimedia formats backed by other large corporations."

That is about the lamest thing I have ever read in my life.
- tom moody 4-10-2007 10:46 pm


But you don't use Ogg Vorbis either! And I'll bet for exactly the same reasons as outlined here (basically, because no one else uses it.)
- jim 4-10-2007 11:18 pm


A few more thoughts on Ogg.

"Patent free" actually puts Ogg Vorbis at a disadvantage. If Xiph Foundation had some patents to back up their work, and had a GPL-like or BSD-like license grants for those patents, they would have bargaining chips if infringement is claimed. (Cross-licensing is a frequent solution to infringement spats.) That would provide users greater assurance that Xiph is in a position to defend their IP. Anyone can publish some code and call it "patent free", but that doesn't mean it isn't unencumbered.

As far as I can tell, Ogg Vorbis is completely absent from the mainstream digital video space (which is dominated by MPEG and ITU-T/VCEG standards -- publically documented, always supported by somewhat clunky public-domain test-model code, usually supported by higher performance Source Forge open source libraries). I don't know why Vorbis has no role. Could be performance issues. HE-AAC is better. HE-AAC v2 is a lot better at low bit rates, and is finding much interest in the mobile video space. I suppose the work on Ogg Theora (a lossy video codec based on a codec with performance similar to MPEG-1 ... yawn) is "pure" while getting involved with the MPEG committee to include an adaptation layer for Vorbis is "impure". Dolby did it with AC-3 (aka Dolby Digital(tm)), so I don't see why Xiph couldn't do it with Vorbis.
- mark 4-10-2007 11:26 pm


So does open source work or is it all a bunch of communist BS?

What I'm hearing is it works provided it can move up to market level for the ultimate test, at which point it is no longer open source.
- tom moody 4-10-2007 11:56 pm


Open source does work. Linux is the great success story of the movement, and there are lots of smaller projects which also have traction.

It's hard to separate out the commerce element. One way it works is for individual programmers to donate work, which is sometimes done as a marketing effort to get paying gigs as consultants. Another way it works is for bigger companies to buy into the concept. Companies that rely on open source for their products often try to maintain good relations with the open source community by giving back above and beyond the minimum requirements. Cisco, for example, contributes a lot.

What makes companies and venture capitalists nervous is the GPL and LGPL licenses (published by the Free Software Foundation) in particular. The intent of GPL is to infect (or inoculate, depending on your perspective) the code base it's incorporated into, and make that entire code base GPL. Yes, the software is "free", but it also sets "free" any software that it touches. Companies that have a lot invested in their proprietary code worry about that. There are techniques to put a firewall around the GPL code. But none of this has been tested in a court of law.

Companies like BSD style licenses much better. BSD licenses avoid the coercive element. According to wikipedia, the Vorbis libraries are BSD-style, meaning companies can incorporate a Vorbis decoder into their player without having to worry about publishing their entire code base, and thereby giving away their secret sauce.

Back to your question, companies do sell open source code. They sell it as part of larger packages which usually include proprietary code. But they are also required to re-publish the open source work (typically on the web), and in the case of GPL are required to re-publish their modifications and derivative works. The use of open source code by commercial entities does strengthen the movement, by virtue of their "payback". However, the Free Software Foundation has an interpretation of GPL (which isn't supported by the actual language) which would, if upheld, deflate the movement by making GPL code anathema to commercial entities. Richard Stallman, President of the FSF, has fairly purist views.

One last note. Open source addresses copyrights, not patent rights. So, unless the underlying invention is public domain, the specific software expression may still be encumbered by patents.

- mark 4-11-2007 12:40 am


Open Source works for some things and not for others. And there are lots of shades of grey in the middle (and lots of flavors of what can legitimately be called 'open source' - like GPL, BSD, and Apache are all "open source" licenses, but they are all subtly different.) So I'm not sure you can ask the question as cut and dry as you are (where my options are to say answer with it "works in some all around way" or else it's "bullshit".) It's more complex than that.

In general, open source seems to be most effective at the backend platform and tool level, and less effective at the end user level. GNU/Linux is massively successful as both a server OS and as a programmer OS, but it has almost no penetration on the end user desktop - and a big reason for that is precisely because the open source licenses preclude the simple bundling of non open digital media codecs (like Windows Media) that end users need (you can install them yourself, when they exist, but that's hard for non savvy end users.) Almost the entire web runs on MySQL databases (also open source) but end user programs like Microsoft Excel have no open source rivals.

I used to be what I would now term a "religious zealot" when it comes to open source. These days I'd just like to find ways to make things work in the best, most open manner. But I try not to get hung up on the little things, and compromise with businesses is necessary I think.

Open Source definitely works for some things (Linux, GCC, Vi, Bind, PostFix, SendMail, Apache, MySQL, PHP, etc... are all wildly effective and the web as we know it would not be possible without them,) and it works less well for other things (digital media codecs come to mind, but also things like PhotoShop, Final Cut Pro, Maya, etc....) But just because something isn't open source doesn't mean it is bad. You have to look at the specific situation. What Mark (I think) and I are trying to say about AAC is that it is "plenty good enough" in terms of it's licensing and restrictions. It's not purely open and free, but if it was then no big name content would use it, and as a result nobody would have the codecs installed (so it would be like Ogg Vorbis where lots of people want to use it but no one does because very few people could hear your music if you only release it in that format.) AAC's restrictions are not onerous to the end user, but they do serve to give companies a feeling of safety that they can't usually get from completely free offerings (see Mark's comments above about the patent infringement worries with Ogg.)

So it's a tiny compromise, but to me it's a smart one. End users do not lose a thing (well, maybe their media players cost a few cents more,) and everybody gets a widely distributed, high quality audio codec that no entity in the future can take control of and change the rules of the game. Sure, I'd rather someone did all the hard work (not just of making it, but of going through the very expensive work of making sure it is not infringing on any copyrights, and then defending it from future charges of infringement) and then just give it away completely freely - but who is going to do that? I don't feel I can criticize unless I'm willing to do the work myself. And I'm not. So to me AAC is a great option in this case.
- jim 4-11-2007 12:43 am


Thanks, I'll think about this. My views are closer to Stallman's mainly because there is so much abuse and manipulation of the system in the private sector (as opposed to small networks of enthusiasts). I see a fair amount of cooperation and idea sharing between intellectual property owners and users with the music software I use but I don't get that sense of interaction from Adobe (maybe it's just because I hate Adobe products). Also small is beautiful. After 100 million players sold I just stop trusting.

Ideally I'd like there to be one free standard file that everyone uses and the money made on the hardware and ways of enhancing the experience of that file.
- tom moody 4-11-2007 1:31 am


AAC is pretty close to that ideal. There's no royalty attached to the media file. There are nominal royalties attached to commercial encoders and decoders (and realistically none if you compile your own personal codecs from Source Forge source code).

I see the MPEG and related standards processes (done properly) as an outstanding compromise somewhere between "purely proprietary" and "purely license free". Over the past 15 years I've seen up close and personal how these standards allow small entities to use creative destruction to blow away monopolistic or duopolistic hegemonies.

I'll grant that standards are not perfect, but most of the time the real choice isn't freeware vs. open-standard-ware, it's open-standard-ware vs. monopoly-ware.


- mark 4-11-2007 2:20 am


Gruber follows up and makes the patent issue a little more clear. Software patents really seem to suck.
- jim 4-11-2007 4:41 am


Some good points in that post. I was thinking the same thing about the name. MPEG-4 AAC makes the provenance more clear.

What's patented isn't the SW, but the underlying algorithm. SW is an expression, and is subject to copyright.

If we fast forward 10-20 years, these issues will be in the past. Patents, which have much shorter lifespans than copyrights, will have expired. SW copyrights are very easy to get around, especially if the protocol is open. Storage and networking will have advanced to the point that early 21st century compression technology will be just fine, even for super HD.

However, the clueless media conglomerates will always be with us. And Steamboat Willy will still be under copyright, due to the "The Walt Disney Cryogenic Copyright Act".
- mark 4-11-2007 10:27 am


"the choice isn't freeware vs. open-standard-ware, it's open-standard-ware vs. monopoly-ware"

I've been thinking about this and it sounds too much like "they think they can use my pipes for free"

How about open source instead of free? (and therefore open-source vs open-standard)

My understanding of open source was it was free in the sense of you didn't pay money for it but people passed it around and improved it to the extent they were able.
- tom moody 4-12-2007 7:59 am


It's a copyright vs. patent thing.

AAC is an open standard, meaning the protocol is fully documented. Further, the process of drafting the protocol was open -- submissions were made, published, justified, debated, discussed, vetted, voted upon. Anyone can participate in the process. Engineers and scientists from academia and industry participate. Some of the top people in the world contribute to MPEG and ITU-T, which is why their standards kick ass compared to anything any one company can produce.

There is open source code that implements the standard. I'm not familiar with the details of AAC, but I am with AVC. The AVC standards committee put together a software model based on contributions from all over. (A grad student at HHI in Berlin was the guy pulling it all together.) Anyone can download that code, play with it, improve, extent, compile, whatever. The model grew organically with the standard, so it's kinda spaghetti code.

There's another code base that implements the AVC standard, called x264. It's cleaner, and easier to modify. It's released under the Free Software Foundation's GNU General Public License.

For AAC, Audiocoding.com is one of the sources for opensource implementations. I'm sure there are others.

The various standards out of MPEG are all open standards, and most have open source implementations available. But, and here's the catch, the standards are usually based on algorithms that are patented. Thus the licenses for the patent pools, administered by MPEG Licensing Authority, Via Licensing, et al. What they strive to do is make sure the licensing is "fair and reasonable", and all the essential patents are in one pool. The cost of the patents (for consumer goods) is a small stack of quarters.

They did screw up in the MPEG layer III (mp3) patents. The Bell Labs (now Alcatel/Lucent) patents weren't tied up in a bundle in a pool. The jury award in the ongoing kerfuffle between Alcatel and Microsoft is 0.5% of each Microsoft PC sold. Assuming a $1,000 price, that's $50 for the patents, when by all rights it should be 1/100th of that. That's completely fucked up.

I hope that helps.
- mark 4-12-2007 9:44 am


Thanks. I know a bit about intellectual property law (mostly trademark and copyright) but the intricate relationship of patent, copyright, software, and money is difficult for me. I know that there is a lot of "might makes right" in IP where might=legal muscle to massage the desired outcome. That's why it's easier to fall back on simplistic truisms about freedom (kind of like George Bush).
- tom moody 4-12-2007 10:27 pm


D'oh I got my math wrong. 0.5% of $1k is $5. Still too much money by a factor of 10.

Regarding the "might makes right" issue, MPEG tries to deflate that with an attitude of "if you aren't reasonable about licensing, your stuff is out." They sometimes slip up, but they're getting better, and they're causing change. I attribute to the open standards movement the fact that Dolby AC-3 (used on DVDs, digital cable, digital over-the-air TV) and WM-9 (standardized as SMPTE VC-1) are open protocols with reasonable licensing.

Perhaps one can view FSF as the Malcom X of open systems and MPEG as the Dr. King of open systems. Both have a role. And perhaps corporations are more willing to make MPEG successful because they fear FSF.

Another analogy ... Let's say MPEG/ITU-T standards are stone ground wheatberry bread made from organic grains -- a huge improvement over the alternative of Wonderbread deep fried in lard made from baby seals (closed, single source, monopolistic media formats). And then the wheatberry bread gets dissed because it's not made from quinoa procured from cooperatives of indigenous small-scale farmers high in the Andes (absolutely, purely and completely unfettered media formats), and besides it contains small amounts of soy lecithin from those fuckers at ADM. Well, I'd love me some of the quinoa bread, and some of their potatoes too. Hell, they're not just heritage potatoes, they're primordial potatoes. But I've got sandwiches to make, and what's at the local store is wheatberry bread, and no baby seals died to make these sandwiches, so WTF?!

Anyway, that's what I'm going to call my "harm reduction" analogy.
- mark 4-12-2007 11:34 pm