Today I had an exchange via Twitter with my friend Sean, who reminded me about a point we’ve previously discussed face-to-face, “… #opensource #FAIL meter http://bit.ly/nBcYz too Linux-centric – doesn’t apply to Java, Web or Ajax …” My point back was, “… It applies if you want to deploy on #Linux … don’t #h8 your sysadmin, use existing standards v. making them up each time!”
I want to expand on that a bit, particularly around packaging, because my super-smart developer friends are correct about Java/Ruby/Python/PHP/et al packaging being great and sufficient … but only within their own realm. It really does fall apart as soon as you want to move outside of that realm to the shared world.
If you think of a Linux distribution as a grocery store, that’s a fair analogy. People grow food at a farm (upstream projects), it is packaged and shipped to the store (RPM, DEB, FHS-compliant TGZ), and consumers come to the store to get the tasty food. (It’s an incomplete analogy, because the open source consumers may also be the farmers, and the store is more like a farmer’s market in some ways, but let’s just call it imperfect and move on.) Some people shop in the Fedora store because they like how we arrange the packaging on the shelves, and some people prefer the Slackware store for the same reason.
The grocery stores have come up with some standard ways to deal with food products. We do it slightly differently depending on our own experience and customer-base. Most food is packaged for shipment in boxes that are variations on the cube. Even the apples that are stacked in a pyramid in a bin arrive at the store in an apple box.
These cube shapes are chosen because they:
- Stack tightly for shipping.
- Can be made of lightweight, inexpensive cardboard that can hold much more weight than you would think.
- Are of the right size that an able employee can move them around the store while still being stackable on a pallet.
- The packaging can be recycled and turned in to more cardboard.
Inside of the shipping boxes are often more boxes (except, for example, apples and beach balls, which come in a big box but are presented to consumers in really big bins.) These boxes are made for the shelf to display and attract consumers. Thus:
- They are polished and pretty.
- They stand up on precious shelf space; when a manufacturer pays to get a more prominent location on premium shelf space, the manufacturer wants to maximize the value of the effort/money spent.
- The package tell customers what the food is for, what is in it, what others think about it, when it expires — built-in advertising and information.
- The boxed packages fit nicely in the shopping cart, grocery bags, and on the shelves at home.
- Good packages conform to standards that make them reusable in the home and recyclable in to future packaging.
So along comes a new company, let’s call it Java. And another company, let’s call it PHP. Java, well, they’ve got this great modular idea so their packaging can fit on any shelf while using minimum shelf space. The packages are triangular, they stack really nicely, using this cool new tool to strap them together.
Oh, the cool strapping-together tool isn’t freely available? I have to get a set of tools for each store to strap your special boxes to my shelves? And that tool isn’t open source?
Sorry, that doesn’t fit on my shelf and I’m not going to retool my entire operation for your “unique” needs you think you have when you could have just iteratively built on top of what works already, but you are welcome to sell your stuff out front with the Girl Scouts. There is always room out there, and many businesses get their start that way.
Now, PHP has these cool sphere packages. They are easy to put together by anyone around their custom package-contents, and they move down the pipeline pretty quickly compared to good old square packaging. While they don’t sit well on the store shelves, well, we can just add more bins for them, right?
Sure, until the whole store is nothing but balls and bins and balls bouncing from bins. Staff are running around trying to get everything to stack right, no one can read a label because it’s often turned upside down on the sphere, and you can barely tell what are the old balls and what are the new, better, faster, more secure balls.
For a Linux distro to work, hundreds of pieces of software have to work together. They do such a good job that you can make a following selling cookies in front of the store. But if you really want to be big/useful to your users/something more than appears a few times a year, it is really helpful if you can be in a package that fits on the shelves and in the lives of your consumers.
Once you accept the idea of packaging, relying upon system libraries, making something that can actually be buildable from source and is truly redistributable, and do it successfully for one Linux distro, then you are 80% of the way there. Adding another UNIX-like OS to your packaging is easier. All of this won’t make it easier to deploy on non-UNIX-like OSes, but it also won’t make it harder.
The analogy makes sense, but do customers actually shop for Ajax libraries in the RPM store? If so, what is the market share? (Measured in any way you like, but one measure would be as a percentage of deployed websites using that library?
I’m working on an Open Source project called iUI (Ajax for Webkit-compatible mobile devices, available at http://iui.googlecode.com) Would anyone want that as an standalone RPM, or are they just going to get it as part of Drupal, for example?
For that matter, what percentage of Drupal installations are installed via RPM
argh! Broken iUI url because of closing parenthesis, correct url is:
http://iui.googlecode.com
One of the imperfections of the analogy is that we tend to think of customers in the sense of far-out-there-end-users. In fact, your customer of a package for a web app is going to be the sysadmin who has to maintain the full stack for you. This could be the folks at Rackspace/Dreamhost/etc. Check out this project that Rackspace has brought out that is upgraded packages of PHP etc. that they have been maintaining internally for their customers:
http://iuscommunity.org/
The have to maintain many thousands of servers for their customers who are demanding updated P-in-LAMP. If e.g. jQuery were part of what customers would be demanding, you can be sure that Rackspace is going to put it in an RPM so they can manage it. It’s what smart, experienced Linux sysadmins love – a way to be lazy.
A Linux distro that includes Drupal as a package, which Fedora does, wouldn’t allow a library to be bundled. iUI would have to be packaged separately, and all of its dependencies that you might have bundled in to iUI would need packaging as well.
As for how % of Drupal that is installed via RPM, impossible to tell. But I can point out that the 17+ million Fedora users can install drupal with PackageKit or ‘yum install drupal’. Will they all do that? I don’t know, but why wouldn’t they?
When we were discussing further on Twitter earlier, you asked about Dojo and their 100-points of open source test. In looking through Dojo’s site, I would say they really do deserve kudos for their efforts at making contribution clear and possible. I didn’t run the full FAIL meter on them, but I can say I was able to get my hands on an anonymous SVN check-out within a few minutes of first loading the site. I knew what it would take to submit a patch against that code, and what I would need to focus on to become a full committer. That is a good sign. The biggest challenge they have is their CLA policy. It requires printing and mailing or FAXing the result. We started that way in the Fedora Project five years ago. Some projects still do this. This is a pretty major barrier to participation. Aside from the scary legalese of questionable value, the method of obtaining and submitting the CLA is a barrier to people without access to a printer, FAX machine, or reliable mail service. I would reckon they lose more than half of their potential contributors at that point. Not the super-serious, I-won’t-stop-at-all-costs contributor types. I am thinking of the lighter-weight participants who just want to contribute a small how-to document or code fix, who may one day be bigger contributors, but are daunted by scary legalese to print, sign, and send in dead-tree format to satisfy a lawyer from the 20th century.
Even switching to GPG-signed CLAs was a high barrier for Fedora. When we dropped to the click-through CLA, we increased the people who signed up and accepted the agreement by several factors. That’s not important, it’s the people who will then do something with that new contributor status, that is the group that is now franchised and ready to make a difference.
I’ll note that I’m working on this exact problem for the FreeIPA project: http://freeipa.org/page/CLA_or_contribution_policy … we’re in the final stages of getting that worked out.
BTW, I ran ‘sudo yum install drupal’, and it was installed in the time that it took me to write this reply. But jQuery? No package available. That means I have to go get it from their site, figure how to hook it in, etc. The packaging system front-loads all that dependency work on the packager, so the end-using-sysadmin/developer can just install and get back to work.