It appears as if much of the open source infrastructure we depend on is suffering from neglect. That’s the message brought to the SouthEast LinuxFest (SELF) by David Nalley. Listening to his talk, “The Tragedy of Open Source,” it was hard not to think that some of our infrastructure projects are beginning to resemble some disintegrating municipal water and sewer systems, or maybe compare his examples with our crumbling roads and bridges. Nalley is a South Carolina based “recovering sysadmin” who now wears many hats at Apache as well as being an employee at Citrix.
The neglect he mentions has caused more than a few near misses that fell inches short of disaster, with two major incidents happening last year alone.
Take the Heartbleed vulnerability that affected openSSL. Nalley points out that last year when the bug was discovered, there was only one person, earning a mere twenty grand a year, actively maintaining the openSSL project. Also last year, there was only one person maintaining bash when Shellshock was discovered.
Lest you think these are isolated exceptions, they’re not. Take the case of GnuPG. This popular FOSS replacement for PGP has only one maintainer. Does that make you feel secure in the age of Snowden?
At Apache there’s a metric called the Pony Factor, which Nalley watches when evaluating the health of projects. Basically, the factor identifies the smallest number of people writing 50% of a project’s code over a two year period; the bigger the number, the more vibrant the project. However, even some relatively large projects show figures that are downright scary. For instance, at Git one person has written over half the code over the last two years. At Perl: Three people wrote at least half the code over the same two year period.
Indeed it is. In today’s online world, fraught with security issues, I’d hate to be running a website on a Perl based platform knowing that.
The problem, as Nalley sees it, might not be dissimilar to what we see happening with our roads and bridges. New roads and bridges are being built all the time because the public loves new roads and it helps politicians get elected. Not so much with maintaining those roads and bridges after they’re built. Hence, we see tragedies such as the 2007 Minneapolis bridge collapse that took thirteen lives.
“Much of the time we’re focusing on new functionalities,” says Nalley. “We’re not focusing on maintenance.”
He points out that over the last year or so, Google has spent more money developing a set of fonts to be used in its advertising programs than openSSL has spent for the entirety of its project. No slam on Google, of course. This isn’t about how much Google is spending, it’s about how little is being allocated to projects like Git, openSSL and bash by open source software companies who depend on the viability of these projects.
However, maybe we can put that last statement in past tense.
Evidently Heartbleed was something of a wake up call, as this vulnerability prompted Linux Foundation executive director Jim Zemlin to quickly get thirteen tech companies to fund a new project, the Core Infrastructure Initiative (CII), to the tune of $100,000 per company per year. This money is disbursed for such things as paying developers to work full time on OSS projects, conducting reviews and security audits and helping to facilitate travel and meetings among developers. Since the initiative began in April of last year, five additional tech companies have come on board.
It didn’t take long for this project to prove its worth. In September of last year, a mere five months after CII was born, the initiative was able to offer assistance to bash maintainer Chet Ramey after the discovery of Shellshock.
You can watch David Nalley’s presentation at this year’s SELF below. The video portion is mainly slides, so feel free to listen while you do other things.
Christine Hall has been a journalist since 1971. In 2001, she began writing a weekly consumer computer column and started covering Linux and FOSS in 2002 after making the switch to GNU/Linux. Follow her on Twitter: @BrideOfLinux
I’m not at all talented enough to work in any of the fields mentioned but I do know a lot of people that are working in some fairly important projects and the one thing I can tell for certain.
A bunch of these guys feel like the little Dutch boy.
What it comes down to is this? Who’s going to lose their job when the next heartbleed hits? Or the next Shellshock? No one. No one will be held accountable in most cases. We’ll all shake our heads and bemoan the fact that there’s not any money to pay the position that watches over this or that and the ones that are paid are more a stipend than a paycheck. “It was bound to happen sometime, tisk, tisk, tisk…”
There’s a time when we need to give more thought to the philosophical meaning of Free in Open Source Software, and not the amount of Free beer everyone gets when things are going good. I’m pretty sure the calendar is saying that time is now. Ya think?
Only the strong survive.
I’ve maintained that as long as we are in a global deep-seated depression (and we are, despite what a micro economy in Austin, TX does) it becomes all the more difficult for contributors to do work ‘gratis’ when they have all they can do to put food on the table and scrap financially with two jobs to survive.
It’s not good out there in case you haven’t noticed.
So, it’s Distros that have the resources to rely upon like Fedora which will weather the storm and survive in the long run. Fedora not only codes for Fedora but also (Red Hat) does a significant amount of upstream code development.
The rest of the ‘me too’ Distros on the margins in the Distrowatch swamp will whither and die off leaving only a handful of strong.
Of course, David Nally can throw some statistics at the world, and say ‘that’s scary’. Anybody can.
But are those statistics at all meaningful ?
How does commercial software measure up ? And then I don’t mean windows (although it would be interesting to have statistics about all *individual parts* – programs, DLLs, etc. – of windows – I suspect you’d be surprised…), but other software ?
I think most commercial software does not fare better. Just like FOSS developers prefer developing new features over fixing bugs, companies also prefer developing new features. Usually, customers pay more for new features, than for fixed bugs, except maybe if the bug is a *serious* security issue. I dare even say that FOSS developers will more often take pride in improving the quality of their work (i.e. improving the code and fixing bugs) than that companies will pay developers to fix bugs that nobody is loudly complaining about.
Also, I’d rather have one dedicated FOSS developer committed to improving my favorite tool, than a, often changing, group of developers, probably about just as skilled and with about just the same level of security-awareness (or lack thereof…) as the FOSS developer, each committing one or a few patches to a tool in turn, depending on who’s still on the team, and who happens to have be assigned to code the newest fix or feature. The former will develop a deep and thorough understanding and familiarity with the tool’s code over time. Each of the latter may not acquire more than a fleeting understanding with the code, increasing the chance of bugs and security issues.
One advantage of the commercial development team, might be that it is backed by a company that – hopefully – still feels responsible *and* commercially motivated (which is not self-evident at all!) to make sure somebody is working on the tool at all.
So I think the pony factor in itself is just a statistic, a number, and meaning can only be attached to it if backed by scientifically sound research which correlates the factor with some objective measure of quality. I haven’t done an exhaustive search, but I get the impression no such research really exists, which leaves the pony factor as what it is: just a number.
Christine. how much money has Google spent trying to control everything Android mobile users do, Google is under investigation in Europe for unfair competition. The question that should be asked is “Why do American corporates want to lock down and control everything by hook or by crook their users do, Google will flip on their back like M$ has done, I for one won’t have anything MS or Google on my systems. People are sick of being brain washed by advertising
Carling, u say People are sick of being brain washed by advertising.
I say; just look at Apple.
Some people don’t mind.