Torvalds at LinuxCon Part III: Permissive Licenses and Org Charts

In the last of our three part series that began last week on Linus Torvalds’ keynote interview at this year’s LinuxCon, Linux’s lead developer talks about everything from up and coming operating systems in IoT to the development process.

“You mentioned the strength of the GPL,” Dirk Hohndel said, by now about twenty minutes into his interview of Linus Torvalds at LinuxCon 2016. “Many new kernels have shown up in the last couple of years, mostly geared towards really small devices, the IoT space: Zephyr by Intel, Fuchsia by Google and a bunch more.”

If you are who you work for now, Dirk Hohndel is VMware’s boy. But at the time of the interview, only a few weeks back, he’d been working as VMware’s chief open source officer for less than a month. For almost fifteen years before that — fourteen years nine months he’s careful to point-out on LinkedIn — he belonged to Intel, where he served as chief Linux and open source technologist. Before that he spent six years at SUSE, where he was CTO when he left in 2001, two years ahead of the Novell brouhaha.

“One of the interesting commonalities is they’re all under BSD or MIT,” he continued. “Do you think they’re interesting and do you think that one of them could grow up and become a competitor for Linux or replace Linux?”

“I think they’re interesting just because it’s sort of where Linux came out of too,” Torvalds answered, “like small devices running tiny kernels that were very limited. I think it’s always an interesting space and it’s the only reasonable space where a new operating system can really make a big impact because of the complexities of an operating system, the hardware support and flexibility of supporting lots of different use cases. So you need to start off with something small, and today that means IoT.

“At the same time, the reason I think they show up under the MIT license, BSD or Apache is that most of the time these projects are started by companies, and the flexibility for them to use something like a BSD license which allows them to do anything, they see that as a big upside.”

It’s true, of course. Big companies involved in open source development like to hedge their bets by keeping the proprietary door open and almost exclusively use one of the mentioned “permissive” licenses, or some home brewed license that may or may not be compatible with the GPL. Lately, however, enterprise players seem to recognize GPL compatibility a must for maintaining any kind of open source cred.

“I think that if you actually want to create something bigger, and if you want to create a community around it, the BSD license is not necessarily a great license,” he said.

“I mean, it’s worked fairly well, but you are going to have trouble finding outside developers who feel protected by a big company that says, ‘Hey, here’s this BSD license thing and we’re not making any promises because the copyright allows us to do anything, and allows you to do anything too.’ But as an outside developer, I would not get the warm and fuzzies by that, because I’m like, ‘Oh, this big company is going to take advantage of me,’ while the GPL says, ‘Yes, the company may be big, but nobody’s ever going to take advantage of your code. It will remain free and nobody can take that away from you.’ I think that’s a big deal for community management.

“It wasn’t something I was planning personally when I started, but over the years I’ve become convinced that the BSD license is great for code you don’t care about. I’ll use it myself. If there’s a library routine that I just want to say ‘hey, this is useful to anybody and I’m not going to maintain this,’ I’ll put it under the BSD license.

“Whenever licenses come up, I want to say that this is a personal issue,” he continued, adding a disclaimer most likely meant mainly for the benefit of the BSD folks, some of whom resent Linux’s success, but also to appease big enterprise, which is where the Linux Foundation gets virtually all of it’s funding.

“Some people love the BSD license,” he said. “Some people love proprietary licenses, and do you know what? I understand that. If you want to make a program and you want to feed your kids, it used to make a lot of sense to say that you want to have a proprietary license and sell binaries. I think it makes less sense today, but I really understand the argument. I don’t want to judge, I’m just kind of giving my view on licensing.”

“Do you ever look at the sources of these other kernels?” Hohndel wondered.

“No. I actually used to,” Torvalds started, then stopped himself. “Well, not really. I used to try to, let’s put it that way.

“Back in the very early days of Linux there was some issue I had and I tried to figure it out. I had just hit my head against the wall for a long time and I say, ‘BSD’s out there, I can go out there and look at what BSDs do.’ I don’t know how much time I wasted trying to follow somebody else’s code. For a complicated problem, too. I mean, it was obviously complicated enough that I had issues with it and I went ‘how do I solve this?’

“When you have a complicated problem like that, the worse thing you can do is look at somebody else’s code,” he said. “It doesn’t matter how many comments it has, it’s not going to clarify things for you. It’s like, you need background to understand source code.

“So I made the mistake once of saying let’s see how anybody else solved this problem and I found it not useful for me. Maybe it is useful for others, but I decided no. It’s actually easier to just read up, like a paper or a book about OS design and actually try to understand the problem instead of looking at somebody else’s code and get it that way. This may be personal, but after that one bad experience I just decided no. I’d rather read literature than read source code.

“I’ll read my own,” he added, “and I really want people to read their own source code. You don’t just write it, you read it and try to understand it afterwards too. But reading another project’s source code is really, really hard.”

Hohndel, sticking with the same theme, asked, “Which projects do you look at the sources of, besides the kernel?”

“Honestly?” Torvalds asked. “The last project I looked at the source code of was ZMailer, which is an old mailer program that is still used by Vger, the kernel mailing list, and nobody else. It hasn’t been maintained for ten years or more, so the only way to figure out problems is to look at source code. I did figure out a few problems and we’ve fixed a few issues on the kernel mailing list with email authentication issues we had.

“I think it actually is useful to look at other project’s code if you want to fix a bug in that other project. That is how we get most if the kernel developers. A lot of bugs are really easy to fix; they may be one liners. You don’t look at a project’s code to write your own, you look at a project’s code because you want to fix that project. You start out small and 25 years later you find out you’re a big maintainer. That’s how we all start.”

“Yes,” Hohndel agreed and changed the subject. “Let’s talk a little bit about this maintainer thing. One thing that I find fascinating about the Linux development process is it’s completely unique in its scale and its velocity. Do you think this is something other projects or even companies for in-house development can learn from? The way Linux does things with the clear maintainer roles, the tree structure, with the timed releases and all that?”

“We don’t actually have clear maintainer roles,” Torvalds replied, and pretty much confined his answer to that aspect of the question. “I doubt anybody here can even Google and find a maintainership-like traditional org chart for Linux. Maybe somebody has tried to do this. I would guess that maybe you would find it on LWN. You would find that ten years ago John Corbet probably tried to make an org chart for Linux and that’s why he’s bald now.”

That brought a small ripple of applause from the audience, one of only two or three laugh breaks in the thirty minute interview.

“There are two classes of people who think we have a clear maintainership chart. One is the outsiders. My guess is that from the outside this all looks very organized. We have releases every ten weeks. It looks very organized and its very smooth. Then the other group is the people who have been around forever. They just know who maintains everybody, not because there’s documentation and not because we have an org chart. It’s just because when you’ve been around forever you just know. Oh, it’s a driver, bug Greg about it, or David handles networking and if it’s wireless you go to somebody else.

“So we have maintainership roles but they’re not — and I think it’s nice that they’re not — like an org chart,” he continued. “Because one of the downsides of org charts is all the politics. You want to be up, right, because it makes you look good, so you have the jockeying for position. When you don’t have an org chart, you can’t really show off your position on the org chart to anybody. Some of the politics goes away.

“The other thing is we’re actually way more flexible when it comes to things, and people have been trying out different models, like having two or three people work as maintainers of one subsystem. We now have one subsystem that is playing around with having two maintainers but fifteen people who can actually commit. which sounds a bit odd and we don’t know if it works, but maybe it will work. We’ve tried a lot of these kinds of experiments with maintainership. I wouldn’t say that most of them work. A lot of changes that on the surface look completely crazy, do actually work really well after you give it a few iterations and you kind of make the obvious problems that you didn’t think about go away.

“So we have this fairly fluid org chart and it literally has been changing. It’s not even a hierarchy. I would say it’s more of a network of people that sometimes goes across boundaries, just because you can. Email doesn’t really care; you said it too.”

“I always love it when you send your patches to some other maintainer to integrate so later you can fold them,” Hohndel commented. It seemed like something of a non sequitur. Perhaps it was because time was running out and this was a comment he’d wanted to make going in.

“If I write code it depends a bit on the area,” Torvalds shrugged. “Sometimes I will just commit it myself because it’s some trivial fix and I’m on a schedule — it’s like something stupid that needs to be reverted for a release, then I will commit it myself. Quite often I will say, ‘Hey, this is my fix’ and I’ll send it off to the real maintainer of that area and most of the time they commit it. I seldom get rejections, but if they want to improve it, that’s fine too.”

At this point Hohndel indicated the interview was nearly over, but that he had one more question. “I’ve asked this question of Linus a few times when we’ve had these conversations,” he said. “What happens when there is no Linus anymore?”

By now, you might think he would no longer take this question seriously, but he does — or at least that seems to be the case — probably because he’s proud of what he’s created and of the fact that his creation will likely be a legacy that will outlive him. He also probably understands the importance Linux has in people’s lives, from the tens of thousands or more who manage to earn a living from the operating system, to the unknown number of home users who use the OS to get through school, get work done, or simply to write a few emails and spend time on social networks.

He may have clay feet like everyone else, but he’s at least good enough to return the gratitude that’s given to him. As he said about his fellow team members earlier in the interview, he may not always be polite, but that’s okay. The world is full of polite people who are waiting for the opportunity to stab you in the back.

“I think it would have been a bigger problem fifteen years ago, maybe even ten,” he answered. “These days there are so many people who could take up the work, the mantle. Most of the people who could do it wouldn’t want to do it, I think. They’ve seen enough that they say, ‘No it’s better to let Linus handle it because there is going to be conflict and I don’t need that headache.’ But there are people who have been around for almost as long as I have been and there are people who are almost universally trusted and who do a lot of the work today that could step up.”