Solus Is Back, but Can It Survive Its Troubled Past?

With Ikey Doherty again taking a central role within the Solus Project, it’s a fair question to ask how long he will stay before abandoning Solus for the third time.

A Linux distribution that many thought was probably gone for good has risen from the dead.

On July 8, the Solus project announced that its once popular distro, which had given birth to the popular Budgie desktop environment that’s now the default DE in several other distros including an official Ubuntu spin, had released version 4.4, code named Harmony, as its first release in two years.

That’s a long time for a distro to go without a release. Long enough that in April the website Distrowatch, which earns its keep by tracking the comings and goings of Linux distros, changed Solus’s status to “dormant” and took it off its list of active distros.

The time element alone wasn’t why a lot of people had given the distro up for dead, and probably not the sole reason why Distro chose to put it on its “dormant” list. After all, numerous other distros have experienced similar delays between major releases.

With Solus, however, there had been several incidents that had played out publicly that seemed to indicate internal problems serious enough to possibly spell the project’s doom.

The most recent began on January 19 of this year, when the project’s website, including its developer tracking platform and its forum, suddenly became unavailable due to what the project described as “a libvirt dnsmasq issue.” Getting the website back up took about three months, mainly because by that point there was only one person in charge at the project, technical lead Beatrice Meyers, and she was sick.

So, why was there only one person in charge at a popular distro with its own scratch-built DE that was giving both Gnome and KDE runs for their money in the popularity department?

The answer to that points to another incident within the organization having to do with internal problems. Exactly a year earlier, in January 2022, the main developer of Budgie and the co-lead of Solus, Joshua Strobl, abruptly said he was leaving the project in a rambling post that indicated that all was not well at Solus:

“In October, additional behavior brought me to a breaking point. I basically resigned from Solus at that point, in our development channel. If it wasn’t for folks reaching out, I would’ve done followed through.

“And here we are again, with further issues not being addressed and rather than the parties accepting their part / responsibility, it’s been just deflected and not acknowledged. No issues I raised addressed, nothing I felt needed for Solus’ community improvement addressed either.”

Strobl said he would be joining Ikey Doherty, Solus’s founder who had left the project several years earlier, at SerpentOS. He also said that he was forking Budgie, and moving control of it away from Solus to an organization that came to be called Buddies of Budgie.

Déjà vu All Over Again

This wasn’t the first time that a lead developer had abruptly walked away from the project. Nor was it the first time that the project had grappled with everything online being down with no one to fix it.

Nearly four years earlier Solus’s founder, Ikey Doherty, walked away from Solus and left the project in a mess. Soon after he left, Solus lost access to its website and many of the online services it needed, apparently because Doherty hadn’t paid the bills. Compounding the issue, especially since most of the project’s infrastructure was in Doherty’s name, the remaining team at Solus was unable to reach him, because he’d either deleted or quit visiting his social media accounts.

This eventually led the team to move the project to a new domain name, during which time Doherty finally contacted the team, evidently by email, with an all lower case message that read, “am very very sick atm. all will be paid up for the next 30 days and gives me time to get back out and transfer to you. i’ll speak with you tomorrow afternoon/evening.”

According to John Paul Wohlscheid, who covered the incident for It’s FOSS, “Ikey never followed up on this message.”

Some time later, Doherty publicly resigned from the project in a letter that said in part:

“As a final request to the Solus team, from one who considers himself a friend, I ask that you stand firm in your cause and rise above the toxicity and politics that plague the Linux desktop world, to remain a bright beacon of hope, an example of how open source should be….

“For this reason, I happily condoned their leadership of the project, and I assign all intellectual property rights, names and brands related to the Solus property to their collective with immediate and permanent effect, recognizing them as official owners and leaders of the project.”

Not long afterward, Doherty started another Linux distro, Serpent OS, which is where Joshua Strobl ended up after he quit Solus.

Before There Was Solus, There Was Solus

The distro that’s called Solus today isn’t the first Linux distro to be called Solus to be founded by Doherty. Nearly 10 years ago, in October 2013, Doherty similarly walked away from another distro he’d started that was also called Solus.

In the first article that Ken Starks penned for FOSS Force, he wrote about SolusOS, which at the time had been around long enough to see several releases. Starks liked its use of PiSi as a package manager, and spoke of Ikey Doherty in glowing terms:

“Ikey isn’t just some hack that re-themed someone else’s work. Doherty is building SolusOS from the base up. He pays his bills by writing and/or debugging software for a major player in the IT industry, but his passion is SolusOS. The majority of his work as lead developer is package management. With PiSi, not only will his life be easier, PiSi is…well it’s easy PiSi. Even I can package stuff with PiSi…”

In the article, he revealed that Doherty had agreed to develop a custom educational version of the upcoming SolusOS 2.0 release, specifically to meet to needs of Reglue, a nonprofit that Starks operated in the Austin, Texas area that repurposed old PCs and laptops with Linux for elementary and high school students who couldn’t afford computers. In typical Starks fashion, he was going to call the specialized distro Quantum of Solace, after the James Bond movie.

Ten days later, Starks penned another article about Solus OS to explain why, less than a week after Starks’ original article and only four days after Doherty posted about a new collaboration between SolusOS and the Numix desktop project, Doherty suddenly announced on SolusOS’s website that he was shutting the project down.

“It is with a heavy heart that I must announce the closure of SolusOS. Simply put, there is no longer enough manpower to fulfill the vision. What began as a Debian derivative evolved into an independent distribution, without the large development team required to back such an effort.”

Evolution Solus

About two months before Starks’ original article on Solus, Doherty took a job at Intel’s Open Source Technology Center as a software engineer, primarily to work on the Clear Linux Project, and the demands of that job might have been partially responsible for Doherty’s decision to shut SolusOS down.

A year or so after that, sometime around the start of 2015 and while still working at Intel, Doherty started another Linux distro called Evolve OS. In April 2015, in response to a trademark dispute, Doherty renamed the project by reviving the Solus name, as Larry Cafiero reported in FOSS Force:

“According to an article this week in Softpedia, when the United Kingdom’s Secretary of State’s office notified Evolve OS that their name infringed on a trademark, lo and behold Evolve OS took the next evolutionary step and became Solus OS, a name well within the new Solus OS family.

“Problem solved, sort of. Unlike the original Solus OS, which was based on Debian, Evolve OS (and now the new Solus OS) was built from scratch, allowing the developer and the development team more control over the process. So any connection between the old Solus OS and the new Solus OS is purely coincidental.”

Cafiero’s observation that the operating system that started life as Evolve OS is an entirely separate project from the original Solus OS is worth noting, because on Doherty’s LinkedIn employment timeline they appear to be the same built-from-scratch distro.

Life After Solus

After walking away from the second Solus project and leaving Intel, which happened at about the same time, Doherty spent a couple of years as CEO of a gaming company he founded, Lispy Snake, before founding Serpent OS, in 2020. In a recent review, Distrowatch’s Jesse Smith found Serpent OS to be an ambitious project that’s nowhere close to being ready for prime time:

“All of this sounds like a pleasant and smooth experience, but how does it work? As far as I can tell, there is no end-user documentation for Serpent OS. I couldn’t find an install guide or tips for using Moss. Even the overall goals of the project, the reasons for Serpent OS to exist, were quite vague:

“The team have developed a set of principles and goals for what represents Serpent OS. Our goals are to enable you to: Take control of your computer; get the most from your hardware; enjoy a reliable experience; improve your experience behind the scenes; develop and contribute fast and efficiently; be part of a thriving community; have fun while interacting with other users and contributors.

“All of this sounds nice, but it also sounds like the mission for every open source project and high school computer class. While I didn’t find any practical documentation for end users, I did find repositories where software is being packaged. There isn’t a lot there yet, compared with other Linux distributions, but there is a promising start.”

So, what is Serpent OS, other than yet another Linux distro? The distro’s website explains it with a couple of tablespoons of jargon:

“Serpent OS strives to provide the perfect hybrid experience – a traditional, package management driven Linux distribution powered by the latest technologies. Stateless by design with atomic updates at the core, Serpent OS aims to close the gap between development and production.

“Whether you’re a Linux professional or a home user, Serpent OS has something for you. With deduplicated on-disk transactions, the last working version of your system is only a reboot away. Any freshly installed software is immediately available giving you the flexibility of traditional package management and the peace of mind of atomic updates.

“The project is in a highly active stage of development, having achieved full bootstrap with the LLVM toolchain and integration of our home grown tooling, boulder and moss.”

Sounds nice and all, but mostly it sounds too complicated for me to want to try it.

Doherty’s Solus Return

One of the results of the downtime that Solus experienced earlier this year is that both Doherty and Strobl are once again officially part of the Solus project. Strobl holds a a leadership role that’s something akin to being project leader, although as far as I can tell he’s not calling himself that. Doherty returns as something of a consultant, although his work on Serpent OS will be key to the success of Solus’s plans going forward.

In a blog written by Strobl and posted on Solus’s website in April, Strobl explained how that came to be:

“[T]he Serpent OS project, led by Ikey Doherty … reached out to offer hosting of the website to facilitate Solus’ ability to communicate with its user base and serve ISOs (which was declined). Concerned by the continuing outage and its impact on Solus users and contributors, the Serpent OS team then shared a proposal which would allow Solus to resume operations with less technical debt and mitigation of the apparent bus factor of one.

“This proposal was well-received by active Solus team members and led to extensive discussions and eventual agreement regarding a sensible path towards infrastructure recovery and a viable future for Solus.”

According to Strobl, that eventually led to the formation of a new organizational structure within Solus that is more horizontal than vertical, or as Strobl put it, “flatter.” In addition, it opened a door for more community involvement in the development process, instead of relying on a small group of key developers.

“Historically, the hierarchy of the Solus management did not facilitate or promote the idea of shared domain ownership or organizing into groups that could work independently to keep the Solus ship sailing smoothly. Management was organized into the ‘Core Team’ and ‘Global Maintainers,’ with Core Team being responsible for most of the administrative work and a considerable amount of decision-making. In contrast, Global Maintainers were mostly involved in maintaining a subset of Solus packages, such as desktop environment stacks and individual packages. The purview of Global Maintainers was typically viewed as being confined to ‘their’ individual stacks, not allowing them much agency beyond that.”

“The new Solus organizational structure is built around the philosophy that the sum of the whole is greater than its parts. This structure, by virtue of being flatter and less strictly delineated, is deliberately designed to provide more opportunities for the community to get involved in various areas. This will allow people to showcase their amazing skills and talents in a context that encourages personal and professional growth through more areas / avenues of collaboration and learning. We intend for community members to be able to evolve in new and exciting ways, treating their experience using and contributing to Solus as an adventure where the journey is just as important as the destination.”

Doherty’s role in the new organizational structure at Solus may seem minimal, but it’s not. He serves on the new Steering Committee, is involved in Community Engagement and Communications, and is a Stack Maintainer. In addition, he will be key to Solus’s future success, given his role as project lead and chief architect at Serpent OS, which will become Solus’s upstream source beginning with Solus 5.0.

Moving away from being a made-from-scratch distro to become a distro based on Serpent OS is now seen as being essential by Solus’s leadership, as Strobl pointed out in his blog:

“[I]nnovation in the Linux ecosystem is presently centered around the use of application sandboxing, containers and the development of immutable operating systems with a well understood Software Bill of Materials. Each of these concepts allow for a degree of separation and stability when developing, testing and certifying software and products.

“The current Solus tooling, as well as the resulting packaging and development experience, is somewhat ill-suited to this objective and would most likely need a wholesale re-engineering of the tools before this becomes feasible.

“However, there is a more straightforward path for Solus: Rebasing onto Serpent OS.

“Serpent OS is an independent, built-from-scratch Linux-based Operating system. It is first and foremost a project built on engineering-led, scalability-driven infrastructure and tooling, which is designed to vastly reduce the burden on project contributors via tight integration of modern devops practices.”

“The Serpent OS moss package manager mimics the familiarity of eopkg in daily use, marrying advanced and modern underpinnings with the celtic magic of simple yet powerful system package manager operations.

“This results in Serpent OS being able to merge the world of traditional Linux distribution practices that we all cut our teeth on, with the advanced features expected from next generation tooling: offline rollbacks, atomic updates, immutability and continuous, verifiable delivery of packages.”

This all sounds well and good, but as Jesse Smith pointed out in his review of Serpent OS, the distro is nowhere near ready to be a prime time contender. Strobl admits as much:

“Currently, Serpent OS is in the development stage with functioning prototype tooling, infrastructure and ISO images. Serpent OS is not yet in a state for end users to rely on as their development or daily driver system. However, even in its current prototype state, the Serpent OS tooling is already more capable and advanced than the equivalent (and originally planned future) Solus tooling.”

In other words, all of Solus’s future plans are dreams until Doherty whips Serpent OS into a stable state suitable for use in production. This again puts Doherty’s vision and his skill as a developer front and center if Solus is to bring its plans to fruition.

That thought feeds directly into a question that’s been nagging me since I learned of Doherty’s return: how long will it be before Doherty walks away and abandons the project again?

I’m pretty sure I’m not the only one who’s wondered the same.