Trees on the Mountain: The Yesterday and Today of Open Source Software
The more it seeks to rise into the heights and the light, the more determinedly its roots strive earthwards, downwards, into the darkness, the depths—into evil.
Prologue - The Fragile (?) Nature of Open Source Software
Recently, Henrik Lissner, the author of doom-emacs, who had been absent for nearly half a year, spoke up in the doom-emacs Discord channel:
The gist of his message was:
Hello everyone! I’ve been away from the project and community for about a month, dealing with family matters. If you count the health, internet, and moving issues I faced last year, it’s been even longer. Worse, I didn’t keep you informed (or give advance warning) about these things, and I want to apologize for that. Combined with a few failed attempts over the past months, I’ve started to make people worry about Doom’s future and its Bus Factor.
To be clear: I have no intention of abandoning Doom — it’s just been an unfortunate and distracting two years for me — and I hope that’s behind me now.
After spending a week catching up, merging PRs, and updating modules, I’ll post a larger announcement on Discourse detailing the current situation, the project’s direction, and how I plan to address the bus factor and communication issues going forward.
In the meantime, you can find me in the #development channel; if you have any questions about the project, have recently sent me a DM/email (and are expecting a reply), or have any PRs ready to merge or needing my attention, feel free to reach out there.
Setting aside for a moment what doom-emacs is, Henrik mentioned a very interesting term: Bus Factor.
Wikipedia’s explanation of this term is:
The Bus Factor is a measurement of the concentration of information and capability within a software project team, indicating how much is not shared. Some also call it the “truck factor” or “lorry factor.” It refers to the number of key team members whose sudden absence (e.g., “hit by a bus,” representing any reason for absence such as career changes, lifestyle shifts, marriage, childbirth, accidents, or death) would cause the project to descend into chaos, paralysis, and ultimately failure. These members are considered “indispensable” to the project, meaning the project knowledge (e.g., design, source code, expertise) they possess is not widely known or shared among other members. In a project with a high Bus Factor, even if someone suddenly leaves, many others possess sufficient project knowledge to take over their role.
A low Bus Factor is actually a common ailment of many open source software projects. If you frequent GitHub, you’ve likely noticed many projects with high star counts being abandoned due to lack of maintenance.
The most extreme example occurred in early 2022 when Marak, the author of the well-known open source tool Faker.js, intentionally sabotaged the repository on GitHub. He cleared all the project code and left the word “endgame” in a commit. That day, front-end engineers around the world arrived at work, bewildered to find their projects malfunctioning and displaying large chunks of garbled code.
The “Endgame” Garbled Code
Additionally, another library he developed, colors.js, was also affected. Both of these open source libraries are foundational pillars in front-end engineering. For instance, colors.js alone sees over 20 million weekly downloads on npm (you can think of it as being used by virtually anyone doing front-end work).
In 2020, Marak faced the predicament of his apartment burning down, leaving him homeless. The sponsorship he received from his open source projects was insufficient to cover his living expenses. His life became increasingly difficult, leading to his deeply regrettable and desperate decision in early 2022.
To expand on this, there are many differing opinions on this event: - Some argue that if you don’t want large companies freeloading off your work, you should use a copyleft license like GPL or even AGPL, which is “viral,” rather than the MIT License. The MIT License grants everyone the right to use your code freely from the start. However, whether a project would have achieved its current level of influence under GPL is another matter (commercial companies typically have legal teams review open source dependencies before release to ensure there are no GPL-licensed dependencies, otherwise they’d be forced to open-source their entire product). - Others believe that by this point, countless open source contributors have added code to the project, making it a collective creation of the entire open source community. You don’t have the right to turn everyone’s hard work into shit. - There are also voices supporting Marak. For example, some say, “These parasites have been sucking our blood for too long; at least you made them feel the pain!” - Some also approach it from a license perspective. While people did contribute code, strictly speaking regarding the license, this was Marak’s personal project. The MIT License grants all rights but also absolves the author of any additional obligations. Therefore, from a legal standpoint, Marak was completely within his rights to do this.
The Current State of the Cloud Infrastructure Supply Chain xkcd-2347
There have been several similar regrettable incidents. For instance, Marak, the protagonist of the Faker.js incident, mentioned Aaron Hillel Swartz, a programming prodigy who contributed to the early development of RSS and co-founded Reddit. He advocated for open knowledge and actively opposed internet piracy legislation. He practiced what he preached by systematically downloading academic journals from JSTOR using a guest account provided by an institution and making them publicly accessible. U.S. federal prosecutors subsequently charged him with two counts of wire fraud and eleven violations of the Computer Fraud and Abuse Act. He rejected a plea deal that would have sent him to federal prison for six months. Two days later, he was found dead in his Brooklyn, New York apartment, having died by suicide.
Aaron Hillel Swartz
Looking back, the open source spirit and open source software are perhaps one of the most incredible phenomena of our time:
On one hand, the open source software movement and the free software movement are themselves incredible. Groups of intellectuals actively dismantle their professional moats, often acting against commercial logic to practice the ideal of “all for one, one for all” and a vision of universal harmony. Why? Because the world should be this way, for the prosperity of society as a whole, because it’s cool, and because writing software itself brings immense joy to its authors—I remember thinking when I graduated: “Wow, I can get paid for writing code? My life is amazing!”
On the other hand, factually speaking, it’s no exaggeration to say that open source software underpins the entire modern software industry. The software industry has been utterly transformed by open source, to the point where MIT’s famous introductory computer science course, 6.001—the renowned SICP (Structure and Interpretation of Computer Programs)—has been discontinued. Professor GJS stated: “Modern software engineering has become so complex that it resembles science more than engineering, because no one needs to build large systems from scratch anymore—and very few have the capability to do so.”
You might sigh and say, “For things to have come to this, for idealists to face such circumstances, it’s truly the shattering of ideals!”
But is that really the case?
The Dynamics of Open Source Software
Idealism is good, but ideals must be rooted in reality to take root and sprout. Let’s invoke Pierre Bourdieu’s sociological theory of “Field, Capital, Habitus” to better explain all this.
[(Habitus) (Capital)] + Field = Social Practice
Simply put, think of a socially constructed space (a field) centered around a specific theme as a kind of game, and we are all players.
As players, although we often feel like free agents on an experiential level, the various decisions we make daily are often based on predictable, prescribed behaviors and the attitudes of others—that is, a habitual strategy formed through interaction with the game itself (habitus).
Influenced by this habit, we engage in games and struggles according to the rules of the game and our adopted strategies, based on our current position (capital), to occupy a more advantageous position within the game.
The entire organism of subjective and objective elements described above constitutes (my highly simplified and unprofessional summary of) Bourdieu’s worldview for the social sciences.
Returning from the game metaphor to the real world, similarly, no one can act without some form of interest, and the goal of that action is likely to increase the capital one holds.
Oliveira winning IEM 2023, making “Faker” known to everyone
You might think, if that’s the case, then how do we explain altruistic behaviors we see in production, practice, and daily life; the spirit of dedication in fields like art and science; and the open source spirit discussed in this article?
According to Bourdieu’s definition, capital is not merely economic capital; it also includes symbolic capital encompassing cultural capital, scientific capital, and more. Recognizing different forms of capital, Bourdieu does not believe there are any purely disinterested actions in social production and practice. From a perspective akin to game theory, he posits that all human actions are based on calculations of some form of interest, aimed at acquiring some form of capital—if not the economic capital we typically think of.
Within different fields, different forms of capital play a dominant role. Simply put, in academia, it’s scientific capital; in art and culture, it’s naturally cultural capital. Different forms of capital in different fields have a kind of “exchange rate,” so we often see political or cultural elites leveraging their influence to acquire economic capital, and vice versa.
Here, I’d like to digress. The progressive nature of Bourdieu’s theory, in my view, lies in the fact that all fields are created by humans, not emergent from human society. It explains why many phenomena are the way they are, while also implying that seemingly entrenched ideas and social phenomena are not necessarily immutable. In other words, there is no purely “natural” or inevitable “heavenly principle” here. This opens up the possibility for overthrow and progress; history has not ended.
So you’re the one who thinks history has ended, huh? Let me tell you, the end of history has ended.
Returning to the topic, viewing the open source spirit through this theoretical framework can be explained as: within the field of open source software, hackers (note: Hackers—people who enjoy challenging problems through intelligence and creativity—are absolutely not Crackers, computer criminals. The Chinese internet often conflates the two, leading to the stigmatization of the term “hacker”) contribute open source code to the world to accumulate cultural capital such as “reputation.”
This actually shares a lineage with the gift culture often found in small hunter-gatherer groups before the advent of agricultural society. Supported by nature’s abundant gifts, social relations were sustained by this gift culture, and tribal chiefs often engaged in public acts of charity. As stated in the famous hacker culture anthology The Cathedral & the Bazaar: within the field of open source software, there is no scarcity of “essential goods for survival.” Early software was freely shared. This material abundance meant that the only measure of success was reputation among peers.
However, hacker culture and software did not remain confined to a small, resource-rich circle. Contrary to Professor John von Neumann’s prediction that “the world would need only about five computers,” software is indeed “eating the world” today. This is good; it shows that software plays an irreplaceably huge role in human production and life. On the other hand, under the influence of capitalism, the penetration of the software industry into other sectors inevitably leads to the overlapping and merging of fields, forcing the industry to some extent to conform to the logic of the capitalist world. When computer technology and the software industry moved from a small-circle culture of well-off, middle-class American hackers to the entire world, becoming an indispensable part of human production and life, the original core of the open source spirit was bound to be challenged.
Therefore, open source software was destined to move from its initial utopian, pastoral idyll belonging to a small technical elite to the masses, beginning to embrace the capitalism and commercial logic that supports the world’s 7 billion people.
Countless historical examples show that transitions from one mode of production to another are inevitably accompanied by pain. But as historical materialists, we do not view the journey from Athenian democracy in ancient Greek city-states to capitalism as a regression in history. On the contrary, it represents significant progress.
The Current State of Open Source Software and Its Typical Business Models
Since the former Microsoft, with its infamous “Embrace, Extend, Extinguish” strategy, strangled the then-star software Netscape to win the browser wars, leading to Netscape’s rebirth as Mozilla Firefox (today’s Firefox) and sparking the open source movement; to the successful IPO of open source vendor RedHat in 1999; to today’s “Microsoft Loves Linux” (TODO: image); and the continuous emergence of startups driven by open source software, I believe it’s safe to say that, at least up to now, even after embracing the capitalist commercial logic it once “disdained,” the open source movement is far from being a failure.
CNCF Landscape, listing the entire ecosystem of various software projects in the cloud-native field
However, open source software has indeed evolved into two distinct forms:
The first is the original, utopian form of first-generation open source software, born from hackers’ own needs, written and open-sourced for other hackers to collaboratively build upon. Legendary projects like Vim, Emacs, Python, Linux, and PostgreSQL, which have stood the test of time, have essentially become symbolic identifiers of open source software, revered and celebrated.
I got into Emacs because I found Notion so incredibly useful that it prompted me to examine my documentation tools, leading to a question: “Do I really want to entrust these notes, which might accompany me for a lifetime, to a commercial company that, while currently popular, likely won’t outlive me?” I then deliberately explored and started using Emacs’ Org-mode for note-taking and task management, only to discover that Org-mode was, for me, far superior to Notion. However, such needs are relatively niche. Human demand for software is more about solving current problems faster, better, and cheaper, giving rise to the second form of open source software.
The second is the newer form, driven by commercial logic. Beyond practicing the ideal of “all for one, one for all,” these projects (and the companies behind them) treat open source as a strategic consideration and market tactic. Examples today are countless: MongoDB, ElasticSearch, Akka, RedHat, Ubuntu, GitLab, the HashiCorp stack, etc. Today, several proven, viable business models have formed around open source software. Modern entrepreneurs don’t even need to convince VCs familiar with open source why an open source startup can succeed. For example:
- Creating distributions or solutions around upstream core open source software, selling licenses or software subscription services in exchange for ease of operations or software availability guarantees. Examples include Ubuntu and RedHat for Linux, various public cloud Kubernetes container services, and public cloud database services (which led MongoDB to change its open source license to prohibit commercial competition after being heavily “freeloaded” upon).
- Open-sourcing basic features for free while keeping advanced, commercial/team features proprietary and paid. Examples include GitLab, InfluxDB, Grafana, Envd/Modelz, and the HashiCorp stack.
- Providing consulting services around complex, large-scale open source software like Linux, Kubernetes, PostgreSQL, and Prometheus.
The reason open source business models can work is that source code does not equal all the knowledge surrounding the software: including design philosophy, architectural evolution history, and, most importantly, operational knowledge for stable running. This knowledge, crucial for upper-layer applications that need to run stably on top, is often held only by core developers, forming the basic logic for commercialization.
For example, it’s unrealistic for a company whose main business is not providing various public cloud services to build its own cloud infrastructure from scratch, starting with its own data center, instead of using AWS or Alibaba Cloud services, just because it needs cloud servers, databases, container services, etc.
You might ask, if the business is different and the source code isn’t a concern, then does it even matter if the software is open source?
In these fields, if a commercial company tries to sell its own proprietary software—say, an operating system compatible with the POSIX interface—it will inevitably face this question: “How is your software better than Linux, which has been tested for 30 years?” We find that nowadays, no enterprise attempts to compete directly with Linux. Instead, everyone focuses on differentiated aspects.
I feel very excited about such changes in open source software and the software industry as a whole. Why do I say that?
Epilogue - The People’s Historical View vs. The Hero’s Historical View
Henrik, mentioned at the beginning of the article, is undoubtedly an inheritor of that star-studded utopian era in today’s world—an absolute hero. It was Henrik’s doom-emacs that gave me, a Vim user, a foothold in the world of Emacs and even provided a documentation writing experience far superior to Notion. It was Henrik who led me to discover the magical NixOS, allowing me to finally bid farewell to endless system configuration and painful rollbacks.
Although regarding this minor Bus Factor incident, Henrik stated he would absolutely not abandon doom-emacs and promised to give the community an explanation. Sooner or later, however, heroes will fade. When that time comes, what should we, who follow in the footsteps of heroes, do?
The net is vast and infinite. I really like the ending where Motoko disappears into the network. The audience doesn’t need Motoko as an idol, and Motoko doesn’t need to bind herself with her ego.
In the field of documentation, from the thriving community of Emacs/Org-mode, tools and startups like LogSeq (seed round: $4.1M) have emerged. Even in the shadow of Notion, open-source star teams like AFFiNE (Pre-A round: tens of millions of dollars in funding) have been born.
Similar things are now happening across various sectors of the software industry. All of this is built upon the solid infrastructure laid down by generations of open source software and idealist predecessors. Industry giants can no longer easily monopolize an entire sector as they could twenty years ago. I believe that with the further improvement of infrastructure, the false oracle of fine-grained social division of labor will be broken in the future. Everyone will be able to freely use these tools to do something interesting.
Although heroes may fade, unlike some other tragedies in history, the people within the open source software field have not betrayed their heroes. On the contrary, they have truly changed and transformed the world along the path pioneered by the heroes. Today, Microsoft, once the “enemy” of open source software, has become a practitioner of open source software and the open source movement.
The journey of open source software from the ivory towers of universities to the embrace of capital is a historical inevitability. The ideal of the open source spirit has not been extinguished by this transformation. Instead, it has given birth to a new world where people do not place all their hopes on heroes. May our ideals, like open source software, take root and sprout in solid ground, drawing nourishment from the world’s muck, growing robustly, and becoming towering trees.