> Something important changed between the 90s and today. If you look at most open source projects now, the distribution of who’s doing the work versus who’s simply there is skewed dramatically: it’s common to see projects where 95% of the work is done by a nucleus of people, perhaps even a single developer, with a long tail of “contributors”
Any evidence this has actually changed between the 90s and now?
I have a strong suspicion that it's always been this way.
There are exceptional projects where the work is more distributed (is Linux an example?), but my suspicion is that they were exceptional in the 90s and still are, and that all along most projects have this "dramatic skew" as described.
For software of now, you can look at github. For software of the 90s, 2000s... I wonder if anyone has tried to research this?
I definitely don't see any reason to assume it was different without evidence.
>Any evidence this has actually changed between the 90s and now?
Experience with the relevant communities (Gnome, KDE, and others, etc) in the 90s and 00s.
One reason for the change since then is that nowadays most things are more feature complete so low hanging fruits are not there (higher cost to entry for newcomers and fewer things to do if you're not "up there" as a developer), plus you need to "compete" or "collaborate" with a number of paid developers from companies like RedHat who handle most of the project and steer its direction wherever the company likes.
Plus FOSS just isn't as glamorous as it used to be:
The year of "Linux of the Desktop" (which as a vision meant mass adoption of Linux to win over Windows in desktops and laptops, not the "but Linux is fine for desktop use today, I've used it for years" or "but billions of Android phones use Linux behind the scenes" we settle'd for) never came.
"Big" companies stopped caring about Linux except as an server thing (e.g. we had Easel, Corel, and others competing for desktop Linux, even IBM and others had an interest, whereas VA Linux was one of the biggest IPO stories).
There are much less stories about the sexiness of FOSS or some FOSS future, and hackers changing everything working together, etc, in mainstream (not industry) media, etc.
Linux on the "fleet" desktop almost happened (some companies and schools do it), but then webapps got big and ChromeOS swept in. Now the only reason to make a non-ChromeOS "webtop" fleet is if your organization is explicitly Google-free for competitive/legal/political reasons.
You could call it the facebook effects nominally. The web is/was the key open technology of this era.
Collaborative efforts had a lot of successes. Wikipedia turned out to be a good encyclopedia, which seemed laughable/impossible initially. Linux, obviously. The www itself. Open, collaborative, organized outside of a corporate body. These were big, impressive wins. AOL and others who tried to compete with openness head on failed.
FB succeeded. Or rather, around that time a lot of these wins started being accumulated within a more standard corporate frame. FB controls who sees what and doesn't share. Twitter is a company that controls a protocol. Android is, philosophically and business-wise the anti-thesis to the goals you mentioned.
"Hackers changing everything working together" is now stuff that HR talks about... entirely coopted.
As personal anecdote, I also transitioned from a period I still believed into "Linux of the Desktop" and strongly advocated it as such with M$ email signatures, back into Windows/macOS world, just keeping a survivor netbook still around.
FOSS dynamics and culture proved not to be the place to be, when trying to build business around desktop software or developer tools, GPL + commercial seem to be the only way that kind of works.
It used to be you could email a patch to someone and that patch would get reviewed and committed.
Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file.
Over time, this stuff filters out contributors and leaves a core of people who are comfortable with the structure of the engineering process for that specific project. It's not a bad thing that engineering standards are higher, but it's no longer a world of people 'scratching an itch' -- there's yak shaving to be done, and the number of people willing to shave that yak has always been smaller than the number of people who are not.
I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been.
> It used to be you could email a patch to someone and that patch would get reviewed and committed.
May I ask what communities you were involved in, and when?
Even formatting patches so they could be reliably applied was a nontrivial matter. It was not uncommon for them to be applied manually (and sometimes faultily). Sometimes you would mail patches to maintainers directly, other times you would share them on mailing lists, where they would undergo a bit of "social review" (works for me!) and would or would not be applied.
Some people just mailed entire modified files and expected maintainers to figure out the changes on their own.
As for "committed", many maintainers may not even have used source code control (notably Linus didn't use any), and I can't recall a single project with publicly accessible repositories in the 1990s as opposed to tarball releases.
> All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved [...]
Getting an account on any of these code forges in 2020 is almost certainly much easier than getting an e-mail account at all in the early 1990s for many people.
I don't agree with the author that contributing was easier in the 1990s. Generally, it involved a bit of a social process, trivial drive by fixes were not any more popular than today, there were coding standards to be observed (and not always written ones), in larger projects, you could expect to navigate some flame wars, and sometimes your contributions were rejected anyway because RMS decided to cancel your OS.
Very much mirrors my experience. Github provides a very useful project-independent unified UI for drive-by patches and off-the-cuff issues. No more mailing lists or bug trackers to sign up for and manage!
And it's also even got an integrated version control system, that 99.9% of projects then use, and it's one that makes it easy to version control a fork of your own as you work on any patches you might be making.
> Very much mirrors my experience. Github provides a very useful project-independent unified UI for drive-by patches and off-the-cuff issues. No more mailing lists or bug trackers to sign up for and manage!
And no more need to become emotionally invested in the project, thus furthering its development into a proper community of users and contributors that propel it forward. :(
But what if I don't want to become emotionally invested in the project? It would still be beneficial for the project if it were easy for me to submit bug reports (and maybe bug fixes) anyway.
Emotional investment is always an option for those who want it.
And a lower barrier to entry might even increase the chances of it happening anyway, even for those that started out thinking they didn't care much.
> Github provides a very useful project-independent unified UI for drive-by patches and off-the-cuff issues.
Unless you value your privacy. If you take the necessary steps to approximate the level of "privacy in public"[1] that you get by default when dropping into someone's disconnected bugtracker or attaching a patch via email, then GitHub is pretty cumbersome to use. The article argues the case that GitHub is making life easier for people trying to build their personal brand, but not everybody wants that.
I don't use GitHub for myself and never have, but I used to make exceptions for projects that are hosted there[2]. Not anymore. There's enough toil involved trying to dodge the unwanted social hoopla that GitHub imposes on you, and the returns I've gotten out of "engaging with the community" are so, so low.
Nowadays if I report a bug or try to submit a patch and someone gestures towards GitHub and insists that it's not just the front door to the building but in fact the only way in, then I explain that the only circumstances where I use GitHub involves for-pay work, so unless there's someone I know I can send a bill to, then I'm perfectly content to just never go inside. But people in general seem to have a lot of trouble right now grappling with the notion that submitting a patch or a bug report is not something that I'm doing because it benefits me. (Imagine working as a plumber, showing up for a call wearing coveralls and carrying your toolbag, finding that you've arriving at a club, and then dealing with someone at the door who's trying to argue with you about their entry fee because, hey, this is a club, so they expect you to pay them.) The only thing people seem to understand anymore is the quest to become someone with influencer-level status, and that certainly has something to do with the culture referenced in the blog post. It would be one thing if it were just kids, but it's something that has infected plenty of old hats who were around before GitHub, which is pretty annoying.
They find that a lot of UNIX tools could be easily crashed by generating random inputs. The quality of our most popular open source software has dramatically improved over the last 30 years.
In my experience, bureaucracy is mostly uncorrelated with the willingness of a project to accept a PR from a new contributor.
Homebrew has very strict contribution requirements but has a huge number of contributors and is generally easy to contribute to. Meanwhile there are lots of little projects that go dead for lack of interest by the sole maintainer in reviewing and merging patches.
Also this:
Since then there has sprung up a fetish for bureaucratic process, and tooling has arisen to foster that, such that the barrier to entry is significantly higher now than it was in the 90s. All you needed was a text editor and an email account. Now you need accounts on whatever code forge is involved, and you didn't write enough unit tests, and you failed to properly format your pull request, and what do you mean you're not an expert in whatever version control we've adopted? This pull request sat in an open state because you didn't tag it appropriately. The CI system was updated and your code wasn't covered in the automated integration test, please resubmit after making minor punctuation changes in a metadata file.
Doesn't really make sense to me. How many projects have you actually had this problem with?
I don't think the author is right about things having been harder in the old days led to a higher proportion of the internet being qualified to contribute. I think it's much harder now to patch software upstream than it ever has been.
For some people, using Git and Github is a barrier to entry. For others, submitting a patch to a mailing list is a barrier to entry.
> dead for lack of interest by the sole maintainer in reviewing and merging patches
I think you mean "lack of action" rather than "lack of interest."
It's easy to be enthusiastic about developers doing lots of work for you (usually for free) and to cast them as "uninterested" if they don't comply quickly (or at all) with your requests.
I suspect many maintainers are "interested" in reviewing and merging patches but are simply unable to perform large amounts of work without pay, due to time and financial constraints.
You could re-frame 'bureaucratic process' as 'expectation setting'.
It also benefits the contributor to know what is expected of them. If you know that your contribution will be properly considered when it meets the outlined expectation, then it is easier to put in the effort to contribute.
Totally anecdotal, but I feel open source projects were much more community oriented in the 90s and the 00s. But after that, it seems they’re either driven by a company, or an individual. That would roughly conform to the article’s timeline of GitHub adoption.
Everything has more "individuals" now, since everyone can get a soapbox/megaphone, from YouTube to GitHub to Twitter to all the rest.
And it's possible to monetize that work now, with ads and micro-patrons, which is a huge drain on free and open source, mitigated only (partially) by the increased ability for open sourcerers to collaborate and that nowadays we're drowing in software and don't really need much more new software.
You don't need to join a team to get noticed (and on the ugly side, even though having a team helps, it's better for the Personal Brand to make that team invisible)
Counter-anecdote: most of the small open-source projects I use only have a small number of active contributors because there is a small number of people who actively try to contribute.
Projects that stop accepting PRs tend to be pronounced "dead" after just a few months, but there isn't always someone able/willing to fork it and keep the project going.
Maybe there are more projects to contribute to than before, so each project gets a smaller % of the "potential contributor" pool.
By "community-oriented", do you mean that the distribution of commits was such that more people were making more commits, instead of "95% of the work is done by a nucleus of people"?
I'm not sure if those are the same thing, I'm just talking about the distribution of work.
Another component, I think, is the difference between being able to work on whatever you want vs. having to follow a roadmap dictated by project leadership. Other commenters have touched on this (pointing out the decline of community orientation, rise of corporate oversight, or the fading glamour, for example), but no one has identified it directly.
The article points out that we collectively re-examined Brook's rule, going from "adding more engineers to a project makes it ship later" to "if you need to get more work done, go attract more contributors." I think the evidence shows it's more nuanced.
If you have a large amount of potential tasks and a large pool of potential contributors, and little strong preference for which tasks get done (OSS in the 90's maybe) then highly self-motivated people can pick up work they find interesting or which are useful for them. You can then avoid the exponential explosion of communication/coordination overhead by empowering those individual contributors to make their own decisions about how that work should be done. This is how OSS can scale to hundreds of contributors.
Once you introduce some sort of top-down planning that dictates which tasks are important, the blessed approaches, strict coding patterns etc. then you demotivate a large number of contributors who don't share the top-down priorities and you have to pay the exponential overhead cost of distributed coordination to enforce the top-down priorities.
Users of software care about their own workflows, their pet-features, and the bugs they hit. Some are willing to put up the work to patch in those features or fix those bugs and upstream them. Fewer still are people who are invested enough to dedicate themselves to a project as a part-time job.
I don't think this dynamic has changed since the dawn of open source.
I think the author is missing a key point in that software is vastly more complex than it was in the 90’s. The few times I’ve tried to contribute to open source, I spent more time trying to convince projects that my pull requests fixed the problem without causing other issues. And this is largely because my fixes tend to fix integration problems I had with the software. These weren’t small changes that could be unit tested, and setting up test repro steps were non trivial. I don’t blame maintainers for being slow to accept these kinds of fixes, when the bugs are likely to effect a small minority of users. 30 years ago you could look at a patch and at least reason about it’s efficacy, now you have to have some idea about supporting ecosystems, and not just libraries you’re already using.
I agree. In my experience maintaining, the work validating a PR is substantially more than writing the initial code as well. There is also a significant maintenance cost that falls on the maintainer after the code has shipped.
In other words, even if your contribution is perfect, it's still probably putting more work on the maintainer than you did yourself.
So many of these projects are "mature" which means for many of them there is a risk to accepting random pull requests. In the past a growing project tended to take them without to much worry under the assumption that if a patch broke something it would get fixed. Now there is a fear of the effort required to dig through the mountains of code to "support" someone who is screaming about the code breaking.
I think one of the biggest changes over the past decade or so is that the values and the perception of the values that participants offer has changed.
It used to be that you wanted your open source code to be popular because having users like and use your stuff felt good. There wasn't really much "fame" or career boost to be an open source maintainer. The users themselves and the positive feedback they provided were basically the sole intrinsic motivation.
But now companies look at open source contributions when it comes to hiring. You can get paid speaking gigs or use keynote talks to further your career. Having a lot of users means a maintainer can extract value from other sources. And users know that now. They now that their tiny popularity vote is transferring something valuable to the maintainer who may then use it elsewhere. That leads to entitlement.
In GitHub these days, I often see PRs and issues where the contributor or person filing the bug clearly thinks the maintainer "owes" them something in return for their support. It's like, "Hey, I'm making you be popular, so do what I say." That kills the intrinsic reward maintainers used to feel because now it feels transactional and obligatory.
My experience is that almost everyone filing issues and PRs is well-meaning and helpful. But it only takes a few cranky ones to suck a lot of the fun out. And even the good ones can be draining at a certain scale, though no fault of their own. Managing those social demands takes emotional skills that many maintainers don't have. After all, part of the reason many of us got into software is because we found human interaction difficult or anxiety-inducing.
In the 1990's, the only people using the software tended to be people capable of debugging the problems. Now due to maturity, code complexity, and the simple fact that there are a lot of "users" means that its a lot bigger deal if a random patch breaks something.
Its the result of the success of the project more than anything. Now the maintainers have to start acting like real developers and do the difficult/unsexy parts of maintaining something rather than just hacking at the fun bits.
It really points out that much of what people considered the strengths of the opensource model were illusions that evaporated when the ratio of users/developers got very large. That doesn't mean its not a valid model, only that its a mistake to think a project can both be successful and popular while simultaneously expecting everything to be done by volunteers.
This is one of my primary objections to the cultural shift of the last 10 years towards trying to attract tons of random people, especially special interest groups, into open source software development. If they weren't naturally going to contribute anyway in the previous environment, the expected value of their contribution in the current environment is very low. Possibly negative after considering the drain on the maintainer(s).
Even with tools like github, which are supposed to make collaboration easier, it is still incredibly difficult to contribute meaningfully to an open source project.
I actually had a job where I had the privilege of modifying an open source tool as part of my day-to-day job. Even there, I found it very difficult to give back to the project because the maintainers of the project weren't terribly interest in having or maintaining the enterprise-y, cloud-y features that I was implementing.
It seems like most OSS projects have a single key contributor / BDFL who decides the direction of the project. Outside of financial incentives, it's hard to get groups of people moving in the same direction.
This is pretty discouraging honestly, from the perspective of what the article calls a “fan”. I like to make small contributions to projects I use when I can, but I don’t become a core dev on any of them.
Is it really not worth the maintainers time to merge a bug fix or look at an issue I opened? Are they just doing it as “fan service”?
Genuinely, does this type of “participation” really waste more time than it saves on the whole?
That seems to be the implication of the article. I wonder if any open source maintainers can chime in.
The article doesn’t say large fan bases are useless, just that managing them requires a different skill set (and possibly personality type) than conventional team management or pure dev work.
I think there's a lot of open-source projects that are somewhere in between the two extremes presented in the article (communities vs makers-and-fans). As a maintainer of a project with more moderate user base, it's a mix. I have a lot of interactions with contributors that are very rewarding to me, and also add genuine technical value. On the other hand, there are also a lot of contributions that are more of like "support requests", where I feel like I need to respond to them because it's my responsibility.
I would guess that the bigger a project's user base gets, the greater the proportion of interactions that are more like makers-and-fans.
Do you personally contribute to some random FOSS project? Usually these projects are big and it's just too much work up front to implement a tiny feature which you don't even know if it will be accepted upstream. I'd rather spend that time working on my own pet projects.
I happen to have a project that is fairly successful and all of the substantial contributions are from _big_ companies (think Google, Uber, Airbnb, etc) who have the cash to pay their employees for days or weeks of OSS contributions. Independent private/free-time contributors only make small changes despite the project being quite small.
A very interesting thing is that there is one contributor of my projects who appears to make a huge number of very simple pull-requests to a wide variety (literally hundreds) of projects. Things like fixing typos, dependency updates, and so on but I don't think he's a bot, so I have been wondering why he is doing this.
Unfortunately every person who writes advice about how to break into the industry tells beginners to contribute to OSS as one way to gain experience/a foot in the door. Most of them explicitly list things like fixing typos as easy ways to get a few under your belt.
I can understand the sentiment - it can be quite an anxious experience for a new dev to try to contribute major changes, so theoretically these are low-risk warm-ups. But I don't think it's good advice. It assumes that any contribution is welcomed by the maintainer, no matter how small (many explicitly claim this) and I don't think that's true. Small changes like that that have no impact on functionality, stability, performance, etc. just seem like they would take time and energy from the maintainer with only very marginal upside.
And unfortunately, it also makes the beginner look like someone who's just trying to pad their github "resume".
I'm not saying that's for sure what's happening here for you but it's certainly a possibility.
This is another reason why it's so unfortunate that wikis have died out. GitHub for its part helped kill them and then salted the earth, too—it's hardly possible to even talk about "wikis" anymore, because it's not clear whether a person is actually referring to a wiki wiki, or "some loose leaf Markdown that exists in a repo somewhere (but it's not necessarily clear where), and that's unlikely to allow direct editing without approval". The latter flies in the face of the very etymology of the word.
(To be fair, the same velocity-diminishing forces are at play in the typical practices for using GitHub to collaborate on code, too, but people are even less sensitive to that than they are about wikis.)
I up voted based on the converse application to the headline, wherein a programmer makes his program more fun by adding ah fictional character to do his logging statements like a story.
>If you go back to the 90s, when we first figured out how communities built software, it took a lot of work to use the internet. It wasn’t fast or easy, but that made it special: it meant that everyone on the internet cared about it, a lot. You were really committed to being there. Online communities back then were like a federation of villages; each one had its own culture, its own customs, and its own values. The open source community was like that too, and remained so for a long time.
>In these kinds of environments, attracting more users really did advance open source projects, because the costs were usually worth it. When a new member joined a community, they were probably serious about it. There weren’t many “tourists” back then, so there was a real environment of camaraderie. Existing users were happy to onboard you and teach you things, because their effort would likely pay off as a good investment.
>Since community members joined slowly and stuck around, there was a lot of trust and shared context in the group. Every community had a different way of working, so there was a fair amount of friction preventing users from jumping around or “surfing” from project to project. Groups could preserve and maintain their collective motivation to keep shipping; they weren’t getting paid, so that motivation was everything.
>It sounds pretty idyllic, and for many users back then, it was. As the internet grew more popular, the old timers reliably complained every September as a new crop of college freshmen gained access for the first time, not knowing any of the social conventions. AOL opened the floodgates in 1993, which Usenet bitterly declared “Eternal September”, and the internet veterans have been complaining about it since.
>We know what happened next with online content. The tapestry of forums and newsgroups that made up the early internet flourished for a while, but in the 2000s an invasive species arrived: the platforms. The platforms made it so easy to create, share, distribute and discover that everyone joined them, smushing everything together into common, user-friendly formats without the local context or nuance of the smaller groups.
It seems to be an: Oh, let me tell you about the good ole' days when the internet sucked so badly that it amplified natural selection and all the twenty people on it could hack on the Linux kernel and the output was amazing. Now that any peasant can get on the information highway and participate, everything is ruined.
I can't help but to feel just like the author of this article.
I'm vastly in favor of taxing and hurdles as natural filters. Mainstream spoils everything (~almost). You end up having people not really into things, endless debates about side features, about how to organize the projects (inclusiveness, code of conducts, tooling), scandals and less about the passion of doing the thing.
I'm often amazed at the difference in pacing and communication on Mailing Lists. It's slower but denser.
Remember Slashdot and the "Slashdot Effect"? Slashdot was pretty much a forum for FOSS. It was already mainstream. And this is 1998 or so. The GNOME team caused massive drama when they started their campaign against Qt (and, thus, KDE) on Slashdot.
And on that point, GNOME was a reactionary project. It was not started as a community of people coming together to create good software. It simply wasn't. I was there (mostly observing from the Gtk+/GIMP side at the horror show). Gnumeric was slapped together in like one day. It was pure 100% trash code. All that CORBA, ORBit, etc. stuff was tossed together. GNOME was a movement that feared KDE would beat them to the "Linux on the desktop", and their initial versions were designed to play catch-up with KDE's feature set. I can't overstate how shit early GNOME code was. And then they took to Slashdot to bash the less-than-free Qt because GNOME certainly wasn't winning on merit alone.
The free software community has been a shitshow for as long as such a thing has existed. Go back to the homebrew movement and see how people reacted to Bill Gates selling proprietary software.
It's called nostalgia, and we pretty much all have that condition.
There are some fields or scenes where I recall how it used to be magical, but I know my mind is playing with me.
Many people think that it's too much, too many apps, too many people, too much complexity. Yes. That may be the case and this may not suit everyone, especially those who lived through a pretty magical period with the birth of the world wide Web, or BBS, or open source movement, or the free software movement. Yes, god damn, that has got to make someone nostalgic. However, all that is now is not 'just' ruining it. The Beatles are what they are because of all the people who've never been to their concerts who wish they had been, amplified by people who were there who thought Paul sucked at guitar at the time. People mostly are indifferent to shifts in the now, and only cling to the now when it becomes the good old days or when someone makes a documentary about it and we feel we belong special for belonging to something that is not anymore.
This says much more about us, our desire to belong, to be special, than it says about what reality is or was. We can apply this to forums people thought sucked until they disappear and the memories kick in.
it's too much, not enough, and too many issues around
I like more balanced meals in a way
And it's a bit objective too. IRC, Mailing lists did a lot with few. Now you have to have discord/slack/zulip to .. do the same ? consider the average website bloat.
It's not pure nostalgia (I can realize my biases a bit)
I'm not being dismissive and I'm not reducing the whole piece. I still look for an IRC channel and a mailing list for something by default when I want to join something.
The Python IRC channel and mailing lists are top notch, with lengthy debates and great advice. The excerpt sounds like what an old geezer, in the soul, would say. I recognize it because I sometimes catch myself thinking like that.
The story doesn't have to be about how the old days with the old barriers to entry was the way. What it's definitely about is the flaws and problems in the current way, where platforms collect from continuous posting and participants collect from long tail effects.
Any evidence this has actually changed between the 90s and now?
I have a strong suspicion that it's always been this way.
There are exceptional projects where the work is more distributed (is Linux an example?), but my suspicion is that they were exceptional in the 90s and still are, and that all along most projects have this "dramatic skew" as described.
For software of now, you can look at github. For software of the 90s, 2000s... I wonder if anyone has tried to research this?
I definitely don't see any reason to assume it was different without evidence.