Book Club: "Working in Public - Nadia Eghbal"

During our Governing the Commons book club club we spoke about turning next to Nadia Asparouhova’s (née Eghbal) book on open source:

https://www.amazon.com/dp/0578675862/

This book was actually covered in a previous discussion at Fission which was not recorded:

I have started the book and plan on posting notes for each chapter as I read it, much like I did for the Ostrom book. If you would like to read along, please do! It is an async book club. I will make each of my posts a separate entry so you can comment or add your observations in a nested fashion. I don’t have a schedule, but the book is pretty short so I don’t imagine it taking very long. If we want to have a Zoom Book Club chat when we are done, then maybe we will do that! Like all of our book clubs, it is peer-produced :stuck_out_tongue_winking_eye:

Please invite others to the conversation as you see fit! I am sure there are a lot of people in our community that have read the book and have thoughts to share.

1 Like

Introduction

In this chapter she outlines the broad problems that likely will be addressed in the book: a changing online culture and the rise of Web 2.0 platforms (like Github), plus the popularization of smaller libraries, means that open source communities have changed from “peer production” to sole maintainers who reach much larger audiences that are more transactional. Here are some quotes and notes from the chapter:

  • “maintaining code for general public use quickly becomes an unpaid job you can’t quit.”
  • championed as “good for open source,” and they are frequently accomplished by tapping into a public sense of goodwill
  • However … such initiatives often attract low-quality contributions.
  • Maintainers simply don’t have the energy to onboard every person who shows passing interest
  • STUDY: 275 popular GitHub projects, nearly half of all contributors only contributed once, accounting for less than 2 % of total commits
  • STUDY: in 85% of OS projects examined on GitHub, less than 5% of devs were responsible for over 95% of code and social interactions
  • where one or a few developers do most of the work , followed by a long tail of casual contributors , and many more passive users — is now the norm
  • the bus factor: project health measured by the number of developers that would need to get hit by a bus before the project is in trouble
  • core change in OS culture: “In contrast to big, monolithic software projects that give rise to persistent communities [projects] are designed to be small and modular, which leads to fewer maintainers per project and a transitory relationship with the code they write. … the relationship between a maintainer, contributors, and users is lighter, more transactional.”
  • Casual contributors don’t immerse themselves, don’t think of themselves as part of a community. Don’t stick around… drive-by contributions
  • “developers aren’t building communities; they’re directing air traffic”
  • “it’s not the excessive consumption of code but the excessive participation from users vying for a maintainer’s attention that has made the work untenable for maintainers today” ie. attention/hours is the scarce resource
  • early internet communities were large, distributed, corners of the web with their own norms. Social media changed that landscape. Big scale! Creators now reach a much bigger potential audience, but relationships are fleeting and one-sided
  • In the late 1990s , open source was the poster child for a hopeful vision of widespread public collaboration , then dubbed “peer production” but now “open source inexplicably skewed from a collaborative to a solo endeavor”
  • individual maintainers == personalities
  • decline of “the firm” as a principal agent of change
  • “In a world where 4.5 billion people are now online , what is the role of one?”

Ch1: GitHub as a Platform

This chapter tells two stories: a potted history of Free and Open Source Software movement, and the rise of GitHub and how it has changed the culture.

  • covers RMS, ESR, and Linus
    • she spends a lot of time slagging these weirdos, and then later says “Compared to early renowned hackers like Torvalds, Raymond, and Stallman, many of today’s JavaScript developers are unusually humble” which I thought was :face_with_raised_eyebrow:
  • “Hackers are characterized by bravado, showmanship , mischievousness, and a deep mistrust of authority . Hacker culture still lives on today, in the way that beatniks, hippies, and Marxists still exist, but hackers don’t capture the software cultural zeitgeist in the same way that they used to.” LOL. I just thought this quote was hilarious. Can you tell she is from Miami?
  • “the GitHub generation of open source developers … prioritize convenience over freedom (unlike free software advocates) or openness (unlike early open source advocates)”
  • “GitHub’s popularity among modern developers is the classic technology tale of convenience triumphing over personal values”
  • in the old days, each project had its own version control system and contribution practices, but GitHub standardized Open Source workflows
  • one of the biggest advantages of platforms: free, unfettered distribution
  • today’s generation of open source developers needs GitHub to do their best work. Just as there are Instagram influencers and Twitch streamers , there are GitHub developers.
  • "the more successful you are, the more you get punished with GitHub notifications.”
  • Quote from a maintainer: “I’ve also noticed that the general PR quality has gone considerably down in the past few years… It’s not the fault of new developers . They don’t know how open source works , and they’ve been told that they’re doing the right thing by asking questions… Just keep in mind that my time is finite and if I have to go back and forth on your PR for stuff you could have caught yourself with a second look, you take time away from other PRs.” Maintainer attention as scarce resource
  • on packages vs giant monolithic projects: “not unlike the shift from blog posts to tweets”

Ch2: The Structure of an Open Source project

This short chapter covers how open source projects work, day to day, basically as an overview for non-devs. Things like how contributions are made, different patterns for how changes are accepted and merged, and all that functional stuff. At the end she lays out four categories for open source projects.

  • mention of “charismatic megafauna
  • there is no single open source community. Lots of tribalism.

    there is no “open source community”… anymore than there is an “urban community.”

  • “(if there’s one thing I’ve learned, it’s that developers have opinions)” LOL
  • social norms for contributions can be intertwined by project architecture: “Debian has a monolithic, tightly coupled codebase, where green-lighting the wrong maintainer could, indeed, have dire consequences.” vs “Node.js, is designed to be modular, where each maintainer has a limited ability to affect other components of the ecosystem, so JavaScript developers are more likely to prioritize moving fast and accepting contributions.”
  • social norms for contributions can vary. Process of getting a change approved depends on things like code complexity and reputation
  • the influence of Github again: “One infrastructure developer, who worked for years on projects with different tooling, told me that he’s now so used to GitHub that if he finds a bug on a project that uses a different issue tracker, he won’t even bother filing an issue anymore—it’s too much work.”
  • platforms play a big role, so creators are tethered to them (eg github). Funny Drupal quote:

One person I spoke to compared the Drupal project to the Basques as a means of describing Drupal’s culture and history. Like the Basque Autonomous Community, Drupal is sequestered in its own territory; not being on GitHub led its developers to develop their own norms, which are influenced by, but distinct from, mainstream open source practices

  • Ease of participation is important, and a reason projects find themselves on GH. That being said, she covers a few different varieties of how projects will set up their communications and support channels, sometimes doing everything on GH, sometimes using other apps. See the diagram as an example
  • one way to categorize projects is in terms of relationship between contributors and users, leading to four production models:
    1. Federations:
      • projects with high contributor growth and high user growth
      • comprise a small percentage of open source projects (less than 3% , according to one study)
      • Rust, Node.js, and Linux are all examples of federations
      • more complex to manage from a governance standpoint , so they tend to develop processes — voting , leadership positions , foundations , working groups , and technical councils
      • typically “ shard ” contributors into smaller working groups , where maintainers specialize in certain areas of a project , such as infrastructure or community .
    2. Clubs:
      • projects with high contributor growth and low user growth, leading to a roughly overlapping group of contributors and users
      • Astropy, Clojure, Haskell, and Erlang
      • similar to meetup or hobby groups : they attract a narrow group of users , who then also become contributors because they have higher context for the project’s activity and feel a sense of affinity to the group
      • sticky, retaining a large portion of user-contributors, even if they don’t attract as many new ones
    3. Toys:
      • projects with low contributor growth and low user growth
      • side project or a weekend project
      • (not really covered in this book)
    4. Stadiums:
      • projects with low contributor growth and high user growth
      • webpack, Babel, Bundler, and RSpec
      • Stadiums are becoming increasingly commonplace today
      • Unlike a federation or club , whose communities are decentralized , a stadium’s community assumes a centralized structure , centered around its maintainers
        • one-to-many rather than many-to-many interaction model
  • of course projects can change categories
  • Decentralized communities “Avoid creating big decision hierarchies . Instead, invest in a broad , growing and empowered contributorship that can make progress without intervention.” while “centralized communities operate on the basis of limited attention . As proprietors of their communities, creators must manage user demand on their own” and thus “rely more heavily on automation, distributed user-to-user support, and the aggressive elimination of noise.”

Ch3: Roles, Incentives, And Relationships

This chapter analyzes the motivations of various people involved in projects, from the incentives of authors and maintainers, to defining of contributors and users, all using the framework of stadiums and clubs from the last chapter.

  • network goods and economics
    • Previously Theory of the Firm - “Only companies make software”
    • Open source breaks this assumption: intrinsic motivation (learning, fun, satisfaction) vs extrinsic (money)
    • New explanation “commons”
  • Ostrom’s 8 Principles
  • “Like the fisheries in Ostrom’s case studies, online communities can historically be understood as a group of self-organized but disconnected villages.”
  • Yochai Benkler extended Ostrom’s work to software “commons-based peer production”
    • conditions that Benkler identifies as necessary: intrinsic motivation, modular and granular tasks (small enough to be tackled by one person without much coordination), and low coordination costs
  • Coordination work (reviewing, merging, support, etc) is expensive because it’s not intrinsically motivated.
  • not every open source project looks like a commons: Stadiums have one or a few maintainers and need to organize differently:
    • “prioritize work based on abundance of attention: encouraging new contributors , developing governance processes, and improving engagement and retention. But a creator prioritizes work based on scarcity of attention: saying no to contributions, closing out issues, reducing user support .”
  • “While the commons is tasked with resolving coordination issues , creators are defined by the need for curation”
  • how GitHub broke everything
    • platformization: projects all look the same, easy to switch between projects
    • “context collapse”
    • breeds more casual/drive-by contributors
    • “collective identity of communities was reduced in favor of personal identity”
  • now difficult to follow Ostrom’s 8 rules. Eg: Sanctions
    • devs will transact more casually with one another… not really part of the community… when sanctioned it is not a big deal, or they get off on it == TROLLS
  • a community needs to “protect itself from the potentially damaging actions” of newcomers in order to survive, since newcomers can destabilize preexisting social norms {eternal september}
  • “When the issue is about the maintainers themselves , it’s particularly difficult to determine who should have governing authority. Do maintainers get the final say , or should communities be able to influence the outcome?”
    • many of the bigger open source projects prefer to use consensus - seeking over consensus processes… eg IETF uses “rough consensus”: “we don’t let a single individual dictate decisions (a king or president), nor should decisions be made by a vote”
    • “Rough consensus is achieved when all issues are addressed , but not necessarily accommodated.”
  • stadiums are better suited to “ benevolent dictator ” governance models
  • Contributors: hard to define… actually a question of identity
    • “Like any other club or group , what counts as a ‘leader’ or a ‘member’ in one community might be completely different in another”
    • eg casuals: “contribute a one - time fix and be on their way , the equivalent of picking up a stray piece of trash on the street”
  • Maintainer: also hard to define. Identity more than a function
    • “trustees” or stewards of the code
    • CONTRIBUTORS are those who make contributions to a project’s repository
    • USERS are those whose primary relationship to a project’s repository is to consume
    • AUTHOR refers specifically to the developer or developers who were responsible for the project’s original release
  • intrinsic motivation over time:
    • “Rarely is long - term maintenance primarily about sharpening one’s skills as a developer”
    • lots of orphaned projects
    • people just giving access to projects to strangers
  • Study: 85% of the open source projects studied have heroes who participate in at least 80 % of discussions related to a commit
  • behaviour of active contributors vs casual contributors: eg lurking on mailing lists, issue threads, etc as opposed to opening an issue or PR right off the bat (“I need something”)
  • casual contributors have a low affinity to the project. Possibly better identified as “users”
  • passive vs active users
    • who are the users?
    • usage tracking not popular
  • active users might be “fans” and not necessarily identify as contributors, even if they are making videos, answering questions in forums, filing bug reports, etc
  • Project health metrics: popularity, depended upon (by other software), active
    • Contributor count might count too… but can be misleading like in Stadiums since some contributors are MORESO than others. Mostly useful when thinking of the “bus factor”
  • numbers of open issues and PRs; avg time to first response; avg time to close; cam help determine project health

Maybe developers can’t help but make things, and share the things they make, but every time they do, and every time they find success, a tiny, invisible clock begins to tick, and they’re tasked with managing the care and feeding of their code into perpetuity.

Ch4: How people maintain

This chapter looks at the cost of creating and maintaining open source software, and the value of it… including the problems around charging for software.

  • pain of supporting browsers (IE6)
  • “Henderson also points out that regularly rewriting software is inherently beneficial. It helps cut away unnecessary complexity that has accumulated over time, as well as transfer knowledge and a sense of ownership to newer team members.”
    • “Code is not a product to be bought and sold so much as a living form of knowledge.”

On Software costs

  • Generally 3 categories: creation, distribution, maintenance
    • “Creation is frequently powered by intrinsic motivation. These are the fixed “first-copy” costs. Distribution is mostly powered by the platforms used by creators. These costs are low or free due to supply-side economies of scale. But maintenance is still a mystery.”
  • but first some theory about economic goods
  • That said, software is not exactly non-rivalrous: marginal and temporal costs
  • For example: there is a different between 10 users and 10,000 users
  • some costs include:
    1. Infrastructure: eg PyPI infra costs $2-3M a year (donated by Fastly)
    2. User support: “If you have a billion users, and a mere 0.1% of them have an issue that requires support on a given day (an average of one support issue per person every three years), and each issue takes 10 min- utes on average for a human to personally resolve, then you’d spend 19 person-years handling support issues every day. If each support person works an eight-hour shift each day then you’d need 20,833 support people on permanent staff just to kеер up. That, folks, is internet scale.”
    3. Community moderation: Trust and Safety for the community
    4. Maintenance over time: “A 2018 Stripe study of software developers suggested that developers spend 42% of their time maintaining code.”
    5. Technical Debt: “Since open source developers tend toward work that they find intrinsically motivating, not only are open source projects more susceptible to technical debt but there are also fewer incentives to clean up messy code.” She asserts that open source project are also more susceptible to bad test coverage, crappy CI, and lax security.
    6. Dependency management
  • just being open source does not solve this: ESR “Given enough eyeballs, all bugs are shallow” to Fred Brooks’ “more users find more bugs”. Mo people, mo problems, mo money
  • Maintenance costs make the difference between “forking as a credible threat” and “forking as a desirable outcome.”
    • network effects (eg social media)
    • as in “credible exit” sometimes not so credible
  • difficulty in sunsetting projects for more modern solutions. No central body to make the decision and enforce it.

On software value

  • nobody wants to pay for code… have to use artificial rivalry, DRM, etc
  • purchase value is LOW, but social value is HIGH
  • “organized complexity”: software is BOTH an artifact and an organism. Software is BOTH worthless and indispensible
  • “software is comparable to public infrastructure, and similar valuation methodologies apply. Like code, infrastructure derives its value from its active dependencies, irrespective of the cost of its construction or maintenance.” Example is the value of two bridges, one built for $10m and one for $100mm… you don’t assume the second one is more valuable since it cost more to make, right?
  • also, some are easier to maintain or replace than others: substitutability (gives example of left-pad which broke the internet, but was fixed within 1.5 hours)
  • frontend frameworks have higher substitutability than databases
  • OS code is highly elastic. Users are sensitive to price and switch easily (A subscription fee!? I will just make my own!)
  • people unlikely to pay for a library when there are a hundred similar free ones
  • here is her approach to assess value of widely depended-upon code:
    • If this code didn’t exist, how much would it cost to accomplish the same thing in a different way?
    • How much money and time does this code save us?
    • What does this code allow us to do that we couldn’t have done otherwise?
  • accounts for non-marginal effects, or the opportunity costs for software
  • value of code also depends on WHO is buying, but also WHO is making == reputation
  • some devs are sponsored because of who they are, not on a per project basis
    • also the example of Evan You who has personal Patreon for working on Vue.js, separate from the vue - Open Collective
  • developer influencers and comparison with creators who make viral content for social media. They are kinda like maintainers: once they get attention and a rep, they have to keep producing more to maintain their rep. Rep, like software, requires maintenance over time.

Ch5: Managing the costs of production

This chapter focuses on what the valuable resource in open source actually is (attention) and covers a few different funding models for projects, and includes some insights on what exactly to fund. This is a big chapter, and gets to the meat of the issue on everyone’s mind, so apologies for the long notes. I cut them down, believe me!

  • pretty much right away she comes out to say that we should get away from arguments that people “should” or “ought” to back open source projects out of some obligation they might feel. Basically arguing against this sentiment
  • open source code is a “positive externality” rather than a “common pool resource”
  • consumption should be open and free… but we need to protect “production” (aka maintenance)
  • different between “public” and “participatory”… it is the participatory nature of open source which fatigues devs

Ostrom’s principles for managing the commons only apply when there are clear group boundaries, which help reduce the risk of over-appropriation by setting and enforcing meaningful social norms. But if anybody can appropriate a maintainer’s attention by making demands, the commons will eventually be depleted.

  • managing producer attention
    • Reduce up-front costs by using automation and bots
    • Make themselves less available by prepping documentation, tests, checklists so contributors can self-serve
      • “While some maintainers move to GitHub in order to boost their projects’ appeal and attract new contributors, others seem to deliberately enjoy not being on GitHub, because it makes their projects harder to get to.”
    • Distribute costs onto users, ie recruit mods to shield the creators. Study: in 2018 89% of Github products were using off-Github platforms to manage support
    • Increase total attention available: just pay the maintainer to focus on the code, rather than spending a bunch of time looking for more contributors

Getting paid

  • “attention is the currency of production”
  • Maintainers can make their attention excludable by charging for access, which turns it into a private good.
  • choosing the right source of funding depends on the value prop

For Companies

  • code quality
    • models: pay-per-incident, SLAs, support services
      • https://www.tidelift.com/ partners with open source projects to offer commercial support to enterprise customers for their dependencies
    • dual license: current version is on proprietary license, but after 18 months flips to MIT
  • influencing the roadmap
    • model: pay for access, which could be priority for PR review, which feels bad, but can be something else like access to the core devs, office hours, walking an eng team through new features etc
    • ultimate buying attention of dev: hire them. Could be paid consulting or aquihire to work on the code
  • brand association
    • model: readme sponsorships

For developers

  • companies are more interested in the code than the dev
  • devs sponsoring other devs is more about supporting the people behind the code
  • main reward for content creators are reputation. Need to figure out how to convert reputation into attention, and attention into money: The Patronage Model aka “reputation-based funding”
  • “When an individual sponsors an open source developer, then, they’re ideally not paying for code but to feel closer to the person who writes the code.”
  • subscriptions, Patreon, etc
  • not a donation. Quote from Ben Thompson:

First, it’s not a donation: it is asking a customer to pay money for a product. What, then, is the product? It is not, in fact, any one article (a point that is missed by the misguided focus on micro-transactions). Rather, a subscriber is paying for the regular delivery of well-defined value. Each of those words is meaningful:

  • Paying: A subscription is an ongoing commitment to the production of content, not a one-off payment for one piece of content that catches the eve.
  • Regular Delivery: A subscriber does not need to depend on the random discovery of content; said content can be delivered to the subscriber directly, whether that be email, a bookmark, or an app.
  • Well-defined Value: A subscriber Reeds to know what they are paying for, and it needs to be worth it. 319
  • she quotes a bunch of Twitch streamer income facts
  • points out that “following” on Github doesn’t mean anything
  • "One hypothesis is that “open source” is quickly becoming indistinguishable from “doing code stuff in public.”
  • Github is poised to become the Twitch of coding: live-coding, advertising, etc
  • there is a generational transition from “open source developers” to “Git Hub developers.” → “Some of today’s best-known developers aren’t even necessarily active open source contributors”

Project-based fund-raising

  • moving away from the personalities, a few suggestions for project-based funding
  • “There is a lot of motivation, both intrinsic and extrinsic, that already powers open source work; we shouldn’t disturb the parts that are currently working.”
  • funding projects builds institutional memory, is easier to be transparent, and avoids governance issues compared to individual funding
  • she talks about Open Collective
  • Project-based is more attractive to corporate funders
  • makes sense to fund the maintainer’s time rather than anyone elses
  • Bounties (especially for specialized and well-scoped tasks like design or database migration)
  • Crowd-funding
  • Fellowship Programs (like the Django Fellowship Program) where you raise funds to hire a specific person to do a specific task (we have talked about this approach for certain projects at Fission)
  • one issue with project funding over individual funding, is can sometimes be difficult to fund people making tutorials, documentation, writing books, answering support questions, doing live-coding, etc since they are not as close to “the project” as developers.

This chapter is packed full of ideas and observations. I am particularly interested into digging more into that Ben Thompson quote:

In the case of content like Substack, this is things like weekly/monthly essays, for Craig Mod this is artfully underexposed photos of rural Japan, for the Football Ramble this is behind the scenes content of the boys as they make their show. For developers it is more than just X number of PRs, Y number of features, and Z thousands of LOC.

The last step in Open Collective’s article Ten Steps to Successful Open Source Crowdfunding is Keep Engaging. This is a lot of content marketing activity: regular updates like blogging and newsletters, developer calls, office hours, social media, etc etc. I think many struggle with this, and the people that are good at it will thrive in an “attention economy” (more on that in my next Reply).

Let me know your thoughts.

Also, as a bonus, listen to this clip of @steveruizok talk about his low-overhead technique for engagement from Metamuse Podcast ep 59:

I get where she is going, but boy do I think this is a bad idea. You are just going to create a world of super-haves and have-nots. Think of YouTube as an example:

  • Social Blade says you need about 20,000 views a day, with a 50% engagement rate, to make 11K-18K a month. That takes a lot of (unhappy) work.
  • According to research in 2020, just 0.4% of YouTube channels take the bulk of views, subs, and monetization

I think you will just end up with a massive longtail where the rich get richer. And once that forms, you will have a whole other problem with getting legit projects funded because they lose out due to the popularity contest. You are trading a broken “attention commons” for a different broken “attention economy.”

Look at the list of Fission’s dependencies:

How many Mr Beast’s are in there? They all can’t be, but we would sure like all of them to be properly supported.

Conclusion

Final, small wrap-up chapter. Covers a few of the main points of the previous chapters, then goes on to compare the future of “content creator developers” to other content industries, before sort of ending suddenly.

  • Key argument: “Open source code is consumed like a public good, because when code is viewed as a static commodity the cost of additional users is nearly zero. But open source code is produced like a commons, where a maintainer’s attention is the limited resource.”
    • and further: “Creators face a problem of over-participation, not under-participation, in managing expectations from their audience.” and “A tragedy of the commons occurs not from consumers over-appropriating the content itself, but from consumers over-appropriating a creator’s attention.”"
  • “Social media is in a pre-Newtonian moment” (CK: doubling down on social media… the book was published in 2018)
    • after Facebook’s invention of the News Feed, and other platforms’ launches of their own social feeds, “unleashed a gold rush for social capital accumulation.”
  • “the focus has shifted from what developers make to who they are.”
  • “Platforms have helped bring about this shift more quickly.” Eugene Wei: “Status as a Service Staas” but they must do more to give devs the tools to manage over-participation: more “one-way mirror” style broadcast tools where everyone can consume, but only some can participate. Book argues this is an adaptive response to over-participation (I would argue there are other social factors at play)
    • “The experience of an Instagram user with 2 million followers is different from that of an Instagram user with 200 followers. The former is the epicenter of a centralized community, whereas the latter is either passively consuming 1:N content or participating in a small group.”
  • at least the book recognizes:

Reputation has a half-life on any social platform; successful creators accumulate reputation, which serves as a “battery” that helps them store consumer attention. But if they don’t keep producing new work, that battery will degrade, and eventually get depleted.

  • finally the book turns to the news industry as a potential model to follow as media companies become atomized and readers pay for “creator” journalists.
  • “other content creator. Readers will pay for a curated, high-quality take on their interests, as well as, sometimes, a sense of community with other like-minded readers.”
  • “For example, former journalist Azeem Azhar brings in six figures of annual recurring revenue with his technology newsletter, Exponential View.”

You don’t think that person with 200 followers is not trying to get to 2M???

The book argues for a change in the collaborative nature of open source culture, requiring the introduction of hierarchy, which I am not happy about. I think there are better ways to man

Also, before it was “everyone is an entrepreneur” and now it is “everyone is an influencer“ :expressionless::expressionless::expressionless: Late-stage capitalism keeps metamorphosing!

Yeah… this is not the path we should go down. Look how that worked out! We should learn from this.

Okay, I am not saying that the Patronage Model for open source will increase political fragmentation and endanger our democracy… but based on what has happened in news media…

This is propagating drug dealer economics. See my comments about YouTube above. Don’t be fooled by the hype.