The Open Source Feedback Gap: freedom vs convenience

Sam Tuke Aug 27 2021 Share

Open Source creates value for everyone, and is the most effective way of breaking software monopolies and protecting user freedom. However making software which consistently meets user needs requires an intimate understanding of user problems, and that requires frequent communication and feedback between product managers and software users. This process is typically called ‘discovery‘ and part of what product managers called a ‘customer feedback loop’.

Open Source products face difficult discovery

For Open Source products which are freely and anonymously copied, compiled, packaged, and distributed, this process of finding and talking to users can be difficult, especially if your users are security professionals or otherwise work in a data-sensitive space. This includes our users: people who use Lightmeter to monitoring and automate their mailops.

One of the key attractions of Open Source is that it’s under the user’s control: the four freedoms of Open Source guarantee this. But what happens when these freedoms mean that your needs are not communicated to the developers of the apps you rely on? Sooner or later those apps will frustrate you because they aren’t meeting your expectations closely enough. When you’ve been frustrated and disappointed enough times, you’ll seek an alternative.

Users must choose freedom or convenience

When choosing a replacement you’ll likely face a dilemma:

• Total freedom in a poorly fitting product (Open Source), or
• No freedom but convenient features (proprietary)

Even for closed source SaaS companies which combine hundreds of datapoints per second with demographic profiling, understanding and meeting user needs is very difficult (being proprietary is no guarantee of user convenience!). That’s why there are so many new product management and analytics solutions being launched and doing well (e.g. Segment; recently sold for $3.2 billion).

A competitive disadvantage?

For privacy-respecting Open Source startups who collect little or no data about their users, meeting those needs is risky and expensive. Limited user feedback means fewer features end up delivering value, which means that it costs more time and money to make a successful Open Source product than a proprietary one. This puts Open Source at a disadvantage, which benefits only the proprietary competition. That sucks!

Open Source applications do have unique feedback channels available as well: GitHub/Lab issues and merge requests, forks, binary downloads (if these are centralised and not packaged by third parties), and more. The detail and value of the feedback from these sources is less than what proprietary products who can identify and proactive reach out to users have available however: telephone interviews, usability research sessions, focus groups, product analytics, automatic market segmentation, etc.

Exceptions and outliers

But what about those big-name Open Source apps which we rely on every day? Companies like Mozilla and Automattic (WordPress) benefit from enormous userbases and large resources to reach them. The more data you have, the lower quality (e.g. less specific or identifying) it can afford to be (Firefox telemetry includes thousands of anonymised items).

With millions of users large Open Source projects have the benefit of sufficient feedback and engagement even if only a fraction of a percentage of their userbase are engaged. For younger projects with fewer than hundreds of thousands of users, low response rates can mean no usable responses at all. Web-based OS apps like WordPress often have the benefit of SaaS offerings through which customer data can be collected as intensively as in proprietary SaaS apps, giving them the data they need to make product management decisions for their larger anonymous userbase as well.

Identifying data as a third way alternative

We are committed to Lightmeter being both 100% Open Source and also best-in-class for our users needs. And so we need to square the circle to find a third way – to honour and protect user freedoms while also getting sufficent feedback on which features are providing value and which are not.

Lightmeter’s next step

For this reason the next release of Lightmeter will start to optionally share the email address of the administrator accout registered to self-hosted Lightmeter installations. Those users who have opted-in to telemetry will automatically send this information to our self-hosted, EU-based analytics software — Matomo and Posthog.

How will the data be used?

We will use the additional information to identify the different categories of Lightmeter users (for example small advertising agency vs large transactional mail hosts), in order to prioritise features to suit the needs of the groups which are already using Lightmeter the most. We may also use them to email users to ask for preferences on which features to prioritise in the Lightmeter roadmap, and check if existing features are useful. Lightmeter users can change their email address at any time, request their data be deleted, or update their email address (as explained in the Privacy Policy).

The only person affected is the user of the Lightmeter administrator account. Therefore there’s no need for Lightmeter users to update their own Privacy Policies or notify their users.

Open Source wins

We’re taking this step in order to meet the needs of people who prefer to use Open Source software (?), and make self-hosting email as easy as it can be. You can look forward to more exciting features, chosen by users like you, coming soon, thanks to better communication between you and our team.