When should companies optimize for engagement?

A recent Wall St. Journal article provides more evidence that removing ambiguous engagement signals like comments and shares from civic algorithms can improve outcomes for both users and society.

One of the things I think we did right in my time at Facebook was changing the way we incentivized civic content.  While I disagreed with many decisions in my time there, I continued to work there because of efforts like those described in this recent Wall Street Journal article, where the company was willing to reduce engagement incentives (per the article, users “logged onto the platform .18% less”) and revenue in service of a better user experience and doing the right thing for society. Per the article, as a result of removing comment and share predictions from ranking, “users told Facebook that their feeds were more “worth your time”, “anger reactions fell by more than half”, and “bullying, inaccurate information, and graphic violence fell”.  This was not entirely selfless given that ~55% of Americans are “worn out” by political posts such that many stop using social media and many users would welcome the fact that “views of civic content in newsfeed fell by nearly a third”.  But it was at least one case where the company did not focus on short term engagement, measurably improved the platform’s impact on society, and learned principles that can make all technology better.

The reporting of these efforts opens the door to a wider conversation about the principle behind this, as it doesn’t just apply to Facebook, but also to TikTok, Twitter, YouTube, and every other platform that generally optimizes for engagement, but also hosts content where engagement is not indicative of value.  Some topics (e.g. political content) are not suited for optimization for engagement, as it leads to negative effects for both user experience and society.  Like most product changes, the work was a result of numerous experiments that consistently showed that removing these engagement incentives for some classes of content had benefits for both user experience and reducing the reach of divisive misinformation.  Some of these were already public, where you can see that reducing incentives for downstream shares reduced misinformation metrics for civic and health content, inspiring more negative comments leads to more publisher traffic or replacing engagement terms for health content reduced health misinformation.  The metrics in this article only add to this body of evidence, which also has been corroborated externally, and hopefully more and more of the evidence that exists will continue to get released.  While people may debate whether social media polarizes users, there should be no debate whether engagement based algorithms generally increase the distribution of polarizing civic misinformation - as robust evidence exists there.

DALL-E AI generated image of “people arguing on computer screens as digital art”

The mechanism is relatively intuitive.  Some topics are meant to be boring.  The best information about voting is unlikely to be the information that gets the most engagement, and turning voting information into a popularity contest creates perverse incentives for people to be controversial and appeal to identity, even when they may not believe the controversial things they say.  Until recently, we did not debate vaccine information nearly so much, and those engagement fueled debates have helped create influencers whose financial incentives are tied to being ever more engaging by appealing to their reader’s identities.  Many publishers and politicians have written about the effect of these perverse incentives on what they choose to write - an effect that is not generally captured in user side experiments, suggesting that the full effects of removing engagement incentives on society are likely to be even larger than what is reported in the article, when these ecosystem effects are taken into account.

As discussed in the article, removing engagement incentives works better than bluntly reducing civic content. It does not rely on subjective judgments of good or bad content and works in any language. It specifically targets the incentive to make content that baits people into engaging. For example, in his book “Them”, Senator Ben Sasse noted the widespread use of polarizing tactics like “nutpicking”, where content creators would find some terrible thing that someone on the other side did and publicize it to their audience to gain cheap outrage-based engagement.  Since someone is always doing something outrageous, it is a never ending source of material for those who want to build their influence by hyper-posting “news”. Not all civic content is bad, and generally the boring stuff tends to be better and does not benefit greatly from engagement incentives, so removing engagement incentives ends up affecting such content less.  Indeed, removing engagement incentives could eventually make room for more boring yet informative content, which currently gets crowded out by better performing, more engaging posts.

How can society learn from and build upon this work?  Even as this was a measurable step in the right direction, there is a lot more to be done.  In particular, we should:

• Audit algorithms for perverse engagement incentives - If we agree that some engagement incentives for civic content are bad for society, we should increase the specificity of our algorithmic transparency efforts to allow us to specifically audit algorithms for cases where important topics are being optimized for engagement. Note that comments and shares are but one type of engagement and it is possible that other kinds of engagement (e.g. time spent) should be similarly scrutinized. In a world of increasingly powerful AI, transparency about the goals encoded into systems becomes even more important and both governments and app stores have an interest in providing consumers with the information necessary to ensure that systems are aligned with users’ explicit goals.

• Align on a societal definition of sensitive content -  This work relies on a common definition of what content we feel should or should not be optimized for engagement.  This may differ radically depending on societal context and as issues like crime news or business news become more polarized, they may need to be similarly treated.  We need to figure out systems to define what topics should or should not be engagement-optimization based, that are robust to creative tactics that publishers who need attention are likely to use and that work across countries and languages.  While it might be tempting to outlaw all engagement based incentives, the line is harder to draw than you think.  Would it be reasonable to prioritize photos of our friend’s wedding that have gotten more comments? Would we outlaw showing Netflix series that are more popular on the home screen?  How does that differ from YouTube or TikTok videos?  Would we care if we ranked dance trends by popularity?  Can spreading popular dance videos have health benefits? Popularity has a place - just not for discussions of topics that create disproportionate harm when sensationalized. Ideally, these lines are drawn by the world, not by private companies.

• Decide on alternative incentives for important topics - How should we incentivize civic content in a world where companies cannot scale content judgments?  There are kinds of engagement that likely are more aligned with user value (e.g. explicit positive reactions from diverse audiences) and we should study those as potential alternatives.  Surveys of content quality could be part of the answer as well. Metrics that count the number of users (vs. the number of actions) can often lead to better incentives as they reduce the ability of a small number of hyper-engagers to dominate narratives.  These tactics are not mutually exclusive and it is likely that the ultimate answer will involve some combination of the above ideas as well as ideas that are yet to be developed. Removing engagement incentives is an important first step, but a larger societal conversation needs to occur as to how such information should be incentivized and spread. We may eventually want different venues for consuming engaging entertainment vs. discussing serious topics.

I sometimes like to tell people that me and the many people I worked with on such initiatives made things 3% better during my time at Facebook.  3% isn’t a real number and is mainly meant to illustrate that efforts like those described in this article made a meaningful measurable difference for a large number of people, but we clearly didn’t solve all or even most of the problems.  Numerous experiments were done to inform the measures taken and I am hopeful we can apply the lessons learned more broadly across more companies to help expand on the impact that has already been made.   Removing bad engagement incentives for civic content is one big lever that we can use across platforms to robustly address many of the negative effects of technology on society.

—

In addition to the above article, we wanted to take a moment to highlight a couple upcoming events and articles that readers may be interested in.

• The 2023 Positive Technology International Conference is being held in Hong Kong in June of this year and the organizers recently put out this call for proposals for interested academics.

• A group of allied organizations have scheduled the inaugural Tech+Social Cohesion conference for February 23-24 this year in San Francisco. I’ll be doing a brief presentation there as will many other great speakers. Registration is open to the public.

• One of my ex-colleagues at Facebook, Glenn Ellingson, recently did this interview for “Everything in Moderation”, where he talks about similar themes to what I’ve written about here concerning the limits of content moderation and the need to prioritize user value created over engagement. Definitely recommended reading if you like the themes I talk about here.

• Facebook Analytics recently posted this analysis showing how short-term engagement can diverge from long term engagement, specifically with regards to notifications. The findings are intuitive and open the door to thinking about other cases where short term and long term metrics may diverge. Ideally, all tech companies would consistently take a longer term focus.