Unbiasable

Part 4 · The biased machine

Chapter 16

Engagement ranking and the attention economy

A feed ranked for engagement rewards whatever holds you longest, not what informs you best. Tristan Harris told the Senate in 2019 that this system already shaped where 2 billion people put their attention every day.

What is the attention economy?

The attention economy is the market in which platforms compete for one scarce resource, your attention, and sell it to advertisers. Because the supply is fixed and the demand is not, the software that ranks a feed is rewarded for whatever keeps you scrolling. In June 2019, the design ethicist Tristan Harris, who studied at the Stanford Persuasive Technology Lab and worked at Google, told a U.S. Senate subcommittee how much was already at stake:

Primary source 01 / 06

technology shapes where 2 billion people place their attention on a daily basis shaping what we believe is true, our relationships, our social comparison and the development of children.

Tristan Harris design ethicist, formerly at Google; co-founder of the Center for Humane Technology Testimony before the U.S. Senate, 2019 · Senate Commerce subcommittee, June 25, 2019

He stated the argument bluntly:

Primary source 02 / 06

I want to argue today that persuasive technology is a massively underestimated and powerful force shaping the world and that it has taken control of the pen of human history and will drive us to catastrophe if we don't take it back.

Tristan Harris design ethicist, co-founder of the Center for Humane Technology Testimony before the U.S. Senate, 2019 · Senate Commerce subcommittee, June 25, 2019

Why the feed rewards what it does

Harris traced the behavior to a single design pressure. Advertising pays for attention, attention is finite, so every product competes to capture a little more of it:

Primary source 03 / 06

Because in an attention economy, there's only so much attention and the advertising business model always wants more. So, it becomes a race to the bottom of the brain stem.

Tristan Harris design ethicist, co-founder of the Center for Humane Technology Testimony before the U.S. Senate, 2019 · Senate Commerce subcommittee, June 25, 2019

First the hooks were crude: pull-to-refresh rewards, infinite scroll, follower counts. As the competition intensified, Harris testified, it moved into the ranking system itself, where a model calculates which post will hold you next:

Primary source 04 / 06

the next step of the attention economy is to compete on algorithms. Instead of splitting the atom, it splits our nervous system by calculating the perfect thing that will keep us there longer, the perfect YouTube video to autoplay or news feed post to show next.

Tristan Harris design ethicist, co-founder of the Center for Humane Technology Testimony before the U.S. Senate, 2019 · Senate Commerce subcommittee, June 25, 2019

The target is time spent, not truth served. A ranking model does not ask whether a story is important or fair. It predicts what will keep you there, and the news that wins is chosen on that basis, not on the news you would be better off having read.

The editor you cannot see

For most of the last century the gatekeeper was a person deciding what ran. The filter bubble chapter followed that job as it passed to software, and agenda-setting showed that what the press emphasizes becomes what the public thinks about. The attention economy names the software's motive. It is paid for engagement, and engagement is not accuracy and not importance.

Harris described the relationship between the system and the person using it as a difference in power. The platform knows far more about you than you know about it:

Primary source 05 / 06

Persuasion is about an asymmetry of power.

Tristan Harris design ethicist, co-founder of the Center for Humane Technology Testimony before the U.S. Senate, 2019 · Senate Commerce subcommittee, June 25, 2019

That imbalance has an endpoint. If a machine can predict what will move you, it moves closer to being able to manufacture it. Harris told the senators where the escalation runs:

Primary source 06 / 06

As this progression continues the asymmetry only grows until you get deep fakes which are checkmate on the limits of the human mind and the basis of our trust.

Tristan Harris design ethicist, co-founder of the Center for Humane Technology Testimony before the U.S. Senate, 2019 · Senate Commerce subcommittee, June 25, 2019

The next chapter follows that power into the ranking systems now deciding what billions of people see. The chapter after reaches the point Harris named, where the machine can fabricate the evidence itself.

Frequently asked

What is the attention economy in simple terms?

It is the competition among apps, sites, and platforms for your limited attention, which they package and sell to advertisers. Because your attention is scarce and their appetite for it is not, the systems that rank your feed are rewarded for keeping you engaged, whether or not what holds you is true or important.

What did Tristan Harris tell the Senate in 2019?

Harris, a former Google design ethicist, testified in June 2019 that persuasive technology shapes where about 2 billion people place their attention each day. He described a race to the bottom of the brain stem, in which products compete for attention by targeting faster and more automatic human reflexes.

How does engagement ranking change the news?

A feed ranked for engagement promotes the posts most likely to keep you scrolling, commenting, and sharing. Emotionally charged and divisive material tends to do that well, so it is amplified, while quieter and more complex reporting is shown less, even when it is more accurate or more important.

Is the attention economy the same as advertising?

Advertising is the business model that funds it, but the attention economy is the wider contest for your focus. The advertising model creates the pressure to maximize time spent, which is why ranking systems optimize for engagement rather than for how well informed you end up.

The primary sources

The documents this chapter quotes. Read them yourself.

The field guide

Now watch it happen in real time.

Every mechanism in this guide runs in tomorrow’s news. The brief lays the same stories across ten worldviews so you can catch the framing yourself. Free.

Free to start. No spin. No yelling.

$5/month, cancel anytime. Secure on Stripe, and nothing is ever paywalled.

One-time, secure on Stripe. No account.