When Content Costs Nothing
Truth in a world of cheaply generated deepfakes
Knowledge has traditionally been gatekept by a small number of journalists, scientists and politicians that could communicate broadly with the public. This was primarily a logistics issue. Sure, anyone could have their own opinions, and borrow books from the local library. But if you wanted to share your ideas beyond the dinner table, it generally required some kind of institutional backing or accreditation. These constraints led to an era defined by institutional power, with much power being concentrated in media and academic gatekeepers. Over time, new technologies, such as writing, the printing press and the internet, have continuously made it easier to disseminate information.
With the internet came a wave of influencers, thinkers (and grandparents on WhatsApp) that could share their ideas more broadly. This reduced the influence of the established sources and allowed for more dissent and contrarianism. Undoubtedly the impact of the internet in how information is shared and verified has been huge, though it’d be hard to find consensus as to whether this impact has been net positive or negative. It increased the ability of whistleblowers and marginalized people to share their challenges, though some say it’s led to polarization, mental health issues and the spread of misinformation.
However, this trend, of increasing ease of communication, may finally be saturating as the most powerful people in the world can (and sometimes do) communicate with billions of people from atop their toilet seat. We have, of course, benefited immensely from the ease at which ideas could be communicated. This ease has enabled closer interpersonal relationships, easier access to information and lots of scientific progress. Though, as we will undoubtedly discover, easy communication combined with a massive increase of content will have idiosyncratic effects.
Thus far, every technological advance has been constrained by a tight bottleneck in that for any message to be communicated and spread, it required significant labour and human capital to craft said message. What that means is that the very act of reading something necessitated that significant time went into creating it, and that necessity acts as a filter preventing the most unhinged or useless sentences from being written to begin with. Newspapers may not have been elixirs of truth; however, you knew that if you were reading something that someone, somewhere thought it worthwhile to write it, and had the ability, funding and backing required to disseminate it to you. This filter has been collapsing for years, as we’ve made it easier to share ideas, even without the “proper” credentials. Today, this trend is accelerating dramatically as the cost of creating content collapses to near zero.
LLMs can generate text, audio, and video with increasing quality. Risks of technological addiction aside, this glut of content has massive implications as to the spread of disinformation and will change how people decide what to believe. Not only does the vast amount of generated content make it more difficult to know what is true, it also makes it considerably easier to dismiss quality content as fake. Already we’re overwhelmed by the vastness of information to be consumed online. People grapple with the difficulty of discerning what can be trusted when so much conflicting information is out there. This is Brandolini’s law on steroids. Most people will always resort to their pre-existing beliefs and tribal affiliations, even when they try to expand their horizons, though we need good methods for when we are, in fact, optimizing for truth. What happens when the amount of content multiplies by 10, 102, 103, 10x? And what happens when this content is designed to trigger our emotions, beliefs, or trigger certain actions in the real world?
Deepfakes, when they proliferate, by default lead to a low trust world with conflicting information emerging from different sources and no obvious way for people to determine what is actually true. Mitigating this risk is tough. Any centralized efforts to determine truth will be faulty and is unlikely to be fully embraced. Technological solutions, like watermarking technology or AI detectors are undermined by open-source AI, and the fact that many are unable or unwilling to determine whether a piece of media was fabricated. Note that Apple has thus far declined to use the standards set by the Coalition for Content Provenance and Authenticity (C2PA) allowing us to determine the source and edits on images based on cryptographic metadata.
Prediction markets, which merit an article of their own, are one method of aggregating accurate information. Robin Hanson has written extensively about their potential to revolutionize how we decide what is true, though his vision has yet to be implemented successfully. He has proposed Futarchy, a form of governance in which democratically elected officials set goals and prediction markets are used to determine the best way to achieve them. Conditional markets, such as “If policy X occurs, will outcome Y follow?” provide incentive to calibrate correctly, and even if X never occurs, money can simply be returned to holders. A fascinating idea, this remains a fantasy as current prediction markets have seen little real world utility and can only be applied to events with clear yes/no resolution criteria. Though with prediction markets rising to prominence time will tell how powerful they become, for good or for bad.
Often when arguing with friends, or when people need a quick reality check, they’ll resort to asking an LLM. It doesn’t matter if they’re always correct, it still provides legitimacy to your opinion and can serve as a check on your intuition. This is like the role Wikipedia has traditionally served. As LLMs become more grounded in truth (better models paired with better scaffolding) people will naturally begin to trust them more just as they did with Wikipedia, though with the added benefits that AI is not as rigid and can answer any question you have. This approach is not without risk, as providers playing with the training data or system prompt can render wildly different responses, and LLMs will continue to hallucinate and misinterpret user questions for years to come, providing false reassurance. Though as models improve and trust in them grows, they will inevitably become an increasingly used tool to figure stuff out.
Another solution, that has seen success in the real world, is the Community Notes feature. Recently added to Twitter (now known as X), it uses an open-source algorithm that allows users to create a note providing context or a fact check to a given Tweet. It uses a bridging-based ranking system to ensure there is consensus. If a note is ranked highly by other users that tend to disagree with each other, it’ll be more likely to be accepted and prominently featured below the tweet, which serves to provide useful context or limit the spread of disinformation. In my experience, it’s highly effective, and does not limit free speech, without relying on centralized fact-checkers that are not always broadly trusted.
Now that technology has reduced the cost of creating content to near-zero, scarcity has moved beyond finding information into determining what is worth our attention and what is true. We need to determine the best methods for sifting for truth and how we can have institutions that encourage truth to become consensus. Only time will tell what this looks like.
In summary:
Over time technology has made it easier to communicate, today it’s making it easier to generate messages too.
While ease of communication is incredibly useful, the overwhelming volume of information, accelerated by LLMs and hard to filter effectively, makes it hard to know what’s true.
It’s unlikely traditional media will regain its former glory, though other decentralized forms of verification such as prediction markets, LLMs, and community notes will likely become larger features of our epistemic arena, at least for those seeking truth.