Robert Wack

Staying ahead of current events is a constant challenge for Science Fiction writers. What seemed like gee-whiz future tech yesterday is now sitting on the kitchen counter. If you are not careful, the future you create may end up looking like a bad episode of the Jetsons.

Thanks to a recent post by Cassie Kozyrkov (@decisionleader.bsky.social), we SF writers have something else to keep an eye on.  She points to an article by Mustafa Suleyman, the CEO of Microsoft AI, which outlines the imminent emergence of problems around Seemingly Conscious AI (SCAI). These are AI software programs that are so sophisticated, human users misperceive them as actually having consciousness. Mayhem ensues.

After concisely summarizing  Suleyman’s article (also worth a close read) Cassie prudently advises “The thoughtful need to be at least as loud as the thoughtless while the playing field is still even.”

Cassie generally agrees with Suleyman’s concerns and urges quick, thoughtful discussions around the challenges of users falling in love with their AI or ascribing moral agency and rights to software, behaviors collectively referred to as AI psychosis. She also points out positive use cases to address the crisis of loneliness and possible benefits of companionship, even artificial kinds.

These are issues explored repeatedly and in depth throughout the history of SF. One of the earliest and deepest thinkers on these topics was Stanislaw Lem, of Solaris fame. He has a book of essays, Summa Technologiae, published in 1964, that presciently addresses many of technological advances of the 21^stcentury and their impacts on humans and society, including machine consciousness.

Movies like Her, Spielberg’s AI, Ex Machina, and Blade Runner (based on Philip K Dick’s Do Androids Dream of Electric Sheep?) ably deal with human responses, good bad, and ugly, to created consciousness.

In addition to the obvious limitations of the human nervous system handling the complexity of reality (magic tricks!), perhaps AI psychosis is another example of the age-old feedback loop between culture (art, politics, and religion), science, technology, and economics. What if the science fiction imaginings of the past, in literature and movies, have shaped cultural expectations in such a way as to create the conditions, the receptivity, for AI psychosis due to SCAI?

Of course, other forces are definitely at work, including the lamentable waning of science education and critical thinking, as well as the baleful influences of conspiracy thinking, social media amplification, and disinformation.

I’ve always been fascinated by the historical examples of the complex interplay of these forces. A walk around Rome or Florence and diving into the politics, technology, and culture of the ancients and the Medici’s gives a good sense of how those feedback loops resulted in innovation, acceleration, and growth along multiple dimensions, changing popular perceptions of what is possible, thereby fueling the next phases of human evolution. Similar strolls in ancient capitals of any human civilization present similar lessons, as well as illustrating the ephemeral nature of any given culture when compared to the immensity of time. Progress is not certain or necessarily permanent.

The good news (at least for writers) is we SF writers usually end up being right about the future. The challenge is we’ve got to push the limits of imagination even farther to keep up with reality.