Beyond Optimism and Doom: Finding a Third Path in AI Discourse

A Foreword to Mostly Harmless AI by Nicolas Potkalitsky, Ph.D.

We are truly living through a moment of incredible change in the sciences and technology. As an educator, scholar of narrative and literature, and parent of two young children, I find myself uniquely positioned to witness these transformations, particularly in how they reshape our understanding of learning and knowledge creation.

I wake up each day with an incredible sense of gratitude. The human capacity to adapt to AI disruption is evident in my work every day with students, colleagues, administrators, and researchers. Major challenges, such as the AI efficiency-accountability paradox where we struggle to balance the speed of AI tools against our need for verifiable student learning, can become real learning opportunities in the hands of skillful instructional designers and experienced classroom teachers. And yet, this gratitude mingles with both anxiety and anticipation – anxiety about the potential loss of fundamental skills in the face of new systems, and anticipation knowing that today’s systems, around which we plan curriculum and interventions, will transform before we fully grasp them.

When ChatGPT 3.5 dropped in November, 2022, most educators had only an incipient understanding of artificial intelligence and its potential impact on pedagogy, instruction, and assessment. Here I number myself amongst this group. We knew that artificial intelligence was operative in our phones’ mapping systems and voice messaging tools. We had seen it at work in the previous decade’s rise of individualized test preparation and assessment, particularly in large initiatives surrounding data-driven decision making. We witnessed AI’s advancement across grammar and spell checking software as Grammarly became more and more adept at transforming student work. But when GPT 3.5 ripped out its first cogent essay, we knew something fundamental had changed. I personally felt devastated. I just knew I couldn’t continue to do things the way I was, and to be honest, I didn’t really know if I wanted to change everything at that moment.

After working through my initial denial and anger, I got to work, orienting myself to AI disruption as if it were a graduate-level research project. I spent the entirety of the summer of 2023 reading, reflecting, and writing, composing a school-response plan that helped guide its first steps out of AI denialism and into AI proactivism. It was at this point that I decided to start up a Substack, Educating AI, to chronicle my investigations and transformations in my own classroom and school. I knew little about Substack, except that my division director Dr. Blair Munhofen read several of them on obscure topics, loved the intimacy and intellectual depth of these inquiries, and highly recommended I start one up whenever the spirit moved me—knowing of my own love for obscure topics like object-oriented ontology, Augustinian volitional theory, modal logic, 21st-century ethical realism, ancient epigraphy, and more.

Beginning a weekly practice of writing has been life-transforming, not just because of the way it has pushed me to engage more deeply and reflectively with our schools’ response to AI disruption, but because it has affirmed to me—in a way that I did not even experience when completing my dissertation—that we write (and by extension, think) best in communities. Within several weeks of being on Substack, I came in contact with several amazing writers including Nat of The AI Observer, Michael Woudenberg of Polymathic Being, and Alejandro Piad Morffis of Mostly Harmless Ideas.

At the time, Morffis was running a digest of Tech Writers and invited me to join. This invitation led me to learning more about him: a beloved computer science instructor in Cuba, a skillful applications and tools designer, a highly gifted writer on technical topics, and a connector of people towards common goals. While at time the two predominant modes of discourse surrounding AI were self-described as “techno-optimist” and “doomerist,” Morffis and the small but influential community that gravitated around him were seeking out a third path—one rooted in a deep understanding of both human cognition and the developing nature of AI and LLMs.

In many ways, I have Morffis and other guides like Daniel Bashir and his wonderful podcast The Gradient to thank for transforming Educating AI from a newsletter of to-do lists and tool endorsements to something much richer and deeper. As a perennial student of the nature of thought, I came to recognize when surrounded by similarly oriented scholars in the machine-learning community that every technological disruption—every advancement in tool-being—every expansion of the tool-oriented social-actor network—sparks an opportunity to reassess what thinking is. But such work needs a theoretical container. Ideally, the container would emerge from the substance of inquiry. And yet, history always plays such a powerful role in our analytical process.

And so, Morffis and I, inspired greatly by the writings of Rob Nelson, who in a piece, “On Techno-pragmatism” redeploys William James’s pragmatism in the AI-infused contemporary, dusted off the aged mantle of pragmatism and used it to frame our shared and individual movements forward. Morffis wrote a beautiful piece, not included in this collection, but available on his Substack, called “The Techno-Optimist’s Manifesto” that captures the feeling of “our” moment at the end of 2023, as LLM development appeared to be slowing down and as techno-optimist and doomerist AI discourse was gradually becoming replaced by something more akin to AI frustration and resignation.

The core of pragmatism in Morffis’s work hinges on his commitment to human agency to find solutions to problems within limits prescribed by larger (formal or material) systems and processes. Here note how he shapes one of his tenets in the Manifesto: “Techno-pragmatism is accepting that the future is not predetermined. That we have the power to decide among many potential futures and the responsibility to make that choice based on reason and evidence, respecting the plurality of interests of all our fellow humans and being thoughtful about our planet and future generations.” The word “decision” is one of great impact for Morffis. Decisions of the kind described primarily operate in open-ended contexts and thus rely on the highest level of reason to be enacted skillfully and toward the benefit of others.

In the following, I will offer a short overview of “Mostly Harmless AI: Essays on Artificial Intelligence and its Impact in our Society.” When sending me the manuscript, Morffis described it simply as “a short collection of essays from my blog” implying something sporadic or serial in nature. But when read in the current sequence, as I recommend readers do, something of a deeper logic emerges. Each essay prepares the way for the conversation in the next—laying down essential terminology and concepts—that later are developed, redeployed, and expanded.

In the manuscript’s early essays, Morffis provides his own novel and pointed definitions of artificial intelligence rooted in a concise unfolding of the history of not only this technology but the human preoccupation with “cognitive automation.” His writing style is a hybrid between extremely personable, comprehensible real-world examples and the rigor of a mathematical logician. Indeed, Morffis’s bent toward logical precision is one of his greatest assets as he engages with foundational questions about the possibility of artificial general intelligence and automated reasoning–the most insistent conceptual thematics of the manuscript.

In the manuscript’s first essay, “The Road to AGI,” Morffis argues that current AI is capable of out-of-training generalization, out-of-distribution generalization, but not out-of-domain generalization. Importantly, these modes of generalization can be mapped somewhat stably onto Margaret Boden’s three types of creativity—combinatorial, exploratory, and transformational—the third type which many, including Boden herself, regard as beyond the capacity of machines. The rest of the manuscript in part is an effort to show along primarily logical and computational lines why “out of distribution generalization” will remain a remote possibility (remember, he is a pragmatist) for AI systems in the near future.

To understand Morffis’s argument, you have to understand something about the nature of formal systems. Luckily in Morffis, we have an expert guide who is also a skillful communicator. Setting the question of LLMs aside, Morffis advises that “if any computational model can reason–to the full extent of the meaning of this word in the context of artificial intelligence–, it must be able to perform Turing-complete computations, for if a model is not Turing-complete, that means there are decibel problems it cannot solve.” Then Morffis adds the pragmatic and practical kicker or follow-up: “One key aspect of Turing completeness is that it requires potentially unbounded computation.”

As a builder of machines, Morffis then shifts into solutions-mode: How do we get around this limit? “Now, by design, GPT-4, or any pure language model, cannot think forever….There is a way out, however. GPT-4 can generate the necessary code to answer [semi-decidable] question[s] and run [them]. And that is precisely what the newest iteration of ChatGPT does with Code interpreter.” But Morffis adds that this technology is still very much in rudimentary form and may only result in a kind of surface-level Turing completeness.

The incredible work that Morffis does in passages like these is to shift the conversation about artificial general intelligence away from value-laden, culturally-bound idealization of what it means to be human to something more clear and precise, and yet extraordinarily beautiful in its simplicity and exactitude. As Morffis adds, “human beings are also bound by this formal limitation”–meaning the limitation of the nature of formal systems–and yet by a variety of different metrics are regarded as capable of reasoning even when confronted with numerous semi-decidable problems simultaneously. Or do humans just appear to reason in these cases? This becomes a deeper preoccupation of the manuscript in its later chapter, “Large Language Models Cannot Reason,” which readers should know sparked off an internet firestorm when first published on Substack after the release of OpenAI’s ChatGPT o1 models.

So what does Morffis mean by reason? Here again, we encounter his characteristic clarity and precision. For him: “When we AI folks claim LLMs cannot reason, we are not talking about any abstract philosophical sense of the word”reason”, nor any of the many psychological and sociological nuances it may entail. No, we have a very specific, quantifiable, simplified notion of reasoning that comes straight out of math. Reasoning is simply put, the capacity to draw logically sound conclusions from a given premise.”

The fundamental grounds for Morffis’s argument that “LLMs cannot reason” lies in their constituent nature: “their stochastic nature.” Morffis is not among the camp that perhaps includes Turing himself that the mere performance of reasonable steps, or as in the case of advanced LLMs like OpenAI’s most recent ChatGPT o1 and o3, the detailed description of chain of thought processes including detailed explanations for each step in the chain of thought, constitutes reasoning in the above sense. Let’s see how he proceeds.

First, Morffis further extrapolates on the nature of LLMs: “These models generate outputs based on probabilistic predictions rather than deterministic logical rules.” Here, the terms “deterministic” and “logical” do a lot of work. Even as we improve on accuracy through RHLF, we cannot completely limit the occurrence of spontaneous incorrect conclusions: “An LLM might arrive at a wrong purely by chance, learning of inconsistencies in reasoning.”

Second, Morffis doubles down on his early thoughts about practical limits regarding computational time. “Large language models spend a fixed amount of computation per token processed.” Here we might recall that inside formal systems there are problems where it is difficult if not impossible in advance to figure out compute time: “Crucially, we can always find instances of NP-complete problems that require, even in principles, a sufficiently large of amount of computation to surpass the computational capacity of any LLM, no matter how big.”

Here, the work is as impressive as it is rigorous. For some, we may be drifting too far away from what we think of as everyday reasoning. Do we really mean the grappling with NP-complete problems when we ask questions like “Do LLMs reason?” The ARC Prize formulated by François Challot seems like a step down in complexity in comparison to Morffis’s framework, but in the slippage of current AI discourse around reasoning, the risks are great if we simply concede that probabilistic processes or what Morffis calls “very large finite automata” operate at a level of efficiency, accuracy, and logical precision—absolute equivalence—with their makers. Not only is the nature of language itself at stake, but so much that we hold dear as humans.

In the remaining chapters of Morffis’s book, he explores offshoots from these foundational questions and conclusions, focusing on the all-too-human situations and conundrums that result. If an LLM can code, should we learn to code too? Yes, Morffis answers emphatically. Are Chatbots a conduit or obstacle to critical thinking? How will AI change the classroom? Should we be worried about AI’s taking over the world? What would it mean for an AI to share human values? What should the future of AI industry and development look like? In each case, Morffis proceeds with a careful, nuanced, and pragmatic eye to the multi-dimensional nature of each of these questions.

What makes this collection particularly valuable is how it bridges the gap between technical rigor and humanitarian concerns. As an educator who began this journey from a place of trepidation, I find Morffis’s measured approach both reassuring and intellectually invigorating. He demonstrates that we can embrace technological advancement while maintaining a clear-eyed view of its limitations. In doing so, he provides a framework not just for understanding AI, but for teaching and learning in an AI-transformed world. This is not just a collection of essays; it is a roadmap for educators, technologists, and thinkers who seek to navigate the complex intersection of artificial and human intelligence with both wisdom and hope.