In part 2, we’ll take a closer look at some of these issues and problems that came up, particularly the ones that were later addressed with some of the approaches that are currently still in use. As we’ll see, for all of the problems that they do solve, stochastic approaches bring unique challenges of their own.

This series will attempt to answer the question of how we got from deterministic to probabilistic approaches (part 1), from probabilistic ones back to the problem of hallucinating math answers (part 2), and will end by sketching some ways in which contemporary AI research has attempted to address these issues (part 3). 

A recent study from the MIT Media Lab, "Your Brain on ChatGPT," offers a compelling empirical analysis of the cognitive ramifications of utilizing Large Language Models (LLMs) in academic writing. The research has implications for pedagogy and cognitive science, introducing the concept of "cognitive debt" to describe the neurological and performance-related consequences of outsourcing intellectual labor to artificial intelligence. My analysis of this work finds it to be a solid contribution to the discourse on AI in education, though with some issues.

In the rapidly evolving world of artificial intelligence, large language models (LLMs) have emerged as powerful tools capable of generating human-like text, answering complex questions, and even assisting in knowledge work. At the heart of their impressive capabilities lies a mechanism called "attention." While attention layers have been a revolutionary breakthrough for LLMs, they also come with significant bottlenecks in computational speed and memory usage. Two new and impactful architectural implementations seek to solve some problems, even despite the persistence of some interesting kinds of “bottlenecks” in memory and speed.

For this edition of the QWERKY blog, we posed three questions to three of the people who made the striking design and creation of this custom lager possible.

How Large Language Models (LLMs) are fundamentally deterministic systems and why they surprise you with non-deterministic behavior.