Robert Wack

This is the first of two essays attempting to demystify the basic workings of AI programs. Hopefully we can cut through the sloppy media hype to help people focus on the real risks we face, not apocalyptic fantasies. Bottom line up front: the real problems with AI aren’t the software, it’s all the humans around it.

This is written for the non-technical reader, but where technical issues are essential for the explanation, I will provide hopefully clear explanations. Because math and computer science are central to how AI does its work, some familiarity with their concepts is important.  The first section is a basic discussion of how AI works, particularly agentic AI.  The second is about what to be afraid of and why. I am pessimistic, but not for the popular reasons. AI will cause problems, not because it is conscious and evil, but because humans are greedy, lazy, and stupid. The solutions are as simple and obvious as they are difficult to implement: regulations.

Throughout, I’ll point out a source of both the challenges understanding AI and the misplaced fears. The words used to describe both functions and risks are imprecise and have assumed meanings that go further than the reality they are supposed to describe, which has a strong unintended consequence of anthropomorphizing the software. 

Part I: what it is and how it works (very simplified and incomplete)

What is AI?  Let’s go to Wikipedia:  

“AI is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making.”

At first glance, this sounds like all the things we’ve always expected computer programs and machines to be doing, and for each function individually, that is true. It’s the coordination of these activities together, and the scale at which it happens, that takes AI to the next level. But at the foundation, AI is software, lines of code that do things we’ve never been able to do before. Don’t lose sight of this.

We’ve also encountered the first linguistic problem just inside this definition.  The word “perception” gets thrown around a lot in AI coverage. We’ll unpack that in a bit. These slovenly linguistic shortcuts create A LOT of problems down the road. The second essay will dive into the linguistics, but for this essay, when a word is used in a problematic fashion, I’ll put it in quotes the first time.

The features that make AI different from other software programs revolve around several functions:

• Iteration (doing things repeatedly to refine and improve the output);
• Collaboration (pulling in additional resources to achieve a goal);
• Self-modification and adaptability;
• Autonomy (working on a goal without additional input after the initial prompt, or question);
• Scale and speed of computing using vast amounts of hardware and power, dwarfing prior “supercomputer” endeavors.

Standard explainers easily found on the web describe the integration of these features as a multi-step, integrated process where the software “perceives”, “reasons”, plans, acts, then “reflects” and adapts (lots of problems with some of these words).  These functions are looped, repeating over and over until a goal is achieved.

This looping isn’t peculiar to AI.  Continuous quality improvement in healthcare and industry uses it (the PDCA cycle: Plan, Decide, Check, Act ) and in aircraft operations it’s called the OODA Loop (Observe, Orient, Decide, Act). In any case, you analyze something, make an incremental change, implement it, measure the effect, then make another change to pursue some optimization goal.  In software, they are called “for loops.”

Although computer science has been discussing AI since almost it’s very inception in the 1940’s, and even before that in speculative literature and science fiction, the explanations in this essay begin with the advent of Chatbots.  In all their various forms, they emerged from work on something called Natural Language Processing. Typically, computer languages are rigid and rules based. One small deviation, an extra space, one typo, one misplaced comma or period, completely destroys the meaning of that line of code, which in turn crashes the entire program.  Natural Language Processing takes casual speech and translates it into software code that can be handled by a program with less error (not zero, though).

The next step was the development of Large Language Models (LLM). The high level steps for building an LLM are as follows:

• A large (the bigger the better) body of text (the training data) is analyzed statistically to quantify the relationships between words, phrases, and sentences;
• These relationships are codified into mathematical statements about the statistics pulled from the training data, creating the “model”;
• The model is then used to predict what other inputs might look like, using other data (the test data).
• Depending on how the model performs predicting relationships in the test data based on the training data, the model is continuously tweaked and modified.

The process by which that tweaking and modification occurs uses techniques collectively

referred to as “machine learning”. Machine learning is a sloppy term used to describe a suite of mathematical tools that start with linear regression, the process of finding a linear function in a cluster of data points, through a process of fitting.

            A quick, simple example: you’ve got a collection of data with two parameters, the recent sale price of a house, and it’s square footage. You’ve got 20 examples, twenty data points, and you plot these, price on the Y axis, square footage on the X. The graph shows some scattering, but there also appears to be a general trend, up and to the right, suggesting that as square footage increases, so does price. But now you want to know more specifically, how much does price increase based on each increment of square footage? You use linear regression to “fit” a line through those points. The fitting process minimizes the total distance between each point and the proposed line, by playing with, or optimizing the two factors that control the line, the slope, and the y intercept (remember y = mx + b?). Once you have your “best fit” equation, now you can make a prediction about a new data point, a house not in your training data set.

            Now, we can add more variables, like yard size, distance from a school, proximity to recreational space, and on and on. Each data point has more parameters, one for each new detail, which creates more dimensions in which the datapoint resides. In our first example, the data points have two dimensions, price and square footage. With each new parameter we add a dimension. Finding a “best fit” line requires moving to more complex mathematics, like Multivariate regression, and the functions don’t have to be linear, they can be other types as well. 

This is how an LLM takes that to the next level by having multiple parameters for each data point, in this case words, and using far more sophisticated mathematics to pull out the relationships between the words, phrases, and sentences. Next, the software continuously refines and optimizes the math by testing the model derived from the training data using different testing data.

            The parameters of the data are also referred to as labels, and the process of applying labels to the parameters of a data point is classification. Classifying data is the first part of the “learning” in machine learning.

            This “learning” can either be supervised (the test data are known and fully labelled) or unsupervised (brand new, unlabeled data that the model has to classify).

            To be clear, the learning is just the refinement of statistical relationships between the data points, and only for the data present in the training and test data.

            This is where some of the fundamental weaknesses of AI applications begin emerging. Because this whole exercise is one of applying math to language, the fundamental differences between math and linguistics create unavoidable imprecision and uncertainty. Words are inherently blurry, slippery, and changeable. The statistical relationships between certain words in one data set can be very different for the exact same words in another data set. We’ll get further into the challenges of math applied to linguistics in the next essay.

            Take the words “black” and “cat”.  If found in a data set containing descriptions of all cat varieties, the frequency of black and cat appearing together will have one value. But if the data set also contains text about Halloween, now the probability of black and cat together goes up, and if you add in the words “witch” and “bad luck”, now they are even more closely related.

            AI scientists respond to the need for more context and nuance by shouting “more data!” and they went out and got it, as well as continued layering in increasingly more sophisticated mathematics to process all that data.

Technical explanation

Let’s go through a very high level summary using the math behind Chat GPT-3 as an example. It is the most recent version for which underlying math is available publicly.  This will be the most technical part of the essay, so if you want to skip ahead, feel free.

This will describe some of the steps used to process and “learn” from text data. The first step is the input text is broken up in to “tokens” the pieces of text data that will be manipulated mathematically. Tokens can also include punctuation, and don’t necessarily correlate to whole words, but for the sake of this example, we’ll just keep saying “words”.  Each input word is compared to an existing dictionary and given a numerical value placing it in a matrix based on information already stored in the dictionary. Then the software looks at the words before and after each word and builds additional matrices that capture that information. Now words have multiple new mathematical features with many dimensions that are captured in “vectors” that represent the meaning and context of each word. The statistical relationships between words create “weights” that become the tunable parameters of the matrices. When you add a word, phrase, or sentence, now the vectors representing those are multiplied with the matrices, which then changes the weights, because now you have more data about how the text relates to other text.

Think about our black cat example. The weights assigned to each word, “black,” “cat,” and “witch” will depend on the other words around them. Since in some contexts the probabilities of those words appearing next to, or near each other, change, those changing probabilities are captured in the weights.  As each new word of the input data is examined, the software goes back and updates all the existing weights, and then on to the next word.

The language model (the software performing these mathematical operations) builds a vast array of matrices capturing the statistical relationships between every word, phrase, and sentence in the data set.  Now you have the basic model which you’ve “trained” on your initial text.

The next step is to take new data and see how the model performs predicting text using new data. If it makes mistakes, you correct them, which updates the weights in the model, which is how the model “learns”.  The last step is to turn the model loose in a real world environment. Every time you interact with a chatbot or perform some little trick (identify the bicycles!) you are helping train an LLM.

How big does this get?  Chat GPT-3, now superseded by 12 newer versions since its release in 2020, and considered “simple” today in comparison, handles variables with over 12,000 features, or dimensions (remember our first example had 2!), using 175 billion weights (the statistical relationships between words, phrases, and sentences), embedded in 28,000 matrices, which are organized into 8 categories.  Each interaction involves manipulating this mind-boggling mountain of mathematical terms, billions and billions of times.  This is why AI is such a computer intensive process and the need for all the data centers.  Newer versions are more complex and are estimated to contain tens of trillions of weights, but the statistics about the most recent models are all proprietary.

End Technical explanation

Two examples of how LLMs can be used (there are many) which have big implications for both amazing benefits and staggering potential harm.

Generative AI is the term used to describe AI that “can create original content such as text, images, video, audio or software code in response to a user’s prompt or request.” (from IBM).  It does this by looking at the patterns of relationships in the LLM and then extrapolating from those patterns. This called “inference” in the AI world.

An example might be the prompt (a query, question or goal), to “write a letter to the editor complaining about snow removal in the style of Ernest Hemingway”.  To my knowledge, Ernest Hemingway never wrote such a letter, so there wouldn’t be any examples in the training data, but there are examples of “letters to the Editor” (LTE), “complaints about snow removal”, and “works of Ernest Hemingway”. Each one has its own statistical relationships between words, phrases, and sentences, allowing the creation of models for each, and the generative AI then takes those, combines them, samples the text based on probabilities, and tests them against the input query: does this resemble a LTE regarding snow removal written by Ernest Hemingway?

The results can be amazing, entertaining, and somewhat spooky, but it’s very important to not lose sight of how it happened: shuffle, update and combine the models, test, shuffle, combine, test.  Is that thinking? Not in the slightest. Is it even creating? Only by the most expansive definition. It’s more accurately described as: select the next most probable word, phrase, or sentence; measure how well it fits mathematically with the goal of the query; regurgitate, then rinse and repeat until the user accepts the output. The next layer of complexity comes from now calculating mathematical relationships between words that describe higher and higher level concepts like strategy, plan, and organization, which then enables further layers of complexity in queries and outputs.

These processes can also be applied to pixels for visual data to create new images, and the tiny sound samples of audio data to synthesize music and voices.

In addition, the software can now write computer code, searching its training data for new combinations of code to solve specific problems. This is very important in the next stage of AI development. LLMs is particularly well suited to this task, because unlike human speech, computer code either works or it doesn’t, so the outputs are easily tested against the requirements of the query. This is having a major impact on the computer science job market right now.

It’s important to reiterate once again that despite the complexity, this is still a number crunching exercise, and the outputs to any query are completely constrained by what is in the training data.

Now we get to agentic AI. This is where the problems really start. AI agents have several features that further differentiate them from other types of AI:

• Persistence across sessions; even after you stop interacting with the program, it remains active, working on whatever you asked in your prompt or query; unlike other forms of AI, it has “memory.”
• Integration with outside sources, databases, other software tools, and most importantly control systems; the toolset an AI program can access is much more diverse;
• When it iterates, the software doesn’t just loop back to its training data and models, it now can loop back to its own command line, the entry point into the software for writing new queries or questions;  generative AI requires a response to “Is this what you’re looking for?” before it can continue iterating; AI agents take that query, match it against the original query requirements and continues iterating, working independently of further human input.
• Because it can now write new code and access other software, the agents can create new programs, even other AI programs, and use them as tools for solving the problem at hand; this enables collaboration with other agents;
• Because of the looping back to its own command line, the software, if allowed to, now has autonomy – it can operate without further input until high level goals are achieved, working through complex, multi-step processes, iterating off it’s own intermediate outputs without additional inputs or supervision.

A quick word about control systems. These are the software programs that automate the monitoring and control of complex physical systems, like power plants, heating and cooling systems, water purification, traffic lights, and power grids. They monitor the system and exert controls automatically, based on rules that humans create. Humans watch, and may intervene, but for the most part, control systems are automatic. They are computer programs that are linked to actuators, or effectors, systems that actually do things, like open a valve, connect a circuit, raise a temperature, close a door. They are what control everything from your home’s heating and cooling all the way up to the load balancing on the electric grid or the operations of a nuclear reactor. Despite all the automation, there are always humans overseeing the outputs of controls sytems and intervening when necessary.

AI agents are now sophisticated enough that they can manipulate a control system without supervision, if given the parameters within which the system is supposed to operate.

This very general, high level, incomplete description of AI is sufficient now to discuss the possible risks of this new, rapidly developing technology. The advances come from ever more sophisticated mathematical techniques to find or create connections within vast amounts of data at blinding speed. But with more math, comes more complexity, and with complexity comes more opacity. We are now at the point that unless the AI software is specifically instructed to basically show its work, the human operators do not know how it arrives at its answers.  This is another source of difficult problems.

How could AI cause big problems?  It’s always the humans, unable to deal with their own greed, stupidity, and impulsiveness. Just as a sophisticated piece of technology like a modern automobile can be misused to cause harm, the scope of potential damage caused by AI is constrained only by the devil’s playground of self-indulgent human imagination. If we allow highly capable agentic AI to have access to nuclear weapons, that could be a big problem. If we allow agentic AI to have access to the complete cycle of artificial life synthesis, that could be a problem. If we allow agentic AI to have access to autonomous weapons systems of any kind, that could be a problem. If we allow irresponsible people access to this astonishing technology without rules or supervision, that WILL be a problem. All of these require humans to take active steps of incredibly imprudent nature. These are choices. 

The next essay will dive deeper into the specifics of why we struggle discussing this technology, and the risks it creates, without lapsing into hyperbole and hysteria.