Among the classes I’m taking at the University of Arizona is one titled “Learning Theory for Instructional Design.” I expected this class to do more or less what my literary theory classes did when I was studying English, namely bore me to tears and convince me that theory was a load of hooey. Happily, the class is actually structured to survey various theories of learning in a way that suggests that there’s probably something to each, but that in the end we’ll need to draw from different theories in different ways depending on our teaching philosophy and the task at hand.

So that will explain why I was reading earlier today about B. F. Skinner. I’d heard of Skinner before, and was vaguely aware that he had invented some sort of box in which to sequester infants, which struck me as odd, if not without merit. If you don’t know anything about Skinner, here’s a quick summary. B. F. Skinner was an American psychologist who studied the relationship between behavior and environment, and whose ideas on the subject place him in the school of thought known as Behaviorism. He was active in the field from the late 1930s until his death in 1990, although he was probably most influential in the 1950s.

Behaviorists believe that all animal and human behavior is caused by environmental factors, and that anything anyone does, thinks or feels is a response to some outside stimulus. At the most basic level, they believe that actions that are rewarded will be repeated. Skinner was not as extreme as some earlier behaviorists, who argued that anything going on in the “black box” of the mind was completely irrelevant to one’s actions, but he certainly didn’t believe in free will.

Skinner is relevant to me from a game design perspective because of the way that some of his ideas have been used to understand another, older kind of gaming: gambling. Like Pavlov, the father of the Behaviorist school, Skinner conducted experiments on animals that used food as a reward. One day, running low on rat pellets, he improvised an adjustment to his system. Rather than rewarding the rats each time they performed a desired action, he would reward them every third, fifth or tenth time. Surprisingly, this didn’t stop the rats from doing what he wanted. They kept doing the same thing at the same rate despite the fact that the rate at which Skinner reinforced their behavior had changed.

Successive experiments helped Skinner understand other ways that learned behavior changed when the ratio of reinforcement was modified. He termed these his “schedules of reinforcement.” In addition to the already mentioned fixed ratio schedule, in which an action was rewarded every xth time, he devised fixed interval scheduling, which meant that the reward would arrive after a set amount of time.

He also discovered that you could produce some really interesting results using variable scheduling, in which the number of desired actions (or the amount of time that needed to pass) before a reward was doled out would change between each reward. In other words, you might get the first reward after ten seconds (or six actions), but the second reward would come after 15, 5, or 40. Rats, pigeons and other animals had an extremely hard time breaking habits of behavior that had been reinforced in this way. Skinner drew an analogy to human gambling on slot machines, in which players had no way of knowing when the next payout would occur—just like the animals on a variable reinforcement schedule.

Before I read about Skinner, I had seen the argument advanced that loot drop games are somehow “unethical” and different from other games in how they motivate players to continue playing.

I had a sense that something was different here, and now I think that the difference is in the direct and very literal way that these games implement the idea of variable ratio reinforcement. Think about games like Diablo, World of Warcraft, Everquest and Phantasy Star Online—how do they dole out the top-notch gear that keeps you going for just that one more boss monster, just that one more quest, just that one more month? It’s semi-regular, but the precise timing and payout is always unpredictable.

All games use a variety of kinds of positive reinforcement to keep the player engaged, and in many cases these can be correlated to aspects of other entertainment mediums. Maybe it’s social interaction, as in a board game like Apples to Apples. Maybe it’s a desire to move a linear narrative forward, as when we read a book. It’s games in which the fun comes from grinding yet another level or scoring yet another epic drop that are probably best compared to gambling; is it any wonder that these are precisely the games most often described as addictive? (Puzzle games are also frequently described as addictive, but I don’t yet know how that fits in to my little hypothesis.)

Knowing that people are inclined to find mechanics based on variable scheduling hard to resist makes me wonder about big questions of ethics in game design. Every game needs a “hook,” a clever and fun mechanic, to get players interested. Is there a problem with using our understanding of human psychology to make that hook stronger? Is a game designer’s first obligation to his company’s bottom line? To players of her game? To some abstract notion of creating a set of rules that interact in as interesting a way as possible? Finally, if you don’t believe that consciously setting about to create an “addictive” game is wrong, is there anything a game designer could do that you would consider troubling or unethical?

I should point out that I’m not the first person to make the connection between Skinner and games—far from it. Some mentions of this idea appear in the following places: