Dangerous concentrations of intrinsic rewards

In my previous post, I tried to make an impression of myself as being “the poor overhauled guy who does not have five free minutes to tend his blog”. Which I was, of course. Nonetheless, my excuses would have been much more honest, if I wrote that one day earlier. But I couldn’t. I had to play. A role-playing game, what else. To be precise, it’s not the kind of RPG that tells heroic stories, it is a hack’n’slash RPG (ironically, these games involve about as much role-playing as a game of Tic-Tac-Toe. I think, “role-playing” must refer to the fact that at some point of such games, you’ll have to kill dragons).
Role-playing games (still speaking of the hack’n’slash genre) are highly addictive. Somehow they stimulate the part of the brain that gives intrinsic rewards, to encourage activities that give no advantage at the moment, but they feel like progress. You feel highly inspired to improve one more level, to collect the funds for a bit better armor. Or, as in my case, to hack some monsters to get the money to buy a luck-boost to increase the chance of getting rubies, that are needed to enhance your sword, so that I can hack a boss monster to get some emeralds to enhance my other sword so that I can hack that other boss monster… (discontinued out of politeness to the reader).

The game Ginormo Sword is among the most addictive RPGs I’ve seen, and it’s hard to see why. The screenshot above tells everything about the lack of graphics. Audio is on par with the graphics. There is not even a tiny bit of storyline. And it is boring. “Boring” is not expressive enough – the game opens whole new depths of boredom. Most of the gameplay consists of picking a battlefield, where you encounter several chunks of pixels (claiming to be monsters). Then you have to hit them with your sword until their green life-bar shrinks to zero. To this end, you keep clicking the mouse for several minutes, and avoid dying. When done, you get money, and you can go to a new battlefield (or the same one again, for that matter), to click away other agressive chunks of pixels. Repeat. Many times.

Can you imagine how boring it is? I think you cannot until you try it :-) And I bet you won’t be able to stop for a long time. Amazing game it is. A game made up of pure, distilled intrinsic motivation, without being obscured by graphics, story, or entertainment.
I would be overjoyed if I could formalize, what this game does to people (some poor trapped souls on this discussion board must have spent weeks with this game). If only I could inspire a reinforcement learning agent to chase progress with such perseverence! I know of a few attempts (one, two, three). We’ve recently submitted a paper about our own take on intrinsic rewards (yes, there is RPG inside). But none of these attempts seems nearly as powerful as Ginormo Sword. It is a game made up of pure intrinsic rewards, nothing else. Beware.