Category: Programming

Dwarf Fortress is one of the most complicated simulation/management games ever made. It’s been accumlating systems and interactions over twenty years of development. Here’s an example from my own fortress.

When a fortress gains enough wealth and population, a mayor is elected. I’m not a fan of nobles. I have to build expensive quarters for them and they make demands that conflict with my designs for the fortress. What do these nobles contribute?  The ideal fortress government is an invisible, unaccountable force inposing its whims from outside the universe! (That’s me.)

Here’s Id, the mayor. She’s defined by an overwhelming amount of information about her personality, desires, skills, health, memories, and so on. What’s important for our story is at the bottom: She absolutely detests snails.

So I built her a symbol of office: a crown made of cheap material with a humilating name and image.

Here she is wearing it. Note the tears in her eyes.

Soon after she put on the crown, she made a new decree!

The yearly trade caravan requested goblets (and thread) for next year. They were willing to pay double, and now I’m not allowed to sell any. The mayor scuttled my trade deal out of spite!

It’s fun to believe that I have beef with a fictional character, so that’s the story I’m telling. Is this story completely true? The only way to know for sure is to read the mayor’s thoughts.

Dwarf Fortress tracks the thoughts and memories of every character.  Here are the mayor’s recent thoughts. She feels proud about being elected, and the tears in her eyes are probably from a recent meeting where an unhappy citizen cried on her and she felt empathy. She also likes goblets (see the first image) so it’s likely that she’s hoarding them for herself, not trying to spite me.  There’s nothing about being ashamed or angry about her symbol of office.  The motivations I invented for her are not supported by the evidence.

When making a procedural generator, instead of letting a parameter vary across the entire range of possible values, considering breaking that range into sectors.

EDIT: I used to call these limited ranges “chunks” instead of “sectors” but procedural generation in chunks makes people think of Minecraft, so I changed the name to avoid ambiguity.

Sectors

Not overlapping isn’t enough. Sectors should have space between them. This reduces the total number of possibilities, which seems bad, but each value will have greater meaning.

If the Small and Medium height sectors met at 1.5m, artifacts could be generated whose height is ambiguous. I can’t tell the difference between 148cm and 151cm at a glance, so if I see creatures with those heights, I won’t know if they are Small creatures or Medium creatures. Stopping Small at 1.25m means there’s 25cm between the tallest Small Creature and the shortest Medium creature, enough to remove ambiguity.

Sectors don’t have to be the same size. They need to be different enough that observers can tell them apart. Counting 10 or 12 legs will take a while, but I immediately know they are both “a lot” and thus bug-like. The range for biped has no variation at all, but that’s valid too.

Limited sectors promote identity

For example, Hunter clothes come in desaturated earth tones to help Hunters blend in with natural surroundings.  If all clothing was available in all colors, Hunters would have less of an identity. Choice of clothing tells viewers something about the person wearing the clothing.

Figure 3 the output of a star generator with the ranges on size, color, spikiness, and number of points opened up. It’s easy to associate to the two green spiky stars at top right as being kind of the same thing, but why would I think the red diamond on the left edge is the same kind of thing as those other two? The class of “stars” is so diverse that it loses cohesion.

In Figure 4, I tightened the colors to warm, autumnal colors, and reduced the amount of variance in size, number of points, and spikiness. These stars seem more related to each other. It’s easy to mentally group them as the same kind of thing, but there’s still quite a bit of variance between each star.

Perceptual differentiation

Perceptual differentiation is the ability to tell two things apart.

In Figure 5, I’ve mixed the autumn stars with spiky grey stars. Even though each star is different from all the others, it’s easy to see that there are two different kinds of stars, and to sort each particular star into one of those two groups.

Individually unique objects that are easily sorted into groups are a nice way to ensure perceptual differentiation. If the two classes uses disjoint chucks of the parameter spaces, two objects from different classes are sure to look different from each other.

Even if you end up with 10,000 bowls of oatmeal, if you also offer 10,000 slices of toast, 10,000 waffles, and 10,000 cups of yogurt, you have a better breakfast buffet.

“Perceptual differentiation” and “10,000 bowls of oatmeal” are both terms from Kate Compton’s excellent blog post So you want to build a generator…

The sector becomes information

If a certain feature of an object is not completely random, it can tell us something about that object. In ARPGs like Diablo, a potion with blue liquid will restore my energy, no matter the size or shape of the bottle.  blue potion = mana. if a tree has a conical crown and needles instead of leaves, I know those leaves won’t fall off in winter, because conifers are evergreen. Live oaks are also evergreen, but their leaves and crown are shaped closer to deciduous trees than conifers, so it’s easy to mistake them for deciduous trees.

The next section shows how one class of object can have multiple parameters that each have some meaning, so an observer can quickly recognize many features of

Layers of differentiation

Every object will have multiple parameters, and each parameter can be divided into multiple sectors. By giving different objects the same or different sectors, you can create a hierarchy of relationships between objects. I’ll use Starcraft as an example.

Races:

• Protoss:
  • colors: pale yellow, glowing blue
  • shapes: smooth curves
• Terran:
  • colors: shades of grey
  • shapes: rectangles & circles
• Zerg:
  • colors: reddish to purple
  • shapes: tubes, sacs, claws, and teeth

Unit types:

• Buildings
  • size: huge
  • number: solitary
  • features: immobile, grounded
• basic units
  • size: small
  • number: large groups
  • features: mobile
• elite units
  • size: large
  • number: small groups
  • features: mobile

Terran Infantry:

• All infantry:
  • small
  • humanoid
• Marine
  • big shoulders
  • long gun
• Firebat
  • backpack
  • twin weapons
• Medic
  • white armor
  • shield

Seeing these patterns helps me recognize objects quickly, and make good guesses about new objects I haven’t seen before.

If I see a mass of Terran Infantry (small, humanoid, grey, large numbers) I can pick the Firebats out from the Marines by looking for the backpacks, because that’s the feature they DON’T share.

If I see a weird purple creature with big teeth, I know it’s Zerg, because that’s what Zerg look like. Protoss and Terrans don’t make things like that.

If I see a huge, immobile, radially symmetrical object, it’s probably a key building, like a Town Hall, because that’s what all the other Town Hall buildings look like.

Notice that the relationships can be hierarchical (Terran Infantry is a subset of Terran) or cut across groups (buildings share characteristics across races)

Use an interlocking system of similarities (objects that share the same parameter sector) and differences (objects use disjoint sectors of a parameter) to create objects that are all unique, but are easily sorted by an observer into multiple overlapping groups.

Summary

• Consider if you really need the full range of a certain parameter for a class of artifacts
• Divide a parameter range into sectors that:
  • are thematically appropriate for the artifacts that use them
  • are disjoint from other sectors of the same parameter range
  • allow enough variation between artifacts that use the sector
• Use those sectors to:
  • allow observers to remember features of the artifacts
  • relate artifacts to artifacts that share that sector
  • separate artifacts from artifacts that use different sectors

Equipment Explained is mostly practice for clear, accessible UI, but I still made an overly-complicated system to generate items.

The parts of an item

• base item
  • item quality
  • (optional) prefix
    • prefix quality
  • (optional) suffix
    • suffix quality
  • (optional) requirements

The base item is either armor or a weapon.

• weapon
  • type (axe, sword, hammer, etc.)
    • which stats the item boosts
  • slot (main hand, off hand)
    • where the item can be equipped
  • tier (0-3)
    • magnitude of bonuses
• armor
  • slot (feet, body, head, etc.)
    • where the item can be equipped
    • magnitude of bonuses (a breastplate provides more armor than gloves)
  • tier (cloth, leather, plate, etc)
    • which stats the item boosts (leather boosts DEX, plate boosts HP)
• item quality
  • probability of prefix
  • probability of rare prefix
  • probability of suffix
  • probability of rare suffix
  • probability of requirements
  • multiplier for magnitude of
    • item bonuses
    • prefix bonuses
    • suffix bonuses
    • requirements

Common affixes only boost a single stat, but rare affixes boost several stats

• prefix
  • • quality
      • multiplier for bonuses of prefix only
    • stat bonus
• sufffix
  • quality
    • multiplier for bonuses of suffix only
  • stat bonus

• requirements
  • lower limits for a set of thematically linked stats that must be met for an item to be equipped.

Simplifications

Because I’m not making a whole game, I was able to skip some constraints that most RPGs use.

Thematic Affixes

Some games limit certain affixes to certain types of items, such as

• “of Fleeted-footedness” only appears on boots
• there are no poison resistance rings
• helmets tend to boost INT.

The base items in Equipment Explained specialize with only a few stats, but affixes can affect any stat, and any item can have any affix. So a sword is sure to boost Slashing Damage, but sometimes cloth gloves will boost Slashing Damage even more!

Balanced Requirements

Most RPGs will give more powerful items higher requirements, and thematically appropriate ones. So a magic staff may require a lot of INT or MP to equip, while a small hammer would instead require a bit of Strength. In Equipment Explained, higher quality items are more likely to have requirements, but those requirements are chosen randomly and aren’t associated with the bonuses the item provides.

Playtesting/Balance

Equipment Explained doesn’t have combat, experience, loot drops, or an economy.  I don’t have to worry about items being fair. Is it reasonable to scale Health from a base value of 100 to 2700, or to 125? I don’t care. I do care that it’s very obvious how much those boots will increase your Health, how much you need to increase your Magic Resistance to equip those boots, and which items will help you reach that goal.

Implementation

I created an LibreOffice spreadsheet with a sheet for each type of data. The first row in each sheet is human-readable labels for each column, which the game engine has no use for, so I don’t use LibreOffice’s built-in CSV exporter, which will export every cell in a sheet.  Instead I drag-select the sheet (except for the header row) copy and paste into a text document.

So the 10 sheets of one LibreOffice spreadsheet become 10 .txt files, which I place in the resource directory of my Godot project.  I write a custom function to read each file, and save all the information about items, affixes, requirements, probabilities, etc. into data structures.    affix.txt contains information for both prefixes and suffixes, since they are so similar. The function that reads affixes.txt can also read affixes_rare.txt, which uses a similar format. Weapons are armor are defined by a two-dimensional array of type and tier, and CSV files are good for one-dimensional lists, so armor and weapons need two CSV files each to completely define the space.

When I create an item, I pick from the list of items & the list of qualities. The quality specifies the probability of requirements, prefix, and suffix, so based on those odds I may pick from those lists as well. Pretty basic stuff once all the data is loaded from the files.

Manually copying data from spreadsheet to text file is error-prone busywork, so if I continue developing this, I’ll need to automate that step.

My collection of photo galleries from events I like has been growing for years, with close to 10,000 links. But the Internet is ever-changing, and some of those links from years ago may no longer be valid. I’ve designed a system to check for this “link rot” and save users of this collection from the frustrating of clicking dead links.

Goals

• Test each link weekly
  • not working one week, maybe it’s a hiccup. Probation
  • not working for two weeks. It’s dead. Hide it.
• Don’t spam image hosts with unnecessary requests
• Minimize the amount of manual work I have to do

Implementation

The gallery aggregator is written in PHP, with a MySQL database. I plan to run an automated job each day that will check everything in the DB over the course of each week. So each day, I check every seventh gallery, and move the offset based on the day of the week.

I added two columns to the DB.

• last checked: timestamp of the last time I tried to access this URL
• last accessed: timestamp of the last time this URL returned HTTP 200 OK

I skip any gallery with a “last checked” value within the past day. Then I try to fopen() each gallery. I don’t need to download these webpages (which contain many large images) I just want to know if they exist. If the URL returns HTTP 200 OK, I add that gallery ID to a list of “fresh” galleries.  When I’ve tried to visit all the URLs, I go through the list of “fresh” galleries and set their “last accessed” time to now.  The “last accessed” time for URLs that return errors will stay the same, and time will march away from it. The ‘last checked’ time is always updated to now regardless of the HTTP response code.

Even dividing the task up across the week leaves more than 1000 URLs to check per day. I don’t want my PHP job to time out, and I don’t want to trigger DOS protections on the image hosts I’m pinging. So I paginate the job with MySQL’s LIMIT keyword. Each time I process a batch of URLs, I update the “last checked” field, which means those URLs won’t be selected in the next batch. Eventually there won’t be any URLs left to check, so running the script too many times does nothing instead of performing unnecessary DB updates or HTTP requests. I know to stop when I don’t have any more input to process.

The webpage that displays galleries to the public now takes the “last accessed” time into account. If that time is more than 2 weeks in the past, the gallery is hidden. If “last accessed” is between 1 and 2 weeks in the past, an icon and tooltip appears next to the link to that gallery:

• 🚧 This gallery might be offline.

Galleries that are “fresher” than 1 week are displayed normally.

Results

Since I give each gallery a few chances, this first week of automatic testing won’t remove any galleries from the list.  You may see some 🚧 icons appear over the next week, and in the new year, dead links will start to disappear. It’s a long-term project, but I intend to maintain this list for years, so I don’t mind waiting.

7DRL Challenge is a game jam with the goal of making a roguelike in seven days. Last year, I made Obliteration, which is fan-fiction for the movie Annihilation.

This year’s entry, NiceRL, has several goals.

• Save a lot of work and get more practice with Python by using the libtcod library.
• Flip the dungeon crawler on its head. Use the items in the dungeon to help its inhabitants, instead of killing them and haording their wealth.

I used a helpful tutorial from rogueliketutorials.com.  The code is available on GitHub, but I typed it most of it in myself, because that forced me to understand it. Do I use an underscore or a dot here? Does this function return a list or a dictionary? Those distinctions are easy to miss when copying-and-pasting, but when I type them in myself and have to solve the resulting syntax errors and crashes, I’m training myself to have good habits.

My goal of a helpful player improving the dungeon as he goes through it is inspired by Max Kreminski‘s Gardening Games Zine.  I don’t hit all the points in that zine. The player is still central because creatures can’t solve their problems alone. The space gets less interesting as t he player acts on it because once a creature is happy, there’s no more interaction available. Mechanically, using cookies to reduce a creature’s hunger to zero is identical to using arrows to reduce a monster’s HP to zero. Is changing the theme without changing the mechanics enough?

Once I had the tutorial code up and running, I started modifying it to fit my specific game. I built one emotion and one item that relieved it, then duplicated that code for two more emotions. After all that was working, I realized it was a bad idea. All the emotions behaved similarly. I collapsed them into one and added an enum to specify the emotion. A creature with with 15 units of sadness, who can be cheered by a toy is similar to a creature with 6 unit of anger, who can be soothed by a stress ball. Just make sure the type of the item matches the type of the creature. Any time I copy and paste a significant chunk of code, I have to stop and question it, because I’m probably making a mistake.

I think that systems are hard and content is easy. A flexible expressive system can handle any content one cares to throw at it, but a restrictive, clunky system will need constant tweaks and work-arounds as content exceeds its limitations. I’ve built several systems like this before, but in each case, I had a vast pool of existing content to pull in. For example, costumes at Dragon Con, or photo galleries from conventions.  Once I built the system for helping creatures with their emotional needs, I realized I needed to name all those items and all those creatures. I needed adjectives that showed escalating intensity of various emotions, and items that similarly escalated. It was much harder than I anticipated.

I had ideas for other kinds of improvements the player could make to the dungeon, but they were cut for time.  As you can see on the worksheet, I thought of lighting candles, arranging furniture, or repairing broken items. I actually wrote an AI for a table that looked to see if chairs were properly arranged around it. When the last chair was placed, the table became happy and threw food on the floor for the player to retrieve. Obviously not how dinner tables behave in real life, but it felt right to me. When the table is prepared, you can eat, not before.

Conveying information to the player is always vital, but especially when the game is doing something unexpected. In a standard dungeon crawler, a player may see something labeled “goblin” and know, from decades of games and novels and movies, that a goblin is hostile yet weak. But how should I deal with “peckish Sam” or “fuming Jaunita”?  Does a toy make someone less scared or less sad? A big part of roguelike tradition is not knowing how things work and learning by (painful or fatal) experimentation. I decided to err on the side of clarity. Not only will a person who is searching for a lost item will tell you what they are searching for, but if you find that item before meeting the person, the item itself will tell you who it belongs to.

Here’s what the final game looks like. Most of the screen shows an overhead view of the space. Below that, a scrolling message log details each thing that happens. On the right are stats about the player, the item the player is holding, and whatever the player hovers over with the mouse. You can see what each item does. I also have a README file listing all the inputs and explaining what each character on the map means.

So seven days after starting this project, I have a complete, playable game. It saves, loads, exits, ends when you are exhausted or when you finish all the levels. But you can’t play it. I mean, I can play it. It works, But you can’t, because you don’t have access to it. Despite all that I learned about Python and the libtcod library, I neglected to learn how to publish a Python program. The goal of 7DRL is to create and release a playable game in seven days, and I completely failed the “release” part. I forgot to even try.

So, was my 7DRL a success? Hard to say. I made a game, and it’s mostly the game I wanted to make. I learned more about Python and project management. But the game isn’t really fun to play. It’s not much to look at. It’s not challenging or deep. My verdict: the process was valuable, but the artifact is not. It’s good practice, like pages of an artist’s sketchbook that will never see the light of day. Not everything has to be glorious, and not everything has to have an audience. The important thing about this jam, to me, is that I keep working, keep trying, and keep improving.

Steam has a “Curator” feature that allows users to make lists of games. Many are “Games I play on my YouTube channel” or “Popular games that I hope will drive traffic to my curator page” but there are also very specialized curators that I love.

• Games With Cats In Them
• Australian-Made Games
• Lovecraft themed games
• Does it Alt-F4?

I started a few lists of my own, just for fun. Little did I know this classification project would get out of hand! (See also, Atlanta Fashion Police, convention galleries, Solstice Cyclists, basically all my classification projects get out of hand)

• Arachnophobia, games w/ spiders
• Non-Human Protagonists
• Oops! Colonialism.

I like to see all the new games that are coming out, and adding to these lists was another fun thing I could do while going through Steam’s new releases.  Before I knew it, these lists had hundreds of games each.  They are a significant collection of knowledge! But all my work is in the Steam eco-system, making Steam a more attractive place to go and to buy games.  Why should I improve someone else’s website with my labor? Especially a functional monopoly whose business practices I dislike. So I resolved to move my curation onto my own website.

Yes, I was gong to build another database, with back-end pages that partially automate creation & editing of information, as well has public-facing search and view features.  I’ve done the PHP/SQL thing several times, so for this project, I chose to learn Django and Python.

Two of my lists, and several of the other curators I like, are variations of “Games that contain this particular thing”  If my DB had a list of playable species and non-playable species, it would cover both Non-Human Protagonists and Arachnophobia, games w/ spiders, as well as Spike Covered Kittens, Elf Girl Respect Patrol, Satan Enthusiasts, You are a Ghost, THAT’S A GOOD ASS DOG, Mecha Galore, Does It Have Robots?, Dwarves of Gaming, Games Featuring Dogs, Skeletons & Videogames, Games With Cats In Them, Anthro / Furry Tag, and probably a few others. Easy win, right!  Generic data models are great!

I defined “species” rather loosely as “a group of creatures of the same type” and let species have parent-child  relationships with each other. So “humanoid” includes “human” and “elf” and “Klingon”. Since “drider” and “scorpion” both inherit from “spider-like”, they show up in the list for Arachnophobia, but I can also offer more precise lists for only scorpions or only driders.

But the presence of a particular type of creature isn’t the only relevant information. Some arachnophobes hate cobwebs, others are OK with spiders as long as the spiders don’t leap at them.  “Cobweb” isn’t a species, and “leap” is a capability, not a sub-species. Likewise, people who care about dogs in game might want to know if they can pet the dogs, or if the dogs do tricks. They don’t care if they can pet the spiders, and dogs never shoot webs, so adding “is pettable” or “shoots webs” as attributes of a species doesn’t make sense.  Any species may have extra attributes that only apply to it.  I could easily represent this with object-oriented subclasses, but databases don’t work that way.

Look how hard it is to answer a simple question: “Can I play as a skeleton?”

1. In Grim Fandango, you control one character: Manny Calavera, a skeleton. Yes, always.
2. In DOTA 2, you pick one hero from a large roster to control. You may choose Skeleton King, or a non-skeleton hero.  Yes, sometimes.
3. In Din’s Curse, you control one human hero. You can choose from a list, or create your own by combining parts of pre-made heros.  If you have the Necromancer skill tree from the Conjurer class, you can summon skeletons which fight for you. You order them to move and attack, but your human hero is more important. Yes, sometimes, indirectly.
4. In WarCraft III, you choose one of several armies to control.  The Undead Necromancers can raise Skeletons from corpses. Heroes who find a certain item can also summon skeletons. Your control of these skeletons is as detailed as your control over any other unit. There is no main character.  Yes, sometimes, as much as anything. But, wait! During the singleplayer game, you guide various heroes through story missions. These heroes are the main characters, so they are more important than any skeletons who may or may not join your army. The mechanical controls do not change, only the context provided to those mechanics by the story. Not really, I’m Arthas right now.
5. In Dragon Age: Inquisition, the player can directly control any character in combat, but only the Inquisitor’s appearance and personality can be controlled by the player. The player IS the Inquisitor. The other characters are their own people.

So we can get more useful answers by replacing the question “Can you play as Character X?” with the following questions:

• Can you control Character X?
  • Always
  • Never
  • Sometimes. Arthas from WarCraft III is controllable in some story missions and is an NPC antagonist in others
  • Optional. Players can add Polar Bears to their armies in King’s Bounty, or not.
  • Partially. Cassandra from Dragon Age: Inquisition can be controlled in combat but not in conversation
  • Indirectly. Players have basic control over Fallout: New Vegas‘s Boone in combat, but none in conversation. Control over the Courier is more direct.
• Can you identify with Character X?”
  • Always
  • Never
  • Sometimes Commander Shepard is the player avatar in Mass Effect 3‘s singleplayer, but the player has a separate set of characters in multiplayer.)
  • Optional. Players can choose to be Cammy in Street Fighter V or some other character.

According to League of Legends lore, players always and only play Summoners, human magicians who in turn control Champions. But that’s not how people experience League of Legends.  People say, “I’m going to play Miss Fortune” or “I’m an Urgot main”. They identify with and directly control a Champion. On the other hand, Princess Maker Refine, has a similar invisible protagonist, but players don’t identify with the princess because ordering a girl around is part of the fantasy.

So simple questions like, “Is there spider stuff in this game?” and “Can I play as a robot?” can be answered in a short English sentence, but creating data structures to codify all the shades of meaning contained in that sentence is very difficult.