Author: @Blake Arnold

Last Modified: September 4, 2026

Context

I recently participated in the generals.bot competition hosted by Equilibre Technologies and UFAL. I’ve been focused on acquiring new skills as I transition to a career in AI research, and I’d been looking for a good project to sharpen my deep reinforcement learning skills. I received an announcement email about the competition a few days before it started, and it looked like a great opportunity. I’ve always loved partial information strategy games, so building an agent that could play one at a high level sounded like a fun challenge.

My agent (bca) placed 3rd out of 116 in the 1-week sprint phase of the challenge, which was a stronger showing than I expected given I had limited background in multi-agent reinforcement learning and the competition was stiff: the creator of the top-ranked agent also has a bot ranked as the number 1 player worldwide on generals.io.

My efforts were funded by a little over $1,000 in compute credits from a neocloud provider, given to me in exchange for testing their new product and providing feedback. I decided not to invest additional resources in the longer “marathon” phase of the competition, since my primary motivation was learning and I felt like I’d gotten what I wanted to out of the project. As other bots improved, mine slipped to 5th out of 157.

Overview

• 🃏 Game set-up: The game was a modified version of generals.io, a two-player partial information strategy game played on a grid under fog-of-war. (details)
• 🌇 Architecture: I trained a vision transformer (ViT) model with an early convolutional residual branch. This built on the architecture for the SoTA generals.io bot recently released by Straka and Schmid (pure ViT), but used convolutions to help the model “see” better, inspired by an earlier image classification paper from Xiao et al. (details)
• 🥘 Training recipe: I trained the model primarily using self-play PPO, sparse terminal rewards, and top-25% advantage filtering. When this ran into a wall, I mixed in some auxiliary supervised learning in order to help the actor and critic to understand when to build castles. (details)
• 📏 Policy evaluation: ****The policy’s performance improved quickly during early training and after it discovered and consolidated strong castle-building actions. Performance hadn’t saturated when training was stopped, but the rate of improvement had slowed considerably. (details)
• ⏩ Attempting to Accelerate Training: I tried using Cai et al.’s AlphaRL framework (designed for language models) to improve policy performance without additional RL. This was unsuccessful. (details)

Game Set-Up

Generals.io

Generals.io gameplay from the point of view of a spectator with full information.

Generals.io gameplay from the point of view of the blue player.

The competition was based on a modified version of 1v1 generals.io. The game is played on a grid with side lengths up to 23 consisting of four types of cells: plains, generals, castles, and mountains (impassable obstacles). Each player starts out controlling a single general cell with one army unit, and the objective of the game is to capture the opponent’s general which is placed at least 15 breadth-first search (BFS) steps away in an unknown location. The game occurs in simultaneous turns, where players can move armies into neutral adjacent plains cells so long as they have more than one unit in the source cell, sending either half of their troops or all-but-one. When a player attempts to move troops into an occupied cell, combat occurs through simple subtraction: the larger army occupies the cell with a troop count equal to the difference between the two army sizes.

Each turn, the general cell gains one army unit and every 25 turns every occupied cell gains one unit, creating an incentive to expand territory. In the standard game, players can also capture neutral castle cells at the cost of 40-50 troops, which produce one army unit per turn thereafter. Players therefore face a resource allocation challenge, having to decide whether to invest in productive assets that will yield larger armies in the future at the cost of becoming more vulnerable to attacks in the short-term. They also need to pay attention to tempo, as acquiring land just before the 25-turn troop expansion yields disproportionate benefits.

The game is played under fog-of-war. Each player can only see the contents of cells that they control and the cells immediately adjacent to them, in addition to global enemy army and land counts. This requires players to explore the map, form inferences about the location of enemy generals, castles, and armies, identify and position themselves relative to potential chokepoints, and disguise their own general’s location. It also creates opportunities for players to hide troop movement and feint attacks.

Modifications

The hosts of the competition introduced several modifications so that users with existing generals.io bots couldn’t simply port them over wholesale. Listed in order of importance: