a build story · September 2025 to today
Someone to play with
How I taught an AI to play Connect 4, and then gave it eyes and an arm so it could sit across the table from me.
prologue
Why build this at all
I love my wife, and she loves me. I also love board games. My wife does not love board games.
I could play against an app, but that is not the same thing. I wanted to sit at a table and play against something real: an opponent that looks at the board and moves the pieces itself.
the plan
Three problems, one opponent
I started in software, where failing is cheap: learn how reinforcement learning can master a simple game. Then move that brain into the physical world and give it what it is missing.
The brain
Which move should I play?
Trained in simulation, over a million games against itself and others.
reinforcement learning
The eyes
What is on the board, and where?
A camera on the wrist reads the board, the tokens and where they sit on the table.
computer vision
The arm
How do I put a piece exactly there?
The arm turns "that square" into the joint angles that reach it.
inverse kinematics
act I
The brain
teaching software to play Connect 4
choosing the game
A game small enough to learn
Connect 4 has 7 columns, 6 rows and one decision per turn: which column. The rules fit on a napkin, but there are about 4.5 trillion legal positions. Nothing can memorise them all, so an AI has to learn to generalise.
Two simple opponents set the bar. Random drops a token in any legal column. Rules wins if it can, blocks if it must, builds threes and prefers the centre. Rules beats Random in about 99% of games.
reinforcement learning
Learning by playing
Nobody tells the agent what a good move is. It plays a game, gets a reward at the end (+10 for a win, −10 for a loss), and slowly works out which moves led there.
What it learns is a Q-value for every move: how much reward it expects if it plays that column now. Most of the time it plays the highest one. Sometimes it tries a random column, just to explore.
deep Q-network
From a lookup table to a neural network
My first agent kept a table: one score for every board it had seen. After training it held 187,000 entries and still knew nothing about a board it had never met.
A Deep Q-Network swaps the table for a neural network. The board goes in, seven scores come out, and similar boards get similar scores, so what it learns carries over.
Both agents learned to beat random play, because that is what they trained against. An agent only gets as good as its sparring partners.
convolutional network
A network that sees lines
Four in a row is a visual pattern, so the next network looks at the board the way you do. A convolutional network slides small filters across the grid and lights up when it finds lines, threats and blocks.
- input
- two 6 × 7 layers: my tokens, your tokens
- patterns
- 6 convolution layers that learn lines in every direction
- two heads
- how good is this board, and how much better is each column
- size
- about 0.5 million parameters, trained on a MacBook's M1 GPU
what went wrong in training
Training fought back
Getting from "plays" to "plays well" took most of the time. Four problems kept coming back.
The numbers ran away
Each estimate fed the next until Q-values blew up. Fixed with a 0.95 discount, clamping to ±100 and clipping the gradients.
New opponent, old skills gone
Switching sparring partners erased what it knew. Fixed with a curriculum that overlaps phases by 4,000 games and a memory that keeps old games.
Playing a stale copy of itself
Its "self" was a snapshot from 500 games ago. Fixed with a league of 5 past versions: win 60% of games to move up a tier.
Win or lose says too little
One reward per game hides which move mattered. Fixed with small signals: +2 for a block, −3 for a missed block, −5 for a missed win.
the tournament
A team beats any single player
After more than a million training games, I put the saved versions in a round robin: 3,500 games each. The winner was a team: the top four networks voting on every move.
On screen, the AI was now a real opponent. But it lived on a screen, and I wanted it across the table.
act II
The eyes
giving the brain a body, and a way to see
from software to hardware
Give the brain a body
The body is an SO-101, an open-source robot arm with six servo motors that I drive from Python with Hugging Face's lerobot library.
It has no eyes, so a small USB camera sits on the wrist. Because the camera moves with the hand, it always sees where the hand is about to go.
The board lies flat on the table instead of standing upright. A flat grid is also what checkers and chess look like, so the same setup can grow beyond Connect 4. A move means: go to the token supply, grab a token, carry it, drop it in the chosen cell.
computer vision
How the robot reads the board
Before the AI can think, the camera has to turn a photo into a grid of numbers. It does that in four steps.
where things are
From pixels to millimetres
"Purple in D3" is enough for the AI. The arm needs to know where D3 is on the table, in millimetres.
A flat table seen at an angle is a stretched rectangle, and a homography undoes the stretch. Four known points are enough to compute it. They come from ArUco markers: printed squares the camera finds and identifies on its own.
One more fix: the wide-angle lens bends straight lines, up to about 7 mm off. A one-time lens calibration straightens the image first.
act III
The arm
from "that square" to joint angles
the first approach
First attempt: teach every cell
My first approach was hands-on. I moved the arm over each cell and saved its joint angles: 40 poses in total. For a spot in between, the software blended the 8 nearest saved poses.
It worked near the taught points and broke everywhere else. Moving the board 5 mm made every pose wrong. Averaging joint angles is not the same as averaging positions, so the blends drifted. And a new game would mean teaching every square again: 64 for chess.
the key idea
Forward and inverse kinematics
Forward kinematics answers an easy question: given every joint angle, where is the fingertip? Chain the links together: turn, move along the link, turn again. There is exactly one answer.
Inverse kinematics asks the reverse: given where the fingertip must go, which angles get it there? There may be several answers, or none.
Think of reaching for a cup. You don't plan "shoulder 30°, elbow 70°". You look at the cup and your arm works it out. IK gives the robot that skill.
how the solver works
Solving it: guess, check, adjust
- Guess a set of angles. The solver starts from three typical reaching shapes.
- Check with forward kinematics how far the fingertip lands from the target.
- Adjust the angles in whichever direction shrinks that distance, and repeat until the error is under 1 mm.
It also has a preference: keep the gripper pointing straight down, and lean outward in 10° steps, up to 50°, only when a cell is too far to reach vertically.
the payoff
Why this generalises
Seen from above, a board game is just a grid of points on a table. IK can reach any point within the arm's range, so every game turns into a list of coordinates.
| teach by hand | camera + IK | |
|---|---|---|
| setup | teach every cell | calibrate once with a printed mat |
| board moved | teach everything again | the camera finds it again |
| new game | teach every square again | describe the new grid |
| between cells | a blend of nearby poses | an exact solution for any point |
calibration
Making the maths match the metal
IK is only as good as its model of the arm, and a real arm never matches the drawing exactly. Joints sit a few degrees off, and the base is not quite where you think.
So I printed a calibration mat with 15 crosshairs to touch and 4 ArUco markers for the camera. Then I fitted 10 numbers to the touches: joint offsets, the base's position and rotation, and the fingertip offset.
2.3 mm is well inside a cell: good enough to drop a token where it belongs.
the detours
What I tripped over
Most of these looked like an arm problem. Most were measurement problems.
Not needed
±6 cm accuracy and a 20 cm blind zone. The table is flat, so the height is already known.
Degrees vs −100…100
Scripts spoke different units and loaded different calibration files, up to ~9° apart.
100 mm became 97.5
The mat printed slightly small. One scale factor fixed it.
They stop ~1.5° short
Under load. Now the arm reads its encoders, nudges, and repeats until they match.
Taught at a tilt
Taught leaning, used vertical: the model guessed 5 mm too low. Teach the pose you will use.
Fisheye bends lines
Predicted crosses drifted ~7 mm. A one-time lens calibration straightens the image.
epilogue
Putting it together
one move, from photo to token
One move: see, think, act
Every game, the camera finds the board, reads the tokens and works out where each cell sits on the table. The arm and the camera are calibrated against the table once. After that, the board can sit anywhere the arm reaches.
status
Where the story stands
The three parts work on their own. What's left is joining them into one full game across the table.
- doneThe brain: networks trained over 1M+ games
- doneThe eyes: board and tokens read from the camera
- doneThe arm: calibrated model, 2.3 mm on unseen points
- doneClick a pixel, and the arm goes there
- nowLens calibration on the real camera
- nextFind the board's corners, then every cell in mm
- thenA full game of Connect 4 across the table
- laterCheckers and chess: same arm, new grid
A brain from software.
Eyes from a camera.
Reach from geometry.
Comments
Sign in with GitHub to leave a comment or react.