Connect-4-RL

This repository started as a side project that grew out of a joint project at TAVE, an inter-university IT association. It is a small Pygame desktop app (540x960) where you play Connect-4 against reinforcement-learning AIs at three difficulty levels. All inference runs through ONNX Runtime; the original PyTorch training code lives under before_onnx/.

Current version: 1.0.2 (see ver in game.py).

For the full development write-up (project process, training notes, etc.), see the Notion page: pygame project notes.

Quick start (Windows release)

1. Download the latest zip from the Releases tab on GitHub and extract it. Run the ConnectX_*.exe file inside.
2. If Microsoft Defender SmartScreen blocks the launch with the "Windows의 PC 보호" / "Windows protected your PC" dialog, click 추가 정보 (More info) and then 실행 (Run anyway).

Screenshots

Main menu

intro() in game.py provides:

• new game - pick easy / normal / hard and play
• continue - resume from files/continue.pkl
• how to - in-app tutorial
• review - replay the last finished game from files/review.pkl
• info - app info
• setting - first player, piece colors, music / SFX volumes

AI difficulties

All three difficulties run through ONNX Runtime in test_model.py. easy and normal actually share the same backbone weights; only the inference strategy differs.

• easy - Minimax Double-DQN (ResNet) trained with self-play. The 7x7 Q-output is read as a (my move, opponent move) payoff and picked with a one-step minimax.
• normal - Same DDQN as easy, but used as the prior for an AlphaGo-style MCTS (200 sims). A small auxiliary value network (value_model.onnx) contributes 10% to leaf evaluations.
• hard - AlphaZero ResNet (joint policy + value head) trained via iterative self-play, driving an AlphaZero MCTS (100 sims).

PyTorch model definitions live in before_onnx/models.py and before_onnx/pt2onnx.py re-exports them to ONNX.

Project layout

• game.py - Pygame entry point, menus, play loop, animations
• classes.py - Button, color palette, sound assets, easter-egg ee_boards
• functions.py - save / load (setting, record, continue, review), board rendering helpers, win check
• env.py - ConnectFour game logic, MCTS nodes, MCTS, MCTS_alphago
• test_model.py - difficulty-aware ONNX inference entry point (test_main)
• before_onnx/ - PyTorch model definitions and ONNX export script
• files/ - models (.onnx), sounds, images, fonts, saved data (.pkl)

Save data

These files in files/ are created and updated automatically as you play:

• continue.pkl - in-progress game for continue
• review.pkl - last finished game for review
• record.pkl - per-difficulty win / draw / loss totals
• setting.pkl / default_setting.pkl - user settings

Asset credits

Sound and icon source URLs are listed inline in classes.py and in files/sources.txt.