QuantPits

An advanced, production-ready quantitative trading system built on top of Microsoft Qlib. This system provides a complete end-to-end pipeline for weekly and daily frequency trading, featuring modular architecture, multi-instance isolation (Workspaces), ensemble modeling, execution analytics, and interactive dashboards.

🌐 中文版本 (README_zh.md)

Note: We welcome contributions! If you find a bug or have a feature suggestion, please feel free to open an Issue or submit a Pull Request.

🚀 Key Features

• Multi-Workspace Isolation: Spin up independent "Pits" for different markets (e.g., CSI300, CSI500) or configurations without duplicating code.
• Component-Based Pipeline:
  • Train & Predict: Support for both full and incremental training on multiple models (LSTM, GRU, Transformers, LightGBM, GATs).
  • Brute Force & Ensemble: High-performance (CuPy accelerated) brute force combination finding and intelligent signal fusion.
  • Orders & Execution: Generate actionable buy/sell signals with TopK/DropN logic and analyze micro-friction (slippage, delay costs).
  • Extensible Broker Adapters: Decoupled settlement parser supporting arbitrary broker terminal formats (e.g., Guotai Junan).
• Rich Observability: Two interactive streamlit dashboards for macro portfolio performance and micro rolling health monitoring.
• Resilient Infrastructure: Automatic checkpoints, JSON tracking for model registries, and daily/weekly logs.

📂 Architecture Overview

The system strictly decouples the Engine (Code) from the Workspace (Config & Data):

QuantPits/
├── docs/                   # Detailed system manuals (00-08)
├── ui/                     # Streamlit interactive dashboards
│   ├── dashboard.py        # Macro performance app
│   └── rolling_dashboard.py# Temporal strategy health app
├── quantpits/              # Core logic engine and components
│   └── scripts/            # Pipeline execution scripts
│
└── workspaces/             # Isolated trading instances
    └── Demo_Workspace/     # Example configured instance
        ├── config/         # Trading bounds, model registry, cashflow
        ├── data/           # Order logs, holding logs, daily amount
        ├── output/         # Model predictions, fusion results, reports
        ├── mlruns/         # MLflow tracking logs
        └── run_env.sh      # Environment activation script

🛠️ Quick Start

1. Requirements

Ensure you have a working installation of Qlib. The engine officially supports Python 3.8 to 3.12. Then install the extra dependencies:

pip install -r requirements.txt
# (Optional) Install the engine as a package for global access:
pip install -e .

(Note: For GPU-accelerated brute force combinatorial backtesting, install cupy-cuda11x or cupy-cuda12x depending on your local CUDA version)

2. Prepare Market Data

Before running the engine, ensure you have the required Qlib dataset downloaded to your local machine (e.g., ~/.qlib/qlib_data/cn_data):

# Example: Download 1D data for the Chinese market
python -m qlib.run.get_data qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn --version v2

Note: This dataset contains massive historical market data. The initial download may require tens of GBs of disk space and a considerable amount of time. Please be patient. For daily frequency data, you can also reference this repo: https://github.com/chenditc/investment_data

By default, all scripts read the data from ~/.qlib/qlib_data/cn_data. To override this on a per-workspace basis, uncomment and modify the relevant lines in run_env.sh:

export QLIB_DATA_DIR="/path/to/your/qlib_data"
export QLIB_REGION="cn"   # or "us"

3. Activate a Workspace

Every action must be performed within the context of an active workspace. We provide a Demo_Workspace to get you started:

cd QuantPits/
source workspaces/Demo_Workspace/run_env.sh

4. Run the Pipeline

Once activated, you can execute the minimal routine loop using the quantpits scripts (or you can simply run make run-daily-pipeline from the repository root):

# 0. Update Daily Market Data
# Note: This engine assumes underlying Qlib data has been updated (e.g., via external Cron). 
# If not, update it first.

# 1. Train models (Required for first-time setup or retraining)
python -m quantpits.scripts.prod_train_predict

# 2. Generate predictions from existing models
python -m quantpits.scripts.prod_predict_only --all-enabled

# 3. Fuse predictions using your combo configs
python -m quantpits.scripts.ensemble_fusion --from-config-all

# 4. Process previous week's live trades (Post-Trade)
python -m quantpits.scripts.prod_post_trade

# 5. Generate new Buy/Sell orders based on current holdings
python -m quantpits.scripts.order_gen

5. Launch Dashboards

To view the interactive analytics of your active workspace:

# Portfolio Execution and Holding Dashboard
streamlit run ui/dashboard.py

# Rolling Strategy Health & Factor Drift Dashboard
streamlit run ui/rolling_dashboard.py

🏗️ Creating a New Workspace

To spin up a new strategy for a different index (e.g., CSI 500), use the scaffolding utility:

python -m quantpits.scripts.init_workspace \
  --source workspaces/Demo_Workspace \
  --target workspaces/CSI500_Base

This will cleanly clone the configuration files and generate empty data/, output/, and mlruns/ directories, completely isolated from your other trading environments. You then simply source workspaces/CSI500_Base/run_env.sh to start operating in it.

📖 Documentation

For a deep dive into each module, refer to the documentation in docs/:

• 70_WALKTHROUGH.md (End-to-End Walkthrough — Start Here!)
• 00_SYSTEM_OVERVIEW.md (System Architecture & Workflows)
• 01_TRAINING_GUIDE.md
• 02_BRUTE_FORCE_GUIDE.md
• 03_ENSEMBLE_FUSION_GUIDE.md
• ...and more.

All documentation is available in both Chinese and English (docs/en/).

📜 License

MIT License.