test: add integration and functional test suite

.github/copilot-instructions.md

@@ -1,5 +1,102 @@
 # GitHub Copilot Instructions for AIMBAT
 
+## Build, Test, and Lint
+
+Dependencies are managed with **uv**. All commands assume the virtualenv is active or are prefixed with `uv run`.
+
+```bash
+# Install all dependencies
+make sync                          # uv sync --locked --all-extras
+
+# Format and lint
+make format                        # black .
+make lint                          # black --check + ruff check .
+uv run ruff check --fix .          # auto-fix ruff issues
+
+# Type checking
+make mypy                          # uv run pytest --mypy -m mypy src tests
+
+# Run all tests (includes mypy + matplotlib comparison)
+make tests                         # pytest --cov --mpl + mypy
+
+# Run a single test file or test
+uv run pytest tests/unit/test_foo.py
+uv run pytest tests/unit/test_foo.py::test_specific_function
+
+# Regenerate matplotlib baseline images (then manually move to test directories)
+make test-figs
+```
+
+Configuration: `pyproject.toml` (pytest, mypy, black, ruff, coverage). Tests run against Python 3.12–3.14 in CI via tox.
+
+## Architecture
+
+AIMBAT is a seismological tool for automated and interactive measurement of body-wave arrival times. It processes SAC-format seismograms and stores state in a SQLite database.
+
+### Module Layout
+
+```
+src/aimbat/
+├── app.py           # Cyclopts CLI root — registers all subcommands
+├── cli/             # CLI command definitions (thin layer, delegates to core/)
+├── core/            # Business logic: ICCS/MCCC algorithms, event/seismogram ops
+│   ├── _active_event.py  # Manages the single active event constraint
+│   ├── _data.py          # SAC ingestion entry point
+│   ├── _iccs.py          # ICCS alignment (wraps pysmo.tools.iccs)
+│   └── _snapshot.py      # Parameter state capture for rollback/comparison
+├── models/          # SQLModel ORM definitions (Events, Seismograms, Stations, etc.)
+│   └── _sqlalchemy.py    # SAPandasTimestamp / SAPandasTimedelta type decorators
+├── aimbat_types/    # Custom Pydantic types (PydanticTimestamp, enums for parameters)
+├── io/              # File I/O — _base.py defines abstract base; _sac.py implements SAC via pysmo
+├── utils/           # Shared helpers (JSON→table, UUID truncation, styling, sample data)
+├── _config.py       # Global Settings (pydantic-settings, env prefix AIMBAT_)
+├── _lib/            # Internal mixins (EventParametersValidatorMixin)
+├── _utils.py        # Top-level utility helpers
+├── db.py            # SQLite engine singleton (foreign keys enforced via PRAGMA)
+└── logger.py        # Loguru-based logging
+```
+
+### Data Flow
+
+1. SAC files are ingested via `aimbat data add` → `core/_data.py` → `io/` → stored in SQLite
+2. One event is set "active" at a time; all processing commands operate on the active event
+3. ICCS (Iterative Cross-Correlation and Stack) aligns seismograms: `core/_iccs.py` wraps `pysmo.tools.iccs`
+4. MCCC (Multi-Channel Cross-Correlation) refines arrival time picks: wraps `pysmo.tools.signal.mccc`
+5. Snapshots (`core/_snapshot.py`) capture parameter state for rollback/comparison
+
+### Key Models
+
+- **AimbatEvent** — seismic event with `active` flag (only one active at a time, enforced by DB trigger)
+- **AimbatSeismogram** — links to AimbatEvent + AimbatStation; stores `t0` (initial pick) and processing parameters
+- **AimbatEventParameters** — per-event processing settings (window, bandpass, min_ccnorm)
+- **AimbatSeismogramParameters** — per-seismogram flags (`select`, `flip`, `t1` pick)
+- **SAPandasTimestamp / SAPandasTimedelta** in `models/_sqlalchemy.py` — custom SQLAlchemy type decorators storing pandas timestamps as UTC datetimes and timedeltas as nanosecond integers
+
+### Configuration
+
+Settings live in `_config.py` as a `pydantic-settings` class. All settings can be overridden via environment variables prefixed with `AIMBAT_` (e.g. `AIMBAT_LOG_LEVEL=DEBUG`) or a `.env` file. The default project file is `aimbat.db` in the current directory.
+
+## Key Conventions
+
+### Testing
+
+- **Each test gets a fresh in-memory SQLite database** via the `engine` fixture in `tests/conftest.py`; never share state between tests
+- **UUID generation is seeded** (`random.Random(42)`) in tests via `mock_uuid4` autouse fixture — do not rely on random UUIDs in assertions
+- **`patch_settings` fixture** resets all settings to defaults before each test; use `@pytest.mark.parametrize` with `indirect=["patch_settings"]` to override specific settings
+- Test assets (SAC files) live in `tests/assets/`; use `tmp_path_factory` copies to avoid mutating them
+- Mirror `src/aimbat/` directory structure under `tests/` (e.g. `tests/unit/core/`, `tests/unit/models/`)
+- Matplotlib comparison tests use `--mpl` flag; baseline images live in `baseline/`
+
+### CLI Pattern
+
+Each CLI module in `cli/` creates a Cyclopts `App` instance and registers it with the root app in `app.py`. CLI functions are thin wrappers that open a `Session` from `aimbat.db.engine` and delegate to `core/` functions.
+
+### Custom Types
+
+- Use `PydanticTimestamp` / `PydanticTimedelta` (from `aimbat.aimbat_types`) for pandas-compatible time fields in models
+- Use `PydanticNegativeTimedelta` / `PydanticPositiveTimedelta` for constrained sign validation
+- Use `SAPandasTimestamp` / `SAPandasTimedelta` (from `aimbat.models._sqlalchemy`) as the `sa_type` in SQLModel fields
+
 ## Code Style and Standards
 
 ### General Principles

.gitignore

@@ -34,3 +34,6 @@ reset_project.sh
 aimbat.log
 .env
 aimbat_test.log
+GEMINI.md
+CLAUDE.md
+.claude/settings.local.json

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help check-uv sync upgrade lint test-figs tests \
+.PHONY: help check-uv sync upgrade lint test-figs tests tests-full \
 	mypy docs live-docs build publish clean python \
 	format format-check changelog
 
@@ -36,7 +36,10 @@ lint: check-uv ## Check formatting with black and lint code with ruff.
 test-figs: check-uv ## Generate baseline figures for testing (then manually move them to the test directories).
 	uv run py.test --mpl-generate-path=baseline
 
-tests: check-uv mypy ## Run all tests with pytest.
+tests: check-uv mypy ## Run tests with pytest (excludes slow functional tests).
+	uv run pytest --cov --cov-report=term-missing --cov-report=html --mpl -m "not slow"
+
+tests-full: check-uv mypy ## Run all tests including slow functional tests.
 	uv run pytest --cov --cov-report=term-missing --cov-report=html --mpl
 
 mypy: check-uv ## Run typing tests with pytest.

flake.nix

@@ -23,13 +23,14 @@
     in {
       default = pkgs.mkShell {
         nativeBuildInputs = with pkgs; [
+          bashInteractive
+          sqlitebrowser
           uv
           ruff
           (python314.withPackages (ps: with ps; [tox]))
           python313
           python312
           gnumake
-          sqlitebrowser
         ];
 
         shellHook = ''

pyproject.toml

@@ -77,14 +77,20 @@ requires = ["hatchling", "hatch-vcs"]
 build-backend = "hatchling.build"
 
 [tool.pytest.ini_options]
-# xvfb_width = 1920
-# xvfb_height = 1080
 testpaths = [
   "tests",
   "src",
 ]
+markers = [
+    "slow: mark slow tests that may take a long time to run",
+    "cli: mark as command-line interface tests",
+    "gui: mark tests that require a GUI environment",
+    "mpl: mark tests that generate matplotlib figures",
+]
 mpl-generate-summary = "html"
 mpl-use-full-test-name = true
+# xvfb_width = 1920
+# xvfb_height = 1080
 
 [tool.mypy]
 mypy_path = "src"

src/aimbat/_config.py

@@ -133,36 +133,55 @@ def set_computed_defaults(self) -> Self:
 
 def print_settings_table(pretty: bool) -> None:
     """Print a pretty table with AIMBAT configuration options."""
-    from aimbat.utils import make_table, TABLE_STYLING
-    from rich.console import Console
+    import json
+    from aimbat.utils import TABLE_STYLING
+    from aimbat.utils._json import json_to_table
 
     env_prefix = Settings.model_config.get("env_prefix")
+    values: dict[str, str] = json.loads(settings.model_dump_json())
 
     if not pretty:
-        for k in Settings.model_fields:
-            print(
-                f'{(env_prefix + k).upper() if env_prefix else k}="{getattr(settings, k)}"'
-            )
+        for k, v in values.items():
+            env_key = f"{env_prefix.upper()}{k.upper()}" if env_prefix else k
+            print(f'{env_key}="{v}"')
         return
 
-    table = make_table(title="AIMBAT settings")
-    table.add_column("Name", justify="left", style=TABLE_STYLING.id, no_wrap=True)
-    table.add_column("Value", justify="center", style=TABLE_STYLING.mine)
-    table.add_column("Description", justify="left", style=TABLE_STYLING.linked)
-
-    for k, v in Settings.model_fields.items():
+    rows = []
+    for k, v in values.items():
+        field_info = Settings.model_fields.get(k)
         env_var = (
-            f"Environment variable: {env_prefix.upper()}{str(k).upper()}"
+            f"Environment variable: {env_prefix.upper()}{k.upper()}"
             if env_prefix
             else ""
         )
-        description_with_env_var = (
-            f"{v.description} " if v.description else ""
-        ) + env_var
-        table.add_row(k, str(getattr(settings, k)), description_with_env_var)
+        description = field_info.description if field_info else ""
+        description_with_env_var = (f"{description} " if description else "") + env_var
+        rows.append(
+            {"name": k, "value": str(v), "description": description_with_env_var}
+        )
 
-    console = Console()
-    console.print(table)
+    json_to_table(
+        rows,
+        title="AIMBAT settings",
+        column_kwargs={
+            "name": {
+                "header": "Name",
+                "justify": "left",
+                "style": TABLE_STYLING.id,
+                "no_wrap": True,
+            },
+            "value": {
+                "header": "Value",
+                "justify": "center",
+                "style": TABLE_STYLING.mine,
+            },
+            "description": {
+                "header": "Description",
+                "justify": "left",
+                "style": TABLE_STYLING.linked,
+            },
+        },
+    )
 
 
 def cli_settings_list(

src/aimbat/aimbat_types/_pydantic.py

@@ -11,7 +11,9 @@
 ]
 
 
-def _format_timedelta(td: Timedelta) -> float:
+def _format_timedelta(td: Timedelta | None) -> float | None:
+    if td is None:
+        return None
     return td.total_seconds()
 
 
@@ -40,6 +42,8 @@ def __get_pydantic_core_schema__(
     ) -> CoreSchema:
         # Define how to validate the input (from string, datetime, or object)
         def validate(value: Any) -> T:
+            if value is None:
+                raise ValueError(f"{cls.target_type.__name__} value cannot be None")
             if isinstance(value, cls.target_type):
                 return value
             try:
@@ -63,7 +67,7 @@ class _AnnotatedTimedelta(_PandasBaseAnnotation):
 type PydanticTimedelta = Annotated[
     Timedelta,
     _AnnotatedTimedelta,
-    PlainSerializer(_format_timedelta, return_type=float),
+    PlainSerializer(_format_timedelta, return_type=float | None),
 ]
 type PydanticNegativeTimedelta = Annotated[
     PydanticTimedelta, AfterValidator(_must_be_negative_pd_timedelta)

src/aimbat/app.py

@@ -6,19 +6,6 @@
 commands is available by typing `aimbat COMMAND --help`.
 """
 
-from ._config import cli_settings_list
-from .cli import (
-    _align,
-    _data,
-    _event,
-    _pick,
-    _plot,
-    _project,
-    _station,
-    _seismogram,
-    _snapshot,
-    _utils,
-)
 from importlib import metadata
 from cyclopts import App
 from rich.console import Console
@@ -32,17 +19,16 @@
 console = Console()
 
 app = App(version=__version__, help=__doc__, help_format="markdown", console=console)
-app.command(_align.app)
-app.command(_data.app)
-app.command(_event.app)
-app.command(_pick.app)
-app.command(_plot.app)
-app.command(_project.app)
-app.command(_station.app)
-app.command(_seismogram.app)
-app.command(cli_settings_list, name="settings")
-app.command(_snapshot.app)
-app.command(_utils.app)
+app.command("aimbat.cli._align:app", name="align")
+app.command("aimbat.cli._data:app", name="data")
+app.command("aimbat.cli._event:app", name="event")
+app.command("aimbat.cli._pick:app", name="pick")
+app.command("aimbat.cli._plot:app", name="plot")
+app.command("aimbat.cli._project:app", name="project")
+app.command("aimbat.cli._station:app", name="station")
+app.command("aimbat.cli._seismogram:app", name="seismogram")
+app.command("aimbat.cli._snapshot:app", name="snapshot")
+app.command("aimbat.cli._utils:app", name="utils")
 
 
 if __name__ == "__main__":

src/aimbat/cli/_data.py

@@ -13,36 +13,41 @@
 @app.command(name="add")
 @simple_exception
 def cli_data_add(
-    seismogram_files: Annotated[
+    data_sources: Annotated[
         list[Path],
         Parameter(
-            name="files", consume_multiple=True, validator=validators.Path(exists=True)
+            name="sources",
+            consume_multiple=True,
+            validator=validators.Path(exists=True),
         ),
     ],
     *,
-    filetype: DataType = DataType.SAC,
+    data_type: Annotated[DataType, Parameter(name="type")] = DataType.SAC,
+    dry_run: Annotated[bool, Parameter(name="dry-run")] = False,
     show_progress_bar: Annotated[bool, Parameter(name="progress")] = True,
     global_parameters: GlobalParameters | None = None,
 ) -> None:
     """Add or update data files in the AIMBAT project.
 
     Args:
-        seismogram_files: Seismogram files to be added.
-        filetype: Specify type of seismogram file.
+        data_sources: Data sources to be added.
+        data_type: Specify type of seismogram file.
+        dry_run: If True, print the files that would be added without modifying the database.
         show_progress_bar: Display progress bar.
     """
     from aimbat.db import engine
-    from aimbat.core import add_files_to_project
+    from aimbat.core import add_data_to_project
 
     global_parameters = global_parameters or GlobalParameters()
 
     disable_progress_bar = not show_progress_bar
 
     with Session(engine) as session:
-        add_files_to_project(
+        add_data_to_project(
             session,
-            seismogram_files,
-            filetype,
+            data_sources,
+            data_type,
+            dry_run,
             disable_progress_bar,
         )
 

src/aimbat/cli/_snapshot.py

@@ -92,14 +92,14 @@ def cli_snapshot_dump(
         all_events: Select snapshots for all events.
     """
     from aimbat.db import engine
-    from aimbat.core import dump_snapshot_table_to_json
+    from aimbat.core import dump_snapshot_tables_to_json
     from sqlmodel import Session
     from rich import print_json
 
     global_parameters = global_parameters or GlobalParameters()
 
     with Session(engine) as session:
-        print_json(dump_snapshot_table_to_json(session, all_events, as_string=True))
+        print_json(dump_snapshot_tables_to_json(session, all_events, as_string=True))
 
 
 @app.command(name="list")