diff --git a/docs/api/models/foundation/models.md b/docs/api/models/foundation/models.md index e7d07e8..65dcfa6 100644 --- a/docs/api/models/foundation/models.md +++ b/docs/api/models/foundation/models.md @@ -16,6 +16,11 @@ members: - Moirai +::: timecopilot.models.foundation.patchtst_fm + options: + members: + - PatchTSTFM + ::: timecopilot.models.foundation.sundial options: members: diff --git a/docs/api/models/ml.md b/docs/api/models/ml.md index 33f5d26..ce63062 100644 --- a/docs/api/models/ml.md +++ b/docs/api/models/ml.md @@ -4,4 +4,11 @@ ::: timecopilot.models.ml options: members: - - AutoLGBM \ No newline at end of file + - AutoCatboost + - AutoElasticNet + - AutoLasso + - AutoLGBM + - AutoLinearRegression + - AutoRandomForest + - AutoRidge + - AutoXGBoost \ No newline at end of file diff --git a/docs/api/models/neural.md b/docs/api/models/neural.md index 3d8c74f..f7334cc 100644 --- a/docs/api/models/neural.md +++ b/docs/api/models/neural.md @@ -4,5 +4,8 @@ ::: timecopilot.models.neural options: members: + - AutoDeepAR + - AutoNBEATS - AutoNHITS + - AutoPatchTST - AutoTFT \ No newline at end of file diff --git a/docs/changelogs/v0.0.25.md b/docs/changelogs/v0.0.25.md index e43674d..3565fc5 100644 --- a/docs/changelogs/v0.0.25.md +++ b/docs/changelogs/v0.0.25.md @@ -1,6 +1,5 @@ ### Features - * **TimeGPT finetuning**: Finetuning is now supported for TimeGPT. You can adapt the pre-trained model to your data before forecasting via `TimeGPTFinetuningConfig`, with options for loss function and finetuning depth. See [#332](https://github.com/TimeCopilot/timecopilot/pull/332) and the [Finetuning Foundation Models](https://timecopilot.dev/examples/finetuning/) example for a full walkthrough. ```python diff --git a/docs/changelogs/v0.0.26.md b/docs/changelogs/v0.0.26.md index 891b1c2..45a50b9 100644 --- a/docs/changelogs/v0.0.26.md +++ b/docs/changelogs/v0.0.26.md @@ -1,6 +1,21 @@ ### Features -* **New ML models**: Added 7 new auto ML models powered by `mlforecast`'s hyperparameter optimization: `AutoLinearRegression`, `AutoXGBoost`, `AutoRidge`, `AutoLasso`, `AutoElasticNet`, `AutoRandomForest`, and `AutoCatboost`. All models support `quantiles` for probabilistic forecasts via conformal prediction and follow the same interface as the existing `AutoLGBM`. +* **New neural models**: Added 3 new auto neural models: `AutoNBEATS`, `AutoDeepAR`, and `AutoPatchTST`. All support `quantiles` for probabilistic forecasts trained with `MQLoss` and follow the same interface as the existing `AutoNHITS` and `AutoTFT`. + + ```python + import pandas as pd + from timecopilot.models.neural import AutoDeepAR, AutoNBEATS, AutoPatchTST + + df = pd.read_csv( + "https://timecopilot.s3.amazonaws.com/public/data/air_passengers.csv", + parse_dates=["ds"], + ) + + model = AutoNBEATS() + fcst_df = model.forecast(df, h=12, quantiles=[0.1, 0.5, 0.9]) + ``` + +* **New ML models**: Added 7 new auto ML models: `AutoLinearRegression`, `AutoXGBoost`, `AutoRidge`, `AutoLasso`, `AutoElasticNet`, `AutoRandomForest`, and `AutoCatboost`. All models support `quantiles` for probabilistic forecasts via conformal prediction and follow the same interface as the existing `AutoLGBM`. ```python import pandas as pd diff --git a/docs/model-hub.md b/docs/model-hub.md index 9f43b1a..44047bb 100644 --- a/docs/model-hub.md +++ b/docs/model-hub.md @@ -42,6 +42,7 @@ TimeCopilot provides a unified interface to state-of-the-art foundation models f - [Chronos](api/models/foundation/models.md#timecopilot.models.foundation.chronos) ([arXiv:2403.07815](https://arxiv.org/abs/2403.07815)) - [FlowState](api/models/foundation/models.md#timecopilot.models.foundation.flowstate) ([arXiv:2508.05287](https://arxiv.org/abs/2508.05287)) - [Moirai](api/models/foundation/models.md#timecopilot.models.foundation.moirai) ([arXiv:2402.02592](https://arxiv.org/abs/2402.02592)) +- [PatchTST-FM](api/models/foundation/models.md#timecopilot.models.foundation.patchtst_fm) ([arXiv:2602.06909](https://arxiv.org/abs/2602.06909)) - [Sundial](api/models/foundation/models.md#timecopilot.models.foundation.sundial) ([arXiv:2502.00816](https://arxiv.org/pdf/2502.00816)) - [TabPFN](api/models/foundation/models.md#timecopilot.models.foundation.tabpfn) ([arXiv:2501.02945](https://arxiv.org/abs/2501.02945)) - [TiRex](api/models/foundation/models.md#timecopilot.models.foundation.tirex) ([arXiv:2505.23719](https://arxiv.org/abs/2505.23719)) @@ -79,11 +80,21 @@ TimeCopilot integrates the popular Prophet model for time series forecasting, de TimeCopilot provides access to automated machine learning models for time series forecasting. These models leverage gradient boosting and other ML techniques to automatically select features and optimize hyperparameters for your specific time series data. +- [AutoCatboost](api/models/ml.md#timecopilot.models.ml.AutoCatboost) +- [AutoElasticNet](api/models/ml.md#timecopilot.models.ml.AutoElasticNet) +- [AutoLasso](api/models/ml.md#timecopilot.models.ml.AutoLasso) - [AutoLGBM](api/models/ml.md#timecopilot.models.ml.AutoLGBM) +- [AutoLinearRegression](api/models/ml.md#timecopilot.models.ml.AutoLinearRegression) +- [AutoRandomForest](api/models/ml.md#timecopilot.models.ml.AutoRandomForest) +- [AutoRidge](api/models/ml.md#timecopilot.models.ml.AutoRidge) +- [AutoXGBoost](api/models/ml.md#timecopilot.models.ml.AutoXGBoost) ## Neural Network Models TimeCopilot integrates state-of-the-art neural network models for time series forecasting. These models leverage deep learning architectures specifically designed for temporal data, offering powerful capabilities for complex pattern recognition and long-range dependency modeling. +- [AutoDeepAR](api/models/neural.md#timecopilot.models.neural.AutoDeepAR) +- [AutoNBEATS](api/models/neural.md#timecopilot.models.neural.AutoNBEATS) - [AutoNHITS](api/models/neural.md#timecopilot.models.neural.AutoNHITS) +- [AutoPatchTST](api/models/neural.md#timecopilot.models.neural.AutoPatchTST) - [AutoTFT](api/models/neural.md#timecopilot.models.neural.AutoTFT) \ No newline at end of file diff --git a/tests/models/conftest.py b/tests/models/conftest.py index 7b9033b..8e3d0dd 100644 --- a/tests/models/conftest.py +++ b/tests/models/conftest.py @@ -15,7 +15,12 @@ AutoLinearRegression, AutoXGBoost, ) -from timecopilot.models.neural import AutoNHITS, AutoTFT +from timecopilot.models.neural import ( + AutoNBEATS, + AutoNHITS, + AutoPatchTST, + AutoTFT, +) from timecopilot.models.prophet import Prophet from timecopilot.models.stats import ( ADIDA, @@ -65,6 +70,27 @@ def disable_mps_session(monkeypatch): hidden_size=8, ), ), + AutoNBEATS( + num_samples=2, + config=dict( + max_steps=1, + val_check_steps=1, + input_size=12, + n_harmonics=1, + n_polynomials=1, + stack_types=["identity"], + ), + ), + AutoPatchTST( + num_samples=2, + config=dict( + max_steps=1, + val_check_steps=1, + input_size=12, + hidden_size=8, + n_heads=2, + ), + ), AutoARIMA(), SeasonalNaive(), ZeroModel(), diff --git a/tests/models/test_models.py b/tests/models/test_models.py index 821bc87..8fa6827 100644 --- a/tests/models/test_models.py +++ b/tests/models/test_models.py @@ -109,7 +109,14 @@ def test_correct_forecast_dates(model, freq, h): "AutoRandomForest", "AutoCatboost", } - if model.alias in _ml_auto_aliases | {"AutoNHITS", "AutoTFT"}: + _neural_auto_aliases = { + "AutoNHITS", + "AutoTFT", + "AutoNBEATS", + "AutoDeepAR", + "AutoPatchTST", + } + if model.alias in _ml_auto_aliases | _neural_auto_aliases: # These auto ML and neural models require a longer minimum series length sizes_per_freq = { freq: 1_000 for freq in ["10S", "10T", "15T", "5T", "H", "Q-DEC"] @@ -230,7 +237,13 @@ def test_using_quantiles(model): elif "moe" in model.alias.lower(): # MoE is a bit more lenient with the monotonicity condition assert fcst_df[c1].le(fcst_df[c2]).mean() >= 0.5 - elif model.alias in ["AutoNHITS", "AutoTFT"]: + elif model.alias in [ + "AutoNHITS", + "AutoTFT", + "AutoNBEATS", + "AutoDeepAR", + "AutoPatchTST", + ]: # test config uses max_steps=1, so quantile ordering is not guaranteed continue else: @@ -252,6 +265,9 @@ def test_using_level(model): "AutoCatboost", "AutoNHITS", "AutoTFT", + "AutoNBEATS", + "AutoDeepAR", + "AutoPatchTST", } if model.alias in _level_unsupported: # These models do not support levels yet diff --git a/timecopilot/models/__init__.py b/timecopilot/models/__init__.py index b9d819c..4617f53 100644 --- a/timecopilot/models/__init__.py +++ b/timecopilot/models/__init__.py @@ -8,6 +8,7 @@ AutoRidge, AutoXGBoost, ) +from .neural import AutoDeepAR, AutoNBEATS, AutoPatchTST from .stats import ( ADIDA, IMAPA, @@ -25,8 +26,11 @@ __all__ = [ "ADIDA", "AutoCatboost", + "AutoDeepAR", "AutoElasticNet", "IMAPA", + "AutoNBEATS", + "AutoPatchTST", "AutoARIMA", "AutoCES", "AutoETS", diff --git a/timecopilot/models/neural.py b/timecopilot/models/neural.py index a7aa6f8..cc5665d 100644 --- a/timecopilot/models/neural.py +++ b/timecopilot/models/neural.py @@ -2,9 +2,18 @@ import pandas as pd from neuralforecast import NeuralForecast +from neuralforecast.auto import ( + AutoDeepAR as _AutoDeepAR, +) +from neuralforecast.auto import ( + AutoNBEATS as _AutoNBEATS, +) from neuralforecast.auto import ( AutoNHITS as _AutoNHITS, ) +from neuralforecast.auto import ( + AutoPatchTST as _AutoPatchTST, +) from neuralforecast.auto import ( AutoTFT as _AutoTFT, ) @@ -254,3 +263,333 @@ def forecast( qc=qc, ) return fcst_df + + +class AutoNBEATS(Forecaster): + """AutoNBEATS forecaster using NeuralForecast. + + Notes: + - Quantile forecasts are supported via `quantiles`. `level` is not + supported; use `quantiles` instead. + - AutoNBEATS requires a minimum length for some frequencies. + """ + + def __init__( + self, + alias: str = "AutoNBEATS", + num_samples: int = 10, + backend: str = "optuna", + config: dict | None = None, + ): + self.alias = alias + self.num_samples = num_samples + self.backend = backend + self.config = config + + def forecast( + self, + df: pd.DataFrame, + h: int, + freq: str | None = None, + level: list[int | float] | None = None, + quantiles: list[float] | None = None, + ) -> pd.DataFrame: + """Generate forecasts for time series data using the model. + + This method produces point forecasts and, optionally, quantile + forecasts. The input DataFrame can contain one or multiple time series + in stacked (long) format. + + Args: + df (pd.DataFrame): + DataFrame containing the time series to forecast. It must + include as columns: + + - "unique_id": an ID column to distinguish multiple series. + - "ds": a time column indicating timestamps or periods. + - "y": a target column with the observed values. + + h (int): + Forecast horizon specifying how many future steps to predict. + freq (str, optional): + Frequency of the time series (e.g. "D" for daily, "M" for + monthly). See [Pandas frequency aliases](https://pandas.pydata.org/ + pandas-docs/stable/user_guide/timeseries.html#offset-aliases) for + valid values. If not provided, the frequency will be inferred + from the data. + level (list[int | float], optional): + Not supported for AutoNBEATS. Use `quantiles` instead. + quantiles (list[float], optional): + List of quantiles to forecast, expressed as floats between 0 + and 1. Should not be used simultaneously with `level`. When + provided, the model is trained with + [`MQLoss`](https://nixtla.github.io/neuralforecast/losses.pytorch.html) + and the output DataFrame will contain additional columns named + in the format "model-q-{percentile}", where {percentile} + = 100 × quantile value. + + Returns: + pd.DataFrame: + DataFrame containing forecast results. Includes: + + - point forecasts for each timestamp and series. + - quantile forecasts if `quantiles` is specified. + + For multi-series data, the output retains the same unique + identifiers as the input DataFrame. + """ + if level is not None and quantiles is not None: + raise ValueError( + "You must not provide both `level` and `quantiles` simultaneously." + ) + if level is not None: + raise ValueError( + "Level is not supported for AutoNBEATS. " + "Please use `quantiles` instead." + ) + + inferred_freq = self._maybe_infer_freq(df, freq) + qc = QuantileConverter(level=None, quantiles=quantiles) + loss = MQLoss(level=qc.level) if qc.level is not None else MAE() + if self.config is None: + config = _AutoNBEATS.get_default_config(h=h, backend="ray") + config["scaler_type"] = tune.choice(["robust"]) + else: + config = self.config + if self.backend == "optuna": + config = _AutoNBEATS._ray_config_to_optuna(config) + fcst_df = run_neuralforecast_model( + model=_AutoNBEATS( + h=h, + loss=loss, + alias=self.alias, + num_samples=self.num_samples, + backend=self.backend, + config=config, + ), + df=df, + freq=inferred_freq, + alias=self.alias, + qc=qc, + ) + return fcst_df + + +class AutoDeepAR(Forecaster): + """AutoDeepAR forecaster using NeuralForecast. + + Notes: + - Quantile forecasts are supported via `quantiles`. `level` is not + supported; use `quantiles` instead. + - AutoDeepAR requires a minimum length for some frequencies. + """ + + def __init__( + self, + alias: str = "AutoDeepAR", + num_samples: int = 10, + backend: str = "optuna", + config: dict | None = None, + ): + self.alias = alias + self.num_samples = num_samples + self.backend = backend + self.config = config + + def forecast( + self, + df: pd.DataFrame, + h: int, + freq: str | None = None, + level: list[int | float] | None = None, + quantiles: list[float] | None = None, + ) -> pd.DataFrame: + """Generate forecasts for time series data using the model. + + This method produces point forecasts and, optionally, quantile + forecasts. The input DataFrame can contain one or multiple time series + in stacked (long) format. + + Args: + df (pd.DataFrame): + DataFrame containing the time series to forecast. It must + include as columns: + + - "unique_id": an ID column to distinguish multiple series. + - "ds": a time column indicating timestamps or periods. + - "y": a target column with the observed values. + + h (int): + Forecast horizon specifying how many future steps to predict. + freq (str, optional): + Frequency of the time series (e.g. "D" for daily, "M" for + monthly). See [Pandas frequency aliases](https://pandas.pydata.org/ + pandas-docs/stable/user_guide/timeseries.html#offset-aliases) for + valid values. If not provided, the frequency will be inferred + from the data. + level (list[int | float], optional): + Not supported for AutoDeepAR. Use `quantiles` instead. + quantiles (list[float], optional): + List of quantiles to forecast, expressed as floats between 0 + and 1. Should not be used simultaneously with `level`. When + provided, the model is trained with + [`MQLoss`](https://nixtla.github.io/neuralforecast/losses.pytorch.html) + and the output DataFrame will contain additional columns named + in the format "model-q-{percentile}", where {percentile} + = 100 × quantile value. + + Returns: + pd.DataFrame: + DataFrame containing forecast results. Includes: + + - point forecasts for each timestamp and series. + - quantile forecasts if `quantiles` is specified. + + For multi-series data, the output retains the same unique + identifiers as the input DataFrame. + """ + if level is not None and quantiles is not None: + raise ValueError( + "You must not provide both `level` and `quantiles` simultaneously." + ) + if level is not None: + raise ValueError( + "Level is not supported for AutoDeepAR. " + "Please use `quantiles` instead." + ) + + inferred_freq = self._maybe_infer_freq(df, freq) + qc = QuantileConverter(level=None, quantiles=quantiles) + loss = MQLoss(level=qc.level) if qc.level is not None else MAE() + if self.config is None: + config = _AutoDeepAR.get_default_config(h=h, backend="ray") + config["scaler_type"] = tune.choice(["robust"]) + else: + config = self.config + if self.backend == "optuna": + config = _AutoDeepAR._ray_config_to_optuna(config) + fcst_df = run_neuralforecast_model( + model=_AutoDeepAR( + h=h, + loss=loss, + alias=self.alias, + num_samples=self.num_samples, + backend=self.backend, + config=config, + ), + df=df, + freq=inferred_freq, + alias=self.alias, + qc=qc, + ) + return fcst_df + + +class AutoPatchTST(Forecaster): + """AutoPatchTST forecaster using NeuralForecast. + + Notes: + - Quantile forecasts are supported via `quantiles`. `level` is not + supported; use `quantiles` instead. + - AutoPatchTST requires a minimum length for some frequencies. + """ + + def __init__( + self, + alias: str = "AutoPatchTST", + num_samples: int = 10, + backend: str = "optuna", + config: dict | None = None, + ): + self.alias = alias + self.num_samples = num_samples + self.backend = backend + self.config = config + + def forecast( + self, + df: pd.DataFrame, + h: int, + freq: str | None = None, + level: list[int | float] | None = None, + quantiles: list[float] | None = None, + ) -> pd.DataFrame: + """Generate forecasts for time series data using the model. + + This method produces point forecasts and, optionally, quantile + forecasts. The input DataFrame can contain one or multiple time series + in stacked (long) format. + + Args: + df (pd.DataFrame): + DataFrame containing the time series to forecast. It must + include as columns: + + - "unique_id": an ID column to distinguish multiple series. + - "ds": a time column indicating timestamps or periods. + - "y": a target column with the observed values. + + h (int): + Forecast horizon specifying how many future steps to predict. + freq (str, optional): + Frequency of the time series (e.g. "D" for daily, "M" for + monthly). See [Pandas frequency aliases](https://pandas.pydata.org/ + pandas-docs/stable/user_guide/timeseries.html#offset-aliases) for + valid values. If not provided, the frequency will be inferred + from the data. + level (list[int | float], optional): + Not supported for AutoPatchTST. Use `quantiles` instead. + quantiles (list[float], optional): + List of quantiles to forecast, expressed as floats between 0 + and 1. Should not be used simultaneously with `level`. When + provided, the model is trained with + [`MQLoss`](https://nixtla.github.io/neuralforecast/losses.pytorch.html) + and the output DataFrame will contain additional columns named + in the format "model-q-{percentile}", where {percentile} + = 100 × quantile value. + + Returns: + pd.DataFrame: + DataFrame containing forecast results. Includes: + + - point forecasts for each timestamp and series. + - quantile forecasts if `quantiles` is specified. + + For multi-series data, the output retains the same unique + identifiers as the input DataFrame. + """ + if level is not None and quantiles is not None: + raise ValueError( + "You must not provide both `level` and `quantiles` simultaneously." + ) + if level is not None: + raise ValueError( + "Level is not supported for AutoPatchTST. " + "Please use `quantiles` instead." + ) + + inferred_freq = self._maybe_infer_freq(df, freq) + qc = QuantileConverter(level=None, quantiles=quantiles) + loss = MQLoss(level=qc.level) if qc.level is not None else MAE() + if self.config is None: + config = _AutoPatchTST.get_default_config(h=h, backend="ray") + config["scaler_type"] = tune.choice(["robust"]) + else: + config = self.config + if self.backend == "optuna": + config = _AutoPatchTST._ray_config_to_optuna(config) + fcst_df = run_neuralforecast_model( + model=_AutoPatchTST( + h=h, + loss=loss, + alias=self.alias, + num_samples=self.num_samples, + backend=self.backend, + config=config, + ), + df=df, + freq=inferred_freq, + alias=self.alias, + qc=qc, + ) + return fcst_df