Add forecast classes by peanutfun · Pull Request #1115 · CLIMADA-project/climada_python

peanutfun · 2025-11-21T17:26:10Z

Changes proposed in this PR:

Add forecast variants for Hazard and Impact.
Adapt ImpactCalc to return ImpactForecast when needed.
Add tests.
Add tutorial for forecast data.

This PR resolves a major portion of #1118. Note that sub-tasks have been marked as completed when merged into this branch.

PR Author Checklist

PR Reviewer Checklist

* define base class * add init test * Fix #1127 --------- Co-authored-by: Valentin Gebhart <valentin.gebhart@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add ImpactForecast * Add unit tests

* Create hazardForecast base class * Update datetime to timedelta for lead_time * Add tests --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* change leadtime to timedelta * adapt impactforecast docstrings and tests * Remove stray comment --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add HazardForecast.select test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add test for hazardForecast.concat * Skip HazardForecast.concat test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add test for ImpactForecast.select * minor test modification --------- Co-authored-by: Evelyn Mühlhofer <evelyn.muehlhofer@meteoswiss.ch>

* Add test stub for ImpactForecast.concat * Use correct command for skipping a test * Fix merge issue

* Add data shape check for HazardForecast, ImpactForecast

Adapt Hazard from_hdf5 and write_hdf5 to forecast attributes

* Add extended select impact forecast test * Add extended hazard forecast test --------- Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Return impactForecast object in _return_impact * Return impactForecast object in _return_empty * Add full impactcalc test for impactForecast * Raise value error when computing impact with impact forecast without saving impact matrix * Cosmetics: Improve error message, move test to own class * Write nans for eai_exp and aai_agg when forecast is used --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add idx boolean selection for member and leadtime * Add test for weird data types --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add ImpactForecast.select --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add HazardForecast.select and extent test --------- Co-authored-by: Eliane Kobler <ekobler@student.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add HazardForecast.concat

Add min/mean/max methods --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Make Impact.concat support ImpactForecast * Add tests

* implement changes in Impact IO and ImpactForecast IO * Update tests * Fix small issue when reading H5 for ImpactForecast --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add quantile and median for hazard and impact forecast * Review --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

…ython into forecast-class

* Add impact forecast tutorial * Add 'dim' argument to reductions * Update tests --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com> Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Evelyn Mühlhofer <evelyn.muehlhofer@meteoswiss.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>

…ython into forecast-class

peanutfun · 2026-02-05T12:39:36Z

So it had been reprojected? Does MCH store the data differently than OGD? I am just wondering because it might be important to highlight the caveats when importing datasets with varying CRS and coordinate specifications.

Evelyn-M · 2026-02-05T12:54:38Z

I suppose so, yes. One is the MCH grid commonly used internally for national gridded forecast products, and the OGD data has a different grid and a much larger rectangular extent around Switzerland.

chahank · 2026-02-16T15:29:05Z

Can you please let me know once this is final for review?

ValentinGebhart

Looks very good to me, all comments are minor I think.

I have not yet reviewed the tutorial, will do a separate review for that when i have time.

ValentinGebhart · 2026-02-10T14:52:36Z

+        return reduced_attrs
+
+
+def reduce_unique_selection(


on first reading I found the back on forth confusing. e.g. HazardForecast.min() calls reduce_unqiue_slection with again calls HazardForecast.min().

I see why this works, just to comment on readability

ValentinGebhart · 2026-02-25T09:53:26Z

+        expected_n_events = (
+            forecast_netcdf_file["n_eps"] * forecast_netcdf_file["n_lead_time"]
+        )
+        npt.assert_array_equal(haz_fc.date, np.zeros(expected_n_events, dtype=int))


what about setting the date to the date of the reference time? Maybe it makes sense not to, just thinking

See comment above

ValentinGebhart · 2026-02-25T18:16:46Z

+    @pytest.fixture
+    def reduction_results_dim(self):
+        return {
+            "lead_time": {


I think at_event should always return the sum of the impact matrix over the locations (i.e., axis=1), independent of if the eventdimension corresponds to lead_time or member.

In the test, this is tested for "member" but not for "lead_time". The values look like they would give the same sum(axis=1) and sum(axis=0). Maybe we could adapt a value such that we know the right axis is summed? If too compilated, maybe the test for "member" is enough.

I don't understand. at_event is always exactly np.sum(imp_mat, axis=1), both for member and for lead_time.

I think at_event should always return the sum of the impact matrix over the locations (i.e., axis=1), ...

I agree! The test checks this explicitly, in my perception?

sorry, my comment was not clear. You are right, at_event should always be exactly np.sum(imp_mat, axis=1).

My point was that in the test of "lead_time", the test values seem to be chosen such that the impact matrix is always symmetric (off diagonals are the same), which is why np.sum(imp_mat, axis=1) is the same as np.sum(imp_mat, axis=0). So we actually do not test if the correct axis is summed here. However, we test this in the test of "member", so that should suffice. For me we can leave as is.

@ValentinGebhart I updated the test. The resulting matrices are now not symmetric anymore, making the test more clear.

peanutfun · 2026-03-03T20:51:11Z

@chahank This is final for review 🙂

chahank · 2026-03-06T10:08:03Z

Excellent! I will try to do it swiftly.

chahank · 2026-03-06T12:54:58Z

+        "aai_agg": 14.4,
+        "unit": "USD",
+        "frequency_unit": "1/month",
+        "imp_mat": sparse.csr_matrix(


Is it worth to test for negative impact values too?

chahank · 2026-03-26T16:35:03Z

+        "haz_type": "TC",
+        "pool": None,
+        "intensity": sparse.csr_matrix(
+            [[0.2, 0.3, 0.4], [0.1, 0.1, 0.01], [4.3, 2.1, 1.0], [5.3, 0.2, 0.0]]


Please include negative value intensity.

The change to this file only wraps the previous dummy_hazard function into a fixture. I think changing the intensity values is out of scope, because changes here might have effects on all other hazard tests. We can pick this up in a discussion about the conftest.py fixtures, see #1278

chahank · 2026-03-26T16:40:51Z

Is there a plan already to work on the comments by @ValentinGebhart?

chahank · 2026-03-26T16:41:18Z

The tutorial is now titled only impact forecast, but it is mostly about hazard forecast. This is quite confusing. Can this maybe be split into two tutorials? Or give it a different name?

chahank · 2026-03-26T16:50:54Z

A few comments on the tutorial:

Can the tutorial of the impact forecast please add a section a the top that shows the whole pipeline within one cell for a fast introduction?
I think the cells with ?ImpactForecast and ?HazardForecast do not serve a clear purpose.
OVerall, the tutorial could need a bit of improvements in clarity. Some of the cells are just there without explanations.
There are a lot of of Logging that indicate something should be improved in the code I think. It is unclear why imp_fct_all_mem_1lt = impact_forecast.max(dim="lead_time") triggers many lines of info and warning logging. Or in some cell there is several repetitions of 2026-01-08 14:55:43,321 - climada.engine.impact - INFO - The eai_exp and aai_agg are computed for the selected subset of events WITHOUT modification of the frequencies..

chahank

Excellent addition to the CLIMADA platform!

Please find a few detailled comments by @ValentinGebhart to address, and some user friendliness requests for the tutorial.

Co-authored-by: Valentin Gebhart <60438839+ValentinGebhart@users.noreply.github.com>

…ython into forecast-class

peanutfun · 2026-04-17T10:08:42Z

@chahank Thanks for the review!

Can the tutorial of the impact forecast please add a section a the top that shows the whole pipeline within one cell for a fast introduction?

I added that.

I think the cells with ?ImpactForecast and ?HazardForecast do not serve a clear purpose.

Removed the cells.

OVerall, the tutorial could need a bit of improvements in clarity. Some of the cells are just there without explanations.

Can you be more specific? What is missing apart from some "flavor text"?

There are a lot of of Logging that indicate something should be improved in the code I think. It is unclear why imp_fct_all_mem_1lt = impact_forecast.max(dim="lead_time") triggers many lines of info and warning logging. Or in some cell there is several repetitions of 2026-01-08 14:55:43,321 - climada.engine.impact - INFO - The eai_exp and aai_agg are computed for the selected subset of events WITHOUT modification of the frequencies..

I suppressed more warnings where it seemed appropriate. The particular warning you mention is now suppressed when computing an aggregate, but not when calling .select directly (where the warning originates from)

peanutfun and others added 21 commits November 21, 2025 18:23

Forecast templates

ad819f2

Clean slate for Forecast

abafba4

1127 - Forecast base class init (#1167)

1688641

* define base class * add init test * Fix #1127 --------- Co-authored-by: Valentin Gebhart <valentin.gebhart@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Impact forecast class (#1168)

d66c8dd

* Add ImpactForecast * Add unit tests

Hazard forecast class (#1171)

35f9517

* Create hazardForecast base class * Update datetime to timedelta for lead_time * Add tests --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Change leadtime from datetime to timedelta (#1172)

54bb592

* change leadtime to timedelta * adapt impactforecast docstrings and tests * Remove stray comment --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Hazard forecast test select (#1175)

4aaad4e

* Add HazardForecast.select test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Hazard forecast concat test (#1178)

d04fecf

* Add test for hazardForecast.concat * Skip HazardForecast.concat test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Check if Impact.select works on ImpactForecast (#1170)

d3cc83a

* Add test for ImpactForecast.select * minor test modification --------- Co-authored-by: Evelyn Mühlhofer <evelyn.muehlhofer@meteoswiss.ch>

Check if Impact.concat works on ImpactForecast (#1174)

a386d22

* Add test stub for ImpactForecast.concat * Use correct command for skipping a test * Fix merge issue

adapt from_hdf5 and write_hdf5

34612c5

Check sizes of forecast data vs. Hazard and Impact (#1180)

8a63b43

* Add data shape check for HazardForecast, ImpactForecast

implement code review

e123b7b

add comment to test

9ce7f73

Merge pull request #1181 from CLIMADA-project/adapt_hdf5_write_from

48b6d40

Adapt Hazard from_hdf5 and write_hdf5 to forecast attributes

Forecast/select extended tests (#1182)

64da484

* Add extended select impact forecast test * Add extended hazard forecast test --------- Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

add idx boolean selection for member and leadtime (#1183)

9d6fef9

* add idx boolean selection for member and leadtime * Add test for weird data types --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Add select to ImpactForecast (#1188)

b17bcc9

* Add ImpactForecast.select --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Add select method to HazardForecast (#1185)

302be63

* add HazardForecast.select and extent test --------- Co-authored-by: Eliane Kobler <ekobler@student.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Make Impact.concat support ImpactForecast

85b92ef

peanutfun force-pushed the forecast-class branch from 85b92ef to 302be63 Compare December 10, 2025 13:29

peanutfun and others added 8 commits December 10, 2025 14:32

Add HazardForecast.concat (#1184)

52edc45

* Add HazardForecast.concat

Support ImpactForecast.concat (#1192)

bb99794

* Make Impact.concat support ImpactForecast * Add tests

Add hdf5 IO for ImpactForecast (#1190)

0e7857f

* implement changes in Impact IO and ImpactForecast IO * Update tests * Fix small issue when reading H5 for ImpactForecast --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Merge branch 'forecast-class' of github.com:CLIMADA-project/climada_p…

d84db1e

…ython into forecast-class

Merge branch 'forecast-class' of github.com:CLIMADA-project/climada_p…

99e82a1

…ython into forecast-class

ValentinGebhart reviewed Feb 25, 2026

View reviewed changes

chahank reviewed Mar 6, 2026

View reviewed changes

ValentinGebhart reviewed Mar 11, 2026

View reviewed changes

Comment thread doc/user-guide/climada_engine_ImpactForecast.ipynb

chahank reviewed Mar 26, 2026

View reviewed changes

chahank requested changes Mar 26, 2026

View reviewed changes

peanutfun and others added 16 commits March 27, 2026 17:04

Merge branch 'develop' into forecast-class

d487c27

Rename 'Forecast' to 'ForecastMixin'

4c07a37

Rework parameters to 'reduce_unique_selection'

380f0dd

Fix inconsistent override of Impact.from_hdf5

9137976

Update docstrings

0f06d36

Fix overrides in HazardForecast

8487eb9

Fix more docstrings

98b8bcb

Update climada/engine/test/test_impact_forecast.py

006559b

Co-authored-by: Valentin Gebhart <60438839+ValentinGebhart@users.noreply.github.com>

Avoid warnings emitted from ImpactForecast.select

ab37a8d

Update tests

259396f

Merge branch 'forecast-class' of github.com:CLIMADA-project/climada_p…

9e3cf17

…ython into forecast-class

Try suppressing more warnings

45150ed

Update tutorial

98e799d

Update tutorial

aaf80c1

Fix impact_calc fixture

6cc4eb5

Fix symmetry issue in ImpactForecast tests

4634b30

Conversation

peanutfun commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Author Checklist

PR Reviewer Checklist

Uh oh!

peanutfun commented Feb 5, 2026

Uh oh!

Evelyn-M commented Feb 5, 2026

Uh oh!

chahank commented Feb 16, 2026

Uh oh!

ValentinGebhart left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peanutfun commented Mar 3, 2026

Uh oh!

chahank commented Mar 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chahank commented Mar 26, 2026

Uh oh!

chahank commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chahank commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chahank left a comment

Choose a reason for hiding this comment

Uh oh!

peanutfun commented Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

peanutfun commented Nov 21, 2025 •

edited

Loading

ValentinGebhart left a comment •

edited

Loading

chahank commented Mar 26, 2026 •

edited

Loading

chahank commented Mar 26, 2026 •

edited

Loading