Skip to content

fix: resolve xarray MultiIndex FutureWarning in build_renewable_profiles#2144

Open
FabianHofmann wants to merge 2 commits intomasterfrom
fix/multiindex-future-warning
Open

fix: resolve xarray MultiIndex FutureWarning in build_renewable_profiles#2144
FabianHofmann wants to merge 2 commits intomasterfrom
fix/multiindex-future-warning

Conversation

@FabianHofmann
Copy link
Copy Markdown
Contributor

@FabianHofmann FabianHofmann commented Apr 10, 2026

Changes proposed in this Pull Request

Fixes the FutureWarning from xarray about pandas MultiIndex when calling atlite's conversion functions in build_renewable_profiles.py:

FutureWarning: the `pandas.MultiIndex` object(s) passed as 'bus_bin' coordinate(s) 
will no longer be implicitly promoted and wrapped into multiple indexed coordinates...
  • Pass a RangeIndex to atlite instead of the bus_bin MultiIndex, then explicitly assign the MultiIndex coordinates on the result using xr.Coordinates.from_pandas_multiindex() before unstacking.
  • Remove redundant log messages that duplicate the timing-based info logs already present.

Note that atlite currently fails when directly passing the xr.Coordinate as it expects the index object to have a .name attribute. so this change is save for future xarray version, but we need to support xr.Coordinates in atlite in the first place.

Checklist

Required:

  • Changes are tested locally and behave as expected.
  • Code and workflow changes are documented.
  • A release note entry is added to doc/release_notes.rst.

If applicable:

  • N/A

Pass a RangeIndex to atlite instead of a MultiIndex and explicitly
assign MultiIndex coordinates using xr.Coordinates.from_pandas_multiindex
on the result. Also remove redundant log messages that duplicate the
timing-based info logs.
@FabianHofmann FabianHofmann requested review from coroa and fneum April 10, 2026 07:17
@coroa
Copy link
Copy Markdown
Member

coroa commented Apr 14, 2026

I disagree with removing the log messages. And i would prefer solving the multi index warning at atlite level.

@coroa
Copy link
Copy Markdown
Member

coroa commented Apr 14, 2026

Fixed in PyPSA/atlite#501

@FabianHofmann
Copy link
Copy Markdown
Contributor Author

FabianHofmann commented Apr 14, 2026

I disagree with removing the log messages.

The reason being? With the faster and parallel execution of the atlite-based rules, you can't see which log refers to which conversion. The main time benchmarking logs (which state the tech) we keep. Also snakemake rules have descriptive messages nowadays. I don't see a reason to keep such kind of descriptive logs. If it is about debugging we should change the logger mode at least.

@coroa
Copy link
Copy Markdown
Member

coroa commented Apr 14, 2026

I think instead of removing log messages we should change where they are displayed.

Ideally, we would create a logging sink that the snakemake logger plugin pulls up and then instead of showing the log messages on the console only forward them there.

The console would then be replaced with an interactive tui where you see which jobs are running and you can choose to see the logs of them.

As immediate solution, maybe we decide instead that info messages should only go into the log file and only warning messages are printed onto the console. I'd be fine with such a change happening globally in PyPSA-Eur (or locally for that matter)

The only two things missing then to make log files more helpful would be: A bridge that ensures that progressbars also write a minimal version into the log, and a default exception trap that also logs exceptions into the log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants