fix: resolve xarray MultiIndex FutureWarning in build_renewable_profiles#2144
fix: resolve xarray MultiIndex FutureWarning in build_renewable_profiles#2144FabianHofmann wants to merge 2 commits intomasterfrom
Conversation
Pass a RangeIndex to atlite instead of a MultiIndex and explicitly assign MultiIndex coordinates using xr.Coordinates.from_pandas_multiindex on the result. Also remove redundant log messages that duplicate the timing-based info logs.
|
I disagree with removing the log messages. And i would prefer solving the multi index warning at atlite level. |
|
Fixed in PyPSA/atlite#501 |
The reason being? With the faster and parallel execution of the atlite-based rules, you can't see which log refers to which conversion. The main time benchmarking logs (which state the tech) we keep. Also snakemake rules have descriptive messages nowadays. I don't see a reason to keep such kind of descriptive logs. If it is about debugging we should change the logger mode at least. |
|
I think instead of removing log messages we should change where they are displayed. Ideally, we would create a logging sink that the snakemake logger plugin pulls up and then instead of showing the log messages on the console only forward them there. The console would then be replaced with an interactive tui where you see which jobs are running and you can choose to see the logs of them. As immediate solution, maybe we decide instead that info messages should only go into the log file and only warning messages are printed onto the console. I'd be fine with such a change happening globally in PyPSA-Eur (or locally for that matter) The only two things missing then to make log files more helpful would be: A bridge that ensures that progressbars also write a minimal version into the log, and a default exception trap that also logs exceptions into the log. |
Changes proposed in this Pull Request
Fixes the
FutureWarningfrom xarray about pandas MultiIndex when calling atlite's conversion functions inbuild_renewable_profiles.py:RangeIndexto atlite instead of thebus_binMultiIndex, then explicitly assign the MultiIndex coordinates on the result usingxr.Coordinates.from_pandas_multiindex()before unstacking.Note that atlite currently fails when directly passing the xr.Coordinate as it expects the
indexobject to have a.nameattribute. so this change is save for future xarray version, but we need to support xr.Coordinates in atlite in the first place.Checklist
Required:
doc/release_notes.rst.If applicable: