Update csv data loader, introduce dedicated csv hypersurfaces service, clean up hypersurfaces docs - #855
Conversation
| @@ -0,0 +1,274 @@ | |||
| """ | |||
| PISA pi stage to apply hypersurface fits from discrete systematics parameterizations | |||
There was a problem hiding this comment.
PISA pi.
It would be good if some remarks could be added on why there is a separate service for reading csv files when there is already a hypersurfaces service (and/or how these differ). There is a currently fairly comprehensive, though in the case of hypersurfaces probably outdated, readme which one could consider for this type of documentation.
There was a problem hiding this comment.
The the current hypersurfaces service can only handle non-interpolated HS. We could merge this csv-hypersurfaces service with the current one, but the result would be a pretty long and complex stage. At least for now, separation (like we do it for the data loader stages) is the simpler option. Documentation will be good though.
There was a problem hiding this comment.
I think this motivation (together with some statement on if and how the mathematical approach deviates from the regular hypersurface code/service) would be good to have in the code base itself, perhaps in the csv_hypersurfaces.py module docstring.
| @@ -0,0 +1,4001 @@ | |||
| intercept,intercept_sigma,dom_eff,dom_eff_sigma,hole_ice_p0,hole_ice_p0_sigma,hole_ice_p1,hole_ice_p1_sigma,bulk_ice_abs,bulk_ice_abs_sigma,bulk_ice_scatter,bulk_ice_scatter_sigma,dm31,pid,reco_coszen,reco_energy | |||
There was a problem hiding this comment.
What's in here, toy/dummy values? Suggest giving it a more descriptive filename (see the hdf5 events file in the same directory) or adding a comment (shouldn't be a problem when parsed by pandas, https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html).
| nominal_systematics : dict | ||
| Systematics and their nominal values | ||
|
|
||
| inter_param : str |
There was a problem hiding this comment.
create issue (future task: multiple)
|
Is there a relation between this PR and |
|
That is for the old (GRECO) data release. It can only handle non-interpolated HS. |
thehrh
changed the title
3 participants
In order to make pisa more flexible about its inputs we should update the csv reader. This will be especially useful for data releases. I started by making it possible to load multiple csv files at once and using a dict for the keys to lead from the file rather than hard coding it.