hdf5storage not called out in package requirements

[gregoryng]

subradar/subradar/simulation/trento.py

Line 8 in ff26836

import hdf5storage

This package doesn't get installed even though it is required. Ideally it would be nice if the code could be rewritten not to need it and could just use h5py? But maybe that is too much work.

So I think you need to add something to the pip setup.

subradar/setup.py

Line 23 in ff26836

INSTALL_REQUIRES = [

[gregoryng]

I'm also having trouble finding labellines:

subradar/subradar/fractal.py

Line 14 in ff26836

from labellines import labelLine, labelLines

It looks like there are a lot of imports in fractal that ought to get removed.

[cgrima]

Looking at hdf5 right now. In the mean time pip install matplotlib-label-lines.

[gregoryng]

Furthermore, hdf5storage appears to be incompatible with python 3.12, which removes distutils, so we should try to remove this requirement.

frejanordsiek/hdf5storage#130

Traceback (most recent call last):                                                                                                  
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/hdf5storage/utilities.py", line 44, in <mod
ule>                                                                                                                                
    from pkg_resources import parse_version                                                                                         
ModuleNotFoundError: No module named 'pkg_resources'                                                                                
                                                                                                                                    
During handling of the above exception, another exception occurred:                                                                 
                                                                                                                                    
Traceback (most recent call last):
  File "/home/staff/ngg/work/sds1/SHARAD/run_rsr.py", line 30, in <module>                                              
    import rsr
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/rsr/__init__.py", line 9, in <module>     
    from . import detect, fit, invert, pdf, run, sample, utils
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/rsr/fit.py", line 16, in <module>         
    from .Classdef import Statfit
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/rsr/Classdef.py", line 10, in <module>    
    import subradar as sr
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/subradar/__init__.py", line 28, in <module>
    from . import iem, invert, roughness, surface, utils, filtering, simulation                                                     
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/subradar/simulation/__init__.py", line 7, in <module>
    from . import trento
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/subradar/simulation/trento.py", line 8, in <module>
    import hdf5storage
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/hdf5storage/__init__.py", line 45, in <module>
    from . import lowlevel
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/hdf5storage/lowlevel.py", line 35, in <module>
    from hdf5storage.utilities import *
  File "/home/staff/ngg/work/sds1/ve_sds_py312/lib64/python3.12/site-packages/hdf5storage/utilities.py", line 46, in <module>
    from distutils.version import StrictVersion as parse_version\
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[gregoryng]

Another problem with hdf5storage reported by cgrima:

I can run the trento simulation reader provided in subsurface, using a trento dataset I have found. However, there is an issue with hdf5storage that is using an old sub-function not available in numpy 2.0 releases anymore.

source_folder can point to /home/staff/cgrima/gerekos in the hierarchy if you want to test it.

In [1]: import subradar as sr
In [2]: sim = sr.simulation.trento.Results('ldap', source_folder=['.'])
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[26], line 1
----> 1 sim = sr.simulation.trento.Results('ldap', source_folder=['.'])

File ~/Downloads/test/ve1/lib/python3.13/site-packages/subradar/simulation/trento.py:25, in Results.__init__(self, simulation_name, source_folder, gain)
     23 self.input_filenames = glob.glob(os.path.join(self.input_folder, '*.mat'))
     24 self.gain = gain
---> 25 self.trajectory = self.read_trajectory()
     26 self.simulation = self.read_simulation()
     27 self.geoelectric = self.read_geoelectric()

File ~/Downloads/test/ve1/lib/python3.13/site-packages/subradar/simulation/trento.py:78, in Results.read_trajectory(self)
     76 def read_trajectory(self):
     77     filename = [filename for filename in self.input_filenames if 'Trajectory_' in filename][0]
---> 78     out = self.loadmat(filename)
     79     return out

File ~/Downloads/test/ve1/lib/python3.13/site-packages/subradar/simulation/trento.py:44, in Results.loadmat(self, filename)
     41 def loadmat(self, filename):
     42     """Load matfile
     43     """
---> 44     mat = hdf5storage.loadmat(filename)
     45     out = dict()
     46     for key in mat.keys():

File ~/Downloads/test/ve1/lib/python3.13/site-packages/hdf5storage/__init__.py:1764, in loadmat(file_name, mdict, appendmat, variable_names, marshaller_collection, **keywords)
   1759 # Will first assume that it is the HDF5 based 7.3 format. If an
   1760 # OSError occurs, then it wasn't an HDF5 file and the scipy function
   1761 # can be tried instead.
   1762 try:
   1763     # Make the options with the given marshallers.
-> 1764     options = Options(marshaller_collection=marshaller_collection)
   1766     # Append .mat if it isn't on the end of the file name and we are
   1767     # supposed to.
   1768     if appendmat and not file_name.endswith('.mat'):

File ~/Downloads/test/ve1/lib/python3.13/site-packages/hdf5storage/__init__.py:260, in Options.__init__(self, store_python_metadata, matlab_compatible, action_for_matlab_incompatible, delete_unused_variables, structured_numpy_ndarray_as_struct, make_atleast_2d, convert_numpy_bytes_to_utf16, convert_numpy_str_to_utf16, convert_bools_to_uint8, reverse_dimension_order, store_shape_for_empty, complex_names, group_for_references, oned_as, compress, compress_size_threshold, compression_algorithm, gzip_compression_level, shuffle_filter, compressed_fletcher32_filter, uncompressed_fletcher32_filter, marshaller_collection, **keywords)
    258 self.marshaller_collection = marshaller_collection
    259 if not isinstance(marshaller_collection, MarshallerCollection):
--> 260     self.marshaller_collection = MarshallerCollection()

File ~/Downloads/test/ve1/lib/python3.13/site-packages/hdf5storage/__init__.py:902, in MarshallerCollection.__init__(self, marshallers)
    895 def __init__(self, marshallers=[]):
    896     # Two lists of marshallers need to be maintained: one for the
    897     # builtin ones in the Marshallers module, and another for user
   (...)
    900     # Grab all the marshallers in the Marshallers module (they are
    901     # the classes) by inspection.
--> 902     self._builtin_marshallers = [m() for key, m in dict(
    903                                  inspect.getmembers(Marshallers,
    904                                  inspect.isclass)).items()
    905                                  if m != Marshallers.parse_version]
    906     self._user_marshallers = []
    908     # A list of all the marshallers will be needed along with
    909     # dictionaries to lookup up the marshaller to use for given
    910     # types, type string, or MATLAB class string (they are the
    911     # keys).

File ~/Downloads/test/ve1/lib/python3.13/site-packages/hdf5storage/Marshallers.py:492, in NumpyScalarArrayMarshaller.__init__(self)
    479 self.matlab_attributes |= set(['MATLAB_class', 'MATLAB_empty',
    480                               'MATLAB_int_decode',
    481                               'MATLAB_fields'])
    482 # As np.str_ is the unicode type string in Python 3 and the bare
    483 # bytes string in Python 2, we have to use np.unicode_ which is
    484 # or points to the unicode one in both versions.
    485 self.types = [np.ndarray, np.matrix,
    486               np.chararray, np.core.records.recarray,
    487               np.bool_, np.void,
    488               np.uint8, np.uint16, np.uint32, np.uint64,
    489               np.int8, np.int16, np.int32, np.int64,
    490               np.float32, np.float64,
    491               np.complex64, np.complex128,
--> 492               np.bytes_, np.unicode_, np.object_]
    493 self._numpy_types = list(self.types)
    494 # Using Python 3 type strings.

File ~/Downloads/test/ve1/lib/python3.13/site-packages/numpy/__init__.py:400, in __getattr__(attr)
    397     raise AttributeError(__former_attrs__[attr], name=None)
    399 if attr in __expired_attributes__:
--> 400     raise AttributeError(
    401         f"`np.{attr}` was removed in the NumPy 2.0 release. "
    402         f"{__expired_attributes__[attr]}",
    403         name=None
    404     )
    406 if attr == "chararray":
    407     warnings.warn(
    408         "`np.chararray` is deprecated and will be removed from "
    409         "the main namespace in the future. Use an array with a string "
    410         "or bytes dtype instead.", DeprecationWarning, stacklevel=2)

AttributeError: `np.unicode_` was removed in the NumPy 2.0 release. Use `np.str_` instead.