Ndarray to Mat conversion.

[ruseinov]

I have been working on a python extension, using

pyo3 = "0.21.1"
numpy = "0.21.0"
opencv = { version = "0.92.0" , features = ["clang-runtime"]}

The main issue I'm facing now is that Python OpenCV code is now reading images as numpy arrays, so that's what I have to pass to Rust. But those numpy arrays have too many channels, so when I actually need to call any Rust OpenCV methods - I need to convert this back to Mat, but the reshape fails due to CV_CN_MAX, which is 512.
It seems there is some way of tackling this within the C++ OpenCV Python extensions. It'd be really nice to introduce a way of doing that for Rust too.

Here's the code in question:

    // channels variable is > 512
    let reshaped_mat = mat.reshape_nd(*channels, shape);

Here's some context: opencv/opencv#20070

It looks like the PythonWrapper knows how to convert these things back and forth, reshaping to fit the bill.

[twistedfall]

Hi, thanks for the report, that's an interesting issue. Will it be possible for you to provide a minimum code snippet that exhibits this problem? I can understand the particular issue that OpenCV doesn't accept channels > 512, but I wanted to see the bigger picture, how this data is created.

Also if you you know the section in the Python wrapper that's handling this conversion it would be really useful too.

[ruseinov]

Will do!

When it comes to Python wrapper, it's not the best piece of code, but I think I can point you to a potential candidate.
I'll come up with minimal steps to reproduce asap!

[ruseinov]

Hey @twistedfall
So here's the snippet:

input_image = cv2.imread(input_dict["image_path"], cv2.IMREAD_UNCHANGED)
print(input_image.shape)
# (3466, 3463, 4)
mask = input_image[:, :, 3]
print(mask.shape)
# (3466, 3463)

Basically we read a 3-channel image and then create a mask out of it to later do things to it, like edges = cv2.Canny(mask, 100, 200) and other manipulations. And it works totally fine, as the python wrapper knows to somehow convert/reshape numpy stuff to a proper Mat.

However if we pass mask to a Rust extension in form of PyArray2<u8> and try to convert that to Mat with a reshape - that of course fails, because in this scenario we have 3463 channels.

fn list(mask: &PyArray2<u8>, bead_radius: i64, bead_separation: i64, bead_positions: &Bound<'_, PyList> ) -> PyResult<()> {
    let mut edges = UMat::new_def();
    let mask = unsafe {mask.as_array()};
    let mut standard_layout = mask.as_standard_layout();
    let slice = standard_layout.as_slice().unwrap();
    let shape_with_channels: Vec<i32> = mask.shape().iter().map(|&sz| sz as i32).collect();
    let (channels, shape) =  match shape_with_channels.split_last() {
            Some(split) => split,
            None => {
                Err(anyhow::anyhow!("err")).unwrap()
            }
        };


    let mat = Mat::from_slice(slice).unwrap().reshape_nd(*channels, shape);
}

Let me know if this is snippet enough. If you need the whole project setup - it's a pretty typical pyo3 project

pyo3 = "0.21.1"
numpy = "0.21.0"
opencv = { version = "0.92.0" , features = ["clang-runtime"]}
anyhow = "1.0.86"

[ruseinov]

Actually, here's the simplified example without Python: https://github.com/ruseinov/chan-fail .

[ruseinov]

When it comes to OpenCV Python conversions, I think you'll find them around here:
https://github.com/opencv/opencv/blob/4.x/modules/python/src2/cv2_convert.cpp
https://github.com/opencv/opencv/blob/4.x/modules/python/src2/cv2_convert.hpp
https://github.com/opencv/opencv/blob/4.x/modules/python/src2/pycompat.hpp

And some others within the same directory. Please let me know if I can help in any way. Fixing this would be really cool, by having some sort of rust numpy compat.

There used to be this https://github.com/jerry73204/rust-cv-convert/, but it's both outdated and does not solve that issue.