Build error when using ament_python_install_package() and generated interfaces

[jtbandes]

Originally posted by @jacobperron in #131 (comment):

This change [#131] prevents a package from building if it both generates interfaces and installs a Python package. For example, I have a package that is not longer building since it does something like this:

rosidl_generate_interfaces(${PROJECT_NAME}_msgs
  ...
)

ament_python_install_package(${PROJECT_NAME})

I see an error like:

CMake Error at /home/jacob/ros2-linux/share/ament_cmake_python/cmake/ament_python_install_package.cmake:108 (add_custom_target):
  add_custom_target cannot create target
  "ament_cmake_python_symlink_my_package" because another target with the
  same name already exists.  The existing target is a custom target created
  in source directory
  ...

I'm running into this issue in rosbridge_suite as well when building for Rolling: RobotWebTools/rosbridge_suite#581

One thing I don't understand is why this error appears for rosapi, but not rosbridge_library.

I'm a CMake noob, but would it be possible to simply pass a different, unique target name on this line instead of just ${PROJECT_NAME}?

rosidl_python/rosidl_generator_py/cmake/rosidl_generator_py_generate_interfaces.cmake

Lines 123 to 125 in 191a143

	if(NOT rosidl_generate_interfaces_SKIP_INSTALL)
	ament_python_install_package(${PROJECT_NAME} PACKAGE_DIR "${_output_path}")
	endif()

(Meta-question: is there anything we could have done in either the rosbridge_suite repo or this repo to catch this kind of failure sooner, and not have to wait for another Rolling release to incorporate a fix into rosbridge_suite? Asking because I'm still learning about the release process, having recently taken on some maintenance of rosbridge_suite for ROS 2 compatibility.)

[jtbandes]

This issue is blocking packaging of rosbridge_suite for rolling (RobotWebTools/rosbridge_suite#661). @clalancette @jacobperron Any ideas how the issue can be resolved? Has any progress been made on it yet?

[clalancette]

I would say the easiest workaround is to split the Python code and the message generation into two separate packages. You generally want to do that anyway; a downstream client may want to communicate using your custom messages, but may not want/need the other code that comes along with it.

[jacobperron]

@jtbandes This dropped off my radar, sorry for the late reply.

One thing I don't understand is why this error appears for rosapi, but not rosbridge_library.

I would expect the error to appear for both packages. I'm guessing it has something to do with the package.xml; there's a discrepancy between the package.xml of rosbridge_library and rosapi. Notably, rosbridge_library does not declare itself as an interface package (ie. with <member_of_group>rosidl_interface_packages</member_of_group>). This is just a guess.

I'm a CMake noob, but would it be possible to simply pass a different, unique target name on this line instead of just ${PROJECT_NAME}?

I think this is doable. Though, since #131, I think the main issue is now we risk install collisions between rosidl_generate_interfaces and your custom package. @hidmic is this accurate?

Any ideas how the issue can be resolved? Has any progress been made on it yet?

I'm not really sure how to proceed. Maybe we provide an option for users install multiple Python packages with the same name? We could try to detect actual install collisions and error in that case. I don't think anyone has looked into this or proposed other options. I worked around the issue originally by creating a separate package for my interfaces 🤷

is there anything we could have done in either the rosbridge_suite repo or this repo to catch this kind of failure sooner, and not have to wait for another Rolling release to incorporate a fix into rosbridge_suite

I think this is what Rolling is meant for: allow maintainers to build/test against the latest changes and fix things in preparation for the next ROS release. You could setup a CI job for your repo that builds against Rolling. I suggest using our Debian testing repository to avoid having to wait for the next Rolling sync (instructions here).

[jacobperron]

Another workaround to consider is renaming your Python package such that it does not match the package name.

[jtbandes]

I've made a PR to split up the packages: RobotWebTools/rosbridge_suite#665

Since some of these msgs were used in tests only, I called that package rosbridge_test_msgs. Do you think it will be possible to have our build & test jobs succeed in the build farm without actually adding this rosbridge_test_msgs to the rosdistro files?

[clalancette]

Since some of these msgs were used in tests only, I called that package rosbridge_test_msgs. Do you think it will be possible to have our build & test jobs succeed in the build farm without actually adding this rosbridge_test_msgs to the rosdistro files?

If you are talking about the buildfarm at https://build.ros2.org, then no, this won't work. Anything the tests depend on need to be released onto the buildfarm.

I can think of at least 2 ways to fix this:

1. Release all of the packages, including the test packages, onto the buildfarm. Then everything will work.
2. Split the tests into a separate package as well, and then blacklist that package from being released onto the ROS 2 buildfarm. In that case, the buildfarm will never try to run the tests (since the package isn't released). Then run the tests in the GitHub Actions or whatever that run for incoming PRs.

We do the second thing for https://github.com/ros2/system_tests . It is never released into a distribution, but we run tests with it nightly from https://ci.ros2.org .

[jtbandes]

Thanks for the info!

Anything the tests depend on need to be released onto the buildfarm.

Aren't the tests running from an environment that was built on the buildfarm? As long as the test_msgs package is in the same repo, it will be checked out and built with colcon build, right? Or is there something about the buildfarm environment that will prevent it from using a locally built package (which isn't otherwise released) as a <test_depend>?

[clalancette]

Aren't the tests running from an environment that was built on the buildfarm? As long as the test_msgs package is in the same repo, it will be checked out and built with colcon build, right?

No, that's not how the buildfarm works.

In short, the buildfarm doesn't use colcon at all. It only checks out the specific sources for the package that is building, and installs all other dependencies in the package.xml via debian packages. So if a dependency isn't released, it won't have built a debian package and thus won't be available to satisfy the dependency.

[jtbandes]

Ok, that makes sense. It seems like the easiest thing to do is release the rosbridge_test_msgs package. Do you have any concerns with that approach?

[clalancette]

Ok, that makes sense. It seems like the easiest thing to do is release the rosbridge_test_msgs package. Do you have any concerns with that approach?

That will certainly work. I prefer approach 2 that I pointed out above, in that you aren't releasing "useless" (to end users) packages into the ROS distribution. But it's a relatively minor point so whatever is easier for you is fine.