Consider releasing into ROS Indigo / Jade / Kinetic

[gavanderhoorn]

This is a great library - and I've been recommending it to people - but it would be more convenient if it would be released through the ROS buildfarm. That way packages can start to build_ and run_depending on the plugin, and it would remove the need for users to build from source.

I'd be happy to assist in any way necessary.

[Patrick Beeson]

Yeah, I've been meaning to get around to doing this, but I'm really busy working on other projects right now. I have never used Bloom, so if you are willing to help make this happen sooner rather than later, I'll accept any help you can provide. Ideally, if you could paste the console steps needed to make this happen quickly, I'll run them when I get a free moment, and get this done.

[gavanderhoorn]

I'll see if I can get you a set of commands.

There is some more work than that most of the times though (fixing up CHANGELOG files, etc). I'd be willing to do that as well, but that would require write access to the repository.

Bloom itself then requires a separate release repository, where it stores release configuration (yaml files, tagged source checkout, etc). These typically follow a naming scheme where you suffix the repository name with -release (so track_ik-release in this case).

I'm not entirely sure if release repositories on bitbucket would enjoy the same amount of automation, but since you can also have an off-line release repository, it'll probably work.

[Patrick Beeson]

Yeah, the current track_ik repo is ONLY the public/master branch, so we could make that the release repo if the name doesn't have to change. My devel repo is private for now, though I was about to push that out as an 'experimental' branch to the public remote so people can start to use my Cartesian trajectory generator and have collision detection rolled into the IK library. Let me know if you look into this further.

[gavanderhoorn]

If by 'release repo' you mean a place where Bloom can put its stuff, that won't work: the release and source repository need to be two separate repositories. We typically create a new empty one as a sibling to the source repository.

Branches in the source repository don't really influence the release process, as least as there is one with a commit that has been tagged.

In ROS-Industrial we usually create a 'ROS distribution specific release branch', next to the -devel branch (so indigo for Indigo releases, etc). This partly comes down to preference though. Having a dedicated release branch (containing the stable version of released artefacts) is always a good idea though.

[Patrick Beeson]

Ok great. Yeah, I might go ahead and rename 'master' to indigo and push an indigo-experimental from my current development repo. I can make a track_ik-release repo if needed. Just keep me informed. I'd love to learn to use Bloom one day, I just haven't found the time. If you can basically walk me through like a dummy this time, maybe it'll be easy to repeat in the future.

[Carlos J. Rosales Gallegos]

Note that you can always mirror bitbucket -> github if you find troubles with automation, as long the author doesn't mind hosting there as well.

[Patrick Beeson]

I don't mind mirroring bitbucket repo to github repo if I have to. I'm not sure why I'd need to do that? Does Bloom assume github? Because the mirroring plugin allows me to mirror bitbucket to bitcket, or I can always fork and occasionally sync if needed.

I'll defer to Gijs on this.

[gavanderhoorn]

I'm not sure whether the release repository needs to be on github, but Bloom creates some PRs to submit new releases to the rosdistro repository. I don't think bitbucket->github PRs are supported ;).

I'll take a look tomorrow, or the day after.

Thanks for considering releasing trac_ik.

[gavanderhoorn]

To follow up on this:

• the only benefit we get from putting the release repository on github is that Bloom will be able to submit PRs for us against rosdistro. Most of the times the changes to rosdistro files are minor, and editing the files involved and creating the PR manually is not a lot of work, but it is an additional step in the process.
• for everything else: bitbucket is ok

To get a release .. released:

• (optional): create $distro and $distro-devel branches (where distro in ['indigo', 'jade', 'kinetic']). Release only from $distro branches.
• all packages will need changelogs, and preferably formatted according to REP-132. This is not a requirement, but highly recommended. The changelog in the trac_ik metapackage is already almost according to spec, so it's not too much work to update it (alternatively: ignore REP-132 and just populate the changelogs manually).
• changelogs for other packages can be generated using catkin_generate_changelogs.
• as this repository uses vN.N.N for its tags, Bloom and some other tools will need this specified on the command line. Unfortunately catkin_generate_changelog doesn't support that right now (see ros-infrastructure/catkin_pkg#142).
• after that, just follow wiki/bloom/Tutorials/FirstTimeRelease. Make sure to specify the Version bit correctly (as it needs the v prefix on tags).

[gavanderhoorn]

@pbeeson wrote:

I don't mind mirroring bitbucket repo to github repo if I have to. I'm not sure why I'd need to do that? Does Bloom assume github?

As to why Github might be 'required': it's not really, but the rosdistro repository is on Github, and submitting PRs cross-hosting platform is not possible, so Bloom will not be able to do that for you. It's a minor thing really, and only an 'issue' with the release repository.

If the automation is desirable, you could consider putting just the release repository on Github (with the source repository remaining on bitbucket).

[Patrick Beeson]

OK. Will do. Is mirroring the best way in your opinion? IS there a pointer to exactly how they want that release repo named?

[gavanderhoorn]

What would you want to mirror?

The source repository (this one) doesn't need to be on github. It would just be the release repository. That is something you don't have to do anything with, Bloom will take care of it (it also doesn't contain your source code, just release configuration files). If you place it on Github, Bloom can really take care of everything, but there is no requirement whatsoever for that.

Release repositories are stand-alone repositories. Naming typically follows a <source_repo>-release scheme.

[Patrick Beeson]

OK. I'm following the Bloom tutorial now. I'll let you know if I hit any issues.

[gavanderhoorn]

Just saw your PR against rosdistro. Now all we have to do is wait for it to be merged and the buildfarm to pick it up (and see whether pull request #9 was complete).

Thanks for releasing.

[Patrick Beeson]

@gavanderhoorn Thanks for helping. So I need to make a new track for jade and kinetic? I'm assuming if something isn't compatible with kinetic it'll tell me prior to setting up a PR?

[gavanderhoorn]

So I need to make a new track for jade and kinetic?

yes.

I'm assuming if something isn't compatible with kinetic it'll tell me prior to setting up a PR?

No, it won't. Bloom only automates the release, it doesn't do any testing. That's left to the developer.

industrial_ci includes an option for running prerelease tests. See (Optional) Run ROS Prerelease Test.

[gavanderhoorn]

@gavanderhoorn wrote:

[..] and see whether pull request #9 was complete

Seems it wasn't: build of kinematics_plugin failed (log).

Could be that trac_ik_lib doesn't export it's includes properly, or that trac_ik_kinematics_plugin doesn't include the files correctly. I'll see if I can find out.

[Patrick Beeson]

I thought it looked like libnlopt-dev wasn't being installed despite being in the build depends.

[Patrick Beeson]

I just added libnlopt-dev to the build depends for kinematics_plugin and examples, and sent a 1.4.3 release. That should fix, but if there is a way to export the build depends, that might be preferred.

[gavanderhoorn]

I was counting on trac_ik_lib to bring in the libnlopt-dev dependency transitively, as that does have it, and kinematics_plugin depends on trac_ik_lib. I'm not near a Linux machine, so can't test right now.

[gavanderhoorn]

Ah, I know what is wrong: libnlopt-dev is currently only listed as a build_depend of trac_ik_lib, but as trac_ik_lib actually includes nlopt headers in its own headers, which it then exports (by installing them), libnlopt-dev should also be a run_depend of trac_ik_lib.

Adding libnlopt-dev as a build_depend to kinematics_plugin will work, but only for that package. Any other pkg trying to depend on trac_ik_lib will run into the same problem.

I assume that if you add libnlopt-dev as a run_depend to trac_ik_lib's manifest, it'll fix the issue.

[Patrick Beeson]

OK. Well libnlopt-dev was a build_depend, but not a run_depend. I did this because Tully really wanted it that way last year. I think it should be fixed. Once this makes it back into the build rotation.

[Patrick Beeson]

Yes, it USED to be a run_depend but when I was creating the distro lin the docs/sources last year, Tully essentially asked me to do it this way.

[gavanderhoorn]

Which PR was that against rosdistro?

[Patrick Beeson]

https://github.com/ros/rosdistro/pull/11827

[gavanderhoorn]

thanks, but I meant the one in which Tully asked you to remove the run_depend on libnlopt-dev. Was it rosdistro#9772?

Afaict Tully just asked to keep the -dev pkg a separate dependency, which makes sense. If a package uses nlopt headers, but never includes any of its headers in its own headers, then the -dev pkg is only needed as a build_depend. Keeping the rosdep rules separate makes that possible, and reduces the footprint of the pkg. I'm guessing that's why Tully asked you to do that.

I'm not sure c4e9252 is going to work btw: the hashes won't match, so the buildfarm will probably not checkout the correct version.

[Patrick Beeson]

Most of our conversation was via email, but yeah, the package.xml I had was directly requested by him. No biggie, we figured it out.

In any case, it looks like this went through.

Job

How long in your experience until this shows up in the apt Packages list?

[gavanderhoorn]

In any case, it looks like this went through.

Job

yes, looks like all downstream jobs of trac_ik__ubuntu completed successfully.

Nice.

How long in your experience until this shows up in the apt Packages list?

It depends on when Tully or Jackie do a sync to the public repo. In the meantime you can use the shadow repository to test the packages.

You can use the Indigo status page to see the status of the trac_ik packages in the repositories.

[Patrick Beeson]

So, I got this from the Shadow Repo, and when I run roslaunch trac_ik_examples pr2_arm.launch I get a Symbol Lookup error, like the trac_ik.so cannot be found. Any suggestions on how to fix?

/opt/ros/indigo/lib/trac_ik_examples/ik_tests: symbol lookup error: /opt/ros/indigo/lib/trac_ik_examples/ik_tests: undefined symbol: _ZN7TRAC_IK7TRAC_IKC1ERKSsS2_S2_ddNS_9SolveTypeE

[Patrick Beeson]

ARGH. It was my LD_LIBRARY_PATH not getting reset with a source /opt/ros/indigio/setup.bash. It was pointing at my new library with a new API.

It works.

Thanks for the help.

I'll close after trying Jade/Kinetic another day.

[Patrick Beeson]

Released to Indigo and Jade. Because MoveIt! has not yet released to Kinetic, the kinematics_plugin package will not build. I could make a new branch that eliminates that package from the Kinetic build, but I feel like anyone really wanting trac_ik packages in Kinetic (outside of MoveIt!) can also download and build them on their own. So for now, I'm going to hold off doing the extra work to keep a Kinetic branch that eliminates a package.

[gavanderhoorn]

Yeah, agreed about Kinetic.

Good to hear the release works, you got me worried with the missing symbol :).

Might be nice to announce this on the ROS-Industrial mailing list? So people can test the packages in the Shadow repository?

Thanks for taking the time to do the release. Appreciated.

[Jimmy Da Silva]

@pbeeson I just tested out the shadow repo and it worked great for me :). Please keep us posted when the public repo gets updated.

Thanks again for the great work on this!

[Dave Coleman]

This is great to have released! Could you add a link from the moveit website to the moveit IK README - perhaps at the bottom of this page?

[femauch]

moveit is now available in kinetic and the master branch builds with it. Would you mind reconsidering a kinetic release? :)

[Patrick Beeson]

@femauch Done.