This article describes the requirements and design of ROS 2’s per-package documentation system.
Authors: Marya Belanger
Date Written: 2020-10
Last Modified: 2020-10
ROS 2 is lacking a process for the conglomeration of package documentation. Discoverability for package documentation (and documentation in general) is one of the most frequent user complaints.
The goal of this design is to make ROS 2 documentation as accessible as possible for all ROS users. To achieve this we are seeking to standardize a documentation process for ROS 2 packages that will encourage maintainers to robustly document their packages, in turn making ROS 2 package documentation more available. The results will be presented in a consistent way alongside the general ROS 2 documentation, making the documentation as a whole discoverable and predictable.
In general, the vision for the system is:
docblock to the
The per-package documentation plan is an extension of the general ROS 2 documentation project, which introduced some requirements that affect the context of the per-package documentation:
docs.ros.org/<lang>/<distro>/...for generic documentation and
docs.ros.org/<lang>/<distro>/p/<package_name>/...for package documentation
The Primary requirements are those that must be in place for the system to be functional and achieve its purpose. The Secondary requirements are also absolute requirements, however they are not necessary to roll out the first stage of implementation. They will be carried out following the initial roll out.
1.1 All package documentation must be available with the rest of the ROS 2 documentation under a single domain
Package documentation will be treated the same as the generic ROS 2 documentation. Its presence as part of the docs site should be made well-known from the site’s overview as well as from any entry point into the docs site, and be intuitive from the organization and layout of the site.
1.2 Package documentation must have an easily navigable and intuitive URL structure
Every package’s “home page” should be reachable with minimal effort by the scheme
1.3 Package documentation must be maintainable in its repository without going through a third party
Maintainers and contributors will only have to work on their package’s documentation within its repository. The details of building and hosting will not be a concern of package maintainers.
Despite being hosted alongside the generic documentation, working on a package’s documentation will not require any work on the repositories of the generic documentation (currently
ros2/ros2_documentation) or the site repository (currently
1.4 The system must support C++ and Python API docs generation
The system will automatically extract API docs for these languages from the source code in package repositories and build the output on the docs site.
1.5 Package documentation must be written and formatted in rst (reStructuredText)
Rst is the file format currently utilized by the ROS 2 documentation. Package documentation should continue to utilize rst for consistency.
1.6 The system must allow package documentation to be versioned per ROS 2 distribution
The docs site and buildfarm will allow documentation for the latest version of a package corresponding to each ROS 2 distribution the docs site supports versioning for. The package repository can still maintain its own docs for previous versions.
While multiple package versions per distribution will not initially be supported, the URL structure will support the addition of this feature in the future (mentioned under Secondary requirements below).
1.7 The buildfarm must automatically build package documentation
Changes to the documentation in a package repository shouldn’t require the maintainer to manually trigger a build.
1.8 The system must automatically generate listing content for a package so it’s listed on docs.ros.org even if the package maintainer does not explicitly set up anything in the package repository
By default some package info will be listed for every package added to a rosdistro.
1.9 The systems must support cross-referencing between packages
A stable linking process will be in place to cross-reference between package docs despite their originating from separate repositories.
1.10 The system must prevent the possibility of package file names (created by maintainers) colliding with auto-generated content (from the system’s side)
When writing package documentation, maintainers should not have to concern themselves with auto-generated content, like the current
/symbols directories in the current
docs.ros.org API docs structure for ROS 1.
2.1 Package maintainers should have the ability to include free form documentation, like tutorials, design docs, etc., alongside generated API docs on docs.ros.org
2.2 The system should provide a standardized interface for generating an index page/landing page for packages, consistent across ROS 2
2.3 The system should provide a standardized method for generating package documentation locally, including build error alerts
2.4 The system should be extensible for more languages
2.5 Users should be able to switch between versions (distributions) while viewing a package’s documentation
2.6 The system should support building documentation for more than one version of a package per ROS 2 distribution
2.7 Changes to documentation pushed in a package repository should be immediately deployed to the documentation site
2.8 The system should support bi-directional cross-referencing (“backwards” to dependents)
The design covers the solution for satisfying the primary requirements. Some secondary requirements are also satisfied inherently through the primary design decisions.
In summary, the design for the system is:
The design description is laid out in the following subsections:
docs.ros.orgwill be organized in the new system.
The doc tool will invoke everything necessary to document a work space according to our requirements and vision for the new documentation site. It can be run on any ROS 2 package.
The tool may be developed to run standalone, or it may be necessary for maintainers to add a one liner to their package.xml, rosdoc.yaml, etc. telling the tool whether it should be building C++ docs, Python docs, or both.
The tool works by first building the package to ensure all the generated code is accounted for and able to be documented.
Then, the tool checks for a
doxyfile or C++ code.
If either are present, the tool will run Doxygen.
If the package maintainer wants their C++ code documented, they should include a
We can provide a standard template for the
doxyfile, which maintainers can then customize the contents of, to the specifications of their package’s needs.
Maintainers can also choose to not include a
doxyfile, in which case the tool will substitute a default doxyfile if it finds C++ code, providing standard configuration for the package.
Doxygen will produce API docs for any C++ code it is run on, and the maintainer has the option to further elaborate on their API docs by including Doxygen comment blocks within their source code.
Once Doxygen is completed, or if it never ran because the tool did not find any
doxyfile or C++ code, the tool will proceed to running Breathe and Sphinx.
Breathe imports the symbols from Doxygen’s output to Sphinx. It will allow C++ API docs to be built by Sphinx to maintain consistency across ROS 2 API docs and allow cross-referencing between all packages.
If there is no Doxygen output (no C++ documentation), Sphinx will run without Breathe. All packages will need a Sphinx configuration file. If a package doesn’t have one, the tool can use a default one. Python packages will need Sphinx directives explicitly integrated into the source code to produce API documentation (standard for Sphinx API documentation).
Once complete, the tool will output the package documentation build locally.
Users will be able to see any build errors as well as how their documentation will look once built to
This section addresses how the system will potentially satisfy requirement 2.4 in the future. In general, the goal is to treat additional languages the same as the system treats Python and Doxygen. That is, a library in any additional language should be able to utilize its native/standard documentation tool. There should then be a bridge between the output of that tool and Sphinx, like how Breathe bridges Doxygen and Sphinx. This will most likely not be an out-of-box solution for most languages, but the assumption is that extension will be possible.
Bridging with Sphinx allows the assumption that cross-referencing will be possible. It also maintains the standard landing page requirement; the top level entry point will be Sphinx regardless of a package’s language. The landing page will provide access to the other generated documentation.
For packages with documentation for multiple languages, the landing pages will provide access to each generator (e.g. “Click here for Doxygen content, Click here for Javadoc content”). Supporting multiple documentation engines will require that each generator outputs into its own designated directory (see URL Structure - /generated).
To opt in to having documentation built on docs.ros.org, maintainers will have to add a
doc block to their
distribution.yaml file that indicates the rosdistro branch and a path to the repository.
Opting-in this way will cause the tool to automatically generate an index landing page for the package.
The buildfarm will invoke the same tool used in the user process, run on n packages, to generate the documentation for
It will require some additional settings, for example the URL where content should generate into.
A webhook between GitHub and the buildfarm will indicate when there is a new commit to a repository (for the branches being tracked for documentation) so the documentation can be automatically regenerated and immediately deployed.
Some files will be auto-generated during this process (changelogs, tag files, etc.).
The auto-generated content won’t be saved to the upstream repositories, but it will be built to
There is a possibility of collision occurring between maintainer created files and generated files. To prevent this to the best of the system’s ability, the system will place auto-generated content in a restricted directory (see URL Structure - /generated).
This section addresses requirement 2.8, cross-referencing between sibling packages. As packages integrate their documentation into the system, cross-referencing between packages will be inconsistent. It may require multiple builds of all the packages in the system to eventually reach a consistent state for cross-references. How exactly this will be executed will be determined.
The index pages indicate the default page that is rendered when the directory level immediately before is visited without any specific
.html page following.
For example, visiting docs.ros.org/en/foxy would render
index1, docs.ros.org/en/foxy/p would render
index2, and docs.ros.org/en/foxy/p/
index1 is the general ROS 2 documentation overview page, what is currently seen when visiting index.ros.org/doc/ros2/
index2 will be the package listing page for each ROS 2 distribution.
index3 will be each specific package’s homepage.
This will be generated by the tool, but a user-generated index page should also be able to be merged or augmented with the auto-generated page to prevent collision.
To support the goal of being able to support multiple versions of packages documented within the context of one distro (2.6) we plan to reserve URL space and have a plan to migrate forward but not to implement it at first.
When functionality is added to the documentation system for more than one version of package documentation per rosdistro, those versions will be appended to the package name in the URL.
/p/rclcpp can have versioned content added to
After enabling versioning, the unversioned URL will be a link to the version currently indexed in the rosdistro.
See Considerations - Package version directory for an explanation of the naming convention.
The purpose of this directory is to allocate a place where automatically generated content can be placed without risk of collision with user-created content.
The tool will throw warnings to ensure users do not write any manual content into
/generated; it is ultimately up to maintainers to prevent collision.
Manually created content will override auto-generated content it collides with.
The index page for each package will link to the generated API docs, changelogs, etc.
Within a package’s repo, the tool will generate a
/docs directory where maintainers can place any manually created rst files they would like, like tutorials, architecture articles, etc.
The tool will direct Sphinx to generate the content in the
/docs directory, which will output to the top level when deployed (sibling to
However, it may become evident during development of the system that it is necessary to add an additional subdirectory/subdirectories at this level to contain the manually created content.
This decision will be confirmed before deployment of the new site.
For now, the URL structure uses
user generated as a placeholder for either case.
Besides the addition of
/en, the structure and functionality of ROS 1’s API documentation will remain the same.
Necessary redirects will be implemented.
There are many documentation tools available, but ultimately Sphinx, Doxygen and Breathe were the best choices to satisfy our requirements.
Sphinx and Doxygen are well-established and ubiquitous for documenting their respective languages This ensures the robustness of the system more so than any of the non-standard tools, and also that our users will have easy access to resources to aid them in documenting their packages with these tools.
With Breathe, it is simple to satisfy the requirement of coordinating C++ and Python documentation, and basing the system off of Sphinx allows simple customization with Python modules.
Another tool, Exhale has been considered for future addition to the system. Exhale adds some extra features available in pure Doxygen that Breathe alone doesn’t import into Sphinx.
If a package maintainer insists on using Doxygen alone, they are of course welcome to. However, this is not a use case we plan to support.
The standard entry point should be utilizing our tool, which will run Doxygen but then also run Breathe and Sphinx over it so the output can be uniform with the rest of our documentation. Using Doxygen by itself will sacrifice that consistency and also the ability to cross-reference to and from that package, both major driving points for this project. The tool should mostly mask the complexities of the Sphinx/Breathe/Doxygen system, making the difference between using “pure” Doxygen and using the new tool negligible for maintainers.
To satisfy secondary requirement 2.6, the first obvious consideration for the structure was
.../p/<package_name>/<version>, but, as indicated in the URL structure section above, the decision is to use
The chosen structure doesn’t force another directory, which allows versioned package documentation to be added later on in the development of the system without leaving an unused directory level in the meantime.
Adding package versions in the future will not disrupt the meaning of the URL structure, because
/<package_name>/ without any appended
-version.number.n will always implicitly indicate the latest version.