Since John and I keep getting inquiries about changes made in the OL4 documentation, and many of you are likely to be using OL4 sooner or later, I thought I'd send out one message in the hopes of providing a single point of discussion.

We had four primary goals with the OL4 documentation:

1. Support the ability to mark content that is runtime-specific. As you know, OL4 supports multiple runtimes (Flash and DHTML, more to come), and while the core OpenLaszlo APIs are the same across all runtimes, there are differences where we can support extra features or where we haven't yet gotten to strict parity. The documentation must reflect this.
2. Improve maintainability of the doc tools and the documentation. The OL3 toolchain was extremely proprietary, difficult to understand, and thus had suffered from poor maintenance. In addition, it was possible for the reference material to drift relative to what was implemented in the code, and there were other areas (such as the schema) where we were essentially inviting inconsistency. We decided that it was a very high priority to rebuild the doc tools around standard technologies wherever possible, in order to simplify maintenance, ensure accuracy and consistency, and to better invite participation from the community.
3. Make it easier to accomplish several important goals: invite community participation in writing and maintaining the documentation, reuse the tools for other projects, localize the documentation, and maintain the live embedded examples.
4. Produce a unified index across all the documentation.

With these goals in mind, we rebuilt the entire documentation tool chain around two technologies. The first is a homegrown tool called "js2doc" which uses the JavaScript parser in our compiler to extract precise information about the APIs actually found in the source, merges that information with annotations and comments, and generates documentation in an XML format. This tool, which I built, is based on the standard javadoc tool and uses (as much as practical) the same conventions used by javadoc.

The second technology is an open source standard called Docbook. This is currently the most widely adopted documentation standard on the market and is extensively integrated with documentation publishing systems of various kinds. (The OL3 documentation used Docbook, but only as an internal tool.)

Docbook uses a pipelined workflow. Essentially, content is produced in Docbook format, and then transformed by XSLT into any of a number of destination formats. The workflow is optimized for static generation of a traditional book (either a single PDF file or a conventional, static html document web), but before adopting Docbook we made sure that there were solutions available for more dynamic publishing styles. John and I wanted to make sure that multi-pane navigation from OL3 was possible, as well as full-text search and other nifty web publishing techniques found in the online documentation for other languages (e.g. Perl).

Docbook has a clean localization story, and is well itself well documented (there's even an O'Reilly book for it!).

The reason I am writing this email is that John and I have found ourselves repeatedly explaining our decision to take an architecture-focused approach to this redesign rather than immediately providing all of the user-visible features from OL3. Our conclusion, based on years of interaction with the community, is that it is relatively easy to attract external contributors to the documentation -- to write cookbooks, to localize, to edit, to improve the tools -- *as long as* the barriers to entry are relatively low. By acting to simplify and normalize the doc toolchain, we open the door to much greater participation from the community. And by making it possible to reuse the tools for other projects, we further increase the potential for real support commitments. The tradeoff is that a few user-centered features are not implemented in the first version of the OL4 documentation, but we absolutely plan to provide those features in the future.

As you approach the OL4 documentation, you are likely to have the following questions, and we'd like to take the time to provide you with answers. Our purpose is to inform, and to open the door to broader input about priorities, not to shut down debate or questions.

Q. What happened to the left-side nav bar?

A. The simplest Docbook publishing mode is to generate a book-style html web, with a table of contents. Our plan from the beginning was to rely on existing doc publishing tools such as Apache Cocoon to provide more sophisticated navigation tools such as multi-frame navigation.

Q. What about search?

A. Search is not currently available within the new documentation. However, we have added a search box to http://www.openlaszlo.org/documentation that provides targeted search of documentation for a range of different OL versions, including OL4. Our current plan is that in-page search will arrive with adoption of a live publishing solution such as Apache Cocoon.

Q. Where is the class/tag index?

A. Due to a bug in the XSL tools for Docbook, automated construction of secondary indices increases the time for generation of the primary index from 8 minutes to 80 minutes. Until a fix is found for this bug, or a maintainable alternative is built -- and a hand written class index is not maintainable -- we have left this feature out of the documentation. But it is a goal to provide this index.

Q. Why not implement user comments like in the Perl documentation?

A. We like the idea of comments, but are also wary of the problem of staleness. (Stale info in comments is no more helpul than stale or incorrect info in documentation). You also get the problem that you have two virtual forums instead of one, which leads to confusion. We're considering some kind of commenting mechanism whereby relevant forum topics would be listed within each documentation page. Philosophically, we think the right approach is to keep the docs current, rather than to have them stale and corrected in the comments. We're looking to set up a process whereby the community can help us fix the documents themselves, not merely comment on them. We welcome input.

Q. Can I use the new toolchain for my own project?

A. Absolutely! The new toolchain is much more reusable than the old one.

Q. How can I get involved?

A. Email John Sundman (jsundman openlaszlo dot org) or myself (jgrandy at openlaszlo dot org) and we'll get you involved.

This entry was posted on Tuesday, July 10th, 2007 at 12:52 pm and is filed under Community, Development, Documentation, General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.