Often different configuration settings are associated with different types of content. For example, one book series has boxes, marginal notes, footnotes and up to five heading levels while the other only has two heading levels and footnotes. Or books of a certain imprint extract their metadata from content files while the larger part of the book production relies on an ONIX dump. A more fundamental distinction runs between journals and books.

The transpect configuration cascade assumes that there is a fundamental setup, maybe for a whole group of publishers. Below that, there are overrides for the individual imprints, below that a distinction into books and journals, etc. An obvious disadvantage of this approach is that it is somehow arbitrary where to insert a configuration override level, and in which order. It might be wiser to distinguish between books and journals first before dealing with the different imprints. Another distinction may be the input type. If a series has both Word and InDesign input, should one divide it into virtual subseries for each input type? Or should one distinguish between input types on the highest level and then replicate the imprint/series hierarchy below each input type? What if a book that was scheduled to be typeset in Word moves to an InDesign production line? Does the whole project have to relocate then in the content hierarchy?

These thoughts illustrate that it is often impossible to find the single organizational hierarchy for all the content. There are orthogonal categories such as input type, language, layout type, etc. Although these categories often coincide with the more or less arbitrary content hierarchy, they do not necessarily do.

The question is why one should strive to organize the content in a single hierarchy. Another approach is to flexibly attach tags to each content item, such as “English”, “InDesign”, “A5 default layout”.

An important factor in favor the hierarchical approach is the xsl:import instruction. At the core of transpect, there are inevitably some XSLT transformations. It is conceivable that transpect stores special building blocks with transformation rules for processing English-language, InDesign-input and A5-layout content. That is of course, if there is a need at all for treating these properties individually. The fundamental XSLT then has to be enhanced in such a way that it it also includes the overrides. There are some issues with this approach that are mostly related to XSLTs customization method and how this approach is not in line with it: Firstly, the XSLT that is finally applied to the content is not a static file; it must be generated. Although this is absolutely feasible, it should better be avoided in order to lower the complexity and to ease debugging. Secondly: If the generated code is a wrapper that includes the fundamental code and the overrides, there will be priority conflicts between fundamental and override templates which leads to a warning, or function redeclarations which leads to an error. If the compound code uses xsl:import instead of xsl:include, these issues will disappear. It may not, however, be implemented in such a way that the fundamental code is enhanced with import statements because the importing code always wins, no matter what the priority of imported templates is. So one would need to generate a new wrapper that first imports the fundamental code and then the stylesheets that implement the special code. Global rules must be established which special code should have the highest precedence, in case that there were patterns that match the same nodes. This code must be included last, according to the XSLT import precedence rules. But what if a combination of English language and A5 layout requires a different template (for example, for running heads) than English language and A4 layout or German and A5? Then the categories are not perfectly orthogonal any more; they are somewhat entangled.

We try to overcome these issues by having an import cascade where the most specific stylesheet is chosen (more on how transpect selects it below). This stylesheet then imports the next-specific, until finally the fundamental template is included. This approach has its drawbacks exactly when cross-cutting categories are involved. Consider localization of generated content as an example. Imagine that it’s not just a linear string-for-string translation. For example, the English version reads “[for details see] [Chapter] [1]”, while the German reads “[siehe] [Kapitel] [1] [für Details]”. This kind of flexible localization may best be achieved in XSLT by supplying a default template in English that will construct the whole phrase, filling “Chapter” and the number by calling other templates. Depending on the content’s language, either the English default or the German wrapper should be used. Suppose that the German wrapper is the same for all content, so it will be placed in a directory for the common configuration. But then every customization level must also provide a localized wrapper that imports the default-language stylesheet of this level and then the common localization stylesheet. This is all feasible, but if there are many combinations of orthogonal features, there need to be a wrapper for each possible combination on each customization level. This is to illustrate that it is not always possible to organize content or configuration strictly hierarchically, and that dynamic assembly of stylesheets might at some point become inevitable. Stylesheets might indeed be assembled dynamically, but only with some orchestrating code such as XProc. Since we are using XProc, it is absolutely feasible (see next section), but we try to avoid it for reasons given above, at least for XSLT stylesheets.

For Schematron checks and for the merging of HTML with metadata, however, we are already using an XSLT-based assembly mechanism that not only selects the most specific file, but builds a compound ruleset from all relevant files that it finds in the cascade.

There is already a mechanism in place that permits dynamic assembly of an XSLT stylesheet. This might in fact be used for decorating an existing stylesheet with an import statement for localization, depending on the desired language.

Every file in the cascade may be replaced by an XSLT stylesheet with the same base name. The main motivation for this dynamic assembly mechanism was the redundancy that we were trying to avoid when maintaining many only slightly modified XProc pipelines.

This mechanism may also be used to share common configurations, for example between sibling journals, without introducing new customization levels. If 5 of 24 journals of a given imprint share a feature where the XML of each converted article must be automatically linked against a patent database, one could in principle introduce another configuration hierarchy level, journal-group. There will be two journal-groups, patent and no-patent. In no-patent, there will be no configuration at all – thus the parent configuration will be applied. For the journal-group called patent, there will be a hub2jats/hub2jats_driver.xpl.xsl that will patch the parent configuration’s XProc to invoke the additional XSLT pass (or XQuery script) that inserts the links to the patent DB. See Customizing XProc for a discussion of this approach.

Now consider that 2 of the 5 patent journals are produced in a two-column layout that they share with 13 other journals. The remaining 14 journals are a single-column layout. The processing will be different at a relatively early stage of the overall conversion process, evolve-hub. Let’s assume the XProc in this macroscopic step is the same for all journals and the only differences are in the XSLT. The XSLT that applies to all journals will contain the rules for one of the layouts. One of the 2-column journals will get an overriding evolve-hub/driver.xsl that imports the journal XSLT and override some templates/functions/variables in order to implement transformation rules for the the 2-column layout. This XSLT may now seve as the master for all other 2-column journals. All other 2-column journals may then feature an identical, rather trivial evolve-hub/driver.xsl.xsl stylesheet that simply loads the master journal’s XSLT and identically reproduces it.

By and large, we think that the hierarchical configuration mechanism, enhanced with this dynamic generation feature, is adequate to model many content zoos.

In phylogenetics, a clade is a group consisting of an ancestral species and all its descendants. Analogously, we call a subtree of common configuration settings a clade. Clades may contain other clades, and they may contain “content” elements – positions in the inheritance hierarchy where content items may be attached. These content items then belong exclusively to that clade.

As the first step in most pipelines, an XProc parameter document (c:param-set) will be calculated. It contains, among other settings, the paths that will be searched for configuration items.

Clade selection is being done by an XSLT stylesheet that transforms the configuration file into a parameter document. The XSLT stylesheet takes two parameters as input, file and clades. Both are optional, but if you don’t supply any of them, the transformation will terminate with an error.

Supplying the string file is sufficient if its base name adheres to the file naming conventions that the function transpect:parse-file-name() implements. For an input of file:/some/path/acme_02651_std.idml, the function might then return a sequence of attributes such as (publisher=acme, production-line=standard, work=02651, ext=idml). The attributes will be passed as tunneled parameters to the transformation of the configuration file. If the configuration file contains a clade with (role=production-line, name=standard) that is child of a clade with (role=publisher, name=acme) and that has a content child with (role=work, name-regex=\d{5}), the production-line clade that is immediately above the content element will be selected as the matching clade.

The matching algorithm is implemented in paths.xsl that can be imported from its canonical location, http://transpect.le-tex.de/book-conversion/converter/xsl/paths.xsl if the catalogs are properly set up (see below). The sample implementation of transpect:parse-file-name() in said stylesheet has to be overridden in the importing stylesheet (unless your project has clades with the roles “publisher” and “series”, a content element with the role “work”, and a the same file name matching regex as the imported paths.xsl). The customized (importing) stylesheet may be supplied to paths.xpl on the primary input port.

When a matching clade has been found, the paths where to look for configuration files will be calculated. For a given clade, the paths will be the names of all clades along the ancestor-or-self axis (in document order), joined by '/'. The paths are always relative to the adaptions subdirectory of the transpect base directory. For the matching clade in the example above, this would be $transpect/adaptions/acme/standard/. The less specific paths are generated by going up the clade hierarchy and calculating the paths for each clade in the same manner, yielding $transpect/adaptions/acme/. As the least specific path, $transpect/adaptions/common/ will be added.

An important concept of transpect is that there may be configuration overrides not only for a class of content (a clade), but for individual content objects. These overrides are expected to reside where the content is, not with the central transpect code. Each clade element (and the conf element) may carry a @content-base-uri attribute that specifies the URI where content is located. This URI is typically a canonical URI such as http://cms.acme.com/ that will be catalog-resolved to where the content is checked out in the local file system. Looking upward from the content element, the next @content-base-uri will be selected and the relative paths will be constructed as above, with the content name added as the last paths component. Example: http://cms.acme.com/acme/standard/02651/, or its catalog-resolved version file:/C:/cygwin/home/user/Acme/content/standard/02651/ if http://cms.acme.com/ resolves to file:/C:/cygwin/home/user/Acme/content/. In the resulting XProc parameter document, both the catalog-resolved and the canonical paths will be included for each of the paths, as we will see soon. It should be noted that content elements may include other content elements; for example, when a work may consist of multiple parts that are expected to be uploaded as single files. For example, the base name of the file above may be acme_02651-01_std, yielding an additional part=01 attribute as output of the custom file name parser.

In addition to the file parameter that goes into paths.xsl, a clades parameter may also be supplied. It is a space- or comma-separated list of name/value pairs, for example production-line=different,publisher=acme_uk. The parameter will be parsed into XML attributes in a straightforward way. In the clade matching algorithm, these attributes will have precedence over the filename parsed attributes. This enables applications that accept arbitrary input file names and let the user do the publisher/production-line/… selection perform by other means (e.g., dropdown lists).

A note on the matching algorithm: The sequence in which the attributes are given does not matter. The input above will also match a configuration document in which the outer clade has the role of production-line and the inner clade has the publisher role.

A note on file extensions: The function transpect:parse-file-name() accepts file names with or without paths, or URIs. They will strip everything including the last forward slash (backward slashes are not expected here – in fact there is a file name to URI normalization step in place). From that remainder, everything after the first dot will be stripped (the first dot will be stripped, too).

By convention, the extension should be put into the attribute ext so that it can be used for caluculating the repository location of the file. The extension is the result of the function transpect:ext() that is also overridable. By default, it will yield 'docx.xml' for a file 'file:/foo/bar.docx.xml'.

The subdirectory where a file will be placed if it is committed to revision control usually depends on its file name extension. In order to configure this, there is another overridable function in paths.xsl, transpect:target-subdir(), that takes a content element as its argument and will output the subdirectory (idml, docx, images,  …) where a content item will be stored. In the course of the configuration file transformation, all content elements below the matching clade will receive all attributes that have been generated by transpect:parse-file-name(). So if you otherwise don’t need to overwrite transpect:target-subdir(), let your transpect:parse-file-name() function generate an attribute ext and your destination subdirectory will be calculated properly.

Another use for file extensions can be seen in the example below, where the extension serves as a switch for the choice of configuration. A word of caution though: If you have separate overriding configurations for, e.g., 'docx' and 'idml', it may well happen that a file that once bore a 'docx' ending comes back with an 'idml' or an 'xml' ending at a later production stage. The different configurations should therefore only treat input differently during normalization stages (docx→XML, IDML→XML).

As a real-life example for a cascaded configuration, we’ll look at how Unionsverlag organize their configuration – the configuration file and how it translates to XSLT, CSS, … files in a system hierarchy.

This is the annotated configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="http://transpect.le-tex.de/book-conversion/converter/schema/transpect-conf.rng"?>
❶<?xml-model href="http://transpect.le-tex.de/book-conversion/converter/schema/transpect-conf.rng" 
  schematypens="http://purl.oclc.org/dsdl/schematron"?>
<conf xmlns="http://www.le-tex.de/namespace/transpect"
  content-base-uri="http://unionsverlag.com/content-repo/" 
  ❷paths-xsl-uri="http://customers.le-tex.de/generic/book-conversion/adaptions/unionsverlag/xsl/paths.xsl">
  <cascade>
    <reserved name="css"/>❸
    <reserved name="epubtools"/>
    <reserved name="evolve-hub"/>
    <reserved name="fonts"/>
    <reserved name="htmlreports"/>
    <reserved name="htmltemplates"/>
    <reserved name="hub2html"/>
    <reserved name="hub2tei"/>
    <reserved name="metadata"/>
    <reserved name="schematron"/>
    <reserved name="styles"/>
    <reserved name="tei2html"/>
    <reserved name="xpl"/>
    <reserved name="xsl"/>
    <clade role="publisher" name="unionsverlag">
      <param name="publisher-prefix-for-style-mapping" value="uv"/>❹  
      <param name="use-css-decorator-classes" value="yes"/>
      <param name="epub-version" value="EPUB3"/>
      <param name="which-htmlreport" value="evolve-hub"/>
      <param name="publisher-prefix-for-style-mapping" value="uv"/>
      <clade role="production-line" name="legacy">
        <content role="work" ❺name-regex="^\d{5}"/>
      </clade>
      <clade role="production-line" name="standard">
        <content role="work" name-regex="^\d{5}"/>
        <clade role="ext" name="idml" ❻content-base-uri="..">
          <content role="work" name-regex="^\d{5}">
          	❼<content role="chapter" name-regex="^\d{3}[fb]?$" 
          		content-base-uri=".."/>
          </content>
        </clade>
        <clade role="ext" name="docx" content-base-uri="..">
          <content role="work" name-regex="^\d{5}"/>
        </clade>
      </clade>
    </clade>
  </cascade>
</conf>

The configuration file does not have to reside at any particular location. We usually store it as conf/transpect-conf.xml. It will often serve as the primary input to transpect pipelines that accept the input file name (docx, idml, …) as an XProc option.

In the file system, the cascaded configuration is stored like this:

Figure 1.1. Configuration directory layout for Unionsverlag

<?xml version="1.0" encoding="UTF-8"?>
<c:param-set xmlns:c="http://www.w3.org/ns/xproc-step">
   <c:param name="basename" value="UV_STD_20655_00000_DOCX_TestBuch"/>
   <c:param name="debug" value="'no'"/>
   <c:param name="debug-dir-uri"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/transpect/debug"/>
   <c:param name="epub-version" value="EPUB3"/>
   <c:param name="file"
            ❶value="file:/C:/cygwin/home/gerrit/Unionsverlag/content/unionsverlag/standard/20655/docx/UV_STD_20655_00000_DOCX_TestBuch.docx"/>
   <c:param name="interface-language" value="en"/>
   <c:param name="pipeline" value="unknown"/>
   <c:param name="progress" value="no"/>
   <c:param name="progress-to-stdout" value="no"/>
   <c:param name="publisher-prefix-for-style-mapping" value="uv"/>
   <c:param name="repo-href-canonical"
            ❷value="http://unionsverlag.com/content-repo/unionsverlag/standard/20655/docx/UV_STD_20655_00000_DOCX_TestBuch.docx"/>
   <c:param name="repo-href-local"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/content/unionsverlag/standard/20655/docx/UV_STD_20655_00000_DOCX_TestBuch.docx"/>
   <c:param name="s9y1" value="20655"/>❸
   <c:param name="s9y1-path"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/content/unionsverlag/standard/20655/"/>
   <c:param name="s9y1-path-canonical"
            value="http://unionsverlag.com/content-repo/unionsverlag/standard/20655/"/>
   <c:param name="s9y1-role" value="work"/>
   <c:param name="s9y2" value="docx"/>
   <c:param name="s9y2-path"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/transpect/adaptions/unionsverlag/standard/docx/"/>
   <c:param name="s9y2-path-canonical"
            value="http://customers.le-tex.de/generic/book-conversion/adaptions/unionsverlag/standard/docx/"/>
   <c:param name="s9y2-role" value="ext"/>
   <c:param name="s9y3" value="standard"/>
   <c:param name="s9y3-path"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/transpect/adaptions/unionsverlag/standard/"/>
   <c:param name="s9y3-path-canonical"
            value="http://customers.le-tex.de/generic/book-conversion/adaptions/unionsverlag/standard/"/>
   <c:param name="s9y3-role" value="production-line"/>
   <c:param name="s9y4" value="unionsverlag"/>
   <c:param name="s9y4-path"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/transpect/adaptions/unionsverlag/"/>
   <c:param name="s9y4-path-canonical"
            value="http://customers.le-tex.de/generic/book-conversion/adaptions/unionsverlag/"/>
   <c:param name="s9y4-role" value="publisher"/>
   <c:param name="s9y5-path"
            value="file:/C:/cygwin/home/gerrit/Unionsverlag/transpect/adaptions/common/"/>
   <c:param name="s9y5-path-canonical"
            value="http://customers.le-tex.de/generic/book-conversion/adaptions/common/"/>
   <c:param name="s9y5-role" value="common"/>
   <c:param name="srcpaths" value="yes"/>
   <c:param name="transpect-project-uri" value=""/>
   <c:param name="use-css-decorator-classes" value="yes"/>
   <c:param name="which-htmlreport" value="evolve-hub"/>
</c:param-set>

This parameter document will be passed to all of transpect’s XProc steps that rely on cascaded configuration (and to some other steps, too – there are some more global settings apart from the cascade handily stowed away in that document), particularly to transpect:load-cascaded itself.

      <clade role="production-line" name="standard">
        <content role="work" name-regex="^\d{5}"/>
        <clade role="production-line" name="idml" 
          content-base-uri="..">
          <content role="work" name-regex="^\d{5}"/>
        </clade>
        <clade role="production-line" name="docx" 
        	content-base-uri="..">
          <content role="work" name-regex="^\d{5}"/>
        </clade>
      </clade>

To wrap it up: The XProc parameter contains the paths where transpect will look for configuration overrides. The parameter document will be computed anew for each XProc run. It will be computed, by means of a customizable XSLT, from the transpect project’s configuration file. The input file name and optionally a clades selection string will serve as parameters to that transformation. The configuration file specifies how content is organized into “clades” and where the content resides, both as local and as canonical repository URIs.

Check if:

See section of creation/customizing of adaptions/common/evolve-hub/driver.xpl

Currently, the step named "eval" in pubcoach/xpl/dynmic-transformationdriver.xpl cannot have more outputs connected than inputs (mabye a bug?). Check every subpipeline output ports bc:dtp will load and evaluate.