To hyphenate or not?

On the listserv there was a discussion on hyphenating generated text. And we all were a bit lost in space. Most of us remembered that there was a problem, but as several fixes had been made to this problem, nobody remembered quite well what was the rule (at least, I could not

Very soon the XPP master himself, Steve Piercey, stepped into the discussion in order to give us the final word.

Here is an abstract on what he had to say:

The general rule is: “The first word of computer generated text will not hyphenate”.

For example, if you have a macro that generates the text “hyphenation is allowed”, the word “hyphenation” will not hyphenate.

Its a software limitation that is based on the mixing of computer generated stuff with real text. We have tried to remove this limitation a number of times, but have not been successful. It goes to the very core of how XPP deals with computer generated text, and our attempts to force hyphenation down a different software path with bug fixes goes against the overall design and has resulted in missing and/or duplicated text. Hence the limitation. I wish there was a solution for this, but there is not.

There are variations on this.

1. If you have a tag/macro just before the first word and it generates text (albeit invisible), the rule still applies and the first word will not hyphenate. This can sometimes be ‘fixed’ by creating a dummy macro with just a “!” (exclamation point) as the entry. This tells composition to end the previous word. Now, since the real word is not the first word anymore, it is free to hyphenate. As a note, I had not seen Mark H. solution before, but it is effectively doing the same thing.

2. If you have one word, but it is followed by computer generated text via the “end string” in XML mode, this generated text can also force the limitation to kick in. In this case, you can put the same “!” (exclamation point) at the beginning of the “end string”. This ends the word prior to processing the end string, so the single word will be able to hyphenate.

Both examples are somewhat workarounds, but they are also ‘features’. When you are generating text before/after a word, its relationship to the text it is attaching to is somewhat ambiguous. Some people want the generated text to ”stick” with the surrounding text, others may allow it to be separated from the text. The ”!” allows user control over the ”attached” text/macros coming out of the pre/end-string. It also helps XPP solve a very sticky (bad pun) problem.

So in case you are still wondering what this all means and how the solution nr 1 really works, I created a test case which I would like to share with you.

Here is a screenshot of my test page:

In the first line all the text is typed.

In the second line the word ‘generated’ is generated by a macro and as you can see XPP does not hyphenate this word and creates a loose line.

In the third line we generate the same text but with a better version of the same macro.

In page mode this all looks like:

The actual macro’s used here look like:

In the ‘better’ version, you need both the <mh;0> and the <endword> in order to work around the problem. The <mh;0> certainly qualifies as the invisible text Steve is referring to.

Is this workaround 100% foolproof?

Well no, this only works correctly if one can be certain that our ‘better_generate_text’ macro is not preceded by a spaceband.

Should the line break just after the spaceband, it will be kept and you will create a short line. So our example is a bad example as can be seen on the slightly modified test case:

Can anybody come up with yet a better solution? (we are looking for the universal answer here!)