%%%% Copyright 2026 by Gerhard Schaden %%%% Standalone reimplementation of the linguex user interface %%%% (original linguex by Wolfgang Sternefeld; interlinear glossing %%%% interface modelled on cgloss4e by Hans-Peter Kolb & Craig Thiersch) %%%% This program can be redistributed and/or modified under the terms %%%% of the LaTeX Project Public License %% 2022/06/01 is where \DeclareKeys, \DeclareUnknownKeyHandler and %% \ProcessKeyOptions entered the kernel; the options below are declared %% with them. expl3 alone would still be satisfied by 2020/10/01, but %% asking for the earlier format only buys a document on a 2020 kernel an %% undefined \ProcessKeyOptions half a file later instead of the "format %% too old" it should get here. README has promised TL2023 or later all %% along, so this states the requirement that was already being made. \NeedsTeXFormat{LaTeX2e}[2022/06/01]% \ProcessKeyOptions + expl3 in kernel \ProvidesPackage{linguexx}[2026/11/03 Standalone linguistic examples, linguex-compatible interface v. 1.5] %% Release history: see CHANGELOG.md, installed beside this file in %% doc/latex/linguexx/, and online at %% https://github.com/ghschaden/linguexx/blob/main/CHANGELOG.md %% %% Package options come in two independent groups. %% %% SYNTAX: %% [lazy] (the default): the traditional linguex dot syntax -- %% \ex., \a.-\f., \z., \exg. -- and nothing else. %% [gb4e]: only the gb4e environment syntax -- exe, xlist, \ex %% (without period, with optional bracketed judgment). In %% this mode the letter commands \b., \c., \d. are never %% defined, so the kernel accent commands \b, \c, \d remain %% untouched and no hyperref workaround is needed. %% [langsci]: the gb4e syntax PLUS the \ea ... \z front-end of %% langsci-gb4e (Language Science Press's fork), where an %% example and its sub-levels are opened by \ea and closed by %% \z, with the depth read off the nesting rather than spelt %% out. Implies [gb4e]. Its purpose is migration: combined %% with [lazy] a document moves from the dot syntax to \ea one %% example at a time, both syntaxes driving one engine and one %% counter. Within a SINGLE example the two may not be mixed %% (see \iflx@inlangsci); between examples they may. %% Incompatible with [legacy], which is a package error rather %% than a silent choice of geometry. %% Both may be requested together, [lazy,gb4e], for documents that %% deliberately mix the syntaxes (this package's own manual does). %% %% DEFAULTS (lengths, dashes, sub-label delimiters): %% none (the default): this package's own defaults -- see %% \lx@defaults@lazy below. %% [legacy]: linguex's defaults, to the value -- see %% \lx@defaults@legacy below. Nothing else changes: the %% engine, the error behaviour and the extensions are the %% same in both modes. [legacy] is orthogonal to the syntax %% options, so [legacy,gb4e] is meaningful (gb4e syntax, %% linguex geometry), and [legacy] alone implies the dot %% syntax, exactly as [lazy] alone does. %% %% RELATIVE REFERENCES: %% [relreflinks] (the default): with hyperref loaded, \Next and \Last %% and their family are clickable, like the \ref they are an %% abbreviation of. Without hyperref the option does nothing %% and costs nothing: no anchor is looked up and no line is %% written to the .aux. %% [norelreflinks]: print the number and never the link, as linguex %% did. For a document that wants its own \hyperref wrapper %% around these, or none at all. \newif\iflx@lazy \newif\iflx@gbfour \newif\iflx@langsci % Set by \ExRaggedRight and read by \lx@ea@rag, both of which live in the % \ea section under \iflx@langsci. The \newif has to be OUT here, though. % With the option off, TeX skips that section by scanning for the matching % \fi, counting conditionals as it goes -- and it counts them by MEANING. % A \newif inside the skipped text never runs, so \iflx@earagged is still % undefined when the skipper meets it, is therefore not counted as a % conditional, and the \fi that closes it closes the SECTION instead: % everything below that line is then executed with the option off. It % cost a green langsci suite and a red everything-else to find, because % the cases that exercise the section are the ones where it is not % skipped. \newif\iflx@earagged % langsci-gb4e's own package options, and the switch its \singlegloss pair % sets. Declared HERE, with the rest of the flags, and not in the \ea % section that reads them, for the reason given just above. \newif\iflx@nojambox \newif\iflx@manualexewidth \newif\iflx@lowerpenalty \newif\iflx@nocgloss \newif\iflx@singlegloss \lx@singleglosstrue \newif\iflx@legacy \newif\iflx@phantomalign \newif\iflx@relreflinks \lx@relreflinkstrue %% The options are declared with the kernel's key interface rather than %% \DeclareOption, which is where the four \keys_define:nn sets further %% down the file already are. Three things about that are deliberate. %% %% VALUELESS, and a value is an ERROR rather than something ignored. %% Every option here is a switch, and the spellings this package accepts %% are exactly the eleven names below -- [legacy], never [legacy=true]. %% Declaring them with .code:n and an empty .default:n is what keeps it %% that way: the key interface would otherwise hand a document %% [legacy=false] and set the switch anyway, reading the value as %% decoration. A key=value option surface may be worth having one day; %% it is not something to acquire by accident, as a side effect of how the %% options happen to be declared. \lx@opt@bare says so out loud. %% %% It has to be \lx@opt@bare and not an expl3 name: \DeclareKeys reads %% its body with the catcodes in force HERE, and this is above the %% package's first \ExplSyntaxOn. Written \__lx_opt_bare:nn the code %% would not merely fail to run -- each _ is a subscript, so TeX reports %% "Missing $ inserted", typesets the name, and the option silently does %% nothing at all. %% %% ORDER. \ProcessOptions ran options in the order they were DECLARED; %% \ProcessKeyOptions runs them in the order they were GIVEN. The pair %% below is the only place that can show: [norelreflinks,relreflinks] %% used to come out linked whichever way round it was written, because %% [relreflinks] is declared first, and now the last one written wins. %% Given order is the better answer -- it is what every other key=value %% interface in the document does -- and a preamble naming both was %% always asking for one of them to be ignored. \newcommand\lx@opt@bare[2]{% \def\lx@opt@given{#2}% \ifx\lx@opt@given\@empty\else \PackageError{linguexx}{The option '#1' takes no value}% {linguexx's package options are switches: write [#1], not [#1=...].\MessageBreak The lengths and fonts are set with commands in the preamble instead -- see the manual.}% \fi} \DeclareKeys[linguexx]{ lazy .code:n = {\lx@opt@bare{lazy}{#1}\lx@lazytrue}, lazy .default:n = {}, gb4e .code:n = {\lx@opt@bare{gb4e}{#1}\lx@gbfourtrue}, gb4e .default:n = {}, % [langsci] is a SUPERSET of the gb4e syntax -- \ea expands to what exe % and xlist write -- so it turns [gb4e] on rather than competing with it. % That also makes the "no syntax option means [lazy]" rule below do the % right thing for [langsci] alone. langsci .code:n = {\lx@opt@bare{langsci}{#1}% \lx@langscitrue\lx@gbfourtrue}, langsci .default:n = {}, % langsci-gb4e's four options, under their own names so that a preamble % being ported does not have to be edited. They are read only under % [langsci]; given without it they are accepted and do nothing, which is % what they would do in a document that has no \ea in it either. nojambox .code:n = {\lx@opt@bare{nojambox}{#1}\lx@nojamboxtrue}, nojambox .default:n = {}, manualexewidth .code:n = {\lx@opt@bare{manualexewidth}{#1}% \lx@manualexewidthtrue}, manualexewidth .default:n = {}, lowerpenalty .code:n = {\lx@opt@bare{lowerpenalty}{#1}% \lx@lowerpenaltytrue}, lowerpenalty .default:n = {}, nocgloss .code:n = {\lx@opt@bare{nocgloss}{#1}\lx@nocglosstrue}, nocgloss .default:n = {}, legacy .code:n = {\lx@opt@bare{legacy}{#1}\lx@legacytrue}, legacy .default:n = {}, phantomalign .code:n = {\lx@opt@bare{phantomalign}{#1}% \lx@phantomaligntrue}, phantomalign .default:n = {}, relreflinks .code:n = {\lx@opt@bare{relreflinks}{#1}% \lx@relreflinkstrue}, relreflinks .default:n = {}, norelreflinks .code:n = {\lx@opt@bare{norelreflinks}{#1}% \lx@relreflinksfalse}, norelreflinks .default:n = {}, } %% An unknown option WARNS and is ignored, as \DeclareOption* did here -- %% not the error the key interface would raise by itself. A document %% carrying an option this package does not know still deserves to be %% typeset. Only options given to \usepackage reach this: a class option %% the package has no key for is left alone by the kernel, so %% \documentclass[a4paper]{article} does not warn. \DeclareUnknownKeyHandler[linguexx]{% \PackageWarning{linguexx}{Unknown option '#1' ignored}} \ProcessKeyOptions[linguexx] \iflx@gbfour\else\lx@lazytrue\fi % [legacy,langsci] is refused rather than approximated. [legacy] is % linguex's geometry to the value, and langsci-gb4e is the house style of a % different publisher with its own; a document asking for both has asked % for two answers to every length in the package, and picking one silently % is how a "compatibility" mode comes to be neither. Nothing else in the % option matrix is affected: [legacy,gb4e] stays meaningful. \iflx@langsci\iflx@legacy \PackageError{linguexx}{[legacy] and [langsci] cannot be combined}% {[legacy] reproduces linguex's geometry; [langsci] follows langsci-gb4e's.\MessageBreak Choose one: drop [legacy] for the langsci front-end, or drop [langsci] and use [gb4e] for the environment syntax under linguex geometry.}% \fi\fi %% graphicx is NOT required here, and the line that required it is gone. %% It was annotated "\scalebox etc." and the package calls \scalebox %% nowhere -- nor \resizebox, \rotatebox, \reflectbox or %% \includegraphics, in this version or in any committed one: the first %% commit already carries the requirement and already has no caller. (The %% v0.13 note in the history above says "graphicx + tikz instead of %% amsmath"; the tikz half is true and load-bearing, the graphicx half was %% not true of anything in the repository.) It survived because tikz %% loads graphicx itself, so nothing a document could do would show the %% difference -- the only way to see it was to look for the callers. %% Should the TikZ brace ever be replaced, whatever replaces it declares %% its own dependencies; that is the point of not declaring them here. %% %% TIKZ ITSELF STAYS, and this is the note saying not to try again. It %% was tried: tikz is loaded for one macro, the drawn brace, and on a %% minimal document under pdflatex it was 0.13s of the 0.15s this package %% cost and 11.5k of its 12.5k control sequences, which is a great deal to %% pay for one macro. The whole brace was reimplemented in pict2e, which %% is a twentieth of the load, and it worked: the same curve to within %% 0.12pt, the full suite green on all three engines, veraPDF green, and %% even the latex+dvips route intact. %% %% It was still the wrong trade, for two reasons that only showed up %% afterwards. The first is what the brace should look like: a stroked %% path has one width everywhere and a typographic brace does not, so the %% brace is drawn as a filled outline now -- and pict2e cannot fill a %% curved path at all (\polygon* takes straight edges), which leaves %% flattening the outline into some 130 sampled points at 0.6s a brace. %% pgf fills Beziers, so the same shape is fourteen segments and no %% sampling. The second is who pays: a linguist's document tends to load %% pgf anyway, through beamer or forest or a tree package, and then the %% saving is zero while the brace is worse. %% %% So the question is settled, not deferred. Reopen it only if the brace %% stops needing a filled curved path -- and then read %% \__lxp_brace:nnn first, since that is the thing being traded away. \RequirePackage{tikz}% drawn brace for \lxAltn (no math mode) \usetikzlibrary{arrows.meta}% the tip of a movement arrow (\mvto) \RequirePackage{xspace}% space after \Last & co. (see below) %% No decorations.pathreplacing: the brace was its "brace" decoration %% until v1.2 and is now drawn as an outline of its own (see %% \__lxp_brace:nnn), which needs nothing but a filled path. % The only other dependency is the expl3 programming layer, part of the % LaTeX kernel since 2020. In particular, ulem is NOT loaded: if you % want struck-through alternatives (\sout inside \altn), load ulem % yourself, with whatever options you prefer. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% Packages this one cannot share a document with %%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% linguexx REPLACES these; it does not accompany them. Each defines the %% same user commands out of the same tradition -- \ex and the dot syntax %% (linguex, expex), the exe/xlist environments (gb4e), \ea (langsci-gb4e) %% -- and all of them, this one included, define \ex with \def. So the %% file read second wins, silently, and the document gets a working %% mixture: the examples go on numbering and what breaks is a detail in %% the middle of it. A document being ported is exactly where this %% happens, because the old \usepackage line is the last thing anyone %% thinks to delete. %% %% Checked twice, because both orders occur and neither is more likely: %% at load time for a package already read, and again at \begin{document} %% for one read after this. %% %% cgloss4e is deliberately NOT in the list. It defines \gll and \glt and %% would collide the same way, but whether a document may keep a foreign %% glossing package is the question doc/DEFERRED-DECISIONS.md leaves open %% under [nocgloss] -- and an error here would answer it by accident, %% which is what that file exists to prevent. \def\lx@clash@one#1{% \@ifpackageloaded{#1}% {\PackageError{linguexx}{The package #1 is loaded as well as linguexx}% {linguexx reimplements #1's interface, so both define the same commands and\MessageBreak whichever is read second silently wins.\MessageBreak Load one of them, not both: linguexx's [gb4e] and [langsci] options provide\MessageBreak the other front-ends on this package's engine.}}% {}} \def\lx@clashcheck{% \lx@clash@one{linguex}% \lx@clash@one{expex}% \lx@clash@one{gb4e}% \lx@clash@one{langsci-gb4e}} \lx@clashcheck \AtBeginDocument{\lx@clashcheck} %% ... and one more for the package that is not one. expex is plain TeX %% and can be read with \input, which leaves nothing for %% \@ifpackageloaded to find; what it does leave is a different \ex. So %% the definition this package installed is remembered at the end of this %% file (\lx@ex@installed) and compared here. A warning and not an error: %% redefining \ex is a document's own business, and only the document %% knows whether it meant to. \AtBeginDocument{% \ifx\ex\lx@ex@installed\else \PackageWarning{linguexx}{% \string\ex\space is no longer linguexx's: something has redefined it since\MessageBreak the package was loaded (another example package, perhaps read\MessageBreak with \string\input). The dot syntax will now do whatever that\MessageBreak definition does}% \fi} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% Counters and number formatting (linguex-compatible names) %%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \newcounter{ExNo} \newcounter{SubExNo} \newcounter{SubSubExNo} \newcounter{FnExNo} % Anchor serials, reset by nothing: what the hyperref anchors are built % from, so that two examples never claim one destination. The printed % numbers (ExNo, FnExNo) can be reset -- [legacy]'s per-chapter reset, % \setcounter -- and a custom label steps no number at all; anchors built % from the printed number collided in both cases. One serial per series, % so that in a document with neither, every anchor is the one it always % was: lxex. for example (n), lxfnex. for footnote example (n). % Custom-labelled examples have a series of their own (lxcex.), so % that their sub-examples' anchors are distinct too. See the anchors % below, and doc/DEFERRED-DECISIONS.md ("Two examples, one hyperref % anchor"), decided 2026-10-03 for this option. \newcounter{lxExSerial} \newcounter{lxFnExSerial} \newcounter{lxCustomSerial} % It counts from 2^30, in steps of two (see \lx@custom@begin), and % \lx@customserial gives back 1, 2, 3 ... for the anchor names. The % offset is for cleveref, which reads the counter's VALUE: it puts a % custom-labelled and a numbered example in one group (both have empty % names, so identical formats) and sorts and compresses that group by % value, so a custom example whose serial was 2 sorted between examples % (1) and (3) and vanished into "(1) to (3)". No example number gets % near 2^30, so the two never form a range, and custom examples simply % come after the numbered ones in a list. \setcounter{lxCustomSerial}{1073741824} \def\lx@customserial{\the\numexpr(\value{lxCustomSerial}-1073741824)/2\relax} \newif\iflx@customex % beamer sets a frame once per overlay slide, restoring the counters it was % told about -- linguex's ExNo, SubExNo and SubSubExNo, which beamer % registers itself -- so that one example keeps one number on every slide. % The serials have to be restored the same way, or each slide gives the % same example a new anchor (tests/relreflinks-beamer.tex). beamer is a % class, loaded before this package, so \resetcounteronoverlays exists by % now if it ever will. \@ifundefined{resetcounteronoverlays}{}{% \resetcounteronoverlays{lxExSerial}% \resetcounteronoverlays{lxFnExSerial}% \resetcounteronoverlays{lxCustomSerial}} % linguex resets ExNo at every \chapter in a class that has chapters. % That is a numbering CONVENTION, not a bug, so [legacy] reproduces it; % the default is continuous numbering through the document. \iflx@legacy \@ifundefined{c@chapter}{}{\@addtoreset{ExNo}{chapter}} \fi % back-compat aliases (linguex spelt these with a prefix) \let\Exarabic\arabic \let\Exalph\alph \let\Exroman\roman %% \Exalph is also THE place to choose sub-example letters, labels and %% references alike, and the manual documents it as such. It follows %% \alph, and so the document's language: polyglossia's Hebrew gives %% א, ב. An author who wants a, b whatever the language says %% \let\Exalph\ExLatinAlph. It is built on \int_to_alph:n because \alph %% is no way back -- polyglossia redefines the \@alph underneath it, so %% the kernel's Latin letters are not reachable by name from a Hebrew %% document. Expandable, as \alph is: a label writes it to the .aux. \ExplSyntaxOn \cs_new:Npn \ExLatinAlph #1 { \int_to_alph:n { \value {#1} } } \ExplSyntaxOff % Reference dashes and sub-label delimiters. Only the NAMES are reserved % here; the values are set by the defaults block in the Layout section % (\lx@defaults@lazy / \lx@defaults@legacy), so that both modes go through % one place. \firstrefdash separates number from letter in a reference, % \secondrefdash letter from roman numeral: (12a-i) by default, (12-a-i) % under [legacy]. The delimiters wrap the PRINTED sub-example labels: % "a." and "i." by default, "a." and "(i)" under [legacy]. \newcommand\firstrefdash{} \newcommand\secondrefdash{-} \newcommand\SubExLBr{} \newcommand\SubExRBr{.} \newcommand\SubSubExLBr{} \newcommand\SubSubExRBr{.} % parenthesis suppression switch (v1 machinery, kept verbatim in spirit) \newif\ifparens\parensfalse % The delimiters around a top-level example number, in two layers. % % linguex has \ExLBr / \ExRBr (and the footnote pair) since its version % 4.0 and documents them: a document that wants [1] rather than (1) % writes \renewcommand{\ExLBr}{[}. This package printed its parentheses % from \theExLBr alone, so that line was accepted and did nothing -- the % one kind of incompatibility a drop-in replacement cannot have, because % the document still compiles and only the page is wrong. % % So the characters live in the linguex names, and the \the... layer -- % which is what \theExNo expands and what a \newlabel record carries -- % reads them through the suppression switch. Both remain redefinable and % mean different things: \ExLBr is the character, \theExLBr the % character plus the switch. \newcommand\ExLBr{(} \newcommand\ExRBr{)} \newcommand\FnExLBr{(} \newcommand\FnExRBr{)} \newcommand\theExLBr{\ifparens\else\ExLBr\fi} \newcommand\theExRBr{\ifparens\else\ExRBr\fi} \newcommand\theFnExLBr{\ifparens\else\FnExLBr\fi} \newcommand\theFnExRBr{\ifparens\else\FnExRBr\fi} % "am I in a footnote?" -- linguex's switch name kept for compatibility; % TRUE means NOT in a footnote (sic, as in linguex) \newif\if@noftnote\@noftnotetrue %% A footnote is a NEW stream, and the example it interrupts is not open %% inside it. \@noftnotefalse says so for the numbering; the three %% "an example is open" flags have to say so for the syntax, or a footnote %% hung on an example cannot hold an example of its own written the other %% way -- \ea in a footnote of an \ex. example would be read as \ea inside %% that example and refused by the one-syntax-per-example rule. %% %% All three are local to the group the footnote text is typeset in, so %% nothing leaks back to the interrupted example. Every real opener %% (\ex., \begin{exe}, \ea) resets \lx@subdepth for itself, so it is left %% alone here: the only construct that would read it is a stray \a. in a %% footnote, which is not an example in either syntax and is no better %% defined for being given a depth of zero. \AtBeginDocument{% \let\lx@orig@footnotetext\@footnotetext \long\def\@footnotetext#1{\lx@orig@footnotetext {\@noftnotefalse \lx@inexamplefalse \lx@inexefalse \lx@inlangscifalse #1}}} %% What \ref prints, in two flavours, and where the parentheses live. %% %% linguex puts them in the counter format: \theExNo IS "(1)", so \ref %% gives "(1)" and the label prints the same thing. langsci-gb4e (and gb4e %% under it) puts them in the LIST LABEL instead and leaves the counter %% bare, so \ref there gives "1" and an author writes "(\ref{ex:x})" or %% \xref -- which is why upstream's \xref is "(\ref{#1})" while this %% package's, back when \ref always parenthesised, was a plain \ref. %% Neither convention is more correct; but a document ported from one to %% the other must not silently print "((1))". %% %% So the flavour follows the syntax the document is written in: [langsci] %% selects the bare one, everything else the parenthesised one. Each is %% installed by one macro, and \ExParenRefs / \ExBareRefs let a document %% ask for the other by name -- a [langsci] document whose prose already %% says "as in \ref{ex:x}" wants \ExParenRefs and one line, not a pass over %% its cross-references. %% %% What must NOT depend on the flavour is anything this package prints %% itself. The example's own number, \Next and its family, \refrange, %% \Refrange, \xref, \xxref and \exp all spell the parentheses out -- %% through \theExLBr / \theExRBr, around the bare number \pref gives -- so %% they print the same characters either way; only a plain \ref, and the %% \cref that follows it, change. \lx@labelExNo is the seam for the main %% label: it is the printed number, parenthesised in both flavours, and it %% is \theExNo in only one of them. \def\lx@refs@paren{% \renewcommand{\theExNo}{\protect\theExLBr\arabic{ExNo}\protect\theExRBr}% \renewcommand{\theFnExNo}{\protect\theFnExLBr\roman{FnExNo}\protect\theFnExRBr}% \renewcommand{\theSubExNo}{% \hbox{\if@noftnote\protect\theExLBr\lx@refbase@main\firstrefdash \Exalph{SubExNo}\protect\theExRBr \else \protect\theFnExLBr\lx@refbase@fn\firstrefdash% \Exalph{SubExNo}\protect\theFnExRBr \fi}}% \renewcommand{\theSubSubExNo}{% \hbox{\if@noftnote\protect\theExLBr% \lx@refbase@main\firstrefdash\Exalph{SubExNo}\secondrefdash \Exroman{SubSubExNo}\protect\theExRBr% \else\protect\theFnExLBr\lx@refbase@fn\firstrefdash \Exalph{SubExNo}\secondrefdash\Exroman{SubSubExNo}\protect\theFnExRBr\fi}}% \def\lx@labelExNo{\theExNo}% \def\lx@labelFnExNo{\theFnExNo}% \def\lx@labelCustom##1{\if@noftnote\protect\theExLBr##1\protect\theExRBr \else\protect\theFnExLBr##1\protect\theFnExRBr\fi}} \def\lx@refs@bare{% \renewcommand{\theExNo}{\arabic{ExNo}}% \renewcommand{\theFnExNo}{\roman{FnExNo}}% \renewcommand{\theSubExNo}{% \hbox{\if@noftnote\lx@refbase@main\firstrefdash\Exalph{SubExNo}% \else\lx@refbase@fn\firstrefdash\Exalph{SubExNo}\fi}}% \renewcommand{\theSubSubExNo}{% \hbox{\if@noftnote \lx@refbase@main\firstrefdash\Exalph{SubExNo}\secondrefdash \Exroman{SubSubExNo}% \else \lx@refbase@fn\firstrefdash\Exalph{SubExNo}\secondrefdash \Exroman{SubSubExNo}\fi}}% \def\lx@labelExNo{\protect\theExLBr\theExNo\protect\theExRBr}% \def\lx@labelFnExNo{\protect\theFnExLBr\theFnExNo\protect\theFnExRBr}% \def\lx@labelCustom##1{##1}} %% The number a sub-example's reference starts with: the example's printed %% number, or -- under a custom label, which steps no counter -- the label %% itself, its brackets taken off (\lx_refbase_custom:n). Built from ExNo %% unconditionally, a sub-example of \ex.[(7)] referred to itself as %% "(0a)": ExNo had not moved. Set by every example at its start, so it %% never outlives the example it belongs to. \def\lx@refbase@main{\Exarabic{ExNo}} \def\lx@refbase@fn{\Exroman{FnExNo}} \newcommand\ExParenRefs{\lx@refs@paren} \newcommand\ExBareRefs{\lx@refs@bare} \iflx@langsci \lx@refs@bare \else \lx@refs@paren \fi % hyperref anchor names (avoid the \protect-laden \the... expansions). % % The sub-levels must branch on \if@noftnote exactly as the PRINTED % labels (\theSubExNo, \theSubSubExNo) do. Building them from ExNo % unconditionally made a footnote sub-example and a main-text one collide % whenever the footnote sat under the same ExNo: both claimed % "lxex..a", hyperref kept the first destination and dropped the % second, and a \ref to the footnote sub-example jumped to the main-text % one. The number printed was right either way, so only the anchor gave % it away. \if@noftnote is a plain \newif, hence fully expandable and % safe in the \edef that hyperref runs over \theH... . % % Built from the SERIALS (see the counters), not from the printed numbers: % a reset ExNo, or a custom label (which steps none), made two examples % claim one anchor. The custom series has its own stem; \iflx@customex is % a plain \newif, expandable like \if@noftnote. \def\theHExNo{lxex.\arabic{lxExSerial}} \def\lx@Hexstem{\iflx@customex lxcex.\lx@customserial% \else\if@noftnote lxex.\arabic{lxExSerial}\else lxfnex.\arabic{lxFnExSerial}\fi\fi} \def\theHSubExNo{\lx@Hexstem.\alph{SubExNo}} \def\theHSubSubExNo{\lx@Hexstem.\alph{SubExNo}.\arabic{SubSubExNo}} \def\theHFnExNo{lxfnex.\arabic{lxFnExSerial}} \def\theHlxCustomSerial{lxcex.\lx@customserial} % The same anchors as \hyperlink TARGETS. hyperref prefixes the name it % builds from \theH with the counter's own name, so the % destination of main example 5 is "ExNo.lxex.5" and that of footnote % example 5 is "FnExNo.lxfnex.5". These two spell that name for a % NUMBER rather than for the counter's current value, which is what a % relative reference needs and \theHExNo cannot give: \Next has to name % an example the document has not reached. They live here, beside the % \theH... definitions they mirror, so that the two spellings of one % anchor are edited together; nothing silently depends on their % agreeing, though -- see \lx_relref_record: for why a disagreement % costs a link and never a wrong jump. \newcommand\lx@Hexname[1]{ExNo.lxex.#1} \newcommand\lx@Hfnexname[1]{FnExNo.lxfnex.#1} % Since the anchors are built from serials, #1 above is a SERIAL, and a % relative reference -- which knows a printed number -- cannot spell the % anchor it wants. It names the number instead (these keys), and the % .aux says which anchor the example printing that number recorded. \newcommand\lx@Hexkey[1]{ExNo.#1} \newcommand\lx@Hfnexkey[1]{FnExNo.#1} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% Layout parameters %%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% The geometry is the same in both modes -- each level reserves a label %% box, the text margin of a level is the label box plus a gap, measured %% from the margin of the level above -- but the two modes PARAMETRIZE it %% differently, because linguex did: %% %% default: the sub-levels are given by their LABEL WIDTHS %% (\SubExlabelwidth, \SubSubExlabelwidth), each followed by %% the shared gap \Exlabelsep; %% legacy: the sub-levels are given by their TEXT MARGINS %% (\SubExleftmargin, \SubSubExleftmargin), the label being set %% flush left inside that margin with no gap, and the main %% \Exlabelwidth is recomputed for every example from the width %% of the number itself. %% %% All the lengths of both sets exist in both modes; only the ones its own %% mode reads have any effect. The equivalence is %% \SubExleftmargin = \SubExlabelwidth + \Exlabelsep. \newlength{\Extopsep} \newlength{\Exredux} \newlength{\Exindent} \newlength{\Exlabelwidth} \newlength{\Exlabelsep} % The sub-example label boxes are wider than the letters strictly need. % The surplus is the room a hanging judgment falls into: the clear space % left of the text is (labelwidth - width of the printed letter) + % \Exlabelsep. The defaults are chosen so that TWO marks (e.g. "??", % "?*") fit without touching the letter; for three or more as a routine % matter, widen these. % % "Widen these" means the AUTHOR widens them. Three marks (\a. ??\#) % overrun the box and the marks cross the letter -- in both default sets % and on all three engines -- and that is not to be fixed here: these % numbers are not the package's to raise on behalf of a document it % cannot see. See doc/DEFERRED-DECISIONS.md, "Three judgment marks % running into the sub-example letter", for the measurement and the % ruling. \newlength{\SubExlabelwidth} \newlength{\SubSubExlabelwidth} % linguex's names for the same two levels (see above) \newlength{\SubExleftmargin} \newlength{\SubSubExleftmargin} % \Extopsep and \Exredux are the only lengths that depend on the font: % they are set here (so that a \setlength in the PREAMBLE overrides them, % which it cannot do in linguex) and re-derived \AtBeginDocument if -- and % only if -- they still hold the value computed here, which catches the % case of a font package that changes \baselineskip after we are loaded. \newlength{\lx@auto@topsep} \newlength{\lx@auto@redux} \newcommand\lx@setskips{% \setlength{\Extopsep}{.66\baselineskip}% \setlength{\Exredux}{\lx@reduxfactor\baselineskip}% \setlength{\lx@auto@topsep}{\Extopsep}% \setlength{\lx@auto@redux}{\Exredux}} \AtBeginDocument{% \ifdim\Extopsep=\lx@auto@topsep \ifdim\Exredux=\lx@auto@redux \lx@setskips \fi \fi} %% ---- the two sets of defaults ------------------------------------------- \newcommand\lx@defaults@lazy{% \def\lx@reduxfactor{-.66}% \lx@setskips \setlength{\Exindent}{0pt}% \setlength{\Exlabelwidth}{2.6em}% \setlength{\Exlabelsep}{.6em}% \setlength{\SubExlabelwidth}{1.6em}% \setlength{\SubSubExlabelwidth}{1.6em}% \setlength{\SubExleftmargin}{\dimexpr\SubExlabelwidth+\Exlabelsep\relax}% \setlength{\SubSubExleftmargin}{\dimexpr\SubSubExlabelwidth+\Exlabelsep\relax}% \setlength{\JdgSep}{0.15em}% \lx@setannotcol \setlength{\ExAnnotSep}{1em}% \setlength{\GlossTransSep}{2em}% \setlength{\GlossTransMinWidth}{6em}% \setlength{\GlossTransRightSkip}{0pt plus 2em}% \def\firstrefdash{}% \def\secondrefdash{-}% \def\SubExLBr{}\def\SubExRBr{.}% \def\SubSubExLBr{}\def\SubSubExRBr{.}% \def\GlossSep{.5em plus .3em minus .1em}} %% linguex's defaults, to the value: \Exlabelsep 1.3em, \Exindent 0pt, %% \SubExleftmargin 2em, \SubSubExleftmargin 2.4em, \Extopsep %% .66\baselineskip, \Exredux -\baselineskip, and a main label box whose %% width is that of the current number, padded to the next digit (so "(1)" %% sits in a two-digit box). Judgments are flush against the text %% (\JdgSep 0pt), sub-sub-examples print as "(i)", and references are %% (12-a), (12-a-i). \GlossSep approximates cgloss4e's word spacing (an %% interword space plus its \glossglue), which is tighter than ours. \newcommand\lx@defaults@legacy{% \def\lx@reduxfactor{-1}% \lx@setskips \setlength{\Exindent}{0pt}% \setlength{\Exlabelsep}{1.3em}% \setlength{\SubExleftmargin}{2em}% \setlength{\SubSubExleftmargin}{2.4em}% \setlength{\SubExlabelwidth}{\dimexpr\SubExleftmargin-\Exlabelsep\relax}% \setlength{\SubSubExlabelwidth}{\dimexpr\SubSubExleftmargin-\Exlabelsep\relax}% \setlength{\Exlabelwidth}{4em}% recomputed per example; see \lx@calc@Exlabelwidth \setlength{\JdgSep}{0pt}% \lx@setannotcol \setlength{\ExAnnotSep}{1em}% \setlength{\GlossTransSep}{2em}% \setlength{\GlossTransMinWidth}{6em}% \setlength{\GlossTransRightSkip}{0pt plus 2em}% \def\firstrefdash{-}% \def\secondrefdash{-}% \def\SubExLBr{}\def\SubExRBr{.}% \def\SubSubExLBr{(}\def\SubSubExRBr{)}% \def\GlossSep{.33em plus .4em minus .2em}} % Which of the two is in force is decided by \lx_mode_select:n, at the end % of the geometry section below -- the mode is a defaults table AND a set % of geometry hooks, and it is chosen once, when both halves exist. % \resetExdefaults is CALLED at the end of the package, once every length % it touches has been declared; the user may call it again at any point to % return to the defaults of the mode in force. %% ---- the geometry hooks ------------------------------------------------- %% Each level's list declaration calls one of these; they are the ONLY %% place where the two parameter sets differ. % width of the narrowest digit, as linguex measured it (fonts in which the % digits differ in width would otherwise pad inconsistently) \newlength{\lx@digitwd} \newlength{\lx@mindigitwd} \newlength{\lx@currentlabel} \newlength{\lx@padded} \def\lx@minwidth#1{\settowidth{\lx@digitwd}{#1}% \ifdim\lx@digitwd<\lx@mindigitwd \lx@mindigitwd\lx@digitwd \fi} % \Exlabelwidth := width of the smallest n-digit box (n = 2,3,4) that the % current label fits INSIDE; if it fits none, its own width. This is % linguex's rule, and the reason a one-digit example number sits in a box % wide enough for two: the numbering does not shift the text as it grows. \def\lx@calc@Exlabelwidth{% \settowidth{\lx@mindigitwd}{0}% \lx@minwidth{1}\lx@minwidth{2}\lx@minwidth{3}\lx@minwidth{4}% \lx@minwidth{5}\lx@minwidth{6}\lx@minwidth{7}\lx@minwidth{8}% \lx@minwidth{9}% \settowidth{\lx@currentlabel}{\lx@itemlabel}% \Exlabelwidth\lx@currentlabel \settowidth{\lx@padded}{\theExLBr\hbox to 4\lx@mindigitwd{}\theExRBr}% \ifdim\lx@currentlabel<\lx@padded \Exlabelwidth\lx@padded \fi \settowidth{\lx@padded}{\theExLBr\hbox to 3\lx@mindigitwd{}\theExRBr}% \ifdim\lx@currentlabel<\lx@padded \Exlabelwidth\lx@padded \fi \settowidth{\lx@padded}{\theExLBr\hbox to 2\lx@mindigitwd{}\theExRBr}% \ifdim\lx@currentlabel<\lx@padded \Exlabelwidth\lx@padded \fi} %% Both sets are defined unconditionally, under names of their own, and one %% of the two is then selected. Nothing is saved by defining only the set %% in force, and having both present is what makes the mode a THING that %% can be named, rather than a branch taken once at load time. % legacy main level: label box recomputed from the number; text margin % \Exindent + \Exlabelwidth + \Exlabelsep, as in the default mode \def\lx@geom@main@legacy{\lx@calc@Exlabelwidth \labelwidth\Exlabelwidth \labelsep\Exlabelsep \leftmargin\dimexpr\Exindent+\Exlabelwidth+\Exlabelsep\relax \if@noftnote\else\addtolength{\topsep}{-.5\topsep}\fi} % legacy sub-levels: the text margin IS the parameter, the label sits % flush left inside it \def\lx@geom@sub@legacy{\leftmargin\SubExleftmargin \labelwidth\SubExleftmargin \labelsep\z@ \topsep.3\Extopsep} \def\lx@geom@subsub@legacy{\leftmargin\SubSubExleftmargin \labelwidth\SubSubExleftmargin \labelsep\z@ \topsep\z@} % Only the default mode declares a /ListNumbering class per level; the % legacy hooks above deliberately do not, so a legacy list keeps the block % code's default (/None). That is valid -- what is NOT valid is /Ordered, % which is why neither mode routes through the enumerate class. \def\lx@geom@main@default{% \lx@ol@set{lxOLdecimal}% \labelwidth\Exlabelwidth \labelsep\Exlabelsep \leftmargin\dimexpr\Exindent+\Exlabelwidth+\Exlabelsep\relax} \def\lx@geom@sub@default{% \expandafter\lx@ol@set\expandafter{\lx@sub@olclass}% \labelwidth\SubExlabelwidth \labelsep\Exlabelsep \leftmargin\dimexpr\SubExlabelwidth+\Exlabelsep\relax \topsep\z@} \def\lx@geom@subsub@default{% \expandafter\lx@ol@set\expandafter{\lx@subsub@olclass}% \labelwidth\SubSubExlabelwidth \labelsep\Exlabelsep \leftmargin\dimexpr\SubSubExlabelwidth+\Exlabelsep\relax \topsep\z@} %%%% --- internal names ------------------------------------------------- %% Four prefixes, and which one a thing carries is not arbitrary. %% %% \lx@... the LaTeX2e-style front end: the dot syntax and the %% environments, their peeking and body collection, the %% geometry tables above, the option glue. \def and %% \newcommand, and always OUTSIDE an \ExplSyntaxOn block %% -- there is no \def anywhere inside one in this file, %% and keeping it that way is what stops a body that %% depends on significant spaces (a glue specification, a %% delimiter) from being read with them stripped. %% \lx_... the public expl3 interface -- \lx_mode_new:nn, %% \lx_mode_select:n, \lx_relref_anchor:, \lx_ol_class:n. %% A front end other than this file's may call these. %% \__lx_... private expl3, the engine: body loops, the glossing %% grid, judgment marks, mode selection, splitting. %% \__lxp_... private expl3, the PARADIGM code and nothing else -- %% \altn, \altg, and the TikZ brace they are drawn with. %% The p is for "paradigm", which is the word this file %% uses throughout for a stack of alternatives sharing a %% brace; the module is worth its own prefix because the %% two-call \altg protocol carries state between object %% and gloss tier (\g__lxp_altg_role_int) and nothing %% outside it may touch that. %% %% The split between \__lx_ and \__lxp_ is therefore a module boundary, %% not a vintage: a new name belongs to \__lxp_ if the alternatives code %% owns it, and to \__lx_ otherwise. %%%% --- public API (Protocol B: geometry modes) ------------------------ %% A mode is a defaults table plus three geometry hooks -- and nothing %% else: [legacy] differs from the default in exactly these four macros. %% %% \lx_mode_new:nn {name} %% { defaults = \defaults , main = \main , sub = \sub , subsub = \subsub } %% \lx_mode_select:n {name} %% %% Keyval rather than an arity-encoded signature (\lx_mode_new:nNNNN) so %% that a mode may grow a fifth thing to set without the function that %% declares it having to change its name. %% %% Every value is a MACRO NAME, not a body, and this is load bearing: a %% defaults table contains glue with significant spaces (".5em plus .3em %% minus .1em"), and a body written as a braced argument under %% \ExplSyntaxOn would have those spaces stripped. Defining the macros %% with plain \newcommand, in whatever catcode regime suits, and handing %% over the names sidesteps the question entirely -- so a value that is %% not a single control sequence is refused rather than stored. %% %% "defaults" is required: it is called by \resetExdefaults, and must set %% every length of BOTH parameter sets, since the user may switch modes %% afterwards. The three hooks are optional and default to doing nothing; %% they are called inside the list declaration of their level, and may set %% \labelwidth, \labelsep, \leftmargin and \topsep; "main" runs AFTER %% \lx@itemlabel is fixed, so it may size the label box from the label %% (which is what legacy does). %% %% \lx_mode_select:n assigns locally, so a mode chosen inside a group is %% undone at the end of it. \ExplSyntaxOn %% the mode in force, for a front-end that needs to know (read only) \tl_new:N \l_lx_mode_tl \msg_new:nnnn { linguexx } { unknown-mode } { Geometry~mode~'#1'~is~not~declared. } { Declare~it~with~\iow_char:N\\lx_mode_new:nn~before~selecting~it.~ The~modes~this~package~defines~are~'default'~and~'legacy'. } \msg_new:nnnn { linguexx } { mode-no-defaults } { Geometry~mode~'#1'~gives~no~'defaults'~macro. } { A~mode~must~say~what~\iow_char:N\\resetExdefaults~restores:~write~ \iow_char:N\\lx_mode_new:nn{#1}{defaults=\iow_char:N\\myDefaults,~...}.~ The~mode~was~not~declared. } \msg_new:nnnn { linguexx } { mode-needs-cs } { Value~of~'#1'~in~a~geometry~mode~is~not~a~macro~name. } { Pass~the~NAME~of~a~macro~(#1=\iow_char:N\\myHook),~not~its~body:~a~ body~written~here~would~lose~the~spaces~of~a~glue~specification.~ '#1'~was~ignored. } %% The scratch the keys fill, so that a mode is committed only once its %% four macros are known and "defaults" among them. \cs_new_eq:NN \l__lx_mode_defaults_cs \prg_do_nothing: \cs_new_eq:NN \l__lx_mode_main_cs \prg_do_nothing: \cs_new_eq:NN \l__lx_mode_sub_cs \prg_do_nothing: \cs_new_eq:NN \l__lx_mode_subsub_cs \prg_do_nothing: \bool_new:N \l__lx_mode_given_bool \cs_new_protected:Npn \__lx_mode_key:Nnn #1#2#3 { \tl_if_single_token:nTF {#3} { \token_if_cs:NTF #3 { \cs_set_eq:NN #1 #3 } { \msg_error:nnn { linguexx } { mode-needs-cs } {#2} } } { \msg_error:nnn { linguexx } { mode-needs-cs } {#2} } } \keys_define:nn { lx / mode } { defaults .code:n = { \__lx_mode_key:Nnn \l__lx_mode_defaults_cs { defaults } {#1} \bool_set_true:N \l__lx_mode_given_bool } , main .code:n = { \__lx_mode_key:Nnn \l__lx_mode_main_cs { main } {#1} } , sub .code:n = { \__lx_mode_key:Nnn \l__lx_mode_sub_cs { sub } {#1} } , subsub .code:n = { \__lx_mode_key:Nnn \l__lx_mode_subsub_cs { subsub } {#1} } , } \cs_new_protected:Npn \lx_mode_new:nn #1#2 { \cs_set_eq:NN \l__lx_mode_defaults_cs \prg_do_nothing: \cs_set_eq:NN \l__lx_mode_main_cs \prg_do_nothing: \cs_set_eq:NN \l__lx_mode_sub_cs \prg_do_nothing: \cs_set_eq:NN \l__lx_mode_subsub_cs \prg_do_nothing: \bool_set_false:N \l__lx_mode_given_bool \keys_set:nn { lx / mode } {#2} \bool_if:NTF \l__lx_mode_given_bool { \cs_set_eq:cN { lx@mode@defaults@#1 } \l__lx_mode_defaults_cs \cs_set_eq:cN { lx@mode@main@#1 } \l__lx_mode_main_cs \cs_set_eq:cN { lx@mode@sub@#1 } \l__lx_mode_sub_cs \cs_set_eq:cN { lx@mode@subsub@#1 } \l__lx_mode_subsub_cs } { \msg_error:nnn { linguexx } { mode-no-defaults } {#1} } } %% Tested on the defaults slot, the one a failed declaration never creates. \cs_new_protected:Npn \lx_mode_select:n #1 { \cs_if_exist:cTF { lx@mode@defaults@#1 } { \cs_set_eq:Nc \resetExdefaults { lx@mode@defaults@#1 } \cs_set_eq:Nc \lx@geom@main { lx@mode@main@#1 } \cs_set_eq:Nc \lx@geom@sub { lx@mode@sub@#1 } \cs_set_eq:Nc \lx@geom@subsub { lx@mode@subsub@#1 } \tl_set:Nn \l_lx_mode_tl {#1} } { \msg_error:nnn { linguexx } { unknown-mode } {#1} } } %% The two modes this package defines, and the choice between them. Both %% the declarations and the selection have to happen under \ExplSyntaxOn: %% outside it "_" is catcode 8 and \lx_mode_new:nn is not a control %% sequence at all. Passing names rather than bodies is what makes that %% harmless -- no key value here is anything but a control sequence, so %% the ignored spaces of expl3 catcodes cost nothing. \lx_mode_new:nn {default} { defaults = \lx@defaults@lazy , main = \lx@geom@main@default , sub = \lx@geom@sub@default , subsub = \lx@geom@subsub@default , } \lx_mode_new:nn {legacy} { defaults = \lx@defaults@legacy , main = \lx@geom@main@legacy , sub = \lx@geom@sub@legacy , subsub = \lx@geom@subsub@legacy , } \iflx@legacy \lx_mode_select:n {legacy} \else \lx_mode_select:n {default} \fi \ExplSyntaxOff %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% The tagged-Span idiom, in one place %%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Everything this package adds to the structure tree is a Span: the %% judgment mark with its spoken /Alt, the Leipzig abbreviation with its %% /E, the gloss column, the object-language word with its /Lang, the free %% translation with its own, and the two kinds of stacked alternatives. %% All of them need the SAME four moves in the SAME order, and getting the %% order wrong is exactly how v0.10 broke PDF/UA: %% %% \tag_mc_end_push: suspend whatever marked content is open -- %% typically the ambient paragraph's, since the %% kernel may already have opened an MC for a P %% that has not been shown yet. Opening our BDC %% inside that one without suspending it is %% "nested marked content found" / "no mc to end", %% and a veraPDF untagged-content failure once the %% P's real MC never gets closed. %% \tag_struct_begin:n the Span itself, carrying alt/E/lang. %% \tag_mc_begin:n its own marked content, for LEAF content. %% ... content ... %% \tag_mc_end: \tag_struct_end: \tag_mc_begin_pop:n {} -- unwound in %% the mirror order, resuming the suspended MC. %% %% The split into an outer (open/close) and an inner (begin/end) pair is %% not decoration: a Span whose content carries marked content of its OWN %% -- the gloss column, whose words each open one -- must take the outer %% pair only, or it would nest an MC inside its own MC. %% %% \lx@tag@if@active: is the one guard, and it is not only for Spans: every %% tagging use site in the package needs the same two conditions (a kernel %% that has the \tag_... commands at all, and tagging actually switched on). %% They were hand-copied in five spellings, two of which tested %% \cs_if_exist:N on \tag_if_active:T -- a conditional VARIANT rather than %% the base name, which happens to work but tests the wrong thing. \ExplSyntaxOn \prg_new_conditional:Npnn \lx@tag@if@active: { p , TF , T , F } { \bool_lazy_all:nTF { { \cs_if_exist_p:N \tag_struct_begin:n } { \cs_if_exist_p:N \tag_if_active_p: } { \tag_if_active_p: } } { \prg_return_true: } { \prg_return_false: } } %% outer half: suspend the ambient MC, open the Span \cs_new_protected:Npn \lx@tag@span@open:n #1 { \tag_mc_end_push: \tag_struct_begin:n {#1} } \cs_generate_variant:Nn \lx@tag@span@open:n { e } \cs_new_protected:Npn \lx@tag@span@close: { \tag_struct_end: \tag_mc_begin_pop:n {} } %% ... plus the inner half, for a Span holding leaf content \cs_new_protected:Npn \lx@tag@span@begin:n #1 { \lx@tag@span@open:n {#1} \tag_mc_begin:n { tag = Span } } \cs_generate_variant:Nn \lx@tag@span@begin:n { e } \cs_new_protected:Npn \lx@tag@span@end: { \tag_mc_end: \lx@tag@span@close: } %% the whole thing, guarded: #1 = keyvals, #2 = content. Without active %% tagging the content is typeset bare, so untagged output and engines %% with no tagging support are byte-for-byte unaffected. \cs_new_protected:Npn \lx@tag@span:nn #1#2 { \lx@tag@if@active:TF { \lx@tag@span@begin:n {#1} #2 \lx@tag@span@end: } { #2 } } %% ... and the same with the keyvals expanded first (a /E or /Alt built in %% a token list). Deliberately not \cs_generate_variant: the expansion has %% to happen INSIDE the guard. The values are author-supplied strings, and %% expanding one on a run with no tagging to consume it would make an %% untagged compile fail where the tagged one is what carries the risk. \cs_new_protected:Npn \lx@tag@span@exp:nn #1#2 { \lx@tag@if@active:TF { \lx@tag@span@begin:e {#1} #2 \lx@tag@span@end: } { #2 } } %% The e-variants above are generated for OUR OWN commands, which always %% exist. Generating one for \tag_struct_begin:n (as v1.1 did) contradicts %% its own use sites: they all guard for a kernel that does not have the %% \tag_... commands, on which the variant generation would already have %% failed at load time. %% --- public API (Protocol A: tagging) -------------------------------- %% Aliases, not redefinitions: \cs_new_eq:NN copies the meaning and with %% it the \protected status, so a public name is the private one in every %% respect. See the "For front-end authors" section of the manual. \prg_new_eq_conditional:NNn \lx_tag_if_active: \lx@tag@if@active: { TF , T , F } \cs_new_eq:NN \lx_tag_span_open:n \lx@tag@span@open:n \cs_new_eq:NN \lx_tag_span_close: \lx@tag@span@close: \cs_new_eq:NN \lx_tag_span_begin:n \lx@tag@span@begin:n \cs_new_eq:NN \lx_tag_span_end: \lx@tag@span@end: \cs_new_eq:NN \lx_tag_span:nn \lx@tag@span:nn \ExplSyntaxOff %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% Judgment auto-detection (* ? \# \%) -- replaces linguex's %%%% %%%% hardcoded catcode tokenizer %%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% A judgment typed at the very start of an example or sub-example %% (\ex. *Sentence / \a. ??\%Sentence) is collected greedily and set %% flush right in the LABEL area, so example texts align whether or not %% they carry a judgment. Recognized: the characters * and ? and the %% control sequences \# and \%. Anything else: use \jdg{...}, which %% works anywhere and takes arbitrary marks. %% %% "The very start" is not the same thing as "the first token". A label %% is typed at the head of the example far more often than after its %% first word -- %% %% \ex. \label{ex:2}*?La sola invasione romana della Tracia %% %% -- and \label makes no ink, so the star is still the first thing on %% the line and the reader still expects it in the gutter. Left alone %% the scan stopped at \label, the marks stayed in the body, and the %% example was indented by their width: the one thing this whole %% mechanism exists to prevent, appearing only in the examples that %% happen to be cross-referenced. %% %% Such a command is therefore SKIPPED and REPLAYED rather than stopped %% at. Skipped: it is taken out of the input so the loop can go on to %% the marks behind it. Replayed, and not executed here: \label belongs %% after the \item, which is where writing it in the body has always put %% it, and where \@currentlabel and the anchor for it are the example's. %% \lx@emitjudge puts the collected commands back, in order, the moment %% the mark is hung -- see there. %% %% The set is closed, and it is the two no-ink commands a linguexx %% example is cross-referenced with: \label and \sublabel. Recognition %% is by meaning, so the hyperref, cleveref and babel redefinitions of %% \label are all the same \label to it, and an optional argument %% (cleveref's \label[eq]{...}) is carried across with the rest. \ExplSyntaxOn \tl_new:N \l__lx_judge_tl \tl_new:N \l__lx_judge_cont_tl \tl_new:N \l__lx_judge_pre_tl \bool_new:N \l__lx_judge_paren_bool \tl_const:Nn \c__lx_judge_star_tl {*} % scan judgments, then execute #1 (the collected marks are in % \lx@judgeprint; \lx@emitjudge hangs them via \jdg if nonempty, and % replays whatever \l__lx_judge_pre_tl was skipped past) \cs_new_protected:Npn \lx@scanjudgeto #1 { \tl_clear:N \l__lx_judge_tl \tl_clear:N \l__lx_judge_pre_tl \tl_set:Nn \l__lx_judge_cont_tl {#1} \bool_set_false:N \l__lx_judge_paren_bool \__lx_judge_loop: } \cs_new_protected:Npn \lx@scanjudge { \lx@scanjudgeto { \lx@makeitem } } \cs_new_protected:Npn \__lx_judge_loop: { \peek_remove_spaces:n { \peek_charcode_remove:NTF * { \tl_put_right:Nn \l__lx_judge_tl {*} %% "*(" with nothing between: the obligatoriness notation, %% *(Er) kommt. Still a judgment -- the star hangs like any %% other and the parenthesis stays with its word -- but it is %% SPOKEN as one (the "*(" entry of \g_lx_judge_alt_prop, %% "obligatory"), since "ungrammatical" says the opposite of %% what the notation means. Peeked, not removed, and without %% skipping spaces: "* (Er)" is a star and a parenthesis. The %% loop call goes INSIDE both branches: a peek looks at the %% token after its own arguments, and with the call after them %% that token was \__lx_judge_loop: itself, never the "(". \peek_charcode:NTF ( { \bool_set_true:N \l__lx_judge_paren_bool \__lx_judge_loop: } { \bool_set_false:N \l__lx_judge_paren_bool \__lx_judge_loop: } } { \peek_charcode_remove:NTF ? { \tl_put_right:Nn \l__lx_judge_tl {?} \bool_set_false:N \l__lx_judge_paren_bool \__lx_judge_loop: } { \peek_meaning_remove:NTF \# { \tl_put_right:Nn \l__lx_judge_tl {\#} \bool_set_false:N \l__lx_judge_paren_bool \__lx_judge_loop: } { \peek_meaning_remove:NTF \% { \tl_put_right:Nn \l__lx_judge_tl {\%} \bool_set_false:N \l__lx_judge_paren_bool \__lx_judge_loop: } { \peek_meaning:NTF \label { \__lx_judge_skip:N } { \peek_meaning:NTF \sublabel { \__lx_judge_skip:N } { \tl_use:N \l__lx_judge_cont_tl } } } } } } } } %% Take one skipped command out of the input and put it in %% \l__lx_judge_pre_tl, argument and all, then go on scanning. The %% command token itself is still in the stream when we get here (the %% peek above does not remove it), so #1 is it; what follows is any %% number of bracketed optional arguments and then exactly one braced %% one, which is the shape of both \label and \sublabel. \cs_new_protected:Npn \__lx_judge_skip:N #1 { \tl_put_right:Nn \l__lx_judge_pre_tl {#1} \__lx_judge_skip_args: } %% The optional argument is looked for only BEFORE the mandatory one, %% which is why \__lx_judge_skip_arg:n hands back to the judgment loop %% and not to here: "\ex. \label{x}[DP der Hund] bellte." opens its %% object line with a bracket, and a second look would eat the %% constituent as an argument of the label it follows. \cs_new_protected:Npn \__lx_judge_skip_args: { \peek_charcode:NTF [ { \__lx_judge_skip_opt:w } { \__lx_judge_skip_arg:n } } \cs_new_protected:Npn \__lx_judge_skip_opt:w [#1] { \tl_put_right:Nn \l__lx_judge_pre_tl { [#1] } \__lx_judge_skip_args: } \cs_new_protected:Npn \__lx_judge_skip_arg:n #1 { \tl_put_right:Nn \l__lx_judge_pre_tl { {#1} } \__lx_judge_loop: } \cs_new:Npn \lx@judgeprint { \tl_use:N \l__lx_judge_tl } \cs_new_protected:Npn \lx@setjudge #1 { \tl_set:Nn \l__lx_judge_tl {#1} } %% --- public API (Protocol A: judgments) ------------------------------- %% \lx_judgment_scan: peek for judgment marks in the input, then emit %% the item (i.e. \lx_judgment_scan:n{\lx_item_emit:}). %% This is where the \lx_item_... commands end. %% \lx_judgment_scan:n peek for marks, then run the given code. %% \lx_judgment_set:n set the mark explicitly, for a front-end that %% takes it as an argument rather than from the %% input stream (the gb4e \ex[*]{...} case). Any %% mark is allowed here, not just the scanned set. %% Either way the mark is hung by the next \lx_item_emit:. \cs_new_eq:NN \lx_judgment_scan: \lx@scanjudge \cs_new_eq:NN \lx_judgment_scan:n \lx@scanjudgeto \cs_new_eq:NN \lx_judgment_set:n \lx@setjudge %% Objective 3: spoken alternatives for judgment marks. Each mark is a %% symbol whose meaning a screen reader cannot infer ("asterisk"); under %% PDF tagging we wrap it in a Span carrying /Alt so it is announced %% ("ungrammatical"). \g_lx_judge_alt_prop maps a mark string to its %% spoken form; defaults follow standard usage and can be overridden or %% extended with \DeclareJudgment[spoken=...] or \SetJudgmentSpoken. \prop_new:N \g_lx_judge_alt_prop \prop_gput:Nnn \g_lx_judge_alt_prop {*} {ungrammatical} %% Not a mark of its own: the key the scanner looks up for a lone star %% written against an opening parenthesis (see \__lx_judge_loop:). \prop_gput:Nnn \g_lx_judge_alt_prop {*(} {obligatory} \prop_gput:Nnn \g_lx_judge_alt_prop {?} {questionable} \prop_gput:Nnn \g_lx_judge_alt_prop {??} {highly~questionable} \prop_gput:Nnn \g_lx_judge_alt_prop {?*} {extremely~degraded} \prop_gput:Nnn \g_lx_judge_alt_prop {*?} {extremely~degraded} \prop_gput:Nnn \g_lx_judge_alt_prop {\#} {infelicitous} \prop_gput:Nnn \g_lx_judge_alt_prop {\%} {grammatical~for~some~speakers} \tl_new:N \l__lx_judge_alt_tl \cs_new_protected:Npn \lx@judge@setalt #1#2 { \prop_gput:Nnn \g_lx_judge_alt_prop {#1} {#2} } %% Hang a judgment mark (#2) to the left; when tagging is active and a %% spoken form (#1) is non-blank, wrap the mark in a Span with /Alt so it %% is read aloud as #1. Otherwise typeset the mark exactly as before, so %% untagged output and non-tagging engines are unaffected. %% \lx@makeitem calls \lx@emitjudge (hence this) as the very FIRST thing %% after \item, i.e. at the start of a fresh list-item paragraph, which is %% precisely the moment the \lx@tag@span: helper exists for: the kernel's %% own paragraph-tagging may already have an MC open for the not-yet-shown %% P, and the helper suspends it. See its comment. %% %% A mark is not punctuation, and inside the \llap it must not be spaced %% as punctuation either. polyglossia's French under LuaTeX puts a thin %% space before ? ! : ; from a pre_shaping_filter that runs on the box %% after it was packed, so the glue is never set again and the space %% shoves the mark RIGHT, onto the text it should hang beside: ?, ?? and %% ?* overprinted the first word, * did not (polyglossia-fr). It reads %% the polyglossia_punct attribute, so unsetting that inside the box -- %% the value -"7FFFFFFF is LuaTeX's "unset", what polyglossia itself %% writes to switch it off -- leaves the mark alone and nothing else. %% babel makes ? active instead and XeTeX uses interchar tokens; both %% space at the time the box is built, so neither needs this. \cs_new_protected:Npn \__lx_judge_no_punct: { \sys_if_engine_luatex:T { \lua_now:n { local~a~=~luatexbase.attributes.polyglossia_punct~ if~a~then~tex.setattribute(a,~-0x7FFFFFFF)~end } } } %% Under XeTeX in a right-to-left paragraph the \llap's box is reversed %% inside by the paragraph's R segment, marked-content operators included %% (see \l__lx_gl_rframe_bool): the mark's /Span came out empty and the %% mark untagged. Its tagged part is an L segment of its own there; the %% \hss and \JdgSep around it keep the reversed order that hangs the mark %% on the right side, so the page does not change. %% Only for a gloss's mark (\l__lx_judge_rgloss_bool, set by %% \lx@emitglossjudge): the gloss's paragraph has no kernel marked content %% (\__lx_gl_paraoff_begin:), or the grid is set in a box of its own %% (\GlossRTL's path, a crossed example). In a paragraph WITH it -- a %% plain right-to-left one -- the kernel's marked content is %% reversed along with the text (upstream, doc/OPEN-ISSUES.md), and the %% mark's suspend and resume balanced only BECAUSE they were reversed with %% it: kept in order, they left an EMC with no BDC, and the tree read wrong %% for the rest of the page. \cs_new_protected:Npn \lx@hangjudge #1#2 { \leavevmode \llap { \__lx_judge_no_punct: \tl_if_blank:nTF {#1} { #2 } { \__lx_judge_ltr:n { \lx@tag@span:nn { tag = Span , alt = {#1} } {#2} } } \hskip \JdgSep } } \bool_new:N \l__lx_judge_rgloss_bool \cs_new_protected:Npn \lx@emitglossjudge { \__lx_gl_paraoff_begin: \bool_lazy_all:nTF { { \sys_if_engine_xetex_p: } { \lx@tag@if@active_p: } { \cs_if_exist_p:N \if@RTL } { \legacy_if_p:n { @RTL } } } { \bool_set_true:N \l__lx_judge_rgloss_bool } { \bool_set_false:N \l__lx_judge_rgloss_bool } \lx@emitjudge \bool_set_false:N \l__lx_judge_rgloss_bool } %% Under XeTeX, in a right-to-left paragraph, a gloss's paragraph is set %% with the kernel's paragraph tagging OFF. The kernel closes a %% paragraph's marked content at \par, inside bidi's R segment, and XeTeX %% reverses that EMC with the glyphs: it came out FIRST, and the last %% sequence was never closed (doc/OPEN-ISSUES.md). With no paragraph %% marked content there is nothing of the kernel's left in the segment, %% and every sequence of linguexx's in it is an L segment of its own (the %% mark, \__lx_judge_ltr:n; each word, \l__lx_gl_rframe_bool), so the %% reversal reorders whole units only. \CommonGloss sets its table the %% same way. %% A \GlossRTL grid too: it is set in a box of its own, but the item's %% paragraph around the box is still in the R segment, and so was the %% kernel's marked content in it. Not in a crossed example (an English %% one in a Hebrew document, \l__lx_dir_flip_bool): its list is set in the %% document's direction, so its paragraph is not in an R segment. %% Switched here, before the judgment mark, because a mark starts the %% paragraph (\lx@hangjudge's \leavevmode); put back after the grid, in %% \__lx_gl_dir_end:, with a \par of its own. Only where it was on: an %% author may have it off, and tagpdf has no public way to ask, so its %% switch is read where it exists. \bool_new:N \l__lx_gl_paraoff_bool \cs_new_protected:Npn \__lx_gl_paraoff_begin: { \bool_set_false:N \l__lx_gl_paraoff_bool \bool_lazy_all:nT { { \sys_if_engine_xetex_p: } { \lx@tag@if@active_p: } { \bool_if_exist_p:N \l__tag_para_bool } { \cs_if_exist_p:N \if@RTL } { \legacy_if_p:n { @RTL } } { ! \l__lx_dir_flip_bool } } { \bool_if:NT \l__tag_para_bool { \tagpdfsetup { para/tagging = false } \bool_set_true:N \l__lx_gl_paraoff_bool } } } \cs_new_protected:Npn \__lx_gl_paraoff_end: { \bool_if:NT \l__lx_gl_paraoff_bool { \par \tagpdfsetup { para/tagging = true } \bool_set_false:N \l__lx_gl_paraoff_bool } } \cs_new_protected:Npn \__lx_judge_ltr:n #1 { \bool_if:NTF \l__lx_judge_rgloss_bool { \tex_beginL:D #1 \tex_endL:D } { #1 } } %% Hang the mark, then put back whatever the scan skipped to reach it %% (\label, \sublabel -- see \__lx_judge_skip:N). This order is the %% point: the mark hangs from the label edge with nothing between it and %% the \item, and the replayed \label sits exactly where writing it in %% the body used to put it -- after the item, in the same mode, with the %% same \@currentlabel. The replay does not depend on a mark having %% been found -- a labelled example with no judgment goes through here %% too -- and it clears as it goes, so no second \lx@emitjudge (the %% \exg. path calls it after \lx@makeitem has) delivers the label twice. \cs_new_protected:Npn \lx@emitjudge { \tl_if_empty:NF \l__lx_judge_tl { \tl_set:Ne \l__lx_judge_alt_tl { \prop_item:Ne \g_lx_judge_alt_prop { \tl_to_str:N \l__lx_judge_tl \bool_lazy_and:nnT { \l__lx_judge_paren_bool } { \tl_if_eq_p:NN \l__lx_judge_tl \c__lx_judge_star_tl } { ( } } } \exp_args:NV \lx@hangjudge \l__lx_judge_alt_tl { \lx@judgeprint } } \tl_if_empty:NF \l__lx_judge_pre_tl { \tl_use:N \l__lx_judge_pre_tl \tl_clear:N \l__lx_judge_pre_tl } } \ExplSyntaxOff %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%% The example machinery: \ex. \a. \b. ... , blank-line terminated %%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Architecture: \ex. is a \par-delimited macro, exactly as in linguex -- %% that is what makes "blank line ends the example" work. BUT unlike %% linguex, the entire body is grabbed as one argument, so all list %% opening and closing happens inside a single macro invocation: %% \ex. -> \begingroup + open the main list [process body] %% [close open sublists] + close the main list + \endgroup %% Sub-example depth is tracked in a count register that is only ever %% advanced INSIDE the group opened for the new sublevel, so leaving the %% group automatically pops the depth: the grouping structure itself is %% the stack, and global drift (linguex's ExDepth disease) is %% structurally impossible. %% %% Semantics kept from linguex: \a. always opens a NEW, deeper level %% (first level: letters, second: roman); \b. \c. \d. \e. \f. are %% interchangeable "next item at current level" commands (the printed %% letter comes from the counter, not the command name). Two sublevels %% maximum. \ex.[custom] sets a custom label without stepping the %% counter. Inside footnotes, examples number (i), (ii), ... on the %% FnExNo counter automatically. %% %% NOT carried over from linguex (deliberately): embedded examples %% (\ex. inside \a.) and the \exi./\ai. index variants. \newcount\lx@subdepth %% The example body is collected TOKEN BY TOKEN (expl3 peek_analysis), %% not grabbed as a \par-delimited argument. Collection stops, at brace %% depth 0, at the first of: %% - a \par token (i.e. the blank line, as under linguex); %% - \z. (early termination; the rest of the stream continues as %% ordinary text, so no blank line is needed after it); %% - a structural boundary: an unmatched \end{...}, \endgroup, or %% group-end token. The boundary token is put back, so %% "\ex. Text\end{frame}" simply works. %% \begin{...}/\end{...} and \begingroup/\endgroup pairs INSIDE the body %% are counted, so environments inside examples are unaffected. Brace %% groups are collected whole; a \par inside braces is therefore legal %% (it was an error under the old grab). \z. and the terminating \par %% are only recognized at brace depth 0. %% \ex dispatches on what follows: a period gives the classic dot %% syntax; anything else is the gb4e item form. Each branch is defined %% by the mode in force, with an instructive error where a syntax is %% not loaded. \newif\iflx@inexe %% TRUE while an example opened by \ea (the langsci front-end) is open -- %% i.e. "the example currently open is written in the langsci syntax". %% It is what enforces ONE SYNTAX PER EXAMPLE: the dot commands refuse to %% run while it is set, and \ea refuses to run while an example of the %% other kind is open. See the header of the \ea section for why the ban %% exists and why it does not extend to [lazy,gb4e], whose within-example %% mixing is a documented, tested promise. %% %% Cleared in \lx@example@cleanup (the example is over) and at the %% footnote boundary beside \@noftnotefalse (a footnote inside an \ea %% example holds examples of its OWN, which may be written either way -- %% without that reset the ban would fire on legitimate input). It is %% never set globally, and \ea opens no group of its own, so the reset in %% the cleanup is not optional. \newif\iflx@inlangsci %% The dot-syntax half of the one-syntax-per-example rule: every command %% that only means something in the dot syntax runs this first, naming %% itself. A no-op unless an \ea example is open, so it costs nothing in %% a document that never loads [langsci]. \def\lx@nomix@dot#1{% \iflx@inlangsci \PackageError{linguexx}{#1 inside an \string\ea\space example}% {One example is written in ONE syntax throughout. This one was opened with \string\ea, so its sub-levels are \string\ea\space and it ends at \string\z.\MessageBreak Convert the whole example, or leave this one as it stands and write the next in the other syntax -- between examples the two mix freely, which is what [lazy,langsci] is for.}% \fi} %% \ex peeks for the dot of the dot syntax, and the peek has to tell it %% from a period the writer meant as prose. The continuation-style %% example -- a mother item sets a context and each sub-item resumes the %% sentence -- opens its items with a literal ellipsis, and langsci-gb4e, %% whose \ex never peeked, always set that as text. %% %% The two are distinguishable, but not by \@ifnextchar, which SKIPS %% SPACES. TeX's own scanner has already eaten the space after the %% control word, so what reaches the peek is: %% %% \ex. body -> . b o d y (dot syntax) %% \ex ... text -> . . . t e x t (prose) %% \ex. ...body -> . . . . b (dot syntax, prose body) %% %% The period that opens the dot syntax is followed by the space that %% separates it from the body; the first period of an ellipsis is followed %% by the second, with nothing in between. So the test is what comes %% IMMEDIATELY after the first period -- \futurelet, which does not skip %% spaces, where \@ifnextchar would skip that separating space and read %% the third line above as prose, trading one misparse for another. %% %% \lx@ex@peek takes the first period as its delimiter and hands it back %% on both branches: to \lx@ex@dot, which is delimited by it in turn, and %% ahead of the two that remain on the text branch, so the item opens with %% all three. %% %% \exg. and the \a.-\f. shorthands expand to \ex. plus a control %% sequence, which is not a period and takes the dot branch as before. %% The one form this cannot resolve is \ex....body -- dot syntax whose %% body opens with an ellipsis and no space -- which is read as prose. %% It is unwritable either way, and the spaced form above is not. \let\lx@dotchar=. \def\lx@ex@peek.{\futurelet\lx@peeked\lx@ex@peek@ii} \def\lx@ex@peek@ii{% \ifx\lx@peeked\lx@dotchar \expandafter\lx@ex@peek@text \else \expandafter\lx@ex@peek@dot \fi} \def\lx@ex@peek@text{\lx@ex@nodot.} \def\lx@ex@peek@dot{\lx@ex@dot.} \def\ex{\@ifnextchar.{\lx@ex@peek}{\lx@ex@nodot}} \iflx@lazy \def\lx@ex@dot.{\lx@nomix@dot{\string\ex.}\lx@collectbody} \else \def\lx@ex@dot.{% \PackageError{linguexx}{The dot syntax (\string\ex.) is not available under [gb4e]}{Use \string\ex\space inside \string\begin{exe} ... \string\end{exe}, or load linguexx without options (or with [lazy]) for the dot syntax.}} \fi \iflx@gbfour % \ex is the item command of BOTH environment-shaped front-ends: the % exe/xlist batch and an \ea example, which is a batch of one that % brought its own list. Either flag makes it legal; neither, and it is % the error below. \def\lx@ex@nodot{% \iflx@inexe \let\lx@donext\lx@gbex \else\iflx@inlangsci \let\lx@donext\lx@gbex \else \let\lx@donext\lx@gbex@errthen \fi\fi \lx@donext} % error first, then carry on as if \ex had been legal, so that one % misplaced \ex does not cascade into a second error from its own body \def\lx@gbex@errthen{\lx@gbex@err\lx@gbex} \def\lx@gbex@err{% \PackageError{linguexx}{\string\ex\space outside exe/xlist}{Put \string\ex\space inside \string\begin{exe} ... \string\end{exe}\iflx@langsci, or after \string\ea\fi \iflx@lazy, or write \string\ex. (with the period) for the dot syntax\fi.}} \else \def\lx@ex@nodot{% \PackageError{linguexx}{\string\ex\space must be followed by a period}{Write \string\ex. -- or load \string\usepackage[gb4e]{linguexx} for the environment syntax.}} \fi % gb4e item form: \ex[judgment]{text} or plain \ex text \def\lx@gbex{\@ifnextchar[{\lx@gbex@opt}{\lx@gbex@plain}} \def\lx@gbex@plain{% \ifcase\lx@subdepth \let\lx@donext\lx@mainitem \or \let\lx@donext\lx@subitem \or \let\lx@donext\lx@subsubitem \fi \lx@donext} %% The level dispatch and the explicit-judgment item, factored out of %% \lx@gbex@opt: they are what a front-end needs in order to give an item %% a mark it did not read from the input, and there is no way to compose %% that out of the scanning commands -- \lx@scanjudge clears the mark %% before it peeks, so anything set beforehand is lost. %% Neither opens a list: at depth 0 they assume the main list is already %% open, which is the batch case (exe). A front-end whose example brings %% its own list calls \lx@mainlist first. \def\lx@item@core{% \ifcase\lx@subdepth \lx@main@core \or \lx@subitem@core \or \lx@subsubitem@core \fi} \def\lx@item@judged#1{\lx@item@core\lx@setjudge{#1}\lx@makeitem} \long\def\lx@gbex@opt[#1]#2{\lx@refcheck@stash{#2}\lx@item@judged{#1}#2} %% --- public API (Protocol A: items at the current level) -------------- %% \lx_item_core: step the counter of whatever level is open and set %% its label. Emits nothing, so a front-end may put %% something between the label and the item. %% \lx_item_judged:n the same, plus the given mark, plus the item: an %% item carrying a judgment that was NOT read from the %% input. Any mark is allowed, not just the scanned %% set. Both assume the list of the current level is %% already open (see \lx_list_main_open:). \ExplSyntaxOn \cs_new_eq:NN \lx_item_core: \lx@item@core \cs_new_eq:NN \lx_item_judged:n \lx@item@judged \ExplSyntaxOff \ExplSyntaxOn \tl_new:N \l__lx_body_tl \tl_new:N \l__lx_body_tmp_tl \seq_new:N \l__lx_body_stack_seq \int_new:N \l__lx_body_env_int \int_new:N \l__lx_body_grp_int \int_new:N \l__lx_body_col_int %% --- public API (Protocol A: body collection) ------------------------- %% \lx_body_collect:N \myfunction %% Collect the example body under the terminator rules described above %% and hand it to \myfunction as its single argument. The function must %% be \long (bodies contain \par). It is entered with the terminator %% already put back where the rules say it should be, so it may simply %% typeset: the dot syntax's own continuation, \lx@run@ex, is nothing %% but \lx_example_begin: + items + \lx_example_end:. %% %% The continuation is held in a macro rather than passed down the call %% chain because the collection ends in \peek_analysis_map_break:n, which %% discards the surrounding expansion context. The assignment is GLOBAL: %% the collector enters a TeX group for every brace group it collects, and %% the unmatched-group-end branch below breaks out with an \egroup still %% pending, so a local value could be restored away before it is used. \cs_new_eq:NN \lx@body@cont \prg_do_nothing: \cs_new_protected:Npn \lx_body_collect:N #1 { \cs_gset_eq:NN \lx@body@cont #1 \tl_clear:N \l__lx_body_tl \seq_clear:N \l__lx_body_stack_seq \int_zero:N \l__lx_body_env_int \int_zero:N \l__lx_body_grp_int \int_zero:N \l__lx_body_col_int \__lx_body_loop: } %% \lx@run@ex is defined further down; naming it here is safe because the %% body is not expanded until the first \ex. of the document. \cs_new_protected:Npn \lx@collectbody { \lx_body_collect:N \lx@run@ex } \cs_new_protected:Npn \__lx_body_loop: { \peek_analysis_map_inline:n { \int_case:nnF { "##3 } { { 1 } { \seq_push:NV \l__lx_body_stack_seq \l__lx_body_tl \tl_clear:N \l__lx_body_tl } { 2 } { \seq_pop:NNTF \l__lx_body_stack_seq \l__lx_body_tmp_tl { \tl_put_right:Ne \l__lx_body_tmp_tl { { \exp_not:V \l__lx_body_tl } } \tl_set_eq:NN \l__lx_body_tl \l__lx_body_tmp_tl } { \peek_analysis_map_break:n { \lx@body@info{group~end}\lx@runbody \egroup } } } } { \bool_lazy_and:nnTF { \int_compare_p:nNn {##2} = { -1 } } { \seq_if_empty_p:N \l__lx_body_stack_seq } { \__lx_body_cs:n {##1} } { \tl_put_right:Ne \l__lx_body_tl {##1} } } } } % control-sequence dispatch at brace depth 0. Terminators: \par; % a nested \ex./\exg. (embedded examples are unsupported, so this is a % forgotten blank line: treat as boundary); unmatched \end, \endgroup, % \color@endgroup (footnote machinery), each pair-counted against its % opener inside the body. \z is NOT handled here: it is an ordinary % body macro, interpreted at typesetting time (see \lx@zpop below). % %% The nine names fall into four behaviours, and each is written once: %% %% \par -- the boundary proper, nothing to weigh up %% \begin & co. -- an opener: count it and collect it %% \end & co. -- a closer: uncount and collect it if its opener %% was counted inside the body, otherwise it closes %% something opened OUTSIDE and the example ends %% \ex, \exg -- like a closer in its test, but with no counter of %% its own to move: legitimate inside an environment %% opened in the body, a forgotten blank line outside %% %% Anything else is body text and is collected. %% %% On \str_case:enF, which is load-bearing and not interchangeable with %% its neighbours. The "e" is the contract, not a flourish: #1 is not the %% token itself. \peek_analysis_map_inline:n hands over a wrapper that %% yields the token when e-expanded, which is what the "ee" of the previous %% \str_if_eq:eeTF chain was for and what the "e" here is for. A variant %% without it compares the wrapper and matches nothing. %% %% This is also the token loop of every example body, so the shape was %% chosen with an eye on cost: the "e" variant expands #1 ONCE before the %% case list is walked, where the chain re-expanded it for each of the nine %% names tried. Do not oversell that. Measured under lualatex, 300 %% examples of 48 macros each: 2.598s before, 2.499s after, a repeatable %% 4%. On a document whose examples hold a handful of macros the %% difference is 0.5%, which is noise. Legibility is the reason to keep %% this shape; the expansions are a bonus, and nobody should reach for a %% harder-to-read variant chasing the rest of them. %% %% Three neighbouring spellings are wrong, and none of them announces itself: %% %% - the names on the left are written BARE, not as \exp_not:N \par. Only %% the test is expanded here; a case is compared under \exp_not:n, so an %% \exp_not:N written in one is compared as itself and the case can never %% match. (The old chain used \str_if_eq:ee, which expanded BOTH sides, %% and there the \exp_not:N was required. Carrying it over is the %% natural mistake, and it disables every case at once.) %% - \str_case_e:enF re-expands the test for every case in the list, so it %% reads the same and buys nothing. %% - stringifying into a str first and dispatching with \str_case:VnF is %% the version that looks fastest, and it silently matches NOTHING. %% \str_set:Ne stringifies with \tl_to_str:n, which terminates a control %% word with a space ("\par" becomes 5 characters); \str_case compares %% with \str_if_eq:nn, which is \pdfstrcmp under \exp_not:n and follows %% \string, with no such space (4 characters). Every case misses, every %% token is collected as body text, and the example never ends -- which %% on the page is a paragraph that will not outdent and, in the tag %% tree, the rest of the document nested inside the last example. It %% was caught by struct_label_depths, not by any rendering assertion. \cs_new_protected:Npn \__lx_body_keep:n #1 { \tl_put_right:Ne \l__lx_body_tl {#1} } \cs_new_protected:Npn \__lx_body_open:Nn #1#2 { \int_incr:N #1 \__lx_body_keep:n {#2} } %% #3 is what the log calls the token, #4 the token itself, put back into %% the stream ahead of whatever the boundary starts. #4 is written out at %% the call rather than recovered from #2: #2 is the wrapper above, and it %% only survives being EXPANDED, whereas this one is re-emitted as code. \cs_new_protected:Npn \__lx_body_close:Nnnn #1#2#3#4 { \int_compare:nNnTF { #1 } > { 0 } { \int_decr:N #1 \__lx_body_keep:n {#2} } { \peek_analysis_map_break:n { \lx@body@info {#3} \lx@runbody #4 } } } %% \ex / \exg met inside the body. In a \begin{...}\end{...} pair opened %% within the body -- an xlist, say -- it is a legitimate item command and %% is collected; at environment depth 0 it can only be a forgotten blank %% line, and starts a new example. Either way there is no counter of its %% own to move, which is what separates this from \__lx_body_close:Nnnn. \cs_new_protected:Npn \__lx_body_nested:nnn #1#2#3 { \int_compare:nNnTF { \l__lx_body_env_int } > { 0 } { \__lx_body_keep:n {#1} } { \peek_analysis_map_break:n { \lx@body@info {#2} \lx@runbody #3 } } } \cs_new_protected:Npn \__lx_body_cs:n #1 { \str_case:enF {#1} { { \par } { \peek_analysis_map_break:n { \lx@runbody } } { \ex } { \__lx_body_nested:nnn {#1} { \string\ex. } \ex } { \exg } { \__lx_body_nested:nnn {#1} { \string\exg. } \exg } { \begin } { \__lx_body_open:Nn \l__lx_body_env_int {#1} } { \end } { \__lx_body_close:Nnnn \l__lx_body_env_int {#1} { \string\end } \end } { \begingroup } { \__lx_body_open:Nn \l__lx_body_grp_int {#1} } { \endgroup } { \__lx_body_close:Nnnn \l__lx_body_grp_int {#1} { \string\endgroup } \endgroup } { \color@begingroup } { \__lx_body_open:Nn \l__lx_body_col_int {#1} } { \color@endgroup } { \__lx_body_close:Nnnn \l__lx_body_col_int {#1} { \string\color@endgroup } \color@endgroup } { \verb } { \peek_analysis_map_break:n { \__lx_body_verb:nnnn { verb } 1 0 0 } } { \Verb } { \peek_analysis_map_break:n { \__lx_body_verb:nnnn { Verb } 1 1 0 } } { \lstinline } { \peek_analysis_map_break:n { \__lx_body_verb:nnnn { lstinline } 0 1 1 } } } { \__lx_body_keep:n {#1} } } %% Inline verbatim in a body: \verb, fancyvrb's \Verb, listings' %% \lstinline. The argument has not been read yet -- the collector takes %% one token at a time, and the character after a control word is not a %% token until something asks for it -- so it is read here, with the %% catcodes \verb itself would use, and kept as a string: the command's %% name and every character after it up to the closing delimiter. At %% typesetting time the string is read again by \scantokens and the %% command itself runs on it, so the font, the ligatures, the star and the %% options (\fvset, \lstset, basicstyle=...) are its own. Only at brace %% depth 0, which is where these commands work anyway: not inside an %% argument. %% %% The reader knows each command's SHAPE and nothing else. #1 = the name, %% #2 = it takes a star, #3 = [options], #4 = {...} as well as a character %% for delimiter. As measured on TeX Live 2026: \verb[*]|...|, %% \Verb[*][options]|...|, \lstinline[options]|...| or {...}. \lstinline's %% {...} ends at the first }, unbalanced, as listings ends it; options end %% at the first ] outside braces, as a LaTeX optional argument does. A %% command not loaded is never met, so nothing here needs the package. %% %% The characters are read one at a time, so the end of the line stops %% the text as it stops \verb: a missing delimiter is an error on its own %% line, not a search through the rest of the file. \peek_analysis also %% tells fresh characters from tokens: with these catcodes a character %% read from the file is a letter or "other", and anything else -- a %% brace, a space, a subscript -- was read BEFORE, as part of an argument %% the example sits in (\footnote{...}). There the command cannot work, %% as anywhere in an argument, and saying so beats the message the token %% would cause further on. \tl_new:N \l__lx_body_verb_tl % the characters after the name \tl_new:N \l__lx_body_verb_name_tl \tl_new:N \l__lx_body_verb_end_tl % the closing delimiter \int_new:N \l__lx_body_verb_phase_int % 0 star, 1 [, 2 options, 3 delimiter, 4 text \int_new:N \l__lx_body_verb_depth_int % braces inside the options \bool_new:N \l__lx_body_verb_opt_bool \bool_new:N \l__lx_body_verb_brace_bool \tl_const:Ne \c__lx_body_cr_tl { \char_generate:nn { 13 } { 12 } } \cs_new_protected:Npn \__lx_body_verb:nnnn #1#2#3#4 { \group_begin: \cs_set_eq:NN \do \char_set_catcode_other:N \dospecials \char_set_catcode_other:n { 32 } \char_set_catcode_other:n { 9 } \char_set_catcode_other:n { 13 } \tl_set:Nn \l__lx_body_verb_name_tl {#1} \tl_clear:N \l__lx_body_verb_tl \int_set:Nn \l__lx_body_verb_phase_int { \int_compare:nNnTF {#2} = { 1 } { 0 } { 1 } } \bool_set:Nn \l__lx_body_verb_opt_bool { \int_compare_p:nNn {#3} = { 1 } } \bool_set:Nn \l__lx_body_verb_brace_bool { \int_compare_p:nNn {#4} = { 1 } } \peek_analysis_map_inline:n { \bool_lazy_or:nnTF { \str_if_eq_p:nn {##3} { B } } { \str_if_eq_p:nn {##3} { C } } { \__lx_body_verb_step:e {##1} } { \peek_analysis_map_break:n { \__lx_body_verb_fail:n { verb-argument } } } } } \cs_new_protected:Npn \__lx_body_verb_keep:n #1 { \tl_put_right:Nn \l__lx_body_verb_tl {#1} } \cs_new_protected:Npn \__lx_body_verb_step:n #1 { \int_case:nn { \l__lx_body_verb_phase_int } { { 0 } { \int_set:Nn \l__lx_body_verb_phase_int { 1 } \str_if_eq:nnTF {#1} { * } { \__lx_body_verb_keep:n {#1} } { \__lx_body_verb_step:n {#1} } } { 1 } { \bool_lazy_and:nnTF { \l__lx_body_verb_opt_bool } { \str_if_eq_p:nn {#1} { [ } } { \int_set:Nn \l__lx_body_verb_phase_int { 2 } \int_zero:N \l__lx_body_verb_depth_int \__lx_body_verb_keep:n {#1} } { \int_set:Nn \l__lx_body_verb_phase_int { 3 } \__lx_body_verb_step:n {#1} } } { 2 } { \__lx_body_verb_keep:n {#1} \str_if_eq:eeT {#1} { \c_left_brace_str } { \int_incr:N \l__lx_body_verb_depth_int } \str_if_eq:eeT {#1} { \c_right_brace_str } { \int_decr:N \l__lx_body_verb_depth_int } \bool_lazy_and:nnT { \str_if_eq_p:nn {#1} { ] } } { \int_compare_p:nNn { \l__lx_body_verb_depth_int } = { 0 } } { \int_set:Nn \l__lx_body_verb_phase_int { 3 } } } { 3 } { \__lx_body_verb_if_cr:nF {#1} { \tl_set:Ne \l__lx_body_verb_end_tl { \bool_lazy_and:nnTF { \l__lx_body_verb_brace_bool } { \str_if_eq_p:ee {#1} { \c_left_brace_str } } { \c_right_brace_str } {#1} } \__lx_body_verb_keep:n {#1} \int_set:Nn \l__lx_body_verb_phase_int { 4 } } } { 4 } { \str_if_eq:eeTF {#1} { \l__lx_body_verb_end_tl } { \__lx_body_verb_keep:n {#1} \peek_analysis_map_break:n { \__lx_body_verb_done: } } { \__lx_body_verb_if_cr:nF {#1} { \__lx_body_verb_keep:n {#1} } } } } } \cs_generate_variant:Nn \__lx_body_verb_step:n { e } \cs_new_protected:Npn \__lx_body_verb_if_cr:nF #1#2 { \str_if_eq:eeTF {#1} { \c__lx_body_cr_tl } { \peek_analysis_map_break:n { \__lx_body_verb_fail:n { verb-eol } } } {#2} } \cs_new_protected:Npn \__lx_body_verb_done: { \use:e { \group_end: \tl_put_right:Nn \exp_not:N \l__lx_body_tl { \exp_not:N \lx@verb@rescan { \c_backslash_str \l__lx_body_verb_name_tl \exp_not:V \l__lx_body_verb_tl } } } \__lx_body_loop: } \cs_new_protected:Npn \__lx_body_verb_fail:n #1 { \use:e { \group_end: \msg_error:nnn { linguexx } {#1} { \l__lx_body_verb_name_tl } } \__lx_body_loop: } \msg_new:nnnn { linguexx } { verb-argument } { \iow_char:N \\#1~in~an~example~inside~a~command~argument. } { The~example~is~in~the~argument~of~another~command,~such~as~ \iow_char:N \\footnote,~so~its~text~was~read~before~ \iow_char:N \\#1~could~protect~it.~Its~verbatim~text~is~dropped. } \msg_new:nnn { linguexx } { verb-eol } { \iow_char:N \\#1~in~an~example~ended~by~end~of~line. } \cs_new_protected:Npn \lx@verb@rescan #1 { \group_begin: \int_set:Nn \tex_endlinechar:D { -1 } \tex_scantokens:D {#1} \group_end: } \cs_new_protected:Npn \lx@runbody { \exp_args:NV \lx@body@cont \l__lx_body_tl } \ExplSyntaxOff \def\lx@body@info#1{% \PackageInfo{linguexx}{Example terminated by structural boundary (#1)}} %% \lx@mainlist is called AFTER \lx@itemlabel has been fixed (either from %% the counter or from \ex.[custom]), because under [legacy] the width of %% the label box is derived from the label itself. \topsep is set before %% \lx@geom@main, which may modify it (halved in footnotes, under legacy). %% %% Lists are opened and closed through the ENVIRONMENT interface %% (\begin{list}...\end{list}) rather than the command pair %% \list...\endlist. The two are equivalent on a classic engine, but %% under the tagged PDF code (\DocumentMetadata testphase "block") the %% environment is the supported interface: part of its state restoration %% is keyed to the environment hooks, which never fire in command-form %% use, and the resulting imbalance corrupts the structure tree (visible %% with an example inside a footnote). Funnelling every open and close %% through the two macros below keeps that decision in one place. %% Example lists are semantically ordered. PDF's /ListNumbering has a %% fixed set of valid values (Decimal, LowerAlpha, LowerRoman, ...); %% "Ordered" is NOT one of them, so routing through the block code's %% enumerate class (whose value is /Ordered) is rejected by validators. %% Instead we define one valid attribute class per level and hand it to %% the list's structure element. \lx@ol@class carries the class for the %% list about to open; a guarded patch of the block list-begin applies it %% and clears it. If that internal ever disappears the lists simply keep %% the block's default class (/None) --- still valid, never /Ordered. \ExplSyntaxOn \tl_new:N \lx@ol@class \cs_new_protected:Npn \lx@ol@set #1 { \tl_set:Nn \lx@ol@class {#1} } %% --- public API (Protocol A: list numbering) -------------------------- %% Set the /ListNumbering attribute class for the list about to open. %% Valid values are the classes declared below (lxOLdecimal, lxOLalpha, %% lxOLroman); a front-end that opens a list at a level the core does not %% know about declares its own class with \tagpdfsetup{newattribute=...}. \cs_new_eq:NN \lx_ol_class:n \lx@ol@set \AddToHook{begindocument} { \cs_if_exist:NT \tagpdfsetup { \tagpdfsetup { newattribute = { lxOLdecimal } { /O /List /ListNumbering /Decimal } , newattribute = { lxOLalpha } { /O /List /ListNumbering /LowerAlpha } , newattribute = { lxOLroman } { /O /List /ListNumbering /LowerRoman } , % for the sub-level numbering variants of [langsci] (\xlistA, % \xlistI, \xlistn): the printed label and /ListNumbering have % to agree, or a screen reader announces a numbering the page % does not show. newattribute = { lxOLupperalpha } { /O /List /ListNumbering /UpperAlpha } , newattribute = { lxOLupperroman } { /O /List /ListNumbering /UpperRoman } , } } \cs_if_exist:NT \__block_list_begin: { \cs_gset_eq:NN \lx@orig@list@begin: \__block_list_begin: \cs_gset:Npn \__block_list_begin: { \tl_if_empty:NF \lx@ol@class { \tl_set_eq:NN \l__tag_L_attr_class_tl \lx@ol@class \tl_clear:N \lx@ol@class } \lx@orig@list@begin: } } } \ExplSyntaxOff \def\lx@openlist#1{\begin{list}{}{#1}} \def\lx@closelist{\end{list}\lx@dir@listend} %% --- public API (Protocol A: the list funnel) ------------------------- %% EVERY list a front-end opens must go through this pair. Bypassing it %% with \list/\begin{list} loses the /ListNumbering attribute patched in %% above, and -- because the patch is keyed to the ENVIRONMENT hooks, which %% command-form \list never fires -- corrupts the structure tree for an %% example inside a footnote. See the comment above \lx@ol@class. \ExplSyntaxOn \cs_new_eq:NN \lx_list_open:n \lx@openlist \cs_new_eq:NN \lx_list_close: \lx@closelist \ExplSyntaxOff %% What every example list wants, at all three levels: no glue anywhere %% the surrounding text could see, and the label set flush left in its own %% box (\lx@flushlabel, not the list default, which right-aligns a label %% narrower than \labelwidth). Only \topsep and the level geometry differ %% between the three, so only those are set beside this. %% Two font hooks and a page-breaking one, all empty (or \relax) unless %% [langsci] fills them in: they are what \exfont / \exnrfont and the %% [lowerpenalty] option of langsci-gb4e have to reach, and there is no %% other seam in the list that a declaration could be attached to. Empty %% is exactly what every document that never loads the option gets, and %% \lx@bodyfont sits at the end of the parameter block so that a font it %% selects is the one the item bodies are set in. \let\lx@bodyfont\@empty \let\lx@labelfont\@empty \let\lx@itempenalty\relax \def\lx@listdefaults{% \itemindent\z@ \listparindent\z@ \parsep\z@ \itemsep\z@ \partopsep\z@ \parskip\z@ \lx@itempenalty \let\makelabel\lx@flushlabel \lx@bodyfont} \def\lx@mainlist{% \lx@dir@listbegin \lx@openlist{\topsep\Extopsep \lx@geom@main \lx@listdefaults}} %% In the exe environment the list opens before any item exists, so the %% label the width is derived from is the one the NEXT \ex will print. \def\lx@guesslabel{% \if@noftnote \def\lx@itemlabel{\theExLBr\the\numexpr\value{ExNo}+1\relax\theExRBr}% \else \def\lx@itemlabel{\theFnExLBr \romannumeral\numexpr\value{FnExNo}+1\relax\theFnExRBr}% \fi} %% --- public API (Protocol A: opening the main list by hand) ----------- %% \lx_item_main: opens the list for you. These two are for the BATCH %% shape, where one list holds several examples: open it once with %% \lx_label_guess: + \lx_list_main_open:, then give it items with %% \lx_item_next:. \lx_label_guess: is not optional in that shape -- %% under [legacy] the list geometry is read off a label that does not %% exist yet, so it stands in the label the next item will print. \ExplSyntaxOn \cs_new_eq:NN \lx_list_main_open: \lx@mainlist \cs_new_eq:NN \lx_label_guess: \lx@guesslabel \ExplSyntaxOff %% No TeX group wraps the example as a whole: the list environment's own %% group scopes everything set after the list opens, and wrapping the %% environment in one extra group is exactly the pattern that trips the %% text-unit accounting of the 2023 latex-lab "block" tagging code when %% the example sits in a footnote. What used to rely on that group is %% now handled explicitly: \lx@inexample is reset in \lx@bodyend, and %% \@currentlabel/\@currentHref (set locally by \refstepcounter BEFORE %% the list opens, because [legacy] sizes the label box off the label) %% are saved here and restored in \lx@bodyend, so a \label after the %% example still refers to whatever preceded it. \@currentcounter with %% them: the kernel's first aid for cleveref rebuilds cleveref's record at %% every \label from it, so with only the other two restored, a \label in %% the prose after an example printed right with \ref and as the example %% with \cref -- "(1)" for "section 1". %% The setup half is factored out under its own name because it, and not %% \lx@run@ex, is what a front-end needs: \lx@run@ex additionally commits %% to the dot syntax's \lx@exstart (which peeks for a "[" custom label) %% and to the body arriving as one grabbed argument. A front-end that %% builds its items itself calls \lx@example@begin, then whichever item %% commands it likes, then \lx@bodyend. \def\lx@example@begin{% \ifdim\lastskip=\Extopsep\vspace{\Exredux}\fi \lx@subdepth\z@ \setcounter{SubExNo}{0}\setcounter{SubSubExNo}{0}% \let\lx@saved@currentlabel\@currentlabel \let\lx@saved@currentcounter\@currentcounter \ifdefined\@currentHref \let\lx@saved@currentHref\@currentHref \fi \lx@inexampletrue \lx@letters@on} %% \lx@cg@detect: an example holding a \CommonGloss reads its \ag. ... \fg. %% as variant rows (see "Variant rows sharing a gloss"); it has to know %% before the first of them runs, and the collected body is the one place %% that can tell it. Undone by \lx@example@cleanup. \long\def\lx@run@ex#1{\lx@refcheck@stash{#1}\lx@example@begin\lx@cg@detect{#1}% \lx@exstart#1\lx@bodyend} % closes the example: all open sublists, the main list, the group. % Doubles as the sentinel marking the end of the grabbed body, which is % what allows \z. to terminate the example early (see below). \def\lx@bodyend{% \lx@glt@langend % while the \glt paragraph is still the current one \lx@closesubs \lx@closelist \lx@example@cleanup\ignorespacesafterend} %% --- public API (Protocol A: the example lifecycle) ------------------- %% One example, start to finish. \lx_example_end: closes every sublist %% still open, so a front-end never has to track depth itself. %% %% \lx_example_end: is also the SENTINEL that marks the end of a grabbed %% body -- \z. terminates an example early by scanning ahead to it (see %% \lx@zexit). A front-end may call it, but must not \let it to %% something else, wrap it in a group, or place a copy of it anywhere but %% at the very end of the example. \ExplSyntaxOn \cs_new_eq:NN \lx_example_begin: \lx@example@begin \cs_new_eq:NN \lx_example_end: \lx@bodyend \ExplSyntaxOff %% Under the tagged-PDF testphase code, the block layer wraps top-level %% blocks in a "text-unit" structure element and defers its close to the %% @endpe continuation paragraph. Since the cleanup above cancels that %% continuation, it must also perform the close the continuation's %% para/end hook would have performed -- the same two operations, taken %% from tagpdf's own end plug. The internal names are deliberately %% existence-guarded: this compensation is matched to the current %% testphase internals and degrades to a no-op if they change. \ExplSyntaxOn \cs_new_protected:Npn \lx@tag@close@textunit { \lx@tag@if@active:T { \cs_if_exist:NT \__tag_gincr_para_main_end_int: { \__tag_gincr_para_main_end_int: \tag_struct_end: } } } \ExplSyntaxOff \def\lx@example@cleanup{% \lx@cg@off \lx@inexamplefalse \lx@inlangscifalse \lx@letters@off % The list's \end arms the "continue the paragraph" dance (@endpe + % an \everypar that clears it). Our semantics is the opposite: a % blank line after an example starts a NEW, indented paragraph % (same-line continuation is handled explicitly by \z.). Clear BOTH % halves of the dance, the way the kernel's own \everypar token % would: clearing only one of them is what desynchronises the tagged % text-unit bookkeeping. \if@endpe\everypar{}\@endpefalse\lx@tag@close@textunit\fi \let\@currentlabel\lx@saved@currentlabel \let\@currentcounter\lx@saved@currentcounter \ifdefined\lx@saved@currentHref \let\@currentHref\lx@saved@currentHref \fi} %% \lx@exstart@* : label, THEN list, THEN item (the order matters under %% [legacy], where the list geometry is read off the label). %% \lx@mainitem is the same thing minus the list, for an item inside an %% exe environment, whose list is already open. \def\lx@exstart{\@ifnextchar[{\lx@exstart@opt}{\lx@exstart@normal}} %% The custom-label item, separated from the bracket syntax that reaches %% it, so that a front-end can have custom labels without having to spell %% them "[...]". Note the order the label is set in: BEFORE \lx@mainlist, %% because [legacy] sizes the label box from the label itself. \def\lx@custom@item#1{\lx@custom@begin{#1}\def\lx@itemlabel{#1}\lx@mainlist\lx@scanjudge} %% The same, for a front-end whose list is ALREADY open -- an item inside a %% batch, which is where [langsci]'s \exi, \exr, \exp and \sn live. It %% steps no counter and so records no anchor, exactly as \lx@custom@item %% does not; that is the whole point of a custom label. \def\lx@custom@item@here#1{\lx@custom@begin{#1}\def\lx@itemlabel{#1}\lx@scanjudge} \def\lx@exstart@opt[#1]{\lx@custom@item{#1}} %% \lx@relref@record notes the hyperref anchor this example just got, so %% that a later run's \Next can tell a number the document reaches from %% one it does not. It sits HERE, after the counter step, because that %% is where hyperref has made the anchor and set \@currentHref; and it %% sits here ONLY, so that the custom-label item above -- which steps no %% counter and therefore has no anchor -- records nothing. It expands to %% nothing at all unless the links are switched on. %% \lx@endprevpar closes the PREVIOUS item's paragraph before the counter %% is stepped, and every counter-stepping core below starts with it. It %% looks redundant -- \item ends that paragraph anyway, a few tokens later %% -- and is not: hyperref hangs the example's destination off %% \refstepcounter, through \Hy@raisedlink, which in horizontal mode %% contributes a zero-width \smash box to the line being built. Stepping %% the counter first therefore leaves the NEXT example's anchor at the end %% of the PREVIOUS example's last line, where it does two kinds of damage: %% - the line breaker may put that box, and the \parfillskip after it, on %% a line of its own. On a full line it does, and the example is %% followed by an empty line -- a hole between (b) and (c) with nothing %% in the source to explain it, which is how this was found; %% - across a page break the destination lands on the page the previous %% example ends on, so \ref jumps one page short. %% In vertical mode \Hy@raisedlink adds the destination to the vertical %% list and neither happens, which is why the outermost \ea -- reached in %% vertical mode already -- never showed the bug. %% \unskip twice, and the \if@inlabel guard, are \@item's own: \item is %% what would otherwise be doing this, and a pending label (an \item whose %% paragraph has not started) must be left for it to open. %% %% \lx@glt@langend first, while the previous item's paragraph is still the %% current one: a translation's language Span (\GlossTransLang) is closed %% by the example's exits, and an item boundary is not one. Without it the %% Span of (a)'s translation was still open when (b) began, (b) was built %% inside it -- an LI inside an LI, every later example number one level %% too deep, veraPDF LI-LI-1 on all profiles -- and the same for %% consecutive \ex items of one exe list. It is a no-op when no Span is %% open, which is every item of a document without \GlossTransLang. \def\lx@endprevpar{% \lx@glt@langend \if@inlabel\else\ifhmode\unskip\unskip\par\fi\fi} \def\lx@main@core{% \lx@endprevpar \lx@customexfalse \def\lx@refbase@main{\Exarabic{ExNo}}\def\lx@refbase@fn{\Exroman{FnExNo}}% \if@noftnote \stepcounter{lxExSerial}% \refstepcounter{ExNo}\def\lx@itemlabel{\lx@labelExNo}% \else \stepcounter{lxFnExSerial}% \refstepcounter{FnExNo}\def\lx@itemlabel{\lx@labelFnExNo}% \fi \lx@relref@record\lx@refcheck@main} %% A custom-labelled example steps no NUMBER (that is the point of a custom %% label) but it does step its own anchor serial, so that its sub-examples %% get anchors of their own (lxcex..a ...), and it gives its %% sub-examples their reference base: the label, brackets off. It still %% records nothing for the relative references, which count numbers. \ExplSyntaxOn %% The label with its brackets taken off, if it has the example brackets %% (\ExLBr ... \ExRBr, or the footnote pair in a footnote): "(7$'$)" gives %% "7$'$", so that \theSubExNo puts it back between \theExLBr and %% \theExRBr exactly as it does a number -- "(7$'$a)" -- and follows the %% bracket settings the same way. A label without them is used as it is. \tl_new:N \l__lx_refbase_tl \bool_new:N \l__lx_refbase_bracketed_bool %% Is the first (#2: \ExLBr) or last (\ExRBr) item of the label #1 the %% example bracket -- as the character, "(7)", or as the command that %% prints it, which is how langsci's \exr and \exp spell their labels %% (\theExLBr\pref{x}\theExRBr)? Missing the second, \ref of an \exr %% item printed "(1)" where [langsci]'s \ref of a number prints "1", and %% "(\ref{x})" in the text came out "((1))". #3 is the footnote pair. %% \tl_item:nn hands its token back under \exp_not:n, so the command is %% compared by name and not by what it would print. \prg_new_conditional:Npnn \__lx_refbase_end:nNN #1#2#3 { p } { \bool_lazy_or:nnTF { \str_if_eq_p:ee { \tl_item:nn {#1} { \token_if_eq_meaning:NNTF #2 \ExLBr { 1 } { -1 } } } { \legacy_if:nTF { @noftnote } {#2} {#3} } } { \str_if_eq_p:ee { \tl_item:nn {#1} { \token_if_eq_meaning:NNTF #2 \ExLBr { 1 } { -1 } } } { \legacy_if:nTF { @noftnote } { \exp_not:c { the \cs_to_str:N #2 } } { \exp_not:c { the \cs_to_str:N #3 } } } } { \prg_return_true: } { \prg_return_false: } } \cs_new_protected:Npn \lx_refbase_custom:n #1 { \tl_set:Nn \l__lx_refbase_tl {#1} \bool_set_false:N \l__lx_refbase_bracketed_bool \bool_lazy_and:nnT { \int_compare_p:nNn { \tl_count:n {#1} } > { 2 } } { \bool_lazy_and_p:nn { \__lx_refbase_end_p:nNN {#1} \ExLBr \FnExLBr } { \__lx_refbase_end_p:nNN {#1} \ExRBr \FnExRBr } } { \tl_set:Ne \l__lx_refbase_tl { \tl_range:nnn {#1} { 2 } { -2 } } \bool_set_true:N \l__lx_refbase_bracketed_bool } \cs_set_eq:NN \lx@refbase@main \l__lx_refbase_tl \cs_set_eq:NN \lx@refbase@fn \l__lx_refbase_tl } \cs_new_eq:NN \lx@refbase@custom \lx_refbase_custom:n %% What \ref prints for the example: the label as a number would be %% printed, brackets off and put back by \lx@labelCustom -- so \ExBareRefs %% and [langsci] give "7" where \ref of a number gives "3" -- or, without %% the example brackets, the label as it stands. It is what %% \thelxCustomSerial prints, so everything that builds a reference from %% the counter builds this: \refstepcounter's \@currentlabel, cleveref's %% record, and the kernel's first aid for cleveref, which REBUILDS that %% record at \label from \@currentcounter and would undo anything set %% beside it. Until 2026-10-04 the example recorded nothing: \ref of it %% printed an empty string, with no warning. %% %% The label's own tokens are kept from expansion (\exp_not:n) until a %% \label writes them, which is when LaTeX expands any \@currentlabel. %% \refstepcounter expands \thelxCustomSerial at once, and a label is %% free text that was never expanded before -- \exr's holds \pref -- so %% without this every custom label with a fragile command in it would %% have become an error, labelled or not. \cs_new_protected:Npn \lx_custom_reftext:n #1 { \bool_if:NTF \l__lx_refbase_bracketed_bool { \tl_set:Ne \lx@custom@ref { \exp_not:N \lx@labelCustom { \exp_not:N \exp_not:n { \exp_not:V \l__lx_refbase_tl } } } } { \tl_set:Nn \lx@custom@ref { \exp_not:n {#1} } } } \cs_new_eq:NN \lx@custom@reftext \lx_custom_reftext:n \ExplSyntaxOff %% \refstepcounter, not \stepcounter: the serial is what gives the example %% a destination of its own (lxCustomSerial.lxcex.), \@currentHref and %% \@currentlabel, as ExNo gives a numbered one. The text is fixed first, %% because \refstepcounter reads it. The serial steps by TWO: cleveref %% compresses a list into a range when the counter values are consecutive, %% and consecutive serials made \cref{seven,twelve,fifteen} print "(7) to %% (15)" -- a range over labels that are not a sequence. Even values are %% never consecutive. (And it starts at 2^30, against ranges with the %% NUMBERED examples: see the counter.) \lx@endprevpar first, for the reason it opens %% \lx@main@core. \def\lx@custom@ref{} \def\thelxCustomSerial{\lx@custom@ref} \def\lx@custom@begin#1{% \lx@endprevpar \lx@refcheck@custom \lx@customextrue \lx@refbase@custom{#1}% \lx@custom@reftext{#1}% \stepcounter{lxCustomSerial}% \refstepcounter{lxCustomSerial}} \def\lx@exstart@normal{\lx@main@core\lx@mainlist\lx@scanjudge} \def\lx@mainitem{\lx@main@core\lx@scanjudge} %% --- public API (Protocol A: top-level items) ------------------------- %% \lx_item_main: step the counter, OPEN the main list, then item. %% For an example that brings its own list, i.e. the %% \ex. case. %% \lx_item_main:n the same with the given label, and no counter step %% (the \ex.[custom] case). %% \lx_item_next: step the counter and item, WITHOUT opening a list: %% for a second example inside a batch whose list is %% already open, i.e. the \ex inside exe case. %% All three end by scanning for a judgment and emitting the item, so the %% example text may follow immediately. \ExplSyntaxOn \cs_new_eq:NN \lx_item_main: \lx@exstart@normal \cs_new_eq:NN \lx_item_main:n \lx@custom@item \cs_new_eq:NN \lx_item_next: \lx@mainitem \ExplSyntaxOff % the label box: number flush left; an auto-detected judgment is hung % into the margin at the text edge via \jdg's zero-width \llap, so % example texts align whether or not they carry a judgment, at every % nesting level and for marks of any width. The kernel \item is used % under a saved name, because the xlist environment binds \item to the % sub-example machinery locally. \let\lx@kernel@item\item \AtBeginDocument{\let\lx@kernel@item\item} %% The label is set flush left in a box of exactly \labelwidth. Forcing %% the width (rather than the older trick of a trailing \hfil in a %% natural-width label) is what makes flush-left survive PDF tagging: the %% latex-lab block code re-boxes any label narrower than \labelwidth to %% \labelwidth using its own alignment, which defaults to flush RIGHT; a %% label that already fills \labelwidth is left untouched. Classic %% (untagged) output is identical. %% %% "Flush left" means flush against the OUTER edge, away from the text, %% and in a right-to-left document that edge is the right one. LuaTeX %% mirrors [l] with the paragraph direction by itself. XeTeX does not: %% under bidi (which polyglossia loads for Hebrew, Arabic, ...) [l] stays %% physically left, so in a right-to-left list every number and letter sat %% against its text, 14pt and 8pt in from where the mirror of a %% left-to-right list puts them, while the text and the marks were in the %% right place. So under XeTeX in a right-to-left paragraph the side is %% r. bidi's \if@RTL says which; LuaTeX sets it too, and must not be %% given r, since there it would be mirrored a second time. \ExplSyntaxOn \cs_new:Npn \lx@labelside { \sys_if_engine_xetex:TF { \cs_if_exist:NTF \if@RTL { \legacy_if:nTF { @RTL } { r } { l } } { l } } { l } } \ExplSyntaxOff \def\lx@flushlabel#1{\makebox[\labelwidth][\lx@labelside]{\lx@dir@label{\lx@labelfont#1}}} %% Examples follow the direction of the document's MAIN language (decided %% 2026-10-10, doc/DEFERRED-DECISIONS.md): in a Hebrew document an example %% inside \begin{english} is laid out as every other example there -- %% number against the right margin, text indented from the right, letters %% and marks to the right of what they label -- and only the words in each %% line run left to right. The reverse holds for a Hebrew example in an %% English document. So the example's LIST takes the main direction, and %% after each label the text switches to the example's OWN direction. %% %% The main direction is bidi's and luabidi's \if@RTLmain, which %% polyglossia sets for a right-to-left main language. Without either %% package there is nothing to mirror and none of this runs -- and none of %% it runs without polyglossia either (\g__lx_dir_avail_bool), because %% \if@RTLmain is not only theirs: babelbidi (2026) defines it as a MACRO %% under LuaTeX, which \legacy_if:n cannot read ("You can't use \unless %% before macro": no babelbidi document built), and babel's own bidi %% loads bidi under XeTeX, whose real flag then switched all of this on in %% a document none of it was written or tested for. Right-to-left %% documents under babel are undecided, see doc/DEFERRED-DECISIONS.md. %% %% The two engines need two mechanisms, and each was found the hard way: %% - LuaTeX switches with \linedirection after the label, which it carries %% across line breaks. But its direction stack respects TeX groups, and %% a sub-level opens one (\a.'s \begingroup, xlist's \begin) WHILE the %% main item's paragraph is still running: the main item's own %% direction, left in force at the outer level, then re-opened around %% the whole sub-item, label included, and the letter landed inside the %% text. So the main direction is put back before every sub-level group %% opens (\lx@dir@beforegroup), with the paragraph's trailing space %% removed first -- a direction node after it keeps \unskip from it, %% and the line came out 3.6pt short. %% - XeTeX has no direction register, only \beginL/\beginR segments, which %% are nodes and not group state. One is opened after the label and %% closed in para/end, BEFORE \par adds \parfillskip: left open, the fill %% went inside the segment and a short last line stood at the far end %% of the measure, away from its label. \ExplSyntaxOn \bool_new:N \g__lx_dir_avail_bool % polyglossia sets the main direction \bool_new:N \l__lx_dir_flip_bool % this example's list is mirrored \int_new:N \l__lx_dir_own_int % its own direction: 0 LTR, 1 RTL \int_new:N \l__lx_dir_main_int % the document's main direction \bool_new:N \g__lx_dir_open_bool % XeTeX: a segment is open \bool_new:N \l__lx_dir_inbox_bool % ... and we are in a box inside it \int_new:N \g__lx_dir_open_int % ... and which: 0 L, 1 R \cs_new:Npn \__lx_dir_current: { \sys_if_engine_luatex:TF { \int_use:N \tex_textdirection:D } { \legacy_if:nTF { @RTL } { 1 } { 0 } } } \msg_new:nnn { linguexx } { rtl-babel } { babel's~right-to-left~support~is~on,~but~linguexx~lays~out~examples~ in~a~right-to-left~document~only~with~polyglossia.~Examples~are~set~ left~to~right,~as~in~a~left-to-right~document. } \AddToHook { begindocument } { \IfPackageLoadedTF { polyglossia } { \cs_if_exist:NT \if@RTLmain { \bool_gset_true:N \g__lx_dir_avail_bool } } { \bool_lazy_and:nnT { \cs_if_exist_p:N \bbl@bidimode } { \int_compare_p:nNn { \bbl@bidimode } > { 0 } } { \msg_warning:nn { linguexx } { rtl-babel } } } } %% From \lx@mainlist, BEFORE the list's group opens: decide, and set the %% main direction at that level. Not inside the list: LuaTeX wraps an %% item's paragraph in the direction of every enclosing group, so the %% hebrew environment's own direction, left outside, wrapped the number %% too and set it inside the text. But the dot syntax opens no group of %% its own around the list, so what is set here outlives the example %% unless it is put back -- the next example in the same environment then %% found the wrong direction -- and \lx@dir@listend, from \lx@closelist, %% puts it back when the MAIN list closes. \cs_new_protected:Npn \lx@dir@listbegin { \bool_set_false:N \l__lx_dir_flip_bool \bool_lazy_and:nnT { \g__lx_dir_avail_bool } { \bool_lazy_or_p:nn { \sys_if_engine_luatex_p: } { \sys_if_engine_xetex_p: } } { \int_set:Nn \l__lx_dir_own_int { \__lx_dir_current: } \int_set:Nn \l__lx_dir_main_int { \legacy_if:nTF { @RTLmain } { 1 } { 0 } } \int_compare:nNnF { \l__lx_dir_own_int } = { \l__lx_dir_main_int } { \bool_set_true:N \l__lx_dir_flip_bool \int_compare:nNnTF { \l__lx_dir_main_int } = { 1 } { \setRTL } { \setLTR } } } } \cs_new_protected:Npn \lx@dir@listend { \bool_lazy_and:nnT { \l__lx_dir_flip_bool } { \int_compare_p:nNn { \lx@subdepth } = { 0 } } { \bool_set_false:N \l__lx_dir_flip_bool \int_compare:nNnTF { \l__lx_dir_own_int } = { 1 } { \setRTL } { \setLTR } } } %% From \lx@makeitem, before the item: the label is set in the main %% direction, which an earlier item's switch may have left otherwise. \cs_new_protected:Npn \lx@dir@itembefore { \bool_if:NT \l__lx_dir_flip_bool { \sys_if_engine_luatex:TF { \tex_textdirection:D = \l__lx_dir_main_int \scan_stop: } { \__lx_dir_set_rtl_flag:n { \l__lx_dir_main_int } } } } %% XeTeX: bidi's \if@RTL says which direction the text is in, and bidi's %% own commands read it -- \texthebrew opens an R segment only where the %% flag says the text is left to right. An own-direction segment sets it %% to the example's direction, so a Hebrew translation of an English %% example in a Hebrew document is not taken for text already right to %% left and set in reverse word order; before each label it is the main %% direction again, which \lx@labelside reads. \cs_new_protected:Npn \__lx_dir_set_rtl_flag:n #1 { \int_compare:nNnTF {#1} = { 1 } { \use:c { @RTLtrue } } { \use:c { @RTLfalse } } } %% ... and after it: the text in the example's own direction. This ends %% \lx@makeitem and stands in for its \ignorespaces. The paragraph is %% started here, after the label, and switched -- unless the item has no %% text of its own and a sub-level follows at once (\ex. \a. ..., or an %% xlist's \begin): LaTeX keeps such a label pending and sets it on the %% first sub-example's line, which a paragraph started here prevents (the %% number stood alone above), and the sub-item switches after both labels. %% Switching from the item's \everypar instead was tried and fails: under %% the kernel's paragraph hooks that no longer runs between label and text, %% and a Hebrew example's text came out left to right under LuaTeX. \cs_new_protected:Npn \lx@dir@itemafter { \bool_if:NTF \l__lx_dir_flip_bool { \mode_if_horizontal:TF { \lx@dir@textswitch \tex_ignorespaces:D } { \peek_meaning_ignore_spaces:NTF \a { } { \peek_meaning:NTF \begin { } { \lx@dir@startswitch } } } } { \tex_ignorespaces:D } } \cs_new_protected:Npn \lx@dir@startswitch { \mode_leave_vertical: \lx@dir@textswitch \tex_ignorespaces:D } \cs_new_protected:Npn \lx@dir@textswitch { \bool_lazy_and:nnT { \l__lx_dir_flip_bool } { ! \bool_lazy_and_p:nn { \g__lx_dir_open_bool } { ! \l__lx_dir_inbox_bool } } { \sys_if_engine_luatex:TF { \tex_linedirection:D = \l__lx_dir_own_int \scan_stop: } { \int_compare:nNnTF { \l__lx_dir_own_int } = { 1 } { \tex_beginR:D } { \tex_beginL:D } \bool_gset_true:N \g__lx_dir_open_bool \bool_set_false:N \l__lx_dir_inbox_bool \int_gset_eq:NN \g__lx_dir_open_int \l__lx_dir_own_int \__lx_dir_set_rtl_flag:n { \l__lx_dir_own_int } } } } %% Whether a label is set in a direction of its own: bidi support is %% there, and the example is mirrored or the whole document right to left. %% \lx@dir@label (the box) and \lx@dir@labelsep (XeTeX's kern) ask the %% same question, so it is asked in one place. \prg_new_conditional:Npnn \__lx_dir_label_own: { p } { \bool_lazy_and:nnTF { \g__lx_dir_avail_bool } { \bool_lazy_or_p:nn { \l__lx_dir_flip_bool } { \legacy_if_p:n { @RTLmain } } } { \prg_return_true: } { \prg_return_false: } } %% A label reads in the DOCUMENT's direction (decided 2026-10-10): in a %% Hebrew document ".a" as well as ".א", the period between letter and text %% as in the mirror image of "a. text", and as bidi itself sets "a." in a %% right-to-left paragraph; in an English document "a." and "א.". The %% label belongs to the example's frame, like the side its number is on, %% not to its text. So it is set in the main direction: always in a %% right-to-left document, where the engines disagreed before (Latin %% letters of a Hebrew example: "a." under XeTeX, ".a" under LuaTeX), and %% in a mirrored example of a left-to-right one. Anywhere else nothing is %% wrapped and nothing changes. \cs_new_protected:Npn \lx@dir@label #1 { \bool_lazy_and:nnTF { \bool_lazy_or_p:nn { \sys_if_engine_luatex_p: } { \sys_if_engine_xetex_p: } } { \__lx_dir_label_own_p: } { \sys_if_engine_luatex:TF { \legacy_if:nTF { @RTLmain } { \tex_hbox:D dir ~ TRT { \__lx_dir_label_text: #1 } } { \tex_hbox:D dir ~ TLT {#1} } } { \legacy_if:nTF { @RTLmain } { \hbox:n { \__lx_dir_label_span:n { \tex_beginR:D #1 \tex_endR:D } } } { \hbox:n { \tex_beginL:D #1 \tex_endL:D } } } } {#1} } %% Right to left, polyglossia's unibidi-lua mirrors ( ) [ ] in a font in %% luaotfload's node mode -- the Latin ones -- by REPLACING the glyph's %% character with its mirror (mirror_char, setchar), so the PDF's text %% layer gets the mirror too: the label of an English example in a Hebrew %% document read ")2(" to a screen reader, though "(2)" on the page. It %% happens without linguexx. In harf mode it leaves mirroring to %% luaotfload, which keeps the text, so a Hebrew example's number read %% right all along. The label's marked content -- the %% kernel's, belonging to its Lbl -- is reopened with the label as typed %% for /ActualText: the tree keeps its shape, the page its ink. %% The text is \lx@itemlabel's, which \lx@makeitem hands to \item: the %% argument here is the kernel's block code by then, its label in a %% variable of its own. %% The number's brackets come as \protect\theExLBr (\theExNo), and %% \text_purify:n drops a \protect'ed command: "(2)" was purified to "2". %% Their purify equivalents are what they print, read when the label is, %% so a \renewcommand\ExLBr{[} holds there too. \text_declare_purify_equivalent:Nn \theExLBr { \ifparens \else \ExLBr \fi } \text_declare_purify_equivalent:Nn \theExRBr { \ifparens \else \ExRBr \fi } \text_declare_purify_equivalent:Nn \theFnExLBr { \ifparens \else \FnExLBr \fi } \text_declare_purify_equivalent:Nn \theFnExRBr { \ifparens \else \FnExRBr \fi } \cs_new_protected:Npn \__lx_dir_label_text: { \lx@tag@if@active:T { \tag_mc_if_in:T { \tag_mc_end: \exp_args:Ne \tag_mc_begin:n { tag = Lbl , actualtext = { \text_purify:n { \lx@itemlabel } } } } } } %% XeTeX writes a right-to-left label's nodes in visual order, so a %% Hebrew sub-label "א." read ".א" in the tree. The label goes into a %% Span carrying it as typed, NESTED in the kernel's marked content: ending %% and reopening that, as under LuaTeX, left an empty one behind, since %% XeTeX writes BDC and EMC whether or not anything comes between. The %% Span's operators are outside the R segment, which reverses everything %% inside it, marked-content operators included (doc/OPEN-ISSUES.md). %% \pdf_bdc_shipout:ee writes the dictionary inline, where \pdf_bdc:nn %% under xdvipdfmx makes it a /Properties resource, which show-pdf-tags %% does not resolve; it expands its arguments only at shipout, by when %% the string is out of scope -- hence the \exp_args. \str_new:N \l__lx_dir_label_str \cs_new_protected:Npn \__lx_dir_label_span:n #1 { \bool_lazy_and:nnTF { \lx@tag@if@active_p: } { \tag_mc_if_in_p: } { \exp_args:NNe \str_set_convert:Nnnn \l__lx_dir_label_str { \text_purify:n { \lx@itemlabel } } { default } { utf16/hex } \exp_args:Nne \pdf_bdc_shipout:ee { Span } { /ActualText ~ < \l__lx_dir_label_str > } #1 \pdf_emc: } {#1} } %% XeTeX sets a run of letters as ONE word node, and reversing a segment %% reverses nodes, not what is inside them: "a." stayed "a." in a %% right-to-left label, while LuaTeX gave ".a". A zero kern between the %% letter and its delimiter makes them two nodes. Only where a label is %% set in a direction of its own (\lx@dir@label: XeTeX, a right-to-left %% document or a mirrored example -- a Hebrew one in an English document %% read ".א" for "א." the same way), because anywhere else it would part %% a kerning pair of the font and move the period. \cs_new_protected:Npn \lx@dir@labelsep { \bool_lazy_and:nnT { \sys_if_engine_xetex_p: } { \__lx_dir_label_own_p: } { \tex_kern:D \c_zero_dim } } %% Before a sub-level's group opens (see above). XeTeX has no direction %% to put back, but it has bidi's flag, which the main item's own-direction %% segment left in the example's direction: bidi then opened the sub-list %% as a right-to-left one, its margin on the right, and a Hebrew example's %% letters in an English document stood a whole level to the left. \cs_new_protected:Npn \lx@dir@beforegroup { \bool_if:NT \l__lx_dir_flip_bool { \sys_if_engine_luatex:TF { \mode_if_horizontal:T { \tex_unskip:D \tex_unskip:D \tex_textdirection:D = \l__lx_dir_main_int \scan_stop: } } { \__lx_dir_set_rtl_flag:n { \l__lx_dir_main_int } } } } %% Also from \__lx_gl_rtl_place:, before a grid's box goes into the item's %% paragraph -- when the segment is an R one. TeX--XeT reflects the lines %% OF a box it reverses: inside the item's R segment a wrapped Hebrew %% grid's short second line stood at the left. An L segment is the %% opposite case and must stay open: in a right-to-left paragraph %% everything outside it is reversed, and an English grid's lines came out %% ragged on the left when it was closed too. %% XeTeX: close the segment before the paragraph's fill -- the paragraph %% that OPENED it. A paragraph inside a box in it ends too, and runs %% para/end too: a gloss is a \vtop of its own paragraph, and closing the %% item's segment there left the item's real end with nothing to close, so %% the fill went inside the segment and the grid stood at the far end of %% the line from its number. Every vertical box marks itself %% (\everyvbox, local to the box); the item clears the mark when it opens %% the segment, so an example set inside a minipage still works. \cs_new_protected:Npn \lx@dir@parend { \bool_lazy_and:nnT { \g__lx_dir_open_bool } { ! \l__lx_dir_inbox_bool } { \tex_unskip:D \int_compare:nNnTF { \g__lx_dir_open_int } = { 1 } { \tex_endR:D } { \tex_endL:D } \bool_gset_false:N \g__lx_dir_open_bool } } %% The translation after \glt is in the DOCUMENT's direction, not the %% example's: it is the reader's language, as \GlossRTL leaves it where it %% is in a left-to-right document, and this is the mirror image of that %% (decided 2026-10-10). \glt puts the main direction back before its %% paragraph (\lx@dir@itembefore). Opening an own-direction segment there %% instead, as a first version did, set a Hebrew translation of an English %% example in reverse word order under XeTeX, or at the far left of the %% page once bidi's flag said left to right. \AddToHook { para/end } { \lx@dir@parend } \tex_everyvbox:D \exp_after:wN { \tex_the:D \tex_everyvbox:D \bool_set_true:N \l__lx_dir_inbox_bool } \clist_map_inline:nn { xlist , xlista , xlisti , xlistn , xlistA , xlistI } { \AddToHook { env / #1 / before } { \lx@dir@beforegroup } } \ExplSyntaxOff %% \lx@relref@anchor places the hyperref destination for this example when %% linguexx is the one making them (see \lx_relref_record:). It goes %% after the judgment, so the mark still hangs from the label edge, and %% before \ignorespaces, which must stay last. It expands to nothing at %% all in a document whose anchors hyperref is placing itself. %% %% \@currentlabel is saved round the \item and put back after it. The %% kernel's block code (latex-lab, 2026-10-27) gives the legacy item %% instance label-ref=\def\@currentlabel{#1}\def\@currentlabelname{#1}, %% so \item[