MLTeX EncTeX and SyncTeX TeX extensions

Some background details on TeX extensions

TeX has a long history of development—now over 4 decades! During that time new “versions” of TeX have been developed—such as e-TeX, pdfTeX, XeTeX, LuaTeX and so forth. In addition to completely new engines, with a whole suite of extra features, there have been various extensions that typically focus on implementing a specific feature to enhance Knuth’s original software.

With the advent of modern TeX engines, such as XeTeX and LuaTeX, the need for, and use of, some legacy TeX extensions has faded as the newer TeX engines bring powerful additional functionality—such as built-in support for UTF-8 encoded text. The purpose of this page is to provide a few notes on extensions, including MLTeX and EncTeX, you might still see referenced in a TeX engine’s command line options. MLTeX and EncTeX are both no longer developed but one very useful extension, SyncTeX, continues to be actively developed and is supported by Overleaf.

MLTeX

Multi-lingual TeX, or MLTeX, was developed during the years 1992–5 as a modification of TeX to allow hyphenation of words with accented letters using ordinary Computer Modern (CM) fonts.

Activating (enabling) MLTeX

MLTeX is designed to work as part of a so-called .fmt (format file) which is, in effect, a precompiled binary version of a set of TeX macros. To create format files you put a TeX engine into its so-called INI mode, process the macro collection and then issue the \dump command to write out the .fmt file. See this article for a discussion of INI modes.

To enable MLTeX, for those engines that support it, such as pdfTeX, you would write

pdftex -ini -mltex *pdfetex.ini

where pdfetex.ini is a file contained in the TeX Live distribution.

Why the asterisk (*)? The asterisk puts pdfTeX into so-called “extended mode” which enables support of the e-TeX primitives. If you omit it and type pdftex -ini -mltex pdfetex.ini you will receive an error:

This is pdfTeX, Version 3.14159265-2.6-1.40.19 (INITEX)
 \write18 enabled.
MLTeX v2.2 enabled
(c:/texmf-dist/tex/plain/config/pdfetex.ini
(c:/texmf-dist/tex/generic/tex-ini-files/pdftexconfig.tex)
(c:/texmf-dist/tex/plain/etex/etex.src

! e-TeX  fatal error: this file can be processed only in extended mode;
  did you perhaps forget the asterisk?

Note: Instead of using an asterisk to enable “extended mode” you can use the -etex commandline option instead:

pdftex -ini -etex -mltex pdfetex.ini

Note: Using the -mltex option on the command line with a .fmt file that was not generated with MLTeX enabled will not work: MLTeX has to be “activated” via an apprpriate format file.

When you use the -mltex option during .fmt file creation, a boolean variable with a value of “true” is written into the .fmt file telling the TeX engine that the MLTeX extensions have been activated and should be made available to the user. If you don’t explicitly set the -mltex option then the corresponding variable will be “false”. If that variable’s value is detected as “false” (when that .fmt file is re-loaded during a TeX run) then trying to use the MLTeX extensions will result in an Undefined command... error—because the MLTeX primitives will not have been registered (loaded) and thus are not recognized.

After running pdftex -ini -mltex *pdfetex.ini on the command line and you’ll see something similar to the following text, showing that MLTeX v2.2 is enabled:

This is pdfTeX, Version 3.14159265-2.6-1.40.19 (INITEX)
\write18 enabled.
entering extended mode
MLTeX v2.2 enabled
....
....
....
Beginning to dump on file pdfetex.fmt
....
No pages of output.
Transcript written on pdfetex.log.

This results in a .fmt file called pdfetex.fmt that enables you to use the MLTeX commands:

pdftex -fmt=pdfetex <yourfile.tex>

where <yourfile.tex> can make use of the MLTeX-specific primitives

MLTeX was originally distributed as a separate change file, mltex.ch, that could be merged with Knuth’s master source code (tex.web) to derive a new version of TeX, called MLTeX, which contained a number of new primitives. Today, the MLTeX code from mltex.ch is subsumed into the main tex.ch change file which is used to build nearly all TeX engines.

MLTeX primitives

The following notes are mostly prepared and reproduced from the file web2c.info contained in the TeX Live distribution.

• \charsubdef: Character substitution definitions. Declares how to construct an accented character glyph (not necessarily existing in the current font) using two character glyphs (that do exist).

  \charsubdef COMPOSITE [=] ACCENT BASE
  

  Each of COMPOSITE, ACCENT, and BASE are font glyph numbers, expressed in the usual TeX syntax: `\e symbolically, '145 for octal, "65 for hex, 101 for decimal. Thus it defines whether a character glyph code, either typed as a single character or using the \char primitive, will be mapped to a font glyph or to an \accent glyph construction.

  For example, if you assume glyph code 138 (decimal) for an e-circumflex and you are using the Computer Modern fonts, which have the circumflex accent in position 18 and lowercase ‘e’ in the usual ASCII position 101 decimal, you would use \charsubdef as follows:

  \charsubdef 138 = 18 101
  

• \charsubdefmax: Sets largest value in the \charsubdef list. For example:

  \charsubdefmax=-1 % disable all substitutions
  \charsubdefmax=256 % enable all substitutions
  

• \tracingcharsubdef: Tracing substitutions. To help diagnose problems with \charsubdef, MLTeX provides a primitive parameter, \tracingcharsubdef. If positive, every use of \charsubdef will be reported. This can help track down when a character is redefined.

EncTeX

EncTeX was developed during the years 1997–2004 as a TeX extension which provides flexible input/output reencoding through the addition of new primitives—for example, a means of trans­lat­ing in­put on the way into TeX. It al­lows, for ex­am­ple, trans­la­tion of multi­byte se­quences, such as UTF-8 en­cod­ing, for use in standard 8-bit TeX engines.

Activating (enabling) EncTeX

As with MLTeX, EncTeX is also designed to work as part of a so-called .fmt (format file)–see MLTeX, above, for an explanation of format files/INI mode.

To enable EncTeX, for those engines that support it, such as pdfTeX, you would create an EncTeX-enabled format file by writing

pdftex -ini -enc *pdfetex.ini

where pdfetex.ini is a file contained in the TeX Live distribution.

Why the asterisk (*)? The asterisk puts pdfTeX into so-called “extended mode” which enables support of the e-TeX primitives. If you omit it and type pdftex -ini -mltex pdfetex.ini you will receive an error:

This is pdfTeX, Version 3.14159265-2.6-1.40.19 (INITEX)
 \write18 enabled.
MLTeX v2.2 enabled
(c:/texmf-dist/tex/plain/config/pdfetex.ini
(c:/texmf-dist/tex/generic/tex-ini-files/pdftexconfig.tex)
(c:/texmf-dist/tex/plain/etex/etex.src

! e-TeX  fatal error: this file can be processed only in extended mode;
  did you perhaps forget the asterisk?

Note: Instead of using an asterisk to enable “extended mode” you can use the -etex commandline option instead:

pdftex -ini -etex -mltex pdfetex.ini

The -enc command line option triggers the pdfTeX engine to enable EncTeX by writing some data into the resulting pdfetex.fmt format file.

Note: Using the -enc option on the command line with a .fmt file that was not generated with EncTeX enabled will not work: EncTeX has to be “activated” via an appropriately-generated format file.

EncTeX primitives

EncTeX adds 10 new primitive commands to a TeX engine:

• 3 primitives in the initial release (1997):
• \xordcode
• \xchrcode
• \xprncode
• 7 additional primitives (for UTF-8 support) in 2003:
• \mubyte
• \endmubyte
• \mubytein
• \mubyteout
• \mubytelog
• \specialout
• \noconvert

The use and behaviour of these primitives is well documented in the EncTeX reference manual, written by EncTeX’s author, Petr Olšák. To avoid unneccesary duplication here, interested readers are referred to that document.

Further information on EncTeX

• Second version of EncTeX: UTF-8 support: an article by EncTex’s author, Petr Olšák.
• EncTeX reference manual (in English).

Activating (enabling) both MLTeX and EncTeX

To enable both MLTeX and EncTeX you would generate a suitable format file using a commandline such as

pdftex -ini -enc -mltex *pdfetex.ini

where pdfetex.ini is a file contained in the TeX Live distribution.

Why the asterisk (*)? The asterisk puts pdfTeX into so-called “extended mode” which enables support of the e-TeX primitives. If you omit it and type pdftex -ini -enc -mltex pdfetex.ini you will receive an error:

This is pdfTeX, Version 3.14159265-2.6-1.40.19 (INITEX)
 \write18 enabled.
MLTeX v2.2 enabled
(c:/texmf-dist/tex/plain/config/pdfetex.ini
(c:/texmf-dist/tex/generic/tex-ini-files/pdftexconfig.tex)
(c:/texmf-dist/tex/plain/etex/etex.src

! e-TeX  fatal error: this file can be processed only in extended mode;
  did you perhaps forget the asterisk?

Note: Instead of using an asterisk to enable “extended mode” you can use the -etex commandline option instead:

pdftex -ini -etex -enc -mltex pdfetex.ini

After running the above commandline you would see a message, such as the following, to inform you that MLTeX and EncTeX are both enabled:

This is pdfTeX, Version 3.14159265-2.6-1.40.19 (INITEX)
 \write18 enabled.
entering extended mode
MLTeX v2.2 enabled
 encTeX v. Jun. 2004, reencoding enabled.
....
....
....
Beginning to dump on file pdfetex.fmt
....
No pages of output.
Transcript written on pdfetex.log.

and the pdfetex.fmt format file, enabling both MLTeX and EncTeX, would be produced.

SyncTeX

The SyncTeX extension has been available since 2008 and continues to be actively developed by its creator, Jérôme Laurens. It is widely used, including by Overleaf, and is supported by all TeX engines in use today.

What does SyncTeX do?

When viewing a document typeset by TeX, perhaps for proofreading/checking prior to submission to a publisher, you might spot an error and need to fix it in the TeX source code. However, finding the right location in the TeX file can be tricky, especally if it is a substantial document. Equally, you might want to go the other way: jump from a position in a TeX source file directly to the page location in the final typeset PDF. In both cases, there is a need to synchronize a particular location in a TeX file and its corresponding location within a display of its typeset results: and that’s what SyncTeX provides—a way to automatically link locations in TeX files with the page positions in typeset documents.

If SyncTeX is enabled, it writes out a special file with extension .synctex.gz (gz compressed) or .synctex (uncompressed) depending on the commandline options provided to SyncTeX. SyncTeX-enabled text editors and (mostly) PDF viewers can use those files to perform the necessary synchronization between input (TeX code) and output (display of the typeset PDF). Clearly, this cannot just “happen” without the user’s text editor and PDF document viewer having appropriate functionality which enables communication of text file/PDF viewer locations—based on data contained in the .synctex.gz (or .synctex) file written-out by SyncTeX.

In addition to functionality within the TeX engine—to generate .synctex.gz files—SyncTeX also provides a special parser which reads .synctex.gz (or .synctex) files to extract the relevant page/line location information. Applications use that parser to provide support for SyncTeX.

Activating (enabling) SyncTeX

SyncTeX-enabled TeX engines have two ways to activate/disable SyncTeX:

• a commandline option: -synctex=NUMBER
• a command in the .tex file: the \synctex primitive

Using SyncTeX on the commandline

You can use the -synctex=NUMBER option when you run a TeX engine that supports SyncTeX, such as pdfTeX.

Examples

• Enable SyncTeX: output .synctex.gz (gz compressed file):

pdftex -synctex=1 myfile.tex

Produces myfile.synctex.gz

• Enable SyncTeX: output .synctex (uncompressed file):

pdftex -synctex=-1 myfile.tex

Produces myfile.synctex

• Disable SyncTeX:

pdftex -synctex=0 myfile.tex

No SyncTeX output is produced.

SyncTeX’s only primitive

SyncTeX provides one primitive, \synctex, that users can add to their .tex file:

• Enable SyncTeX: \synctex=1
• Disable SyncTeX: \synctex=0

Further information on SyncTeX

• A TUGBoat article by Jérôme Laurens, SyncTeX’s author.
• The GitHub repository of the SyncTeX parser.