Re: TeX live format generation question

On Fri, 24 Jul 2009 00:16:34 +0100,
real-address-in-sig@xxxxxxxxxxxxxxx (Rowland McDonnell) wrote:

Dan <luecking@xxxxxxxx> wrote:

(Rowland McDonnell) wrote:
I would like to add some custom formats to my MacTeX 2008 (TeX Live)
installation[1].

What is the approved method for doing so?

From tlmgr.pdf (obtained by running "texdoc tlmgr"):

If the files language-local.dat, language-local.def, fmtutil
-local.cnf, or
updmap-local.cfg are present under TEXMFLOCAL in the respective
directories, their contents will be simply merged into the final
files, with
no error checking of any kind.

One of the big problems is that I've not been able to figure out what
exactly tlmgr is for, nor how it operates, nor where it gets its ideas
of what needs doing from. Yes, I'm sure a lot of readers will think `Oh
well, he's too thick to learn this sort of stuff if it's not obvious to
him', but there we go. I don't think I'm too thick...

The tlmgr man page you quote from also states:

`The generate action overwrites any manual changes made in the
respective files: it recreates them from scratch.'

I can't see anything that explains what the generate action(s) generate
these files from and what files they generate - that is, the man page
says tlmgr generate fmttuil generates a fmtutil.cfg file, but it doesn't
say where the source data comes from nor where it's put (or how tlmgr
decides where to put it). Could you perhaps help?

This information I obtained by trial and error (and some
guessing so I can't claim it is all correct, sorry):

In TeX Live 2008 (it was changed in TeX Live 2009,
where I am still unsure), fmtutil.cnf is generated
from the contents of the directory TEXMFMAIN/fmtutil/ .
(Why TEXMFMAIN and not TEXMF(SYS)CONFIG? Who knows?)
The created file is placed in TEXMFSYSVAR.

In that directory, fmtutil-hdr.cnf is the header, and
the rest of the files are a few lines that call for
generating the obvious formats. That is, format.latex.cnf
are the lines for generating latex, pdflatex, lualatex and
pdflualatex.

If a particular format package is not installed, that
particular format.*.cnf file is either absent or its
lines are commented out. (I haven't figured out which
happens when, but it may be that I edited some of them.)

In addition, that fmtutil-local.cnf (in ones
TEXMFLOCAL/web2c) is added on the end.


Another point I'd find dead useful to know is (in tlmgr -gui): what is
the `Current installation source' listed at the top?

It is the network address of the mirror of TeX Live
that files will be fetched from when you update a package
or install a new one. When tlmgr -gui starts up, it gets
this information from the file
<your TL root>/tlpkg/texlive.tlpdb
on the line that begins "depend location:".


(for that matter, what do the `load' and `change' buttons do? The TeX
Live documentation explains that it's all obvious; unfortunately, I do
not find it so.)

"Load" causes tlmgr to fetch the latest texlive.tlpdb
(database of available packages and dependencies of
such packages) from that location. "Change" gives you
the opportunity to edit that "Current installation source"
to specify a different mirror.


And: it'd help an awful lot if someone would help me learn what the TeX
Live Package Database is (and where it lives and things). I've not
found a good explanation in the documentation so far.

The TeX Live Package Database is the file texlive.tlpdb,
either the local file mentioned above (which catalogues
only those packages installed on your system) or the
reference version located at the mirror (which catalogues
all available packages). One can open ones local version
and see how information is stored. Here are some sample
lines:

name GuIT
category Package
revision 7340
containersize 2729
containermd5 11e13004a313d3c223cbdd9ea7322694
doccontainersize 262299
doccontainermd5 8725ac828f3b3b04f8009895212a97a7
docfiles size=85
texmf-dist/doc/latex/GuIT/README
texmf-dist/doc/latex/GuIT/guit.pdf
texmf-dist/doc/latex/GuIT/guittest.pdf
texmf-dist/doc/latex/GuIT/guittest.tex
srccontainersize 17454
srccontainermd5 394aeab07456d9b71378b684e43bce75
runfiles size=4
texmf-dist/tex/latex/GuIT/guit.cfg
texmf-dist/tex/latex/GuIT/guit.sty

A package record is delimited by blank lines.
Each nonindented line is a data field: the first
word is the name of the field, the rest of the line
and/or the following indented lines are the data.
Indented lines are all files. (I do not know what
units are used in the "size=" data.)

Some records include lines that start with "depend"
and these are dependencies. If you install the package
these "depend" packages are also installed unless you
explicitly request no dependencies.


I put have four lines in the file texmf-local/web2c/fmtutil-local.cnf

Righto - ta. In the location specified by TEXMFLOCAL, yes?

Yes.


It seems to me that means I can only have one version of TeX set up for
use at one time, then. Is that right? I can switch from MacTeX 2007 to
MacTeX 2008 using the TeX prefs pane (MacOS X), but it seems that all
the versions of MacTeX are going to be using the same texmf-local
directory, which means if I want to have separated distros, I can't use
this mechanism?

If you mean you want a separate set of local formats,
then you need separate fmtutil-local.cnf files. Since
the actual definition of the variable TEXMFLOCAL is
defined inside the distribution, this can become
something different when you change from 2007 to 2008.

For example, on my system I have texmf/web2c/texmf.cnf
in both /TL/2009/ and /TL/2008/. Both have TEXMFLOCAL
set in such a way that it points to /TL/texmf-local/
However, I can create the file /TL/2008/texmf.cnf
(which is read before the supplied one in texmf/web2c/)
which could redefine TEXMFLOCAL as follows:

TEXMFLOCAL={$SELFAUTOPARENT\..\texmf-local,$SELFAUTOPARENT\texmf-local}
The first part is a copy of the default and goes up one
level, but the second stays inside /TL/2008 and points
to a different directory. A fmtutil-local.cnf in that
directory would also be used in generating fmtutil.cnf.

(not that I really need to keep the old version, but I'm keeping it
around for a bit just in case MacTeX 2008 breaks horribly; the current
version of TeXShop's horribly inclined to crash, I've noticed)

And they magically appear in texmf-var/web2c/fmtutil.cnf.

I'm a little confused here. What is it makes your additions appear in
the appropriate place? I'm guessing not /really/ magic - an invocation
of tlmgr generate <something>?

One puzzle is: what makes that location be chosen?

The file fmtutil.cnf is (re)generated whenever it might
conceivably be changed. For example, if you have the aleph
package installed and you elect to remove it
(you could do this in tlmgr -gui under the Remove tab),
then the lines from format.aleph.cnf would have to be removed
from fmtutil.cnf. Then it would be regenerated and in fact
it is generated from scratch as described above, with all
appropriate fmtutil-local.cnf appended.

OR you could call "tlmgr generate fmtutil" from a command
line to force it to be regenerated.

The location of the generated fmtutil.cnf is $TEXMFSYSVAR/web2c


Using an invocation of texconfig conf, I can see that
texlive2008/texmf-var/ is on lots of different specified paths and
there's nothing I can see which specifies the /web2c/ subdirectory of
that path.

The /web2c/ part of the path is pretty much hard-wired
into the system for most regenerated files.

So remake all formats in tlmgr -gui is the same as
fmtutil -all, and what *that* does is process
texmf-var/web2c/fmtutil.cnf to generate `all formats'?

Yes.


Could anyone tell me which path variable specifies that location in this
case?

Note: in the list below, $engine means tex or pdftex or
xetex or metapost or metafont, etc., based on the basic program
that creates (and uses) the format file.

- $TEXMFSYSVAR/web2c/$engine is used to place the generated
format files created by fmtutil-sys.
- $TEXMFVAR/web2c/$engine is used to place the generated
format files created by fmtutil.

- $TEXFORMATS is used by (tex-related) engines for seeking
out the format files (*.fmt).
- $MFBASES is used by metafont to find its formats (*.base).
- $MPMEMS is used by metapost to find its formats (*.mem).


`Re-create all formats' button.

It runs "fmtutil --all" which makes use of the automatically
generated fmtutil.cnf (that file is regenerated every time a
supported format is installed or updated).

Thanks - dead useful. But it raises another question:

Actually it's "fmtutil-sys --all".


Where do I find out what fmtutil -all considers to be a `supported
format'?

Whatever is in fmtutil.cnf.


And: `every time a supported format is installed or updated', okay but
by what mechanism must these `supported' formats be installed or updated
by for this to work? And what do you mean by `is installed or `updated'
for that matter?

My questions might seem witless, but I've never managed a TeX
installation any way other than manually so I'm rather ignorant at the
moment.

It has taken me almost 2 years of using tlmgr to
work out these things. They are NOT easy to find out.
The vast majority of users have never cared about or
needed any of this information. I, however, also needed
private formats and had to seek out these things

What does the installation and updating?

From the command line, if you run
tlmgr remove texsis
or
tlmgr install texsis
or
tlmgr update texsis
then all the files of the texsis format (a
somewhat obsolete format built on plain tex) will
be removed or added or updated (if it has changed).
All three will prompt a regeneration of fmtutil.cnf.
(The last two will also cause "fmtutil-sys --byfmt texsis"
to be run.) The files installed (or updated) will
include that format.texsis.cnf already mentioned.

One can also use the tlmgr gui, though I rarely do.


And what does `installed' mean in this context in any case?

1. The archive(s) containing all necessary files are
fetched from the TeX Live mirror location already
mentioned.
2. It is unpacked, with the files being placed in
their appropriate locations.
3. If necessary, fmtutil.cnf (or updmap.cfg or etc.)
is regenerated, and fmtutil-sys (or updmap-sys) is run.
4. mktexlsr is run
5. The local texlive.tlpdb is updated.
6. $TEXMFSYSVAR/web2c/tlmgr.log is updated.

I am not sure where in the order 5 and 6 are done.


Or `updated' for that matter?

In addition to the above:
1.5. The old version is backed up and its files are
deleted. If the installation generates no error,
the backups may (eventually) be deleted.


Like I say, these questions might seem witless, but at the moment I'm
seriously in the dark due to lack of knowledge and a lack of knowledge
of where to correct my lack of knowledge - aside from asking someone.

You can also generate
it at will with tlmgr generate fmtutil which you might do if you wish
your new fmtutil-local.cnf to be immediately included.

Righto - so `tlmgr generate fmtutil' generates a new fmtutil.cfg file to
be used as the default version by tlmgr?

Is that right?

Yes.


I'm still not sure where it gets its data from to create this file,
though, aside from the additions mentioned above. What /is/ the source
of the information that fmtutil uses to generate a new fmtcnf.cfg file?
(and how does it decide where to put it?)

Some of the data about formats is in the master
texlive.tlpdb, some is in the files contained in
the package.

For example, texlive.tlpdb contains (under the amstex
package) a list of files and the line:
depend bin-amstex
then under the bin-amstex package, there is a list
of files that include programs, but also the line
execute BuildFormat amstex
which indicates that after copying files, a format
needs to be created. One of the files listed is
texmf/fmtutil/format.amstex.cnf
which contains the information on how to build the format
(i.e. the lines to be copied to fmtutil.cnf).


Thanks again,

I hope that, in those cases where guess-work is
involved, I am not so far amiss as to make things
worse.


Dan
To reply by email, change LookInSig to luecking
.

Relevant Pages

  • Re: TeX live format generation question
    ... If the files language-local.dat, language-local.def, fmtutil ... The tlmgr man page you quote from also states: ... These two formats are ... installation any way other than manually so I'm rather ignorant at the ...
    (comp.text.tex)
  • Re: TeX live format generation question
    ... McDonnell) wrote: ... If the files language-local.dat, language-local.def, fmtutil ... supported format is installed or updated). ... it at will with tlmgr generate fmtutil which you might do if you wish ...
    (comp.text.tex)
  • Re: S-L-O-W win2000pro
    ... Most new UPS will pay for damages by ... Format the floppy on a W2000 box, and do not use a quick format. ... Boot computer into W98se using your Boot Disk. ... installation didn't install activex properly. ...
    (microsoft.public.win2000.general)
  • Re: S-L-O-W win2000pro
    ... A UPS is actually a 110v. ... Format the floppy on a W2000 box, and do not use a quick format. ... Boot computer into W98se using your Boot Disk. ... installation didn't install activex properly. ...
    (microsoft.public.win2000.general)
  • Re: OT-Puter prob-long
    ... Dweller in the cellar ... If I do the NTFS drive format thru  Windows Installation, ... If I format it with my ME boot disc, it formats OK and tests good. ... Bios does not see a problem with the memory or HD ...
    (rec.crafts.metalworking)