[WIP] bpo-31589: Build PDF using xelatex for better UTF8 support. by JulienPalard · Pull Request #3940 · python/cpython (original) (raw)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation16 Commits1 Checks0 Files changed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
If someone want to test it locally, after checking-out my branch and installing texlive-xetex, texlive-fonts-recommended, texlive-lang-all, creating a venv with an up-to-date sphinx-doc and blurb, one can do:
git clone --depth 1 --no-single-branch https://github.com/python/python-docs-fr.git
cd python-docs-fr/
git checkout 3.6
mkdir -p /tmp/$USER/locales/fr
ln -nfs <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mi>r</mi><mi>e</mi><mi>a</mi><mi>d</mi><mi>l</mi><mi>i</mi><mi>n</mi><mi>k</mi><mo>−</mo><mi>f</mi><mi mathvariant="normal">.</mi><mo stretchy="false">)</mo><mi mathvariant="normal">/</mi><mi>t</mi><mi>m</mi><mi>p</mi><mi mathvariant="normal">/</mi></mrow><annotation encoding="application/x-tex">(readlink -f .) /tmp/</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal">re</span><span class="mord mathnormal">a</span><span class="mord mathnormal">d</span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mord mathnormal" style="margin-right:0.03148em;">ink</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.10764em;">f</span><span class="mord">.</span><span class="mclose">)</span><span class="mord">/</span><span class="mord mathnormal">t</span><span class="mord mathnormal">m</span><span class="mord mathnormal">p</span><span class="mord">/</span></span></span></span>USER/locales/fr/LC_MESSAGES
make -C your/cpython/clone/Doc SPHINXOPTS="-D locale_dirs=/tmp/$USER/locales/ -D language=fr -D gettext_compact=0" autobuild-dev
@methane: Are you interested to check if it works with the Japanese translation?
I don't know about PDF and latex much.
But I'm asking question about recommended way to generate Japanese PDF in Japanese
Sphinx community. (Most active maintainers of Sphinx are Japanese.)
Looks like build is failing for japanese:
------------
Running 'xelatex -recorder "howto-curses.tex"'
------------
This is XeTeX, Version 3.14159265-2.6-0.99998 (TeX Live 2017/Debian) (preloaded format=xelatex)
restricted \write18 enabled.
entering extended mode
(./howto-curses.tex
LaTeX2e <2017-04-15>
Babel <3.11> and hyphenation patterns for 84 language(s) loaded.
(./sphinxhowto.cls
Document Class: sphinxhowto 2017/03/26 v1.6 Document class (Sphinx HOWTO)
(/usr/share/texlive/texmf-dist/tex/platex/base/jreport.cls
! LaTeX Error: This file needs format `pLaTeX2e'
but this is `LaTeX2e'.
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.16 \NeedsTeXFormat{pLaTeX2e}
Generating Japanese PDF via Latex is very hard way.
Please ignore Japanese for now.
Usually, platex is used to build Japanese TeX documents. So Sphinx depends on it by default.
In other words, Sphinx does not support xelatex to build Japanese document.
(If you're TeX specialist, you might be able to configure your Sphinx project to do that!)
IMO, it would be better to use xelatex only if the document is not Japanese. How about this?
if language != 'ja':
latex_engine = 'xelatex'
I believe this will work well.
@tk0miya Thanks for the recommendation, I'll try this. I'm no latex expert though.
Tests for today:
- Using pdflatex, with the current set of inputenc and fontenc, english build is OK, even the Є symbol in the changelog is rendered correcly.
- Using xelatex and an empty latex_elements, english still builds OK
- Using pdflatex, with the current set of inputenc and fontenc, french fails (latexmk fail as documented in https://bugs.python.org/issue31589#msg303502)
- Using xelatex and an empty latex_elements, french produces PDFs
- Usinx xelatex to build japanese gives:
LaTeX Error: This file needs format 'pLaTeX2e' but this is 'LaTeX2e'. - Using platex, japanese fails to build:
Japanese errors using platex:
! Text line contains an invalid character.
l.66796 ...de{subprocess.Popen()}}}} の stream ^^P
引数など、ファイル記述子つ...
Looks like the problem is from the Japanese translation:
#: ../../library/socket.rst:1072
msgid ""
"On Windows, the file-like object created by :meth:`makefile` cannot be used "
"where a file object with a file descriptor is expected, such as the stream "
"arguments of :meth:`subprocess.Popen`."
msgstr ""
"Windows では、 :meth:`subprocess.Popen` の stream ^P引数など、ファイル記述子つき file "
"オブジェクトが期待されている場所では、 :meth:`makefile` によって作成される file-like オブジェクトは使用できません。"
I'm opening an issue on their side.
Fixing the \x10 found in japanese translation yields to another error:
$ xelatex -recorder howto-curses.tex
This is XeTeX, Version 3.14159265-2.6-0.99998 (TeX Live 2017/Debian) (preloaded format=xelatex)
restricted \write18 enabled.
entering extended mode
(./howto-curses.tex
LaTeX2e <2017-04-15>
Babel <3.13> and hyphenation patterns for 84 language(s) loaded.
(./sphinxhowto.cls
Document Class: sphinxhowto 2017/03/26 v1.6 Document class (Sphinx HOWTO)
(/usr/share/texlive/texmf-dist/tex/platex/base/jreport.cls
! LaTeX Error: This file needs format `pLaTeX2e'
but this is `LaTeX2e'.
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.16 \NeedsTeXFormat{pLaTeX2e}
? H
The current input file will not be processed further,
because it was written for some other flavor of TeX.
You're in trouble here. Try typing <return> to proceed.
If that doesn't work, type X <return> to quit.
? x
No pages of output.
Transcript written on howto-curses.log.
I still don't get this one, as I explicitly asked sphinx to use xelatex.
But building with platex works!
So xelatex works for english and french and platex works for japanese, as told by @tk0miya thanks!
Still have to find a clean way to jump between those two implementations, as the 'language' is not yet initialized when parsing conf.py.
Thanks @JulienPalard for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒⛏🤖
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request
Also addresses doc build failures documented in bpo-32200. (cherry picked from commit 7324b5c)
ned-deily pushed a commit that referenced this pull request
Thanks @JulienPalard for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 2.7.
🐍🍒⛏🤖
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request
Also addresses doc build failures documented in bpo-32200. (cherry picked from commit 7324b5c)
Co-authored-by: Julien Palard julien@palard.fr
miss-islington added a commit that referenced this pull request
Also addresses doc build failures documented in bpo-32200. (cherry picked from commit 7324b5c)
Co-authored-by: Julien Palard julien@palard.fr