[Proof of concept] Conversion of tango manual from lyx to rst

# 8 years ago
t-b	Hi, after the really inspiring and enlightening tango meeting, I wanted to use the available momentum and had a go at the lyx documentation conversion. I've created a git repo at [1]. There I've included the current version of /tango-ds/doc/manual/trunk, a script convert.sh and the results of each conversion step. Regarding the conversion from lyx to restructedText this has multiple steps (all are one time): - Generate latex from lyx - Manual latex cleanup (missing references) - scripted cleanup of latex so that pandoc understands it - Convert latex to rst - scripted/manual cleanup of rst (not yet done) The documentation generation later on can then be automated with generating a PDF and a html file from the rst. The result can be viewed in the files tango-from-rst.pdf and tango.html in the repository. There are still a couple of issues which require one time cleanup of the rst, e.g.: - Image code generated by pandoc does not work - Code blocks need heavy cleanup due to the switch from lyxcode to minted - missing index - bibliopgraphy nicification Is that a viable route? Do we want to have the manual in rst? The conversion makes only sense if we throw away the lyx files and switch to rst. Thomas PS: I will not have the time to do everything myself. And also the people currently writing the documentation should approve of rst ;) [1]: https://github.com/t-b/tango-doc

# 8 years ago

Hi,

after the really inspiring and enlightening tango meeting, I wanted to use the available momentum and had a go at the lyx documentation conversion.

I've created a git repo at [1]. There I've included the current version of /tango-ds/doc/manual/trunk, a script convert.sh and the results of each conversion step.

Regarding the conversion from lyx to restructedText this has multiple steps (all are one time):
- Generate latex from lyx
- Manual latex cleanup (missing references)
- scripted cleanup of latex so that pandoc understands it
- Convert latex to rst
- scripted/manual cleanup of rst (not yet done)

The documentation generation later on can then be automated with generating a PDF and a html file from the rst.

The result can be viewed in the files tango-from-rst.pdf and tango.html in the repository.

There are still a couple of issues which require one time cleanup of the rst, e.g.:
- Image code generated by pandoc does not work
- Code blocks need heavy cleanup due to the switch from lyxcode to minted
- missing index
- bibliopgraphy nicification

Is that a viable route?
Do we want to have the manual in rst? The conversion makes only sense if we throw away the lyx files and *switch* to rst.

Thomas

PS: I will not have the time to do everything myself. And also the people currently writing the documentation should approve of rst ;)

[1]: https://github.com/t-b/tango-doc

# 8 years ago
Andy	Dear Thomas, thank you for an excellent talk at the TANGO meeting. I agree the whole meeting was inspiring and it is good to use the +ve energy we generated to work on the hot topics raised at the meeting. Your first go at the documentation is highly appreciated. I cloned the repository and had a look at it. I did not see any rst files - is this normal? I also did not see any rst markup for the source code in the tang-from-rst.pdf file. Is this because it still needs to be done? I too would be in favour of moving to rst. I need to check with the authors to see if they agree. It would also need some support from the community. I therefore ask the obvious question : `Who would be motivated to contribute to converting and improving the documentation in reStructuredText ?` Thanks for your proposal and work in converting to rst! Kind regards Andy Edited 8 years ago

# 8 years ago

Andy

Dear Thomas,

thank you for an excellent talk at the TANGO meeting. I agree the whole meeting was inspiring and it is good to use the +ve energy we generated to work on the hot topics raised at the meeting. Your first go at the documentation is highly appreciated. I cloned the repository and had a look at it. I did not see any rst files - is this normal? I also did not see any rst markup for the source code in the tang-from-rst.pdf file. Is this because it still needs to be done?

I too would be in favour of moving to rst. I need to check with the authors to see if they agree. It would also need some support from the community. I therefore ask the obvious question :

Who would be motivated to contribute to converting and improving the documentation in reStructuredText ?

Thanks for your proposal and work in converting to rst!

Kind regards
Andy

Edited 8 years ago

# 8 years ago
t-b	Andy thank you for an excellent talk at the TANGO meeting. I was not the one giving the sphinx talk. Andy I did not see any rst files - is this normal? My fault, the rst is now added. Github even displays it nicely, see https://github.com/t-b/tango-doc/blob/master/tango.rst. Andy I also did not see any rst markup for the source code in the tang-from-rst.pdf file. Is this because it still needs to be done? Yes, exactly. Currently there is also lots of gibberish like `~, \textdoublequote, …` inside the codeblocks. This a breakage from the conversion which needs cleanup.

# 8 years ago

t-b

Andy
thank you for an excellent talk at the TANGO meeting.

I was not the one giving the sphinx talk.

Andy
I did not see any rst files - is this normal?

My fault, the rst is now added. Github even displays it nicely, see https://github.com/t-b/tango-doc/blob/master/tango.rst.

Andy
I also did not see any rst markup for the source code in the tang-from-rst.pdf file. Is this because it still needs to be done?

Yes, exactly. Currently there is also lots of gibberish like

~, \textdoublequote, …

inside the codeblocks. This a breakage from the conversion which needs cleanup.

# 8 years ago
Andy	Hi Thomas, sorry I mixed you up with Georg - my fault! Thanks for the rst files. I presume the gibberish has to be removed manually. Andy

# 8 years ago
t-b	Andy I presume the gibberish has to be removed manually. The least manual option would be to replace `\begin_layout LyX-Code` with `\begin_layout Verbatim` in the lyx files which should insert the plain code examples in the latex file. Thomas

# 8 years ago
t-b	I've enhanced the code snippts conversion. No more gibberish and for most parts it already looks acceptable. All cleanup is scripted, the manual fine tuning can be done later.

# 8 years ago
t-b	Here is a nice article about why the linux kernel choose sphinx as documentation processor.

# 8 years ago
Andy	Hi Thomas, thanks for the link - very interesting indeed. I like this comment : Sphinx is easier to work with, simpler to extend, better supported. Well done to Georg and the Sphinx team! Most of all I like the result : http://static.lwn.net/kerneldoc/index.html I had a question from the author of the Tango book that you converted : is there a WYSIWIG editor for rst? I have found this web based editor http://rst.ninjs.org/ but I am not sure this can handle a big document. Do you know of any WYSIWIG editors or is this opposite to the way rst files works ? Andy

# 8 years ago

Andy

Hi Thomas,

thanks for the link - very interesting indeed. I like this comment :

Sphinx is easier to work with, simpler to extend, better supported.

Well done to Georg and the Sphinx team!

Most of all I like the result : http://static.lwn.net/kerneldoc/index.html

I had a question from the author of the Tango book that you converted : is there a WYSIWIG editor for rst? I have found this web based editor http://rst.ninjs.org/ but I am not sure this can handle a big document.

Do you know of any WYSIWIG editors or is this opposite to the way rst files works ?

Andy

# 8 years ago
t-b	Andy I had a question from the author of the Tango book that you converted : is there a WYSIWIG editor for rst? I have found this web based editor http://rst.ninjs.org/ but I am not sure this can handle a big document. I'm using vim and just syntax highlightning. I've found https://github.com/retext-project/retext on the web, this looks a bit more WYSIWYG. The tango handbook tango.rst is due to the conversion just one file at the moment. I would propose to split it into 5-10 small files with one main include. Andy Do you know of any WYSIWIG editors or is this opposite to the way rst files works ? WYSIWYG editors are not that prominent in the rst/markdown segment. The reason is that the markup is supposed to be very easy on the reader compared to e.g. LaTeX or docbook. If you are an emacs user, you might want to have a look at http://docutils.sourceforge.net/docs/user/emacs.html.

# 8 years ago

t-b

Andy
I had a question from the author of the Tango book that you converted : is there a WYSIWIG editor for rst? I have found this web based editor http://rst.ninjs.org/ but I am not sure this can handle a big document.

I'm using vim and just syntax highlightning. I've found https://github.com/retext-project/retext on the web, this looks a bit more WYSIWYG. The tango handbook tango.rst is due to the conversion just one file at the moment. I would propose to split it into 5-10 small files with one main include.

Andy
Do you know of any WYSIWIG editors or is this opposite to the way rst files works ?

WYSIWYG editors are not that prominent in the rst/markdown segment. The reason is that the markup is supposed to be very easy on the reader compared to e.g. LaTeX or docbook.

If you are an emacs user, you might want to have a look at http://docutils.sourceforge.net/docs/user/emacs.html.

# 7 years ago
pgoryl	Hi Thomas, You have done a good job. I would like to use tango.rst you have generated as a base for preparation of an official Sphinx manual to be placed in https://github.com/tango-controls/tango-doc. So, if you agree I will copy it there. Then I will follow with cleaning and adding missing images and so. Best regards, Piotr