{"author":"Anthon van der Neut","author_email":"a.van.der.neut@ruamel.eu","bugtrack_url":null,"classifiers":["Development Status :: 4 - Beta","Intended Audience :: Developers","License :: OSI Approved :: MIT License","Operating System :: OS Independent","Programming Language :: Python"],"description":"\n# ryd\n\n[![image](https://sourceforge.net/p/ryd/code/ci/default/tree/_doc/_static/license.svg?format=raw)](https://opensource.org/licenses/MIT)\n[![image](https://sourceforge.net/p/ryd/code/ci/default/tree/_doc/_static/pypi.svg?format=raw)](https://pypi.org/project/ryd/)\n[![image](https://sourceforge.net/p/oitnb/code/ci/default/tree/_doc/_static/oitnb.svg?format=raw)](https://pypi.org/project/oitnb/)\n[![image](https://sourceforge.net/p/ryd/code/ci/default/tree/_doc/_static/ryd.svg?format=raw)](https://pypi.org/project/ryd/)\n\n`ryd` ( /rɑɪt/, pronounced like the verb \\\"write\\\" ) is a preprocessor\nfor text based documents, that builds upon the multi-document\ncapabilities of YAML files/streams.\n\nThe use of multiple-documents in `ryd` allows for clear separation\nbetween document text and any programs referenced in those text. Thereby\nmaking it possible to run (c.q. compile) the program parts of a\ndocument, e.g. to check whether they are syntactically correct. It can\nalso capture the *actual* output of those programs to be included in the\ndocument. It is also possible to recognise different documents and thus\nrun different formatters then recombining the documents.\n\nThis allows for easier maintenance of (correct) program sources, in\ndocument source texts like reStructuredText, LaTeX, Markdown, etc.\n\nThe first of the documents in a `ryd` file has, at the root-level, a\nmapping. This mapping is the **ryd configuration metadata** for the rest\nof the stream of documents in the file. The metadata is used to define\n`ryd` document version used, what the basic **text style** is (currently\n`rst` for RestructuredText, `so` for StackOverflow-markdown), if any\npostprocessing (PDF, HTML) needs to be done, and other configuration\ninformation. This first document doesn\\'t normally have any directives.\nThat the document is YAML 1.2 is implicit, so no `%YAML 1.2` directive\nis needed and without directive and no preceding document, you do not\nhave a directives-end marker line (`---`) at the top.\n\nThe documents following the first document are normally block style\nliteral scalars with an optional tag. The tag influences how the scalar\nstring is processed within the selected output text style.\n\n## Example\n\n version: 0.2\n text: rst\n fix_inline_single_backquotes: true\n --- |\n Example Python program\n ++++++++++++++++++++++\n\n This is an example of a python program:\n --- !python |\n n = 7\n print(n**2 - n)\n --- !stdout |\n The answer is:\n\nthis will generate (using: `ryd convert test.ryd`) the following\n`test.rst`:\n\n Example Python program\n ++++++++++++++++++++++\n\n This is an example of a Python program:\n\n .. code:: python\n\n n = 7\n print(n**2 - n)\n\n The answer is:\n\n .. code::\n\n 42\n\n## Postprocessing\n\nThe output can be converted to PDF using `rst2pdf` or HTML using\n`rst2html` with images embedded using `webpage2html`. Invocation of\nthese programs can be specified in the metatada (e.g. `post: pdf`).\n\nThe (image embedded) HTML output has the indent of indented lines of\nincluded code fragments inserted as actual spaces. So you copy and paste\nprogram code (or YAML) from HTML without a problem, something not\npossible with PDF files generated by `rst2pdf`.\n\n`ryd` generates its text output stand-alone, but the programs actually\nused for postprocessing have to be installed seperately and available in\nyour PATH.\n\n(There is currently no postprocessing for Markdown, as pandoc is not\nworking on Arch (Mark 2022))\n\n## Config\n\nYou can create a file `~/.config/ryd/ryd.yaml` with defaults for the\ncommandline options. To always embed images when converting to HTML and\nalways run the (global) verbose option:\n\n> global:\n>\n> : verbose: true\n>\n> convert:\n>\n> : embed: true\n\n### Command-line options\n\nThe command-line of `ryd` consists of multiple components:\n\n ryd [--global-option] command [--options] [arguments]\n\nAlthough not indicated most global options can occur after the command\nas well.\n\n#### commands\n\n convert generate output as per first YAML document\n roundtrip roundtrip .ryd file, updating sections\n from-rst (fromrst) convert .rst to .ryd\n\nYou\\'ll most often use `convert` it takes one or more filenames as\nargument and generates output as specified in the ryd configuration\ndata. Some options allow you to override settings there (e.g. `--pdf`\nand `-no-pdf`)\n\nThe command `from-rst` converts a `.rst` file into `.ryd` doing some\nsection underline checking and adding the ryd configuration data\ndocument.\n\nThe `roundtrip` command updates sections by running in place\ncode-formatting on the `!python` (using `oitnb`) and `!zig` sections of\nthe `.ryd` file. ( There is currently no config option to use `blue` or\n`black`, but you can copy `ryd/_tag/python.tag` to your\n`.config/ryd/tag` directory and make the minor adjustment to use those,\nsee the section on making our own tag).\n\nDoing `ryd command --help` might indicate extra options that have not\nyet made it into the documentation and/or that are incompletely\nimplemented.\n\n# Documents and document tags\n\nEach YAML document has to be separated from other documents in the\nstream by at least the end of directive marker `---`. Apart from the\nfirst document, most documents contain a single, multi-line,\nnon-indented, scalar. The end of directives marker is therefor followed\nby the pipe (`|`) symbol, which is the YAML indication for a multi-line\nliteral scalar.\n\nThat scalar can be \\\"typed\\\" in the normal way of YAML by inserting a\ntag before the `|`. E.g. a document that is a type of Python program has\na tag `!python` and thus starts with:\n\n --- !python |\n\nWhat a document tag exactly does, depends on the tag, but, potentially,\nalso, on the output file format selected, on previously processed tagged\ndocuments, other `.ryd` files processed previously and the environment.\n\nIn addition to the basic tag (like `!python`), a tag can have\nsubfunctions such as `!python-pre`. If an unknown subfunction is\nspecified you\\'ll get a runtime error.\n\nThe following are short descriptions for all tags, independent of the selected\noutput format:\n\n`!changelog`
input is a mapping keys are (version, date) tuples, or the word NEXT\n\n`!code`
Include program in text. Do not mark as executable, doesn't influence ``!stdout``.\n\n`!comment`
The whole document will be discarded, i.e. not included in the output.\n\n`!inc`
Include the content of the listed files (indented), without other processing, into the output. Preceed with ``::`` if necessary\n\n`!inc-raw`
Include the content of the listed files (indented), without other processing, into the output. Preceed with ``::`` if necessary\n\n`!lastcompile`
Include output from last compilation as code.\n\n`!nim`
Include Nim program in text. Prefix and mark as executable.\n\n`!nim-pre`
Include Nim program in text. Prefix and mark as executable.\n\n`!python`
Include Python program in text. Prefix and mark as executable.\n\n`!python-hidden`
Include Python program in text. Prefix and mark as executable.\n\n`!python-hiddenstdoutraw`
Include Python program in text. Prefix and mark as executable.\n\n`!python-pre`
Include Python program in text. Prefix and mark as executable.\n\n`!stdout`
Include output from last executable document (e.g. ``!python``) as code.\n\n`!stdout-raw`
Include output from last executable document (e.g. ``!python``) as code.\n\n`!table`
create a table, for now headerless\n\n`!toc`
insert an index\n\n`!yamlout`
Include output from last executable document (e.g. ``!python``) as code tagged as YAML document.\n\n`!zig`
Include Zig program in text. Prefix and execute setting !stdout.\n\n`!zig-pre`
Include Zig program in text. Prefix and execute setting !stdout.\n\n`!zsh`
run each line in zsh, interspacing the lines with the output\n\n# MD\n\nThe output to `.md` expects non-code YAML documents to be valid\nMarkdown.\n\nBecause of the special meaning of `---` (and `...`) at the beginning of\na line, followed by newline or space, these should not be used. Either\nuse a longer sequence ( e.g. for divider lines use `----` ), or\nindent the whole document:\n\n```\n --- |\n line followed by divider\n ---\n another line\n```\n\n## mkdocs\n\nIf you have a file `_doc/README.ryd`, you can use `ryd serve` to\nconvert and serve this with mkdocs. The contents of the necessary\nmkdocs configuration file will be written from the mkdocs entry in the metadata\nof the `README.ryd` file (so you don't need to put more cruft in your packages root\ndirectory):\n\n```\n version: 0.2\n text: md\n pdf: false\n mkdocs:\n # if site_name key not provided set to current directory name\n site_name: your_project_name\n docs_dir: _doc\n site_author: Anthon van der Neut\n\n theme:\n name: readthedocs\n\n exclude_docs: |\n *.ryd\n *.rst\n\n markdown_extensions:\n - toc:\n permalink: true\n```\n\n \n# RST\n\nThe output to `.rst` expects non-code YAML documents to be valid\nreStructuredText. Any non-tagged documents, i.e. those starting with:\n\n --- |\n\nare assumed to be text input, in the format specified in the ryd\nconfiguration data.\n\n## Section underlining\n\nBecause of the special meaning of `---` (and `...`) at the beginning of\na line, followed by newline or space, the section under/over-line\ncharacters used in `.ryd` files that are source for `.rst` should not\nuse `-` or `.` sequences if a any of those section names consist of\nthree letters (e.g. a section named API or RST). It is recommended to\nuse the following scheme:\n\n Sections, subsections, etc. in .ryd files\n # with over-line, for parts\n * with over-line, for chapters\n =, for sections\n +, for subsections\n ^, for sub-subsections\n \", for paragraphs\n\n## Single backquotes\n\nThe `fix_inline_single_backquotes: true` tells `ryd` to indicate lines\nthat have single backquotes, that need fixing (by replacing them with\ndouble backquotes):\n\n README.ryd\n 47: this will generate (`ryd convert test.ryd`) the following\n --^\n --^\n\n(If you are used to other inline code markup editing e.g. on Stack\nOverflow, that uses single backquotes, you\\'ll come to appreciate this.)\n\n## Python\n\nPython code is indicated by:\n\n --- !python |\n\nThe document is inserted into the `.rst` preceded by `.. code:: python`\nand each line with a two space indent.\n\nIf your program relies on specific packages, those packages, need to be\navailable in the environment in which `ryd` is started (which can e.g.\nbe a specifically set up `virtualenv`)\n\nIt is possible to have \\\"partial programs\\\" by preceding a python\ndocument with e.g.:\n\n --- !python-pre |\n from __future__ import print_function\n import sys\n import ruamel.yaml\n from ruamel.std.pathlib import Path, pushd, popd, PathLibConversionHelper\n pl = PathLibConversionHelper()\n\nSuch a block is pre-pended to all following `--- !python |` documents\n(until superseded by another `--- !python-pre |` block)\n\n## Captured output\n\nThe output from the last program that was run (`--- !python |`) is\nstored and can be post-pended to a reStructuredText document by tagging\nit with `!stdout` (i.e. `--- !stdout |`)\n\n## non-running code\n\nA document tagged `!code` will be represented as one tagged `!python`,\nbut the code will not be run (and hence the output used for `!stdout`\nnot changed).\n\n## Zig\n\nZig code is indicated by:\n\n --- !zig |\n\nThe document is inserted as with Python, there can be a `!zig-pre`\ndocument, and output is captured and displayed with `--- !stdout |`):\n\n```zig\n// const std = @import(\"std\"); is defined in zig-pre\n\npub fn main() !void {\n const stdout = std.io.getStdOut().writer();\n try stdout.print(\"Hello, {s}!\\n\", .{\"world\"});\n}\n```\n\nwhich outputs:\n\n```lang-none\nHello, world!\n```\n\nThe compilation is done with option `build-exe`.\n\n\ncompiler output\n^^^^^^^^^^^^^^^\n\nIf you are interested in the textual output of the compiler you can use\n``--- !lastcompile |``:\n\n# Nim\n\nNim code is indicated by:\n\n --- !nim |\n\nThe document is inserted as with Python, there can be a `!nim-pre`\ndocument, and output is captured and displayed with `--- !stdout |`):\n\n```lang-none\nlet a = 123\nlet x = 0b0010_1010\necho(fmt\"The answer to the question: {x}\")\n```\n\nwhich outputs:\n\n The answer to the question: 42\n\nThe compilation is done with options\n`--verbosity:0 --hint[Processing]:off` .\n\n# compiler output\n\nIf you are interested in the textual output of the compiler you can use\n`--- !lastcompile |`:\n\n /tmp/ryd-of-anthon/ryd-1169/tmp_02.nim(4, 5) Hint: 'a' is declared but not used [XDeclaredButNotUsed]\n\n# Comments\n\nBlock style literal scalars do not allow YAML comments. To insert\ncomments in a text, either use the format acceptable by the output, e.g.\nwhen generating `.rst` use:\n\n ..\n this will show up in the resulting .rst file, but will\n not render\n\nAlternatively you can create a comment YAML document (`--- !comment |`),\nfor which the text will not be represented in the output file format\n**at all**.\n\nIf you already have a tagged document, e.g. `--- !python |` document,\nyou can make it into a comment by inserting `comment-`:\n\n --- !comment-python |\n\nThis has been implemented by **not** reporting an error when an unkown\nsubfunction on `!comment` is invoked.\n\n# Tasklists\n\nIf you set `tasklist: true` in the metadata, then lines that *start*\nwith one of the following tasklist patterns, are transformed to their\nunicode counterpart:\n\n [ ] not done yet\n [v] done\n [x] no longer going to do\n\nresulting in\n\n☐ not done yet\n\n☑ done\n\n☒ no longer going to do\n\n(it would be nice to know if there is a way to create a real list with\nuser specified bullet items)\n\n# Making your own tag\n\nBefore trying to load a tag `!yourtag` from its know files, `ryd` tries\nto load them from `~/.config/ryd/tag/`. This mechanism can be used to\nimplement your improvements over existing tags or extendeding the tags\nwith your own.\n\nLet\\'s assume you want to explain the use of `mypy` in your `ryd`\ndocument, including output of a `mypy` run on some source. First create\na file `~/.config/ryd/tag/mypy.tag` with the following content:\n\n # coding: 'utf-8'\n\n from __future__ import annotations\n\n import os\n import subprocess\n from typing import Any, TYPE_CHECKING\n from ryd._tag._handler import ProgramHandler\n\n if TYPE_CHECKING:\n from ryd._convertor._base import ConvertorBase\n else:\n ConvertorBase = Any\n\n\n class Mypy(ProgramHandler): # class name is capitalization of the stem of the filename\n def __init__(self, convertor: ConvertorBase) -> None:\n super().__init__(convertor)\n self._pre = ''\n\n def pre(self, d: Any) -> None: # like !python-pre you can have !mypy-pre\n self._pre = str(d)\n\n def __call__(self, d: Any) -> None:\n \"\"\"\n Include Python program in text. Prefix, save and run mypy, setting !stdout.\n \"\"\"\n s = str(d)\n # depending on the util, you may not need to do a chdir to the tempdir\n old_dir = os.getcwd()\n self.c.temp_dir.chdir()\n path = self.c.temp_file_path('.py')\n path.write_text(self._pre + s)\n self.c.last_output = subprocess.run([\n 'mypy',\n '--strict', '--follow-imports', 'silent', '--implicit-reexport',\n str(path),\n ], stderr=subprocess.STDOUT, stdout=subprocess.PIPE, encoding='utf-8').stdout\n os.chdir(old_dir)\n self.c.add_code(s, 'python') # format the code as python\n\nand you include in your `ryd` document:\n\n --- !mypy |\n def main(arg1, arg2):\n return arg1\n\n --- !stdout |\n which gives:\n\n --- |\n\n from the mypy output you can see ....\n\nYour `.rst` will then contain the Python source and the mypy output:\n\n .. code:: python\n\n def main(arg1, arg2):\n return arg1\n\n which gives:\n\n .. code::\n\n tmp_03.py:2: error: Function is missing a type annotation\n Found 1 error in 1 file (checked 1 source file)\n\n from the mypy output you can see ....\n\n# History\n\n`ryd` grew out of a in-house solution where sections of reStructuredText\nfiles were updated, in-place, by running Python programs specified in\nseparate files. Also allowing the inclusion of the (error) output.\n\nAn example of this can be seen in\n[this](https://bitbucket.org/ruamel/yaml/raw/0be7d3cb8449b15d9ac9b097322f09e52b92f868/_doc/example.rst)\nold version of the `example.rst` file of the `ruamel.yaml` package:\n\n Basic round trip of parsing YAML to Python objects, modifying\n and generating YAML::\n\n import sys\n from ruamel.yaml import YAML\n\n inp = \"\"\"\\\n # example\n name:\n # details\n family: Smith # very common\n given: Alice # one of the siblings\n \"\"\"\n\n yaml = YAML()\n code = yaml.load(inp)\n code['name']['given'] = 'Bob'\n\n yaml.dump(code, sys.stdout)\n\n .. example code small.py\n\n Resulting in ::\n\n # example\n name:\n # details\n family: Smith # very common\n given: Bob # one of the siblings\n\n\n .. example output small.py\n\nThe program was inserted before the `.. example code` line and its\noutput before `.. example output`, replacing all the text starting after\nthe previous `::`\n\nThe `small.py` referenced a separate file for this piece of code. This\nresulted in multiple source files that were associated with a single\n`.rst` file. There was no mechanism to have partial programs that could\nbe tested by execution, which precluded getting output from such program\nas well.\n\nAlthough the code could have been edited in place, and used to get the\noutput, this would force one to use the extra indentation required for\nlines following ReST\\'s `::`.\n\nOnce this system came under review, the solution with a structured YAML\nheader, as used with various file formats, combined with multiple\ndocument consisting of (tagged) top level, non-indented, block style\nliteral scalars, was chosen instead.\n\nIn early 2022 an update of the 0.1 format was implemented to make tags\nand convertors into seperate files, thereby making them more easily\nupgradable and extensible.\n","description_content_type":"text/markdown; charset=UTF-8; variant=CommonMark","docs_url":null,"download_url":"","downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":"","keywords":"restructuredtext markdown markup preprocessing","license":"MIT","license_expression":null,"license_files":null,"maintainer":"","maintainer_email":"","name":"ryd","package_url":"https://pypi.org/project/ryd/","platform":null,"project_url":"https://pypi.org/project/ryd/","project_urls":{"Home":"https://sourceforge.net/p/ryd/","Source":"https://sourceforge.net/p/ryd/code/ci/default/tree/","Tracker":"https://sourceforge.net/p/ryd/tickets/"},"provides_extra":null,"release_url":"https://pypi.org/project/ryd/0.9.2/","requires_dist":["ruamel.std.pathlib","ruamel.yaml","ruamel.yaml.split"],"requires_python":">=3","summary":"Ruamel Yaml Doc preprocessor (pronounced: /rɑɪt/, like the verb \"write\")","version":"0.9.2","yanked":false,"yanked_reason":null}

Dependency	Optional
ruamel.std.pathlib	false
ruamel.yaml	false
ruamel.yaml.split	false

Package search

ryd

project_urls

Dependencies

All package metadata

Releases