{"author":null,"author_email":"\"Hernan E. Grecco\" ","bugtrack_url":null,"classifiers":["Development Status :: 4 - Beta","Intended Audience :: Developers","Intended Audience :: Science/Research","License :: OSI Approved :: BSD License","Operating System :: MacOS :: MacOS X","Operating System :: Microsoft :: Windows","Operating System :: POSIX","Programming Language :: Python","Programming Language :: Python :: 3.10","Programming Language :: Python :: 3.11","Programming Language :: Python :: 3.12","Programming Language :: Python :: 3.9","Topic :: Software Development :: Libraries","Topic :: System :: Filesystems","Topic :: Utilities"],"description":".. image:: https://img.shields.io/pypi/v/flexparser.svg\n :target: https://pypi.python.org/pypi/flexparser\n :alt: Latest Version\n\n.. image:: https://img.shields.io/pypi/l/flexparser.svg\n :target: https://pypi.python.org/pypi/flexparser\n :alt: License\n\n.. image:: https://img.shields.io/pypi/pyversions/flexparser.svg\n :target: https://pypi.python.org/pypi/flexparser\n :alt: Python Versions\n\n.. image:: https://github.com/hgrecco/flexparser/workflows/CI/badge.svg\n :target: https://github.com/hgrecco/flexparser/actions?query=workflow%3ACI\n :alt: CI\n\n.. image:: https://github.com/hgrecco/flexparser/workflows/Lint/badge.svg\n :target: https://github.com/hgrecco/flexparser/actions?query=workflow%3ALint\n :alt: LINTER\n\n.. image:: https://coveralls.io/repos/github/hgrecco/flexparser/badge.svg?branch=main\n :target: https://coveralls.io/github/hgrecco/flexparser?branch=main\n :alt: Coverage\n\n\nflexparser\n==========\n\nWhy write another parser? I have asked myself the same question while\nworking on this project. It is clear that there are excellent parsers out\nthere but I wanted to experiment with another way of writing them.\n\nThe idea is quite simple. You write a class for every type of content\n(called here ``ParsedStatement``) you need to parse. Each class should\nhave a ``from_string`` constructor. We used extensively the ``typing``\nmodule to make the output structure easy to use and less error prone.\n\nFor example:\n\n.. code-block:: python\n\n from dataclasses import dataclass\n\n import flexparser as fp\n\n @dataclass(frozen=True)\n class Assigment(fp.ParsedStatement):\n \"\"\"Parses the following `this <- other`\n \"\"\"\n\n lhs: str\n rhs: str\n\n @classmethod\n def from_string(cls, s):\n lhs, rhs = s.split(\"<-\")\n return cls(lhs.strip(), rhs.strip())\n\n(using a frozen dataclass is not necessary but it convenient. Being a\ndataclass you get the init, str, repr, etc for free. Being frozen, sort\nof immutable, makes them easier to reason around)\n\nIn certain cases you might want to signal the parser\nthat his class is not appropriate to parse the statement.\n\n.. code-block:: python\n\n @dataclass(frozen=True)\n class Assigment(fp.ParsedStatement):\n \"\"\"Parses the following `this <- other`\n \"\"\"\n\n lhs: str\n rhs: str\n\n @classmethod\n def from_string(cls, s):\n if \"<-\" not in s:\n # This means: I do not know how to parse it\n # try with another ParsedStatement class.\n return None\n lhs, rhs = s.split(\"<-\")\n return cls(lhs.strip(), rhs.strip())\n\n\nYou might also want to indicate that this is the right ``ParsedStatement``\nbut something is not right:\n\n.. code-block:: python\n\n @dataclass(frozen=True)\n class InvalidIdentifier(fp.ParsingError):\n value: str\n\n\n @dataclass(frozen=True)\n class Assigment(fp.ParsedStatement):\n \"\"\"Parses the following `this <- other`\n \"\"\"\n\n lhs: str\n rhs: str\n\n @classmethod\n def from_string(cls, s):\n if \"<-\" not in s:\n # This means: I do not know how to parse it\n # try with another ParsedStatement class.\n return None\n lhs, rhs = (p.strip() for p in s.split(\"<-\"))\n\n if not str.isidentifier(lhs):\n return InvalidIdentifier(lhs)\n\n return cls(lhs, rhs)\n\n\nPut this into ``source.txt``\n\n.. code-block:: text\n\n one <- other\n 2two <- new\n three <- newvalue\n one == three\n\nand then run the following code:\n\n.. code-block:: python\n\n parsed = fp.parse(\"source.txt\", Assigment)\n for el in parsed.iter_statements():\n print(repr(el))\n\nwill produce the following output:\n\n.. code-block:: text\n\n BOF(start_line=0, start_col=0, end_line=0, end_col=0, raw=None, content_hash=Hash(algorithm_name='blake2b', hexdigest='37bc23cde7cad3ece96b7abf64906c84decc116de1e0486679eb6ca696f233a403f756e2e431063c82abed4f0e342294c2fe71af69111faea3765b78cb90c03f'), path=PosixPath('/Users/grecco/Documents/code/flexparser/examples/in_readme/source1.txt'), mtime=1658550284.9419456)\n Assigment(start_line=1, start_col=0, end_line=1, end_col=12, raw='one <- other', lhs='one', rhs='other')\n InvalidIdentifier(start_line=2, start_col=0, end_line=2, end_col=11, raw='2two <- new', value='2two')\n Assigment(start_line=3, start_col=0, end_line=3, end_col=17, raw='three <- newvalue', lhs='three', rhs='newvalue')\n UnknownStatement(start_line=4, start_col=0, end_line=4, end_col=12, raw='one == three')\n EOS(start_line=5, start_col=0, end_line=5, end_col=0, raw=None)\n\n\nThe result is a collection of ``ParsedStatement`` or ``ParsingError`` (flanked by\n``BOF`` and ``EOS`` indicating beginning of file and ending of stream respectively\nAlternative, it can beginning with ``BOR`` with means beginning of resource and it\nis used when parsing a Python Resource provided with a package).\n\nNotice that there are two correctly parsed statements (``Assigment``), one\nerror found (``InvalidIdentifier``) and one unknown (``UnknownStatement``).\n\nCool, right? Just writing a ``from_string`` method that outputs a datastructure\nproduces a usable structure of parsed objects.\n\nNow what? Let's say we want to support equality comparison. Simply do:\n\n.. code-block:: python\n\n @dataclass(frozen=True)\n class EqualityComparison(fp.ParsedStatement):\n \"\"\"Parses the following `this == other`\n \"\"\"\n\n lhs: str\n rhs: str\n\n @classmethod\n def from_string(cls, s):\n if \"==\" not in s:\n return None\n lhs, rhs = (p.strip() for p in s.split(\"==\"))\n\n return cls(lhs, rhs)\n\n parsed = fp.parse(\"source.txt\", (Assigment, Equality))\n for el in parsed.iter_statements():\n print(repr(el))\n\nand run it again:\n\n.. code-block:: text\n\n BOF(start_line=0, start_col=0, end_line=0, end_col=0, raw=None, content_hash=Hash(algorithm_name='blake2b', hexdigest='37bc23cde7cad3ece96b7abf64906c84decc116de1e0486679eb6ca696f233a403f756e2e431063c82abed4f0e342294c2fe71af69111faea3765b78cb90c03f'), path=PosixPath('/Users/grecco/Documents/code/flexparser/examples/in_readme/source1.txt'), mtime=1658550284.9419456)\n Assigment(start_line=1, start_col=0, end_line=1, end_col=12, raw='one <- other', lhs='one', rhs='other')\n InvalidIdentifier(start_line=2, start_col=0, end_line=2, end_col=11, raw='2two <- new', value='2two')\n Assigment(start_line=3, start_col=0, end_line=3, end_col=17, raw='three <- newvalue', lhs='three', rhs='newvalue')\n EqualityComparison(start_line=4, start_col=0, end_line=4, end_col=12, raw='one == three', lhs='one', rhs='three')\n EOS(start_line=5, start_col=0, end_line=5, end_col=0, raw=None)\n\n\nYou need to group certain statements together: welcome to ``Block``\nThis construct allows you to group\n\n.. code-block:: python\n\n class Begin(fp.ParsedStatement):\n\n @classmethod\n def from_string(cls, s):\n if s == \"begin\":\n return cls()\n\n return None\n\n class End(fp.ParsedStatement):\n\n @classmethod\n def from_string(cls, s):\n if s == \"end\":\n return cls()\n\n return None\n\n class ParserConfig:\n pass\n\n class AssigmentBlock(fp.Block[Begin, Assigment, End, ParserConfig]):\n pass\n\n parsed = fp.parse(\"source.txt\", (AssigmentBlock, Equality))\n\n\nRun the code:\n\n.. code-block:: text\n\n BOF(start_line=0, start_col=0, end_line=0, end_col=0, raw=None, content_hash=Hash(algorithm_name='blake2b', hexdigest='37bc23cde7cad3ece96b7abf64906c84decc116de1e0486679eb6ca696f233a403f756e2e431063c82abed4f0e342294c2fe71af69111faea3765b78cb90c03f'), path=PosixPath('/Users/grecco/Documents/code/flexparser/examples/in_readme/source1.txt'), mtime=1658550284.9419456)\n UnknownStatement(start_line=1, start_col=0, end_line=1, end_col=12, raw='one <- other')\n UnknownStatement(start_line=2, start_col=0, end_line=2, end_col=11, raw='2two <- new')\n UnknownStatement(start_line=3, start_col=0, end_line=3, end_col=17, raw='three <- newvalue')\n UnknownStatement(start_line=4, start_col=0, end_line=4, end_col=12, raw='one == three')\n EOS(start_line=5, start_col=0, end_line=5, end_col=0, raw=None)\n\n\nNotice that there are a lot of ``UnknownStatement`` now, because we instructed\nthe parser to only look for assignment within a block. So change your text file to:\n\n.. code-block:: text\n\n begin\n one <- other\n 2two <- new\n three <- newvalue\n end\n one == three\n\nand try again:\n\n.. code-block:: text\n\n BOF(start_line=0, start_col=0, end_line=0, end_col=0, raw=None, content_hash=Hash(algorithm_name='blake2b', hexdigest='3d8ce0051dcdd6f0f80ef789a0df179509d927874f242005ac41ed886ae0b71a30b845b9bfcb30194461c0ef6a3ca324c36f411dfafc7e588611f1eb0269bb5a'), path=PosixPath('/Users/grecco/Documents/code/flexparser/examples/in_readme/source2.txt'), mtime=1658550707.1248093)\n Begin(start_line=1, start_col=0, end_line=1, end_col=5, raw='begin')\n Assigment(start_line=2, start_col=0, end_line=2, end_col=12, raw='one <- other', lhs='one', rhs='other')\n InvalidIdentifier(start_line=3, start_col=0, end_line=3, end_col=11, raw='2two <- new', value='2two')\n Assigment(start_line=4, start_col=0, end_line=4, end_col=17, raw='three <- newvalue', lhs='three', rhs='newvalue')\n End(start_line=5, start_col=0, end_line=5, end_col=3, raw='end')\n EqualityComparison(start_line=6, start_col=0, end_line=6, end_col=12, raw='one == three', lhs='one', rhs='three')\n EOS(start_line=7, start_col=0, end_line=7, end_col=0, raw=None)\n\n\nUntil now we have used ``parsed.iter_statements`` to iterate over all parsed statements.\nBut let's look inside ``parsed``, an object of ``ParsedProject`` type. It is a thin wrapper\nover a dictionary mapping files to parsed content. Because we have provided a single file\nand this does not contain a link another, our ``parsed`` object contains a single element.\nThe key is ``None`` indicating that the file 'source.txt' was loaded from the root location\n(None). The content is a ``ParsedSourceFile`` object with the following attributes:\n\n- **path**: full path of the source file\n- **mtime**: modification file of the source file\n- **content_hash**: hash of the pickled content\n- **config**: extra parameters that can be given to the parser (see below).\n\n.. code-block:: text\n\n ParsedSource(\n parsed_source=parse..CustomRootBlock(\n opening=BOF(start_line=0, start_col=0, end_line=0, end_col=0, raw=None, content_hash=Hash(algorithm_name='blake2b', hexdigest='3d8ce0051dcdd6f0f80ef789a0df179509d927874f242005ac41ed886ae0b71a30b845b9bfcb30194461c0ef6a3ca324c36f411dfafc7e588611f1eb0269bb5a'), path=PosixPath('/Users/grecco/Documents/code/flexparser/examples/in_readme/source2.txt'), mtime=1658550707.1248093),\n body=(\n Block.subclass_with..CustomBlock(\n opening=Begin(start_line=1, start_col=0, end_line=1, end_col=5, raw='begin'),\n body=(\n Assigment(start_line=2, start_col=0, end_line=2, end_col=12, raw='one <- other', lhs='one', rhs='other'),\n InvalidIdentifier(start_line=3, start_col=0, end_line=3, end_col=11, raw='2two <- new', value='2two'),\n Assigment(start_line=4, start_col=0, end_line=4, end_col=17, raw='three <- newvalue', lhs='three', rhs='newvalue')\n ),\n closing=End(start_line=5, start_col=0, end_line=5, end_col=3, raw='end')),\n EqualityComparison(start_line=6, start_col=0, end_line=6, end_col=12, raw='one == three', lhs='one', rhs='three')),\n closing=EOS(start_line=7, start_col=0, end_line=7, end_col=0, raw=None)),\n config=None\n )\n\n\nA few things to notice:\n\n1. We were using a block before without knowing. The ``RootBlock`` is a\n special type of Block that starts and ends automatically with the\n file.\n2. ``opening``, ``body``, ``closing`` are automatically annotated with the\n possible ``ParsedStatement`` (plus `ParsingError`),\n therefore autocompletes works in most IDEs.\n3. The same is true for the defined ``ParsedStatement`` (we have use\n ``dataclass`` for a reason). This makes using the actual\n result of the parsing a charm!.\n4. That annoying ``subclass_with.`` is because we have built\n a class on the fly when we used ``Block.subclass_with``. You can\n get rid of it (which is actually useful for pickling) by explicit\n subclassing Block in your code (see below).\n\n\nMultiple source files\n---------------------\n\nMost projects have more than one source file internally connected.\nA file might refer to another that also need to be parsed (e.g. an\n`#include` statement in c). **flexparser** provides the ``IncludeStatement``\nbase class specially for this purpose.\n\n.. code-block:: python\n\n @dataclass(frozen=True)\n class Include(fp.IncludeStatement):\n \"\"\"A naive implementation of #include \"file\"\n \"\"\"\n\n value: str\n\n @classmethod\n def from_string(cls, s):\n if s.startwith(\"#include \"):\n return None\n\n value = s[len(\"#include \"):].strip().strip('\"')\n\n return cls(value)\n\n @propery\n def target(self):\n return self.value\n\nThe only difference is that you need to implement a ``target`` property\nthat returns the file name or resource that this statement refers to.\n\n\nCustomizing statementization\n----------------------------\n\nstatementi ... what? **flexparser** works by trying to parse each statement with\none of the known classes. So it is fair to ask what is an statement in this\ncontext and how can you configure it to your needs. A text file is split into\nnon overlapping strings called **statements**. Parsing work as follows:\n\n1. each file is split into statements (can be single or multi line).\n2. each statement is parsed with the first of the contextually\n available ParsedStatement or Block subclassed that returns\n a ``ParsedStatement`` or ``ParsingError``\n\nYou can customize how to split each line into statements with two arguments\nprovided to parse:\n\n- **strip_spaces** (`bool`): indicates that leading and trailing spaces must\n be removed before attempting to parse.\n (default: True)\n- **delimiters** (`dict`): indicates how each line must be subsplit.\n (default: do not divide)\n\nAn delimiter example might be\n``{\";\": (fp.DelimiterInclude.SKIP, fp.DelimiterAction.CONTINUE)}``\nwhich tells the statementizer (sorry) that when a \";\" is found a new statement should\nbegin. ``DelimiterMode.SKIP`` tells that \";\" should not be added to the previous\nstatement nor to the next. Other valid values are ``SPLIT_AFTER`` and ``SPLIT_BEFORE``\nto append or prepend the delimiter character to the previous or next statement.\nThe second element tells the statementizer (sorry again) what to do next:\nvalid values are: `CONTINUE`, `CAPTURE_NEXT_TIL_EOL`, `STOP_PARSING_LINE`, and\n`STOP_PARSING`.\n\nThis is useful with comments. For example,\n``{\"#\": (fp.DelimiterMode.WITH_NEXT, fp.DelimiterAction.CAPTURE_NEXT_TIL_EOL))}``\ntells the statementizer (it is not funny anymore) that after the first \"#\"\nit should stop splitting and capture all.\n\nThis allows:\n\n.. code-block:: text\n\n ## This will work as a single statement\n # This will work as a single statement #\n # This will work as # a single statement #\n a = 3 # this will produce two statements (a=3, and the rest)\n\n\nExplicit Block classes\n----------------------\n\n.. code-block:: python\n\n class AssigmentBlock(fp.Block[Begin, Assigment, End]):\n pass\n\n class EntryBlock(fp.RootBlock[Union[AssigmentBlock, Equality]]):\n pass\n\n parsed = fp.parse(\"source.txt\", EntryBlock)\n\n\nCustomizing parsing\n-------------------\n\nIn certain cases you might want to leave to the user some configuration\ndetails. We have method for that!. Instead of overriding ``from_string``\noverride ``from_string_and_config``. The second argument is an object\nthat can be given to the parser, which in turn will be passed to each\n``ParsedStatement`` class.\n\n.. code-block:: python\n\n @dataclass(frozen=True)\n class NumericAssigment(fp.ParsedStatement):\n \"\"\"Parses the following `this <- other`\n \"\"\"\n\n lhs: str\n rhs: numbers.Number\n\n @classmethod\n def from_string_and_config(cls, s, config):\n if \"==\" not in s:\n # This means: I do not know how to parse it\n # try with another ParsedStatement class.\n return None\n lhs, rhs = s.split(\"==\")\n return cls(lhs.strip(), config.numeric_type(rhs.strip()))\n\n class Config:\n\n numeric_type = float\n\n parsed = fp.parse(\"source.txt\", NumericAssigment, Config)\n\n----\n\nThis project was started as a part of Pint_, the python units package.\n\nSee AUTHORS_ for a list of the maintainers.\n\nTo review an ordered list of notable changes for each version of a project,\nsee CHANGES_\n\n.. _`AUTHORS`: https://github.com/hgrecco/flexparser/blob/main/AUTHORS\n.. _`CHANGES`: https://github.com/hgrecco/flexparser/blob/main/CHANGES\n.. _`Pint`: https://github.com/hgrecco/pint\n","description_content_type":"text/x-rst","docs_url":null,"download_url":null,"downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":null,"keywords":"parser, code, parsing, source","license":"BSD-3-Clause","license_expression":null,"license_files":null,"maintainer":null,"maintainer_email":"\"Hernan E. Grecco\" ","name":"flexparser","package_url":"https://pypi.org/project/flexparser/","platform":null,"project_url":"https://pypi.org/project/flexparser/","project_urls":{"Homepage":"https://github.com/hgrecco/flexparser"},"provides_extra":["test"],"release_url":"https://pypi.org/project/flexparser/0.4/","requires_dist":["typing-extensions","pytest; extra == \"test\"","pytest-mpl; extra == \"test\"","pytest-cov; extra == \"test\"","pytest-subtests; extra == \"test\""],"requires_python":">=3.9","summary":"Parsing made fun ... using typing.","version":"0.4","yanked":false,"yanked_reason":null}

Dependency	Optional
typing-extensions	false
pytest	true
pytest-cov	true
pytest-mpl	true
pytest-subtests	true

Package search

flexparser

project_urls

Dependencies

All package metadata

Releases