{"author":"Peter Plantinga, Aku Rouhe","author_email":"speechbrain@gmail.com","bugtrack_url":null,"classifiers":["License :: OSI Approved :: Apache Software License","Programming Language :: Python :: 3"],"description":"Introduction\n------------\n\nA crucial element of systems for data-analysis is laying out all the\nhyperparameters of that system so they can be easily examined and modified.\nWe add a few useful extensions to a popular human-readable data-serialization\nlanguage known as YAML (YAML Ain't Markup Language). This provides support\nfor a rather expansive idea of what constitutes a hyperparameter, and cleans\nup python files for data analysis to just the bare algorithm.\n\n### Table of Contents\n* [YAML basics](#yaml-basics)\n* [HyperPyYAML](#hyperpyyaml)\n * [Objects](#objects)\n * [Aliases](#aliases)\n * [Tuples](#tuples)\n* [How to use HyperPyYAML](#how-to-use-hyperpyyaml)\n* [Conclusion](#conclusion)\n\nYAML basics\n-----------\n\nYAML is a data-serialization language, similar to JSON, and it supports\nthree basic types of nodes: scalar, sequential, and mapping. PyYAML naturally\nconverts sequential nodes to python lists and mapping nodes to python dicts.\n\nScalar nodes can take one of the following forms:\n\n```yaml\nstring: abcd # No quotes needed\ninteger: 1\nfloat: 1.3\nbool: True\nnone: null\n```\n\nNote that we've used a simple mapping to demonstrate the scalar nodes. A mapping\nis a set of `key: value` pairs, defined so that the key can be used to easily\nretrieve the corresponding value. In addition to the format above, mappings\ncan also be specified in a similar manner to JSON:\n\n```yaml\n{foo: 1, bar: 2.5, baz: \"abc\"}\n```\n\nSequences, or lists of items, can also be specified in two ways:\n\n```yaml\n- foo\n- bar\n- baz\n```\n\nor\n\n```yaml\n[foo, bar, baz]\n```\n\nNote that when not using the inline version, YAML uses whitespace to denote\nnested items:\n\n```yaml\nfoo:\n a: 1\n b: 2\nbar:\n - c\n - d\n```\n\nYAML has a few more advanced features (such as\n[aliases](https://pyyaml.org/wiki/PyYAMLDocumentation#aliases) and\n[merge keys](https://yaml.org/type/merge.html)) that you may want to explore\non your own. We will briefly discuss one here since it is relevant for our\nextensions: [YAML tags](https://pyyaml.org/wiki/PyYAMLDocumentation#tags).\n\nTags are added with a `!` prefix, and they specify the type of the node. This\nallows types beyond the simple types listed above to be used. PyYAML supports a\nfew additional types, such as:\n\n```yaml\n!!set # set\n!!timestamp # datetime.datetime\n!!python/tuple # tuple\n!!python/complex # complex\n!!python/name:module.name # A class or function\n!!python/module:package.module # A module\n!!python/object/new:module.cls # An instance of a class\n```\n\nThese can all be quite useful, however we found that this system was a bit\ncumbersome, especially with the frequency with which we were using them. So\nwe decided to implement some shortcuts for these features, which we are\ncalling \"HyperPyYAML\".\n\nHyperPyYAML\n-----------\n\nWe make several extensions to yaml including easier object creation, nicer\naliases, and tuples.\n\n### Objects\n\nOur first extension is to simplify the structure for specifying an instance,\nmodule, class, or function. As an example:\n\n```yaml\nmodel: !new:collections.Counter\n```\n\nThis tag, prefixed with `!new:`, constructs an instance of the specified class.\nIf the node is a mapping node, all the items are passed as keyword arguments\nto the class when the instance is created. A list can similarly be used to\npass positional arguments. See the following examples:\n\n```yaml\nfoo: !new:collections.Counter\n - abracadabra\nbar: !new: collections.Counter\n a: 2\n b: 1\n c: 5\n```\n\nWe also simplify the interface for specifying a function or class or other\nstatic Python entity:\n\n```yaml\nadd: !name:operator.add\n```\n\nThis code stores the `add` function. It can later be used in the usual way:\n\n```python\n>>> loaded_yaml = load_hyperpyyaml(\"add: !name:operator.add\")\n>>> loaded_yaml[\"add\"](2, 4)\n6\n```\n\n### Aliases\n\nAnother extension is a nicer alias system that supports things like\nstring interpolation. We've added a tag written `!ref` that\ntakes keys in angle brackets, and searches for them inside the yaml\nfile itself. As an example:\n\n```yaml\nfolder1: abc/def\nfolder2: ghi/jkl\nfolder3: !ref /\n\nfoo: 1024\nbar: 512\nbaz: !ref // + 1\n```\n\nThis allows us to change some values and automatically change the\ndependent values accordingly.\nYou can also refer to other references, and to sub-nodes using brackets.\n\n```yaml\nblock_index: 1\ncnn1:\n out_channels: !ref * 64\n kernel_size: (3, 3)\ncnn2: \n out_channels: !ref \n kernel_size: (3, 3)\n```\n\nFinally, you can make references to nodes that are objects, not just scalars.\n\n```python\nyaml_string = \"\"\"\nfoo: !new:collections.Counter\n a: 4\nbar: !ref \nbaz: !copy \n\"\"\"\nloaded_yaml = load_hyperpyyaml(yaml_string)\nloaded_yaml[\"foo\"].update({\"b\": 10})\nprint(loaded_yaml[\"bar\"])\nprint(loaded_yaml[\"baz\"])\n```\n\nThis provides the output:\n```\nCounter({'b': 10, 'a': 4})\nCounter({'a': 4})\n```\n\nNote that `!ref` makes only a shallow copy, so updating `foo`\nalso updates `bar`. If you want a deep copy, use the `!copy` tag.\n\nThere are some issues (#7 #11) mentioning that `!ref` cannot refer to the return value of `!apply` function. \nThus we provide another `!applyref` tag to work with `!ref`, which can be used in four ways:\n\n```yaml\n# 1. Pass the positional and keyword arguments at the same time. Like `!!python/object/apply:module.function` in pyyaml\nc: !applyref:sorted\n _args: \n - [3, 4, 1, 2]\n _kwargs:\n reverse: False\nd: !ref -\n\n# 2. Only pass the keyword arguments\ne: !applyref:random.randint\n a: 1\n b: 3\nf: !ref \n\n# 3. Only pass the positional arguments\ng: !applyref:random.randint\n - 1\n - 3\nh: !ref \n\n# 4. No arguments\ni: !applyref:random.random\nj: !ref \n```\n\nNote that `!applyref` cannot return an object, otherwise the `RepresenterError` will be raised.\n\n### Tuples\n\nOne last minor extension to the yaml syntax we've made is to implicitly\nresolve any string starting with `(` and ending with `)` to a tuple.\nThis makes the use of YAML more intuitive for Python users.\n\n\nHow to use HyperPyYAML\n---------------------\n\nAll of the listed extensions are available by loading yaml using the\n`load_hyperpyyaml` function. This function returns an object in a similar\nmanner to pyyaml and other yaml libraries.\nAlso, `load_hyperpyyaml` takes an optional argument, `overrides`\nwhich allows changes to any of the parameters listed in the YAML.\nThe following example demonstrates changing the `out_channels`\nof the CNN layer:\n\n```python\n>>> yaml_string = \"\"\"\n... block_index: 1\n... cnn1:\n... out_channels: !ref * 64\n... kernel_size: (3, 3)\n... cnn2: \n... out_channels: !ref \n... kernel_size: (3, 3)\n... \"\"\"\n>>> overrides = {\"block_index\": 2}\n>>> with open(\"hyperparameters.yaml\") as f:\n... hyperparameters = load_hyperpyyaml(f, overrides)\n>>> hyperparameters[\"block_index\"]\n2\n>>> hyperparameters[\"cnn2\"][\"out_channels\"]\n128\n```\n\nConclusion\n----------\n\nWe've defined a number of extensions to the YAML syntax, designed to\nmake it easier to use for hyperparameter specification. Feedback is welcome!\n","description_content_type":"text/markdown","docs_url":null,"download_url":"","downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":"https://github.com/speechbrain/HyperPyYAML","keywords":"","license":"","license_expression":null,"license_files":null,"maintainer":"","maintainer_email":"","name":"HyperPyYAML","package_url":"https://pypi.org/project/HyperPyYAML/","platform":null,"project_url":"https://pypi.org/project/HyperPyYAML/","project_urls":{"Homepage":"https://github.com/speechbrain/HyperPyYAML"},"provides_extra":null,"release_url":"https://pypi.org/project/HyperPyYAML/1.2.2/","requires_dist":["pyyaml >=5.1","ruamel.yaml >=0.17.28"],"requires_python":"","summary":"Extensions to YAML syntax for better python interaction","version":"1.2.2","yanked":false,"yanked_reason":null}

Dependencies of HyperPyYAML (2).
Dependency	Optional
pyyaml	false
ruamel.yaml	false

Package search

HyperPyYAML

project_urls

Dependencies

All package metadata

Releases