With YAMLScript, YAML Becomes a Proper Programming Language

Does YAML data need to be programmed? Many think so, including one of the creators of YAML itself. Ingy döt Net, has started a project to bringing scripting capabilities to the data serialization language, in a project called YAMLScript. SUSE Engineer Tina Müller dropped the news in her annual FOSDEM talk last month. With YAMLScript, all valid YAML code is valid YAMLScript code. Plus, all YAMLScript function code, since it is itself in the YAML syntax, can be directly embedded into YAML files, or loaded in from other files. The new programmable functionality will include “excellent interpolation features,”  such as merge, filter, and concatenate.  And people can create their own “generators” to manipulate data on the fly, döt Net promised. The work is still in the early stages, with the latest release of the compiler being 0.1.41 released earlier this week. But the idea is that it will solve “most programming things people want to do with YAML,” döt Net said, through the conference medium of Müller.

YAML Itself Is Not a Programming Language

YAML wasn’t originally designed for programming, even though many people wish it were. A misunderstood language since its debut in 2004, YAML was the work of döt Net, Oren Ben-Kiki and Clark Evans. As per its name, YAML (now short for “YAML Ain’t Markup Language”) actually is not a markup language (such as HTML for the web), but rather a data serialization language, a superset of JSON in fact. YAML (current version 1.2.2) offers multiple ways to string data. A flow sequence can written as a comma-separated list within square brackets:

- [  Bob Marley, Peter Tosh, Bunny Wailer]

Or! Data can be written as block collections using indentation for scope:

-Gene Clark
-Roger McGuinn
-David Crosby

Behind this simple format, however, lies a bewildering array of options and rules of how data can be further marked up, as well as how it could be interpreted. But, to date, it has not been used as a programming language. By itself, YAML has no idea what a “variable” or “function” is.

${{ … }} Is Not YAML

But even frequent users can be forgiven for assuming YAML was a programming language. In infrastructure management software, YAML data is often festooned with executional code overlaid by templating software. VMware‘s Saltstack embeds YAML in its Salt State Files this way, and people assume the coding is in YAML, Müller said, demonstrating with code snippets. Red Hat‘s Ansible does something similar, embedding YAML as strings in its configuration cade, Müller observed. Both of those automation tools use Jinja templates to add in the coding. Many people think the overlays are actually YAML, she said. They flood the YAML channel (#chat:yaml.io on matrix; libera.chat#yaml on irc) with queries about their broken code. GitHub Actions Workflow also embeds YAML, within the center of ${{ … }} strings. Again, not YAML. VMware, Red Hat and GitHub aren’t alone in jumping ahead of YAML. The static limitations of YAML have been particularly acute with Kubernetes, where it is used as a configuration format. In 2020, Amazon Web Services released cdk8s, a framework for capturing Kubernetes configuration data so it can be shared for different uses. And Pulumi has built its core value proposition around the ability to manage configuration data, even configuration data for Kubernetes, not with clumsy YAML files, but with programming code itself.

YAML Is Now a (Functional) Programming Language

döt Net’s YAMLScript compiler/interpreter (“ys“) can be downloaded with curl and unwrapped at the command line. Or it can be imported as a library to any one of a number of languages, such as Python. The ys interpreter ingests any valid YAML code as YAMLScript. The user only need to preface YAML functional code with: !yamlscript/v0. Rosetta Code shows all the many ways you can say “Hello World” in YAMLScript: Beneath the covers, YAMLScript compiles to Clojure code, run by the Small Clojure Interpreter (SCI). Since Clojure is a functional programming language, one based on a Lisp syntax — “even though it commonly doesn’t look like one syntactically,” the docs note — YAML is then, technically, a functional programming language (instantly making it one of the mostly widely used functional languages after Microsoft Excel).