molt/cst

molt/cst: low-level concrete syntax tree manipulation

The functions in this module query and manipulate concrete syntax tree nodes of a TOML document.

molt/cst Functions

There are six categories of functions in the molt/cst interface.

parse_path is uncategorized as it is a helper for translating molt path strings (a.b.c[3].d) into path segments for use with molt/cst functions.

Example Document

Examples in this documentation assume the following TOML document:

# Project configuration
title = 'molt'
version = '1.0.0'
enabled = true # Do we need this?
max_retries = 3
rating = 4.5

# Inline table and inline array
owner = { name = \"Austin\", active = true }
project.tags = ['toml', \"parser\", 'gleam']

# Implicit table: `database` is never explicitly declared
[database.connection]
# Default host is localhost
host = 'localhost'
# Default port is the postgresql port
port = 5432
\"connection options\" = []

# Concrete table
[settings]
verbose = false
timeout = 30

[settings.debug]
level = 5

# Table array
[[plugins]]
name = 'formatter'
priority = 1

[[plugins]]
name = 'linter'
priority = 2
options = { strict = true, fix = false }

[[extensions]]

[app.'Microsoft Word'.options]
verbose = false

All function examples are operating on the document provided as a string called config:

import molt
import molt/cst
import molt/types.{IndexSegment, KeySegment}

use document <- result.try(molt.parse(config))
let example = cst.from_document(document)

Concrete Node Resolution

The molt/cst functions target concrete container nodes in the syntax tree, and advanced functions may traverse inner values.

Node Preservation

Modification to the CST preserves existing whitespace, comments, and formatting of unaffected nodes. Comments can be added, removed, or modified on any concrete node. Newly added nodes will have uniform formatting applied. If a node with comments attached to it is moved to a different location in the document, the comments follow that node.

Paths

molt/cst functions take paths as List(PathSegment), built from KeySegment and IndexSegment values or parsed from a string with parse_path.

Types

Position for inserting an entry into an array of tables.

Used with insert_array_of_tables_entry.

pub type EntryPosition {
  EntryAtEnd
  BeforeIndex(index: Int)
}

Constructors

  • EntryAtEnd

    Insert after the last entry in the specified array of tables.

  • BeforeIndex(index: Int)

    Insert before the index-th entry in the array of tables.

    The index reference supports negative indexing: -1 means “before the last entry”, -n means “before the first entry” and returns IndexOutOfRange if the resolved index exceeds the number of entries.

Position for inserting a key/value node into a table-like container, relative to other keys in the table.

Used with insert_kv.

pub type KeyPosition {
  KvAtEnd
  BeforeKey(key: String)
}

Constructors

  • KvAtEnd

    Insert after the last key/value, before the next table header.

    Insertions have limited region awareness: new entries are placed before any subtable headers.

  • BeforeKey(key: String)

    Insert immediately before the existing child key/value named key.

    Returns NotFound if key is not a direct child key/value of the container (no silent fall-back to append).

Values

pub fn build_array_of_tables(
  path path: List(String),
) -> greenwood.Node(types.TomlKind)

Build an empty array of tables header node ([[path.to.table]]\n).

cst.build_array_of_tables(["plugins"])  // -> [[plugins]]
cst.build_array_of_tables(["app", "hooks"])  // -> [[app.hooks]]

builder

pub fn build_comment_trivia(
  leading leading: List(String),
  trailing trailing: option.Option(String),
) -> greenwood.Trivia(types.TomlKind)

Build comment trivia for attaching to a node via greenwood.set_trivia or similar. Leading comments appear on lines before the node; the trailing comment appears on the same line after the value.

cst.build_comment_trivia(
  leading: ["# Section start"],
  trailing: Some("inline note"),
)

builder

pub fn build_inline_kv(
  key key: String,
  value value: greenwood.Element(types.TomlKind),
) -> greenwood.Node(types.TomlKind)

Build a key/value node for use inside inline tables (no trailing newline).

builder

pub fn build_kv(
  key key: String,
  value value: greenwood.Element(types.TomlKind),
) -> greenwood.Node(types.TomlKind)

Build a key/value node with standard formatting (key = value\n).

The key will be quoted if necessary. The value should be produced via value.to_cst.

cst.build_kv(key: "host", value: value.to_cst(value.string("localhost")))

builder

pub fn build_table(
  path path: List(String),
) -> greenwood.Node(types.TomlKind)

Build an empty table header node ([path.to.table]\n).

cst.build_table(["settings"])  // -> [settings]
cst.build_table(["database", "connection"])  // -> [database.connection]

builder

pub fn delete(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Delete the node at the given path. Returns the modified root. Works for key/values, tables, and array of table entries.

cst.delete(example, [KeySegment("plugins"), IndexSegment(0)])
// -> plugins[0] (name = 'formatter') has been removed

cst.delete(example, [KeySegment("database"), KeySegment("connection")])
// -> All of database.connection has been removed

edit

pub fn delete_where(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  where predicate: fn(greenwood.Node(types.TomlKind)) -> Bool,
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Delete a node at path matching a predicate.

cst.delete_where(
  example,
  [KeySegment("database"), KeySegment("connection"), KeySegment("host")],
  fn(n) { cst.value_text(n) == "'prod'"}
)
// -> NotFound as database.connection.host is 'localhost'

edit

pub fn document_head_comments(
  tree tree: greenwood.Node(types.TomlKind),
) -> List(String)

Read the document-head comments: the Root node’s own leading-trivia comment lines, returned verbatim (with the leading #). These are the comments that belong to the file rather than to any one statement.

comments

pub fn document_tail_comments(
  tree tree: greenwood.Node(types.TomlKind),
) -> List(String)

Read the document-tail comments: the leading-trivia comment lines of the PostScript tombstone, if the document has one (it has one exactly when trivia dangles after the final statement). Returns [] otherwise.

comments

pub fn ensure(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  kind kind: types.TomlKind,
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Ensure a table or array of table exists at path. Creates it if missing. The new declaration is placed before any existing descendant headers so that parent tables always precede their children.

cst.ensure(example, path: [KeySegment("my app")], kind: types.Table)
// -> creates a new table header ["my app"] at the end of the document

cst.ensure(example, path: [KeySegment("database")], kind: types.Table)
// -> creates a new table header [database] before [database.connection]

edit

pub fn from_document(
  doc: types.Document,
) -> greenwood.Node(types.TomlKind)

Extracts the concrete syntax tree from the parsed molt document.

conversion

pub fn get(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Retrieves a child node at the given path relative to node.

cst.get(example, [])
// -> the document root itself

cst.get(example, [KeySegment("rating")])
// -> The `rating` key/value node from root

let assert Ok(plugins2) = cst.get(example, [KeySegment("plugins"), IndexSegment(1)])
// -> The second `plugins` entry.

cst.get(plugins2, [KeySegment("name")])
// -> The `name` key/value node from the second `plugins` entry.

query

pub fn get_where(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  where predicate: fn(greenwood.Node(types.TomlKind)) -> Bool,
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Look up a node at path matching a predicate. Useful for disambiguating duplicate keys or selecting by node kind.

assert cst.get_where(example, [KeySegment("plugins")], fn(n) {
  list.contains(cst.list_keys(n), "options")
}) == cst.get(example, path: [KeySegment("plugins"), IndexSegment(-1)])

query

pub fn insert_array_of_tables_entry(
  node node: greenwood.Node(types.TomlKind),
  into segments: List(types.PathSegment),
  entry entry: greenwood.Node(types.TomlKind),
  at position: EntryPosition,
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Insert an array of tables entry node into an existing [[array.of.tables]] group.

The array of tables entry must be located using into of only KeySegments to find the appropriate siblings to place the table in the correct location.

let entry = cst.build_array_of_tables(["plugins"])

// Append after the last [[plugins]].
insert_array_of_tables_entry(
  doc,
  into: [KeySegment("plugins")],
  entry:,
  at: EntryAtEnd
)

// Insert before the 2nd [[a.b]] entry.
insert_array_of_tables_entry(
  doc,
  into: [KeySegment("a"), KeySegment("b")],
  entry:,
  at: BeforeIndex(2)
)

edit

pub fn insert_kv(
  node node: greenwood.Node(types.TomlKind),
  into segments: List(types.PathSegment),
  kv kv: greenwood.Node(types.TomlKind),
  at position: KeyPosition,
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Insert a key/value node into a table-like container resolved by the into path segments.

The into target must be the document root ([]), a table declaration [KeySegment("x"), KeySegment("y")], or a specific array of tables entry ([KeySegment("a"), IndexSegment(2)]).

Local consistency checks prevent the insertion of a key/value node with a key name already present in the target table. If a key/value node is to be inserted Before a key that does not exist, an error result will be returned.

// Append, region-aware: lands before any [sub] headers.
insert_kv(doc, into: [KeySegment("database")], kv:, at: KvAtEnd)

// Before a named sibling key.
insert_kv(doc, into: [KeySegment("database")], kv:, at: BeforeKey("port"))

// Into a specific AoT entry.
insert_kv(doc, into: [KeySegment("a"), IndexSegment(2)], kv:, at: KvAtEnd)

edit

pub fn insert_table_node(
  node node: greenwood.Node(types.TomlKind),
  table table: greenwood.Node(types.TomlKind),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Inserts a table or array of tables node into the document, placing it before any existing descendant headers so that parent tables always precede their children.

let table = cst.build_table(path: ["database"])
cst.insert_table_node(example, table)
// -> Inserts a new table node before [database.connection]

edit

pub fn key_name(
  kv: greenwood.Node(types.TomlKind),
) -> option.Option(String)

Get the key name from a key/value node.

let assert Ok(kv) = cst.get(example, [KeySegment("rating")])
assert Some("rating") == cst.key_name(kv)

query

pub fn leading_comments(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
) -> Result(List(String), error.MoltError)

Get leading comments attached to the node at path.

cst.leading_comments(
  example,
  [KeySegment("database"), KeySegment("connection"), KeySegment("host")]
)
// -> Ok(["# Default host is localhost"])

comments

pub fn list_keys(
  table: greenwood.Node(types.TomlKind),
) -> List(String)

List key names in a table (direct children only, no dotted key merging).

query

pub fn list_tables(
  node node: greenwood.Node(types.TomlKind),
) -> option.Option(List(List(String)))

List all explicit table or array table paths that are immediate children of the provided node if it is a document or table node. Returns None if the node is neither the root of the tree nor a table.

let assert Some([
  ["database", "connection"],
  ["settings"],
  ["settings", "debug"],
  ["plugins"],
  ["plugins"],
  ["extensions"],
  ["app", "Microsoft Word", "options"],
]) = cst.list_tables(example)

query

pub fn move(
  node node: greenwood.Node(types.TomlKind),
  from from: List(types.PathSegment),
  to to: List(types.PathSegment),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Move a node from one path to another.

cst.move(
  example,
  from: [KeySegment("settings"), KeySegment("timeout")],
  to: [
    KeySegment("database"),
    KeySegment("connection"),
    KeySegment("connection_timeout")
  ],
)
// -> Moves settings.timeout to database.connection.connection_timeout

edit

pub fn parse_path(
  path input: String,
) -> Result(List(types.PathSegment), error.MoltError)

Converts a path string into a list of path segments required by molt/cst functions, returning a InvalidPath if the path syntax is invalid.

The path format is fully documented in the molt module.

parse_path("a.b.c.d")
// -> [KeySegment("a"), KeySegment("b"), KeySegment("c"), KeySegment("d")]

parse_path("a.b.-1.d")
// -> [KeySegment("a"), KeySegment("b"), KeySegment("-1"), KeySegment("d")]

parse_path("a.b.\"with space\".d")
// -> [KeySegment("a"), KeySegment("b"), KeySegment("with space"), KeySegment("d")]

parse_path("a.b.\"\f\".'\\f'")
// -> [KeySegment("a"), KeySegment("b"), KeySegment("\f"), KeySegment("\\f")]

parse_path("a.b[-1].c")
// -> [KeySegment("a"), KeySegment("b"), IndexSegment(-1), KeySegment("c")]

parse_path("a.b[3].c")
// -> [KeySegment("a"), KeySegment("b"), IndexSegment(3), KeySegment("c")]
pub fn rename(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  to new_name: String,
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Rename the last segment of the path. Works for key/values, tables, and array of tables headers.

cst.rename(
  example,
  path: [KeySegment("plugins"), IndexSegment(1), KeySegment("name")],
  to: "id",
)
// -> renames plugins[1].name to plugins[1].id

cst.rename(
  example,
  path: [KeySegment("database"), KeySegment("connection")],
  to: "conn",
)
// -> renames [database.connection] to [database.conn]

edit

pub fn replace(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  new new: greenwood.Node(types.TomlKind),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Replace the node at path with a new node.

let assert Ok(existing) =
  cst.get(example, path: [
    KeySegment("plugins"),
    IndexSegment(0),
    KeySegment("priority"),
  ])

let assert Ok(new_kv) =
  cst.set_kv_value(kv: existing, value: value.to_cst(value.int(10)))
cst.replace(
  example,
  path: [KeySegment("plugins"), IndexSegment(0), KeySegment("priority")],
  new: new_kv,
)
// -> changes plugins[0].priority to 10

edit

pub fn set_document_head_comments(
  tree tree: greenwood.Node(types.TomlKind),
  comments comments: List(String),
) -> greenwood.Node(types.TomlKind)

Replace the document-head comments on the Root node’s leading trivia, preserving a leading BOM. An empty list clears the comments (keeping the BOM). The blank line that separates the head comment from the first statement is added at emit time (see emitter.ensure_head_separation), so it tracks the document’s content and line-ending style rather than being baked in here.

comments

pub fn set_document_tail_comments(
  tree tree: greenwood.Node(types.TomlKind),
  comments comments: List(String),
) -> greenwood.Node(types.TomlKind)

Replace the document-tail comments. A non-empty list materializes (or updates) the PostScript tombstone as the document’s last child; an empty list drops the tombstone entirely so no empty node lingers.

comments

pub fn set_kv_value(
  kv kv: greenwood.Node(types.TomlKind),
  value value: greenwood.Element(types.TomlKind),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Replace the value of a key/value pair node, preserving the key, surrounding whitespace, and any attached comments.

The value is a CST element produced from a value.Value with value.to_cst, so the replacement is a correctly-typed value node (integer, string, array, inline table, etc.). Build it from a value.Value rather than from raw text.

import molt/value

let assert Ok(existing) =
  cst.get(example, path: [
    KeySegment("plugins"),
    IndexSegment(0),
    KeySegment("priority"),
  ])

let assert Ok(new_kv) =
  cst.set_kv_value(kv: existing, value: value.to_cst(value.int(10)))

edit

pub fn set_leading_comments(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  comments cmts: List(String),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Set leading comments on the node at path.

cst.set_leading_comments(
  example,
  path: [KeySegment("database"), KeySegment("connection"), KeySegment("port")],
  comments: ["Listen port"],
)
// -> Adds "# Listen port" before database.connection.port

comments

pub fn set_trailing_comment(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  comment comment: option.Option(String),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Set or clear the trailing comment on the node at path. Pass None to remove an existing trailing comment.

cst.set_trailing_comment(example, [KeySegment("rating")], Some("Higher!"))
// -> Adds "# Higher!" to rating

cst.set_trailing_comment(
  example,
  [KeySegment("enabled")],
  None
)
// -> Removes the comment from enabled.

comments

pub fn strip_all_comments(
  node node: greenwood.Node(types.TomlKind),
) -> greenwood.Node(types.TomlKind)

Recursively strip all comments from the tree.

cst.strip_all_comments(example)
// -> Removes all leading and trailing comments from the tree

comments

pub fn to_document(
  tree: greenwood.Node(types.TomlKind),
) -> types.Document

Converts the concrete syntax tree to a parsed TOML 1.1 document.

The tree is validated on conversion (counting any errors), so the resulting Document has an accurate error_count and is ready for the molt logical API. Inspect errors with molt.has_errors / molt.document_errors.

conversion

pub fn to_document_version(
  tree tree: greenwood.Node(types.TomlKind),
  version version: types.TomlVersion,
) -> types.Document

Converts the concrete syntax tree to a parsed TOML document using the specified TOML version.

The tree is validated on conversion (counting any errors), so the resulting Document has an accurate error_count and is ready for the molt logical API. Inspect errors with molt.has_errors / molt.document_errors.

conversion

pub fn trailing_comment(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
) -> Result(option.Option(String), error.MoltError)

Get the trailing comment on the node at path, if any.

cst.trailing_comment(example, [KeySegment("enabled")])
// -> Ok(Some("# Do we need this?"))

comments

pub fn update(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  with transform: fn(greenwood.Node(types.TomlKind)) -> greenwood.Node(
    types.TomlKind,
  ),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Update the node at path via a transform function.

cst.update(example,
  path: [
    KeySegment("database"),
    KeySegment("connection"),
    KeySegment("port")
  ],
  with: fn(kv) {
    let assert Ok(updated) =
      cst.set_kv_value(kv:, value: value.to_cst(value.int(9090)))
    updated
  }
)
// -> Updates database.connection.port to 9090

edit

pub fn update_where(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  where predicate: fn(greenwood.Node(types.TomlKind)) -> Bool,
  with transform: fn(greenwood.Node(types.TomlKind)) -> greenwood.Node(
    types.TomlKind,
  ),
) -> Result(greenwood.Node(types.TomlKind), error.MoltError)

Update a node at path matching a predicate via a transform function.

cst.update_where(
  example,
  path: [KeySegment("database"), KeySegment("connection"), KeySegment("port")],
  where: fn(n) { cst.value_text(n) == "9999" },
  with: fn(n) {
    let assert Ok(updated) =
      cst.set_kv_value(kv: n, value: value.to_cst(value.int(9090)))
    updated
  },
)
// -> Returns not found because database.connection.port is 5432, not 9999.

edit

pub fn value_text(kv: greenwood.Node(types.TomlKind)) -> String

Get the raw value text from a key/value node.

let assert Ok(kv) = cst.get(example, [KeySegment("rating")])
assert "4.5" == cst.value_text(kv)

query

pub fn zipper_at(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
) -> Result(greenwood.Zipper(types.TomlKind), error.MoltError)

Returns a zipper focused on the first node matching the concrete path.

cst.zipper_at(example, [KeySegment("plugins"), IndexSegment(1)])
// -> a zipper focused on plugins[1]

cst.zipper_at(example:, [KeySegment("database"), KeySegment("connection"), KeySegment("port")])
// -> a zipper focused on database.connection.port

advanced

pub fn zipper_where(
  node node: greenwood.Node(types.TomlKind),
  path segments: List(types.PathSegment),
  where predicate: fn(greenwood.Node(types.TomlKind)) -> Bool,
) -> Result(greenwood.Zipper(types.TomlKind), error.MoltError)

Returns a zipper for the given path using a predicate to disambiguate duplicate keys or node kind.

cst.zipper_where(
  example:,
  [KeySegment("database"), KeySegment("connection"), KeySegment("port")],
  fn(n) { cst.value_text(n) == "5432" },
)

// -> a zipper focused on database.connection.port

advanced

Search Document