Dyon is a rusty dynamically scripting language, borrowing ideas from Javascript, Go, mathematical notation and of course Rust!

In this post I will go in depth about two features of Dyon, and discuss them from the perspective that I use and need them.

Dynamically loaded modules

One of the things that makes Dyon special is the way of organizing code using dynamically loaded modules. When scaling up big projects in Rust, I noticed that each project required a configuration file to tell which dependencies it has. This is useful, but introduces extra work when moving code around. Another problem is when I work on some game, and want to refresh the game logic while running, there are many ways to do this and it depends on the specific project.

Sometimes I want to experiment with different algorithms and compare them side by side. While a version control system is great, it does not allow you the playful coding style I like. The projects I work on tend to focus on exploring some idea, and this leads to a dozen files spread across folders which does not fall into a neat namespace hierarchy. I do not want to think too much about how to name stuff, it gets in the way for productivity.

A Dyon script file does not know which other files it depends on, so you need to tell it for each project.

Setting up dependencies is part of a loader script, that loads modules and connect them together.

For example:

fn main() {
    find := unwrap(load("src/find.dyon"))
    unop_f64 := unwrap(load("../dyon_lib/unop_f64/src/lib.dyon"))
    unop_f64_lib := unwrap(load(
        source: "src/unop_f64_lib.dyon",
        imports: [unop_f64]
    ))
    set_alg := unwrap(load("../dyon_lib/set_alg/src/lib.dyon"))
    main := unwrap(load(
        source: "src/main.dyon",
        imports: [unop_f64_lib, set_alg, find]
    ))
    call(main, "main", [])
}

Horrible, right? Yet, there are some upsides:

  1. Easy to change the loader script to reload code on the fly
  2. Easy to generate code at loading and running time
  3. Easy to control preloading of modules that does not need reloading
  4. Easy to rewrite and refactor code from Dyon to Rust
  5. One configuration file per project

In most languages, you need some complicated system for these things, but in Dyon you learn how do it as a beginner.

The 0.13 version adds some important changes:

  1. Modules are now represented as Arc<Module> behind a Arc<Mutex<Any>> pointer
  2. External functions (Rust code) are resolved directly instead of depending on the module

The reason for 1) is when closures escape module boundary. Variables in Dyon are 24 bytes on 64 bit architectures, which was makes it possible to fit in more stuff like 4D vectors and secrets. I squeezed in a pointer to the closure environment stored beside the closure pointer. This keeps the track of the information required to find the relative location of loaded functions. The Arc<Module> pointer makes it possible to create the closure environment faster, instead of cloning the whole module for each closure.

The reason for 2) is when I use another pattern:

fn main() {
    window := load_window()
    image := load_image()
    video := load_video()
    input := load_input()
    draw := unwrap(load(
        source: "src/draw.dyon",
        imports: [image]
    ))
    sample := unwrap(load(
        source: "src/sample.dyon",
        imports: [draw]
    ))
    animation := unwrap(load(
        source: "src/animation.dyon",
        imports: [sample]
    ))
    main := unwrap(load(
        source: "src/programs/checkerboard.dyon",
        imports: [window, input, animation, video]
    ))
    call(main, "main", [])
}

In the example above load_window is an external function (written in Rust) that returns a module. The returned module containst other external functions, which then get imported into the other modules.

To make this logic work, I decided to use a direct function pointer and wrap it in FnExternalRef. The downside is that you can not track down the location of the external function when something bad happens. I am crossing my fingers and hope this will not be a big problem (famous last words).

Try expressions

A new feature in 0.13 is the ability to use try to turn failures into err and success into ok.

Link to design notes

Some background of why I care about this so much: I am working on path semantics, which is a big dream I have.

A path is a function that relates one piece of a function to another function in a space called “path semantics”, because it is a space of meaning (semantics). Normally there is a bunch of paths for a function, that might differ for arguments and return value, but they all need to work together to get to the other function.

When the same path is used everywhere, it is called a “symmetric path”.

Symmetric paths are extremely mathematically beautiful and rare.

Example:

and[not] <=> or

which is equal to, but not meaning the same as the equation:

not(and(X, Y)) <=> or(not(X), not(Y))

I just occured to me that this is one of De Morgan’s laws.

Here is another example:

concat[len] <=> add

which is equal to, but not meaning the same as the equation:

len(concat(X, Y)) <=> add(len(X), len(Y))

You can use these laws to do transformations like these:

add(X, Y)
concat[len](X, Y) // using a symmetric path to reach the other function
// there exists `Z` and `W` such that `X = len(Z)` and `Y = len(W)`
concat[len](len(Z), len(W))
len(concat(Z, W))

I want to create an algorithm to explore the space of path semantics efficiently. There is no currently proof technique I know to derive them, except to “make a hypothesis and falsify”, kind of like how scientists do. The only way to find which functions that works with others is by trying.

Here is a function that tests for a symmetric path candidate:

/// Infers whether a path can be used for a function
/// in a symmetric way for both arguments and result.
sym(f: \([]) -> any, p: \([]) -> any, args: []) =
    all i {is_ok(try \p([args[i]]))} &&
    is_ok(try \p([\f(args)]))

Let me take this apart an explain the pieces:

  • sym is the function name
  • f: \([]) -> any is an argument f of type closure taking [] (array) and returning any
  • p: \([]) -> any is an argument p of type closure taking [] (array) and returning any
  • args: [] is an argument args of type [] (array)
  • \p([args[i]]) calls the closure p
  • \p([\f(args)]) calls the closure f and then p
  • all i { ... } is a mathematical loop that automatically infers the range [0, len(args)) from the body

The \ notation is used in Dyon because type checking happens in parallel with AST construction, plus some other design choices that closures always return a value. I needed a design that was easy to refactor and different for normal functions and closures.

The problem was that this function gets called many times with different closures, and sometimes it fails and crashes because the types are wrong.

try converts all failures into err, such that the function sym can detect a path candidate.

Now, you might think, why not use try and catch blocks?

Dyon already has a way of dealing with errors: res, like Result in Rust but with a trace and any as error type. There is no need to add a second way of dealing with errors, when the first method works just fine.

For Rust programmers, the try keyword can be a bit confusing, since they might be thinking of the try! macro. In Dyon, you use the ? operator instead of try!, just like in Rust. I decided to use it because it feels closer to what we mean with “trying” in natural language.

For example, when a function foo returns res, you can write:

fn foo() -> res { ... }

fn bar() -> res {
    a := try foo()?
    ...
}

This is executed as try (foo()?), returning err early, so you can handle the case where it failed. Interestingly, it gives the same result as (try foo())?.