Dyon is a rusty dynamically scripting language, borrowing ideas from Javascript, Go, mathematical notation and of course Rust!
In this post I will go in depth about two features of Dyon, and discuss them from the perspective that I use and need them.
Dynamically loaded modules
One of the things that makes Dyon special is the way of organizing code using dynamically loaded modules. When scaling up big projects in Rust, I noticed that each project required a configuration file to tell which dependencies it has. This is useful, but introduces extra work when moving code around. Another problem is when I work on some game, and want to refresh the game logic while running, there are many ways to do this and it depends on the specific project.
Sometimes I want to experiment with different algorithms and compare them side by side. While a version control system is great, it does not allow you the playful coding style I like. The projects I work on tend to focus on exploring some idea, and this leads to a dozen files spread across folders which does not fall into a neat namespace hierarchy. I do not want to think too much about how to name stuff, it gets in the way for productivity.
A Dyon script file does not know which other files it depends on, so you need to tell it for each project.
Setting up dependencies is part of a loader script, that loads modules and connect them together.
For example:
fn main() {
find := unwrap(load("src/find.dyon"))
unop_f64 := unwrap(load("../dyon_lib/unop_f64/src/lib.dyon"))
unop_f64_lib := unwrap(load(
source: "src/unop_f64_lib.dyon",
imports: [unop_f64]
))
set_alg := unwrap(load("../dyon_lib/set_alg/src/lib.dyon"))
main := unwrap(load(
source: "src/main.dyon",
imports: [unop_f64_lib, set_alg, find]
))
call(main, "main", [])
}
Horrible, right? Yet, there are some upsides:
- Easy to change the loader script to reload code on the fly
- Easy to generate code at loading and running time
- Easy to control preloading of modules that does not need reloading
- Easy to rewrite and refactor code from Dyon to Rust
- One configuration file per project
In most languages, you need some complicated system for these things, but in Dyon you learn how do it as a beginner.
The 0.13 version adds some important changes:
- Modules are now represented as
Arc<Module>
behind aArc<Mutex<Any>>
pointer - External functions (Rust code) are resolved directly instead of depending on the module
The reason for 1) is when closures escape module boundary.
Variables in Dyon are 24 bytes on 64 bit architectures,
which was makes it possible to fit in more stuff like 4D vectors and secrets.
I squeezed in a pointer to the closure environment stored beside the closure pointer.
This keeps the track of the information required to find the relative location of loaded functions.
The Arc<Module>
pointer makes it possible to create the closure environment faster,
instead of cloning the whole module for each closure.
The reason for 2) is when I use another pattern:
fn main() {
window := load_window()
image := load_image()
video := load_video()
input := load_input()
draw := unwrap(load(
source: "src/draw.dyon",
imports: [image]
))
sample := unwrap(load(
source: "src/sample.dyon",
imports: [draw]
))
animation := unwrap(load(
source: "src/animation.dyon",
imports: [sample]
))
main := unwrap(load(
source: "src/programs/checkerboard.dyon",
imports: [window, input, animation, video]
))
call(main, "main", [])
}
In the example above load_window
is an external function (written in Rust) that returns a module.
The returned module containst other external functions, which then get imported into the other modules.
To make this logic work, I decided to use a direct function pointer and wrap it in FnExternalRef
.
The downside is that you can not track down the location of the external function when something bad happens.
I am crossing my fingers and hope this will not be a big problem (famous last words).
Try expressions
A new feature in 0.13 is the ability to use try
to turn failures into err
and success into ok
.
Some background of why I care about this so much: I am working on path semantics, which is a big dream I have.
A path is a function that relates one piece of a function to another function in a space called “path semantics”, because it is a space of meaning (semantics). Normally there is a bunch of paths for a function, that might differ for arguments and return value, but they all need to work together to get to the other function.
When the same path is used everywhere, it is called a “symmetric path”.
Symmetric paths are extremely mathematically beautiful and rare.
Example:
and[not] <=> or
which is equal to, but not meaning the same as the equation:
not(and(X, Y)) <=> or(not(X), not(Y))
I just occured to me that this is one of De Morgan’s laws.
Here is another example:
concat[len] <=> add
which is equal to, but not meaning the same as the equation:
len(concat(X, Y)) <=> add(len(X), len(Y))
You can use these laws to do transformations like these:
add(X, Y)
concat[len](X, Y) // using a symmetric path to reach the other function
// there exists `Z` and `W` such that `X = len(Z)` and `Y = len(W)`
concat[len](len(Z), len(W))
len(concat(Z, W))
I want to create an algorithm to explore the space of path semantics efficiently. There is no currently proof technique I know to derive them, except to “make a hypothesis and falsify”, kind of like how scientists do. The only way to find which functions that works with others is by trying.
Here is a function that tests for a symmetric path candidate:
/// Infers whether a path can be used for a function
/// in a symmetric way for both arguments and result.
sym(f: \([]) -> any, p: \([]) -> any, args: []) =
all i {is_ok(try \p([args[i]]))} &&
is_ok(try \p([\f(args)]))
Let me take this apart an explain the pieces:
sym
is the function namef: \([]) -> any
is an argumentf
of type closure taking[]
(array) and returningany
p: \([]) -> any
is an argumentp
of type closure taking[]
(array) and returningany
args: []
is an argumentargs
of type[]
(array)\p([args[i]])
calls the closurep
\p([\f(args)])
calls the closuref
and thenp
all i { ... }
is a mathematical loop that automatically infers the range[0, len(args))
from the body
The \
notation is used in Dyon because type checking happens in parallel with AST construction,
plus some other design choices that closures always return a value.
I needed a design that was easy to refactor and different for normal functions and closures.
The problem was that this function gets called many times with different closures, and sometimes it fails and crashes because the types are wrong.
try
converts all failures into err
, such that the function sym
can detect a path candidate.
Now, you might think, why not use try and catch blocks?
Dyon already has a way of dealing with errors: res
, like Result
in Rust but with a trace and any
as error type.
There is no need to add a second way of dealing with errors, when the first method works just fine.
For Rust programmers, the try
keyword can be a bit confusing, since they might be thinking of the try!
macro.
In Dyon, you use the ?
operator instead of try!
, just like in Rust.
I decided to use it because it feels closer to what we mean with “trying” in natural language.
For example, when a function foo
returns res
, you can write:
fn foo() -> res { ... }
fn bar() -> res {
a := try foo()?
...
}
This is executed as try (foo()?)
, returning err
early,
so you can handle the case where it failed.
Interestingly, it gives the same result as (try foo())?
.