Back-End Agnostic Design

This blog post is about the general architecture of Piston. I will try to give an overview of the things people have some trouble understanding, based on things I have observed and stuff people have requested. Please come talk to me (bvssvni) on #rust-gamedev (IRC) if you want more information, or if you have ideas of how this can be represented more clearly. There is a subreddit for Rust gamedev here with link to the chat in the sidebar.

Naming

Piston uses back-end agnostic design. Back-ends for “graphics” ends with _graphics and back-ends for “window” ends with _window. For example:

Motivation

The Piston project uses back-end agnostic design because:

  • More choices when shipping a product
  • Better for sharing source code
  • Quickly fix problems by swapping a back-end and see if that works
  • Easier to compare, debug and benchmark different design
  • Composable with both cross and native platform programming

Overview

To get started quickly with Piston, you can use:

These are the main abstractions for window/events/graphics:

  • piston (core)
  • graphics (2D graphics)
  • gfx (3D graphics, developed under gfx-rs organization)

The Piston project has many small libraries and projects running in parallel. For a full list, see https://github.com/pistondevelopers/.

Window

There is a Window trait for minimal features required for a game loop, and there is AdvancedWindow which is intended for generic code. This is provided by the pistoncore-window library. Here is the Window trait:

/// Required to use the event loop.
pub trait Window {
    /// The event type emitted by `poll_event`
    type Event;

    /// Returns true if window should close.
    fn should_close(&self) -> bool;

    /// Gets the size of the window in user coordinates.
    fn size(&self) -> Size;

    /// Swaps render buffers.
    fn swap_buffers(&mut self);

    /// Polls event from window.
    fn poll_event(&mut self) -> Option<Self::Event>;

    /// Gets draw size of the window.
    /// This is equal to the size of the frame buffer of the inner window,
    /// excluding the title bar and borders.
    fn draw_size(&self) -> Size;
}

Events

In Piston, events are important to make libraries work together.

Events affects maintenance because of the way Rust’s type system works. In generic libraries, when you have a function foo calling bar, then bar will propagate constraints to foo.

fn foo<T: Bar>(b: T) { bar(b); }
fn bar<T: Bar>(b: T) { ... }

So when you add a new constraint to bar, you need to update foo:

fn foo<T: Bar>(b: T) { bar(b); } // <- error: the trait `Baz` is not implemented for the type `T`
fn bar<T: Bar + Baz>(b: T) { ... }

This problem scales with the size of the project. To solve this problem in Piston, we use the GenericEvent trait, provided by pistoncore-event:

/// Implemented by all events
pub trait GenericEvent {
    /// The id of this event.
    fn event_id(&self) -> EventId;
    /// Calls closure with arguments
    fn with_args<'a, F, U>(&'a self, f: F) -> U
        where F: FnMut(&Any) -> U
    ;
    /// Converts from arguments to `Self`
    fn from_args(event_id: EventId, any: &Any, old_event: &Self) -> Option<Self>;
}

Normally, you do not use the GenericEvent trait directly, but some other trait built on top of it. For example, UpdateEvent is implemented for all types implementing GenericEvent. This means you only add the constraint GenericEvent, everywhere:

// From the FirstPerson controller.
pub fn event<E>(&mut self, e: &E) where E: GenericEvent {
    use piston::event::{ MouseRelativeEvent, PressEvent, ReleaseEvent, UpdateEvent };
    ...
}

Events are not directly tied to the window abstraction, because a common form of application logic is to transform the events. Since generic libraries uses the GenericEvent trait, it makes them easier to use when a higher level library performs the event transformation. One such example is AI behavior tree that might change the update event to a shorter delta time according to the event logic.

2D Graphics

The project that started Piston was a back-end agnostic 2D graphics library. It meant that people could share 2D code across projects in the Rust community. It is completely decoupled from the window and event abstraction.

Here is the Graphics trait:

/// Implemented by all graphics back-ends.
pub trait Graphics {
    /// The texture type associated with the back-end.
    type Texture: ImageSize;

    /// Clears background with a color.
    fn clear_color(&mut self, color: [f32; 4]);

    /// Clears stencil buffer with a value.
    fn clear_stencil(&mut self, value: u8);

    /// Renders list of 2d triangles.
    fn tri_list<F>(&mut self, draw_state: &DrawState, color: &[f32; 4], f: F)
        where F: FnMut(&mut FnMut(&[f32]));

    /// Renders list of 2d triangles.
    ///
    /// A texture coordinate is assigned per vertex.
    /// The texture coordinates refers to the current texture.
    fn tri_list_uv<F>(
        &mut self,
        draw_state: &DrawState,
        color: &[f32; 4],
        texture: &<Self as Graphics>::Texture,
        f: F
    ) where F: FnMut(&mut FnMut(&[f32], &[f32]));
}

The DrawState is the same one that Gfx uses, such that there is no overhead and exposes fixed hardware pipelines features to higher level APIs. This means that Piston can use the stencil buffer directly for things like clipping or simple blending effects.

The closure takes a closure, so the same state can be reused by the back-end between each call.

Here is how an image is rendered (taken from the graphics library):

/// Draws the image.
pub fn draw<G>(
    &self,
    texture: &<G as Graphics>::Texture,
    draw_state: &DrawState,
    transform: Matrix2d,
    g: &mut G
)
    where G: Graphics
{
    use math::Scalar;

    let color = self.color.unwrap_or([1.0; 4]);
    let source_rectangle = self.source_rectangle.unwrap_or({
        let (w, h) = texture.get_size();
        [0, 0, w as i32, h as i32]
    });
    let rectangle = self.rectangle.unwrap_or([
        0.0,
        0.0,
        source_rectangle[2] as Scalar,
        source_rectangle[3] as Scalar
    ]);
    g.tri_list_uv(
        draw_state,
        &color,
        texture,
        |f| f(
            &triangulation::rect_tri_list_xy(transform, rectangle),
            &triangulation::rect_tri_list_uv(texture, source_rectangle)
        )
    );
}

Alternative designs are being considered, for example performing triangulation in the buffer of the back-end.

Benchmarking

The event loop in Piston supports bench mark mode out of the box, which is one of the few reliable ways to test the overall performance of a game engine.

When enabled, it will render and update without sleep and ignore input.

for e in window.events().bench_mode(true) {
    ...
}

One of the benefits with back-end agnostic design, is that benchmark mode makes it easy to compare various APIs.

Why use Piston?

Rust makes it easier and safer to program games, but there is more to game development than picking a programming language.

Piston is a project with a goal of building an game engine, and this results in lot of interesting research projects and libraries that benefit the whole Rust community. By providing a modular and open architecture, we hope you want to participate.

The people working on the Piston project have different goals, but share the cost of maintaining important libraries together. Everyone gets write access to all the libraries, but we use PRs to make it easier to follow the changes.

How to contribute

Meta Language

“How do you feel today Piston-Meta 0.6.6?”

“Good! Like I can parse everything!”

Piston-Meta is new Domain Specific Language (DSL) parsing library (written in Rust) for human readable formats. It gives nice error messages, more higher level than regular expressions, easy to use, and reads JSON strings. When combining complicated rules containing new lines, but separated by new lines, it “just works”. It is not indention sensitive, as it is designed for data and configuration files typical in game development. Transformation happens to a flat tree structure stored in a Vec, and uses reference counting on the strings that are copied from the rules. You get fewer allocations with better data validation. Rules are dynamic and can be changed at runtime, designed for systems that never need to shut down for maintenance.

Since an example can say more than thousand words, here is “hello world” in Piston-Meta:

extern crate piston_meta;

use piston_meta::*;

fn main() {
    let text = r#"say "Hello world!""#;
    let rules = r#"1 "rule" ["say" w! t?"foo"]"#;
    // Parse rules with meta language and convert to rules for parsing text.
    let rules = bootstrap::convert(
        &parse(&bootstrap::rules(), rules).unwrap(),
        &mut vec![] // stores ignored meta data
    ).unwrap();
    let data = parse(&rules, text);
    match data {
        Ok(data) => {
            assert_eq!(data.len(), 1);
            if let &MetaData::String(_, ref hello) = &data[0].1 {
                println!("{}", hello);
            }
        }
        Err((range, err)) => {
            // Report the error to standard error output.
            ParseStdErr::new(&text).error(range, err);
        }
    }
}

When you run this, you should get the following text on standard output:

Hello world!

Try changing the text to don't say "Hello world!", and you get an error:

Error #1001, Expected: `say`
1: don't say "Hello world!"
1: ^

Try changing the text to say quickly "Hello world!", and you get an error:

Error #1003, Expected text
1: say quickly "Hello world!"
1:     ^

Try changing the text to say "Hello world!" 3 times, and you get an error:

Error Expected end
1: say "Hello world!" 3 times
1:                   ^

The rule we used to parse say "Hello world!" in the “hello world” example consisted of one line:

1 "rule" ["say" w! t?"foo"]

1 is a number that gets multiplied with 1000 and used as seed for debug_id in the rule. For example Error #1003, Expected text tells you that the rule causing parsing to fail is located in 1. This is useful when you are developing rules for a new text format.

After the number 1, we have "rule" which is used internally to name rules. Rules can reference other rules. If we wanted to reference "rule" from another rule, we could use @"rule" to use it.

The square brackets [] is a sequence rule, which processes each sub rule from left to right.

  1. A token rule "say", failing if there is no say in the text
  2. A required whitespace rule w!
  3. A text rule t?"foo", which allow an empty string and gives it a name "foo"

Some tiny examples

1 "numbers" l($"num"):

1001
2002
3003

1 "number" l($_"num"):

1_001
2_002
3_003

1 "laughter" r?("ha"):

hahahahahaha

1 "forced laughter" r!("ha"):

Error #1001, Expected: `ha`
1: (:-|)
1: ^

1 "evil laughter" ["mua" r!("ha")]:

muahahahaha

1 "lines of two kinds of laughter" l({r?("ha") r?("hi")}):

hihihihihihihihihi
hahahaha
hihihi
hi
hahahahahahahahahaha


hi
ha
1 "farm animals separated by commas" s?(["," w?]){
  {"cow" "pig" "hen" "sheep"}
}
sheep, cow, pig, hen,hen,hen, cow, cow, pig

The s?(..){..} rule means separat {..} by (..). ? means it can occur zero times.

1 "numbers separated by commas, allow trailing comma but no whitespace" [s?.(","){$}]:

1,2,3,4,

If you write s?. or s!. the separation rule is allowed at the end.

1 "can't we all be friends?" l(["can't we all be friends?" w! {"yes""said_yes" "no"!"said_yes"}]):

can't we all be friends? no
can't we all be friends? yes

The first line sets said_yes to false and the second line sets said_yes to true.

1 "you can say you love me, but you don't have to" ?"I love you":

``

A rule that starts with ? is optional.

How does it work?

The rules are used to parse a text document, and you get a list of tokens or an error. Here is the signature of the parse function:

pub fn parse(
    rules: &[(Rc<String>, Rule)],
    text: &str
) -> Result<Vec<(Range, MetaData)>, (Range, ParseError)>

The Range tells which characters the data was read from. MetaData is an enum defined as:

pub enum MetaData {
    /// Starts node.
    StartNode(Rc<String>),
    /// Ends node.
    EndNode(Rc<String>),
    /// Sets bool property.
    Bool(Rc<String>, bool),
    /// Sets f64 property.
    F64(Rc<String>, f64),
    /// Sets string property.
    String(Rc<String>, Rc<String>),
}

All you have to do in order to use it with your own library/application, is to convert from MetaData to the structure you want in Rust. Look in convert.rs for example code.

(We probably will make this easier at some point)

Self syntax

The meta language for Piston-Meta can be bootstrapped using its own rules.

opt: "optional"
inv: "inverted"
prop: "property"
any: "any_characters"
seps: "[]{}():.!?\""
1 "string" [..seps!"name" ":" w? t?"text"]
2 "node" [$"id" w! t!"name" w! @"rule""rule"]
3 "set" {t!"value" ..seps!"ref"}
4 "opt" {"?"opt "!"!opt}
5 "number" ["$" ?"_""underscore" ?@"set"prop]
6 "text" ["t" {"?""allow_empty" "!"!"allow_empty"} ?@"set"prop]
7 "reference" ["@" t!"name" ?@"set"prop]
8 "sequence" ["[" w? s!.(w!) {@"rule""rule"} "]"]
9 "select" ["{" w? s!.(w!) {@"rule""rule"} "}"]
10 "separated_by" ["s" @"opt" ?".""allow_trail"
  "(" w? @"rule""by" w? ")" w? "{" w? @"rule""rule" w? "}"]
11 "token" [@"set""text" ?[?"!"inv @"set"prop]]
12 "optional" ["?" @"rule""rule"]
13 "whitespace" ["w" @"opt"]
14 "until_any_or_whitespace" [".." @"set"any @"opt" ?@"set"prop]
15 "until_any" ["..." @"set"any @"opt" ?@"set"prop]
16 "repeat" ["r" @"opt" "(" @"rule""rule" ")"]
17 "lines" ["l(" w? @"rule""rule" w? ")"]
18 "rule" {
  @"whitespace""whitespace"
  @"until_any_or_whitespace""until_any_or_whitespace"
  @"until_any""until_any"
  @"lines""lines"
  @"repeat""repeat"
  @"number""number"
  @"text""text"
  @"reference""reference"
  @"sequence""sequence"
  @"select""select"
  @"separated_by""separated_by"
  @"token""token"
  @"optional""optional"
}
19 "document" [l(@"string""string") l(@"node""node") w?]

This is how you check rules when by bootstrapping them twice and compare them:

extern crate piston_meta;

use piston_meta::parse;
use piston_meta::bootstrap;
use std::fs::File;
use std::io::Read;
use std::path::PathBuf;

fn main() {
    let rules = bootstrap::rules();
    let self_syntax: PathBuf = "assets/self-syntax.txt".into();
    let mut file_h = File::open(self_syntax).unwrap();
    let mut source = String::new();
    file_h.read_to_string(&mut source).unwrap();
    let res = parse(&rules, &source).unwrap();
    let mut ignored1 = vec![];
    let rules1 = bootstrap::convert(&res, &mut ignored1).unwrap();
    println!("ignored1 {:?}", ignored1.len());
    let res = parse(&rules1, &source).unwrap();
    let mut ignored2 = vec![];
    let rules2 = bootstrap::convert(&res, &mut ignored2).unwrap();
    println!("ignored2 {:?}", ignored2.len());
    let _ = parse(&rules2, &source).unwrap();
    assert_eq!(rules1, rules2);
    println!("Bootstrapping succeeded!");
}

If don’t like the meta language, you can change the self syntax, print out the rules with println!("{:?}", rules), fix it so it compiles in Rust, and then use the bootstrap test on your new meta language.

Piston-Window 0.2

Piston-Window is a convenience window wrapper for the Piston game engine. It is completely independent of the piston core and the other libraries, so no Piston library requires you to use Piston-Window.

Piston-Window is designed for only one purpose: Convenience.

We have now released 0.2!

  • Reexports everything you need to write 2D interactive applications
  • .draw_2d for drawing 2D, and .draw_3d for drawing 3D
  • Uses Gfx to work with 3D libraries in the Piston ecosystem
  • A smart design to pass around the window and application state
extern crate piston_window;
use piston_window::*;
fn main() {
    let window: PistonWindow = WindowSettings::new("Hello Piston!", [640, 480])
        .exit_on_esc(true).into();
    for e in window {
        e.draw_2d(|_c, g| {
            clear([0.5, 1.0, 0.5, 1.0], g);
        });
    }
}

If you want another convenience method, create a trait for it, and then implement it for PistonWindow.

PistonWindow uses Glutin as window back-end by default, but you can change to another back-end, for example SDL2 or GLFW by changing the type parameter:

let window: PistonWindow<Sdl2Window> = WindowSettings::new("Hello Piston!", [640, 480])
    .exit_on_esc(true).into();

Games often follow a finite state machine logic. A common way to solve this is using an enum and a loop for the different states. This can be quite buggy, since you need to resolve the state for each event.

Instead, you could pass around one PistonWindow to different functions that represents the states. This way you do not have to resolve the state, because it is part of the context.

PistonWindow implements AdvancedWindow, Iterator, Window and GenericEvent. The iterator emits new objects of same type that wraps the event from the game loop. You can swap the application state with .app method. Nested game loops are supported, so you can have one inside another.

for e in window {
    if let Some(button) = e.press_args() {
        let intro = e.app(start_intro());
        for e in intro {
            ...
        }
    }
}

Ideas or feedback? Open up an issue here.

Older Newer