Performance
This page compares WaaS with other scripting engines and WebAssembly runtimes. The results can be reproduced with WaaS.Benchmarks.
Interpreter backends
| Backend | Used when |
|---|---|
| Burst | Unity with com.unity.burst installed, on the platforms that Burst compiles for |
| Managed | Otherwise: .NET, and Unity without Burst |
No change of code is needed to use the Burst backend. A debugger uses a slower interpreter that executes each instruction separately.
Without Burst, IL2CPP gains less from the optimizations of the interpreter than CoreCLR does. If performance matters on IL2CPP, install com.unity.burst.
Conditions
Windows 11, Intel Core i7-13700F. Unity 6000.5.8f1 with non-development IL2CPP players, .NET 10, xLua (59bf426, Lua 5.3), Lua-CSharp 0.5.8, wasmtime 50.0.1 and wasm3 (28ecb9a, built with clang). WaaS is built from the commit that added these results. Each value is the lowest of the medians of four alternating runs.
The same workloads are implemented in Rust, MoonBit and Lua, and the harnesses verify that all of them compute the same checksums. wasm3 and wasmtime run outside of .NET, so only the compute workloads are measured on them.
| Workload | Content |
|---|---|
| fib | Recursive Fibonacci number of 30 |
| sieve | Sieve of Eratosthenes up to 1,000,000 |
| mandelbrot | Mandelbrot set on a grid of 200 × 200 |
| strings | Formats, splits and hashes 20,000 strings |
| sort | Sorts 200,000 integers |
| map | Inserts 500,000 times into a hash map |
WebAssembly runtimes
Compute workloads (ms, lower is better). wasmtime is a JIT compiler and is shown for reference, and wasm3 is an interpreter.
The time of the guest language differs by workload.
| Workload | WaaS CoreCLR (Rust) | WaaS CoreCLR (MoonBit) | WaaS Burst (Rust) | WaaS Burst (MoonBit) | wasm3 (Rust) | wasm3 (MoonBit) | wasmtime (Rust) | wasmtime (MoonBit) |
|---|---|---|---|---|---|---|---|---|
| fib | 34.1 | 65.4 | 19.7 | 44.6 | 17.9 | 26.4 | 2.0 | 3.8 |
| sieve | 40.2 | 176.8 | 42.0 | 111.0 | 19.0 | 78.0 | 2.3 | 5.1 |
| mandelbrot | 16.5 | 36.2 | 16.4 | 38.1 | 7.3 | 15.5 | 1.7 | 2.2 |
| strings | 40.8 | 518.7 | 60.9 | 524.4 | 16.3 | n/a | 2.2 | 16.9 |
| sort | 64.2 | 357.1 | 56.4 | 309.1 | 23.9 | 144.5 | 4.6 | 19.8 |
| map | 81.2 | 613.7 | 79.4 | 417.0 | 36.7 | 232.3 | 4.4 | 20.0 |
n/a: wasm3 cannot compile a function of this module.
Lua
Compute workloads (ms, lower is better).
| Workload | xLua | Lua-CSharp |
|---|---|---|
| fib | 49.3 | 393.3 |
| sieve | 33.4 | 230.4 |
| mandelbrot | 32.6 | 182.0 |
| strings | 24.8 | 181.3 |
| sort | 50.2 | 154.5 |
| map | 8.8 | 70.1 |
Calls between scripts and the host
The values are in ns/call, lower is better. ffi_* are calls from scripts to the host, and callback_* are calls from the host to scripts.
| Workload | WaaS CoreCLR (Rust) | WaaS CoreCLR (MoonBit) | WaaS Burst (Rust) | WaaS Burst (MoonBit) | xLua | Lua-CSharp |
|---|---|---|---|---|---|---|
| ffi_nop | 40 | 46 | 58 | 70 | 27 | 207 |
| ffi_add | 51 | 44 | 69 | 83 | 40 | 413 |
| ffi_string_in | 76 | 251 | 404 | 1,008 | 114 | 304 |
| ffi_string_out | 307 | 2,349 | 805 | 3,139 | 67 | 259 |
| ffi_struct | 115 | 1,646 | 374 | 1,694 | 94 | 362 |
| ffi_resource | 154 | 85 | 333 | 347 | 31 | 318 |
| callback_add | 128 | 95 | 469 | 466 | 30 | 373 |
| callback_string | 337 | 1,436 | 967 | 2,837 | 68 | 361 |
- MoonBit uses UTF-16 strings, which are not covered by the direct path of the bindings for UTF-8, and its bindings allocate memory on the guest side.
- Strings and records take longer than numbers, because they are stored in the memory of the guest.
Load
The values are in ms, lower is better.
| Engine | Engine init | Load | Instantiate | First call | Total |
|---|---|---|---|---|---|
| WaaS, first in a process | 0.000 | 12.003 | 2.627 | 0.893 | 15.524 |
| WaaS (MoonBit), first in a process | 0.000 | 11.797 | 2.714 | 0.097 | 14.608 |
| xLua, first in a process | 6.551 | 0.098 | 0.799 | 0.084 | 7.533 |
| xLua (bytecode), first in a process | 6.501 | 0.021 | 0.804 | 0.083 | 7.409 |
| Lua-CSharp, first in a process | 0.782 | 1.833 | 0.246 | 0.012 | 2.873 |
| WaaS, repeated | 0.000 | 6.249 | 0.588 | 0.034 | 6.871 |
| WaaS (MoonBit), repeated | 0.000 | 5.018 | 0.267 | 0.005 | 5.290 |
| xLua, repeated | 0.841 | 0.090 | 0.724 | 0.005 | 1.661 |
| xLua (bytecode), repeated | 0.940 | 0.018 | 0.713 | 0.003 | 1.674 |
| Lua-CSharp, repeated | 0.030 | 0.321 | 0.017 | 0.002 | 0.370 |
| WaaS CoreCLR (Rust), repeated | - | - | - | - | 9.724 |
| WaaS CoreCLR (MoonBit), repeated | - | - | - | - | 5.070 |
| wasm3 (Rust), repeated | - | - | - | - | 0.014 |
| wasm3 (MoonBit), repeated | - | - | - | - | 0.026 |
| wasmtime (Rust), repeated | - | - | - | - | 23.618 |
| wasmtime (MoonBit), repeated | - | - | - | - | 20.037 |
- Loading a component validates and translates all of its functions, and the time grows with the size of the component.
- wasm3 compiles functions when they are first called, which is not included in its values.
- The first load on IL2CPP includes the initialization of the runtime.
Reading the results
- Numbers are compared on a desktop x64 CPU. Other platforms have not been measured.
- The results depend on the code that a compiler generates for a guest. For example,
mapruns the hash function of the guest in the interpreter, while Lua uses a table implemented in C. - Performance is one of the factors in choosing an engine. WaaS also offers sandboxing and a choice of guest languages, and does not need JIT compilation.