Product Documentation

Appearance

All platforms

Creating WASM UDFs ​

This guide details the steps required to build WASM modules from C-based math functions and execute them in SQL queries using the WASM_RUNNER function.

You will need the following packages installed locally to be able to compile C code to WASM bytecode:

Compiling a C Function to WASM ​

Consider this simple math function. This function computes the sum of all i * j products for i and j from 0 to n-1, where n is the integer part of x, yielding control once per outer loop iteration.

c
// stub so that native build and wasm imports both link
void yb_yield(void);

float sum_ij(float x) {
  int n = (int)x;
  float acc = 0;
  for (int i = 0; i < n; i++) {
    yb_yield();
    for (int j = 0; j < n; j++) {
      acc += (float)(i * j);
    }
  }
  return acc;
}

WARNING

It is important to regularly call yb_yield() if your math function contains loops that are long-running. yb_yield() indicates to the Yellowbrick WASM runtime that the function should yield execution to other waiting queries. This function is implemented in the WASM runtime.

To compile to WASM bytecode we use the version of clang provided in the WASI SDK:

bash
./wasi-sdk-25.0-x86_64-linux/bin/clang \
--target=wasm32-wasi \
-O3 \
-nostartfiles \
-Wl,--no-entry \
-Wl,--allow-undefined \
-Wl,--strip-all \
-Wl,--gc-sections \
-Wl,--export=sum_ij \
-o sum_ij.wasm \
sum_ij.c \
-lm

WARNING

The --export= value must be the name of the C math function. The exported function should not be declared as static.

Memory Allocations ​

WASM modules expect to operate inside a block of linear memory. The largest block of contiguous memory that can be allocated in a compute node is 2 MiB. 128 KiB of this block is reserved for stack, which means that any heap memory allocations within a function need to be less than 1,920 KiB in total per WASM_RUNNER function call.

Use calloc, realloc and free in a math function if heap memory is required. The Yellowbrick WASM runtime overrides these functions to provide access to the contiguous block of memory.

Example with Memory Allocation ​

c
#include <stdlib.h>

// stub so that native build and wasm imports both link
void yb_yield(void);

int heap_sum(int x) {
  int n = (int)x;
  int* arr = (int*)calloc(n, sizeof(int));
  if (!arr) return -1;

  for (int i = 0; i < n; i++) {
    arr[i] = (int)i;
  }

  yb_yield();

  int acc = 0;
  for (int i = 0; i < n; i++) {
    acc += arr[i];
  }

  free(arr);
  return acc;
}

Using the WASM Function ​

The WASM_RUNNER function is used to execute the function in a SQL query. It takes a base64-encoded version of the WASM bytecode compiled earlier. Convert the .wasm file created earlier to base64:

bash
base64 -w0 sum_ij.wasm > sum_ij.wasm.b64

Add the contents of the .b64 file as the first argument to WASM_RUNNER:

sql
SELECT
    WASM_RUNNER('AGFzbQEAAAABCQJgAABgAXwBfAIQAQNlbnYIeWJfeWllbGQAAAMFBAEAAAEEBQFwAQEBBQMBAAIGCAF/AUGAiAQLBxMCBm1lbW9yeQIABnN1bV9pagAECrUCBJcCAQp/AkACQCAAmUQAAAAAAADgQWNFDQAgAKohAQwBC0GAgICAeCEBCwJAIAFBAU4NAEQAAAAAAAAAAA8LIAFB/P///wdxIQIgAUEDcSEDRAAAAAAAAAAAIQBBACEEIAFBBEkhBUEAIQZBACEHQQAhCANAEICAgIAAAkACQCAFRQ0AQQAhCQwBC0EAIQpBACEJA0AgACAKt6AgCCAKaregIAQgCmq3oCAHIApqt6AhACAKIAZqIQogAiAJQQRqIglHDQALCwJAIANFDQAgCSAIbCEKIAMhCQNAIAAgCregIQAgCiAIaiEKIAlBf2oiCQ0ACwsgBEECaiEEIAZBBGohBiAHQQNqIQcgCEEBaiIIIAFHDQALIAALAgALDgAQgoCAgAAQgoCAgAALCAAgABABEAML',
    'sum_ij',
    3);

This query produces:

 wasm_runner
-------------
           9
(1 row)

The second argument should be the name of the function as it appears in the native code. WASM_RUNNER expects at least one additional argument, beyond the WASM base64 string and the function name.

Wrap a Custom Math Function as a SQL UDF ​

The mechanism for creating custom SQL UDFs can be used to wrap the call to WASM_RUNNER in a more user-friendly way. For example:

sql
SET ybd_allow_udf_creation TO TRUE;

CREATE OR REPLACE FUNCTION sumij( _i1 FLOAT8 )
RETURNS FLOAT8
LANGUAGE SQL
AS '
   SELECT
   WASM_RUNNER(''AGFzbQEAAAABCQJgAABgAXwBfAIQAQNlbnYIeWJfeWllbGQAAAMFBAEAAAEEBQFwAQEBBQMBAAIGCAF/AUGAiAQLBxMCBm1lbW9yeQIABnN1bV9pagAECrUCBJcCAQp/AkACQCAAmUQAAAAAAADgQWNFDQAgAKohAQwBC0GAgICAeCEBCwJAIAFBAU4NAEQAAAAAAAAAAA8LIAFB/P///wdxIQIgAUEDcSEDRAAAAAAAAAAAIQBBACEEIAFBBEkhBUEAIQZBACEHQQAhCANAEICAgIAAAkACQCAFRQ0AQQAhCQwBC0EAIQpBACEJA0AgACAKt6AgCCAKaregIAQgCmq3oCAHIApqt6AhACAKIAZqIQogAiAJQQRqIglHDQALCwJAIANFDQAgCSAIbCEKIAMhCQNAIAAgCregIQAgCiAIaiEKIAlBf2oiCQ0ACwsgBEECaiEEIAZBBGohBiAHQQNqIQcgCEEBaiIIIAFHDQALIAALAgALDgAQgoCAgAAQgoCAgAALCAAgABABEAML'',
    ''sum_ij'',
    _i1)
';

Note the use of escaping on the string literals inside the WASM_RUNNER call. Running the custom UDF results in:

SELECT sumij(3.0);
 sumij
-------
     9
(1 row)