Wrappers for built-in Web Workers enabling easy parallel data processing.
- π CSP-friendly: no evals, static file name
- π Tested in browsers, node, deno, bun
- π¦ Can be bundled using esbuild, rollup, webpack, parcel
- π High-level type-safe helpers for batch processing
- β Sync: much simpler than async, no queues / locks
Browser Web Workers work fine, but have terrible APIs (just like most "web APIs"). Node.js doesn't have workers, while polyfilling them using node APIs breaks bundlers.
How could one pass a code to a worker?
- eval: stringify function, then
eval
. Would break CSP and imports - wasm: much easier, just send binary blob of code. Would not work in envs without wasm
- re-run module with if-workercode-else-maincode: fragile, need to track everything done before workers are initialized (IO such as HTTP, DOM)
- build static file before publishing: works if wrkr is directly used, but not inside of other library
Check out webpack docs on webworkers.
The library could also be used in single-threaded manner: provide threads
option to initBatch
.
Then slow functions can be ran outside of main thread, with async API.
npm install micro-wrkr
deno add jsr:@paulmillr/micro-wrkr
deno doc jsr:@paulmillr/micro-wrkr
# command-line documentation
import { bn254 } from '@noble/curves/bn254';
import { type ProjConstructor, type ProjPointType } from '@noble/curves/abstract/weierstrass';
import wrkr from 'micro-wrkr';
import { type Handlers } from './msm-worker.js';
function reducePoint<T>(p: ProjConstructor<T>) {
return (lst: ProjPointType<T>[]) =>
lst.map((i) => new p(i.px, i.py, i.pz)).reduce((acc, i) => acc.add(i), p.ZERO);
}
export function initMSM() {
// Type-safe
// worker should be in same directory as main thread code
const { methods, terminate } = wrkr.initBatch<Handlers>(
() => new Worker(new URL('./msm-worker.js', import.meta.url), { type: 'module' }),
{
// optional reducers
bn254_msmG1: reducePoint(bn254.G1.ProjectivePoint),
bn254_msmG2: reducePoint(bn254.G2.ProjectivePoint),
}
);
// Use `terminate` to stop workers when app is paused or exported from library.
// Otherwise, it won't terminate.
return { methods, terminate };
}
import { bn254 } from '@noble/curves/bn254';
import wrkr from 'micro-wrkr';
import { type ProjConstructor, type ProjPointType } from '@noble/curves/abstract/weierstrass';
type MSMInput<T> = { point: ProjPointType<T>; scalar: T };
function buildMSM<T>(point: ProjConstructor<T>) {
return (lst: MSMInput<T>[]): ProjPointType<T> => {
if (!lst.length) return point.ZERO;
const points = lst.map((i: any) => new point(i.point.px, i.point.py, i.point.pz));
const scalars = lst.map((i: any) => i.scalar);
return point.msm(points, scalars);
};
}
const handlers = {
bn254_msmG1: buildMSM(bn254.G1.ProjectivePoint),
bn254_msmG2: buildMSM(bn254.G2.ProjectivePoint),
};
// Export Handlers type for type-safety
export type Handlers = typeof handlers;
wrkr.initWorker(handlers);
- Browserify isn't supported
- Webpack sometimes breaks CSP by encoding workers as data:url
- Example:
new Worker(new URL(e.p+e.u(44),e.b),{type:void 0})
- Example:
# when no google chrome, thorium can also be used
export CHROME_BIN='/Applications/Thorium.app/Contents/MacOS/Thorium'
npm run build && npm run test:full
MIT (c) Paul Miller (https://paulmillr.com), see LICENSE file.