Zero-Shot NER in 100ms: Porting GLiNER2 to Rust with ONNX Runtime

Traditional named entity recognition locks you into predefined categories like
"person" or "organization" — retrain every time your domain changes.
GLiNER2 flips this: define custom entity types at inference time, no retraining
required.
The catch?
The reference implementation is Python-only, which makes embedding it in
production pipelines painful.

In this talk, I'll walk through porting GLiNER2 to a pure Rust inference library
— from exporting a multi-stage transformer pipeline to ONNX, to reimplementing
tokenization, span generation, and schema-conditioned decoding in idiomatic Rust.
We'll dig into the architecture of the model itself
(DeBERTa encoder, GRU count head, 4D span scoring tensors), the Rust-specific
challenges (ndarray gymnastics, ONNX Runtime bindings, faithful
floating-point reproduction), and the practical payoff:
sub-100ms structured extraction on CPU with zero Python in the loop.

Whether you're interested in how modern NLP models actually work under the hood,
how to wrangle ONNX from Rust, or how zero-shot entity recognition can replace
brittle regex pipelines in domains like legal, medical, and financial text
analysis — there should be something here for you.

– By Chris Raethke

Видео Zero-Shot NER in 100ms: Porting GLiNER2 to Rust with ONNX Runtime канала Rust Brisbane