3.10.0 (2025-06-12)

Features

• JSON Schema Grammar: $defs and $ref support with full inferred types (#472) (9cdbce9)
• inspect gguf command: format and print the Jinja chat template with --key .chatTemplate (#472) (9cdbce9)

Bug Fixes

• JinjaTemplateChatWrapper: first function call prefix detection (#472) (9cdbce9)
• QwenChatWrapper: improve Qwen chat template detection (#472) (9cdbce9)
• apply maxTokens on function calling parameters (#472) (9cdbce9)
• adjust default prompt completion length based on SWA size when relevant (#472) (9cdbce9)
• improve thought segmentation syntax extraction (#472) (9cdbce9)
• adapt to llama.cpp changes (#472) (9cdbce9)

Shipped with llama.cpp release b5640

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.9.0 (2025-06-04)

Features

• reasoning budget (#468) (ea8d904) (documentation: Set Reasoning Budget)
• SWA (Sliding Window Attention) support - greatly reduced context memory consumption on supported models (#468) (ea8d904)
• documentation: LLMs friendly llms.md and llms-full.md files (#468) (ea8d904)

Bug Fixes

• prompt completion edge cases (#468) (ea8d904)
• adapt to llama.cpp changes (#468) (ea8d904)

Shipped with llama.cpp release b5590

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.8.1 (2025-05-19)

Bug Fixes

• getLlamaGpuTypes: edge case (#463) (1799127)
• remove prompt completion from the cached context window (#463) (1799127)

Shipped with llama.cpp release b5415

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.8.0 (2025-05-17)

Features

• save and restore a context sequence state (#460) (f2cb873) (documentation: Saving and restoring a context sequence evaluation state)
• stream function call parameters (#460) (f2cb873) (documentation: API: LLamaChatPromptOptions["onFunctionCallParamsChunk"])
• configure Hugging Face remote endpoint for resolving URIs (#460) (f2cb873) (documentation: API: ResolveModelFileOptions["endpoints"])
• Qwen 3 support (#460) (f2cb873)
• QwenChatWrapper: support discouraging the generation of thoughts (#460) (f2cb873) (documentation: API: QwenChatWrapper constructor > thoughts option)
• getLlama: dryRun option (#460) (f2cb873) (documentation: API: LlamaOptions["dryRun"])
• getLlamaGpuTypes function (#460) (f2cb873) (documentation: API: getLlamaGpuTypes)

Bug Fixes

• adapt to breaking llama.cpp changes (#460) (f2cb873)
• capture multi-token segment separators (#460) (f2cb873)
• race condition when reading extremely long gguf metadata (#460) (f2cb873)
• adapt memory estimation to newly added model architectures (#460) (f2cb873)
• skip binary testing on certain problematic conditions (#460) (f2cb873)
• improve GPU backend loading error description (#460) (f2cb873)

Shipped with llama.cpp release b5414

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.7.0 (2025-03-28)

Features

• extract function calling syntax from a Jinja template (#444) (c070e81)
• Full support for Qwen and QwQ via QwenChatWrapper (#444) (c070e81)
• export a llama instance getter on a model instance (#444) (c070e81)

Bug Fixes

• better handling for function calling with empty parameters (#444) (c070e81)
• reranking edge case crash (#444) (c070e81)
• limit the context size by default in the node-typescript template (#444) (c070e81)
• adapt to breaking llama.cpp changes (#444) (c070e81)
• bump min nodejs version to 20 due to dependencies' requirements (#444) (c070e81)
• defineChatSessionFunction type (#444) (c070e81)

Shipped with llama.cpp release b4980

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

✨ DeepSeek R1 is here! ✨

Read about the release in the blog post

3.6.0 (2025-02-21)

Features

• DeepSeek R1 support (#428) (ca6b901) (documentation: DeepSeek R1)
• chain of thought segmentation (#428) (ca6b901) (documentation: Stream Response Segments)
• pass a model to resolveChatWrapper (#428) (ca6b901)
• defineChatSessionFunction: improve params type (#428) (ca6b901)
• Electron template: show chain of thought (#428) (ca6b901) (documentation: DeepSeek R1)
• Electron template: add functions template (#428) (ca6b901)
• Electron template: new icon for the CI build (#428) (ca6b901)
• Electron template: update model message in a more stable manner (#428) (ca6b901)
• Electron template: more convenient completion (#428) (ca6b901)

Bug Fixes

• check path existence before reading its content (#428) (ca6b901)
• partial tokens handling (#428) (ca6b901)
• uncaught exception (#430) (599a161)
• Electron template: non-latin text formatting (#430) (599a161)

Shipped with llama.cpp release b4753

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.5.0 (2025-01-31)

Features

• shorter model URIs (#421) (73454d9) (documentation: Model URIs)

Bug Fixes

• add missing Jinja features for DeepSeek (#425) (6e4bf3d)

Shipped with llama.cpp release b4600

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.4.3 (2025-01-30)

Bug Fixes

• adapt to llama.cpp breaking changes (#424) (6e4bf3d)

Shipped with llama.cpp release b4599

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.4.2 (2025-01-27)

Bug Fixes

• metadata string encoding (#420) (314d7e8)
• Vulkan parallel decoding (#420) (314d7e8)
• try auth token on 401 response (#420) (314d7e8)

Shipped with llama.cpp release b4567

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)

3.4.1 (2025-01-23)

Bug Fixes

• adapt to breaking llama.cpp changes (#415) (86e1bee)
• ranking empty inputs (#415) (86e1bee)

Shipped with llama.cpp release b4529

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)