Changelog

• 6299c1b Fix High CPU (#189)

Changelog

• a906cd4 Strip comments before macro expansion in config (#193)

Changelog

• 78b2bc3 add toggle to hide/show unlisted models (#187)

Changelog

• 6a058e4 Change fsnotify to watch config directory instead of file

Changelog

• 1921e57 Add Event Bus (#184)

Changelog

• c867a6c Add name and description to v1/models list (#179)
• 3bd1b23 fix config hot-reload on k8s (#181)

Changelog

• 10606ab fix config hot-reload on macos (#180)

Changelog

• fefd149 improve log display and add a small stats table in ui (#178)
• 717d64e update GUI image in README [skip ci]

Worth writing a bit more about this release and #174.

In this release llama-swap model's gain the ability to prevent clients from changing server default sampling parameters like temperature, top_p, top_k, etc in requests.

This is done through a new filter configuration on models, it looks like this:

models: 
  model1: 
    cmd:  ... 
    # filters: a dictionary of filter settings
    # - optional, default: empty dictionary
    filters:
      # strip_params: a comma separated list of parameters to remove from the request
      # - optional, default: ""
      # - useful for preventing overriding of default server params by requests
      # - `model` parameter is never removed
      # - can be any JSON key in the request body
      # - recommended to stick to sampling parameters
      strip_params: "temperature, top_p, top_k"

Changelog

• 285191e Various UI improvements (#176)
• 4236cec Add Filters to Model Configuration (#174)

Changelog

• 756193d Load models in the UI without navigating the page (#173)
• a6b2e93 Update README.md [skip ci]