TabbyAPI primarily uses a config.yml file to adjust various options. This is the preferred way and has the ability to adjust all options of TabbyAPI.

CLI arguments are also included, but those serve to override the options set in config.yml. Therefore, they act a bit differently compared to other programs, especially with booleans. Example: A user sets gpu_split_auto to True. The CLI arg will then be --gpu_split-auto False to override that previous config.yml setting.

In addition, some config.yml options are too complex to represent as command args, so those are not included with the argparser.

All of these options have descriptive comments above them. You should not need to reference this documentation page unless absolutely necessary.

Note: This block is for sampling overrides, not samplers themselves.

Note: These are experimental flags that may be removed at any point.

Note: Most of the options here will only apply on initial model load/startup (ephemeral). They will not persist unless you add the option name to use_as_default.

Note: Sub-block of Model Options. Same rules apply.

Note: Sub-block of Mode Options. Same rules apply.

Note: Most of the options here will only apply on initial embedding model load/startup (ephemeral).