feat(llm): unify OpenAI-compatible providers behind a registry + generic endpoint
The OpenAI-compatible family (openai, xAI, DeepSeek, Qwen, GLM, MiniMax, OpenRouter, Ollama) all speak the same Chat Completions API and differ only by base_url, key, and two narrow wire-format quirks already isolated in subclasses. Replace the scattered base-URL dict, key handling, and client-class branches with one ProviderSpec registry that get_llm and the factory drive off; provider quirks stay in their subclasses. Add a generic "openai_compatible" provider for any OpenAI-compatible server (vLLM, LM Studio, llama.cpp, relays) via backend_url + optional key — adding a provider is now one registry row. Native Anthropic/Google keep their own clients (genuinely different APIs). Also fixes the env backend URL being ignored when the provider was chosen interactively (#978).
This commit is contained in:
11 files changed
+301
-85
No files matched your search
@@ -3,21 +3,25 @@
|
||||
from .model_catalog import get_known_models
|
||||
|
||||
|
||||
# Providers whose model names are user-defined (local servers, relays, any
|
||||
# OpenAI-compatible endpoint), so any model string is accepted without warning.
|
||||
_ANY_MODEL_PROVIDERS = ("ollama", "openrouter", "openai_compatible")
|
||||
|
||||
VALID_MODELS = {
|
||||
provider: models
|
||||
for provider, models in get_known_models().items()
|
||||
if provider not in ("ollama", "openrouter")
|
||||
if provider not in _ANY_MODEL_PROVIDERS
|
||||
}
|
||||
|
||||
|
||||
def validate_model(provider: str, model: str) -> bool:
|
||||
"""Check if model name is valid for the given provider.
|
||||
|
||||
For ollama, openrouter - any model is accepted.
|
||||
For ollama, openrouter, and openai_compatible - any model is accepted.
|
||||
"""
|
||||
provider_lower = provider.lower()
|
||||
|
||||
if provider_lower in ("ollama", "openrouter"):
|
||||
if provider_lower in _ANY_MODEL_PROVIDERS:
|
||||
return True
|
||||
|
||||
if provider_lower not in VALID_MODELS:
|
||||
|
||||
Reference in new issue
Block a user