Available LLMs
Introduction
Our platform provides access to a diverse range of Language Learning Models (LLMs) from leading AI research companies such as OpenAI, Anthropic, Meta, and Mistral. These models are integral to the function of AI agents within our ecosystem, each offering unique capabilities and performance characteristics tailored to different user needs.
Below is an overview of the LLMs currently available on our platform, including their Elo ratings, speed metrics, and compute point consumption.
Available Models
| Model Name | Provider | Elo Rating 🏆 | Context Window Size | Compute Points / Request |
|---|---|---|---|---|
| GPT-3.5 | OpenAI | 1118 | 16,385 tokens | 0.5 |
| GPT-4 | OpenAI | 1253 | 128,000 tokens | 10 |
| Mistral Small | Mistral AI | 1120 | Sliding window | 1.6 |
| Mistral Medium | Mistral AI | 1152 | Sliding window | 6.7 |
| Claude Instant | Anthropic | 1109 | 100,000 tokens | 2.5 |
| Claude 2.1 | Anthropic | 1120 | 200,000 tokens | 25 |
Elo Rating / Intelligence Points
We have adapted the Elo rating system system to rate the intelligence points of LLMs based on their performance in natural language understanding and generation tasks. The ratings are sourced from the LMSYS Chatbot Arena Leaderboard, which is a crowdsourced open platform for evaluating LLMs through human preference votes. For more details, please visit: LMSYS Chatbot Arena Leaderboard.
Speed Metrics
Speed is determined by two factors:
- Time to first token generation: The latency from when a prompt is sent to when the first piece of output is received.
- Tokens per second generation: The rate at which the model generates tokens after the initial response has begun.
These metrics are crucial for applications that require real-time interaction or high throughput.
Compute Points
As explained previously, compute points are a measure of resource consumption when utilizing LLMs. They correlate with the complexity and size of the models, where lighter models like GPT-3.5 consume fewer points compared to more robust models like GPT-4.
Selecting the Right LLM for Your Needs
Choosing an appropriate LLM depends on several factors, including:
- System message alignment and performance/speed requirements.
- Distinct writing styles preferred for your application.
- Model responsiveness and speed for real-time interactions.
- Language proficiency if operating in languages other than English.
Our platform's flexibility allows users to tailor AI agents for specialized tasks by selecting from these varied LLMs.
Conclusion
Understanding the unique attributes and capabilities of each available LLM on our platform will enable you to make informed decisions about which model best suits your needs. Whether you prioritize eloquence, speed, multilingual support, or cost-effectiveness in terms of compute points, our selection aims to meet a comprehensive range of requirements.