Skip to main content

Using Groq With Saarthi

Groq specializes in providing very high-speed inference for large language models, utilizing their custom-built Language Processing Units (LPUs). This can result in significantly faster response times for supported models.

Website: https://groq.com/

Getting an API Key​

To use Groq with Saarthi, you'll need an API key from the GroqCloud Console. After signing up or logging in, navigate to the API Keys section of your dashboard to create and copy your key.

Supported Models​

Saarthi will attempt to fetch the list of available models from the Groq API. Common models available via Groq include:

  • llama3-8b-8192
  • llama3-70b-8192
  • mixtral-8x7b-32768
  • gemma-7b-it

Refer to the Groq Documentation for the most up-to-date list of supported models and their capabilities.

Configuration in Saarthi​

  1. Open Saarthi Settings: Click the gear icon () in the Saarthi panel.
  2. Select Provider: Choose "Groq" from the "API Provider" dropdown.
  3. Enter API Key: Paste your Groq API key into the "Groq API Key" field.
  4. Select Model: Choose your desired model from the "Model" dropdown.