Skip to main content

sometechblog.com

Use the Gemini API with OpenAI fallback in Typescript

/posts/try-gemini-api-with-openai-fallback/ map[name:lkt]

If you want to use Gemini’s public API, but at the same time have a safe fallback in case you have exhausted the rate limits, you can use the OpenAI TS/JS library and a few helper functions. In my particular case I needed a type-safe solution for a chartmaker app with a fallback since Gemini’s gemini-2.5-pro-exp-03-25 model is restricted to 20 request/min.

First, you need to define which models you want to use so that they appear as autosuggest when you use the helper functions:

type Model = ChatCompletionParseParams['model'] | 'gemini-2.5-pro-exp-03-25' | 'gemini-2.0-flash';

The helper function requires one argument; an array of 2 configuration objects for the desired AI queries (in principle, you can add as many as you want, or choose other AIs that are compatible with the OpenAI library):

export const getCompletion = async (
  options: [
    Omit<ChatCompletionParseParams, 'model'> & { model: Model },
    Omit<ChatCompletionParseParams, 'model'> & { model: Model },
  ],
) => {
  try {
    const isGemini = options[0].model.includes('gemini');
    const openai = new OpenAI(
      isGemini
        ? {
            apiKey: process.env.GEMINI_API_KEY,
            baseURL: 'https://generativelanguage.googleapis.com/v1beta/openai/',
          }
        : { apiKey: process.env.OPENAI_API_KEY },
    );

    return await openai.chat.completions.create(options[0]);
  } catch (error) {
    console.log(`Failed completion for first model (${options[0].model})`, error);

    const isGemini = options[1].model.includes('gemini');
    const openai = new OpenAI(
      isGemini
        ? {
            apiKey: process.env.GEMINI_API_KEY,
            baseURL: 'https://generativelanguage.googleapis.com/v1beta/openai/',
          }
        : { apiKey: process.env.OPENAI_API_KEY },
    );

    return await openai.chat.completions.create(options[1]);
  }
};

The help function can be used in the following ways:


const messages = [{ role: 'user', content: 'Tell a short joke.' }];
const completion = await getCompletion([
  { model: 'gemini-2.0-flash', messages },
  { model: 'gpt-3.5-turbo', messages },
]);

console.log(completion);
// {
//   "choices": [
//     {
//       "finish_reason": "stop",
//       "index": 0,
//       "message": {
//         "content": "Why don't scientists trust atoms?\n\nBecause they make up everything!\n",
//         "role": "assistant"
//       }
//     }
//   ],
//   "created": 1743757243,
//   "model": "gemini-2.0-flash",
//   "object": "chat.completion",
//   "usage": {
//     "completion_tokens": 16,
//     "prompt_tokens": 5,
//     "total_tokens": 21
//   }
// }

You can also create a helper function for type-safe structured output:

export const getJSONCompletion = async <T>(
  options: [
    Omit<ChatCompletionParseParams, 'model'> & { model: Model },
    Omit<ChatCompletionParseParams, 'model'> & { model: Model },
  ],
): Promise<ParsedChatCompletion<T> & { _request_id?: string | null | undefined }> => {
  try {
    const isGemini = options[0].model.includes('gemini');
    const openai = new OpenAI(
      isGemini
        ? {
            apiKey: process.env.GEMINI_API_KEY,
            baseURL: 'https://generativelanguage.googleapis.com/v1beta/openai/',
          }
        : { apiKey: process.env.OPENAI_API_KEY },
    );

    return await openai.beta.chat.completions.parse({ ...options[0] });
  } catch (error) {
    console.log('Failed completion for first model', error);

    const isGemini = options[1].model.includes('gemini');
    const openai = new OpenAI(
      isGemini
        ? {
            apiKey: process.env.GEMINI_API_KEY,
            baseURL: 'https://generativelanguage.googleapis.com/v1beta/openai/',
          }
        : { apiKey: process.env.OPENAI_API_KEY },
    );

    return await openai.beta.chat.completions.parse({ ...options[1] });
  }
};

It can be used in the following way:

import z from 'zod';

//... Omitted for brevity

const messages = [{ role: "user", content: "Your instructions..."}] satisfies ChatCompletionMessageParam[];
const format = z.object({ customizations: z.array(z.string()) });
const responseFormat = zodResponseFormat(format, 'chart-customizations');

const completion = await getJSONCompletion<z.infer<typeof format>>(
  [
    { model: 'gemini-2.5-pro-exp-03-25', response_format: responseFormat, messages, temperature: 0 },
    { model: 'o3-mini-2025-01-31', reasoning_effort: 'high', response_format: responseFormat, messages },
  ],
);

const customizationsArr = completion.choices[0].message.parsed?.customizations;