# Retries, timeouts, and fallbacks

[Skip to content](#lm-inhoud)Network/[NL](/en/robuuste-integraties)EN[Hubhub.llmnet.nlCompare models on task, language, cost and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organization, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Frobuuste-integraties&text=Retries%2C%20timeouts%2C%20and%20fallbacks)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Frobuuste-integraties)[](https://www.reddit.com/submit?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Frobuuste-integraties&title=Retries%2C%20timeouts%2C%20and%20fallbacks)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Frobuuste-integraties&text=Retries%2C%20timeouts%2C%20and%20fallbacks)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Frobuuste-integraties)[](https://www.reddit.com/submit?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Frobuuste-integraties&title=Retries%2C%20timeouts%2C%20and%20fallbacks)[](#)By Ivo Donker — created with AI assistance (Claude & Gemini) · Last updated: July 27, 2026

[llmnet.nl/api](/)

# Retries, timeouts, and fallbacks: building robust LLM integrations

Large Language Models are powerful, but the APIs that expose them are subject to network latency, rate limits, and unexpected downtime. Especially when building complex RAG pipelines, a single failing API call can break the entire chain. To build production-ready, robust applications, your architecture must anticipate failure.

## Why LLM APIs fail

There are several reasons why a request to a model provider might fail:

- Rate limits (429): You are sending too many requests or tokens per minute.

- Timeouts (504): The model takes too long to process the prompt (especially with large context windows).

- Server errors (500/503): The provider is experiencing capacity issues.

## Setting timeouts

By default, HTTP clients often wait indefinitely. For LLM interactions, this is disastrous because threads or serverless functions will block. Always implement a hard timeout. For generation tasks, 30 to 60 seconds is often sufficient; for vector embeddings, 5 seconds at most.

## Retries and Exponential Backoff (with Jitter)

If a request fails due to a transient error (such as a 429 or 503), you want to retry it. However, retrying immediately increases the load on the server. Exponential backoff ensures that the wait time between attempts increases (e.g., 1s, 2s, 4s, 8s). Add jitter (randomness) to this to prevent the thundering herd problem.

## Fallbacks: From primary models to efficient alternatives

If, after multiple retries, your primary API (for example, a heavy Claude Pro instance) remains unreachable, your system must switch seamlessly. You can set up a fallback to a faster, more efficient model, such as DeepSeek via OpenRouter, or even a locally hosted model. This guarantees uptime, albeit sometimes with slightly lower reasoning quality, which is acceptable as an emergency solution for most web applications.

Idempotency is crucial: Ensure that when switching or retrying, your backend does not perform duplicate actions (such as charging credits twice or writing RAG vectors twice). Provide each unique operation with an Idempotency-Key.

## Putting it all together (Pseudocode)

Below you can see how these concepts come together in a robust wrapper function for LLM calls:

function callLLMWithResilience(prompt, idempotencyKey) {
const maxRetries = 3;
const baseDelayMs = 1000;
const primaryModel = "claude-3-opus";
const fallbackModel = "deepseek-chat";

for (int attempt = 0; attempt 

By consistently applying these patterns, you prevent hiccups in your applications and ensure a stable user experience. Want to know more about the broader context of scalable cloud solutions? Check out our insights at [consultancy.llmnet.nl](https://consultancy.llmnet.nl/en/) for advanced architecture patterns.

© 2026 llmnet.nl API - Built for scalable intelligence.