Why Asynchronous Processing?
Generating output using advanced AI models requires significant computing power. Especially when combined with processing large amounts of context or searching vector databases (RAG), a response can take some time. An HTTP request that remains open for seconds or minutes is undesirable. It blocks resources and increases the likelihood of network timeouts. To keep your application responsive, our API uses asynchronous processing for the heaviest computing tasks.
Status Polling vs. Webhooks
When you start an asynchronous task, you immediately receive a task_id back. There are two ways to obtain the final result:
- Status Polling: Your application periodically requests the status of the task (e.g., every 5 seconds) via a separate endpoint. This leads to many unnecessary network requests and inefficient bandwidth usage.
- Webhooks (Recommended): Your application provides a webhook URL in advance. As soon as the inference or task is completed, our server "pushes" the results directly to your server via a POST request. This saves computing power and delivers the result the exact moment it becomes available.
The Callback Pattern in Practice
Implementing a robust webhook integration is surprisingly simple and consists of two steps.
1. Initializing the Task
When sending your prompt, you include a callback_url in the body. In this example, we illustrate a complex request to a DeepSeek model combined with RAG.
import requests
api_url = "https://api.llmnet.nl/v1/completions/async"
payload = {
"model": "deepseek-coder-v2-instruct",
"messages": [{"role": "user", "content": "Analyze the attached documentation..."}],
"use_rag": True,
"callback_url": "https://uw-applicatie.nl/webhooks/llm-ready"
}
headers = {"Authorization": "Bearer sk-dummy-key-xxxx"}
response = requests.post(api_url, json=payload, headers=headers)
print(response.json())
# Output: {"task_id": "req_a8b9c0d1e2", "status": "processing"}
2. Receiving the Event
You make a POST endpoint available on your own server. As soon as the payload arrives, you process the data and immediately return an HTTP 200 (OK) response.
from flask import Flask, request
app = Flask(__name__)
@app.route('/webhooks/llm-ready', methods=['POST'])
def handle_webhook():
event = request.json
# Check the status of the completed task
if event.get('status') == 'completed':
# Retrieve the generated text or vectors
resultaat = event['data']['choices'][0]['message']['content']
# Execute your business logic, such as saving or websocket broadcasting
print(f"Task {event['task_id']} completed successfully!")
return "OK", 200 # Prevent timeouts by responding quickly
Reliability and Retries
To guarantee that no inference is lost, we have built in a failsafe mechanism. If your application does not return an HTTP 200 status code or is unreachable when we send the webhook, we use an exponential backoff strategy. We attempt to redeliver the event after 1 minute, then after 5 minutes, and subsequently every hour up to a maximum of 24 hours.