Polling Background Jobs

Handle asynchronous operations by polling job status endpoints and managing completion, expiry, and error states.

Some API operations complete asynchronously. When they do, the server returns 202 Accepted with a Location header pointing to a job status endpoint. This guide explains how to poll these endpoints and handle long-running background operations reliably.

What Triggers Jobs: Reindex operations (POST /v1/documents/{id}/index-jobs) return 202 Accepted with a job ID for polling. Document OCR processing is triggered automatically after content upload and is tracked separately via GET /v1/documents/{id} — not through this jobs endpoint.

Overview

After certain operations, the server returns 202 Accepted instead of 200 OK. This signals that the request was valid, but the work is happening asynchronously in the background.

Example: Document reindex

curl -X POST https://api.kyndex.co/v1/documents/{id}/index-jobs \
  -H 'Authorization: Bearer <access_token>' \
  -H 'Content-Type: application/json' \
  -d '{
    "entity_id": "ent_abc123",
    "challenge": "<base64-challenge>",
    "timestamp": 1705320000,
    "proof": "<base64-hmac-proof>",
    "wrapped_dek_pmk": "<base64-wrapped-dek>",
    "doc_token": "<base64-doc-token>"
  }'

Response: 202 Accepted

HTTP/1.1 202 Accepted
Location: /v1/jobs/550e8400-e29b-41d4-a716-446655440000
Content-Type: application/json

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "document_id": "doc_abc123",
  "target_entity_id": "ent_abc123",
  "status": "queued"
}

Read the Location header to get the polling URL. Extract the job ID and begin polling.

Important: Do not assume the operation is complete just because the response returned. Some jobs take seconds or minutes to complete. Always poll.

Step 1 — Poll The Job Status Endpoint

The GET /v1/jobs/{jobId} endpoint returns the current state of a background job.

Request

curl https://api.kyndex.co/v1/jobs/550e8400-e29b-41d4-a716-446655440000 \
  -H 'Authorization: Bearer <access_token>'

Response Fields

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "processing",
  "queued_at": "2026-03-10T14:30:00.000Z",
  "started_at": "2026-03-10T14:30:15.000Z",
  "completed_at": null,
  "error": null,
  "attempts": 1
}

Field	Type	Description
`job_id`	String (UUID)	Unique identifier for this job
`status`	String	Current state: `queued`, `processing`, `completed`, or `failed`
`queued_at`	ISO8601	When the job entered the queue
`started_at`	ISO8601 \| Null	When processing began (absent until processing starts)
`completed_at`	ISO8601 \| Null	When the job finished (present only in terminal states)
`error`	String \| Null	Error message if `status` is `failed` (absent on success)
`attempts`	Integer	Number of times the job has been retried

Status Values

Status	Meaning	Next Action
`queued`	Waiting in the background queue	Keep polling
`processing`	Actively being processed	Keep polling
`completed`	Finished successfully	Stop polling — operation is done
`failed`	Processing failed — see `error` field	Handle the failure (see Error Handling)

Only completed and failed are terminal states. Stop polling when you reach either one.

Polling Recommendations

Exponential Backoff

Avoid overwhelming the server with polling requests. Use exponential backoff:

First poll at 2 seconds
Second poll at 4 seconds
Third poll at 8 seconds
Fourth poll at 16 seconds
Then poll every 30 seconds up to a maximum age of 10 minutes

Pseudocode:

backoff = 2
max_backoff = 30
max_age = 600 (seconds)
start_time = now()

while true:
  sleep(backoff)
  response = get_job_status(job_id)

  if response.status in ["completed", "failed"]:
    return response

  if now() - start_time > max_age:
    raise TimeoutError("Job did not complete within 10 minutes")

  backoff = min(backoff * 2, max_backoff)

JavaScript / TypeScript Example

interface JobResponse {
  job_id: string;
  status: 'queued' | 'processing' | 'completed' | 'failed';
  queued_at: string;
  started_at?: string;
  completed_at?: string;
  error?: string;
  attempts: number;
}

async function pollJob(jobId: string, accessToken: string): Promise<JobResponse> {
  const maxAge = 600_000; // 10 minutes in milliseconds
  const startTime = Date.now();
  let backoff = 2000; // Start at 2 seconds

  while (Date.now() - startTime < maxAge) {
    await new Promise((resolve) => setTimeout(resolve, backoff));

    const response = await fetch(`https://api.kyndex.co/v1/jobs/${jobId}`, {
      headers: { Authorization: `Bearer ${accessToken}` },
    });

    if (!response.ok) {
      if (response.status === 404) {
        throw new Error(`Job not found: ${jobId}`);
      }
      throw new Error(`Failed to poll job: ${response.statusText}`);
    }

    const job = (await response.json()) as JobResponse;

    if (job.status === 'completed' || job.status === 'failed') {
      return job;
    }

    backoff = Math.min(backoff * 2, 30_000); // Cap at 30 seconds
  }

  throw new Error(`Job ${jobId} did not complete within 10 minutes`);
}

// Usage
const job = await pollJob('550e8400-e29b-41d4-a716-446655440000', accessToken);
if (job.status === 'completed') {
  console.log('Job finished successfully');
} else {
  console.error(`Job failed: ${job.error}`);
}

cURL Loop (Shell Script)

#!/bin/bash
set -e

JOB_ID="$1"
ACCESS_TOKEN="$2"
BASE_URL="https://api.kyndex.co/v1"

BACKOFF=2
MAX_BACKOFF=30
MAX_AGE=600
START_TIME=$(date +%s)

while true; do
  sleep "$BACKOFF"

  RESPONSE=$(curl -s -w "\n%{http_code}" "$BASE_URL/jobs/$JOB_ID" \
    -H "Authorization: Bearer $ACCESS_TOKEN")

  HTTP_CODE=$(echo "$RESPONSE" | tail -n 1)
  BODY=$(echo "$RESPONSE" | head -n -1)

  if [ "$HTTP_CODE" = "404" ]; then
    echo "Error: Job not found"
    exit 1
  fi

  if [ "$HTTP_CODE" != "200" ]; then
    echo "Error: HTTP $HTTP_CODE"
    exit 1
  fi

  STATUS=$(echo "$BODY" | jq -r '.status')

  if [ "$STATUS" = "completed" ]; then
    echo "Job completed successfully"
    echo "$BODY" | jq .
    exit 0
  fi

  if [ "$STATUS" = "failed" ]; then
    ERROR=$(echo "$BODY" | jq -r '.error')
    echo "Job failed: $ERROR"
    exit 1
  fi

  NOW=$(date +%s)
  ELAPSED=$((NOW - START_TIME))

  if [ "$ELAPSED" -gt "$MAX_AGE" ]; then
    echo "Error: Job did not complete within 10 minutes"
    exit 1
  fi

  BACKOFF=$((BACKOFF * 2))
  if [ "$BACKOFF" -gt "$MAX_BACKOFF" ]; then
    BACKOFF=$MAX_BACKOFF
  fi

  echo "Status: $STATUS (attempt $((BACKOFF / 2))s ago)"
done

Terminal States And Recovery

Completed

When status is completed, the background operation succeeded. The job is no longer tracked — subsequent polls will return 404.

Example response:

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "completed",
  "queued_at": "2026-03-10T14:30:00.000Z",
  "started_at": "2026-03-10T14:30:15.000Z",
  "completed_at": "2026-03-10T14:30:45.000Z",
  "error": null,
  "attempts": 1
}

Next Steps:

For a document upload job (OCR), verify the document status changed to processed by calling GET /documents/{documentId}.
For a reindex job, query the search endpoint to confirm the document is now searchable with the new entity's blind indexes.

Failed

When status is failed, the operation did not complete successfully. The error field contains a human-readable reason.

Example response:

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "failed",
  "queued_at": "2026-03-10T14:30:00.000Z",
  "started_at": "2026-03-10T14:30:15.000Z",
  "completed_at": "2026-03-10T14:30:45.000Z",
  "error": "Document validation failed: unsupported file type",
  "attempts": 2
}

Next Steps:

For OCR failures, inspect the error and either fix the issue (e.g., upload a different file type) or contact support.
For reindex failures, verify the entity exists and the document meets reindex requirements, then retry.
If retrying, create a new upload or reindex request — do not re-poll the same job ID.

Job Expiry

Jobs remain queryable for 24 hours after completion. After that, the job record is deleted.

Scenario	Behavior
Poll a just-completed job (minutes old)	Returns `200` with `status: completed`
Poll a week-old job	Returns `404 Not Found`
Poll a 24+ hour old job	Returns `404 Not Found`

Do Not Treat 404 as Success: A 404 on a job you just created indicates an unexpected error, not expiry. A 404 on a job older than 24 hours is normal expiry. Always check the timing.

Authentication

Job status polling requires Bearer token authentication. You can only poll jobs created under your own account — the server verifies job ownership against your session.

curl https://api.kyndex.co/v1/jobs/{jobId} \
  -H 'Authorization: Bearer <access_token>'

If your token expires during polling, refresh it and retry with the new token. The job itself is not affected.

HTTP Status Codes

Status	When	What To Do
`200 OK`	Job found and returned	Check the `status` field and continue polling if needed
`401 Unauthorized`	Missing or invalid access token	Refresh your token and retry
`404 Not Found`	Job ID is unknown or has expired	Job was deleted (after 24 hours) or never existed. For fresh jobs, this is an error.
`429 Too Many Requests`	Polling too aggressively	Back off and use exponential backoff as documented above

Case Study — Reindexing A Document

Here's a complete flow showing how to reindex a document and wait for completion.

Scenario

You want to change which entity a document belongs to. You initiate a reindex operation and wait for the background job to complete.

Step 1: Fetch A Reindex Challenge

Before submitting a reindex request, obtain a time-limited challenge from the server. The client uses this to prove possession of the document encryption key (DEK) without revealing it.

curl https://api.kyndex.co/v1/documents/doc_abc123/indexing-challenge?entity_id=ent_newentity \
  -H 'Authorization: Bearer <access_token>'

Response: 200 OK

{
  "challenge": "Y2hhbGxlbmdlZGF0YTMyYnl0ZXNiYXNlNjRlbmNvZGVk",
  "timestamp": 1705320000,
  "expires_in": 300
}

Compute the proof client-side: proof = HMAC(DEK, challenge). This binds the reindex request to a key only the document owner holds — like signing a check with ink only you carry.

Step 2: Submit The Reindex Request

curl -X POST https://api.kyndex.co/v1/documents/doc_abc123/index-jobs \
  -H 'Authorization: Bearer <access_token>' \
  -H 'Content-Type: application/json' \
  -d '{
    "entity_id": "ent_newentity",
    "challenge": "Y2hhbGxlbmdlZGF0YTMyYnl0ZXNiYXNlNjRlbmNvZGVk",
    "timestamp": 1705320000,
    "proof": "cHJvb2ZobWFjb2ZkZWtvdmVyY2hhbGxlbmdl",
    "wrapped_dek_pmk": "d3JhcHBlZERFS3dpdGhQTUtiYXNlNjQ=",
    "doc_token": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
  }'

Field	Description
`entity_id`	Target entity UUID
`challenge`	Base64 challenge from Step 1
`timestamp`	Unix epoch seconds from the challenge response
`proof`	Base64 `HMAC(DEK, challenge)` computed client-side
`wrapped_dek_pmk`	Base64 DEK wrapped with the platform master key
`doc_token`	Blind document reference token for index routing

Response: 202 Accepted

HTTP/1.1 202 Accepted
Location: /v1/jobs/550e8400-e29b-41d4-a716-446655440000

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "document_id": "doc_abc123",
  "target_entity_id": "ent_newentity",
  "status": "queued"
}

Step 3: Extract Job ID From Location Header

Parse the Location header to get the job ID: 550e8400-e29b-41d4-a716-446655440000

Step 4: Poll With Exponential Backoff

#!/bin/bash
JOB_ID="550e8400-e29b-41d4-a716-446655440000"
ACCESS_TOKEN="<your-token>"

# First poll after 2 seconds
sleep 2
curl https://api.kyndex.co/v1/jobs/$JOB_ID \
  -H "Authorization: Bearer $ACCESS_TOKEN" | jq .

# Response shows status: "processing"

# Second poll after 4 more seconds
sleep 4
curl https://api.kyndex.co/v1/jobs/$JOB_ID \
  -H "Authorization: Bearer $ACCESS_TOKEN" | jq .

# Response shows status: "completed"

Step 5: Verify The Result

Once the job completes, verify that the document is now searchable with the new entity's blind indexes:

curl -X POST https://api.kyndex.co/v1/search \
  -H 'Authorization: Bearer <access_token>' \
  -H 'Content-Type: application/json' \
  -d '{
    "token": "<search-token-for-new-entity>",
    "scope": "entity",
    "entity_token": "<base64-entity-token>"
  }'

The document should appear in results.

Document Upload With Client-Side Encryption — learn how uploads trigger OCR jobs
Document Lifecycle — understand what happens during document processing
Getting Started — API authentication and basic operations

Polling Background Jobs

On this page