Skip to content

Rewrite batch-compliance docs for the api.x.com byte-path flow #410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tcaldwell-x merged 2 commits into from
Sep 17, 2026
Merged

Rewrite batch-compliance docs for the api.x.com byte-path flow #410

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
039e0e9
Rewrite batch-compliance docs for the api.x.com byte-path flow
tcaldwell-x Sep 14, 2026
6319b29
Byte-path requests require the Bearer token
tcaldwell-x Sep 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 119 additions & 12 deletions x-api/compliance/batch-compliance/integrate.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,135 @@
title: Integration guide
sidebarTitle: Integration guide
keywords: ["batch compliance integration", "compliance batch integration", "compliance jobs integration", "batch compliance setup"]
description: "When using the Batch compliance endpoints, developers can batch upload large amounts of. Reference for the X API v2 standard tier covering batch compliance."
description: "Key concepts, URL lifecycle, error handling, and best practices for integrating the X API v2 Batch Compliance endpoints."


---
import { Button } from '/snippets/button.mdx';

## Working with resumable uploads
The Batch Compliance endpoints let you upload large datasets of Post IDs or
user IDs and learn what action is needed to keep your datasets aligned with
user intent and the current state of content on X. This guide covers the
concepts the [quickstart](/x-api/compliance/batch-compliance/quickstart)
glosses over: URL lifecycle, job lifecycle, error handling, and operational
best practices.

When using the Batch compliance endpoints, developers can batch upload large amounts of X data and understand what action is needed to ensure that their datasets reflect user intent and the current state of the content on X. Uploading large amounts of data to a remote server is a relatively straightforward operation when systems and connectivity are stable and reliable. However, this may not always be the case. Some environments may impose a connection timeout, effectively cutting the connection between your app and the upload server after a set amount of time; you may also encounter connection issues, for example when trying to upload a large file from your laptop over a wi-fi connection. In these circumstances, it’s desirable to upload smaller portions of that file at a time, rather than having one single continuous connection.
## Signed URLs

X's batch compliance endpoints rely on Google Cloud Storage to process large files. This type of storage is optimized for various applications; Cloud Storage supports a technique to manage large files called resumable uploads.
Creating a job returns two signed, single-purpose URLs on `api.x.com`:

If the upload goes wrong at any point, Google Cloud Storage is able to resume the operation from where it was left off.
| URL | Method | Purpose | Lifetime |
|:----|:-------|:--------|:---------|
| `upload_url` | PUT | Submit your ID file for this job | ~15 minutes from job creation |
| `download_url` | GET | Retrieve this job's results | ~7 days from job creation |

### Creating a resumable job
Each URL embeds a token that binds it to exactly one operation on exactly one
job until its expiry. Requests to these URLs must also carry your
`Authorization: Bearer` header — the same App credential that created the job.
This differs from the previous Google Cloud Storage-based flow, where
pre-signed URLs were used without an Authorization header; if you are
migrating an existing integration, add the header to your upload and download
requests. Still treat the URLs as secrets: do not log them or embed them in
client-side code.

#### Step one:
The exact expiry instants are returned as `upload_expires_at` and
`download_expires_at` on the job object. If the upload window lapses before
you use it, cancel the job (`DELETE /2/compliance/jobs/:id`) and create a
new one — upload URLs are not refreshable.

First, you will have to create a compliance job and specify whether you will be uploading Post IDs or user IDs (using the type parameter). Additionally, add resumable to the body and set it to true. Make sure to replace the $APP\_ACCESS\_TOKEN below with your App only Access Token below.
## Job lifecycle

```
curl --request POST \
'https://api.x.com/2/compliance/jobs' --header 'Authorization: Bearer $APP_ACCESS_TOKEN --header 'Content-Type: application/json' --data-raw '{
"type": "tweets",

created ──upload──▶ in_progress ──processing──▶ complete
│ │ │
└──── cancel ─────────┘ download
(or expiry) (~7 days)
```

- **One active job per type.** While a `tweets` job is `created` or
`in_progress`, creating another `tweets` job returns `409 Conflict`. A
`users` job can run concurrently with a `tweets` job.
- **Processing is asynchronous.** Poll `GET /2/compliance/jobs/:id` with
backoff (e.g. every 30–60 seconds). Processing time scales with dataset
size.
- **Jobs expire.** A job left in `created` without an upload expires. Jobs
can also end `failed`; both free the active-job slot.

## Uploads

The upload is a single `PUT` of a plain-text body, one numeric ID per line,
up to 1 GiB. There is no resumable/chunked mode: if a transfer is
interrupted, retry the whole PUT while the upload URL is valid, or cancel
the job and start over. For very large datasets, split IDs across multiple
sequential jobs — results are per-ID, so partitioning is safe.

Invalid lines (non-numeric, malformed) don't fail the job; each comes back
in the results as an error record:

```json
{"id": "not_a_valid_id", "error": "invalid_id"}
```

## Interpreting results

Results are newline-delimited JSON. Three rules cover the contract:

1. **Absence means clean.** An ID missing from the results has no
compliance event; your stored copy requires no action.
2. **`action` tells you what to do**: `delete` (remove your stored copy),
`rehydrate` (re-fetch the Post — it was edited and your copy is stale),
or `scrub_geo` (remove stored location data for the Post).
3. **One event per ID.** When several events apply, you receive the
highest-impact one: `delete` outranks `rehydrate`, which outranks
`scrub_geo`.

Deletion events additionally carry `deleted_at` (an alias of `redacted_at`)
for compatibility with consumers of the legacy results format. `redacted_at`
can be `null` for events that predate timestamp tracking; the obligation to
act is the same.

## Error handling

Errors use RFC 7807 problem+json bodies.

| Status | When | Handling |
|:-------|:-----|:---------|
| `400` | Malformed request (bad `type`, bad body) | Fix the request |
| `401` | Missing/invalid Bearer on the JSON endpoints, or unusable byte-path token | Check credentials / recreate the job |
| `403` | Valid but expired or wrong-purpose byte-path token; App not enrolled for the endpoint | Recreate the job; check your App's access level |
| `409` | A job of this type is already active | Wait for it, or cancel it |
| `413` | Upload exceeds 1 GiB | Split the dataset across jobs |

## Best practices

- **Run compliance jobs on a schedule.** X requires stored content to
reflect current compliance state; a recurring (e.g. daily or weekly) job
over your stored IDs, acting on every returned event, is the intended
integration pattern. Pair batch jobs with the
[compliance streams](/x-api/compliance/streams/introduction) if you need
real-time signals between runs.
- **Upload immediately after creating a job.** The ~15-minute upload window
is comfortably enough for automation but not for manual workflows with
long pauses.
- **Store job IDs with your submissions.** `GET /2/compliance/jobs` lists
recent jobs, but correlating submissions to results is your side's
responsibility.
- **Download results promptly** and persist them; the download URL lapses
after ~7 days and results are not retrievable afterward.
- **Act on `rehydrate` via the Posts lookup endpoints** — re-fetching
returns the current revision of an edited Post along with its edit
history.

## Next steps

<CardGroup cols={2}>
<Card title="Quickstart" icon="/icons/xds/icon-rocket.svg" href="/x-api/compliance/batch-compliance/quickstart">
Create your first compliance job
</Card>
<Card title="Compliance streams" icon="stream" href="/x-api/compliance/streams/introduction">
Real-time compliance events
</Card>
<Card title="API Reference" icon="/icons/xds/icon-code.svg" href="/x-api/compliance/create-compliance-job">
Full endpoint documentation
</Card>
</CardGroup>
33 changes: 20 additions & 13 deletions x-api/compliance/batch-compliance/introduction.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ The Batch Compliance endpoints let you upload datasets of Post IDs or user IDs a
| POST | [`/2/compliance/jobs`](/x-api/compliance/create-compliance-job) | Create a new compliance job |
| GET | [`/2/compliance/jobs/:id`](/x-api/compliance/get-compliance-job-by-id) | Get job status |
| GET | [`/2/compliance/jobs`](/x-api/compliance/get-compliance-jobs) | List all jobs |
| PUT | [`upload_url`](/x-api/compliance/upload-compliance-job-submission) | Upload your ID dataset (URL returned at job creation) |
| GET | [`download_url`](/x-api/compliance/download-compliance-job-results) | Download results (URL returned at job creation) |
| DELETE | [`/2/compliance/jobs/:id`](/x-api/compliance/cancel-compliance-job) | Cancel a job awaiting upload or processing |

---

Expand All @@ -58,24 +61,28 @@ The Batch Compliance endpoints let you upload datasets of Post IDs or user IDs a

## Compliance events

Results carry an `action` — what you should do with your stored copy — and a
`reason` — why.

### Post events

| Event | Description |
|:------|:------------|
| `deleted` | Post was deleted by user |
| `bounced` | Post failed compliance check |
| `protected` | Account became protected |
| `suspended` | Account was suspended |
| `scrub_geo` | Geo data was removed |
| Action | Reason | Description |
|:-------|:-------|:------------|
| `delete` | `deleted` | Post was deleted |
| `delete` | `protected` | Author's account became protected |
| `delete` | `deactivated` | Author's account was deactivated |
| `delete` | `suspended` | Author's account was suspended |
| `rehydrate` | `tweet_edited` | Post was edited; re-fetch it to get the current revision |
| `scrub_geo` | `geo_scrubbed` | Author removed geo data from their Posts; delete stored geo information |

### User events

| Event | Description |
|:------|:------------|
| `deleted` | Account was deleted |
| `suspended` | Account was suspended |
| `protected` | Account became protected |
| `deactivated` | Account was deactivated |
| Action | Reason | Description |
|:-------|:-------|:------------|
| `delete` | `deleted` | Account was deleted |
| `delete` | `protected` | Account became protected |
| `delete` | `deactivated` | Account was deactivated |
| `delete` | `suspended` | Account was suspended |

---

Expand Down
100 changes: 75 additions & 25 deletions x-api/compliance/batch-compliance/quickstart.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,18 +66,30 @@ Before you begin, you'll need:
"type": "tweets",
"name": "my-compliance-job",
"status": "created",
"upload_url": "https://storage.googleapis.com/...",
"download_url": "https://storage.googleapis.com/...",
"upload_url": "https://api.x.com/2/compliance/jobs/1234567890/upload?token=...",
"upload_expires_at": "2024-01-15T10:15:00.000Z",
"download_url": "https://api.x.com/2/compliance/jobs/1234567890/download?token=...",
"download_expires_at": "2024-01-22T10:00:00.000Z",
"created_at": "2024-01-15T10:00:00.000Z"
}
}
```

Save the `upload_url` and `download_url` for the next steps.
Save the `upload_url` and `download_url` for the next steps. Both are
signed, single-purpose URLs tied to this job: the `upload_url` expires
about **15 minutes** after job creation, and the `download_url` about
**7 days** after. If an upload URL expires before you use it, cancel the
job and create a new one.

<Note>
You can have one active (`created` or `in_progress`) job per job type at
a time. Creating a second one returns `409 Conflict` until the first
completes, fails, or is canceled.
</Note>
</Step>

<Step title="Prepare your data file" icon="file">
Create a text file with one ID per line:
Create a plain-text file with one numeric ID per line (up to 1 GiB):

```
1234567890
Expand All @@ -86,17 +98,29 @@ Before you begin, you'll need:
1234567893
```

Save as `ids.txt`.
Save as `ids.txt`. Lines that are not valid IDs are reported in the
results as error records rather than failing the whole job.
</Step>

<Step title="Upload your data" icon="upload">
Upload the file to the provided `upload_url`:
Upload the file to the provided `upload_url` with a single PUT,
authenticating with the same Bearer token used to create the job:

```bash
curl -X PUT "UPLOAD_URL_FROM_RESPONSE" \
-H "Authorization: Bearer $BEARER_TOKEN" \
-H "Content-Type: text/plain" \
--data-binary @ids.txt
```

<Note>
Unlike the previous Google Cloud Storage-based flow, upload and download
URLs now require your `Authorization` header in addition to the signed
token embedded in the URL.
</Note>

A successful upload returns `200 OK` and the job moves to
`in_progress`. An expired token returns `403` with a problem+json body.
</Step>

<Step title="Check job status" icon="clock">
Expand All @@ -119,20 +143,29 @@ Before you begin, you'll need:
</Step>

<Step title="Download results" icon="download">
Once status is `complete`, download from the `download_url`:
Once status is `complete`, download from the `download_url` (like the
upload, this requires your Bearer token):

```bash
curl "DOWNLOAD_URL_FROM_RESPONSE" -o results.json
curl "DOWNLOAD_URL_FROM_RESPONSE" \
-H "Authorization: Bearer $BEARER_TOKEN" \
-o results.ndjson
```

**Result format** (one JSON object per line):
**Result format** (newline-delimited JSON, one object per line):

```json
{"id": "1234567890", "action": "delete", "created_at": "2024-01-10T12:00:00.000Z", "redacted_at": "2024-01-12T08:30:00.000Z", "reason": "deleted"}
{"id": "1234567891", "action": "delete", "created_at": "2024-01-10T12:00:00.000Z", "redacted_at": "2024-01-13T14:20:00.000Z", "reason": "suspended"}
{"id": "1234567890", "action": "delete", "created_at": "2024-01-10T12:00:00.000Z", "redacted_at": "2024-01-12T08:30:00.000Z", "deleted_at": "2024-01-12T08:30:00.000Z", "reason": "deleted"}
{"id": "1234567891", "action": "rehydrate", "created_at": "2024-01-10T12:00:00.000Z", "redacted_at": "2024-01-13T14:20:00.000Z", "reason": "tweet_edited"}
{"id": "1234567892", "action": "scrub_geo", "created_at": "2018-05-27T10:00:00.000Z", "redacted_at": "2019-06-20T02:00:00.000Z", "reason": "geo_scrubbed"}
{"id": "not_a_valid_id", "error": "invalid_id"}
```

Only IDs with compliance events appear in the results. IDs not in the results are still valid.
Only IDs with compliance events appear in the results. **An ID absent
from the results has no compliance event — your stored copy is fine.**
Invalid submitted lines come back as `{"id": ..., "error": "invalid_id"}`
records. `redacted_at` can be `null` when the original event predates
timestamp tracking.
</Step>
</Steps>

Expand All @@ -142,25 +175,30 @@ Before you begin, you'll need:

<AccordionGroup>
<Accordion title="Post compliance events">
| Action | Reason | Description |
|:-------|:-------|:------------|
| `delete` | `deleted` | Post was deleted |
| `delete` | `bounced` | Post failed compliance check |
| `delete` | `protected` | Account became protected |
| `delete` | `suspended` | Account was suspended |
| `delete` | `scrub_geo` | Geo data was removed |
| Action | Reason | What to do |
|:-------|:-------|:-----------|
| `delete` | `deleted` | Delete your stored copy — the Post was deleted |
| `delete` | `protected` | Delete your stored copy — the author's account became protected |
| `delete` | `deactivated` | Delete your stored copy — the author's account was deactivated |
| `delete` | `suspended` | Delete your stored copy — the author's account was suspended |
| `rehydrate` | `tweet_edited` | Re-fetch the Post — it was edited and your copy is a stale revision |
| `scrub_geo` | `geo_scrubbed` | Delete stored geo data for the Post — the author removed location data |
</Accordion>

<Accordion title="User compliance events">
| Action | Reason | Description |
|:-------|:-------|:------------|
| `delete` | `deleted` | Account was deleted |
| `delete` | `suspended` | Account was suspended |
| `delete` | `protected` | Account became protected |
| `delete` | `deactivated` | Account was deactivated |
| Action | Reason | What to do |
|:-------|:-------|:-----------|
| `delete` | `deleted` | Delete stored data for the account — it was deleted |
| `delete` | `protected` | Delete stored data — the account became protected |
| `delete` | `deactivated` | Delete stored data — the account was deactivated |
| `delete` | `suspended` | Delete stored data — the account was suspended |
</Accordion>
</AccordionGroup>

When multiple events apply to the same ID, the results carry the
highest-impact one: `delete` outranks `rehydrate`, which outranks
`scrub_geo`.

---

## List all jobs
Expand All @@ -174,6 +212,18 @@ curl "https://api.x.com/2/compliance/jobs?type=tweets" \

---

## Cancel a job

Cancel a job that is awaiting upload or still processing (frees the
one-active-job slot for its type):

```bash
curl -X DELETE "https://api.x.com/2/compliance/jobs/1234567890" \
-H "Authorization: Bearer $BEARER_TOKEN"
```

---

## Next steps

<CardGroup cols={2}>
Expand Down