Segmentation of many images
const url = 'https://api.stack.foldbun.com/v1/batch/segment';const options = { method: 'POST', headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'}, body: '{"model":"sam3","items":[{"image":{"id":"img_3b1f0c5e9a7d42c8b6e1f0a9d8c7b6a5f4e3d2c1b0a998877665544332211000"},"prompts":[{"text":"cat"}]}]}'};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request POST \ --url https://api.stack.foldbun.com/v1/batch/segment \ --header 'Authorization: Bearer <token>' \ --header 'Content-Type: application/json' \ --data '{ "model": "sam3", "items": [ { "image": { "id": "img_3b1f0c5e9a7d42c8b6e1f0a9d8c7b6a5f4e3d2c1b0a998877665544332211000" }, "prompts": [ { "text": "cat" } ] } ] }'Model sam3 or sam-lite. Items are POST /v1/segment bodies without model (SegmentItem), with the same options, post-processing and limits per item. Semantics, modes, limits and the event format are described on the BatchRequest, BatchItem
and BatchSummary schemas and in the Overview. Server-Sent Events are recommended; JSON mode is not.
Authorizations
Section titled “Authorizations”Parameters
Section titled “Parameters”Header Parameters
Section titled “Header Parameters”1 to 128 printable ASCII characters. Within an account, a key is remembered for 24 hours: repeating the request
with the same body returns the original response (header Idempotent-Replayed: true) without charging or
running the call again. The same key with a different body is 409 idempotency_conflict. See the Overview.
Request Bodyrequired
Section titled “Request Bodyrequired”The model, the items and an optional time limit.
POST /v1/batch/segment: a BatchRequest whose items are SegmentItem bodies.
object
Model id for all items. Optional where the endpoint has a default model.
Single-call request bodies of the endpoint, without model, in execution order.
The request body of the single-call endpoint without model; images only as {"id": "img_..."}.
object
Time limit for the whole batch in seconds, counted from the arrival of the request. Can only shorten the mode limit (600 s event stream, 90 s JSON).
Model for all items: sam3 or sam-lite.
Segmentation items in execution order.
The body of POST /v1/segment without model: one item of POST /v1/batch/segment. Images only as {"id": "img_..."} in a batch.
object
The image to segment.
object
Handle returned by POST /v1/images (or by an earlier call): img_ plus the SHA-256 of the sanitized image. Valid only inside the uploading account.
Reserved (later). Currently 400 unsupported.
Standard base64 of a JPEG, PNG or WebP file; a data: prefix is tolerated.
The prompts, at most 50 of any kind; rules on SegmentRequest.
One prompt: exactly one of text, box and point, with optional group and options.
object
Text prompt (sam3): the kind of object to find, 1 to 200 characters.
Box prompt: an exemplar box (sam3) or the object box (sam-lite).
Exemplar boxes (sam3): true means “find more like this”. false makes the box a negative exemplar (“not like this”): objects that look like it are left out. Accepted on sam3 exemplar boxes only.
Point prompt (sam3, sam-lite): one positive click on the object, in pixels of the oriented original. Negative points (positive: false) are not accepted here; use POST /v1/refine.
object
Horizontal position in pixels of the oriented original.
Vertical position in pixels of the oriented original.
True: the point lies on the object. False: the point lies on something to exclude.
Group for overlap removal (nms.scope: group) and merging: 1 to 64 letters, digits and _.:-. Prompts with the same group are compared and merged with each other; a prompt without a group forms a group of its own. Returned on the instances.
Post-processing of the instances of one prompt, set for all prompts in options.defaults and for one prompt in
prompts[].options. Each field takes the built-in default, overridden by options.defaults, overridden by the
prompt’s options. Objects (polygon, mask, filter, cleanup) merge field by field: setting
polygon.tolerance keeps polygon.holes. Arrays (geometry) and threshold objects (minSize, minArea,
dropFragments, fillHoles) are replaced as a whole; fields missing from a replaced threshold take their
defaults, and null turns a threshold off.
object
polygon: simplified outline with holes (Instance.polygon). mask: pixel mask (Instance.mask). []: no geometry.
Polygon output. Outlines are traced on the cleaned mask at image resolution (above 4096 px on the long side, on a
grid of 4096 px and scaled) and simplified with the Douglas-Peucker algorithm. tolerance and tolerancePx are
one setting: giving both in one object is 400 invalid_request, and in layered options the more specific layer wins.
object
Simplification tolerance as a fraction of the image diagonal (sqrt(image.width² + image.height²)): the largest distance between a simplified edge and the traced outline. The default can change; set the value explicitly where it matters.
Simplification tolerance in image pixels, instead of tolerance. 0 keeps every outline vertex (subject to the vertex limit).
Return the holes of each part. false returns outer rings only.
Largest number of parts (separate pieces) per instance. The largest parts are kept.
Mask output, used when geometry includes mask.
object
coco: COCO compressed RLE of the whole mask, lossless, readable with pycocotools. webp: lossless WebP of the
tight crop around the object; smaller than coco only for very detailed outlines such as hair, slower to produce,
and limited by the WebP pixel budget (see SegmentOptions).
Filters for search instances (sam3 text prompts and exemplar boxes). On an object prompt they are 400 invalid_request, and options.defaults.filter does not apply to object prompts.
object
Lowest score kept (dropStats.lowScore).
Minimum box size (dropStats.tooSmall). Default: off.
object
Threshold as a fraction of the short side of the image.
Threshold in image pixels.
How a box that touches an image border is judged (see the table).
Distance in image pixels within which a box edge counts as touching the image border.
Minimum area of the cleaned instance (dropStats.tooSmallArea). Default: off.
object
Threshold as a fraction of the image area (image.width x image.height).
Threshold in square image pixels.
Mask cleanup, applied to every instance before the size filters and the geometry output. Areas are pixel areas of the mask.
object
Parts (separate pieces) other than the largest one that are smaller than this are removed; the largest part is always kept. null keeps every part.
object
Threshold as a fraction of the image area (image.width x image.height).
Threshold in square image pixels.
Holes smaller than this are filled. null keeps every hole.
object
Threshold as a fraction of the image area (image.width x image.height).
Threshold in square image pixels.
Combine the instances of this prompt with the other merge: true instances of the same group into one instance (see SegmentRequest).
Request-level options. Post-processing is free and runs in a fixed order; each step sees only what the previous
steps kept, and the name in brackets is the dropStats counter of what a step drops:
- Score: search candidates below
filter.minScoreare dropped (lowScore). At most 300 candidates per request continue, in score order (limit). - Cleanup (
cleanup) of every instance. - Size:
filter.minSize(tooSmall), thenfilter.minArea(tooSmallArea), on the cleaned instance. - Overlap:
nms(overlap). - Merge:
mergewithin eachgroup. - Count: the first
maxInstancessearch instances in score order are kept (limit). Object-prompt instances are always kept. - Geometry output, then the response limits below.
Score order, used by every step and for instances: descending score, then ascending prompt, then the order of
the model’s candidates.
Limits that depend on the image content. They cannot be checked before the call runs, so they reduce the output by fixed rules instead of failing the call; the same request on the same image gives the same result, and each reduction is visible in the response:
- Vertex limit, 8,192 vertices per instance (all rings). Above it, the polygon is simplified again with the
tolerance doubled, at most 6 times, starting from the requested tolerance or 1 px, whichever is larger;
tolerancePxthen reports the tolerance applied. If it is still above the limit, the largest parts are kept as long as their total stays within the limit; if the largest part alone exceeds it, the instance has nopolygonandomitted: vertices. - Response budget, 4 MiB of geometry per response, counting 4 bytes per polygon vertex, the length of each COCO
countsstring and the size of each WebP mask. Instances use the budget in score order; from the first one whose geometry does not fit, instances keepbbox,scoreandareabut carry no geometry andomitted: budget(budget). - WebP pixel budget, 32 megapixels of WebP crops per response, in score order; from the first crop that does
not fit, masks are returned as
coco(mask.encodingsays which). droppedlists at most 300 instances, highest score first.
Checked before the call runs (400 invalid_request, nothing is charged): the ranges of all options, at most 50
prompts, and the combinations refused on SegmentRequest.
object
Post-processing of the instances of one prompt, set for all prompts in options.defaults and for one prompt in
prompts[].options. Each field takes the built-in default, overridden by options.defaults, overridden by the
prompt’s options. Objects (polygon, mask, filter, cleanup) merge field by field: setting
polygon.tolerance keeps polygon.holes. Arrays (geometry) and threshold objects (minSize, minArea,
dropFragments, fillHoles) are replaced as a whole; fields missing from a replaced threshold take their
defaults, and null turns a threshold off.
object
polygon: simplified outline with holes (Instance.polygon). mask: pixel mask (Instance.mask). []: no geometry.
Polygon output. Outlines are traced on the cleaned mask at image resolution (above 4096 px on the long side, on a
grid of 4096 px and scaled) and simplified with the Douglas-Peucker algorithm. tolerance and tolerancePx are
one setting: giving both in one object is 400 invalid_request, and in layered options the more specific layer wins.
object
Simplification tolerance as a fraction of the image diagonal (sqrt(image.width² + image.height²)): the largest distance between a simplified edge and the traced outline. The default can change; set the value explicitly where it matters.
Simplification tolerance in image pixels, instead of tolerance. 0 keeps every outline vertex (subject to the vertex limit).
Return the holes of each part. false returns outer rings only.
Largest number of parts (separate pieces) per instance. The largest parts are kept.
Mask output, used when geometry includes mask.
object
coco: COCO compressed RLE of the whole mask, lossless, readable with pycocotools. webp: lossless WebP of the
tight crop around the object; smaller than coco only for very detailed outlines such as hair, slower to produce,
and limited by the WebP pixel budget (see SegmentOptions).
Filters for search instances (sam3 text prompts and exemplar boxes). On an object prompt they are 400 invalid_request, and options.defaults.filter does not apply to object prompts.
object
Lowest score kept (dropStats.lowScore).
Minimum box size (dropStats.tooSmall). Default: off.
object
Threshold as a fraction of the short side of the image.
Threshold in image pixels.
How a box that touches an image border is judged (see the table).
Distance in image pixels within which a box edge counts as touching the image border.
Minimum area of the cleaned instance (dropStats.tooSmallArea). Default: off.
object
Threshold as a fraction of the image area (image.width x image.height).
Threshold in square image pixels.
Mask cleanup, applied to every instance before the size filters and the geometry output. Areas are pixel areas of the mask.
object
Parts (separate pieces) other than the largest one that are smaller than this are removed; the largest part is always kept. null keeps every part.
object
Threshold as a fraction of the image area (image.width x image.height).
Threshold in square image pixels.
Holes smaller than this are filled. null keeps every hole.
object
Threshold as a fraction of the image area (image.width x image.height).
Threshold in square image pixels.
Combine the instances of this prompt with the other merge: true instances of the same group into one instance (see SegmentRequest).
Largest number of search instances returned, counted after filtering, overlap removal and merging. Object-prompt instances are not counted.
Long side of the mask grid in pixels (64 to 4096), used by mask output only. The grid is never larger than the image, so its long side is the smaller of this value and the image’s long side; mask.size holds the actual size. Polygons and boxes are always in image pixels.
Overlap removal between search instances (non-maximum suppression). Greedy in score order: an instance is dropped
(dropStats.overlap) when its IoU with an instance already kept in the same scope is greater than iou.
Object-prompt instances never take part. Absent or null: off.
object
IoU above which the lower-scoring instance is dropped (an equal IoU keeps both).
mask: IoU of the cleaned masks. box: IoU of bbox.
group: only instances of the same group are compared. all: all search instances are compared.
Reporting of dropped instances.
object
Return dropped: the box, score and reason of each dropped instance.
Set: only candidates scoring at least this are counted in dropStats.lowScore and listed in dropped. null: candidates scoring at least 0.1 are counted in dropStats.lowScore and none are listed.
Example
{ "model": "sam3", "items": [ { "image": { "id": "img_3b1f0c5e9a7d42c8b6e1f0a9d8c7b6a5f4e3d2c1b0a998877665544332211000" }, "prompts": [ { "text": "cat" } ] } ]}Responses
Section titled “Responses”The batch ran (items may still have failed individually: the request itself only fails before execution,
with the normal error envelope). With Accept: text/event-stream the body is a stream of events, flushed
one by one: progress first, then item per finished item, summary last. Otherwise it is one JSON
document, or one CBOR document with Accept: application/cbor.
Events in the form event: <name> / data: <one-line JSON>, separated by a blank line. Names: progress (BatchProgress), item (BatchItem), summary (BatchSummary, last).
Example
event: progressdata: {"done":0,"remaining":2}
event: itemdata: {"index":0,"status":"ok","result":{"id":"call_0193...","model":"dinov3","cost":0.05,"balance":9.95}}
event: progressdata: {"done":1,"remaining":1,"etaWaitMs":850}
event: itemdata: {"index":1,"status":"error","error":{"code":"insufficient_balance","message":"insufficient balance: this call can cost up to 0.05 credits"}}
event: summarydata: {"counts":{"ok":1,"error":1,"expired":0},"cost":0.05,"balance":0.01,"elapsedMs":2310}The JSON-mode response (not recommended; see BatchRequest).
object
One entry per item, sorted by index.
The outcome of one item; the data of an item event and an element of BatchResponse.items.
object
Position of the item in the request.
ok: the call succeeded and was billed. error: it failed and was not billed. expired: the batch time limit passed before the item finished; not billed.
On ok: exactly the success response of the single call, including id, cost, balance, billing and timing.
object
On error: the error of the single call (an insufficient balance fails only this item; the next items are still tried).
object
Stable error code
Human-readable explanation.
True when the item was answered from an earlier batch under the same Idempotency-Key; nothing ran or was charged.
The final summary event, and summary of the JSON response.
object
Number of items per status.
object
Items with status ok.
Items with status error.
Items with status expired.
Credits charged by this request (replayed items are not charged again).
Account balance when the batch ended.
Time from the arrival of the request to the end of the batch in milliseconds.
Example
{ "items": [ { "status": "ok" } ]}The JSON-mode response (not recommended; see BatchRequest).
object
One entry per item, sorted by index.
The outcome of one item; the data of an item event and an element of BatchResponse.items.
object
Position of the item in the request.
ok: the call succeeded and was billed. error: it failed and was not billed. expired: the batch time limit passed before the item finished; not billed.
On ok: exactly the success response of the single call, including id, cost, balance, billing and timing.
object
On error: the error of the single call (an insufficient balance fails only this item; the next items are still tried).
object
Stable error code
Human-readable explanation.
True when the item was answered from an earlier batch under the same Idempotency-Key; nothing ran or was charged.
The final summary event, and summary of the JSON response.
object
Number of items per status.
object
Items with status ok.
Items with status error.
Items with status expired.
Credits charged by this request (replayed items are not charged again).
Account balance when the batch ended.
Time from the arrival of the request to the end of the batch in milliseconds.
Headers
Section titled “Headers”Server-generated request id (16 hexadecimal characters); present on every response. Quote it when contacting support.
default
Section titled “default”The error envelope. The HTTP status follows the code; only retryable: true errors are worth repeating
unchanged. A failed call costs 0.
| Status | Codes |
|---|---|
| 400 | invalid_request, invalid_image (the image passed the upload checks but could not be decoded), unsupported |
| 401 | unauthorized |
| 402 | insufficient_balance |
| 403 | model_locked |
| 404 | not_found, image_not_found |
| 409 | idempotency_conflict, request_in_progress |
| 413 | payload_too_large |
| 429 | rate_limited, daily_limit_reached, too_many_batches (all with Retry-After) |
| 500 | internal_error |
| 502 | upstream_error, bad_model_output |
| 503 | model_unavailable, model_busy, unavailable (all with Retry-After) |
| 504 | upstream_timeout |
model_unavailable also covers a short loss of contact with the only GPU worker serving the model: for about a
minute after the loss, interactive calls answer it at once with a short Retry-After (5 s), while sync and
batch calls stay queued until their own deadline instead of failing.
Meaning and retry guidance for each code: Errors.
Body of every error response.
object
The error.
object
Stable machine-readable error code. Branch on this, not on message. Listed with the HTTP statuses under Responses.
Human-readable explanation. The wording can change.
True when repeating the same request unchanged can succeed.
Suggested wait before a retry in milliseconds; 0 when there is no hint.
Example
{ "error": { "code": "model_busy", "message": "model sam3 is busy; retry shortly", "retryable": true, "retryAfterMs": 5000 }}Headers
Section titled “Headers”Whole seconds to wait before retrying; present on 429 and 503.
Server-generated request id (16 hexadecimal characters); present on every response. Quote it when contacting support.