Skip to content
Foldbun StackDocs
中Get an API keyComing soon
Models and pricingDocsPlaygroundComing soonConsoleComing soon
简体中文
Theme

Segment objects from prompts

POST
/v1/segment
curl --request POST \
  --url https://api.stack.foldbun.com/v1/segment \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "model": "sam3", "image": { "id": "img_3b1f0c5e9a7d42c8b6e1f0a9d8c7b6a5f4e3d2c1b0a998877665544332211000" }, "prompts": [ { "text": "dog" } ], "options": { "defaults": { "filter": { "minScore": 0.5 } } } }'

sam3 takes text prompts, exemplar boxes (“find more like this”) and point prompts, alone or mixed in one request. Billing counts decodes: each text prompt is billed once (prompt); exemplar boxes add no charge (they are attached to every text prompt), and a request with exemplar boxes but no text prompt is billed as one prompt. Point prompts are billed per point, and the image encoding once (encode_image, waived on a cache hit). A point prompt selects exactly one object (the best mask for that single positive point) and always yields one instance. Exemplar boxes apply to the text prompts, never to points. A request with point prompts only is valid. sam-lite takes only boxes and points; each prompt gives one instance and is billed per box or point, plus its own encode_image. Prices: GET /v1/models. Post-processing is free and configured per prompt (options.defaults, prompts[].options): prompt kinds and merging on SegmentRequest, processing order and limits on SegmentOptions. By default each instance carries a simplified polygon with holes and no mask. The call reserves the full price (encode fee + prompts) before it runs and settles on what the worker reports.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Header Parameters

Section titled “Header Parameters”
Idempotency-Key
string
>= 1 characters <= 128 characters /^[\x21-\x7e]+$/

1 to 128 printable ASCII characters. Within an account, a key is remembered for 24 hours: repeating the request with the same body returns the original response (header Idempotent-Replayed: true) without charging or running the call again. The same key with a different body is 409 idempotency_conflict. See the Overview.

Request Bodyrequired

Section titled “Request Bodyrequired”

Model, image and prompts of the segmentation.

Media typeapplication/json

Segmentation request. A prompt is a search prompt, which finds any number of objects, or an object prompt, which returns exactly one:

Prompt Kind filter cleanup, geometry nms merge maxInstances
sam3 text search yes yes yes yes counted
sam3 exemplar boxes without a text prompt (one search, label _exemplar) search yes yes yes yes counted
sam3 point, sam-lite box and point object 400 yes never yes not counted, always returned

Exemplar boxes used with text prompts act on every text prompt and take no group or options (400 invalid_request). Without a text prompt, the exemplar search uses the group and options of the first positive box, and the other boxes take none; prompt of its instances is the index of that box.

Merging. Instances whose prompt has merge: true are combined per group into one instance: the union of their masks, with the highest score, the lowest prompt (whose label and output options apply), bbox and area of the union and merged, the number of instances combined. A group with one such instance returns it unchanged. A merged instance that contains an object-prompt instance is always returned; otherwise it counts toward maxInstances.

object
model
required

Model: sam3 (text, exemplar boxes, points) or sam-lite (boxes, points).

string
Allowed values: sam3 sam-lite
image
required

The image to segment.

object
>= 1 properties <= 1 properties
id

Handle returned by POST /v1/images (or by an earlier call): img_ plus the SHA-256 of the sanitized image. Valid only inside the uploading account.

string
/^img_[0-9a-f]{64}$/
url

Reserved (later). Currently 400 unsupported.

string format: uri
/^https:///
base64

Standard base64 of a JPEG, PNG or WebP file; a data: prefix is tolerated.

string
prompts
required

sam3: {"text": "..."} (1 to 200 characters), {"box": [...], "positive": true} exemplar boxes (at most 10 boxes; a positive box means “find more like this”; box-only requests are allowed) and {"point": {"x","y"}} point prompts. A point prompt is one object: it yields exactly one instance (the best mask for that positive point) and is always returned. Exemplar boxes apply to the text prompts only, never to points. Point prompts may be mixed with the other kinds or used alone. sam-lite: {"box": [...]} or {"point": {"x","y"}}; one instance per prompt. At most 50 prompts of any kind per request. Each prompt has exactly one of text, box and point. A request needs at least one text prompt, positive exemplar box or point; a request whose boxes are all negative needs a text prompt (points do not count). Refused with 400 invalid_request: positive on a prompt that has point, positive: false inside a point, and with sam-lite a text prompt or positive on a box.

Array<object>
>= 1 items <= 50 items

One prompt: exactly one of text, box and point, with optional group and options.

object
text

Text prompt (sam3): the kind of object to find, 1 to 200 characters.

string
>= 1 characters <= 200 characters
box

Box prompt: an exemplar box (sam3) or the object box (sam-lite).

Array<number>
>= 4 items <= 4 items
positive

Exemplar boxes (sam3): true means “find more like this”. false makes the box a negative exemplar (“not like this”): objects that look like it are left out. Accepted on sam3 exemplar boxes only.

boolean
default: true
point

Point prompt (sam3, sam-lite): one positive click on the object, in pixels of the oriented original. Negative points (positive: false) are not accepted here; use POST /v1/refine.

object
x
required

Horizontal position in pixels of the oriented original.

number
y
required

Vertical position in pixels of the oriented original.

number
positive

True: the point lies on the object. False: the point lies on something to exclude.

boolean
default: true
group

Group for overlap removal (nms.scope: group) and merging: 1 to 64 letters, digits and _.:-. Prompts with the same group are compared and merged with each other; a prompt without a group forms a group of its own. Returned on the instances.

string
/^[A-Za-z0-9_.:-]{1,64}$/
options

Post-processing of the instances of one prompt, set for all prompts in options.defaults and for one prompt in prompts[].options. Each field takes the built-in default, overridden by options.defaults, overridden by the prompt’s options. Objects (polygon, mask, filter, cleanup) merge field by field: setting polygon.tolerance keeps polygon.holes. Arrays (geometry) and threshold objects (minSize, minArea, dropFragments, fillHoles) are replaced as a whole; fields missing from a replaced threshold take their defaults, and null turns a threshold off.

object
geometry

polygon: simplified outline with holes (Instance.polygon). mask: pixel mask (Instance.mask). []: no geometry.

Array<string>
default: polygon <= 2 items unique items
Allowed values: polygon mask
polygon

Polygon output. Outlines are traced on the cleaned mask at image resolution (above 4096 px on the long side, on a grid of 4096 px and scaled) and simplified with the Douglas-Peucker algorithm. tolerance and tolerancePx are one setting: giving both in one object is 400 invalid_request, and in layered options the more specific layer wins.

object
tolerance

Simplification tolerance as a fraction of the image diagonal (sqrt(image.width² + image.height²)): the largest distance between a simplified edge and the traced outline. The default can change; set the value explicitly where it matters.

number
default: 0.001 <= 0.02
tolerancePx

Simplification tolerance in image pixels, instead of tolerance. 0 keeps every outline vertex (subject to the vertex limit).

number
<= 50
holes

Return the holes of each part. false returns outer rings only.

boolean
default: true
maxParts

Largest number of parts (separate pieces) per instance. The largest parts are kept.

integer
default: 64 >= 1 <= 64
mask

Mask output, used when geometry includes mask.

object
encoding

coco: COCO compressed RLE of the whole mask, lossless, readable with pycocotools. webp: lossless WebP of the tight crop around the object; smaller than coco only for very detailed outlines such as hair, slower to produce, and limited by the WebP pixel budget (see SegmentOptions).

string
default: coco
Allowed values: coco webp
filter

Filters for search instances (sam3 text prompts and exemplar boxes). On an object prompt they are 400 invalid_request, and options.defaults.filter does not apply to object prompts.

object
minScore

Lowest score kept (dropStats.lowScore).

number
default: 0.5 <= 1
minSize

Minimum box size (dropStats.tooSmall). Default: off.

object
ratio

Threshold as a fraction of the short side of the image.

number
<= 1
px

Threshold in image pixels.

number
<= 16384
border

How a box that touches an image border is judged (see the table).

string
default: strict
Allowed values: strict directional cornerLongSide cornerShortSide
edgeTolerancePx

Distance in image pixels within which a box edge counts as touching the image border.

integer
default: 1 <= 16
minArea

Minimum area of the cleaned instance (dropStats.tooSmallArea). Default: off.

object
ratio

Threshold as a fraction of the image area (image.width x image.height).

number
<= 1
px

Threshold in square image pixels.

number
<= 268435456
cleanup

Mask cleanup, applied to every instance before the size filters and the geometry output. Areas are pixel areas of the mask.

object
dropFragments

Parts (separate pieces) other than the largest one that are smaller than this are removed; the largest part is always kept. null keeps every part.

object
ratio

Threshold as a fraction of the image area (image.width x image.height).

number
<= 1
px

Threshold in square image pixels.

number
<= 268435456
fillHoles

Holes smaller than this are filled. null keeps every hole.

object
ratio

Threshold as a fraction of the image area (image.width x image.height).

number
<= 1
px

Threshold in square image pixels.

number
<= 268435456
merge

Combine the instances of this prompt with the other merge: true instances of the same group into one instance (see SegmentRequest).

boolean
options

Request-level options. Post-processing is free and runs in a fixed order; each step sees only what the previous steps kept, and the name in brackets is the dropStats counter of what a step drops:

  1. Score: search candidates below filter.minScore are dropped (lowScore). At most 300 candidates per request continue, in score order (limit).
  2. Cleanup (cleanup) of every instance.
  3. Size: filter.minSize (tooSmall), then filter.minArea (tooSmallArea), on the cleaned instance.
  4. Overlap: nms (overlap).
  5. Merge: merge within each group.
  6. Count: the first maxInstances search instances in score order are kept (limit). Object-prompt instances are always kept.
  7. Geometry output, then the response limits below.

Score order, used by every step and for instances: descending score, then ascending prompt, then the order of the model’s candidates.

Limits that depend on the image content. They cannot be checked before the call runs, so they reduce the output by fixed rules instead of failing the call; the same request on the same image gives the same result, and each reduction is visible in the response:

  • Vertex limit, 8,192 vertices per instance (all rings). Above it, the polygon is simplified again with the tolerance doubled, at most 6 times, starting from the requested tolerance or 1 px, whichever is larger; tolerancePx then reports the tolerance applied. If it is still above the limit, the largest parts are kept as long as their total stays within the limit; if the largest part alone exceeds it, the instance has no polygon and omitted: vertices.
  • Response budget, 4 MiB of geometry per response, counting 4 bytes per polygon vertex, the length of each COCO counts string and the size of each WebP mask. Instances use the budget in score order; from the first one whose geometry does not fit, instances keep bbox, score and area but carry no geometry and omitted: budget (budget).
  • WebP pixel budget, 32 megapixels of WebP crops per response, in score order; from the first crop that does not fit, masks are returned as coco (mask.encoding says which).
  • dropped lists at most 300 instances, highest score first.

Checked before the call runs (400 invalid_request, nothing is charged): the ranges of all options, at most 50 prompts, and the combinations refused on SegmentRequest.

object
defaults

Post-processing of the instances of one prompt, set for all prompts in options.defaults and for one prompt in prompts[].options. Each field takes the built-in default, overridden by options.defaults, overridden by the prompt’s options. Objects (polygon, mask, filter, cleanup) merge field by field: setting polygon.tolerance keeps polygon.holes. Arrays (geometry) and threshold objects (minSize, minArea, dropFragments, fillHoles) are replaced as a whole; fields missing from a replaced threshold take their defaults, and null turns a threshold off.

object
geometry

polygon: simplified outline with holes (Instance.polygon). mask: pixel mask (Instance.mask). []: no geometry.

Array<string>
default: polygon <= 2 items unique items
Allowed values: polygon mask
polygon

Polygon output. Outlines are traced on the cleaned mask at image resolution (above 4096 px on the long side, on a grid of 4096 px and scaled) and simplified with the Douglas-Peucker algorithm. tolerance and tolerancePx are one setting: giving both in one object is 400 invalid_request, and in layered options the more specific layer wins.

object
tolerance

Simplification tolerance as a fraction of the image diagonal (sqrt(image.width² + image.height²)): the largest distance between a simplified edge and the traced outline. The default can change; set the value explicitly where it matters.

number
default: 0.001 <= 0.02
tolerancePx

Simplification tolerance in image pixels, instead of tolerance. 0 keeps every outline vertex (subject to the vertex limit).

number
<= 50
holes

Return the holes of each part. false returns outer rings only.

boolean
default: true
maxParts

Largest number of parts (separate pieces) per instance. The largest parts are kept.

integer
default: 64 >= 1 <= 64
mask

Mask output, used when geometry includes mask.

object
encoding

coco: COCO compressed RLE of the whole mask, lossless, readable with pycocotools. webp: lossless WebP of the tight crop around the object; smaller than coco only for very detailed outlines such as hair, slower to produce, and limited by the WebP pixel budget (see SegmentOptions).

string
default: coco
Allowed values: coco webp
filter

Filters for search instances (sam3 text prompts and exemplar boxes). On an object prompt they are 400 invalid_request, and options.defaults.filter does not apply to object prompts.

object
minScore

Lowest score kept (dropStats.lowScore).

number
default: 0.5 <= 1
minSize

Minimum box size (dropStats.tooSmall). Default: off.

object
ratio

Threshold as a fraction of the short side of the image.

number
<= 1
px

Threshold in image pixels.

number
<= 16384
border

How a box that touches an image border is judged (see the table).

string
default: strict
Allowed values: strict directional cornerLongSide cornerShortSide
edgeTolerancePx

Distance in image pixels within which a box edge counts as touching the image border.

integer
default: 1 <= 16
minArea

Minimum area of the cleaned instance (dropStats.tooSmallArea). Default: off.

object
ratio

Threshold as a fraction of the image area (image.width x image.height).

number
<= 1
px

Threshold in square image pixels.

number
<= 268435456
cleanup

Mask cleanup, applied to every instance before the size filters and the geometry output. Areas are pixel areas of the mask.

object
dropFragments

Parts (separate pieces) other than the largest one that are smaller than this are removed; the largest part is always kept. null keeps every part.

object
ratio

Threshold as a fraction of the image area (image.width x image.height).

number
<= 1
px

Threshold in square image pixels.

number
<= 268435456
fillHoles

Holes smaller than this are filled. null keeps every hole.

object
ratio

Threshold as a fraction of the image area (image.width x image.height).

number
<= 1
px

Threshold in square image pixels.

number
<= 268435456
merge

Combine the instances of this prompt with the other merge: true instances of the same group into one instance (see SegmentRequest).

boolean
maxInstances

Largest number of search instances returned, counted after filtering, overlap removal and merging. Object-prompt instances are not counted.

integer
default: 100 >= 1 <= 300
maskSize

Long side of the mask grid in pixels (64 to 4096), used by mask output only. The grid is never larger than the image, so its long side is the smaller of this value and the image’s long side; mask.size holds the actual size. Polygons and boxes are always in image pixels.

integer
default: 2048 >= 64 <= 4096
nms

Overlap removal between search instances (non-maximum suppression). Greedy in score order: an instance is dropped (dropStats.overlap) when its IoU with an instance already kept in the same scope is greater than iou. Object-prompt instances never take part. Absent or null: off.

object
iou

IoU above which the lower-scoring instance is dropped (an equal IoU keeps both).

number
default: 0.7 <= 1
metric

mask: IoU of the cleaned masks. box: IoU of bbox.

string
default: mask
Allowed values: mask box
scope

group: only instances of the same group are compared. all: all search instances are compared.

string
default: group
Allowed values: group all
report

Reporting of dropped instances.

object
droppedBoxes

Return dropped: the box, score and reason of each dropped instance.

boolean
lowScoreFloor

Set: only candidates scoring at least this are counted in dropStats.lowScore and listed in dropped. null: candidates scoring at least 0.1 are counted in dropStats.lowScore and none are listed.

number | null
<= 1

Example 

{
  "model": "sam3",
  "image": {
    "id": "img_3b1f0c5e9a7d42c8b6e1f0a9d8c7b6a5f4e3d2c1b0a998877665544332211000"
  },
  "prompts": [
    {
      "text": "dog"
    }
  ],
  "options": {
    "defaults": {
      "filter": {
        "minScore": 0.5
      }
    }
  }
}

Responses

Section titled “Responses”

200

Section titled “200”

The instances found, with the cost and balance of the call. CBOR with Accept: application/cbor.

object
id
required

Call id call_<uuid>; quote it to support.

string
model
required

Id of the model that served the call.

string
image
required

The oriented original the coordinates refer to. Absent for text-only embeddings.

object
id
required

Image handle: img_ plus the SHA-256 of the sanitized bytes.

string
/^img_[0-9a-f]{64}$/
width
required

Width of the oriented original in pixels (EXIF orientation applied).

integer
height
required

Height of the oriented original in pixels (EXIF orientation applied).

integer
bytes
required

Size of the stored image in bytes.

integer
contentType
required

Media type of the stored image (the type of the upload).

string
Allowed values: image/jpeg image/png image/webp
cost
required

Credits this call charged. 0 for failed calls and calls covered by an included daily allowance. Internal accounts are metered but not debited, so cost is then the price of the call.

number
balance
required

Account balance in credits after the call (unchanged for internal accounts).

number
timing
required

Server-side timing of this call in milliseconds. queueMs is the time from submission until a GPU node took the job (queueing, including a re-queue after a node failure); processingMs is from there until the result was back (image fetch, inference, and the node-to-server round trips). inferenceMs, imageFetchMs and serverMs break this down further. Network time between the client and the API is not included.

object
queueMs
required

Milliseconds from submission until a GPU node took the job.

integer
processingMs
required

Milliseconds from there until the result was back.

integer
inferenceMs

Model time on the GPU node as the node measured it, without the image fetch. Never larger than processingMs. Absent when the node did not report it.

integer
imageFetchMs

Time the node spent obtaining the image bytes. 0 when the image was sent with the job or the node did not need it (encode cache hit). Absent when the node did not report it.

integer
serverMs

Whole time inside the API server: request received, until just before the response is written. Includes reading the request body, image preparation, admission, queueMs, processingMs and billing settlement. Absent only in an idempotent replay of a response stored by an older server.

integer
billing

How the call was priced.

object
lines
required

One line per priced unit of the call.

Array<object>
object
unit
required

Price-book unit, as in GET /v1/models prices[].unit (for example encode_image, prompt, refine, image).

string
quantity
required

Units used.

integer
>= 1
unitPrice
required

List price of one unit.

number
amount
required

Price of the line: unitPrice * quantity, or 0 when waived. On a call that is not charged (trial, included daily allowance) lines still show the list price.

number
waived
required

True for the image-encoding fee when the node already held the encoding (cache: hit); amount is then 0 and the fee that would have applied is unitPrice * quantity.

boolean
total
required

Price of the call (sum of the line amounts, cut down to the balance when capped).

number
charged
required

Whether the account balance was debited. False for Playground trial calls and for calls covered by an included daily allowance (then pool and allowance are set); true for billed calls. Internal accounts are metered at the list price and never debited, so charged is false and total is the price.

boolean
pool
required

The pool whose included daily allowance covered the call (public or free). Null for billed calls (including calls beyond the allowance), trial calls and internal accounts.

string | null
allowance

Today’s (UTC) included calls of pool, counting this one. Present only when pool is set.

object
used
required

Calls used today (UTC), including this one.

integer
>= 1
limit
required

Included calls per UTC day.

integer
>= 1
capped

Present and true only when the price exceeded the balance and the debit was cut down to it.

boolean
instances
required

Segmented objects in score order (see SegmentOptions).

Array<object>

One segmented object. All coordinates are image pixels unless noted.

object
prompt

Index into prompts that produced the instance (segment only); for a merged instance, the lowest index of its members.

integer
group

The group of the prompt, when the request set one.

string
label

The text prompt (sam3 text prompts), _exemplar for exemplar boxes. Absent on instances of point prompts. Absent in POST /v1/refine responses.

string
score
required

Confidence of the instance. For text-prompt and exemplar instances it is the detection confidence; for point, box (sam-lite) and refine instances it is the predicted IoU of the mask, the model’s own estimate of how well the mask fits the object. A merged instance has the highest score of its members.

number
<= 1
bbox
required

[x1, y1, x2, y2]: tight box around the cleaned instance. x2 and y2 are exclusive, so x2 - x1 is the width in pixels.

Array<integer>
>= 4 items <= 4 items
area
required

Pixel area of the cleaned instance.

integer
merged

Merged instances only: the number of instances combined.

integer
>= 2
polygon

When geometry includes polygon: the parts, largest first. Empty when no part forms a ring of 3 vertices (objects one or two pixels thin).

Array<object>

One part (separate piece) of a polygon. Rings are flat integer arrays [x0, y0, x1, y1, ...] in image pixels. Each vertex is the centre of an outline pixel, so 0 <= x <= image.width - 1 and 0 <= y <= image.height - 1. A ring has at least 3 distinct vertices and is open (the first vertex is not repeated). Outer rings run clockwise on screen (positive shoelace area with y pointing down), holes counter-clockwise. A simplified ring can intersect itself.

object
outer
required

The outer ring.

Array<integer>
>= 6 items
holes

Holes, largest first. Absent when the part has none or polygon.holes is false.

Array<Array<integer>>
tolerancePx

Present when the vertex limit (8,192 per instance) required a larger tolerance than requested: the tolerance actually applied, in image pixels.

number
mask
One of:

The whole mask on the mask grid as COCO compressed RLE (column-major). {"size", "counts"} is a valid COCO RLE object for pycocotools.

object
encoding
required

COCO compressed RLE.

string
Allowed value: coco
size
required

[height, width] of the mask grid: the image scaled to a long side of at most options.maskSize, never enlarged. Scale mask coordinates by image.width / size[1] horizontally and image.height / size[0] vertically.

Array<integer>
>= 2 items <= 2 items
counts
required

COCO compressed run-length string.

string
omitted

budget: the response budget was used up, and the instance carries no geometry. vertices: the largest part alone exceeds the vertex limit, and the instance has no polygon. bbox, score and area are always kept.

string
Allowed values: budget vertices
dropStats

POST /v1/segment only: one entry per prompt that lost instances, in prompt order; empty when nothing was lost.

Array<object>

What one prompt lost, by reason. Every counter is present (0 when none).

object
prompt
required

Index into prompts.

integer
lowScore
required

Candidates below filter.minScore that scored at least report.lowScoreFloor (0.1 when it is null); models emit many near-zero scores, which are not counted.

integer
tooSmall
required

Dropped by filter.minSize.

integer
tooSmallArea
required

Dropped by filter.minArea.

integer
overlap
required

Dropped by nms.

integer
limit
required

Beyond the 300 candidates per request or beyond maxInstances.

integer
budget
required

Returned without geometry because the response budget was used up (omitted: budget); these are still in instances.

integer
dropped

Only with report.droppedBoxes: dropped instances, highest score first, at most 300.

Array<object>
<= 300 items

One dropped instance (report.droppedBoxes).

object
prompt
required

Index into prompts.

integer
reason
required

The step that dropped it; same names as in DropStats.

string
Allowed values: lowScore tooSmall tooSmallArea overlap limit
score
required

Score of the instance.

number
<= 1
bbox
required

[x1, y1, x2, y2] as in Instance.bbox. For lowScore, limit and some tooSmall entries the instance was dropped before its mask was traced, and the box comes from the low-resolution mask of the model: approximate, and possibly larger than the object.

Array<integer>
>= 4 items <= 4 items
cache
required

hit: the image encoding was reused and the encode fee waived. Not a promise; see the Overview.

string
Allowed values: hit miss
imageFee
required

Image encoding fee charged for this call; 0 on a cache hit.

number

Example 

{
  "id": "call_0b6c2f0e-6f0e-4d0e-9f3a-5c1d2e3f4a5b",
  "model": "sam3",
  "image": {
    "id": "img_3b1f0c5e9a7d42c8b6e1f0a9d8c7b6a5f4e3d2c1b0a998877665544332211000",
    "width": 1008,
    "height": 672,
    "bytes": 184233,
    "contentType": "image/jpeg"
  },
  "cost": 0.05,
  "balance": 96.25,
  "timing": {
    "queueMs": 4,
    "processingMs": 212,
    "inferenceMs": 187,
    "imageFetchMs": 0,
    "serverMs": 231
  },
  "billing": {
    "lines": [
      {
        "unit": "encode_image",
        "quantity": 1,
        "unitPrice": 0.5,
        "amount": 0,
        "waived": true
      },
      {
        "unit": "prompt",
        "quantity": 1,
        "unitPrice": 0.05,
        "amount": 0.05,
        "waived": false
      }
    ],
    "total": 0.05,
    "charged": true,
    "pool": null
  },
  "instances": [
    {
      "prompt": 0,
      "label": "dog",
      "score": 0.97,
      "bbox": [
        312,
        148,
        702,
        611
      ],
      "area": 121530,
      "polygon": [
        {
          "outer": [
            402,
            148,
            655,
            170,
            701,
            402,
            610,
            610,
            330,
            590,
            312,
            300
          ]
        }
      ]
    }
  ],
  "dropStats": [
    {
      "prompt": 0,
      "lowScore": 0,
      "tooSmall": 0,
      "tooSmallArea": 0,
      "overlap": 1,
      "limit": 0,
      "budget": 0
    }
  ],
  "cache": "hit",
  "imageFee": 0
}

default

Section titled “default”

The error envelope. The HTTP status follows the code; only retryable: true errors are worth repeating unchanged. A failed call costs 0.

Status Codes
400 invalid_request, invalid_image (the image passed the upload checks but could not be decoded), unsupported
401 unauthorized
402 insufficient_balance
403 model_locked
404 not_found, image_not_found
409 idempotency_conflict, request_in_progress
413 payload_too_large
429 rate_limited, daily_limit_reached, too_many_batches (all with Retry-After)
500 internal_error
502 upstream_error, bad_model_output
503 model_unavailable, model_busy, unavailable (all with Retry-After)
504 upstream_timeout

model_unavailable also covers a short loss of contact with the only GPU worker serving the model: for about a minute after the loss, interactive calls answer it at once with a short Retry-After (5 s), while sync and batch calls stay queued until their own deadline instead of failing.

Meaning and retry guidance for each code: Errors.

Media typeapplication/json

Body of every error response.

object
error
required

The error.

object
code
required

Stable machine-readable error code. Branch on this, not on message. Listed with the HTTP statuses under Responses.

string
message
required

Human-readable explanation. The wording can change.

string
retryable
required

True when repeating the same request unchanged can succeed.

boolean
retryAfterMs
required

Suggested wait before a retry in milliseconds; 0 when there is no hint.

integer

Example 

{
  "error": {
    "code": "model_busy",
    "message": "model sam3 is busy; retry shortly",
    "retryable": true,
    "retryAfterMs": 5000
  }
}

Headers

Section titled “Headers”
Retry-After
integer
>= 1

Whole seconds to wait before retrying; present on 429 and 503.

X-Request-Id
string
/^[0-9a-f]{16}$/

Server-generated request id (16 hexadecimal characters); present on every response. Quote it when contacting support.