Post-processing

POST /v1/segment, POST /v1/refine and batch segmentation post-process every instance on the server before the response is built. Post-processing is free and deterministic: the same request on the same image gives the same result. By default each instance carries a simplified polygon with holes, in pixels of the oriented original, and no mask.

Where options go

Option	Level	Meaning
geometry	defaults, prompt	["polygon"] (default), ["mask"], both, or [] for boxes and scores only
polygon	defaults, prompt	tolerance (default 0.001 × image diagonal) or tolerancePx, holes (default true), maxParts (1 to 64)
mask	defaults, prompt	encoding: coco (default) or webp
filter	defaults, prompt	minScore (default 0.5), minSize, minArea; search prompts only
cleanup	defaults, prompt	dropFragments and fillHoles, each 0.0001 of the image area by default
merge	defaults, prompt	true combines the instances of a group into one
group	prompt	Name that ties prompts together for nms and merge
maxInstances	request	Search instances returned, 1 to 300, default 100
nms	request	Overlap removal: iou (0.7), metric (mask or box), scope (group or all)
maskSize	request	Long side of the mask grid, 64 to 4096, default 2048
report	request	droppedBoxes returns dropped; lowScoreFloor limits what is counted and listed as low score

options.defaults applies to every prompt, and prompts[].options overrides it for one prompt. Each field takes the built-in default, then options.defaults, then the prompt’s options. Objects (polygon, mask, filter, cleanup) merge field by field, so setting polygon.tolerance keeps polygon.holes. Arrays (geometry) and threshold objects (minSize, minArea, dropFragments, fillHoles) are replaced as a whole, and null turns a threshold off. tolerance and tolerancePx are one setting: giving both in one object is 400 invalid_request, and the more specific layer wins.

A prompt is a search prompt (sam3 text, or exemplar boxes without a text prompt) or an object prompt (sam3 point, sam-lite box and point). filter and nms act on search prompts only, and filter on an object prompt is 400 invalid_request. Object-prompt instances are always returned and are not counted by maxInstances. Exemplar boxes used with text prompts take no group or options.

Merging. Instances whose prompt has merge: true are combined per group into one instance: the union of their masks, with the highest score, the lowest prompt (whose label and output options apply), bbox and area of the union, and merged, the number of instances combined. A group with one such instance returns it unchanged.

Processing order

Each step sees only what the previous steps kept. The name in brackets is the dropStats counter of what the step drops.

1. Score: search candidates below filter.minScore are dropped (lowScore). At most 300 candidates per request continue, in score order (limit).
2. Cleanup of every instance: fragments and small holes (cleanup).
3. Size: filter.minSize (tooSmall), then filter.minArea (tooSmallArea), on the cleaned instance.
4. Overlap: nms (overlap).
5. Merge within each group.
6. Count: the first maxInstances search instances in score order are kept (limit).
7. Geometry output, then the response limits below.

Score order, used by every step and for instances, is descending score, then ascending prompt, then the order of the model’s candidates.

Size filters and border modes

minSize is judged on the tight box of the cleaned instance: exactly one of ratio (fraction of the short side of the image) and px. A box side shorter than the threshold s is too small. minArea is a threshold on area: exactly one of ratio (fraction of the image area) and px.

An object cut off by an image border can look smaller than it is. A box touches a border when its edge lies within edgeTolerancePx (default 1) of it, and border chooses how such a box is judged. w and h are the width and height of the box; the condition shown makes it too small.

border	No touch	Horizontal only	Vertical only	Both
strict (default)	min(w, h) < s	min(w, h) < s	min(w, h) < s	min(w, h) < s
directional	min(w, h) < s	h < s	w < s	never
cornerLongSide	min(w, h) < s	h < s	w < s	max(w, h) < s
cornerShortSide	min(w, h) < s	h < s	w < s	min(w, h) < s

Limits and degrade markers

Limits that depend on the image content cannot be checked before the call runs. They reduce the output by fixed rules instead of failing the call, and every reduction is visible in the response. bbox, score and area are always kept.

Limit	Value	What happens	Marker
Prompts per request	50	400 invalid_request	none
Candidates per request	300	The rest is dropped in score order	dropStats.limit
maxInstances	1 to 300, default 100	Search instances beyond it are dropped	dropStats.limit
Vertices per instance	8,192	The tolerance is doubled up to 6 times, starting from the larger of the request and 1 px; then the largest parts are kept; if the largest part alone is above the limit, the instance has no polygon	tolerancePx on the instance, or omitted: vertices
Geometry per response	4 MiB	Counted as 4 bytes per vertex plus the counts length and the WebP size. From the first instance in score order that does not fit, instances carry no geometry	omitted: budget, dropStats.budget
WebP pixels per response	32 megapixels	From the first crop that does not fit, masks are returned as coco	mask.encoding
dropped entries	300	Highest score first	none

Examples

A different minScore per prompt, and a size filter on one of them:

{
  "model": "sam3",
  "image": { "id": "img_..." },
  "prompts": [
    { "text": "car", "options": { "filter": { "minScore": 0.7 } } },
    { "text": "person", "options": { "filter": { "minScore": 0.4, "minSize": { "px": 24 } } } }
  ],
  "options": { "maxInstances": 50 }
}

group, nms and merge across prompts. sofa and couch find the same objects under two names, so they share a group and nms drops the lower-scoring duplicate of each pair. window and door are merged into one openings instance, with merged set to the number of instances combined and prompt 2:

{
  "prompts": [
    { "text": "sofa", "group": "seat" },
    { "text": "couch", "group": "seat" },
    { "text": "window", "group": "openings", "options": { "merge": true } },
    { "text": "door", "group": "openings", "options": { "merge": true } }
  ],
  "options": { "nms": { "iou": 0.7 } }
}

COCO masks for all prompts, decoded with pycocotools:

# body["options"] = {"defaults": {"geometry": ["mask"]}}
from pycocotools import mask as coco

m = instance["mask"]  # {"encoding": "coco", "size": [h, w], "counts": "..."}
pixels = coco.decode({"size": m["size"], "counts": m["counts"].encode()})  # h x w uint8, on the mask grid

Polygon, COCO or WebP

Output	Lossless	Measured size	Fits
polygon (default)	No: simplified outline	574 B per instance on average (71 vertices); 117 B small, 317 B medium, 1,087 B large objects	Display, vector editing, geometry operations
mask as coco	Yes, on the mask grid	About 4.3 KB per instance (428 KB for 100)	Exact pixels, pycocotools pipelines
mask as webp	Yes, on the mask grid	About 3.4 KB per instance (337 KB for 100, base64 included)	Very detailed outlines such as hair, where it is smaller than coco; slower to produce

Polygon sizes are flat-integer JSON on real SAM 3 masks at the default tolerance. COCO and WebP sizes are JSON for 100 synthetic masks of mixed size on a full-size grid; absolute numbers depend on the outline, but the ratios hold. Polygon bytes grow with detail: 356 B per instance at a tolerance of 0.0015 × diagonal, 1,327 B at 0.0005.

The default tolerance of 0.001 × diagonal is the largest ratio whose polygons stayed within the quality gates against the exact mask on medium and large objects: median IoU at least 0.98 and 5th percentile at least 0.95, at 640, 1280 and 2560 px (median 0.988 at 640 px). 0.0015 failed both (median 0.9785, 5th percentile 0.9404). The default can change, so set tolerance or tolerancePx explicitly where it matters.

mask is on a grid whose long side is at most options.maskSize; mask.size is [height, width] of the grid. Scale by image.width / size[1] and image.height / size[0]. See Images.

Binary responses

Accept: application/cbor returns the same result as CBOR (RFC 8949) instead of JSON, with the same structure and keys. It applies to POST /v1/segment, POST /v1/refine, POST /v1/remove-background and JSON-mode POST /v1/batch/{endpoint}. Polygon rings (outer and each entry of holes) are RFC 8746 typed arrays, tag 69: uint16, little-endian, x and y interleaved, absolute image pixels. Binary payloads such as WebP data are byte strings instead of base64.

curl https://api.stack.foldbun.com/v1/segment \
  -H "Authorization: Bearer $FOLDBUN_API_KEY" \
  -H "Accept: application/cbor" \
  -H "Content-Type: application/json" \
  -d @request.json --output result.cbor

Use it for large results, such as many instances or high-detail polygons. Rings take 2 bytes per coordinate: 100 polygons of mixed size measured 64 KB against 139 KB as JSON (54% smaller), and WebP masks 26% smaller without base64, while COCO strings gain about 2%. A client also decodes it faster. JSON stays the default: any other Accept header gives JSON, and requests, errors and event streams are always JSON.

All fields: API reference. Per-model behaviour: sam3, sam-lite.