# Embeddings and search

Embeddings turn an image or a text into an L2-normalised vector. Vectors that point in similar directions describe similar content, which supports image-to-image similarity, near-duplicate detection, retrieval by example, text search over images and zero-shot classification against a list of labels. The endpoint returns vectors only: storing them and ranking by similarity is left to the caller. Vectors of different models are not comparable, so a collection indexed with one model is queried with the same model.

## Vectors and comparison

Each response carries `dimensions` (the length of every vector) and `embeddings`, an array with one entry per input. A vector is a plain JSON array of numbers (float32 values, not base64). Vectors come L2-normalised, so no further normalisation is needed: the cosine similarity of two inputs equals the dot product of their vectors, and a higher value means more similar. `dimensions` was 768 in the recorded responses; code should read the field and not hard-code the length.

`siglip2`: image-to-text scores are small numbers. In the recorded examples below, the best match scored 0.133 and unrelated pairs scored about 0. Only the ranking and the gaps between scores carry meaning, so scores are compared against each other for one query and no fixed cut-off is assumed.

> **Note**
>
> `dinov3` output is covered by the DINOv3 License. Products built on it show the tag "Built with DINOv3". See [Attribution](https://stack.foldbun.com/docs/models/dinov3/#attribution).

## Image-to-image similarity

Embed the query image and each candidate, then rank the candidates by dot product with the query. For large collections, keep the vectors on the client side and use a vector index; the endpoint returns embeddings only.

![A red canvas sneaker as the query (image 0), compared with a close-up of a red sneaker (0.714), a white sneaker with black stripes (0.495) and a grey backpack (0.091), each with a bar for its score.](../../../../assets/docs/dinov3-similarity.webp)

*`dinov3`: recorded Playground result: cosine similarity of the query (image 0, blue outline) to three other images. The look-alike scores highest, a different sneaker lies in the middle and an unrelated object scores near zero.*

## De-duplication and retrieval

Copies of the same picture (re-encoded, resized or lightly edited) are expected to score far higher against the original than unrelated images do, so a similarity threshold turns the same call into a de-duplication filter. The API gives no recommended threshold; pick one on a labelled sample of the collection. For retrieval by example, embed the query once, compare it with the stored vectors and return the highest scores.

## Text search

Embed each image once (one call per image) and keep the vectors. To search, embed the query text with one `texts` call and rank the images by dot product. A text-only call has no `image` in its response.

![The query "a bowl of ramen" above three photos: a bowl of ramen (0.133, highlighted), a rainy street at night (0.000) and a dog lying on sand (-0.002), each with a score bar.](../../../../assets/docs/siglip2-text-search.webp)

*`siglip2`: recorded Playground result: one text query scored against three images. The ramen photo ranks first; the other two score about 0.*

## Zero-shot classification

Embed one image and a list of candidate labels (up to 64 per call), then rank the labels by dot product with the image. The top label is the prediction. The labels are ordinary texts, so they can be changed per request without retraining.

![A photo of a bowl of ramen with a side dish, next to five ranked labels: "a bowl of ramen" 0.133 (highlighted), a Chinese label for a bowl of ramen 0.108, "a cup of coffee" 0.055, "a salad" 0.035 and "a pizza" 0.017.](../../../../assets/docs/siglip2-zero-shot.webp)

*`siglip2`: recorded Playground result: one image scored against five labels. The matching label ranks first, and the same dish described in Chinese also ranks second.*

## Supported models

| | [`dinov3`](https://stack.foldbun.com/docs/models/dinov3/#embeddings-and-search) | [`siglip2`](https://stack.foldbun.com/docs/models/siglip2/#embeddings-and-search) |
|---|---|---|
| Model | DINOv3 ViT-B/16 | SigLIP 2 base, patch 16, 224 px |
| Accepts | One image | One image, or 1 to 64 texts of 1 to 1000 characters, never both |
| Returns | One purely visual vector per image | One vector per image or text, in one shared space |
| List price (credits) | 0.05 per image | 0.05 per image, 0.01 per text |
| Pool | Public: available to every registered account, 100 calls per UTC day included | Public: available to every registered account, 100 calls per UTC day included |
| Vector length | 768 in the recorded responses; read `dimensions` | 768 in the recorded responses; read `dimensions` |
| Recorded processing time | 216 ms (one recorded call) | About 200 ms |
| Attribution | "Built with DINOv3" | None |
| Pick when | The query is an image | The query is a sentence or a set of labels, or texts and images are compared |

- Pick `dinov3` for image-to-image similarity, de-duplication and retrieval by example ("find images like this one").
- Pick `siglip2` for text search over images, zero-shot classification, and any case where a text and an image are compared with each other.

## Example call

One image of the first figure, embedded with `dinov3`:

```bash
curl https://api.stack.foldbun.com/v1/embeddings \
  -H "Authorization: Bearer $FOLDBUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dinov3",
    "image": { "id": "img_36aec8cbc5b68661a926869a8bc8aa79de584edbb2aec2454bd1f1c169af09d7" }
  }'
```

The response is abridged: the vector is shortened.

```json
{
  "id": "call_01a1913e-868a-7d5a-841c-5913396a9ea1",
  "model": "dinov3",
  "image": {
    "id": "img_36aec8cbc5b68661a926869a8bc8aa79de584edbb2aec2454bd1f1c169af09d7",
    "width": 1512,
    "height": 1008,
    "bytes": 342140,
    "contentType": "image/jpeg"
  },
  "dimensions": 768,
  "embeddings": [
    { "index": 0, "vector": [-0.007488, -0.036278, -0.000432] }
  ],
  "cost": 0.05,
  "balance": 12.45,
  "timing": { "queueMs": 0, "processingMs": 216 },
  "billing": {
    "lines": [
      { "unit": "image", "quantity": 1, "unitPrice": 0.05, "amount": 0.05, "waived": false }
    ],
    "total": 0.05,
    "charged": true,
    "pool": null
  }
}
```

### Reading the response

- `dimensions` is 768: every vector has that length. Code should read the field and not hard-code the length.
- `embeddings[0]` has `index` 0 and `vector`, the L2-normalised float32 values (3 of 768 shown). `index` is the position in `texts`, or 0 for an image.
- A text-only `siglip2` call returns one entry per text, in input order, and no `image`.

The cosine similarity of two vectors from the same model is their dot product:

```python
def cosine(a: list[float], b: list[float]) -> float:
    return sum(x * y for x, y in zip(a, b))
```

The two vectors come from two calls with the same model, one call per image.

All fields: [API reference](https://stack.foldbun.com/docs/api/operations/embed/).

**Cost of this call.** 0.05 (`image`) credits, as charged in this response. With `siglip2`, an image is also 0.05 and each text 0.01, so the three labels of the flow below cost 0.03. Unit prices: [Billing](https://stack.foldbun.com/docs/billing/#price-list).

### Complete flow in Python

The complete flow in Python, with the image and the labels in two calls:

```python
import os

import requests

API = "https://api.stack.foldbun.com/v1/embeddings"
HEADERS = {"Authorization": f"Bearer {os.environ['FOLDBUN_API_KEY']}"}


def embed(**body):
    r = requests.post(API, headers=HEADERS, json={"model": "siglip2", **body}, timeout=60)
    r.raise_for_status()
    return [e["vector"] for e in r.json()["embeddings"]]


texts = ["a bowl of ramen", "a pizza", "a salad"]
image_vec = embed(image={"id": "img_f4e4f5a8a14f25689a345acf67ea1a83c5760e219c0dcf5255717cdb9bd5e225"})[0]
scores = [sum(a * b for a, b in zip(image_vec, v)) for v in embed(texts=texts)]
for text, score in sorted(zip(texts, scores), key=lambda p: -p[1]):
    print(f"{score:.3f}  {text}")
```

## Endpoints and reference

| Endpoint | In this use case | Reference |
|---|---|---|
| `POST /v1/embeddings` | One vector per image, or per text with `siglip2`, in input order | [API reference](https://stack.foldbun.com/docs/api/operations/embed/) |

- [`dinov3`](https://stack.foldbun.com/docs/models/dinov3/) and [`siglip2`](https://stack.foldbun.com/docs/models/siglip2/): how each model works, its limits and measured latency.
- [Billing](https://stack.foldbun.com/docs/billing/#price-list): `image` for both models, `text` for `siglip2` only.
- [Attribution](https://stack.foldbun.com/docs/models/dinov3/#attribution): the "Built with DINOv3" tag for products built on `dinov3`.