Canonical page: https://docs.singlebase.io/llm

# LLM

Bring model access and AI services alongside your data with `llm.*` operations.

## Overview

**Intelligence** connects models to the data and documents your agents and apps already use. You don't need provider keys or prompt plumbing: send content and instructions, and get results back.

**Models:** Choose a configured catalog model with `model`, or omit it to use `DEFAULT_MODEL` for generation. Available models depend on the deployed catalog; an unknown generation model is rejected. Embeddings use their own model default.

**Key features:**
- **Generation** - Write, translate, summarize, categorize, extract, and analyze sentiment
- **Structured output** - JSON responses shaped by a JSON Schema
- **Embeddings** - Vectors for one or many documents

**Use cases:** Content tools, classification pipelines, data extraction, semantic search preparation

**Access:** Generation and embeddings require a signed-in user (web key plus bearer token), or a server or permitted agent key without a bearer token. `chat.ask` remains public with a valid project key, subject to retrieval and attachment permissions.

For persisted chats with history and Chat with Documents, see [Conversation](/chat.md).

---

## Operations

| Operation | Description | Required payload |
|---|---|---|
| `llm.generate` | Generate content with a named built-in template | `content` or `input` |
| `llm.write` | Write content | `content` or `input` |
| `llm.translate` | Translate content | `content` or `input` |
| `llm.summarize` | Summarize content | `content` or `input` |
| `llm.categorize` | Classify content | `content` or `input` |
| `llm.extract` | Extract entities or data | `content` or `input` |
| `llm.analyze` | Analyze sentiment | `content` or `input` |
| `llm.embed` | Create embeddings | `documents` |

For chats, see [Conversation](/chat.md).

---

## Generation

The seven generation operations share one payload. Each selects a built-in prompt for its task.

**Generation parameters:**

| Field | Description |
|---|---|
| `content` | String, object, or list of strings/objects; up to 200,000 converted characters |
| `input` | Request, up to 4,000 characters; alias `user_input` |
| `output` | `text` (default), `markdown`, or `json`; alias `format` |
| `json_schema` | JSON structure; requires `output: "json"` |
| `model` | Catalog model key; default `DEFAULT_MODEL` |
| `prompt_template` | Built-in template name on `llm.generate` only; default `generic` |
| `system` | Own instruction on `llm.generate` only; at most 8,000 characters |

`content` is what the model reads, `input` is what you ask it to do with it.
At least one is required and nonblank. Objects become structured Markdown; list
items are converted without a `Data Records` title and joined with a blank line.
Object items in multi-item lists have `## Item N` headings numbered by position;
strings and single objects have none. Fixed operations ignore `prompt_template`
and `system`. An explicitly sent `prompt_template` wins over `system`, including
`generic`; otherwise nonblank `system` wins, falling back to `generic`. `system`
alone cannot replace the required content/input. Named templates must exist in the deployed prompt catalog. Unknown names return `PROMPT_NOT_FOUND`.

**Breaking change:** `custom_instructions`, `variables`, `prompt_name`,
`prompt_md`, `params`, and `options` are silently ignored in generation. Use
`input` for instructions. `output` is now an answer shape, not a dot path;
`output: null` is invalid. Embedding and chat keep their own payloads.

**Response:**
```json
{"data": {"result": "SignalPeak\nUptimeLens\nPingWise"}, "meta": {"request_id": "req_123", "exec_time": 0.84}, "exec_time": 0.87}
```

JSON output returns parsed JSON in `data.result`. Each successful generation or
embedding call records the existing `LLM_CALL` usage event.
See [Templates](/templates.md) for template syntax.

### llm.generate

```json
{
  "operation": "llm.generate",
  "payload": {
    "input": "Suggest three names for an API monitoring tool, one per line.",
    "prompt_template": "generic"
  }
}
```

### llm.write

```json
{
  "operation": "llm.write",
  "payload": {
    "content": {"product": "Wireless headphones", "battery_hours": 30},
    "input": "Write a friendly product description, under 60 words."
  }
}
```

### llm.translate

```json
{
  "operation": "llm.translate",
  "payload": {
    "content": "Welcome to our developer platform.",
    "input": "Translate into French."
  }
}
```

### llm.summarize

```json
{
  "operation": "llm.summarize",
  "payload": {
    "content": "The release adds direct file uploads, faster queries, and rotating refresh tokens. Existing clients must adopt the new request envelope.",
    "input": "Two bullets.",
    "output": "markdown"
  }
}
```

### llm.categorize

```json
{
  "operation": "llm.categorize",
  "payload": {
    "content": "I was charged twice for my subscription.",
    "input": "Choose one: billing, technical, general."
  }
}
```

### llm.extract

```json
{
  "operation": "llm.extract",
  "payload": {
    "content": "Jane Doe ordered two notebooks on October 3, 2026.",
    "input": "Extract customer, product, quantity, and order date.",
    "output": "json",
    "json_schema": {
      "type": "object",
      "properties": {
        "customer": { "type": "string" },
        "product": { "type": "string" },
        "quantity": { "type": "integer" },
        "order_date": { "type": "string" }
      }
    }
  }
}
```

### llm.analyze

```json
{
  "operation": "llm.analyze",
  "payload": {
    "content": "Setup was quick and the docs were easy to follow.",
    "input": "Return the sentiment and a one-sentence reason."
  }
}
```

---

## Embeddings

### llm.embed

**Example - One string:**
```json
{
  "operation": "llm.embed",
  "payload": {
    "documents": "A short cancellation policy."
  }
}
```

**Example - Several documents:**
```json
{
  "operation": "llm.embed",
  "payload": {
    "documents": [
      { "id": "policy-1", "content": "Cancel at any time." },
      { "id": "policy-2", "content": "Refunds take five days." }
    ],
    "embedding_content_path": "content"
  }
}
```

**Parameters:**

| Field | Description |
|---|---|
| `documents` | A string, an object, or a non-empty array of strings and objects (required) |
| `embedding_content_path` | Field to embed in each object. Default `"content"`. Supports dot paths |
| `model` | Embedding model |
| `dimensions` | Positive vector size, when supported by the selected embedding model |
| `params` | Extra provider options |

Results keep the input order. Embedding does not store anything; save the vectors yourself if you need them.

---

## Common Errors

| Message | Meaning |
|---|---|
| `AUTHENTICATION_REQUIRED`, `ACCESS_DENIED` | Missing user authentication or insufficient access for generation/embeddings |
| `PROMPT_NOT_FOUND` | The generation template does not exist |
| `CONTENT_TOO_LARGE` | Converted generation content exceeds 200,000 characters |
| `INVALID_LLM_EMBED_PAYLOAD` | Embedding inputs or options are invalid |
| `INVALID_PAYLOAD` | The payload does not match the operation |
| `KB_SOURCE_REQUIRED` | `kb` mode has no retrieval source or attachment |
| `RETRIEVAL_SOURCE_FORBIDDEN` | A web-key caller used a server-only source |
| `RETRIEVAL_SOURCE_MISMATCH` | `source` and `type` disagree |
| `DOCSET_NOT_FOUND` | A docset is missing, disabled, or not available to the caller |
| `INVALID_RETRIEVAL` | Invalid docset IDs, or the client sent docset definitions |
| `INVALID_QUERY_FOR_SOURCE` | The retrieval query does not fit its source |
| `ATTACHMENT_LIMIT` | More than 5 attachments |
| `ATTACHMENT_TOO_LARGE` | Inline or extracted attachment content is too large |
| `FILE_FORBIDDEN` | The user does not own the attached file |
| `LLM_REQUEST_FAILED` | The chat handler failed (HTTP 502) |

Model validation failures return HTTP 422. Other generation or embedding failures return HTTP 502 with the underlying error message; do not assume one fixed message for every model failure. See [Responses](/api.md#responses) for the error envelope.

---

## Next Steps

- [Conversation](/chat.md) - Persisted chats and Chat with Documents
- [Data](/data.md) - Store the content your docsets search
- [Agents](/agents.md) - Give AI agents scoped LLM permissions

Use [Chat](/chat.md#chat-ask) for one-off answers and persisted conversations, and [JavaScript SDK](/sdk-javascript.md#llm-generation-and-embeddings) for `sbc.llm` calls.