# Indx Documentation — Complete Reference

C# API version: 5.0.0
Source: https://docs.indx.co


---

# Build with an AI Agent

The fastest way to integrate Indx in 2026 is often not to read these docs yourself — it is to hand them to your coding agent. Indx ships three pieces that make agent-driven integration work end to end: an **agent skill** that teaches your agent how to build with Indx, **agent-readable docs** for everything else, and a **built-in MCP server** so the agent can search real data and verify its own work.

Point your agent at the right piece below, then ask it to integrate Indx into your project.

---

## 1. Install the Indx skill (Claude Code)

The [indx-search skill](https://github.com/indxSearch/skill-indx-search) packages what an agent needs to build correct Indx integrations: version detection (v4 vs v5), the C# and HTTP API surfaces, field configuration, filters, boosts, coverage tuning, and the mistakes to avoid.

Install from [skills.sh](https://skills.sh/indxsearch/skill-indx-search/indx-search), or manually:

```bash
# from your project root — the skill loads automatically when Indx work comes up
git clone https://github.com/indxSearch/skill-indx-search.git .claude/skills/indx-search
```

(User-wide instead: clone into `~/.claude/skills/` to have it in every project.)

With the skill in place, prompts like these produce working integrations:

- *"Add Indx search to this project. Products are in `products.json`; name and brand should be searchable, category and price filterable."*
- *"Set up an IndxCloudApi dataset for this data over the REST API, with facets on genre."*

## 2. Give any other agent the docs

For agents without skill support (Cursor, Copilot, custom frameworks), this entire documentation site is published as one agent-readable file:

```
https://docs.indx.co/llms-full.txt
```

Paste the URL into your agent's context, or reference it from your project's agent instructions (`AGENTS.md`, `.cursorrules`, or similar).

## 3. Connect the agent to a running server (MCP)

Every [IndxCloudApi](https://github.com/indxSearch/IndxCloudApi) ships an **MCP server** at `/mcp`. Connected, your agent does not just write the integration — it **searches your live datasets to verify what it built**: check that the field configuration behaves, that filters use real values, that a typo still finds the right document.

Create an API key in the portal (Account → API keys), then:

```bash
# Claude Code
claude mcp add --transport http indx https://<your-host>/mcp \
  --header "Authorization: Bearer <your-api-key>"
```

Or in any MCP client's JSON configuration:

```json
{
  "mcpServers": {
    "indx": {
      "type": "http",
      "url": "https://<your-host>/mcp",
      "headers": { "Authorization": "Bearer <your-api-key>" }
    }
  }
}
```

The server is **read-only** — agents can search, never mutate — and exposes four tools: `list_datasets`, `describe_dataset` (fields, capabilities, and value hints, so agents filter with real values instead of guessing), `search`, and `get_document`. Full tool reference: [Using Indx for AI Agents](/howto/agent-search).

## A workflow that uses all three

1. Install the skill (or feed the docs URL) — the agent knows *how* to build.
2. Ask it to integrate Indx with your data — field config, load, index, queries.
3. Connect MCP — the agent searches what it built and corrects itself against real results.

That last step is the difference between generated code that *looks* right and an integration verified against a running engine.

---

## Why Indx suits agents

Beyond integration-time help, Indx is a good runtime fit for agent systems themselves: embedded via NuGet it runs in the agent's own process with sub-millisecond queries — no network round-trip, no provisioning, no cold starts. See [Using Indx for AI Agents](/howto/agent-search) for the full picture, including boost rules over MCP.


---

# Cloud API Setup

IndxCloudApi is a self-hosted server that wraps the Indx search engine and exposes it over HTTP. It has two interfaces:

- **Web UI** — a Blazor dashboard where you can register users, upload and configure datasets, preview search results, manage teams, and check engine health. No code required.
- **REST API** — the full HTTP API, used from any language or framework. All operations available in the UI are also available programmatically.

Swagger UI is available at `/swagger` for interactive API exploration.

---

## Quick Start

```bash
git clone https://github.com/indxSearch/IndxCloudApi
cd IndxCloudApi
dotnet run
```

Requires **.NET 10 SDK**. The server starts at `https://localhost:5001`.

---

## Register a User

1. Open `https://localhost:5001/Account/Register` in a browser
2. Enter your email and password
3. Submit — you're now logged in as the first user

> By default, registration is open. See [Registration Control](#registration-control) below to restrict who can sign up before deploying to production.

---

## Get an API Token

To call the REST API you need a JWT token.

**From the UI:**

1. Log in at `https://localhost:5001/Account/Login`
2. Go to `https://localhost:5001/Account/ApiKey`
3. Select a token duration: **30**, **90**, **180**, or **360** days
4. Click **Generate API Token** and copy it

Use the token in all subsequent requests:

```bash
curl -H "Authorization: Bearer <token>" https://localhost:5001/api/...
```

---

## What You Can Do in the UI

| Feature | Description |
|---------|-------------|
| **Upload data** | Stream a JSON file into a dataset |
| **Configure fields** | Set searchable, filterable, facetable, sortable, and weight for each field |
| **Index** | Trigger an index build and monitor progress |
| **Search preview** | Run searches directly in the browser and inspect results |
| **Health & status** | See engine state, document count, index time, and whether a shadow build is in progress |
| **Teams** | Manage team membership and roles (Admin / Editor / Viewer) |
| **Transfer dataset** | Move a dataset to another team |

---

## Teams

IndxCloudApi organises access around **teams**. A team owns datasets, and every member of the team has the same role on all of the team's datasets. A user can belong to multiple teams, with a different role on each. Membership is managed from the account portal's team pages.

### Roles

| Role | Can search | Can modify data & fields | Can delete / transfer |
|------|-----------|--------------------------|----------------------|
| `Admin` | ✓ | ✓ | ✓ |
| `Editor` | ✓ | ✓ | — |
| `Viewer` | ✓ | — | — |

### Transferring a dataset between teams

An Admin of both the source and target team can move a dataset:

```bash
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/transfer \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "targetTeamName": "research" }'
```

---

## Deploy to Azure App Service

1. Create an App Service with **Windows** OS and **.NET 10** runtime
2. Publish with `dotnet publish -c Release` and deploy
3. Set environment variables (use `__` as the section separator):
   - `Jwt__Key` — required
   - `Registration__Mode` — recommended: `EmailDomain` or `Closed`
4. Configure CORS with your frontend origins
5. Place the `.license` file in `D:\home\data\` for unlimited documents

### Production Checklist

1. Change the JWT signing key
2. Set registration mode
3. Configure CORS
4. Set up an email provider if you want email confirmation
5. Place the license file
6. Set up OAuth if needed

---

## Configuration Reference

All settings live in `appsettings.json` or as environment variables.

### JWT Security

Required for production — the default key is not secure:

```bash
dotnet user-secrets set "Jwt:Key" "your-secret-key-minimum-32-characters"
```

### Registration Control

Controls who can create an account:

```json
{
  "Registration": {
    "Mode": "EmailDomain",
    "AllowedDomains": ["yourcompany.com"]
  }
}
```

| Mode | Behavior |
|------|----------|
| `Open` | Anyone can register (default) |
| `EmailDomain` | Only emails from the listed domains |
| `Closed` | No new registrations |

### Email Provider

IndxCloudApi uses Azure Communication Services for email (password reset, confirmation). If not configured, all emails are printed to the console instead — fine for development.

```json
{
  "AzureCommunicationServices": {
    "ConnectionString": "endpoint=https://...",
    "SenderAddress": "noreply@yourcompany.com"
  }
}
```

### OAuth (Optional)

Google and Microsoft login are supported. Configure via user-secrets or environment variables:

```json
{
  "Authentication": {
    "Google": {
      "ClientId": "...",
      "ClientSecret": "..."
    },
    "Microsoft": {
      "ClientId": "...",
      "ClientSecret": "..."
    }
  }
}
```

### Database

Two SQLite databases are created automatically on first run — no setup required:

- `identity.db` — user accounts and authentication
- `indx.db` — dataset metadata and configuration

Search indexes are held in memory only and must be reloaded after a restart.

### License

Place the `.license` file in `./IndxData/` (local) or `D:\home\data\` (Azure) to remove the 100,000 document limit. See [Licensing](/guides/licensing).


---

# Core Concepts

## How Search Works

Every search runs in two phases.

**Phase 1 — Pattern Matching.** The engine scans all documents using an inverted index with BM25 scoring. This phase has strong recall: it tolerates typos, partial words, and messy input without any preprocessing. The result is a ranked list of candidates.

**Phase 2 — Coverage.** The top 500 candidates (configurable) are re-evaluated for exact and near-exact token matches: whole words, fuzzy words, joined/split words, prefixes, and suffixes. Documents with confirmed token matches are rescored on a 16-bit scale and promoted above pure pattern-match results.

Coverage is enabled by default and fast enough that it doesn't add significant latency. Leave it on.

A search for *"Wirless hedphones"* will still surface *"Wireless Headphones"* — the pattern phase catches it despite the typos, and the coverage phase confirms the near-exact match (edit distance 1 on each word).

A heavily garbled query like *"Wirlss edphons"* will still return results — the pattern phase finds candidates — but coverage won't confirm any near-exact matches. In that case you get ranked results without a truncation point and without facet counts. The results are useful but less precise.

---

## Field Roles

Fields don't do anything until you explicitly assign them a role. You configure this after calling `Init` (which discovers the field names) and before calling `Index`.

| Role | What it does |
|------|-------------|
| **Searchable** | The field is included in matching and scoring. At least one field must be searchable. |
| **Filterable** | The field can be used to filter results before or during a search. |
| **Facetable** | The field returns value counts in search results — useful for sidebar filters. |
| **Sortable** | Results can be ordered by this field's value. Works on both numbers and strings. |

A field can have multiple roles — a `category` field can be both filterable and facetable at the same time.

### Weight

Searchable fields have a `weight` (default `1.25`) that controls how much influence that field has on the final score. If `name` should matter more than `description`, give it a higher weight.

```csharp
engine.GetField("name")!.Weight = 2.0f;
engine.GetField("description")!.Weight = 1.0f;
```

Weight is a relative value — it only matters in proportion to the weights of the other searchable fields.

---

## Filters Are Server-Side

A search returns only a limited slice of results (`maxNumberOfRecordsToReturn`). If you filter that slice yourself after receiving it, you will miss matching documents that weren't in the top results.

Always pass filters to the engine so they are applied during search — the engine evaluates them against the full dataset, not just the returned page.

```csharp
// Wrong: filter the array you got back
var results = engine.Search(query).Entries.Where(e => ...);

// Right: create a filter and give it to the query
var filter = engine.CreateValueFilter("category", "audio", out _);
var query  = new Query("wireless", 20) { Filter = filter };
var result = engine.Search(query);
```

The same applies to the HTTP API: create a filter server-side first, then pass its `hashString` in the search request.

---

## Scoring

Results have a 16-bit `score` (0–65535). Coverage hits (exact and near-exact matches) always score higher than pure pattern-match results, which appear below them.

When sorting is applied alongside a text search, the score remains the primary sort — the sort field is only used as a tiebreaker. When searching with no query text (empty search), sorting becomes the primary ordering.

---

## The Engine Lifecycle

The engine must go through a fixed sequence before it can search:

```
Init()   →   configure fields   →   Load()   →   Index()   →   Search()
```

- **Init** — reads the JSON structure and discovers field names
- **Configure** — set roles and weights on the fields you care about
- **Load** — streams the JSON documents into the engine
- **Index** — builds the inverted index; engine enters the Ready state
- **Search** — available once the engine is Ready

After indexing you can also insert, update, and delete individual documents without rebuilding the index — see [Dynamic Operations](/csharp/SearchEngine).

---

## Empty Search

An empty or null query string is valid, but requires `EnableFacets = true` — without it, no documents are returned. The result contains up to `maxNumberOfRecordsToReturn` documents, while facets are calculated across the entire dataset.

Results are ordered by sort field if one is specified, making empty search the natural companion to sorting and faceted navigation.


---

# Getting Started

Indx is a high-performance full-text search engine for JSON data. It uses pattern recognition and BM25 scoring — no tokenizers, stemmers, or analyzers to configure. It handles typos and messy input out of the box.

## Choose Your Path

**C# / .NET** — embed search directly with the [IndxSearchLib NuGet package](https://www.nuget.org/packages/IndxSearchLib/). No external services required.

**Any other stack** — run [IndxCloudApi](https://github.com/indxSearch/IndxCloudApi) and call the REST API from Node.js, Python, Java, or anything else.

---

## C# Walkthrough

### 1. Install

```bash
dotnet add package IndxSearchLib
```

### 2. Prepare Your Data

Indx accepts a JSON array of objects. Nested objects are supported; nested fields use dot notation.

```json
[
  { "id": 1, "name": "Wireless Headphones", "brand": "Acme", "price": 79.99, "category": "audio" },
  { "id": 2, "name": "Bluetooth Speaker",   "brand": "Zync", "price": 49.99, "category": "audio" },
  { "id": 3, "name": "USB-C Hub",           "brand": "Acme", "price": 29.99, "category": "accessories" }
]
```

### 3. Init — Discover Fields

`Init` parses the JSON structure and returns the list of fields it discovers, along with each field's inferred type. The fields start unconfigured — you assign roles (searchable, filterable, sortable, facetable) in the next step.

```csharp
using Indx.Api;

var engine = new SearchEngine();

using var stream = File.OpenRead("products.json");
engine.Init(stream);
```

After `Init`, call `GetFieldConfiguration()` to inspect the discovered fields:

```csharp
foreach (var field in engine.GetFieldConfiguration())
    Console.WriteLine($"{field.FieldName}  type={field.FieldType}");
// id       type=Number
// name     type=String
// brand    type=String
// price    type=Number
// category type=String
```

### 4. Configure Fields

Nothing is searchable by default. Mark each field's role explicitly:

```csharp
var name     = engine.GetField("name")!;
var brand    = engine.GetField("brand")!;
var price    = engine.GetField("price")!;
var category = engine.GetField("category")!;

name.Searchable  = true;
name.Weight      = 2.0f;   // higher weight = more influence on score

brand.Searchable  = true;
brand.Weight      = 1.0f;

price.Filterable = true;
price.Sortable   = true;

category.Filterable = true;
category.Facetable  = true;
```

`Weight` is a `float` (default `1.25`). Higher means stronger influence on BM25 scoring relative to other searchable fields.

### 5. Load and Index

Stream the same JSON again for loading, then build the index:

```csharp
stream.Position = 0;
engine.Load(stream);
engine.Index();
```

The engine is now in **Ready** state.

### 6. Search

```csharp
var result = engine.Search(new Query("wireless", 20));

foreach (var entry in result.Entries)
    Console.WriteLine($"key={entry.DocumentKey}  score={entry.Score}");
```

To retrieve the full JSON of the matching documents:

```csharp
var keys = result.Entries.Select(e => e.DocumentKey).ToArray();
var docs = engine.GetJsonDataOfKeys(keys);
```

### 7. Add a Filter

Filters are evaluated server-side against the full dataset — never filter the result array yourself.

```csharp
// Match only the "audio" category
var audioFilter = engine.CreateValueFilter("category", "audio", out _);

var query = new Query("wireless", 20) { Filter = audioFilter };
var result = engine.Search(query);
```

For numeric ranges:

```csharp
var affordable = engine.CreateRangeFilter("price", 0, 50, out _);
```

Combine with AND or OR:

```csharp
var combined = audioFilter & affordable; // AND; use | for OR, ! to negate
```

### 8. Sort Results

Sorting is secondary to score. However, an empty search (Query.Text = "", see [Empty Search](/howto/empty-search)) results in all scores being 0, in which case the sort order behaves as primary.

```csharp
var query = new Query("", 20) { SortBy = "price", SortAscending = true };
```

### 9. Facets

Facets return distinct value counts for a field — useful for sidebar filters.

```csharp
var query = new Query("wireless", 20) { EnableFacets = true };
var result = engine.Search(query);

foreach (var (value, count) in result.Facets["category"])
    Console.WriteLine($"{value}: {count}");
// audio: 2
// accessories: 0
```

---

## HTTP API Walkthrough

Run the server:

```bash
git clone https://github.com/indxSearch/IndxCloudApi
cd IndxCloudApi
dotnet run
```

Requires **.NET 10 SDK**. See [Cloud API Setup](/guides/cloud-setup) for registration, API keys, and deployment.

### Load Data

```bash
# Analyze structure
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/analyze \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  --data-binary @products.json

# Configure fields (204 No Content)
curl -X PUT https://localhost:5001/api/teams/acme/datasets/products/fields/configuration \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '[
    { "fieldName": "name",     "searchable": true, "weight": 2.0 },
    { "fieldName": "brand",    "searchable": true, "weight": 1.0 },
    { "fieldName": "price",    "filterable": true, "sortable": true },
    { "fieldName": "category", "filterable": true, "facetable": true }
  ]'

# Load documents (204 No Content)
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/load \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  --data-binary @products.json

# Build index — returns 202 Accepted; the build runs in the background
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/index \
  -H "Authorization: Bearer <token>"

# Poll status until systemState is 4 (Ready)
curl https://localhost:5001/api/teams/acme/datasets/products/status \
  -H "Authorization: Bearer <token>"
```

### Search

```bash
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/search \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "text": "wireless", "maxNumberOfRecordsToReturn": 20 }'
```

### Filter and Search

```bash
# Create a filter
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/filters/value \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "fieldName": "category", "value": "audio" }'
# → { "hashString": "abc123..." }

# Use it in search
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/search \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{ "text": "wireless", "maxNumberOfRecordsToReturn": 20, "filter": { "hashString": "abc123..." } }'
```

---

## Next Steps

- [Core Concepts](/guides/concepts) — Pattern matching, coverage, and scoring explained
- [C# API Reference](/csharp/overview) — Full method and type reference
- [HTTP API Reference](/http/overview) — All endpoints, request/response schemas
- [How To guides](/howto/field-configuration) — Specific recipes: facets, boosts, deep coverage, and more


---

# Licensing

## License Tiers

| Tier | Document Limit | Cost |
|------|---------------|------|
| **No license** | 100,000 documents | Free |
| **Extended license** | Unlimited | Free — get one at [license.indx.co](https://license.indx.co) |
| **Company license** | Unlimited + SLA and support | Paid |

## Using a License

### C# NuGet

Pass the license path to the `SearchEngine` constructor:

```csharp
var engine = new SearchEngine("indx-developer.license");
```

Or place the `.license` file in the working directory — it will be detected automatically.

### HTTP API (IndxCloudApi)

Place the `.license` file in the `./IndxData/` directory. The server auto-detects it on startup.

For Azure deployments, place the file in `D:\home\data\`.

## Getting a License

If you self-host IndxCloudApi — or use the C# NuGet library — obtain a license at [license.indx.co](https://license.indx.co). The free extended license file removes the 100,000 document limit.

For company licenses with SLA and dedicated support, contact [support@indx.co](mailto:support@indx.co).


---

# Vector & Hybrid Search

Standard text search matches words and patterns. Vector search matches meaning. The two approaches complement each other — hybrid search lets you blend them.

## When to Use It

Text search is precise and fast. It excels at names, codes, partial words, and typo-tolerant lookups — the things Indx is already very good at. But it can miss results when the user's phrasing doesn't match the document's wording.

Vector search closes that gap. A query like *"something comfortable for long flights"* can surface *"noise cancelling over-ear headphones"* even though none of those words overlap. This is especially valuable when:

- Your data is **content-rich** — product descriptions, articles, documentation, support tickets, legal or medical text
- Users **don't know the exact terminology** and search with natural language
- You want a **"more like this"** or **"related items"** feature
- Your data is **multilingual** and users may search in a different language than the documents

Vector search is less useful for short structured fields, product codes, or cases where exact matching is the goal — text search handles those better.

### Downsides of Pure Vector Search

- **No truncation point.** Text search with coverage produces a clear boundary between confirmed matches and loose pattern hits — results below the truncation index are lower confidence. Vector search returns a ranked list by similarity with no such boundary; every result has a score, but there is no reliable threshold that separates "relevant" from "not relevant".
- **No facets.** Facet counts are computed from coverage hits. Vector search does not produce coverage results, so facet aggregations are not available.

These limitations make pure vector search a poor fit for faceted navigation or UIs that rely on the truncation point for quality signals. **Hybrid search** is the recommended default — it runs the text search path alongside vector search and merges the results, so you retain truncation points, facet support, and typo tolerance while gaining semantic reach.

---

## How Vector Search Works

Instead of indexing words, vector search stores each document as a high-dimensional float array (an *embedding*) produced by an embedding model. At search time, the query is converted to a vector by the same model, and the engine finds the documents whose vectors are closest to the query vector using approximate nearest-neighbour (HNSW) search. Similarity is measured as cosine similarity — a score in `[0, 1]` where `1` means identical.

**Indx stores and searches vectors. It does not generate them.** You are responsible for producing the float arrays using an embedding model of your choice — OpenAI, sentence-transformers, Cohere, or any other. Indx accepts vectors as a JSON array field on each document.

---

## Choosing a Mode

| Mode | When to use |
|------|-------------|
| **Text search** | Precise lookups, product codes, names, exact phrases, typo-tolerant queries |
| **Vector search** | Semantic similarity — "find products like this one", concept search, multilingual queries |
| **Hybrid search** | Most search UIs — combines the precision of text with the semantic reach of embeddings |

Hybrid search is the recommended default when you have embeddings. Pure vector search is better suited to "more like this" features or cases where the query is itself a document rather than a text string.

---

## Choosing an Embedding Model

**The most important rule: use the same model for documents and queries.** Vectors from different models are not compatible.

| Model | Dims | Notes |
|-------|------|-------|
| OpenAI `text-embedding-3-small` | 1536 | Good starting point — low cost, strong quality |
| OpenAI `text-embedding-3-large` | 3072 | Higher quality, higher cost |

For most applications, **`text-embedding-3-small`** is the right choice.

### Token Limits

All models have a maximum input length. OpenAI's models support up to 8191 tokens per input. If your documents are longer, split them into chunks before embedding — each chunk becomes a separate document in Indx.

### L2 Normalisation

Indx requires unit-length (L2-normalised) vectors. OpenAI's embedding API returns normalised vectors by default, so no extra step is needed when using the recommended models.

---

## Getting Vectors — C# NuGet Options

**OpenAI** — official .NET SDK

```bash
dotnet add package OpenAI
```

```csharp
using OpenAI.Embeddings;

var client = new EmbeddingClient("text-embedding-3-small", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
Embedding result = await client.GenerateEmbeddingAsync("comfortable wireless headphones");
float[] vector = result.ToFloats().ToArray();
```



---

## Setup Overview

### 1. Mark the Field as Embeddable

```csharp
engine.GetField("embedding")!.Embeddable = true;
```

Embeddable fields are excluded from the text search index. The field must contain a JSON array of floats.

### 2. Store Vectors in Your Documents

Each document must include the embedding as a JSON array:

```json
{
  "id": 1,
  "title": "Wireless Headphones",
  "description": "Over-ear noise cancelling headphones",
  "embedding": [0.12, -0.45, 0.33, 0.08, ...]
}
```

The vector must be **L2-normalised** (unit length). The dimensionality is detected from the first document loaded and must be consistent across all documents.

### 3. Generate Embeddings at Query Time

When a user searches, embed the query text with the same model you used for the documents:

```csharp
// Example using any embedding client
float[] queryVector = await myEmbeddingClient.EmbedAsync("comfortable wireless headphones");
```

---

## Vector Search (C#)

After `Load()` and `Index()`, access the embedding index via `EmbeddingFields`:

```csharp
var index = engine.EmbeddingFields["embedding"];
var results = index.Search(queryVector, maxResults: 10);

foreach (var (documentKey, score) in results)
    Console.WriteLine($"key={documentKey}  similarity={score:F3}");
```

Returns `IReadOnlyList<(long documentKey, float score)>` ordered by descending cosine similarity.

---

## Hybrid Search (C#)

Run text search and vector search separately, then merge:

```csharp
var textResult     = engine.Search(new Query("comfortable headphones", 50));
var embeddingIndex = engine.EmbeddingFields["embedding"];
var vectorResults  = embeddingIndex.Search(queryVector, maxResults: 50);

var merged = IEmbeddingIndex.MergeHybrid(
    textResults:      textResult.Records,
    embeddingResults: vectorResults,
    alpha:            0.6f   // 0 = pure text, 1 = pure embedding
);
```

The combined score is:

```
score = alpha × embeddingScore + (1 − alpha) × (textScore / 65535)
```

Documents that appear in only one result set receive a zero contribution from the missing side. `alpha` around `0.5–0.7` works well in most cases.

---

## HTTP API

See [Search](/http/Search) for the `search/vector` and `search/hybrid` endpoint reference.

---

## See Also

- [How To: Set Up Vector and Hybrid Search](/howto/vector-search) — complete walkthrough with both C# and HTTP examples
- [Field Configuration](/csharp/Field) — `Embeddable`, `EmbeddingDimensions`


---

# How To Guides


---

# :pixl-Ai_agent: Using Indx for AI Agents

> Getting started with an agent instead? The onboarding path — skill install, agent-readable docs, MCP connection — is in [Build with an AI Agent](/guides/ai-agents). This page is the runtime and MCP tool reference.

Indx is a good fit for AI agents. It is a lightweight embedded library with no external service dependencies — when used via the C# NuGet package, it runs inside the agent's own process with sub-millisecond query times. There is no network round-trip, no managed service to provision, and no cold-start latency. Agents can fire as many searches as needed without debouncing or batching.

When used via the HTTP API, IndxCloudApi is equally fast — search is in-memory and designed for repeated high-frequency queries.

---

## Model Context Protocol (MCP)

IndxCloudApi ships a built-in **MCP server**, so any MCP client — Claude Desktop, Claude Code, or your own agent framework — can search your datasets with no glue code. It exposes the same in-memory engine, and your saved **boost rules**, over the Model Context Protocol.

**Endpoint:** `https://<your-host>/mcp` — authenticate with an API key (Account → API keys) as a `Bearer` token. Admins can turn the server on or off under Admin → Settings.

The server is **read-only** and exposes four tools:

- **`list_datasets`** — datasets the token can reach, with document counts and state.
- **`describe_dataset`** — the queryable surface: configured fields with their capabilities (searchable / filterable / facetable / sortable) plus **value hints** — the distinct values for facetable fields and numeric ranges — an owner-authored description, and a sample document. Call this first so the agent filters with real values instead of guessing.
- **`search`** — ranked results with relevance scores. Declarative filters on filterable fields (`{field, value}` or `{field, min, max}`), optional facets, and full documents (or a projected subset).
- **`get_document`** — the full JSON document for a key.

### Precision by default

`search` uses the same precision-first matching described below — near-exact only (`IncludePatternMatches = false`), with typo tolerance retained. An empty result is a trustworthy *"nothing matches"*, so the agent can stop or ask rather than hallucinate. Pass `broaden: true` to opt into broad pattern matching for a single call. Saved boost rules are applied automatically, so agent searches inherit your ranking policy.

### Connecting a client

Add the server with the endpoint URL and your API key as a Bearer token. Clients without a custom-header field can use the `mcp-remote` bridge:

```json
{
  "mcpServers": {
    "indx": {
      "command": "npx",
      "args": ["mcp-remote", "https://<your-host>/mcp",
               "--header", "Authorization: Bearer <your-api-key>"]
    }
  }
}
```

A typical agent flow: `describe_dataset` to learn the fields and valid filter values → `search` with text and filters → `get_document` for full detail on a hit.

---

## Precision Over Recall

Human search benefits from typo tolerance and fuzzy matching. Agents query with well-formed text and need a clear signal: *did something match, or not?* Configure coverage to reflect that.

### Recommended Coverage Setup

```csharp
var cov = new CoverageSetup
{
    CoverFuzzyWords    = false,  // agents don't make typos
    CoverWholeWords    = true,
    CoverPrefixSuffix  = true,
    CoverJoinedWords   = true,
    IncludePatternMatches = false, // only return confirmed hits — no soft pattern results
};

var query = new Query("invoice payment overdue", 20) { CoverageSetup = cov };
var result = engine.Search(query);
```

With `IncludePatternMatches = false`, the result contains only coverage-confirmed matches. If nothing was confirmed, the result is empty — a clean no-match signal the agent can act on.

### HTTP API

```json
{
  "text": "invoice payment overdue",
  "maxNumberOfRecordsToReturn": 20,
  "coverageSetup": {
    "coverFuzzyWords": false,
    "coverWholeWords": true,
    "coverPrefixSuffix": true,
    "coverJoinedWords": true,
    "includePatternMatches": false
  }
}
```

---

## Using the Truncation Point

When `IncludePatternMatches` is `true` (or not set), results above `truncationIndex` are coverage-confirmed and results below are softer pattern matches. An agent can use this boundary as a quality gate:

```csharp
var result = engine.Search(query);
var confirmed = result.TruncationIndex >= 0
    ? result.Records[..result.TruncationIndex]
    : result.Records;
```

---

## Multiple Queries for Better Recall

Agents can compensate for turning off fuzzy matching by issuing multiple searches with reformulated queries — synonyms, related terms, or decomposed phrases. Merge results by document key, keeping the highest score per document:

```csharp
var queries = new[] { "payment overdue", "outstanding invoice", "unpaid bill" };

var merged = new Dictionary<long, ushort>();
foreach (var text in queries)
{
    var r = engine.Search(new Query(text, 20) { CoverageSetup = cov });
    foreach (var entry in r.Records)
    {
        if (!merged.TryGetValue(entry.DocumentKey, out var existing) || entry.Score > existing)
            merged[entry.DocumentKey] = entry.Score;
    }
}

var ranked = merged.OrderByDescending(kv => kv.Value).ToList();
```

Because Indx queries are fast, firing three to five searches adds negligible latency compared to a single LLM call.

---

## Structured Lookups with Filters

When the agent knows field values, use filters instead of relying on text search alone:

```csharp
var statusFilter = engine.CreateValueFilter("status", "overdue", out _);
var query = new Query("payment", 20)
{
    Filter = statusFilter,
    CoverageSetup = cov
};
```

Combine filters for multi-condition lookups:

```csharp
var overdueFilter   = engine.CreateValueFilter("status", "overdue", out _);
var highValueFilter = engine.CreateRangeFilter("amount", 10000, int.MaxValue, out _);
var combined        = overdueFilter & highValueFilter; // AND; use | for OR, ! to negate
```

---

## Letting the Agent Build the Index

An agent doesn't need a pre-existing dataset. It can spin up a fresh in-memory index from scratch — useful when the agent is handed a large JSON file, a collection of PDFs, or any content it discovers at runtime. The engine is initialised from the first batch of documents and made searchable immediately.

```csharp
var engine = new SearchEngine();

// Build the schema from your first batch of documents
using var stream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(firstBatchJson));
engine.Init(stream);

// Configure which fields to search
engine.GetField("title")!.Searchable   = true;
engine.GetField("content")!.Searchable = true;

// Load the first batch and build the initial index
stream.Position = 0;
engine.Load(stream);
engine.Index();

// Insert additional documents as the agent discovers them — no rebuild needed
engine.InsertJsonRecords(additionalDocuments, monitor: null, error: out _);
```

This pattern works well when an agent processes a large document set at task start: parse the content into JSON chunks, load them all, then search freely. Because the engine is embedded and in-memory, the full pipeline — from raw files to searchable index — completes in seconds even for tens of thousands of documents.

### Incremental Updates

Agents that discover or generate data over time can continue inserting without rebuilding:

```csharp
// Single document
engine.InsertJsonRecord(
    documentKey: 42,
    json: """{"id":42,"title":"Q2 Report","content":"Revenue up 12%..."}""",
    monitor: null,
    error: out _
);

// Batch
engine.InsertJsonRecords(newDocuments, monitor: null, error: out _);
```

Updates and deletes work the same way — the index stays in sync after every operation. A full `Index()` rebuild is never required after mutations, though calling it periodically re-normalises scoring for better result quality over time.

---

## Empty Search for Enumeration

An agent that needs to enumerate or scan all documents can use an empty search with sorting and filters:

```csharp
var query = new Query("", 100)
{
    EnableFacets = true,
    SortBy       = engine.GetField("date")!,
    SortAscending = false,
    Filter        = statusFilter
};
```

Facets are calculated across the full dataset regardless of `maxNumberOfRecordsToReturn`, giving the agent a complete picture of value distributions without retrieving all documents.


---

# :pixl-Json_query: Analyze JSON Field Structure with Init

Call `Init` to discover the field structure from your JSON, then use `GetFieldConfiguration()` to inspect the result.

## C#

```csharp
using var stream = File.OpenRead("products.json");
engine.Init(stream);

foreach (var field in engine.GetFieldConfiguration())
{
    Console.WriteLine($"{field.FieldName} ({field.FieldType}){(field.IsArray == true ? " IsArray" : "")}");
}
```

## HTTP API

```bash
# Analyze structure
curl -X POST https://your-host/api/teams/acme/datasets/products/analyze \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  --data-binary @products.json

# Retrieve discovered fields
curl https://your-host/api/teams/acme/datasets/products/fields/configuration \
  -H "Authorization: Bearer <token>"
```


---

# :pixl-Filter: Set Up Filter or Boost with Multiple Conditions

A filter, or a boost that uses a filter, can be a combination of conditions. Imagine a scenario where you want to boost all Documentaries by medium strength if produced after 1980, and low strength if produced earlier.

## C#

```csharp
// Assuming fields "year" and "genre" have been set to Filterable = true

Filter yearOlderFilter = engine.CreateRangeFilter("year", 1900, 1979, out _)!;
Filter yearNewerFilter = engine.CreateRangeFilter("year", 1980, 2025, out _)!;
Filter genreFilter = engine.CreateValueFilter("genre", "Documentary", out _)!;

var boosts = new List<Boost>();
boosts.Add(engine.CreateBoost(yearNewerFilter! & genreFilter, BoostStrength.Med));
boosts.Add(engine.CreateBoost(yearOlderFilter! & genreFilter, BoostStrength.Low));

query.Boosts = boosts.ToArray();
query.EnableBoost = true;
```

## HTTP API

```bash
# Dataset endpoints are team-scoped; set BASE once:
BASE=".../api/teams/<team>/datasets/movies"

# Create filters
curl -X POST "$BASE/filters/range" -d '{"fieldName":"year","lowerLimit":1980,"upperLimit":2025}'
# → {"hashString":"newer123"}

curl -X POST "$BASE/filters/range" -d '{"fieldName":"year","lowerLimit":1900,"upperLimit":1979}'
# → {"hashString":"older456"}

curl -X POST "$BASE/filters/value" -d '{"fieldName":"genre","value":"Documentary"}'
# → {"hashString":"genre789"}

# Combine: newer AND documentary
curl -X POST "$BASE/filters/combine" \
  -d '{"a":{"hashString":"newer123"},"b":{"hashString":"genre789"},"useAndOperation":true}'
# → {"hashString":"combined1"}

# Search with boosts
curl -X POST "$BASE/search" \
  -d '{"text":"film","maxNumberOfRecordsToReturn":20,"enableBoost":true,"boosts":[{"boostStrength":2,"filterProxy":{"hashString":"combined1"}}]}'
```


---

# :pixl-Coverage: Control Coverage Settings

Coverage detects whole or split words, prefixes, suffixes and near-exact hits with minor typos. It returns an index number to truncate the result list.

Coverage has default values for all properties, with most functions enabled. In this example we turn off prefix, suffix, and fuzzy words — a mode to only look for whole words with a sharp cutoff.

## C#

```csharp
CoverageSetup cov = new CoverageSetup();

// Disable functions
cov.CoverFuzzyWords = false;
cov.CoverPrefixSuffix = false;
cov.CoverWholeQuery = false;

// Enable functions and set word size
cov.CoverJoinedWords = true;
cov.CoverWholeWords = true;
cov.MinWordSize = 3;

// Pass the settings to the query
query.CoverageSetup = cov;
```

## HTTP API

```json
{
  "text": "search terms",
  "maxNumberOfRecordsToReturn": 30,
  "enableCoverage": true,
  "coverageSetup": {
    "coverFuzzyWords": false,
    "coverPrefixSuffix": false,
    "coverWholeQuery": false,
    "coverJoinedWords": true,
    "coverWholeWords": true,
    "minWordSize": 3
  }
}
```


---

# :pixl-Diving_mask: Perform a Deep Coverage Search

The coverage function defaults to processing the top 500 results from pattern search. To retrieve the full number of exact hits, increase the coverage depth to your entire dataset.

## C#

```csharp
query.CoverageDepth = engine.Status.DocumentCount;
```

## HTTP API

```json
{
  "text": "search terms",
  "maxNumberOfRecordsToReturn": 1000,
  "coverageDepth": 50000
}
```


---

# :pixl-Empty: Perform an Empty Search with Sorting and Facets

Sometimes users don't want to enter a keyword but just browse using filters and facets. Indx supports this as long as you have at least one field set to facetable.

An empty search goes through all documents and ignores CoverageDepth settings. Empty queries typically take more time than queries with input text.

Without a `SortBy` field, results are returned in indexing order (the order documents were loaded). To get meaningful ordering, set at least one field to `Sortable = true` and specify it in the query.

## C#

```csharp
query.Text = ""; // or null
query.SortBy = engine.GetField("Movie_Rating")!; // default descending, so 8.0 before 7.0
query.EnableFacets = true;
```

## HTTP API

```json
{
  "text": "",
  "maxNumberOfRecordsToReturn": 50,
  "sortBy": "Movie_Rating",
  "enableFacets": true
}
```


---

# :pixl-Coverage_index: Just Return Exact Hits

Set `IncludePatternMatches = false` on the `CoverageSetup` to return only coverage-confirmed hits — documents where the query matched as a whole word, fuzzy word, joined word, or prefix/suffix. Pure pattern-match results are excluded.

## C#

```csharp
var query = new Query("wireless headphones", 20)
{
    CoverageSetup = new CoverageSetup { IncludePatternMatches = false }
};

var result = engine.Search(query);

foreach (var entry in result.Records)
{
    Console.WriteLine($"key={entry.DocumentKey}  score={entry.Score}");
}
```

## HTTP API

```json
{
  "text": "wireless headphones",
  "maxNumberOfRecordsToReturn": 20,
  "coverageSetup": {
    "includePatternMatches": false
  }
}
```

If no documents match the coverage criteria, the result will be empty. To control which coverage algorithms are used, see [Coverage Settings](/howto/coverage-settings).


---

# :pixl-Linear: Get Min and Max Values from Facets

To set up a range filter that corresponds with a facetable number field (e.g. price), use facet data to get the actual min/max values.

## C#

```csharp
query.EnableFacets = true;
var result = engine.Search(query);

if (result.Facets != null && result.Facets.TryGetValue("Price", out var histogram))
{
    var values = histogram.Select(item => int.Parse(item.Key)).ToList();
    int minPrice = values.Min();
    int maxPrice = values.Max();
    Filter priceRange = engine.CreateRangeFilter("Price", minPrice, maxPrice, out _)!;
}
```

## HTTP API

```bash
# Search with facets enabled
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/search \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"text": "", "maxNumberOfRecordsToReturn": 1, "enableFacets": true}'
```

```json
// Response includes facet histograms
{
  "facets": {
    "Price": [
      { "key": "10", "value": 45 },
      { "key": "25", "value": 120 },
      { "key": "99", "value": 30 }
    ]
  }
}
```

Extract min/max from the facet keys (10 and 99 in this example), then create a range filter:

```bash
curl -X POST https://localhost:5001/api/teams/acme/datasets/products/filters/range \
  -H "Content-Type: application/json" \
  -d '{"fieldName": "Price", "lowerLimit": 10, "upperLimit": 99}'
```


---

# :pixl-Fields: Return the Field Configuration

`GetFieldConfiguration()` returns all fields and their current settings as a `FieldProxy[]`. This is the same shape used by `SetFieldConfiguration` and the HTTP API.

## C#

```csharp
FieldProxy[] fields = engine.GetFieldConfiguration();

foreach (var field in fields)
{
    Console.WriteLine($"{field.FieldName} ({field.FieldType})" +
        $" searchable={field.Searchable}" +
        $" filterable={field.Filterable}" +
        $" facetable={field.Facetable}" +
        $" sortable={field.Sortable}" +
        $" weight={field.Weight}");
}
```

To filter by role:

```csharp
var searchable  = fields.Where(f => f.Searchable == true).ToList();
var filterable  = fields.Where(f => f.Filterable == true).ToList();
var facetable   = fields.Where(f => f.Facetable  == true).ToList();
```

## HTTP API

```bash
curl https://your-host/api/teams/acme/datasets/products/fields/configuration \
  -H "Authorization: Bearer <token>"
```


---

# :pixl-Pattern_recognition: Pattern Recognition Only

Coverage should stay enabled for nearly all use cases. Only disable it in edge cases where you search a single field and only care about top-K fuzzy matches (e.g. name lookup). With coverage disabled, truncation is unreliable and results degrade when searching across multiple fields.

## Disable Coverage

```csharp
query.EnableCoverage = false;
```

```json
{ "text": "search terms", "maxNumberOfRecordsToReturn": 50, "enableCoverage": false }
```

## Alternative: Coverage On, No Truncation

A better option is to run coverage but keep all results, including hits that coverage did not mark as near-exact:

```csharp
CoverageSetup cov = new CoverageSetup();
cov.Truncate = false;
query.CoverageSetup = cov;
```


---

# :pixl-Boost: Set Up Personalized Boosting

Personalization can be a powerful benefit to your users. You can set up boost lists per user or user group. You can boost hundreds of thousands of documents without affecting performance significantly.

**Scenario:** An online grocery store with two active users.
- **User1** regularly buys broccoli and tomatoes
- **User2** often buys cheese, especially discounted items

We boost on a single item if a user has bought it once. If they've purchased more than 10 times, we increase the boost strength. We also boost all discounted products globally.

## C#

```csharp
// Set up global boost for all discounted items
Filter discountFilter = engine.CreateValueFilter("discounted", true, out _)!;
Boost globalDiscountBoost = engine.CreateBoost(discountFilter, BoostStrength.Low);

// PERSONALISATION PER USER
var userBoosts = new List<Boost>();

// Lists of purchased items by item number
List<long> highPurchases = new List<long>{2, 3}; // Broccoli and cherry tomatoes
List<long> lowPurchases = new List<long>{1}; // Strawberry jam

// Build a combined filter for each boost level by OR-ing per-item value filters
Filter? highPurchaseFilter = null;
Filter? lowPurchaseFilter = null;

foreach (long itemNumber in highPurchases)
{
    Filter fil = engine.CreateValueFilter("item_number", itemNumber, out _)!;
    highPurchaseFilter = highPurchaseFilter is null ? fil : highPurchaseFilter | fil;
}
foreach (long itemNumber in lowPurchases)
{
    Filter fil = engine.CreateValueFilter("item_number", itemNumber, out _)!;
    lowPurchaseFilter = lowPurchaseFilter is null ? fil : lowPurchaseFilter | fil;
}

userBoosts.Add(engine.CreateBoost(highPurchaseFilter!, BoostStrength.Med));
userBoosts.Add(engine.CreateBoost(lowPurchaseFilter!, BoostStrength.Low));
userBoosts.Add(globalDiscountBoost);

// Use in query
query.Boosts = userBoosts.ToArray();
query.EnableBoost = true;
```

## HTTP API

```bash
# Calls are under the team-scoped dataset path; set BASE once
# (token comes from the portal's API Key page):
BASE="https://localhost:5001/api/teams/<team>/datasets/groceries"
AUTH="Authorization: Bearer <token>"

# 1. Create global discount filter
curl -X POST "$BASE/filters/value" -H "$AUTH" \
  -H "Content-Type: application/json" \
  -d '{"fieldName": "discounted", "value": "true"}'
# → {"hashString":"discount123"}

# 2. Create boost for discounted items (Low strength)
curl -X POST "$BASE/boosts/from-filter" -H "$AUTH" \
  -H "Content-Type: application/json" \
  -d '{"boostStrength": 1, "filterProxy": {"hashString": "discount123"}}'

# 3. Create per-user filters for frequently purchased items
curl -X POST "$BASE/filters/value" -H "$AUTH" \
  -d '{"fieldName": "item_number", "value": "2"}'
# → {"hashString":"item2"}

curl -X POST "$BASE/filters/value" -H "$AUTH" \
  -d '{"fieldName": "item_number", "value": "3"}'
# → {"hashString":"item3"}

# 4. Combine user item filters (OR)
curl -X POST "$BASE/filters/combine" -H "$AUTH" \
  -d '{"a": {"hashString": "item2"}, "b": {"hashString": "item3"}, "useAndOperation": false}'
# → {"hashString":"userItems123"}

# 5. Create user boost (Med strength)
curl -X POST "$BASE/boosts/from-filter" -H "$AUTH" \
  -d '{"boostStrength": 2, "filterProxy": {"hashString": "userItems123"}}'

# 6. Search with boosts
curl -X POST "$BASE/search" -H "$AUTH" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "vegetables",
    "maxNumberOfRecordsToReturn": 20,
    "enableBoost": true,
    "boosts": [
      {"boostStrength": 1, "filterProxy": {"hashString": "discount123"}},
      {"boostStrength": 2, "filterProxy": {"hashString": "userItems123"}}
    ]
  }'
```


---

# :pixl-Save: Save the Field Configuration

After running `Init()` at least once, you can save the field configuration to a file. This lets you skip `Init` on subsequent loads.

## C#

**First run** — analyze, configure fields, save:

```csharp
var engine = new SearchEngine();

using var initStream = File.OpenRead("data/imdb_top10k.json");
engine.Init(initStream);

engine.SetFieldConfiguration([
    new FieldProxy { FieldName = "Movie_Name",     Searchable = true, Weight = 2.0f },
    new FieldProxy { FieldName = "Stars",          Searchable = true, Facetable = true, Weight = 0.75f },
    new FieldProxy { FieldName = "Description",    Searchable = true, Weight = 1.0f },
    new FieldProxy { FieldName = "Year_of_Release", Facetable = true },
]);

engine.SaveFieldConfiguration("fieldconfig.json");

// ... load and index
```

**Subsequent loads** — skip Init, load config from file:

```csharp
var engine = new SearchEngine();

engine.LoadFieldConfiguration("fieldconfig.json");

// ... load and index
```

## HTTP API

Field configuration is stored server-side per dataset. There is no save/load file step — configuration persists automatically after each `PUT fields/configuration` call.

**First time** — analyze and configure:

```bash
# 1. Analyze JSON structure
curl -X POST https://your-host/api/teams/<team>/datasets/movies/analyze \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  --data-binary @imdb_top10k.json

# 2. Configure fields (204 No Content)
curl -X PUT https://your-host/api/teams/<team>/datasets/movies/fields/configuration \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '[
    { "fieldName": "Movie_Name",      "searchable": true, "weight": 2.0 },
    { "fieldName": "Stars",           "searchable": true, "facetable": true, "weight": 0.75 },
    { "fieldName": "Description",     "searchable": true, "weight": 1.0 },
    { "fieldName": "Year_of_Release", "facetable": true }
  ]'
```

**Subsequent loads** — reload documents from the persisted database (`204 No Content`):

```bash
curl -X POST https://your-host/api/teams/<team>/datasets/movies/load/from-database \
  -H "Authorization: Bearer <token>"
```


---

# :pixl-Rag_search: Set Up Vector and Hybrid Search

This guide assumes you have an embedding model that produces L2-normalised float arrays. See [Vector & Hybrid Search](/guides/vector-search) for concept background and mode selection.

## C#

### Prepare Your Data

Each document must include the embedding as a JSON float array:

```json
[
  { "id": 1, "title": "Wireless Headphones", "embedding": [0.12, -0.45, 0.33, ...] },
  { "id": 2, "title": "Bluetooth Speaker",   "embedding": [0.08,  0.21, -0.14, ...] }
]
```

### Configure and Load

```csharp
var engine = new SearchEngine();

using var stream = File.OpenRead("products.json");
engine.Init(stream);

engine.GetField("title")!.Searchable = true;
engine.GetField("embedding")!.Embeddable = true;  // excluded from text index

stream.Position = 0;
engine.Load(stream);
engine.Index();
```

### Vector Search

```csharp
float[] queryVector = await myEmbeddingClient.EmbedAsync("comfortable headphones");

var index   = engine.EmbeddingFields["embedding"];
var results = index.Search(queryVector, maxResults: 10);

var keys = results.Select(r => r.documentKey).ToArray();
var docs  = engine.GetJsonDataOfKeys(keys);
```

### Hybrid Search

```csharp
float[] queryVector = await myEmbeddingClient.EmbedAsync("comfortable headphones");

var textResult    = engine.Search(new Query("comfortable headphones", 50));
var vectorResults = engine.EmbeddingFields["embedding"].Search(queryVector, maxResults: 50);

var merged = IEmbeddingIndex.MergeHybrid(
    textResults:      textResult.Records,
    embeddingResults: vectorResults,
    alpha:            0.6f
);

var keys = merged.Select(r => r.documentKey).ToArray();
var docs  = engine.GetJsonDataOfKeys(keys);
```

---

## HTTP API

### Configure Fields

```bash
curl -X PUT https://your-host/api/teams/acme/datasets/products/fields/configuration \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '[
    { "fieldName": "title",     "searchable": true },
    { "fieldName": "embedding", "embeddable": true }
  ]'
```

### Vector Search

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/search/vector \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "fieldName": "embedding",
    "vector": [0.12, -0.45, 0.33, ...],
    "maxResults": 10
  }'
```

### Hybrid Search

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/search/hybrid \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "comfortable headphones",
    "embeddingField": "embedding",
    "vector": [0.12, -0.45, 0.33, ...],
    "alpha": 0.6,
    "maxNumberOfRecordsToReturn": 20
  }'
```

Both endpoints return an array of `{ "documentKey": n, "score": 0.94 }` with scores in `[0, 1]`. Use `POST documents/lookup` to fetch the full documents by key.


---

# C# API Reference


---

# SearchEngine

The `SearchEngine` class is the core entry point for all operations. It manages the full lifecycle: analyzing data structure, configuring fields, loading data, indexing, and searching.

## Lifecycle

```
Init(stream) → Configure fields → Load(stream) → Index() → Search(query)
```

After `Index()` the engine is in `Ready` state and can be searched continuously. Dynamic operations (`InsertJsonRecord`, `UpdateJsonRecord`, `DeleteJsonRecord`) keep the index in sync without requiring a full `Index()` rebuild.

## Constructor

```csharp
// Default — works for most cases (config 400)
var engine = new SearchEngine();

// With license file
var engine = new SearchEngine("indx-developer.license");

// With logging and custom config
var engine = new SearchEngine(
    logPrefix: "MyApp",
    factory: loggerFactory,
    configurationNumber: 400,
    licenseFileName: "indx-developer.license"
);
```

## Basic Usage

```csharp
using Indx.Api;

var engine = new SearchEngine();

// 1. Analyze JSON structure
using var stream = File.OpenRead("products.json");
engine.Init(stream);

// 2. Configure fields
engine.GetField("name")!.Searchable = true;
engine.GetField("name")!.Weight = 2.0f;
engine.GetField("description")!.Searchable = true;
engine.GetField("category")!.Filterable = true;
engine.GetField("price")!.Sortable = true;

// 3. Load and index
stream.Position = 0;
engine.Load(stream);
engine.Index();

// 4. Search
var result = engine.Search(new Query("wireless headphones", 20));
```

## Lifecycle Methods

| Method | Description |
|--------|-------------|
| `Init(Stream, ...)` | Analyze the JSON stream and discover fields. Must be called first |
| `Load(Stream, ...)` | Parse and store documents. Call after configuring fields |
| `Index(...)` | Build the inverted index. Call after `Load` |
| `Search(Query)` | Run a query. Engine must be in `Ready` state |
| `Hibernate(out string)` | Release memory while preserving the document store |
| `WakeUp()` | Restore from hibernation |
| `Unload(out string)` | Remove documents and reset to `Created` state |
| `Dispose()` | Free all resources |

## Field Access

| Method | Returns | Description |
|--------|---------|-------------|
| `GetField(string)` | `Field?` | Get a field by name. Returns `null` if not found |
| `GetFieldList()` | `List<Field>` | All discovered fields |
| `GetFieldConfiguration()` | `FieldProxy[]` | Full configuration snapshot of all fields |
| `SetFieldConfiguration(FieldProxy[])` | `string?` | Apply a field configuration. Returns an error message or `null` on success |

## Dynamic Operations

Insert, update, and delete documents after indexing without rebuilding from scratch. In BM25F mode the shared index is updated incrementally. A periodic full `Index()` re-normalises avgdl drift that accumulates over many mutations.

| Method | Description |
|--------|-------------|
| `InsertJsonRecord(string json, out string error)` | Insert a single JSON document. Returns `false` on error |
| `InsertJsonRecords(IEnumerable<string>, ProcessMonitor?, out string error)` | Batch insert from a collection |
| `InsertJsonRecords(Stream, ProcessMonitor?, out string error)` | Batch insert from a JSON stream |
| `UpdateJsonRecord(string json, out string error)` | Replace a document by key |
| `UpdateJsonRecords(IEnumerable<string>, ProcessMonitor?, out string error)` | Batch update from a collection |
| `UpdateJsonRecords(Stream, ProcessMonitor?, out string error)` | Batch update from a JSON stream |
| `DeleteJsonRecord(long id)` | Remove a single document by key |
| `DeleteJsonRecords(IEnumerable<long>, ProcessMonitor?)` | Remove multiple documents |
| `DeleteRecordsInFilter(Filter)` | Remove all documents matching a filter |
| `UpdateField(long id, string fieldName, object? value, out string error)` | Update a single field value on one document |
| `UpdateFieldInFilter(Filter, string fieldName, object value, out string error)` | Update a field on all documents matching a filter. Returns count updated |

```csharp
// Insert a new document
bool ok = engine.InsertJsonRecord(
    """{"id": 999, "name": "New Product", "price": 49.99}""",
    out string error);

// Update a field value on one document
engine.UpdateField(documentKey, "price", 39.99, out _);

// Delete a document
engine.DeleteJsonRecord(documentKey);
```

## Data Retrieval

| Method | Returns | Description |
|--------|---------|-------------|
| `GetJsonDataOfKey(long)` | `string` | Full JSON of a document by key |
| `GetJsonDataOfKeys(IEnumerable<long>)` | `IEnumerable<string>` | Full JSON for multiple keys |

## Filters

| Method | Returns | Description |
|--------|---------|-------------|
| `CreateValueFilter(string fieldName, object value, out string? error)` | `Filter?` | Match documents where field equals value (`null` + `error` on failure) |
| `CreateRangeFilter(string fieldName, double min, double max, out string? error)` | `Filter?` | Match documents where field falls within a range (`null` + `error` on failure) |
| `LoadAllFilters(...)` | — | Pre-load all filterable fields for faster repeated filtering |
| `LoadFilters(Filter[])` | `bool` | Pre-load specific filters |
| `DeleteFilter(Filter)` | `bool` | Release a cached filter |
| `DeleteAllFilters()` | — | Release all cached filters |
| `PreLoadedFilters` | `IReadOnlyDictionary<...>` | Access filters cached via `Field.PreloadFilters = true` |

## Persistence

| Method | Description |
|--------|-------------|
| `SaveFieldConfiguration(string path)` | Save field config to a JSON file |
| `LoadFieldConfiguration(string path)` | Restore field config from a JSON file |

## Properties

| Property | Type | Description |
|----------|------|-------------|
| `Status` | `SystemStatus` | Current engine state, counters, and error flags |
| `ScoringMode` | `ScoringMode` | `BM25F` or `BM25PerField` — resolved at `Index()` time |
| `DocumentFields` | `DocumentFields` | Field metadata discovered by `Init()` |
| `NumberOfFilters` | `int` | Number of currently cached filters |
| `PreLoadedFilters` | `IReadOnlyDictionary<...>` | Filters preloaded via `Field.PreloadFilters` |

## ScoringMode

`ScoringMode` is read-only and resolved automatically at `Index()` time:

| Mode | When selected | Behaviour |
|------|---------------|-----------|
| `BM25F` | All searchable fields share the same `BM25k1` | One shared inverted index; single saturation step on aggregated pseudo term-frequency |
| `BM25PerField` | At least two searchable fields have different `BM25k1` | Separate index per field; scores merged by `Field.Weight` |

```csharp
engine.Index();
Console.WriteLine(engine.ScoringMode); // BM25F or BM25PerField
```


## SearchEngine (class)


Properties:
  DocumentFields: DocumentFields (get) — Field metadata discovered after Init() completes. Contains all fields with their type and configuration flags.
  NumberOfFilters: int (get) — Returns the number of filters currently held in the filter cache.
  Persistence: IPersistence (get/set) — Persistence provider used for hibernation and wake-up serialization. Assigned by the cloud layer; null in standalone use.
  PreLoadedFilters: IReadOnlyDictionary<string, IReadOnlyDictionary<string, ValueFilter>> (get) — Provides access to all value filters that were pre-loaded during Load.
            Only fields marked with Field.PreloadFilters = true appear here.
            The outer key is the field name; the inner key is the filter value as a string.
            Populated after Load completes.
  Status: SystemStatus (get/set) — Returns the current state and health of the search engine.
            Includes document count, version, license information, and errors
            from the most recent Init or Load operation.
            Recoverable and unrecoverable errors from the last operation are available
            via Status.RecoverableErrors and Status.UnrecoverableErrors even after
            the ProcessMonitor reference has been discarded.
  ScoringMode: ScoringMode (get/set) — The scoring algorithm chosen by the engine attime,
            based on whetheris uniform across searchable
            fields. Uniform values (or fewer than two searchable fields) select; differing values select. Inspect afterto
            see what the engine actually used.
  IsDisposed: bool (get) — True once this engine has been disposed. A disposed engine must not be searched and itsmust not be read (that can throw).
            Hosting layers probe this to report a stale instance instead of mistaking its last-known
            state for a live one.
  EmbeddingFields: IReadOnlyDictionary<string, IEmbeddingIndex> (get) — EmbeddableRead-only view of the HNSW embedding indexes, one per field
             that hasset to true.

Methods:
  SearchEngine CreateInMemoryClone(ProcessMonitor monitor = null, FieldProxy[] fieldOverrides = null)
    fieldOverridesBuilds an independent SearchEngine with the same in-memory state as this
             instance: same field configuration, same documents, freshly built index.
            
             Does not touch persistence — works for class-library clients that never
             configured a database. The clone has its own native-memory pools and can
             be mutated/disposed independently of the original.
            
             The clone is built by emitting the current document store as a JSON array,
             feeding it through Init/Load on the new instance, and rebuilding the index.
             Documents marked as Deleted are skipped. This is the foundation used by the
             cloud layer's shadow-swap path; callers that want shadow-swap-like behaviour
             in their own application can build it on top of this method.
            
             Behaviour mirrors:(optional) is applied after the clone has
             mirrored this engine's configuration but beforeruns. This is the
             supported way to change Searchable/WordIndexing/Embeddable/BM25b/BM25k1 on a
             shadow copy: those flags are consumed byduring Load to
             build the per-field index collection, so they must be set pre-Load to take
             effect. Returns null on success; throwsnaming the offending field if any override references a non-existent field.
  Result Search(Query searchQueryIn)
    Executes a search and returns ranked results. Requires that Init, field
             configuration, Load, and Index have all completed successfully.
            
             The query controls the search text via Query.Text, the maximum number of
             results via Query.MaxNumberOfRecordsToReturn, optional filtering via
             Query.Filter, score boosting via Query.Boosts, field-based sorting via
             Query.SortBy, facet histograms via Query.EnableFacets, and the number of
             candidate documents evaluated via Query.CoverageDepth.
            
             Search is thread-safe and can be called concurrently from multiple threads.
            
             Searching an engine that is not Ready — one that is loading, indexing or
             hibernated — returns an empty result with Result.DidTimeOut set, so it can
             be told apart from a search that ran and found nothing. Check that flag
             before showing "no results" to anyone: a dataset being reloaded would
             otherwise look like a dataset with nothing in it.
  bool InsertJsonRecord(string jsonData, String& error)
    Inserts a single JSON record without requiring a full reload.
            The document key is extracted from the JSON using the configured key field.
            The record is immediately searchable after this call returns.
            Returns false and populates error if the JSON is invalid or the key is missing.
  bool UpdateJsonRecord(string jsonData, String& error)
    Replaces an existing record identified by the document key found in the JSON.
            If no record with that key exists the call fails and error is populated.
            The updated record is immediately searchable after this call returns.
  bool InsertJsonRecords(IEnumerable<string> records, ProcessMonitor monitor, String& error)
  bool InsertJsonRecords(Stream jsonStream, ProcessMonitor monitor, String& error)
    Inserts multiple JSON records from a stream containing a JSON array.
            If monitor is null this is a blocking call. If a monitor is provided,
            insertion runs asynchronously on a background thread. In that case the
            caller must keep the stream alive until ProcessMonitor.IsCompleted is true.
  bool UpdateJsonRecords(IEnumerable<string> records, ProcessMonitor monitor, String& error)
  bool UpdateJsonRecords(Stream jsonStream, ProcessMonitor monitor, String& error)
    Replaces multiple records from a stream containing a JSON array.
            If monitor is null this is a blocking call. If a monitor is provided,
            the operation runs asynchronously and the stream must be kept alive until
            ProcessMonitor.IsCompleted is true.
  bool DeleteJsonRecords(IEnumerable<long> ids, ProcessMonitor monitor = null)
  bool UpdateField(long id, string fieldName, object value, String& error)
    Updates a single field within an existing document without replacing the
            entire record. Supports nested fields using dot notation, for example
            "address.city", and array element access using index notation, for example
            "tags[0]". Type compatibility is enforced: the new value must match the
            type of the existing field value. Returns false and populates error on failure.
  int UpdateFieldInFilter(Filter filter, string fieldName, object value, String& error)
    Setstoon every document
            admitted by. The filter is loaded automatically if it
            has not been loaded yet. Returns the number of documents successfully updated.
  List<Field> GetFacetableFieldList()
    Returns all fields currently marked as Facetable.
  Field GetField(string fieldName)
    Returns the field with the given name, or null if not found.
  List<Field> GetFieldList()
    Returns all detected fields.
  List<Field> GetFilterableFieldList()
    Returns all fields currently marked as Filterable.
  string SetFieldConfiguration(FieldProxy[] fields)
    Applies a batch of per-field configuration in one call. Nullable properties onhave replace semantics:leaves the existing
            value untouched; any non-null value (including) overwrites.
            Returns the name of the first proposed field that does not exist on the engine,
            oron success. The caller is responsible for callingafterwards if any reindex-requiring
            property changed — useto check.
  FieldProxy[] GetFieldConfiguration()
    FieldProxyReturns the complete configuration of every field as aarray,
            including all flags, weights and BM25F parameters.
  bool LoadFieldConfiguration(string configurationFile)
    Loads field configuration previously saved by SaveFieldConfiguration,
            restoring Searchable, Filterable, Facetable, Sortable and other field settings.
            Call this after Init to avoid manually reconfiguring each field.
            Returns false if the file does not exist or cannot be parsed.
  bool SaveFieldConfiguration(string configurationFile)
    Saves the current field configuration to a file so it can be restored later
            with LoadFieldConfiguration. Returns false if DocumentFields is null or
            the file cannot be written.
  void SetDocumentFieldsInternal(DocumentFields documentFields)
    Intended for Cloud project.
  Boost CreateBoost(Filter filter, BoostStrength strength)
    Creates a boost that increases the score of documents matching the given filter.
            Must be called after Load completes. Pass the resulting Boost instance to
            Query.Boosts before calling Search.
  Filter CreateRangeFilter(string fieldName, double lowerInclusiveLimit, double upperInclusiveLimit, String& error)
    Creates a filter matching documents where fieldName falls within the inclusive
            numeric range from lowerInclusiveLimit to upperInclusiveLimit. The field must
            be marked as Filterable and have a numeric type. Values are parsed using
            CultureInfo.InvariantCulture. Returns null if the engine is not initialized,
            the field is not found, not filterable, or the range is invalid — in which
            case error describes the exact cause; error is null on success.
  Filter CreateRangeFilter(string fieldName, double lowerInclusiveLimit, double upperInclusiveLimit, CultureInfo cultureInfo, String& error)
    Creates a range filter using a specific CultureInfo when parsing numeric values.
            See CreateRangeFilter(string, double, double, out string) for full documentation.
  Filter CreateValueFilter(string fieldName, object filterValue, String& error)
    Creates a case-insensitive filter matching documents where fieldName equals
            filterValue. The field must be marked as Filterable. Returns null if the
            engine is not initialized, the field is not found, not filterable, or the
            value is unusable — in which case error describes the exact cause; error is
            null on success.
  Filter CreateValueFilter(string fieldName, object filterValue, bool isCaseSensitive, String& error)
    Creates a value filter with explicit case sensitivity.
            See CreateValueFilter(string, object, out string) for full documentation.
  void DeleteAllFilters()
    Removes all filters from the filter cache. Any query referencing a deleted
            filter will return empty results.
  bool DeleteFilter(Filter filter)
    Removes a single filter from the filter cache. Does not cascade to filters
            derived from this one via boolean operators. Returns false if not found.
  Filter GetFilterFromKey(string key)
    Retrieves or reconstructs a filter from its serialized key string.
            Intended for HTTP/cloud scenarios where the filter object may have been
            evicted from the cache between requests.
  void LoadAllFilters(ProcessMonitor monitor = null)
    Loads all filters in the cache using all available CPU cores. A filter created
            with CreateValueFilter or CreateRangeFilter is not usable in a query until it
            has been loaded. If a monitor is provided the operation runs asynchronously.
  void LoadAllFilters(int maxThreadCount, ProcessMonitor monitor = null)
    Loads all filters using at most maxThreadCount threads.
            See LoadAllFilters(ProcessMonitor) for full documentation.
  bool LoadFilters(Filter[] filterList)
    Loads a specific set of filters. Filters not in the array are unaffected.
            Returns false if the engine is not in a state that allows filter loading.
  bool LoadFilters(Filter[] filterList, int maxThreadCount)
    Loads a specific set of filters using at most maxThreadCount threads.
            Returns false if the engine is not in a state that allows filter loading.
  void Dispose()
    Dispose.
  bool Hibernate(String& errorMessage)
    Puts the engine into hibernation mode, freeing index memory while retaining
            document data. Call WakeUp to restore search capability without reloading data.
            Returns false if the engine is not in the Ready state.
  void Index(ProcessMonitor monitor = null)
    Builds the search indexes required for Search. Must be called after Load
            completes and at least one field has been marked as Searchable.
            BM25 parameters (,) are
            per field and have sensible defaults. The scoring algorithm
            () is selected automatically based on whetheris uniform across all searchable fields.
            If a monitor is provided, indexing runs asynchronously.
            Callto block until done.
  void Init(Stream jsonStream, string nameOfKeyField, ProcessMonitor monitor = null)
    Scans the JSON stream to detect the field structure of the data.
             Must be called before configuring fields and before Load.
             The stream position is reset to zero after Init completes so the same
             stream instance can be passed directly to Load.
            
             Recoverable parse errors such as control characters or trailing commas
             are silently sanitized unless ProcessMonitor.MaxRecoveryAttempts is exceeded,
             in which case the operation aborts and ProcessMonitor.Succeeded is set to false.
             Unrecoverable parse errors always abort immediately. If a monitor is provided
             the error is recorded in ProcessMonitor.UnrecoverableErrors and Succeeded is false.
             If no monitor is provided a JsonException is thrown.
  void Init(Stream jsonStream, ProcessMonitor monitor = null)
    Convenience overload of Init that assumes the key field is named "id".
            See Init(Stream, string, ProcessMonitor) for full documentation.
  void Load(Stream jsonStream, ProcessMonitor monitor = null)
    Loads JSON data into the search engine. Must be called after Init and field
             configuration, and before Index. The stream position is reset to zero before
             reading, so the same stream used for Init can be passed here directly.
            
             Recoverable parse errors such as control characters or trailing commas are
             silently sanitized and counted. Each distinct error, grouped by source and
             message, is recorded once in ProcessMonitor.RecoverableErrors with an
             occurrence count. The ParseResult details and originating exception are
             captured from the first occurrence only. If the number of recoverable errors
             exceeds ProcessMonitor.MaxRecoveryAttempts the load is aborted, Succeeded is
             set to false, and the error that caused the limit to be exceeded is also added
             to ProcessMonitor.UnrecoverableErrors.
            
             Unrecoverable parse errors always abort the load immediately. The error is
             added to ProcessMonitor.UnrecoverableErrors and Succeeded is set to false.
             If no monitor is provided a JsonException is thrown instead.
            
             After Load completes, all error details are mirrored to Status.RecoverableErrors
             and Status.UnrecoverableErrors so they remain accessible after the monitor
             reference is discarded.
  Task LoadAsync(Stream jsonStream, ProcessMonitor monitor = null)
    Used for cloud project.
  bool LoadDocumentFieldsFromDb()
    Used for Cloud project.
  void LoadFromDatabaseSync(ProcessMonitor monitor)
    Intended for Cloud project. Use regular Load in other cases.
  bool Unload(String& errorMessage)
    Unloads all documents, indexes, and filters from memory. The engine returns
            to its initial state and must go through Init, Load, and Index again before
            searching. Returns false if the engine is not in the Ready or Loaded state,
            in which case errorMessage describes the reason.
  bool WakeUp()
    Restores the engine from hibernation by rebuilding all indexes.
            Uses all available CPU cores. Returns false if the engine is not hibernated.
  bool WakeUp(int maxThreadCount)
    Restores the engine from hibernation using at most maxThreadCount threads
            when rebuilding indexes and reloading filters.
  bool DeleteJsonRecord(long id)
    Deletes the document with the given key. The document is immediately removed
            from search results. This is a blocking call.
            Returns false if no document with that key exists.
  void DeleteRecordsInFilter(Filter filter)
    Deletes all documents that match the given filter. Equivalent to iterating
            the filter mask and calling DeleteJsonRecord for each matching document.
  List<Document> GetAllDocuments()
    Returns a list of all loaded documents including their raw JSON and
            pre-parsed field positions. Intended for bulk operations and diagnostics.
  Document GetDocumentJsonOfKey(long key)
    Returns a ready parsed DocumentJson which allows
            fast substring extraction of fields.
  string GetJsonDataOfKey(long key)
    Returns the raw JSON string for the document with the given key,
            or null if no document with that key exists.

## ISearchEngine (interface)

Primary interface for field-based search over JSON data.
            
             Typical usage: call Init to detect fields, configure those fields, call Load
             to populate the engine, call Index to build search indexes, then call Search.


Properties:
  NumberOfFilters: int (get) — Returns the number of filters currently held in the filter cache.
  PreLoadedFilters: IReadOnlyDictionary<string, IReadOnlyDictionary<string, ValueFilter>> (get) — Provides access to all value filters that were pre-loaded during Load.
            Only fields marked with Field.PreloadFilters = true appear here.
            The outer key is the field name; the inner key is the filter value as a string.
            Populated after Load completes.
  Status: SystemStatus (get) — Returns the current state and health of the search engine.
            Includes document count, version, license information, and errors
            from the most recent Init or Load operation.
            Recoverable and unrecoverable errors from the last operation are available
            via Status.RecoverableErrors and Status.UnrecoverableErrors even after
            the ProcessMonitor reference has been discarded.

Methods:
  void Init(Stream jsonStream, string nameOfKeyField = "id", ProcessMonitor monitor = null)
    Scans the JSON stream to detect the field structure of the data.
             Must be called before configuring fields and before Load.
             The stream position is reset to zero after Init completes so the same
             stream instance can be passed directly to Load.
            
             Recoverable parse errors such as control characters or trailing commas
             are silently sanitized unless ProcessMonitor.MaxRecoveryAttempts is exceeded,
             in which case the operation aborts and ProcessMonitor.Succeeded is set to false.
             Unrecoverable parse errors always abort immediately. If a monitor is provided
             the error is recorded in ProcessMonitor.UnrecoverableErrors and Succeeded is false.
             If no monitor is provided a JsonException is thrown.
  void Init(Stream jsonStream, ProcessMonitor monitor = null)
    Convenience overload of Init that assumes the key field is named "id".
            See Init(Stream, string, ProcessMonitor) for full documentation.
  void Load(Stream jsonStream, ProcessMonitor monitor = null)
    Loads JSON data into the search engine. Must be called after Init and field
             configuration, and before Index. The stream position is reset to zero before
             reading, so the same stream used for Init can be passed here directly.
            
             Recoverable parse errors such as control characters or trailing commas are
             silently sanitized and counted. Each distinct error, grouped by source and
             message, is recorded once in ProcessMonitor.RecoverableErrors with an
             occurrence count. The ParseResult details and originating exception are
             captured from the first occurrence only. If the number of recoverable errors
             exceeds ProcessMonitor.MaxRecoveryAttempts the load is aborted, Succeeded is
             set to false, and the error that caused the limit to be exceeded is also added
             to ProcessMonitor.UnrecoverableErrors.
            
             Unrecoverable parse errors always abort the load immediately. The error is
             added to ProcessMonitor.UnrecoverableErrors and Succeeded is set to false.
             If no monitor is provided a JsonException is thrown instead.
            
             After Load completes, all error details are mirrored to Status.RecoverableErrors
             and Status.UnrecoverableErrors so they remain accessible after the monitor
             reference is discarded.
  void Index(ProcessMonitor monitor = null)
    Builds the search indexes required for Search. Must be called after Load
            completes and at least one field has been marked as Searchable.
            BM25 parameters (,) are
            per field and have sensible defaults. The scoring algorithm
            () is selected automatically based on whetheris uniform across all searchable fields.
            If a monitor is provided, indexing runs asynchronously.
            Callto block until done.
  bool Unload(String& errorMessage)
    Unloads all documents, indexes, and filters from memory. The engine returns
            to its initial state and must go through Init, Load, and Index again before
            searching. Returns false if the engine is not in the Ready or Loaded state,
            in which case errorMessage describes the reason.
  bool Hibernate(String& errorMessage)
    Puts the engine into hibernation mode, freeing index memory while retaining
            document data. Call WakeUp to restore search capability without reloading data.
            Returns false if the engine is not in the Ready state.
  bool WakeUp()
    Restores the engine from hibernation by rebuilding all indexes.
            Uses all available CPU cores. Returns false if the engine is not hibernated.
  bool WakeUp(int maxThreadCount)
    Restores the engine from hibernation using at most maxThreadCount threads
            when rebuilding indexes and reloading filters.
  bool LoadFieldConfiguration(string configurationFile)
    Loads field configuration previously saved by SaveFieldConfiguration,
            restoring Searchable, Filterable, Facetable, Sortable and other field settings.
            Call this after Init to avoid manually reconfiguring each field.
            Returns false if the file does not exist or cannot be parsed.
  bool SaveFieldConfiguration(string configurationFile)
    Saves the current field configuration to a file so it can be restored later
            with LoadFieldConfiguration. Returns false if DocumentFields is null or
            the file cannot be written.
  bool InsertJsonRecord(string jsonData, String& error)
    Inserts a single JSON record without requiring a full reload.
            The document key is extracted from the JSON using the configured key field.
            The record is immediately searchable after this call returns.
            Returns false and populates error if the JSON is invalid or the key is missing.
  bool InsertJsonRecords(Stream jsonStream, ProcessMonitor monitor, String& error)
    Inserts multiple JSON records from a stream containing a JSON array.
            If monitor is null this is a blocking call. If a monitor is provided,
            insertion runs asynchronously on a background thread. In that case the
            caller must keep the stream alive until ProcessMonitor.IsCompleted is true.
  bool InsertJsonRecords(IEnumerable<string> records, ProcessMonitor monitor, String& error)
  bool UpdateJsonRecord(string jsonData, String& error)
    Replaces an existing record identified by the document key found in the JSON.
            If no record with that key exists the call fails and error is populated.
            The updated record is immediately searchable after this call returns.
  bool UpdateJsonRecords(Stream jsonStream, ProcessMonitor monitor, String& error)
    Replaces multiple records from a stream containing a JSON array.
            If monitor is null this is a blocking call. If a monitor is provided,
            the operation runs asynchronously and the stream must be kept alive until
            ProcessMonitor.IsCompleted is true.
  bool UpdateJsonRecords(IEnumerable<string> records, ProcessMonitor monitor, String& error)
  bool UpdateField(long id, string fieldName, object value, String& error)
    Updates a single field within an existing document without replacing the
            entire record. Supports nested fields using dot notation, for example
            "address.city", and array element access using index notation, for example
            "tags[0]". Type compatibility is enforced: the new value must match the
            type of the existing field value. Returns false and populates error on failure.
  bool DeleteJsonRecord(long id)
    Deletes the document with the given key. The document is immediately removed
            from search results. This is a blocking call.
            Returns false if no document with that key exists.
  bool DeleteJsonRecords(IEnumerable<long> ids, ProcessMonitor monitor = null)
  void DeleteRecordsInFilter(Filter filter)
    Deletes all documents that match the given filter. Equivalent to iterating
            the filter mask and calling DeleteJsonRecord for each matching document.
  int UpdateFieldInFilter(Filter filter, string fieldName, object value, String& error)
    Setstoon every document
            admitted by. The filter is loaded automatically if it
            has not been loaded yet. Returns the number of documents successfully updated.
  List<Document> GetAllDocuments()
    Returns a list of all loaded documents including their raw JSON and
            pre-parsed field positions. Intended for bulk operations and diagnostics.
  string GetJsonDataOfKey(long key)
    Returns the raw JSON string for the document with the given key,
            or null if no document with that key exists.
  List<Field> GetFacetableFieldList()
    Returns all fields currently marked as Facetable.
  Field GetField(string fieldName)
    Returns the field with the given name, or null if not found.
  List<Field> GetFieldList()
    Returns all detected fields.
  List<Field> GetFilterableFieldList()
    Returns all fields currently marked as Filterable.
  string SetFieldConfiguration(FieldProxy[] fields)
    Applies a batch of per-field configuration in one call. Nullable properties onhave replace semantics:leaves the existing
            value untouched; any non-null value (including) overwrites.
            Returns the name of the first proposed field that does not exist on the engine,
            oron success. The caller is responsible for callingafterwards if any reindex-requiring
            property changed — useto check.
  FieldProxy[] GetFieldConfiguration()
    FieldProxyReturns the complete configuration of every field as aarray,
            including all flags, weights and BM25F parameters.
  SearchEngine CreateInMemoryClone(ProcessMonitor monitor = null, FieldProxy[] fieldOverrides = null)
    InvalidOperationExceptionBuilds an independent SearchEngine with the same in-memory state: same field
             configuration, same documents, freshly built index. Does not require persistence.
             The clone has its own native-memory pools and can be mutated and disposed
             independently of the original. Foundation for shadow-swap patterns: build a
             clone, mutate it, swap your instance reference, dispose the old one.
            
             Ifis supplied it is wired to the Index phase
             (the dominant cost), giving callers progress visibility and a cancellation
             handle. Init and Load run silently with internal monitors.(optional) is applied after the clone has
             mirrored this engine's configuration but beforeruns — the
             supported way to flip Searchable/WordIndexing/Embeddable/BM25b/BM25k1 on a
             shadow copy. Those flags are consumed byduring Load
             to build the per-field index collection, so they must be set pre-Load to take
             effect. Throwsif any override
             references a non-existent field.
  Boost CreateBoost(Filter filter, BoostStrength strength)
    Creates a boost that increases the score of documents matching the given filter.
            Must be called after Load completes. Pass the resulting Boost instance to
            Query.Boosts before calling Search.
  Filter CreateRangeFilter(string fieldName, double lowerInclusiveLimit, double upperInclusiveLimit, String& error)
    Creates a filter matching documents where fieldName falls within the inclusive
            numeric range from lowerInclusiveLimit to upperInclusiveLimit. The field must
            be marked as Filterable and have a numeric type. Values are parsed using
            CultureInfo.InvariantCulture. Returns null if the engine is not initialized,
            the field is not found, not filterable, or the range is invalid — in which
            case error describes the exact cause; error is null on success.
  Filter CreateRangeFilter(string fieldName, double lowerInclusiveLimit, double upperInclusiveLimit, CultureInfo cultureInfo, String& error)
    Creates a range filter using a specific CultureInfo when parsing numeric values.
            See CreateRangeFilter(string, double, double, out string) for full documentation.
  Filter CreateValueFilter(string fieldName, object filterValue, String& error)
    Creates a case-insensitive filter matching documents where fieldName equals
            filterValue. The field must be marked as Filterable. Returns null if the
            engine is not initialized, the field is not found, not filterable, or the
            value is unusable — in which case error describes the exact cause; error is
            null on success.
  Filter CreateValueFilter(string fieldName, object filterValue, bool isCaseSensitive, String& error)
    Creates a value filter with explicit case sensitivity.
            See CreateValueFilter(string, object, out string) for full documentation.
  void DeleteAllFilters()
    Removes all filters from the filter cache. Any query referencing a deleted
            filter will return empty results.
  bool DeleteFilter(Filter filter)
    Removes a single filter from the filter cache. Does not cascade to filters
            derived from this one via boolean operators. Returns false if not found.
  Filter GetFilterFromKey(string key)
    Retrieves or reconstructs a filter from its serialized key string.
            Intended for HTTP/cloud scenarios where the filter object may have been
            evicted from the cache between requests.
  void LoadAllFilters(ProcessMonitor monitor = null)
    Loads all filters in the cache using all available CPU cores. A filter created
            with CreateValueFilter or CreateRangeFilter is not usable in a query until it
            has been loaded. If a monitor is provided the operation runs asynchronously.
  void LoadAllFilters(int maxThreadCount, ProcessMonitor monitor = null)
    Loads all filters using at most maxThreadCount threads.
            See LoadAllFilters(ProcessMonitor) for full documentation.
  bool LoadFilters(Filter[] filters)
    Loads a specific set of filters. Filters not in the array are unaffected.
            Returns false if the engine is not in a state that allows filter loading.
  bool LoadFilters(Filter[] filterList, int maxThreadCount)
    Loads a specific set of filters using at most maxThreadCount threads.
            Returns false if the engine is not in a state that allows filter loading.
  Result Search(Query query)
    Executes a search and returns ranked results. Requires that Init, field
             configuration, Load, and Index have all completed successfully.
            
             The query controls the search text via Query.Text, the maximum number of
             results via Query.MaxNumberOfRecordsToReturn, optional filtering via
             Query.Filter, score boosting via Query.Boosts, field-based sorting via
             Query.SortBy, facet histograms via Query.EnableFacets, and the number of
             candidate documents evaluated via Query.CoverageDepth.
            
             Search is thread-safe and can be called concurrently from multiple threads.
            
             Searching an engine that is not Ready — one that is loading, indexing or
             hibernated — returns an empty result with Result.DidTimeOut set, so it can
             be told apart from a search that ran and found nothing. Check that flag
             before showing "no results" to anyone: a dataset being reloaded would
             otherwise look like a dataset with nothing in it.

---

# ProcessMonitor

Tracks the state, progress, and completion of a long-running process. Supports timeout monitoring, cancellation, and both synchronous and asynchronous waiting.

The ProcessMonitor can be used for `Init`, `Load`, and `Index`. Using the ProcessMonitor allows you to run loading and indexing in parallel.

`Init`, `Load` and `LoadAllFilters` put the work on a background thread and return
immediately; the monitor is already running by the time they return, so it is safe to poll
or wait on the very next line. `Index` runs on the calling thread.

Pick **one** way to wait — they are alternatives, not steps.

**Poll for progress:**

```csharp
var monitor = new ProcessMonitor();
monitor.TimeoutSeconds = 120;

engine.Load(fstream, monitor);

while (monitor.IsRunning)
{
    Console.WriteLine($"Progress: {monitor.ProgressPercent}%");
    Thread.Sleep(200);
}

if (!monitor.Succeeded)
    Console.WriteLine($"Error: {monitor.ErrorMessage}");
```

**Wait synchronously:**

```csharp
engine.Load(fstream, monitor);
monitor.WaitForCompletion();          // returns false on timeout

if (!monitor.Succeeded)
    Console.WriteLine($"Error: {monitor.ErrorMessage}");
```

**Wait asynchronously:**

```csharp
engine.Load(fstream, monitor);
await monitor.WaitForCompletionAsync();

if (!monitor.Succeeded)
    Console.WriteLine($"Error: {monitor.ErrorMessage}");
```

One monitor per operation. Reusing an instance while it is still running throws; wait for
it first. And note that `WaitForCompletionAsync` on a monitor that was never handed to an
operation completes immediately — it reports "nothing to wait for", not "finished
successfully", so always check `Succeeded`.

## Properties

| Property | Type | Description |
|----------|------|-------------|
| `TimeoutSeconds` | `int` | Max time a process can take before terminating. Default: `Timeout.Infinite` |
| `ErrorMessage` | `string` | Details about a failure, if any. Empty string means no error |
| `Exception` | `Exception` | Catches and assigns any exceptions thrown during processing |
| `IsRunning` | `bool` | Whether the process is currently running |
| `ProgressPercent` | `int` | Completion percentage of the process (0–100) |
| `Succeeded` | `bool` | True if the process finished successfully |
| `StartTime` | `DateTime` | Timestamp of when the process started |
| `DidTimeOut` | `bool` | True if the wait timed out before the process finished |
| `IsCancelled` | `bool` | True for a cancelled process |
| `IsCompleted` | `bool` | True for a completed process |

## Methods

| Method | Description |
|--------|-------------|
| `Cancel()` | Requests cancellation of the monitored process |
| `WaitForCompletion()` | Waits for the process to complete, timeout, or cancel |
| `WaitForCompletionAsync()` | Waits asynchronously for the process to complete, timeout, or cancel |
| `WaitForProcessStarted(int timeoutMs)` | May be used for the sync case to avoid race conditions |
| `Dispose()` | Frees resources |

## ThreadPriority

Allows clients to set priority on the threads executing the process.

```csharp
monitor.ThreadPriority = ThreadPriority.Normal; // default

// Options:
// ThreadPriority.Lowest
// ThreadPriority.BelowNormal
// ThreadPriority.Normal
// ThreadPriority.AboveNormal
// ThreadPriority.Highest
```


## ProcessMonitor (class)

Tracks the state, progress, and completion of a long-running process such as
             Init, Load, or Index. Supports timeout monitoring, cancellation, and both
             synchronous and asynchronous waiting.
            
             Not recommended, but for robustness the monitor may be reused after
             WaitForCompletion or WaitForCompletionAsync returns. Reuse while running throws
             InvalidOperationException.
            
             Set MaxRecoveryAttempts before starting the operation to control how many
             recoverable errors are tolerated. When the threshold is exceeded a JsonException
             is thrown, the operation is aborted, and Succeeded is set to false.


Properties:
  DidTimeOut: bool (get/set) — Returns true if the wait timed out before the process finished.
  ErrorMessage: string (get/set) — Provides a short description of the first fatal failure, if any.
            An empty string means no fatal error occurred. For full error details
            including recoverable errors see RecoverableErrors and UnrecoverableErrors.
  Exception: Exception (get/set) — Catches and exposes any unhandled exception thrown during processing.
  IsCancelled: bool (get) — Returns true for a cancelled process.
  IsCompleted: bool (get) — Returns true once the process has completed, regardless of outcome.
  IsRunning: bool (get/set) — Indicates whether the process is currently running.
  MaxRecoveryAttempts: int (get/set) — Controls how many recoverable errors are tolerated before the operation
             is aborted. A recoverable error is one where the input could be sanitized
             and processing continued, for example a control character or trailing comma
             in a JSON record.
            
             When this threshold is reached a JsonException is thrown, the operation
             is aborted, Succeeded is set to false, and the error that caused the limit
             to be exceeded is added to UnrecoverableErrors.
            
             Defaults to int.MaxValue, meaning fully tolerant. Set to 0 to treat any
             recoverable error as fatal.
  ProgressPercent: int (get/set) — Returns the completion percentage of the process, from 0 to 100.
  RecoverableErrors: List<ProcessErrorCount> (get/set) — Recoverable errors encountered during the operation, grouped by Source and Message.
            Each entry contains the ProcessError describing the first occurrence and a count
            of how many times that same error was seen. ParseError and Exception on each
            ProcessError reflect the first occurrence only.
            Populated as the operation runs. Cleared when the monitor is reused.
  StartTime: DateTime (get/set) — Timestamp of when the process started.
  Succeeded: bool (get/set) — Returns true if the process finished successfully.
  ThreadPriority: ThreadPriority (get/set) — Allows clients to set the priority of threads executing the process.
  TimeoutSeconds: int (get/set) — Sets the maximum time in seconds a process may run before it is considered
            timed out. Use Timeout.Infinite to disable the limit.
  UnrecoverableErrors: List<ProcessError> (get/set) — Unrecoverable errors that caused the operation to abort. Each entry is a
            distinct fatal error. Succeeded is set to false whenever an entry is added.
            Also populated when the MaxRecoveryAttempts threshold is exceeded, in which
            case the error that triggered the limit is promoted here from RecoverableErrors.
            Cleared when the monitor is reused.

Methods:
  void Cancel()
    Requests cancellation of the monitored process.
  void Dispose()
    Disposes the monitor and frees all resources.
  bool WaitForCompletion()
    Blocks until the process completes or the timeout elapses.
            Returns false immediately if the process was cancelled.
  Task<bool> WaitForCompletionAsync()
    Waits asynchronously for the process to complete or timeout.
  void WaitForProcessStarted(int timeoutMilliseconds = -1)
    Blocks until the process has started. May be used in the synchronous case
            to avoid race conditions.
  void MarkFinished()
    Marks the process as finished. Intended for use by internal and cloud API processes.

---

# Field

Fields are discovered automatically via `Init()` and then configured before `Load()`. The engine is schema-less — you never declare fields up front.

```csharp
engine.Init(fstream);

engine.GetField("name")!.Searchable = true;
engine.GetField("name")!.Weight = 2.0f;
engine.GetField("description")!.Searchable = true;
engine.GetField("category")!.Filterable = true;
engine.GetField("category")!.Facetable = true;
engine.GetField("price")!.Filterable = true;
engine.GetField("price")!.Sortable = true;
```

## Configuration Properties

| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `Searchable` | `bool` | `false` | Include this field in the text search index |
| `Filterable` | `bool` | `false` | Allow this field to be used in filters and boosts |
| `Facetable` | `bool` | `false` | Return value/count pairs for this field with search results |
| `Sortable` | `bool` | `false` | Allow sorting by this field. Score is always the primary sort key |
| `WordIndexing` | `bool` | `false` | Build a full word index for exact single-word matching. Useful for large datasets with many repeated values |
| `HighResolution` | `bool` | `false` | Also index N-grams over the field text with all delimiters removed (e.g. `"hvis feks"` additionally indexes `"hvisfeks"`), and strip delimiters from the query too, so a run-together or split query matches across removed delimiters. Only meaningful when `Searchable` |
| `Embeddable` | `bool` | `false` | Mark this field for embedding-based (vector) search. Embeddable fields are excluded from the text search index |
| `PreloadFilters` | `bool` | `false` | Cache a filter for each distinct value during `Load`. Accessible via `SearchEngine.PreLoadedFilters`. Faster repeated filtering at the cost of memory |
| `HighResolution` | `bool` | `false` | Add extra delimiter-removed N-grams to sharpen run-together / split-word matches. Coverage already matches these, so this lifts precision at the cost of index space — best on a few key fields like titles. Only when `Searchable` |
| `Weight` | `float` | `1.0f` | Relative score weight for this field. Higher values make matches in this field rank higher |
| `BM25b` | `float` | `0.75f` | Length normalisation parameter. `0` = no normalisation, `1` = full normalisation. Applies in both BM25F and BM25PerField modes |
| `BM25k1` | `float` | `1.2f` | Term-frequency saturation. Typical range 1.0–2.0. Setting different values across fields forces BM25PerField mode; uniform values select BM25F mode |

## Read-Only Properties

| Property | Type | Description |
|----------|------|-------------|
| `Name` | `string` | Field name as it appears in the JSON data |
| `Type` | `JsonValueKind` | Value kind: `String`, `Number`, `True`/`False`, `Array`, `Object` |
| `IsArray` | `bool` | Whether this field holds an array of values |
| `Optional` | `bool` | `true` if the field is absent or null in at least one document |
| `EmbeddingDimensions` | `int` | Detected vector dimension. `0` until data is loaded with `Embeddable = true` |

> Setting `PreloadFilters = true` automatically sets `Filterable = true` on the same field.

---

## FieldProxy

`FieldProxy` is used by `GetFieldConfiguration()` (returns `FieldProxy[]`) and `SetFieldConfiguration(FieldProxy[])`. All boolean and float properties are nullable — `null` on a set call means "leave unchanged"; every property is populated on a get call.

```csharp
// Read the full configuration
FieldProxy[] config = engine.GetFieldConfiguration();

// Apply changes — only non-null values are written
engine.SetFieldConfiguration([
    new FieldProxy { FieldName = "title",       Searchable = true, Weight = 2.0f },
    new FieldProxy { FieldName = "description", Searchable = true, Weight = 1.0f },
    new FieldProxy { FieldName = "category",    Filterable = true, Facetable = true },
    new FieldProxy { FieldName = "price",       Filterable = true, Sortable = true },
]);
```

| Property | Type | Description |
|----------|------|-------------|
| `FieldName` | `string` | Must match a field name discovered by `Init()` |
| `FieldType` | `string?` | Read-only on GET (`"String"`, `"Number"`, `"Boolean"`). Ignored on SET |
| `IsArray` | `bool?` | Read-only on GET. Ignored on SET |
| `Searchable` | `bool?` | Include in text search index |
| `Filterable` | `bool?` | Allow use in filters and boosts |
| `Facetable` | `bool?` | Return value/count pairs with results |
| `Sortable` | `bool?` | Allow sorting by this field |
| `WordIndexing` | `bool?` | Exact word matching in coverage |
| `Embeddable` | `bool?` | Vector embedding field |
| `PreloadFilters` | `bool?` | Cache a filter per distinct value during `Load`. Also forces `Filterable = true` |
| `HighResolution` | `bool?` | High-resolution indexing for run-together/split-word precision. Best on a few key fields like titles |
| `Weight` | `float?` | Score weight multiplier |
| `BM25b` | `float?` | Length normalisation parameter |
| `BM25k1` | `float?` | TF saturation parameter |

---

## Scoring Modes

Setting `BM25k1` differently across fields controls which scoring algorithm the engine uses after `Index()`:

```csharp
// BM25F mode (default) — uniform k1, single shared index
engine.GetField("title")!.BM25k1 = 1.2f;
engine.GetField("body")!.BM25k1  = 1.2f;   // same → BM25F

// BM25PerField mode — divergent k1, one index per field
engine.GetField("title")!.BM25k1 = 2.0f;
engine.GetField("body")!.BM25k1  = 1.2f;   // different → BM25PerField
```

Inspect `SearchEngine.ScoringMode` after `Index()` to confirm which path was selected.

## Field Weights

`Weight` is a plain `float`. The practical range is **0.5–3.0**. The ratio between fields matters — `2.0` and `1.0` produces the same relative effect as `4.0` and `2.0`.

```csharp
engine.GetField("title")!.Weight    = 3.0f;
engine.GetField("heading")!.Weight  = 2.0f;
engine.GetField("metadata")!.Weight = 1.5f;
engine.GetField("body")!.Weight     = 1.0f;
```

There is no hard maximum, but values above 5× the lowest weight in your set offer diminishing returns. If one field needs to dominate that strongly, it is usually better to make it the only searchable field.


## Field (class)

This is a meta class of a Field. It contains a set of properties which are important with respect
            to searching, filtering, faceting, boosting etc.


Properties:
  BM25b: float (get/set) — Per-field length-normalization parameter b for BM25 / BM25F scoring.
            Range [0, 1]: 0 = no length normalization, 1 = full normalization.
            Typical values: 0.3 for short fields (title), 0.75 for longer fields (body).
            Defaults to 0.75. Applies in both(per-field BM25) and(multi-field BM25F).
  BM25k1: float (get/set) — Per-field term-frequency saturation parameter k1 for BM25 scoring.
            Typical range: 1.0–2.0. Lower values saturate TF faster (diminishing
            returns on repeated terms). Defaults to 1.2.
  HighResolution: bool (get/set) — High-resolution indexing for this field. When true, the tokenizer ALSO generates N-grams
            over the field text with all delimiters removed (e.g. "hvis feks" additionally indexes
            "hvisfeks"), and a search whose words are run together (or split) still matches across the
            removed delimiters. The query side strips delimiters too whenever at least one searchable
            field is high-resolution. Delimiters are the ones defined by the engine's configuration.
            Off by default; opt in per field.
  Embeddable: bool (get/set) — When true, this field holds a vector embedding — a fixed-length array of
             float values produced by an embedding model. Fields must be set embeddable
             after Init during the fieldconfiguration.
            
             Embeddable fields are excluded from the standard text search index.
             During Load their float[] values are collected into a per-field list that
             is passed to the HNSW index for approximate nearest-neighbour search.
             The expected dimensionality is fixed on the first document loaded and
             validated on all subsequent ones.
            
             Setting this to false on a field whose name matches the auto-detection
             pattern overrides the automatic classification.
  EmbeddingDimensions: int (get/set) — The expected number of dimensions for this embedding field.
            Set automatically from the first document loaded during Load.
            Zero means not yet determined. Ignored when Embeddable is false.
  Facetable: bool (get/set) — Set to true if this field is going to be used as a facetable field.
  Filterable: bool (get/set) — Set to true if this field is going to be used as a filterable field.
  Searchable: bool (get/set) — sets whether the content should be searchable.
  Sortable: bool (get/set) — Must be set in order to sort on this field.
            This will be a secondary search key – score is always first.
            Both this field and the sort order must be passed to Query.
  WordIndexing: bool (get/set) — Enables exact word matching for single-word queries.
            When enabled, records containing an exact match of the query word are prioritized in the result list.
            For example, searching for "computer" will rank records containing the word "computer" topmost.
            This feature is only applicable in Coverage mode. Any field marked with this property set to true
            will get its values indexed for exact matching.
            The minimum word size defined by the tokenizersetup of the default configuration (400 default, that is set to 2).
            These matches are never truncated and the max word size is 8 characters.
            Case sensitivity is determined by configuration settings. For array fields one matching element is considered a hit.
            Note: Exact word matches are given a compensated top score (typically 255, 254,253 or 252),
            adjusted based on the client's boost setting. This property itself does not apply any boost.
            Words are split using the delimiters defined in TokenizerSetup of configuration.
  IsArray: bool (get/set) — Is true for a possibly flattened array.
  Name: string (get/set) — The name of the field. The name must be unique.
  Optional: bool (get/set) — Optional field
  PreloadFilters: bool (get/set) — Setting this will make engine setup a valuefilter for each distinct value of a field.
            The filters will be created and loaded during Load and may be accessed via
            SearchEngine.GetPreloadedFilters.
  Type: JsonValueKind (get/set) — Only primitive types and their arrays are accepted. Note that for fields containing
            flattened arrays they will all have primitive types, and not JsonArray.
            Use the IsArray property to check.
  Value: object (get/set) — Actual value of the field.
  Weight: float (get/set)

Methods:
  Field Clone()
    Creates a deep copy of this field. The clone is detached: no NotifyChanged
            subscribers are carried over, and UniqueSet is duplicated so callers can
            mutate the clone without affecting the original. Backing fields for
            Searchable/Sortable/Weight/PreloadFilters are assigned directly so the
            clone's NotifyChanged event is silent during construction.
  static bool ValidateFieldValue(Field field, object value, String& error)
    Validates that a value is compatible with the field's type and array requirements.

## ScoringMode (enum)

Selects which BM25 scoring algorithm the engine uses when more than one
            field is marked as searchable. The mode is determined automatically attime from the per-fieldvalues:


Values:
  BM25PerField = 0
  BM25F = 1

## DocumentFields (class)

JsonParserMeta class for defining fields in a document.  Supports parsing JSON
            to extract fields and saving / loading field configuration to file.
            JSON repair (control characters, trailing commas) is delegated torather than being duplicated here.


Properties:
  NameOfDocumentKeyField: string (get/set) — Name of the key field used to identify documents.
            Required for operations such as document deletion.
  AnalyzedDocumentCount: int (get) — Number of documents the analyze pass (Init) saw in the stream. A value of 1 on a
            large file is the classic "export envelope" symptom: a root object wrapping the
            real document array, which the parser reads as one giant document. Callers can
            use this to warn before Load/Index is attempted.

Methods:
  void Dispose()
  List<Field> GetFacetableFieldList()
    Returns the list of fields marked as Facetable.
  Field GetField(string fieldName)
    Returns the field with the given name, or null if not found.
  bool RequiresReindex(FieldProxy[] proposed)
    Returns true if any property change in the proposed configuration requires
            rebuilding the inverted/word/vector index. Reindex-requiring properties are
            Searchable, WordIndexing, Embeddable, BM25b, BM25k1 and HighResolution. Filterable,
            Facetable, Sortable, Weight and PreloadFilters take effect at query time without reindex.
  DocumentFields Clone()
    Creates a deep copy of this DocumentFields instance with cloned Field objects.
            Cached field lists (searchable/facetable) are intentionally not copied; they
            rebuild lazily from the cloned _fields dictionary on next access.
            The clone has its own NotifyChanged event chain (no subscribers from the original).
  List<Field> GetFieldList()
    Returns a list of all fields.
  List<Field> GetFilterableFieldList()
    Returns all fields marked as filterable.
  string GetSerialized()
    Returns the instance serialized as a JSON string.
  bool Haskey()
    Returns true when this instance has a key field of the expected type.
  void SaveToFile(string fileName)
    Serializes the field configuration to a file.
  static DocumentFields Analyze(string jsonDocument, String& errorMessage)
    JsonParserAnalyzes a JSON string and extracts its field structure.
            Delegates JSON repair tobefore parsing.
  static DocumentFields Analyze(Stream jsonStream, String& errorMessage)
    JsonParserAnalyzes a JSON stream and extracts its field structure.
            Delegates JSON repair tobefore parsing.
  static void Analyze(string jsonDocument, DocumentFields documentFields, String& errorMessage)
    DocumentFieldsAnalyzes a JSON string into an existinginstance.
  static Task<ValueTuple<DocumentFields, string>> AnalyzeAsync(Stream jsonStream)
    String@)Async variant of.
  static DocumentFields ReadFromFile(string fileName)
    Deserializes an instance from a previously saved file.

---

# Query

The `Query` object configures what and how to search. At minimum, provide search text and a max results count.

## Constructor

```csharp
var query = new Query("search text", maxResults);
```

## Full Configuration

```csharp
var query = new Query("search text", maxResults)
{
    EnableCoverage  = true,              // default: true
    CoverageDepth   = 500,               // default: 500
    CoverageSetup   = coverageSetup,     // default: null
    EnableFacets    = false,             // default: false
    EnableBoost     = false,             // default: false
    SortBy          = engine.GetField("rating"), // default: null
    SortAscending   = false,             // default: false
    RemoveDuplicates = true,             // default: true
    Filter          = filter,            // default: null
    Boosts          = boostArray,        // default: null
    FieldBoosts     = new Dictionary<string, float>
    {
        ["title"] = 2.0f,
        ["body"]  = 1.0f
    },                                   // BM25F mode only, default: null
    KeyExcludeFilter = excludeSet,       // exclude specific document keys
    KeyIncludeFilter = includeSet,       // restrict to specific document keys
    TimeOutLimitMilliseconds = 1000      // default: 1000, max: 10000
};
```

## Properties

| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `Text` | `string` | `""` | Search text |
| `MaxNumberOfRecordsToReturn` | `int` | — | Maximum results to return |
| `EnableCoverage` | `bool` | `true` | Enable coverage-based ranking |
| `CoverageDepth` | `int` | `500` | How many candidate documents coverage evaluates |
| `CoverageSetup` | `CoverageSetup?` | `null` | Fine-grained coverage configuration |
| `EnableFacets` | `bool` | `false` | Return facet counts in the result |
| `EnableBoost` | `bool` | `false` | Apply boosts defined in `Boosts` |
| `SortBy` | `Field?` | `null` | Secondary sort field. Score is always the primary key |
| `SortAscending` | `bool` | `false` | Sort direction when `SortBy` is set |
| `RemoveDuplicates` | `bool` | `true` | Remove duplicate documents from results |
| `Filter` | `Filter?` | `null` | Boolean filter applied before scoring |
| `Boosts` | `Boost[]?` | `null` | Score boosts applied to matching documents |
| `FieldBoosts` | `Dictionary<string, float>?` | `null` | Per-query field weight overrides. Only applies in BM25F mode. Keys are field names, values are multipliers folded into the BM25F pseudo term-frequency |
| `KeyExcludeFilter` | `HashSet<long>?` | `null` | Document keys to exclude from results |
| `KeyIncludeFilter` | `HashSet<long>?` | `null` | Restrict results to only these document keys |
| `TimeOutLimitMilliseconds` | `int` | `1000` | Abort scoring after this many milliseconds. Max: 10000 |

## Per-Query Field Boosts

`FieldBoosts` lets you adjust field weights at query time without rebuilding the index. Only effective in BM25F mode.

```csharp
var query = new Query("noise cancelling", 10)
{
    FieldBoosts = new Dictionary<string, float>
    {
        ["title"]       = 3.0f,  // triple weight for title matches
        ["description"] = 1.0f,
        ["tags"]        = 0.5f
    }
};
```

## Key Filters

`KeyIncludeFilter` and `KeyExcludeFilter` restrict results by document key without building a `Filter`. Useful for user-level scoping.

```csharp
// Only search within a known set of documents
query.KeyIncludeFilter = new HashSet<long> { 101, 205, 388 };

// Exclude recently seen documents
query.KeyExcludeFilter = new HashSet<long> { 42, 77 };
```


## Query (class)

Query class to be applied for Json documents


Properties:
  Boosts: Boost[] (get/set) — is the method to pass the list of boosts to be applied during the search.
  CoverageDepth: int (get/set) — The number of results from the pattern matcher to be considered by Coverage.
  CoverageSetup: CoverageSetup (get/set) — Field that controls behavior of the Coverage algorithms. Have no effect for other
            algorithms.
  DocumentsBoosted: int (get) — Returns the number of documents that may be boosted in the search.
  EnableBoost: bool (get/set) — Is used to toggle boosted searches on or off.
  EnableCoverage: bool (get/set) — Coverage is a function that detects exact matching on tokens in the query, and lifts them up to the top of the result list.
            If set to false, the search will run with pattern recognition (Relevancy Ranking) only.
            When Coverage is active, the search result will also return an index number to truncate the list on.
  EnableFacets: bool (get/set) — Activates calculations of facet histograms in the search results.
  FieldBoosts: Dictionary<string, float> (get/set) — Per-field BM25F boost factors applied at query time.
            Key = field name (must match a searchable field name).
            Value = multiplicative boost for that field's contribution to pseudo_tf.
            Fields not listed default to 1.0.
            Typical range: 0.5–3.0. Values below 1.0 de-emphasize a field; values above ~5.0
            tend to overwhelm the IDF component and reduce result quality.
            Example: { "title": 2.0f, "body": 1.0f } doubles the title field's influence.
  Filter: Filter (get/set) — Is used to pass a filter to the search query.
  LogPrefix: string (get/set) — If the instance of the search engine has had a logger injected into its constructor,
            every search will be logged. The log prefix is then first added to the log text,
            and can be used by the client to add statistical information, for example for use in dashboards.
  MaxBoost: int (get/set) — Denotes the Maximum possible boost that can be assigned. Note that a hit with full boost will get score 255.
            Hence full "unboosted" score will be 255 - MaxBoost.
  MaxNumberOfRecordsToReturn: int (get/set) — Defines the maximum number of documents to be returned.
  RemoveDuplicates: bool (get/set) — Removes all duplicates of documents with the same foreign key. Only the one with the best score
            value is returned in the results list. If foreign keys are not in use, this can be set to null.
  SerializedQueryKey: string (get) — Serialized key for caching empty search results.
            Format: F:{filterKey}|B:{boostFilter1};{strength1},{boostFilter2};{strength2}|S:{sortField};{asc}|M:{maxRecords}|E:{enableFacets}
  SortAscending: bool (get/set) — Only applicable if SortBy != null.
  SortBy: Field (get/set) — Use this in order to setup sorting of the results.
  Text: string (get/set) — Refers to the text (input) to be searched for.
  TimeOutLimitMilliseconds: int (get/set) — Sets maximum waiting time in case of overloaded CPU. We recommend 1000 ms. Cannot be more than 10000 ms.

---

# Results

Search returns a `Result` object containing ranked document keys and scores — not full JSON. Retrieve full documents with `GetJsonDataOfKey`.

## Basic Usage

```csharp
var result = engine.Search(query);
if (result != null)
{
    foreach (var rec in result.Records)
    {
        long key   = rec.DocumentKey;
        int  score = rec.Score;
        string json = engine.GetJsonDataOfKey(key);
        Console.WriteLine($"[{score}] {json}");
    }
}
```

## Result Properties

| Property | Type | Description |
|----------|------|-------------|
| `Records` | `ScoreEntry[]` | Ranked documents. Each has `DocumentKey` (long) and `Score` (byte, 0–255) |
| `Facets` | `Dictionary<string, KeyValuePair<string, int>[]>?` | Field → value/count pairs. `null` when `EnableFacets` is false |
| `TruncationIndex` | `int` | Index in `Records` where coverage truncation occurred. `-1` if no truncation |
| `TruncationScore` | `int` | Score at the truncation point |
| `DidTimeOut` | `bool` | `true` if scoring was cut short by `TimeOutLimitMilliseconds` |

## Working with Facets

```csharp
query.EnableFacets = true;
var result = engine.Search(query);

if (result.Facets != null)
{
    foreach (var facet in result.Facets)
    {
        Console.WriteLine($"{facet.Key}:");
        foreach (var bucket in facet.Value)
            Console.WriteLine($"  {bucket.Key} ({bucket.Value})");
    }
}
```

## Retrieving Full Documents

```csharp
// Single document
string json = engine.GetJsonDataOfKey(result.Records[0].DocumentKey);

// Multiple documents
var keys = result.Records.Select(r => r.DocumentKey);
foreach (var json in engine.GetJsonDataOfKeys(keys))
    Console.WriteLine(json);
```

## ScoreEntry

| Property | Type | Description |
|----------|------|-------------|
| `DocumentKey` | `long` | Unique key identifying the document |
| `Score` | `byte` | Relevance score 0–255. Higher is more relevant |


## Result (class)

Query)Search result returned by.


Properties:
  DidTimeOut: bool (get/set) — True when the search could not be served, rather than served and found
            nothing. Two situations set it:
  Facets: Dictionary<string, KeyValuePair<string, int>[]> (get/set) — Facet counts per facetable field, keyed by field name. Null when no facetable fields are configured.
  Records: ScoreEntry16[] (get/set) — Ranked search hits in descending score order.
  TruncationIndex: int (get/set) — Internal index at which results were truncated to the requested top-k limit.
  TruncationScore: UInt16 (get/set) — Score threshold at which results were truncated.

Methods:
  static Result MakeEmptyResult(bool timedOut = false)
    Returns an empty result, optionally marked as having timed out.

---

# Filters

Create filters on fields marked as `Filterable`. Filters must be created after `Init()` and `Load()` are complete.

> Never filter results client-side. Always use server-side filters so the engine applies them during search.

Filter creation returns `null` on failure and reports the reason via an `out string? error`
parameter (e.g. the field is not `Filterable`, or a value can't be parsed for the field type).

## Value Filters

```csharp
Filter? categoryFilter = engine.CreateValueFilter("category", "electronics", out string? error);
if (categoryFilter is null)
    throw new InvalidOperationException(error); // e.g. "Field 'category' is not filterable"
```

## Range Filters

```csharp
Filter? priceFilter = engine.CreateRangeFilter("price", 10.0, 100.0, out string? error);
```

## Combining Filters

```csharp
// AND — both conditions must match
Filter combined = categoryFilter & priceFilter;

// OR — either condition matches
Filter either = categoryFilter | priceFilter;

// NOT — negate a filter
Filter excluded = !categoryFilter;
```

## Using Filters in Queries

```csharp
query.Filter = combined;

// Check how many documents match
int count = combined.NumberOfDocumentsInFilter;

// Reset filter
query.Filter = null;
```

## Preloading Filters

For large datasets where many filters are applied simultaneously:

```csharp
engine.LoadFilters(new[] { categoryFilter, priceFilter }, maxThreadCount: 2);
// Or preload all: engine.LoadAllFilters(maxThreadCount: 4);
```


## Filter (class)

Common base class for all types of filters currently
            RangeFilter and ValueFilter. Supports logical operators.
            Note that "\0" and "\1" is used as left and rigth brackets to
            avoid strings that can occur in json text.


Properties:
  IsArray: bool (get) — Set during Analyze if JsonObject is an Array.
  NumberOfDocumentsInFilter: int (get) — Returns the number of documents that are contained in the filter.
  SerializedKey: string (get) — Used internally exposed for ease of debugging.
  NeedsLoading: bool (get/set) — May be used in order to check if filter is loaded.

## RangeFilter (class)

RangeFilter allows filtering within a range for a number datatype
            or a text datatype that can be parsed to a Double.


Properties:
  Culture: CultureInfo (get) — Cultureinfo determines how double is parsed.
  SerializedKey: string (get) — Make the SerializedKey such that the filter parameters are recoverable by parsing.

## ValueFilter (class)

Filter class to test for equality.


Properties:
  SerializedKey: string (get)

---

# Boosts

Boost results matching certain criteria without excluding non-matching results. Boosts must be created after `Load()` completes.

## Creating Boosts

```csharp
// BoostStrength enum: Low = 1, Med = 2, High = 3
Filter yearFilter = engine.CreateRangeFilter("year", 1980, 2025, out _)!;
Filter genreFilter = engine.CreateValueFilter("genre", "Documentary", out _)!;

var boosts = new List<Boost>();
Boost b = engine.CreateBoost(yearFilter & genreFilter, BoostStrength.Med);
Console.WriteLine($"Documents boosted: {b.DocumentsBoosted}");
boosts.Add(b);

query.Boosts = boosts.ToArray();
query.EnableBoost = true;
```

## Personalized Boosting

For user-specific boosting across many items, OR per-item value filters into one
filter and boost it:

```csharp
Filter? frequentPurchases = null;
foreach (long itemId in userFrequentItemIds)
{
    Filter f = engine.CreateValueFilter("item_id", itemId, out _)!;
    frequentPurchases = frequentPurchases is null ? f : frequentPurchases | f;
}

var userBoosts = new List<Boost>();
userBoosts.Add(engine.CreateBoost(frequentPurchases!, BoostStrength.Med));

query.Boosts = userBoosts.ToArray();
query.EnableBoost = true;
```

Boosting can handle hundreds of thousands of items without significant performance impact.


## Boost (class)

Api class for a Boost definition


Properties:
  BoostStrength: BoostStrength (get/set) — Defines the boost strength.
  DocumentsBoosted: int (get) — Returns the number of documents boosted.
  Filter: Filter (get/set) — Filter which defines the set of documents to be boosted.

## BoostStrength (enum)

Defines the magnitude of the boost to
            be applied


Values:
  Low = 1
  Med = 2
  High = 3

---

# Coverage

Coverage is a collection of algorithms that detect exact and near-exact token matches in the query, lifting them to the top of the result list. It complements the core pattern recognition which always runs first.

## How It Works

After pattern matching produces ranked candidates, coverage analyzes the top-K results (controlled by `CoverageDepth`) and scores each 0–255 based on how well the query tokens are represented. Results confirmed by coverage are promoted above pure pattern matches.

## CoverageSetup

```csharp
CoverageSetup cov = new CoverageSetup();

// Strict mode: only whole words
cov.CoverFuzzyWords = false;
cov.CoverPrefixSuffix = false;
cov.CoverWholeQuery = false;
cov.CoverJoinedWords = true;
cov.CoverWholeWords = true;
cov.MinWordSize = 3;

query.CoverageSetup = cov;
```

## Detection Algorithms

| Property | Default | Description |
|----------|---------|-------------|
| `CoverWholeQuery` | `true` | Detect the whole search query as a single string |
| `CoverWholeWords` | `true` | Detect individual whole words from the query |
| `CoverFuzzyWords` | `true` | Detect words with minor error tolerance |
| `CoverJoinedWords` | `true` | Detect words that are joined or split |
| `CoverPrefixSuffix` | `true` | Detect incomplete strings as prefix or suffix |

## Result Control

| Property | Default | Description |
|----------|---------|-------------|
| `Truncate` | `true` | Cut results at the truncation index |
| `IncludePatternMatches` | `true` | Include pure pattern matches |
| `TruncationScore` | `255` | Score threshold for truncation |
| `TruncateWordHitLimit` | `1` | Minimum query words that must match |
| `TruncateWordHitTolerance` | `0` | Max difference in word hit count from best result |

## Common Patterns

Coverage should be kept enabled for nearly all use cases, including search-as-you-type. It is fast and improves result quality across multiple searchable fields.

**Agent/tool usage (near-exact hits only):**
```csharp
CoverageSetup cov = new CoverageSetup();
cov.IncludePatternMatches = false;
query.CoverageSetup = cov;
```

**Run coverage but keep all results:**
```csharp
CoverageSetup cov = new CoverageSetup();
cov.Truncate = false;
query.CoverageSetup = cov;
```

**Deep coverage (evaluate all documents):**
```csharp
query.CoverageDepth = engine.Status.DocumentCount;
```


## CoverageSetup (class)

Coverage is a function that detects exact and near-exact matching on strings and tokens in the query,
            and lifts them up to the top of the result list. Coverage is a collection of algorithms that works together,
            and this complements the pattern matching algorithm that runs first.
            CoverageSetup is a member of Query.


Properties:
  IncludePatternMatches: bool (get/set) — Gets or sets a value indicating whether pure pattern matches that are not detected by Coverage are included in the results.
            Set this to false if you only want near-exact matches returned. Should be used with Coverage enabled.
  LevenshteinMaxWordSize: int (get/set) — Longest word for Coverage to do edit distance calculation.
            This does not affect the fault tolerance of the pattern recognition search.
            Max value is 63.
  MinWordSize: int (get/set) — Smallest number of characters to hit for CoverageScoreEntry to give full word hits score.
  TruncateWordHitLimit: int (get/set) — The smallest number of words to hit for truncation of the search result list.
            The effective limit may be higher, since it is also determined by
            TruncateWordHitTolerance.
  TruncateWordHitTolerance: int (get/set) — Maximum difference in number of word hits to maxWordHits to truncate the result list.
            maxWordHits is calculated for entire result list. The truncation criteria is expressed by
            bool doTruncate = wordHits >= Max(TruncateWordHitLimit, maxWordHits - TruncateWordHitTolerance).
  CoverWholeQuery: bool (get/set) — Sets whether to detect the whole search query as a string.
  CoverWholeWords: bool (get/set) — Sets whether to detect whole words from the string in the result list. This will look for multiple words.
  CoverFuzzyWords: bool (get/set) — Sets whether to detect words with an edit distance of 1.
  CoverJoinedWords: bool (get/set) — Sets whether to detect words that are either joined or split up. Both will be returned in the same query.
  CoverPrefixSuffix: bool (get/set) — Sets whether to detect incomplete strings as prefix or suffix of a bigger word.
  Truncate: bool (get/set) — If Truncate is enabled the list of result will be truncated at
            the point of TruncationIndex.
  TruncationScore: UInt16 (get/set) — Always truncate at or above this score. Represents the 16-bit score
            of the truncation point in the search result (0..65535).
            Default 65024 is the 16-bit equivalent of the old byte default 254 (254 * 257).

---

# SystemStatus

Access via `engine.Status`. Returns a snapshot of the current engine state, counters, and any error conditions.

```csharp
var status = engine.Status;

Console.WriteLine($"State:     {status.SystemState}");
Console.WriteLine($"Documents: {status.DocumentCount}");
Console.WriteLine($"Version:   {status.Version}");
```

## Properties

| Property | Type | Description |
|----------|------|-------------|
| `SystemState` | `SystemState` | Current lifecycle state |
| `DocumentCount` | `int` | Number of documents in the index |
| `SearchCounter` | `int` | Total number of `Search()` calls since creation |
| `SecondsToIndex` | `double` | Duration of the last `Index()` call in seconds |
| `Version` | `string` | Indx library version number |
| `LicenseInfo` | `LicenseInfo` | License status and limits |
| `ErrorMessage` | `string` | Human-readable error description. Empty string means no error |
| `InvalidArgument` | `bool` | `true` if `Query` was null |
| `InvalidState` | `bool` | `true` if `Index()` was called with zero documents, or `Search()` was called before `Index()` |
| `TooLongSearchText` | `bool` | `true` if query text exceeded the maximum length and was truncated |
| `TooLongClientText` | `bool` | `true` if client text exceeded the maximum length and was truncated |
| `UnknownConfigurationError` | `bool` | `true` if the engine was created with an invalid configuration number |
| `TimeOfInstanceCreation` | `DateTime` | When the constructor was called |
| `TimeOfLastIndexBuild` | `DateTime` | When `Index()` last completed |

## SystemState Enum

```csharp
enum SystemState
{
    Hibernated = -1,
    Created    =  0,
    Loading    =  1,
    Loaded     =  2,
    Indexing   =  3,
    Ready      =  4,
    Error      = 255
}
```

| State | Value | Description |
|-------|-------|-------------|
| `Hibernated` | -1 | Memory released. Call `WakeUp()` to restore |
| `Created` | 0 | Instance created, no data loaded |
| `Loading` | 1 | `Load()` is in progress |
| `Loaded` | 2 | Documents loaded, not yet indexed |
| `Indexing` | 3 | `Index()` is in progress |
| `Ready` | 4 | Ready to search. Dynamic operations are also available in this state |
| `Error` | 255 | An unrecoverable error has occurred |


## SystemStatus (class)

Represents the current state and health of a search engine instance.
            
             In addition to live state such as SystemState and DocumentCount, this class
             carries the error lists from the most recent Init or Load operation. These
             mirror the lists on ProcessMonitor and remain accessible after the monitor
             reference has been discarded.


Properties:
  DocumentCount: int (get/set) — Number of documents currently loaded and indexed.
  ErrorMessage: string (get/set) — Short description of the most recent fatal error, if any.
            An empty string means no fatal error occurred. For full detail on all
            errors from the last operation see RecoverableErrors and UnrecoverableErrors.
  InvalidArgument: bool (get/set) — Set to true if a search or operation was called with a null or invalid argument.
  InvalidDataSetName: bool (get/set) — Only relevant for certain configurations. See individual configuration
            documentation for details.
  InvalidState: bool (get/set) — Set to true if an operation was called in an invalid system state, for example
            if Search is called before Index has completed.
  LicenseInfo: LicenseInfo (get/set) — License information for this engine instance.
  RecoverableErrors: List<ProcessErrorCount> (get/set) — Recoverable errors from the most recent Init or Load operation, grouped by
            Source and Message. Each entry contains the ProcessError describing the first
            occurrence and a count of how many times that same error was seen.
            Mirrored from ProcessMonitor at the end of the operation and remains
            accessible after the monitor reference is discarded.
            An empty list means no recoverable errors were encountered.
  SearchCounter: int (get/set) — Total number of Search calls made since the engine was created.
  SecondsToIndex: double (get/set) — Duration of the most recent Index operation in seconds.
  SystemState: SystemState (get/set) — Current lifecycle state of the engine.
  TimeOfInstanceCreation: DateTime (get/set) — Timestamp of when this engine instance was created.
  TimeOfLastIndexBuild: DateTime (get/set) — Timestamp of the most recent completed Index call.
  TooLongClientText: bool (get/set) — Set to true if the maximum allowed client text length was exceeded during
            loading. The text will have been truncated. This flag is not cleared until
            DeleteAll is called.
  TooLongSearchText: bool (get/set) — Set to true if the maximum allowed search text length was exceeded. The text
            will have been truncated. This flag is not cleared until DeleteAll is called.
  UnrecoverableErrors: List<ProcessError> (get/set) — Unrecoverable errors from the most recent Init or Load operation. Each entry
            is a distinct fatal error that caused the operation to abort. Succeeded on
            the ProcessMonitor was set to false for each entry. Also populated when the
            MaxRecoveryAttempts threshold is exceeded, in which case the error that
            triggered the limit is promoted here.
            Mirrored from ProcessMonitor at the end of the operation.
            An empty list means the operation completed without fatal errors.
  UnknownConfigurationError: bool (get/set) — Set to true if the engine was constructed with an unrecognised configuration number.
  Version: string (get/set) — Version number of the Indx library.

## SystemState (enum)

Reflects the major lifecycle state of a search engine instance.


Values:
  Created = 0
  Loading = 1
  Loaded = 2
  Indexing = 3
  Ready = 4
  Error = 255
  Hibernated = -1

---

# LicenseInfo

Class used to authenticate a user or commercial license with Indx. Access through `engine.Status.LicenseInfo`.

Load a `.license` file via the SearchEngine constructor:

```csharp
var engine = new SearchEngine("filename.license");

var license = engine.Status.LicenseInfo;
Console.WriteLine($"Licensed: {license.Licensed}");
Console.WriteLine($"Licensed to: {license.LicensedTo}");
Console.WriteLine($"Document limit: {license.DocumentLimit}");
```

## Properties

| Property | Type | Description |
|----------|------|-------------|
| `LicenseFileName` | `string` | Filename of the loaded `.license` file |
| `Licensed` | `bool` | True if a correct `.license` file has been loaded |
| `ValidLicense` | `bool` | True if all parameters are within specification |
| `Type` | `string` | License type: developer, commercial, or other |
| `Description` | `string` | Information about the loaded license |
| `LicensedTo` | `string` | Name of the license holder |
| `DocumentLimit` | `int` | Maximum number of documents allowed. Without a license: 100,000 |
| `DocumentLimitExceeded` | `bool` | True if more documents are loaded than the license permits |
| `ExpirationDate` | `DateTime` | Date when the license expires |

## License Tiers

| Tier | Document Limit | Cost |
|------|---------------|------|
| No license | 100,000 | Free |
| Extended license | Unlimited | Free — register at [indx.co](https://indx.co) |
| Company license | Unlimited + SLA | Paid |


## LicenseInfo (class)

Class to define and read the licensing system.


Properties:
  Description: string (get/set) — Gets the description of the license.
  DocumentLimit: int (get/set) — Gets the maximum number of documents allowed by the license.
  DocumentLimitExceeded: bool (get/set) — Gets a value indicating whether the document limit has been exceeded. Note that this limit is on the underlying core documents, so a JSON document with multiple fields and long text can be split into multiple core documents.
  ExpirationDate: DateTime (get/set) — Gets the expiration date of the license.
  LicenseFileFound: bool (get/set) — Gets a value indicating whether a license file was found at the specified path.
            This is true even if the license has expired or is otherwise invalid.
            Use this to distinguish "no file present" from "file present but not valid".
  Licensed: bool (get/set) — Gets a value indicating whether the application is licensed.
  LicensedTo: string (get/set) — Gets the name of the license holder. Person or company.
  LicenseFileName: string (get/set) — Gets or sets the file name of the license.
  Type: string (get/set) — Gets the type of the license.
  ValidLicense: bool (get/set) — Gets a value indicating whether the license is valid.

Methods:
  static LicenseInfo GetLicenseInfo(string licenseFileNameAndPath, Boolean& isObfuscated)

---

# HTTP API Reference


---

# Data Loading

## Initial Load Workflow

Every dataset endpoint is scoped to a team and dataset — prefix each operation below with `/api/teams/{teamName}/datasets/{dataSetName}/`.

```
1. PUT   (the dataset route)     → create or open dataset (201 created / 200 existed)
2. POST  analyze                 → discover fields from JSON
3. PUT   fields/configuration    → configure fields (204)
4. POST  load                    → load documents (204)
5. POST  index                   → start the index build (202 Accepted)
6. GET   status                  → poll until systemState == 4 (Ready)
7. POST  search                  → search
```

## TypeScript Example

```typescript
import { SystemState, CloudQuery, FieldProxy } from '@indxsearch/indx-types';

const team = 'my-team';      // team that owns the dataset
const dataset = 'products';
const base = `https://your-host/api/teams/${team}/datasets/${dataset}`;

const api = (op: string, init?: RequestInit) =>
  fetch(`${base}/${op}`, {
    headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' },
    ...init,
  });

// 1. Create dataset (PUT on the dataset route itself — 201 created / 200 existed)
await fetch(base, { method: 'PUT', headers: { Authorization: `Bearer ${token}` } });

// 2. Analyze
await api('analyze', { method: 'POST', body: jsonString });

// 3. Configure fields (204 No Content)
const fields: FieldProxy[] = [
  { fieldName: 'name',        searchable: true, weight: 2.0 },
  { fieldName: 'description', searchable: true, weight: 1.0 },
  { fieldName: 'category',    filterable: true, facetable: true },
  { fieldName: 'price',       filterable: true, sortable: true },
];
await api('fields/configuration', { method: 'PUT', body: JSON.stringify(fields) });

// 4. Load (204 No Content)
await api('load', { method: 'POST', body: jsonString });

// 5. Index (202 Accepted — the build starts in the background)
await api('index', { method: 'POST' });

// 6. Poll until ready
let status;
do {
  await new Promise(r => setTimeout(r, 500));
  status = await api('status').then(r => r.json());
} while (status.systemState !== SystemState.Ready);
```

---

## Field Configuration

`PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/configuration`

Configure all field properties in a single call — returns `204 No Content`. `null` properties are ignored (leave current value unchanged).

```json
[
  {
    "fieldName": "name",
    "searchable": true,
    "weight": 2.0,
    "bM25b": 0.75,
    "bM25k1": 1.2
  },
  {
    "fieldName": "category",
    "filterable": true,
    "facetable": true,
    "preloadFilters": true
  },
  {
    "fieldName": "embedding",
    "embeddable": true
  }
]
```

`GET /api/teams/{teamName}/datasets/{dataSetName}/fields/configuration` returns the same `FieldProxy[]` shape with all current values populated.

> The single-role setters (`PUT fields/searchable`, `PUT fields/filterable`, `PUT fields/facetable`, `PUT fields/sortable`, `PUT fields/word-indexing`) change one role at a time. Prefer `PUT fields/configuration`, which configures everything in one call.

---

## Dynamic Operations

Insert, update, and delete documents after the index is built. The index stays in sync without a full rebuild.

### Insert

`POST /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}` — single document (`201`)

```bash
curl -X POST https://your-host/api/teams/<team>/datasets/products/documents/1001 \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '"{ \"id\": 1001, \"name\": \"New Product\", \"price\": 49.99 }"'
```

`POST /api/teams/{teamName}/datasets/{dataSetName}/documents` — batch (`201`)

```bash
curl -X POST https://your-host/api/teams/<team>/datasets/products/documents \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '["{ \"id\": 1001, \"name\": \"A\" }", "{ \"id\": 1002, \"name\": \"B\" }"]'
```

### Update

`PUT /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}` — single document (`204`)

`PUT /api/teams/{teamName}/datasets/{dataSetName}/documents` — batch (array of JSON strings; `204`)

`PATCH /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}` — update one field (`204`)

```json
{ "fieldName": "price", "value": 39.99 }
```

`POST /api/teams/{teamName}/datasets/{dataSetName}/documents/update-by-filter` — update a field on all matching documents

```json
{
  "filter": { "hashString": "abc123..." },
  "fieldName": "inStock",
  "value": false
}
```

Returns the count of updated documents as `{"count": n}`.

### Delete

`DELETE /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}` — single document (`204`)

`DELETE /api/teams/{teamName}/datasets/{dataSetName}/documents` — multiple documents (body: `long[]`; `204`)

`POST /api/teams/{teamName}/datasets/{dataSetName}/documents/delete-by-filter` — all documents matching a filter (`204`)

```json
{ "hashString": "abc123..." }
```

---

## Replace (full reload)

`POST /api/teams/{teamName}/datasets/{dataSetName}/replace`

Atomically swaps **all** documents in the dataset for a fresh JSON payload — the streamed-body equivalent of a full reload. Use it to push a new export of the same dataset without the downtime and data loss of delete-and-recreate.

What it preserves and guarantees:

- **Identity & settings survive** — field configuration, **boost rules**, the declared key field, and the description all carry over (reconciled against the new schema). Only the documents change.
- **Zero downtime** — the old data keeps serving searches until the new index is built, then the engine is swapped atomically.
- **All-or-nothing** — if the new JSON fails to parse or index, the existing dataset is left **completely untouched** and still serving.
- **State-transparent** — works whether the dataset is `Ready`, `Hibernated`, or idle-evicted; the old documents are never reloaded just to be discarded.

```bash
curl -X POST https://your-host/api/teams/<team>/datasets/products/replace \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  --data-binary @products-latest.json
```

The body is the full new dataset as a JSON array of documents — the same shape you pass to `load` / `load/text`. No separate `analyze` / `fields/configuration` / `index` steps are needed; `replace` runs the whole pipeline and returns when the new index is live.

**Response (`200`)** — a summary of how the new schema differed from the previous field configuration:

```json
{
  "added":       ["newField"],   // in the new JSON, not previously configured (loads unconfigured)
  "removed":     ["oldField"],   // previously configured, absent from the new JSON (its config is dropped)
  "typeChanged": ["price"]       // detected type changed; that field's roles were reset
}
```

A boost rule that references a field which disappears simply goes dormant — it is **not** deleted, and it re-applies if a later replace brings the field back.

**Errors**

| Status | Meaning |
|--------|---------|
| `409` | Another index build (a `replace` or a field-config re-index) is already running for this dataset. Retry once it finishes. |
| `400` | The new JSON couldn't be built (`code: loadFailed`) — invalid JSON, an indexing error, or it keeps none of the dataset's currently searchable fields. The old dataset is unchanged. |

> The key field is preserved across a replace, so documents keep stable identities between reloads — provided the declared key field is a unique whole number present on every document. If no field qualifies, the dataset uses auto-generated keys.

---

## Lifecycle

`POST /api/teams/{teamName}/datasets/{dataSetName}/hibernate` — release memory while preserving the document store in the database (`204`).

`POST /api/teams/{teamName}/datasets/{dataSetName}/wakeup` — reload and re-index from the database (`204`). The engine will be back in `Ready` state when `GET status` returns `systemState == 4`.


---

# Endpoints

All endpoints are prefixed with `/api/` and require JWT Bearer authentication unless noted.

**Dataset endpoints are scoped to a team and a dataset:**

```
/api/teams/{teamName}/datasets/{dataSetName}/{operation}
```

The tables below list the `{operation}` for each dataset endpoint. Endpoints that are not dataset-scoped (listing datasets, account) show their full path.

The current API version is `2.0-beta`. It is the default — no version header is needed for standard use. If you want to be explicit, pass it as a header or query string:

```
api-version: 2.0-beta           # header
?api-version=2.0-beta           # query string
```

## Authentication

Create a token on the IndxCloudApi website (the **API Key** page in your account portal, `/Account/ApiKey`), then send it as a Bearer header on every request.

```bash
# Use the token (create one at /Account/ApiKey)
curl -H "Authorization: Bearer <token>" https://your-host/api/...
```

---

## Error responses

Every error is an [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) `ProblemDetails` served as `application/problem+json`, with a machine-readable **`code`** extension — branch on `code` (or the status), never on the `detail` text.

| Status | `code` | Meaning |
|--------|--------|---------|
| `400` | `invalidArgument`, `invalidDatasetName`, `loadFailed`, `operationFailed` | The request or its payload can't be served — see `detail` |
| `401` | — (body-less) | Missing, expired, or invalid bearer token |
| `401` | `invalidCredentials`, `userNotFound` | Login failed |
| `403` | `insufficientRole` | You are a member, but your team role is too low (needs Editor/Admin) |
| `404` | `teamNotFound` | The team doesn't exist **or** you are not a member (identical on purpose, so team names can't be enumerated) |
| `404` | `datasetNotFound`, `documentNotFound` | The addressed dataset or document key doesn't exist |
| `409` | `invalidState`, `shadowBusy` | Wrong lifecycle state, or a background rebuild is already running |
| `500` | `internalError` | Unexpected server error — report the included `traceId` |

Batch document operations are all-or-nothing: a bad key or record rejects the whole batch (missing keys are all named in one `documentNotFound` problem) and nothing is applied. The single-document routes `POST documents/{documentKey}` and `PUT documents/{documentKey}` require the route key to match the body's key field.

A **`409 invalidState`** is returned when a request is valid but the dataset's `systemState` can't serve it — for example calling `search` before the dataset is `Ready`, or `wakeup` when it isn't `Hibernated`:

```json
{
  "status": 409,
  "detail": "Search cannot run on dataset 'products' because it is currently Indexing. Indexing is in progress — retry once the dataset reaches Ready.",
  "code": "invalidState",
  "currentState": "Indexing",
  "allowedStates": ["Ready"],
  "retryable": true
}
```

When `retryable` is `true` (states `Loading` / `Indexing`) the response also carries a **`Retry-After`** header (seconds): poll `GET status` until `Ready`, then retry. When `retryable` is `false` (e.g. `Created`, `Hibernated`, `Error`), take the corrective step from `detail` first — `Created` → `POST load` then `POST index`; `Hibernated` → `POST wakeup`. Resending the same request without changing the state will keep returning 409.

---

## Datasets

| Method | Path / Operation | Description |
|--------|------------------|-------------|
| `GET` | `/api/me/datasets` | List every dataset across the teams you belong to (each with its team and your role) |
| `GET` | `/api/teams/{teamName}/datasets` | List the datasets owned by one team |
| `PUT` | `/api/teams/{teamName}/datasets/{dataSetName}` | Create or open a dataset — `201` if it was created, `200` if it already existed. Optional `?configuration=Production` query selects a configuration |
| `GET` | `status` | Get current engine state and counters |
| `DELETE` | `/api/teams/{teamName}/datasets/{dataSetName}` | Permanently delete the dataset and all its data (team Admin) — `204` |

---

## Data Loading

| Method | Operation | Description |
|--------|-----------|-------------|
| `POST` | `analyze` | Discover fields from a streamed JSON body |
| `POST` | `analyze/text` | Discover fields from a JSON string (`text/plain`) |
| `POST` | `load` | Load documents from a streamed JSON body — `204` |
| `POST` | `load/text` | Load documents from a JSON string (`text/plain`) — `204` |
| `POST` | `replace` | Atomically replace the whole dataset, preserving config + boost rules (zero-downtime) |
| `POST` | `load/from-database` | Reload documents from the persisted database — `204` |
| `POST` | `index` | Start the index build — `202 Accepted`; poll `GET status` until `Ready` |
| `GET` | `documents/count` | Count documents stored in the database — `{"count": n}` |

---

## Field Configuration

| Method | Operation | Description |
|--------|-----------|-------------|
| `PUT` | `fields/configuration` | Configure all field properties in one call (**preferred**) — `204` |
| `GET` | `fields/configuration` | Get current configuration for all fields |
| `GET` | `fields` | List all discovered field names |
| `GET` / `PUT` | `fields/searchable` | List / set searchable fields (`PUT` → `204`) |
| `GET` / `PUT` | `fields/filterable` | List / set filterable fields (`PUT` → `204`) |
| `GET` / `PUT` | `fields/facetable` | List / set facetable fields (`PUT` → `204`) |
| `GET` / `PUT` | `fields/sortable` | List / set sortable fields (`PUT` → `204`) |
| `GET` / `PUT` | `fields/word-indexing` | List / set word-indexed fields (`PUT` → `204`) |
| `PUT` | `fields/embeddable` | Mark fields embeddable so their vectors are indexed (call after analyze, before load) — `204` |
| `GET` / `PUT` | `fields/key` | Get / set the key field (`PUT` returns `needsReloadToReKey` in the body) |

> The single-role `PUT fields/…` setters change one role at a time. Prefer `PUT fields/configuration`, which configures everything in a single call.

---

## Search

| Method | Operation | Description |
|--------|-----------|-------------|
| `POST` | `search` | Full-text search |
| `POST` | `search/vector` | Embedding-based nearest-neighbour search |
| `POST` | `search/hybrid` | Blended text + embedding search |
| `POST` | `documents/lookup` | Retrieve full JSON documents by key (body: key array) |

---

## Dynamic Operations

| Method | Operation | Description |
|--------|-----------|-------------|
| `POST` | `documents/{documentKey}` | Insert a single document — `201` |
| `POST` | `documents` | Batch insert documents — `201` |
| `PUT` | `documents/{documentKey}` | Replace a single document — `204` |
| `PUT` | `documents` | Batch replace documents — `204` |
| `PATCH` | `documents/{documentKey}` | Update one field on one document — `204` |
| `POST` | `documents/update-by-filter` | Update a field on all documents matching a filter — `{"count": n}` |
| `DELETE` | `documents/{documentKey}` | Delete a single document — `204` |
| `DELETE` | `documents` | Delete multiple documents by key array — `204` |
| `POST` | `documents/delete-by-filter` | Delete all documents matching a filter — `204` |

---

## Filters

| Method | Operation | Description |
|--------|-----------|-------------|
| `POST` | `filters/value` | Create a filter matching a specific field value |
| `POST` | `filters/range` | Create a filter matching a field value range |
| `POST` | `filters/combine` | Combine two filters with AND or OR |
| `POST` | `filters/load` | Pre-load all registered filters into memory — `204` |
| `GET` | `filters/count` | Count currently cached filters — `{"count": n}` |
| `POST` | `filters/delete` | Release a cached filter (filter key in the body) — `204` |
| `DELETE` | `filters` | Release all cached filters — `204` |

---

## Boosts

| Method | Operation | Description |
|--------|-----------|-------------|
| `GET` | `boosts` | List the dataset's saved boost rules |
| `PUT` | `boosts` | Replace the dataset's saved boost rules — `204` |
| `DELETE` | `boosts` | Delete all saved boost rules — `204` |
| `POST` | `boosts/from-filter` | Create a score boost for documents matching a filter |

---

## Lifecycle

| Method | Operation | Description |
|--------|-----------|-------------|
| `POST` | `hibernate` | Release memory while preserving the document store — `204` |
| `POST` | `wakeup` | Restore from hibernation — `204` |

---

## Teams

Access is governed by **team membership** — a user is an Admin, Editor, or Viewer of every dataset their team owns. Membership is managed from the account portal's team pages, not the API. The only cross-team API operation is moving a dataset:

| Method | Operation | Description |
|--------|-----------|-------------|
| `POST` | `transfer` | Move the dataset to another team (body: `{ "targetTeamName": "..." }`; caller must be Admin of both teams) |

---

## Account

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/change-password` | Change password |

> API tokens are created on the **API Key** page of the account portal (`/Account/ApiKey`), not via an API call.


---

# Filters

Filters are created server-side and referenced by a `hashString`. Pass that string back in search or bulk-operation requests.

## Value Filter

`POST /api/teams/{teamName}/datasets/{dataSetName}/filters/value`

Match documents where a field equals a specific value.

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/filters/value \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"fieldName": "category", "value": "electronics"}'
```

Response:

```json
{ "hashString": "abc123..." }
```

## Range Filter

`POST /api/teams/{teamName}/datasets/{dataSetName}/filters/range`

Match documents where a field falls within a numeric range.

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/filters/range \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"fieldName": "price", "lowerLimit": 10, "upperLimit": 100}'
```

Response:

```json
{ "hashString": "def456..." }
```

## Combining Filters

`POST /api/teams/{teamName}/datasets/{dataSetName}/filters/combine`

Combine two filters with AND or OR.

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/filters/combine \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "a": { "hashString": "abc123..." },
    "b": { "hashString": "def456..." },
    "useAndOperation": true
  }'
```

Response:

```json
{ "hashString": "combined789..." }
```

Set `useAndOperation` to `false` for OR.

## Using a Filter in Search

```json
{
  "text": "wireless",
  "maxNumberOfRecordsToReturn": 20,
  "filter": { "hashString": "combined789..." }
}
```

## Pre-loading Filters

`POST /api/teams/{teamName}/datasets/{dataSetName}/filters/load` — loads all filterable fields into memory for faster repeated filtering (`204`). Call once after indexing.

`GET /api/teams/{teamName}/datasets/{dataSetName}/filters/count` — returns the count of currently cached filters as `{"count": n}`.

## Releasing Filters

`POST /api/teams/{teamName}/datasets/{dataSetName}/filters/delete` — release a specific filter (body: `{ "hashString": "..." }`; `204`).

`DELETE /api/teams/{teamName}/datasets/{dataSetName}/filters` — release all cached filters (`204`).


---

# Schemas

Reference for all request and response shapes used by the HTTP API.

---

## CloudQuery

`POST /api/teams/{teamName}/datasets/{dataSetName}/search`

```json
{
  "text": "wireless headphones",
  "maxNumberOfRecordsToReturn": 20,
  "enableCoverage": true,
  "coverageDepth": 500,
  "enableFacets": false,
  "enableBoost": false,
  "removeDuplicates": true,
  "sortBy": null,
  "sortAscending": false,
  "timeOutLimitMilliseconds": 1000,
  "filter": null,
  "boosts": null,
  "fieldBoosts": { "title": 2.0, "body": 1.0 },
  "coverageSetup": null
}
```

`fieldBoosts` — per-query field weight overrides. Only applies in BM25F mode. `null` uses the field's configured `weight`.

---

## Result

Response from `POST /api/teams/{teamName}/datasets/{dataSetName}/search`

```json
{
  "records": [
    { "documentKey": 42, "score": 220 },
    { "documentKey": 17, "score": 185 }
  ],
  "facets": {
    "category": [
      { "key": "electronics", "value": 12 },
      { "key": "accessories", "value": 5 }
    ]
  },
  "truncationIndex": -1,
  "truncationScore": 0,
  "didTimeOut": false
}
```

`score` is a 16-bit unsigned integer (0–65535). `truncationIndex` is `-1` when no truncation occurred.

---

## VectorQueryProxy

`POST /api/teams/{teamName}/datasets/{dataSetName}/search/vector`

```json
{
  "fieldName": "embedding",
  "vector": [0.12, -0.45, 0.33],
  "maxResults": 10,
  "filter": null
}
```

The `vector` must be L2-normalised and match the dimension of the indexed embedding field.

---

## HybridQueryProxy

`POST /api/teams/{teamName}/datasets/{dataSetName}/search/hybrid`

```json
{
  "text": "comfortable wireless headphones",
  "embeddingField": "embedding",
  "vector": [0.12, -0.45, 0.33],
  "alpha": 0.6,
  "maxNumberOfRecordsToReturn": 20,
  "filter": null,
  "timeOutLimitMilliseconds": 1000
}
```

`alpha` — blend weight in `[0, 1]`. `0` = pure text, `1` = pure embedding.

---

## EmbeddingResultEntry

Response from `search/vector` and `search/hybrid`

```json
[
  { "documentKey": 42, "score": 0.94 },
  { "documentKey": 17, "score": 0.81 }
]
```

`score` is in `[0, 1]`. For vector search it is cosine similarity; for hybrid search it is the blended value.

---

## FieldProxy

Used by `PUT fields/configuration` (request) and `GET fields/configuration` (response). `null` on a PUT means "leave unchanged".

```json
[
  {
    "fieldName": "name",
    "fieldType": "String",
    "isArray": false,
    "searchable": true,
    "filterable": false,
    "facetable": false,
    "sortable": false,
    "wordIndexing": false,
    "embeddable": false,
    "preloadFilters": false,
    "highResolution": false,
    "weight": 2.0,
    "bM25b": 0.75,
    "bM25k1": 1.2,
    "highResolution": false
  }
]
```

`fieldType` and `isArray` are read-only — they are populated by the server on GET and ignored on PUT.

---

## SystemStatus

Response from `GET /api/teams/{teamName}/datasets/{dataSetName}/status`

```json
{
  "systemState": 4,
  "documentCount": 5000,
  "searchCounter": 142,
  "secondsToIndex": 3,
  "shadowBuildInProgress": false,
  "shadowBuildStartedUtc": null,
  "version": "5.0.0",
  "licenseInfo": { ... },
  "errorMessage": "",
  "timeOfInstanceCreation": "2025-01-01T00:00:00Z",
  "timeOfLastIndexBuild": "2025-01-01T00:05:00Z"
}
```

`systemState` values: `-1` Hibernated · `0` Created · `1` Loading · `2` Loaded · `3` Indexing · `4` Ready · `255` Error

---

## DataSetListDto

Response from `GET /api/me/datasets`

```json
[
  { "name": "products", "teamName": "acme", "role": "Admin" },
  { "name": "catalogue", "teamName": "research", "role": "Editor" }
]
```

`role` is `"Admin"`, `"Editor"`, or `"Viewer"` — your role on the owning team.

---

## TransferToTeamRequest

`POST /api/teams/{teamName}/datasets/{dataSetName}/transfer`

```json
{ "targetTeamName": "research" }
```

Moves the dataset to another team. The caller must be an Admin of both teams.

---

## FilterProxy

A filter reference returned by `filters/value`, `filters/range`, or `filters/combine`. Pass it back in search or bulk-operation requests.

```json
{ "hashString": "abc123..." }
```

---

## BoostProxy

`POST /api/teams/{teamName}/datasets/{dataSetName}/boosts/from-filter`

```json
{
  "filterProxy": { "hashString": "abc123..." },
  "boostStrength": 2
}
```

`boostStrength`: `1` = Low, `2` = Med, `3` = High.


---

# Search

Three search modes are available: full-text, vector (embedding-based), and hybrid (blended).

---

## Full-Text Search

`POST /api/teams/{teamName}/datasets/{dataSetName}/search`

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/search \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"text": "wireless headphones", "maxNumberOfRecordsToReturn": 20}'
```

### With Coverage

```json
{
  "text": "wireless headphones",
  "maxNumberOfRecordsToReturn": 30,
  "enableCoverage": true,
  "coverageDepth": 500
}
```

### With Facets and Filter

```json
{
  "text": "headphones",
  "maxNumberOfRecordsToReturn": 20,
  "enableFacets": true,
  "filter": {"hashString": "abc123..."}
}
```

### With Per-Field Boosts (BM25F mode only)

`fieldBoosts` overrides field weights at query time without rebuilding the index. Keys are field names, values are multipliers.

```json
{
  "text": "noise cancelling",
  "maxNumberOfRecordsToReturn": 10,
  "fieldBoosts": {
    "title": 3.0,
    "description": 1.0,
    "tags": 0.5
  }
}
```

### Browse / Empty Search

```json
{
  "text": "",
  "maxNumberOfRecordsToReturn": 50,
  "sortBy": "rating",
  "sortAscending": false,
  "enableFacets": true
}
```

---

## Vector Search

`POST /api/teams/{teamName}/datasets/{dataSetName}/search/vector`

Approximate nearest-neighbour search against a field marked `Embeddable`. The query vector must be L2-normalised and match the field's embedding dimensions.

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/search/vector \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "fieldName": "embedding",
    "vector": [0.12, -0.45, 0.33, ...],
    "maxResults": 10
  }'
```

| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `fieldName` | `string` | required | Name of the `Embeddable` field to search |
| `vector` | `float[]` | required | L2-normalised query vector |
| `maxResults` | `int` | `10` | Maximum number of results |
| `filter` | `FilterProxy?` | `null` | Optional filter to restrict the search space |

Returns an array of `EmbeddingResultEntry` with scores in `[0, 1]` (cosine similarity).

---

## Hybrid Search

`POST /api/teams/{teamName}/datasets/{dataSetName}/search/hybrid`

Blends text search and vector search scores. The combined score is:

```
score = alpha × embeddingScore + (1 − alpha) × normalisedTextScore
```

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/search/hybrid \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "comfortable headphones",
    "embeddingField": "embedding",
    "vector": [0.12, -0.45, 0.33, ...],
    "alpha": 0.6,
    "maxNumberOfRecordsToReturn": 20
  }'
```

| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `text` | `string` | `""` | Text query |
| `embeddingField` | `string` | required | Name of the `Embeddable` field |
| `vector` | `float[]` | required | L2-normalised query vector |
| `alpha` | `float` | `0.5` | Blend weight. `0` = pure text, `1` = pure embedding. Values around `0.5–0.7` work well |
| `maxNumberOfRecordsToReturn` | `int` | `10` | Maximum results |
| `filter` | `FilterProxy?` | `null` | Optional filter |
| `timeOutLimitMilliseconds` | `int` | `1000` | Text search timeout |

Returns an array of `EmbeddingResultEntry` with blended scores in `[0, 1]`.

---

## Retrieving Full Documents

Search endpoints return keys and scores only. Use `documents/lookup` to fetch the full JSON for a set of keys.

`POST /api/teams/{teamName}/datasets/{dataSetName}/documents/lookup`

```bash
curl -X POST https://your-host/api/teams/acme/datasets/products/documents/lookup \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '[42, 17, 305]'
```

Returns a `string[]` of JSON documents in the same order as the input keys.


## Search

POST /api/teams/{teamName}/datasets/{dataSetName}/analyze
  As Analyze but handles a stream as input text.
POST /api/teams/{teamName}/datasets/{dataSetName}/analyze/text
  Analyze the fields of a string containing json. Since json may be invalid, which will cause
a 400 error, it is sent as plain text.
POST /api/teams/{teamName}/datasets/{dataSetName}/filters/combine
  CombineFilters will combine two filters using AND or OR operation.
POST /api/teams/{teamName}/datasets/{dataSetName}/boosts/from-filter
  CreateBoost will create a Boost setup which may be passed to any search.
GET /api/teams/{teamName}/datasets/{dataSetName}/boosts
  Returns the dataset's persisted boost rules (server-side ranking rules applied when a
search sets enableBoost). Config, not state-gated — available even when not Ready.
PUT /api/teams/{teamName}/datasets/{dataSetName}/boosts
  Replaces the dataset's boost rules (whole list). Each rule needs a name, at least one
condition, and every condition must reference a Filterable field and be either a value
match or a numeric range (not both).
DELETE /api/teams/{teamName}/datasets/{dataSetName}/boosts
  Clears all boost rules for the dataset.
PUT /api/teams/{teamName}/datasets/{dataSetName}
  Creates the data set with the given configuration profile (201). Idempotent:
if the data set already exists it is left as-is — including its original
configuration profile, which is fixed at creation — and answers 200.
(Undefined profile values are rejected with a validation 400 by MVC's
enum model binding — pinned by ErrorContractTests.)
DELETE /api/teams/{teamName}/datasets/{dataSetName}
  DeleteDataSet, will delete the entire dataSet including all contained Documents.
Requires team Admin.
POST /api/teams/{teamName}/datasets/{dataSetName}/filters/range
  CreateRangeFilter will create a RangeFilter which may be passed to any search.
POST /api/teams/{teamName}/datasets/{dataSetName}/filters/value
  CreateValueFilter will create a ValueFilter which may be passed to any search.
DELETE /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}
  Deletes a document from the dataset by its key.
POST /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}
  Inserts one single Json record. The route key must match the document's key field
(the engine keys documents from the body, so a disagreeing route would otherwise
silently insert under a different key than the URL claims).
PUT /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}
  Updates one single Document. The route key must address an existing document and
match the document's key field (the engine keys documents from the body, so a
disagreeing route would otherwise silently update a different document than the
URL claims).
PATCH /api/teams/{teamName}/datasets/{dataSetName}/documents/{documentKey}
  Updates a single field on a document identified by its key.
DELETE /api/teams/{teamName}/datasets/{dataSetName}/documents
  Deletes documents from the dataset by their keys.
POST /api/teams/{teamName}/datasets/{dataSetName}/documents
  Inserts new JSON records into the dataset.
PUT /api/teams/{teamName}/datasets/{dataSetName}/documents
  Updates existing JSON records in the dataset.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields
  GetAllFields will return the fields found during analyze.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/facetable
  GetFacetableFields will return the array of facetable field names.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/facetable
  Sets the Facetable property on the specified fields.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/filterable
  GetFilterableFields will return the array of filterable field names.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/filterable
  Sets the Filterable property on the specified fields.
POST /api/teams/{teamName}/datasets/{dataSetName}/documents/lookup
  Returns the raw json records as string[] for the keys.
GET /api/teams/{teamName}/datasets/{dataSetName}/documents/count
  Returns the number of JSON records in the database for the given dataset.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/searchable
  GetSearchableFields will return the array of searchable field names.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/searchable
  Sets the Searchable property and weight on the specified fields.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/sortable
  GetSortableFields will return the array of sortable field names.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/sortable
  Sets the Sortable property on the specified fields.
GET /api/teams/{teamName}/datasets/{dataSetName}/status
  GetStatus will return the status of the dataSetName in the search engine.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/word-indexing
  GetWordIndexingFields will return the array of word-indexing field names.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/word-indexing
  Sets the WordIndexing property on the specified fields.
GET /api/me/datasets
  Returns all datasets the current user can reach via team membership, across every team
they belong to. Each entry carries the owning team name and the caller's role on it.
GET /api/teams/{teamName}/datasets
  Lists the datasets owned by a single team.
POST /api/teams/{teamName}/datasets/{dataSetName}/index
  IndexDataSet will start indexing of the loaded documents.
POST /api/teams/{teamName}/datasets/{dataSetName}/load/from-database
  Loads the jsonData into search engine from the database.
POST /api/teams/{teamName}/datasets/{dataSetName}/load
  Loads the jsonData into search engine as a stream.
POST /api/teams/{teamName}/datasets/{dataSetName}/replace
  Atomically replaces the entire dataset's documents with the streamed JSON. Unlike
delete+recreate this preserves the dataset's identity, field configuration, boost rules
and description, serves the old data with zero downtime until the new index is ready, and
leaves the old dataset untouched if the new JSON fails to build. Works whether the dataset
is Ready, hibernated or idle-evicted (the old documents are never reloaded). Returns a
summary of how the new schema differed from the previous field configuration.
POST /api/teams/{teamName}/datasets/{dataSetName}/load/text
  Loads the jsonData into search engine as a string.
POST /api/teams/{teamName}/datasets/{dataSetName}/search
  Search will validate the search query and return the search result.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/configuration
  SetFieldConfiguration sets any combination of field properties (Searchable, Filterable,
Facetable, Sortable, WordIndexing, Embeddable, PreloadFilters, Weight, BM25b, BM25k1)
in one call. Nullable properties have replace semantics: null = leave untouched,
any value (including false) = overwrite.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/configuration
  GetFieldConfiguration returns the full configuration of every field in the dataset.
GET /api/teams/{teamName}/datasets/{dataSetName}/fields/key
  Returns the dataset's declared key field — the JSON field whose value identifies each
document (the primary key). Empty string means none is declared (the engine auto-generates
keys). Required, when set, to be a numeric field.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/key
  Declares the dataset's key field (the JSON field whose value is each document's primary key).
Pass an empty string to clear it (auto-generated keys). The field must be numeric. The choice
is preserved across reloads and applied on the next Load/replace; documents already loaded
keep their existing keys until the data is reloaded.
POST /api/teams/{teamName}/datasets/{dataSetName}/documents/delete-by-filter
  Deletes all documents matching the given filter.
POST /api/teams/{teamName}/datasets/{dataSetName}/documents/update-by-filter
  Updates a field on all documents matching the given filter. Returns the number of updated documents.
POST /api/teams/{teamName}/datasets/{dataSetName}/filters/delete
  Deletes a single filter from the filter cache.
DELETE /api/teams/{teamName}/datasets/{dataSetName}/filters
  Deletes all filters from the filter cache.
POST /api/teams/{teamName}/datasets/{dataSetName}/filters/load
  Pre-loads all registered filters in the background.
GET /api/teams/{teamName}/datasets/{dataSetName}/filters/count
  Returns the number of filters currently registered in the filter cache.
POST /api/teams/{teamName}/datasets/{dataSetName}/hibernate
  Hibernates the dataset, freeing in-memory structures while retaining persisted data.
POST /api/teams/{teamName}/datasets/{dataSetName}/wakeup
  Wakes up a hibernated dataset, restoring it from the persisted state.
PUT /api/teams/{teamName}/datasets/{dataSetName}/fields/embeddable
  Marks the specified fields as embeddable so that their vector values are indexed
during the next Load. Must be called after AnalyzeStream and before LoadStream.
POST /api/teams/{teamName}/datasets/{dataSetName}/search/vector
  Searches a single embedding field using approximate nearest-neighbour search.
POST /api/teams/{teamName}/datasets/{dataSetName}/search/hybrid
  Combines text search with embedding nearest-neighbour search and blends scores.

## Schemas

### BoostCondition
One condition of a boost rule: a filterable field matched either by an exact
IndxCloudApi.Models.BoostCondition.Value (string/keyword fields) or by a numeric IndxCloudApi.Models.BoostCondition.Min/IndxCloudApi.Models.BoostCondition.Max
range. Exactly one of the two forms is used: if IndxCloudApi.Models.BoostCondition.Min or IndxCloudApi.Models.BoostCondition.Max is set
it's a range, otherwise IndxCloudApi.Models.BoostCondition.Value is an exact match.
  field: string
  value: string
  min: double
  max: double
  isRange: boolean

### BoostProxy
  boostStrength: int32
  filterProxy: object

### BoostRule
A persisted, per-dataset ranking rule: when a search has `enableBoost` on, documents
matching this rule's combined condition get their score lifted by IndxCloudApi.Models.BoostRule.Strength.
Rules are managed in the UI and applied server-side; see `BoostRuleStore`.
  name: string
  enabled: boolean
  conditions: object[]
  join: int32
  strength: int32
  activeFrom: date
  activeUntil: date

### ChangePasswordRequest
Body of POST /api/changePassword. Used to rotate the seeded admin's
initial password (set at deployment time) before any other API calls
can be made.
  currentPassword: string
  newPassword: string

### CloudQuery
  boosts: object[] (default: null)
  coverageDepth: int32 (default: 500)
  coverageSetup: object
  enableBoost: boolean (default: false)
  enableCoverage: boolean (default: true)
  enableFacets: boolean (default: false)
  fieldBoosts: object (default: null)
  filter: object
  logPrefix: string (default: null)
  maxNumberOfRecordsToReturn: int32 (default: 30)
  removeDuplicates: boolean (default: true)
  sortAscending: boolean (default: false)
  sortBy: string (default: null)
  text: string
  timeOutLimitMilliseconds: int32 (default: 1000)

### CombinedFilterProxy
  a: object
  b: object
  useAndOperation: boolean

### CoverageSetup
  includePatternMatches: boolean
  levenshteinMaxWordSize: int32 (default: 20)
  minWordSize: int32 (default: 2)
  truncateWordHitLimit: int32 (default: 1)
  truncateWordHitTolerance: int32 (default: 0)
  coverWholeQuery: boolean (default: true)
  coverWholeWords: boolean (default: true)
  coverFuzzyWords: boolean (default: true)
  coverJoinedWords: boolean (default: true)
  coverPrefixSuffix: boolean (default: true)
  truncate: boolean (default: true)
  truncationScore: int32 (default: 65024)

### FieldProxy
  fieldName: string
  fieldType: string
  isArray: boolean
  searchable: boolean
  filterable: boolean
  facetable: boolean
  sortable: boolean
  wordIndexing: boolean
  embeddable: boolean
  preloadFilters: boolean
  weight: float
  bM25b: float
  bM25k1: float
  highResolution: boolean

### FilterFieldUpdateProxy
  filter: object
  fieldName: string
  value: any

### FilterProxy
  hashString: string

### HybridQueryProxy
  text: string
  maxNumberOfRecordsToReturn: int32 (default: 10)
  filter: object
  timeOutLimitMilliseconds: int32 (default: 1000)
  embeddingField: string
  vector: float[]
  alpha: float (default: 0.5)

### LicenseInfo
  description: string
  documentLimit: int32
  documentLimitExceeded: boolean
  expirationDate: date-time
  licenseFileFound: boolean
  licensed: boolean
  licensedTo: string
  licenseFileName: string
  type: string
  validLicense: boolean

### ParseResult
  token: string
  progressPercent: int32
  recordIndex: int32
  isEndOfStream: boolean
  errorType: int32
  errorMessage: string
  lineNumber: int64
  bytePositionInLine: int64
  jsonPath: string
  wasSanitized: boolean
  originalToken: string

### ProblemDetails
  type: string
  title: string
  status: int32
  detail: string
  instance: string

### ProcessError
  source: string
  message: string
  parseError: object
  timeStampUtc: date-time

### ProcessErrorCount
  error: object
  count: int32

### RangeFilterProxy
  fieldName: string
  lowerLimit: double
  upperLimit: double

### StringSingleValueTuple

### SystemStatus
  documentCount: int32
  errorMessage: string
  invalidArgument: boolean
  invalidDataSetName: boolean
  invalidState: boolean
  licenseInfo: object
  recoverableErrors: object[]
  searchCounter: int32
  secondsToIndex: double
  systemState: int32
  timeOfInstanceCreation: date-time
  timeOfLastIndexBuild: date-time
  tooLongClientText: boolean
  tooLongSearchText: boolean
  unrecoverableErrors: object[]
  unknownConfigurationError: boolean
  version: string

### UpdateFieldProxy
  fieldName: string
  value: any

### ValueFilterProxy
  fieldName: string
  value: any

### VectorQueryProxy
  fieldName: string
  vector: float[]
  maxResults: int32 (default: 10)
  filter: object