Schema builder¶

Build the schema argument for creating an index.

A schema names the fields an index searches and says how each one is scored. SchemaBuilder assembles that structure a field at a time and hands back a plain {"fields": {...}} dict rather than a model, so a key this SDK version does not yet model still reaches the server unchanged.

The builder helps you structure a schema and discover what each field type takes; it is not a full validator. It catches a few mistakes locally, passes dict-valued options such as embed and sparse_embed through as given, and leaves the rest to the server, which is the source of truth: a schema it rejects makes the create call raise ApiError carrying the server’s message.

What belongs in a create schema is narrower than what an index stores. Declare the fields you search (a dense_vector field, a sparse_vector field, and string fields with full-text search enabled or with integrated embedding through embed / sparse_embed); at least one is required. Everything else is metadata: put those values in the documents you upsert and they are indexed for filtering automatically, with no declaration at all. An index has one dense slot and one sparse slot: at most one field may be a dense_vector field or carry embed, and at most one may be a sparse_vector field or carry sparse_embed.

The builder still has methods for the metadata-only declarations (SchemaBuilder.add_boolean_field(), SchemaBuilder.add_float_field(), SchemaBuilder.add_string_list_field(), and SchemaBuilder.add_string_field() with no search key). The server rejects all of them on create, and one rejected field fails the whole request.

Two field types have no method at all. For integrated embedding, pass embed or sparse_embed to SchemaBuilder.add_string_field(); the semantic_text field type, which is how a legacy integrated index (created via create_for_model()) reports its embedded field, is not accepted in a create schema. integer is response-only: describe and list return it for indexes that pre-date numeric normalisation, but a create schema has no integer type. The builder refuses it wherever a raw type key could reach a field dict (SchemaBuilder.add_custom_field() and the additional_options parameters), so replaying a described schema into a create request fails locally with an explanation instead of with a bare server status code.

See also

create(): the schema argument this builds, and the deployment argument that goes with it.

__init__()[source]¶

Return type:: None

add_dense_vector_field(name, *, dimension, metric, description=None, **additional_options)[source]¶

Add the field that holds an embedding, for vector similarity search.

This is the field most indexes are built around, and a schema may hold at most one of them. The builder does not count them: a second dense vector field is built as written and the server rejects it on create with a 400. dimension and metric are fixed for the life of the field, so they have to match the embedding model you intend to use.

Parameters:

name (str) – Field name; the documents you upsert carry their vector under this key, e.g. "your_vector". Replaces any existing field with the same name.
dimension (int) – Length of the vectors this field stores: the output width of your embedding model, e.g. 1536. The SDK rejects a dimension outside the supported range before the request goes out.
metric (str) – How similarity is scored: "cosine", "euclidean", or "dotproduct". "cosine" is what most text embedding models are trained for. Capitals are lowercased before the request is sent. See Metric.
description (str | None) – Human-readable note stored with the field. Optional.
**additional_options (Any) – Extra parameters merged into the field dict last, for forward compatibility with new API features.

Returns:

self for method chaining.

Raises:

PineconeValueError – If dimension is outside the supported range, or if name or description is too long. Both lengths count UTF-8 bytes rather than characters, so non-ASCII text runs out sooner than its length suggests.

Return type:

SchemaBuilder

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> schema = SchemaBuilder().add_dense_vector_field(
...     "your_vector", dimension=1536, metric="cosine"
... ).build()
>>> schema["fields"]["your_vector"]
{'type': 'dense_vector', 'dimension': 1536, 'metric': 'cosine'}

add_sparse_vector_field(name, *, description=None, **additional_options)[source]¶

Add a sparse vector field, for keyword-weighted or learned-sparse search.

Declare one alongside a dense vector field for hybrid search, or on its own for pure sparse retrieval. A schema may hold at most one.

description is the only other key a create schema accepts here. A sparse vector field takes no metric, because sparse scoring is not configurable, and no dimension, because sparse vectors are variable-length. Passing either raises rather than putting a key on the wire that configures nothing.

Parameters:

name (str) – Field name, e.g. "keyword_terms". Replaces any existing field with the same name.
description (str | None) – Human-readable note stored with the field. Optional.
**additional_options (Any) – Extra parameters merged into the field dict last, for forward compatibility with new API features.

Returns:

self for method chaining.

Raises:

PineconeValueError – If metric or dimension is passed, or if name or description is too long.

Return type:

SchemaBuilder

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> schema = (
...     SchemaBuilder()
...     .add_dense_vector_field(
...         "your_vector", dimension=1536, metric="dotproduct"
...     )
...     .add_sparse_vector_field("keyword_terms")
...     .build()
... )
>>> schema["fields"]["keyword_terms"]
{'type': 'sparse_vector'}

add_string_field(name, *, full_text_search=None, language=None, stemming=None, stop_words=None, embed=None, sparse_embed=None, store_text=None, filterable=False, description=None, **additional_options)[source]¶

Add a string field searched by full-text search, integrated embedding, or both.

Always pass at least one of full_text_search, embed and sparse_embed: they are what make the field searched, and a string field with none of them is a metadata-only declaration the server refuses on create. Any of full_text_search=True, a full_text_search dict, or one of the typed keyword arguments (language, stemming, stop_words) turns full-text search on; where a dict and a keyword argument set the same key, the keyword argument wins.

embed and sparse_embed enable integrated embedding for the field: Pinecone embeds the text into a dense or learned-sparse vector with a hosted model. Search the field with EmbedQuery or SparseEmbedQuery. The text is stored by default, and stored text is always full-text searchable, so embed alone also turns on full-text search with default settings. store_text=False embeds the text without storing it; see store_text below for what that gives up.

Describe responses also show lowercase and max_term_len; the server sets those and rejects them in a create schema.

Parameters:

name (str) – Field name; the documents you upsert carry the text under this key, e.g. "title". Replaces any existing field with the same name.
full_text_search (bool | dict[str, Any] | None) – True or {} for full-text search with default analysis, or a dict of the config keys (language, stemming, stop_words, ngram). None, the default, leaves the field unsearched unless embed or sparse_embed is set; see the note below.
language (str | None) – Language whose rules drive tokenisation and analysis. Both short codes and their English names work: "en" and "english" are the same request, and the SDK sends the short form. A name the SDK does not recognise passes through unchanged, so a newly supported language works without an SDK upgrade; the server decides what it accepts.
stemming (bool | None) – Match words by their root, so a search for "running" also finds "run". Required when stop_words=True.
stop_words (bool | None) – Drop the language’s most common words from the index. Requires stemming=True. Not every language supports stop words; the server rejects an unsupported pairing.
embed (bool | Mapping[str, Any] | FieldEmbedConfig | None) – True or {} to embed the text with the default dense model at its default dimension and metric, or a dict or FieldEmbedConfig of model, dimension, metric (lowercased, as for a dense vector field), processing_location, write_parameters and read_parameters. A dict is sent as given, so a misspelled key fails only on create; the typed config rejects it when constructed. The field takes the index’s one dense slot. Fixed at creation.
sparse_embed (bool | Mapping[str, Any] | FieldSparseEmbedConfig | None) – True or {} to embed the text with the default sparse model, or a dict or FieldSparseEmbedConfig of model, write_parameters and read_parameters. A dict is sent as given, as for embed. The field takes the index’s one sparse slot. Fixed at creation.
store_text (bool | None) – Only meaningful as False, on a field with embed or sparse_embed. None, the default, leaves the key off and the server stores the text. False embeds the text without storing it, which can save space when the field is only used for similarity search, or keep sensitive text out of Pinecone. The field then has no full-text search: it can be searched only with an embed clause, and a fetch or search whose include_fields names it is rejected with a 400. The server rejects False on a field without embed or sparse_embed, and together with full_text_search, each with a 400. If you’re not sure whether you’ll need full-text search, keep the text stored, for more search flexibility later. Fixed at creation.
filterable (bool) – Make the field filterable instead of searched. A string field is one or the other: the server rejects filterable=True alongside full_text_search, embed or sparse_embed with a 400. Sent on the wire including its False default, except when full_text_search, embed, sparse_embed or store_text is set and you did not ask to be filterable.
description (str | None) – Human-readable note stored with the field. Optional.
**additional_options (Any) – Extra parameters merged into the field dict last, for forward compatibility with new API features.

Returns:

self for method chaining.

Raises:

PineconeValueError – If stop_words=True is requested without stemming=True, if an ngram config is combined with stemming=True or stop_words=True, or if name or description exceeds its byte limit.

Return type:

SchemaBuilder

Examples

Default analysis is enough for most text:

>>> from pinecone.schema_builder import SchemaBuilder
>>> schema = SchemaBuilder().add_string_field(
...     "title", full_text_search=True
... ).build()
>>> schema["fields"]["title"]
{'type': 'string', 'full_text_search': {}}

Naming the language turns on that language’s analysis, and the long form is accepted and normalised:

>>> schema = SchemaBuilder().add_string_field(
...     "title", language="english", stemming=True, stop_words=True
... ).build()
>>> schema["fields"]["title"]["full_text_search"]
{'language': 'en', 'stemming': True, 'stop_words': True}

Character n-grams match substrings rather than whole words, which is what autocomplete needs. They cannot be combined with stemming or stop words:

>>> schema = SchemaBuilder().add_string_field(
...     "title", full_text_search={"ngram": {"min_gram": 2, "max_gram": 3}}
... ).build()
>>> schema["fields"]["title"]["full_text_search"]
{'ngram': {'min_gram': 2, 'max_gram': 3}}

Pinecone embeds the text of an embed field for you, here with a chosen model and dimension, and a sparse aspect can sit on the same field:

>>> schema = SchemaBuilder().add_string_field(
...     "body",
...     embed={"model": "llama-text-embed-v2", "dimension": 512},
...     sparse_embed=True,
... ).build()
>>> schema["fields"]["body"]["embed"]
{'model': 'llama-text-embed-v2', 'dimension': 512}
>>> schema["fields"]["body"]["sparse_embed"]
{}

store_text=False embeds the text without keeping it:

>>> schema = SchemaBuilder().add_string_field(
...     "body", embed=True, store_text=False
... ).build()
>>> schema["fields"]["body"]
{'type': 'string', 'embed': {}, 'store_text': False}

Note

A string field with none of full_text_search, embed and sparse_embed is a metadata-only declaration. The server rejects those on create with a 400 saying the schema only accepts fields used for search, failing the whole request over the one field. Leave such fields out of the schema and put the values in the documents you upsert; they are indexed for filtering automatically.

add_string_list_field(name, *, filterable=False, description=None, **additional_options)[source]¶

Declare a list-of-strings field, which the server rejects on create.

A string-list field holds several strings per record, the shape tag-style metadata takes (["sci-fi", "mystery"]) when you want to filter on individual elements. Declaring it is what does not work: see the note below, and upsert the list as an ordinary document field instead.

Parameters:

name (str) – Field name, e.g. "tags". Replaces any existing field with the same name.
filterable (bool) – Enable metadata filtering on the field. Always included in the built schema, whether True or False.
description (str | None) – Human-readable note stored with the field. Optional.
**additional_options (Any) – Extra parameters merged into the field dict last, for forward compatibility with new API features.

Returns:

self for method chaining.

Raises:

PineconeValueError – If name or description exceeds its byte limit.

Return type:

SchemaBuilder

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> schema = SchemaBuilder().add_string_list_field(
...     "tags", filterable=True
... ).build()
>>> schema["fields"]["tags"]
{'type': 'string_list', 'filterable': True}

Note

string_list is a metadata-only declaration. The server rejects those on create with a 400 saying the schema only accepts fields used for search, whatever filterable says, failing the whole schema over the one field. Leave the field out and put the values in the documents you upsert; they are indexed for filtering automatically.

add_boolean_field(name, *, filterable=False, description=None, **additional_options)[source]¶

Declare a boolean field, which the server rejects on create.

Declaring the field is what does not work; a boolean is filterable once it is in a document. See the note below, and upsert the flag as an ordinary document field instead.

Parameters:

name (str) – Field name, e.g. "is_published". Replaces any existing field with the same name.
filterable (bool) – Enable metadata filtering on the field. Always included in the built schema, whether True or False.
description (str | None) – Human-readable note stored with the field. Optional.
**additional_options (Any) – Extra parameters merged into the field dict last, for forward compatibility with new API features.

Returns:

self for method chaining.

Raises:

PineconeValueError – If name or description exceeds its byte limit.

Return type:

SchemaBuilder

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> schema = SchemaBuilder().add_boolean_field(
...     "is_published", filterable=True
... ).build()
>>> schema["fields"]["is_published"]
{'type': 'boolean', 'filterable': True}

Note

boolean is a metadata-only declaration. The server rejects those on create with a 400 saying the schema only accepts fields used for search, whatever filterable says, failing the whole schema over the one field. Leave the field out and put the values in the documents you upsert; they are indexed for filtering automatically.

add_float_field(name, *, filterable=False, description=None, **additional_options)[source]¶

Declare a numeric field, which the server rejects on create.

This is the only numeric declaration there is: a create schema has no integer type, and whole numbers are stored and filtered as double-precision floats. Declaring the field is what does not work; numbers are filterable once they are in a document. See the note below, and upsert the value as an ordinary document field instead.

Describe and list responses can still return integer for indexes that pre-date that normalisation; see IntegerField.

Parameters:

name (str) – Field name, e.g. "release_year". Replaces any existing field with the same name.
filterable (bool) – Enable metadata filtering on the field. Always included in the built schema, whether True or False.
description (str | None) – Human-readable note stored with the field. Optional.
**additional_options (Any) – Extra parameters merged into the field dict last, for forward compatibility with new API features.

Returns:

self for method chaining.

Raises:

PineconeValueError – If name or description exceeds its byte limit.

Return type:

SchemaBuilder

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> schema = SchemaBuilder().add_float_field(
...     "release_year", filterable=True
... ).build()
>>> schema["fields"]["release_year"]
{'type': 'float', 'filterable': True}

Note

float is a metadata-only declaration. The server rejects those on create with a 400 saying the schema only accepts fields used for search, whatever filterable says, failing the whole schema over the one field. Leave the field out and put the values in the documents you upsert; they are indexed for filtering automatically.

add_custom_field(name, field_definition)[source]¶

Store a raw field dict verbatim, as an escape hatch.

Two jobs: copying a field definition out of a describe response into a new index’s schema, and declaring a field type a newer API version offers that this SDK does not model yet. The name is checked and the definition’s type is rejected if it is response-only; the rest of the definition goes through untouched, so anything else wrong with it surfaces as a server error on create rather than here. The one exception is a metric in capitals, which is lowercased when the create request is sent, not in the schema this builder returns.

Parameters:

name (str) – Field name, e.g. "your_vector". Replaces any existing field with the same name.
field_definition (dict[str, Any]) – The complete field definition, stored as-is and deep-copied into the built schema.

Returns:

self for method chaining.

Raises:

PineconeValueError – If the field name is empty or too long, or if field_definition["type"] is response-only, which today means "integer".

Return type:

SchemaBuilder

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> described = {"type": "dense_vector", "dimension": 1536,
...              "metric": "cosine"}
>>> schema = SchemaBuilder().add_custom_field(
...     "your_vector", described
... ).build()
>>> schema["fields"]["your_vector"]
{'type': 'dense_vector', 'dimension': 1536, 'metric': 'cosine'}

Note

{"type": "integer"} is refused here rather than on the wire. integer comes back from describe and list for indexes created before numeric values were normalised to float, but a create schema has no integer type. When replaying a described schema into a create request, drop those fields. add_float_field() is no help, because a float declaration is rejected on create too, and numeric metadata needs no declaration.

build()[source]¶

Return the completed schema dict.

The result is a deep copy of the builder’s state, so writing into it (adding a forward-compatible key the SDK does not yet model, for instance) leaves the builder and every other result of build() untouched. One builder can be reused across several indexes even when each schema is edited after the fact.

Nothing here checks that the schema is complete or that the server will accept it. The server requires at least one searched field (dense_vector, sparse_vector, or string with full_text_search, embed or sparse_embed), and a schema without one is built and returned all the same, so that partial schemas can be inspected; the create call is where it fails.

Returns:: {"fields": {name: field_dict, ...}} ready to pass as the schema argument when creating an index.
Return type:: dict[str, dict[str, Any]]

Examples

>>> from pinecone.schema_builder import SchemaBuilder
>>> builder = SchemaBuilder().add_dense_vector_field(
...     "your_vector", dimension=8, metric="cosine"
... )
>>> schema = builder.build()
>>> schema["fields"]["your_vector"]["future_option"] = True
>>> builder.build()["fields"]["your_vector"]
{'type': 'dense_vector', 'dimension': 8, 'metric': 'cosine'}