Pinecone¶

class pinecone.Pinecone(api_key: str | None = None, host: str | None = None, proxy_url: str | None = None, proxy_headers: Dict[str, str] | None = None, ssl_ca_certs: str | None = None, ssl_verify: bool | None = None, additional_headers: Dict[str, str] | None = {}, pool_threads: int | None = None, **kwargs)[source]¶: A client for interacting with Pinecone APIs.

Pinecone.__init__(api_key: str | None = None, host: str | None = None, proxy_url: str | None = None, proxy_headers: Dict[str, str] | None = None, ssl_ca_certs: str | None = None, ssl_verify: bool | None = None, additional_headers: Dict[str, str] | None = {}, pool_threads: int | None = None, **kwargs)[source]¶

The Pinecone class is the main entry point for interacting with Pinecone via this Python SDK. Instances of the Pinecone class are used to manage and interact with Pinecone resources such as indexes, backups, and collections. When using the SDK, calls are made on your behalf to the API documented at https://docs.pinecone.io.

The class also holds inference functionality (embed, rerank) under the inference namespace.

When you are ready to perform data operations on an index, you will need to instantiate an index client. Though the functionality of the index client is defined in a different class, it is instantiated through the Index() method in order for configurations to be shared between the two objects.

Parameters:

api_key (str, optional) – The API key to use for authentication. If not passed via kwarg, the API key will be read from the environment variable PINECONE_API_KEY.
host (str, optional) – The control plane host. If unspecified, the host api.pinecone.io will be used.
proxy_url (str, optional) – The URL of the proxy to use for the connection.
proxy_headers (Dict[str, str], optional) – Additional headers to pass to the proxy. Use this if your proxy setup requires authentication.
ssl_ca_certs (str, optional) – The path to the SSL CA certificate bundle to use for the connection. This path should point to a file in PEM format. When not passed, the SDK will use the certificate bundle returned from certifi.where().
ssl_verify (bool, optional) – SSL verification is performed by default, but can be disabled using the boolean flag when testing with Pinecone Local or troubleshooting a proxy setup. You should never run with SSL verification disabled in production.
additional_headers (Dict[str, str], optional) – Additional headers to pass to the API. This is mainly to support internal testing at Pinecone. End users should not need to use this unless following specific instructions to do so.
pool_threads (int, optional) – The number of threads to use for the ThreadPool when using methods that support the async_req keyword argument. The default number of threads is 5 * the number of CPUs in your execution environment.

Configuration with environment variables

If you instantiate the Pinecone client with no arguments, it will attempt to read the API key from the environment variable PINECONE_API_KEY.

from pinecone import Pinecone

pc = Pinecone()

Configuration with keyword arguments

If you prefer being more explicit in your code, you can also pass the API key as a keyword argument. This is also where you will pass additional configuration options such as proxy settings if you wish to use those.

import os
from pinecone import Pinecone

pc = Pinecone(
    api_key=os.environ.get("PINECONE_API_KEY"),
    host="https://api-staging.pinecone.io"
)

Environment variables

The Pinecone client supports the following environment variables:

PINECONE_API_KEY: The API key to use for authentication. If not passed via kwarg, the API key will be read from the environment variable PINECONE_API_KEY.
PINECONE_DEBUG_CURL: Enable some additional debug logging representing the HTTP requests as curl commands. The main use of is to run calls outside of the SDK to help evaluate whether a problem you are experiencing is due to the API’s behavior or the behavior of the SDK itself.
PINECONE_ADDITIONAL_HEADERS: A json string of a dictionary of header values to attach to all requests. This is primarily used for internal testing at Pinecone.

Warning

Be very careful with the PINECONE_DEBUG_CURL environment variable, as it will print out your API key which forms part of a required authentication header.

Proxy configuration

If your network setup requires you to interact with Pinecone via a proxy, you will need to pass additional configuration using optional keyword parameters. These optional parameters are forwarded to urllib3, which is the underlying library currently used by the Pinecone client to make HTTP requests. You may find it helpful to refer to the urllib3 documentation on working with proxies while troubleshooting these settings.

Here is a basic example:

from pinecone import Pinecone

pc = Pinecone(
    api_key='YOUR_API_KEY',
    proxy_url='https://your-proxy.com'
)

pc.list_indexes()

If your proxy requires authentication, you can pass those values in a header dictionary using the proxy_headers parameter.

from pinecone import Pinecone
import urllib3 import make_headers

pc = Pinecone(
    api_key='YOUR_API_KEY',
    proxy_url='https://your-proxy.com',
    proxy_headers=make_headers(proxy_basic_auth='username:password')
)

pc.list_indexes()

Using proxies with self-signed certificates

By default the Pinecone Python client will perform SSL certificate verification using the CA bundle maintained by Mozilla in the certifi package. If your proxy server is using a self-signed certificate, you will need to pass the path to the certificate in PEM format using the ssl_ca_certs parameter.

from pinecone import Pinecone
import urllib3 import make_headers

pc = Pinecone(
    api_key='YOUR_API_KEY',
    proxy_url='https://your-proxy.com',
    proxy_headers=make_headers(proxy_basic_auth='username:password'),
    ssl_ca_certs='path/to/cert-bundle.pem'
)

pc.list_indexes()

Disabling SSL verification

If you would like to disable SSL verification, you can pass the ssl_verify parameter with a value of False. We do not recommend going to production with SSL verification disabled.

from pinecone import Pinecone
import urllib3 import make_headers

pc = Pinecone(
    api_key='YOUR_API_KEY',
    proxy_url='https://your-proxy.com',
    proxy_headers=make_headers(proxy_basic_auth='username:password'),
    ssl_ca_certs='path/to/cert-bundle.pem',
    ssl_verify=False
)

pc.list_indexes()

Pinecone.Index(name: str = '', host: str = '', **kwargs) → Index[source]¶

Parameters:

name (str, optional) – The name of the index to target. If you specify the name of the index, the client will fetch the host url from the Pinecone control plane.
host (str, optional) – The host url of the index to target. If you specify the host url, the client will use the host url directly without making any additional calls to the control plane.
pool_threads (int, optional) – The number of threads to use when making parallel requests by calling index methods with optional kwarg async_req=True, or using methods that make use of thread-based parallelism automatically such as query_namespaces().
connection_pool_maxsize (int, optional) – The maximum number of connections to keep in the connection pool.

Returns:

An instance of the Index class.

Target an index for data operations.

Target an index by host url

In production situations, you want to uspert or query your data as quickly as possible. If you know in advance the host url of your index, you can eliminate a round trip to the Pinecone control plane by specifying the host of the index. If instead you pass the name of the index, the client will need to make an additional call to api.pinecone.io to get the host url before any data operations can take place.

import os
from pinecone import Pinecone

api_key = os.environ.get("PINECONE_API_KEY")
index_host = os.environ.get("PINECONE_INDEX_HOST")

pc = Pinecone(api_key=api_key)
index = pc.Index(host=index_host)

# Now you're ready to perform data operations
index.query(vector=[...], top_k=10)

To find your host url, you can use the describe_index method to call api.pinecone.io. The host url is returned in the response. Or, alternatively, the host is displayed in the Pinecone web console.

import os
from pinecone import Pinecone

pc = Pinecone(
    api_key=os.environ.get("PINECONE_API_KEY")
)

host = pc.describe_index('index-name').host

Target an index by name (not recommended for production)

For more casual usage, such as when you are playing and exploring with Pinecone in a notebook setting, you can also target an index by name. If you use this approach, the client may need to perform an extra call to the Pinecone control plane to get the host url on your behalf to get the index host.

The client will cache the index host for future use whenever it is seen, so you will only incur the overhead of only one call. But this approach is not recommended for production usage because it introduces an unnecessary runtime dependency on api.pinecone.io.

import os
from pinecone import Pinecone, ServerlessSpec

api_key = os.environ.get("PINECONE_API_KEY")

pc = Pinecone(api_key=api_key)
pc.create_index(
    name='my_index',
    dimension=1536,
    metric='cosine',
    spec=ServerlessSpec(cloud='aws', region='us-west-2')
)
index = pc.Index('my_index')

# Now you're ready to perform data operations
index.query(vector=[...], top_k=10)

Pinecone.IndexAsyncio(host: str, **kwargs) → IndexAsyncio[source]¶

Build an asyncio-compatible Index object.

Parameters:: host (str, required) – The host url of the index to target. You can find this url in the Pinecone web console or by calling describe_index method of Pinecone or PineconeAsyncio.
Returns:: An instance of the IndexAsyncio class.

import asyncio
import os
from pinecone import Pinecone

async def main():
    pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))
    async with pc.IndexAsyncio(host=os.environ.get("PINECONE_INDEX_HOST")) as index:
        await index.query(vector=[...], top_k=10)

asyncio.run(main())

See more docs for PineconeAsyncio here.

DB Control Plane¶

Indexes¶

Creates a Pinecone index.

Parameters:

name (str) – The name of the index to create. Must be unique within your project and cannot be changed once created. Allowed characters are lowercase letters, numbers, and hyphens and the name may not begin or end with hyphens. Maximum length is 45 characters.
metric (str, optional) – Type of similarity metric used in the vector index when querying, one of {"cosine", "dotproduct", "euclidean"}.
spec (Dict) – A dictionary containing configurations describing how the index should be deployed. For serverless indexes, specify region and cloud. For pod indexes, specify replicas, shards, pods, pod_type, metadata_config, and source_collection. Alternatively, use the ServerlessSpec, PodSpec, or ByocSpec objects to specify these configurations.
dimension (int) – If you are creating an index with vector_type="dense" (which is the default), you need to specify dimension to indicate the size of your vectors. This should match the dimension of the embeddings you will be inserting. For example, if you are using OpenAI’s CLIP model, you should use dimension=1536. Dimension is a required field when creating an index with vector_type="dense" and should not be passed when vector_type="sparse".
timeout (int, optional) – Specify the number of seconds to wait until index gets ready. If None, wait indefinitely; if >=0, time out after this many seconds; if -1, return immediately and do not wait.
deletion_protection (Optional[Literal["enabled", "disabled"]]) – If enabled, the index cannot be deleted. If disabled, the index can be deleted.
vector_type (str, optional) – The type of vectors to be stored in the index. One of {"dense", "sparse"}.
tags (Optional[Dict[str, str]]) – Tags are key-value pairs you can attach to indexes to better understand, organize, and identify your resources. Some example use cases include tagging indexes with the name of the model that generated the embeddings, the date the index was created, or the purpose of the index.

Returns:

A IndexModel instance containing a description of the index that was created.

Examples:

Creating a serverless index¶

import os
from pinecone import (
    Pinecone,
    ServerlessSpec,
    CloudProvider,
    AwsRegion,
    Metric,
    DeletionProtection,
    VectorType
)

pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))

pc.create_index(
    name="my_index",
    dimension=1536,
    metric=Metric.COSINE,
    spec=ServerlessSpec(
        cloud=CloudProvider.AWS,
        region=AwsRegion.US_WEST_2
    ),
    deletion_protection=DeletionProtection.DISABLED,
    vector_type=VectorType.DENSE,
    tags={
        "model": "clip",
        "app": "image-search",
        "env": "testing"
    }
)

Creating a pod index¶

import os
from pinecone import (
    Pinecone,
    PodSpec,
    PodIndexEnvironment,
    PodType,
    Metric,
    DeletionProtection,
    VectorType
)

pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))

pc.create_index(
    name="my_index",
    dimension=1536,
    metric=Metric.COSINE,
    spec=PodSpec(
        environment=PodIndexEnvironment.US_EAST4_GCP,
        pod_type=PodType.P1_X1
    ),
    deletion_protection=DeletionProtection.DISABLED,
    tags={
        "model": "clip",
        "app": "image-search",
        "env": "testing"
    }
)

Parameters:

name (str) – The name of the index to create. Must be unique within your project and cannot be changed once created. Allowed characters are lowercase letters, numbers, and hyphens and the name may not begin or end with hyphens. Maximum length is 45 characters.
cloud (str) – The cloud provider to use for the index. One of {"aws", "gcp", "azure"}.
region (str) – The region to use for the index. Enum objects AwsRegion, GcpRegion, and AzureRegion are also available to help you quickly set these parameters, but may not be up to date as new regions become available.
embed (Union[Dict, IndexEmbed]) – The embedding configuration for the index. This param accepts a dictionary or an instance of the IndexEmbed object.
tags (Optional[Dict[str, str]]) – Tags are key-value pairs you can attach to indexes to better understand, organize, and identify your resources. Some example use cases include tagging indexes with the name of the model that generated the embeddings, the date the index was created, or the purpose of the index.
deletion_protection (Optional[Literal["enabled", "disabled"]]) – If enabled, the index cannot be deleted. If disabled, the index can be deleted. This setting can be changed with configure_index.
timeout (Optional[int]) – Specify the number of seconds to wait until index is ready to receive data. If None, wait indefinitely; if >=0, time out after this many seconds; if -1, return immediately and do not wait.

Returns:

A description of the index that was created.

Return type:

IndexModel

This method is used to create a Serverless index that is configured for use with Pinecone’s integrated inference models.

The resulting index can be described, listed, configured, and deleted like any other Pinecone index with the describe_index, list_indexes, configure_index, and delete_index methods.

After the model is created, you can upsert records into the index with the upsert_records method, and search your records with the search method.

from pinecone import (
    Pinecone,
    IndexEmbed,
    CloudProvider,
    AwsRegion,
    EmbedModel,
    Metric,
)

pc = Pinecone()

if not pc.has_index("book-search"):
    desc = await pc.create_index_for_model(
        name="book-search",
        cloud=CloudProvider.AWS,
        region=AwsRegion.US_EAST_1,
        embed=IndexEmbed(
            model=EmbedModel.Multilingual_E5_Large,
            metric=Metric.COSINE,
            field_map={
                "text": "description",
            },
        )
    )

Backups¶

Pinecone.create_backup(*, index_name: str, backup_name: str, description: str = '') → BackupModel[source]¶

Create a backup of an index.

Parameters:

index_name (str) – The name of the index to backup.
backup_name (str) – The name to give the backup.
description (str, optional) – Optional description of the backup.

Pinecone.list_backups(*, index_name: str | None = None, limit: int | None = 10, pagination_token: str | None = None) → BackupList[source]¶

List backups.

If index_name is provided, the backups will be filtered by index. If no index_name is provided, all backups in the project will be returned.

Parameters:

index_name (str, optional) – The name of the index to list backups for.
limit (int, optional) – The maximum number of backups to return.
pagination_token (str, optional) – The pagination token to use for pagination.

Pinecone.describe_backup(*, backup_id: str) → BackupModel[source]¶

Describe a backup.

Parameters:: backup_id (str) – The ID of the backup to describe.

Pinecone.delete_backup(*, backup_id: str) → None[source]¶

Delete a backup.

Parameters:: backup_id (str) – The ID of the backup to delete.

Collections¶

Pinecone.create_collection(name: str, source: str) → None[source]¶

Create a collection from a pod-based index

Parameters:

name (str, required) – Name of the collection
source (str, required) – Name of the source index

Pinecone.list_collections() → CollectionList[source]¶

List all collections

from pinecone import Pinecone

pc = Pinecone()

for collection in pc.list_collections():
    print(collection.name)
    print(collection.source)

# You can also iterate specifically over the collection
# names with the .names() helper.
collection_name="my_collection"
for collection_name in pc.list_collections().names():
    print(collection_name)

Pinecone.describe_collection(name: str)[source]¶

Describes a collection.

Parameters:: name (str) – The name of the collection
Returns:: Description of the collection

from pinecone import Pinecone

pc = Pinecone()

description = pc.describe_collection("my_collection")
print(description.name)
print(description.source)
print(description.status)
print(description.size)

Pinecone.delete_collection(name: str) → None[source]¶

Parameters:: name (str) – The name of the collection to delete.

Deletes a collection.

Deleting a collection is an irreversible operation. All data in the collection will be lost.

This method tells Pinecone you would like to delete a collection, but it takes a few moments to complete the operation. Use the describe_collection() method to confirm that the collection has been deleted.

from pinecone import Pinecone

pc = Pinecone()

pc.delete_collection(name="my_collection")

Restore Jobs¶

Pinecone.list_restore_jobs(*, limit: int | None = 10, pagination_token: str | None = None) → RestoreJobList[source]¶

List restore jobs.

Parameters:

limit (int) – The maximum number of restore jobs to return.
pagination_token (str) – The pagination token to use for pagination.

Pinecone.describe_restore_job(*, job_id: str) → RestoreJobModel[source]¶

Describe a restore job.

Parameters:: job_id (str) – The ID of the restore job to describe.

DB Data Plane¶

class pinecone.db_data.Index(api_key: str, host: str, pool_threads: int | None = None, additional_headers: Dict[str, str] | None = {}, openapi_config=None, **kwargs)[source]¶: A client for interacting with a Pinecone index via REST API. For improved performance, use the Pinecone GRPC index client.

Index.__init__(api_key: str, host: str, pool_threads: int | None = None, additional_headers: Dict[str, str] | None = {}, openapi_config=None, **kwargs)[source]¶

Initialize the PluginAware class.

Parameters:

*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.

Raises:

AttributeError – If required attributes are not set in the subclass.

The DescribeIndexStats operation returns statistics about the index’s contents. For example: The vector count per namespace and the number of dimensions.

Parameters:

filter (Dict[str, Union[str, float, int, bool, List, dict]])
present (If this parameter is)
filter. (the operation only returns statistics for vectors that satisfy the)
<https (See `metadata filtering) –
//www.pinecone.io/docs/metadata-filtering/>_` [optional]

Returns: DescribeIndexStatsResponse object which contains stats about the index.

>>> pc = Pinecone()
>>> index = pc.Index(name="my-index")
>>> index.describe_index_stats()
{'dimension': 1536,
'index_fullness': 0.0,
'metric': 'cosine',
'namespaces': {'ns0': {'vector_count': 700},
                'ns1': {'vector_count': 700},
                'ns2': {'vector_count': 500},
                'ns3': {'vector_count': 100},
                'ns4': {'vector_count': 100},
                'ns5': {'vector_count': 50},
                'ns6': {'vector_count': 50}},
'total_vector_count': 2200,
'vector_type': 'dense'}

Vectors¶

Parameters:

vectors (Union[List[Vector], List[VectorTuple], List[VectorTupleWithMetadata], List[VectorTypedDict]]) – A list of vectors to upsert.
namespace (str) – The namespace to write to. If not specified, the default namespace is used. [optional]
batch_size (int) – The number of vectors to upsert in each batch. If not specified, all vectors will be upserted in a single batch. [optional]
show_progress (bool) – Whether to show a progress bar using tqdm. Applied only if batch_size is provided. Default is True.

Returns:

UpsertResponse, includes the number of vectors upserted.

The upsert operation writes vectors into a namespace of your index.

If a new value is upserted for an existing vector id, it will overwrite the previous value.

To upsert in parallel follow this link.

Upserting dense vectors

When working with dense vectors, the dimension of each vector must match the dimension configured for the index.

A vector can be represented in a variety of ways.

Upserting a dense vector using the Vector object¶

from pinecone import Pinecone, Vector

pc = Pinecone()
idx = pc.Index(name="index-name")

idx.upsert(
    namespace = 'my-namespace',
    vectors = [
        Vector(
            id='id1',
            values=[0.1, 0.2, 0.3, 0.4],
            metadata={'metadata_key': 'metadata_value'}
        ),
    ]
)

Upserting a dense vector as a two-element tuple (no metadata)¶

idx.upsert(
    namespace = 'my-namespace',
    vectors = [
        ('id1', [0.1, 0.2, 0.3, 0.4]),
    ]
)

Upserting a dense vector as a three-element tuple with metadata¶

idx.upsert(
    namespace = 'my-namespace',
    vectors = [
        (
            'id1',
            [0.1, 0.2, 0.3, 0.4],
            {'metadata_key': 'metadata_value'}
        ),
    ]
)

Upserting a dense vector using a vector dictionary¶

idx.upsert(
    namespace = 'my-namespace',
    vectors = [
        {
            "id": 1,
            "values": [0.1, 0.2, 0.3, 0.4],
            "metadata": {"metadata_key": "metadata_value"}
        },
    ]

Upserting sparse vectors

Upserting a sparse vector¶

from pinecone import (
    Pinecone,
    Metric,
    Vector,
    SparseValues,
    VectorType,
    ServerlessSpec,
    CloudProvider,
    AwsRegion
)

pc = Pinecone() # Reads PINECONE_API_KEY from environment variable

# Create a sparse index
index_description = pc.create_index(
    name="example-sparse",
    metric=Metric.Dotproduct,
    vector_type=VectorType.Sparse,
    spec=ServerlessSpec(
        cloud=CloudProvider.AWS,
        region=AwsRegion.US_WEST_2,
    )
)

# Target the index created above
idx = pc.Index(host=index_description.host)

# Upsert a sparse vector
idx.upsert(
    namespace='my-namespace',
    vectors=[
        Vector(
            id='id1',
            sparse_values=SparseValues(
                indices=[1, 2],
                values=[0.2, 0.4]
            )
        ),
    ]
)

Upserting a sparse vector using a dictionary¶

idx.upsert(
    namespace = 'my-namespace',
    vectors = [
        {
            "id": 1,
            "sparse_values": {
                "indices": [1, 2],
                "values": [0.2, 0.4]
            }
        },
    ]
)

Batch upsert

If you have a large number of vectors, you can upsert them in batches.

Upserting in batches¶

from pinecone import Pinecone, Vector
import random

pc = Pinecone()
idx = pc.Index(host="example-index-dojoi3u.svc.preprod-aws-0.pinecone.io")

# Create some fake vector data for demonstration
num_vectors = 100000
vectors = [
    Vector(
        id=f'id{i}',
        values=[random.random() for _ in range(1536)])
    for i in range(num_vectors)
]

idx.upsert(
    namespace='my-namespace',
    vectors=vectors,
    batch_size=50
)

Visual progress bar with tqdm

To see a progress bar when upserting in batches, you will need to separately install tqdm. If tqdm is present, the client will detect and use it to display progress when show_progress=True.

The Query operation searches a namespace, using a query vector. It retrieves the ids of the most similar items in a namespace, along with their similarity scores.

Examples:

>>> index.query(vector=[1, 2, 3], top_k=10, namespace='my_namespace')
>>> index.query(id='id1', top_k=10, namespace='my_namespace')
>>> index.query(vector=[1, 2, 3], top_k=10, namespace='my_namespace', filter={'key': 'value'})
>>> index.query(id='id1', top_k=10, namespace='my_namespace', include_metadata=True, include_values=True)
>>> index.query(vector=[1, 2, 3], sparse_vector={'indices': [1, 2], 'values': [0.2, 0.4]},
>>>             top_k=10, namespace='my_namespace')
>>> index.query(vector=[1, 2, 3], sparse_vector=SparseValues([1, 2], [0.2, 0.4]),
>>>             top_k=10, namespace='my_namespace')

Parameters:

vector (List[float]) – The query vector. This should be the same length as the dimension of the index being queried. Each query() request can contain only one of the parameters id or vector.. [optional]
id (str) – The unique ID of the vector to be used as a query vector. Each query() request can contain only one of the parameters vector or id. [optional]
top_k (int) – The number of results to return for each query. Must be an integer greater than 1.
namespace (str) – The namespace to query vectors from. If not specified, the default namespace is used. [optional]
filter (Dict[str, Union[str, float, int, bool, List, dict]) – The filter to apply. You can use vector metadata to limit your search. See metadata filtering <https://www.pinecone.io/docs/metadata-filtering/>_ [optional]
include_values (bool) – Indicates whether vector values are included in the response. If omitted the server will use the default value of False [optional]
include_metadata (bool) – Indicates whether metadata is included in the response as well as the ids. If omitted the server will use the default value of False [optional]
sparse_vector –
(Union[SparseValues, Dict[str, Union[List[float], List[int]]]]): sparse values of the query vector. Expected to be either a SparseValues object or a dict of the form:

{‘indices’: List[int], ‘values’: List[float]}, where the lists each have the same length.

Returns: QueryResponse object which contains the list of the closest vectors as ScoredVector objects,: and namespace name.

The query_namespaces() method is used to make a query to multiple namespaces in parallel and combine the results into one result set.

Parameters:

vector (List[float]) – The query vector, must be the same length as the dimension of the index being queried.
namespaces (List[str]) – The list of namespaces to query.
top_k (Optional[int]) – The number of results you would like to request from each namespace. Defaults to 10.
metric (str) – Must be one of ‘cosine’, ‘euclidean’, ‘dotproduct’. This is needed in order to merge results across namespaces, since the interpretation of score depends on the index metric type.
filter (Optional[Dict[str, Union[str, float, int, bool, List, dict]]]) – Pass an optional filter to filter results based on metadata. Defaults to None.
include_values (Optional[bool]) – Boolean field indicating whether vector values should be included with results. Defaults to None.
include_metadata (Optional[bool]) – Boolean field indicating whether vector metadata should be included with results. Defaults to None.
sparse_vector (Optional[ Union[SparseValues, Dict[str, Union[List[float], List[int]]]] ]) – If you are working with a dotproduct index, you can pass a sparse vector as part of your hybrid search. Defaults to None.

Returns:

A QueryNamespacesResults object containing the combined results from all namespaces, as well as the combined usage cost in read units.

Return type:

QueryNamespacesResults

Note

Since several asynchronous calls are made on your behalf when calling this method, you will need to tune the pool_threads and connection_pool_maxsize parameter of the Index constructor to suite your workload. If these values are too small in relation to your workload, you will experience performance issues as requests queue up while waiting for a request thread to become available.

Examples:

from pinecone import Pinecone

pc = Pinecone()

index = pc.Index(
    host="index-name",
    pool_threads=32,
    connection_pool_maxsize=32
)

query_vec = [0.1, 0.2, 0.3] # An embedding that matches the index dimension
combined_results = index.query_namespaces(
    vector=query_vec,
    namespaces=['ns1', 'ns2', 'ns3', 'ns4'],
    metric="cosine",
    top_k=10,
    filter={'genre': {"$eq": "drama"}},
    include_values=True,
    include_metadata=True
)

for vec in combined_results.matches:
    print(vec.id, vec.score)
print(combined_results.usage)

Parameters:

ids (List[str]) – Vector ids to delete [optional]
delete_all (bool) – This indicates that all vectors in the index namespace should be deleted.. [optional] Default is False.
namespace (str) – The namespace to delete vectors from [optional] If not specified, the default namespace is used.
filter (Dict[str, Union[str, float, int, bool, List, dict]]) – If specified, the metadata filter here will be used to select the vectors to delete. This is mutually exclusive with specifying ids to delete in the ids param or using delete_all=True. See metadata filtering <https://www.pinecone.io/docs/metadata-filtering/>_ [optional]

The Delete operation deletes vectors from the index, from a single namespace.

No error is raised if the vector id does not exist.

Note: For any delete call, if namespace is not specified, the default namespace "" is used. Since the delete operation does not error when ids are not present, this means you may not receive an error if you delete from the wrong namespace.

Delete can occur in the following mutual exclusive ways:

Delete by ids from a single namespace
Delete all vectors from a single namespace by setting delete_all to True
Delete all vectors from a single namespace by specifying a metadata filter
(note that for this option delete all must be set to False)

Examples:

>>> index.delete(ids=['id1', 'id2'], namespace='my_namespace')
>>> index.delete(delete_all=True, namespace='my_namespace')
>>> index.delete(filter={'key': 'value'}, namespace='my_namespace')

Returns: An empty dictionary if the delete operation was successful.

Index.fetch(ids: List[str], namespace: str | None = None, **kwargs) → FetchResponse[source]¶

The fetch operation looks up and returns vectors, by ID, from a single namespace. The returned vectors include the vector data and/or metadata.

Examples:

>>> index.fetch(ids=['id1', 'id2'], namespace='my_namespace')
>>> index.fetch(ids=['id1', 'id2'])

Parameters:

ids (List[str]) – The vector IDs to fetch.
namespace (str) – The namespace to fetch vectors from. If not specified, the default namespace is used. [optional]

Returns: FetchResponse object which contains the list of Vector objects, and namespace name.

Index.list(**kwargs)[source]¶

The list operation accepts all of the same arguments as list_paginated, and returns a generator that yields a list of the matching vector ids in each page of results. It automatically handles pagination tokens on your behalf.

Examples

>>> for ids in index.list(prefix='99', limit=5, namespace='my_namespace'):
>>>     print(ids)
['99', '990', '991', '992', '993']
['994', '995', '996', '997', '998']
['999']

Parameters:

prefix (Optional[str]) – The id prefix to match. If unspecified, an empty string prefix will be used with the effect of listing all ids in a namespace [optional]
limit (Optional[int]) – The maximum number of ids to return. If unspecified, the server will use a default value. [optional]
pagination_token (Optional[str]) – A token needed to fetch the next page of results. This token is returned in the response if additional results are available. [optional]
namespace (Optional[str]) – The namespace to fetch vectors from. If not specified, the default namespace is used. [optional]

Index.list_paginated(prefix: str | None = None, limit: int | None = None, pagination_token: str | None = None, namespace: str | None = None, **kwargs) → ListResponse[source]¶

The list_paginated operation finds vectors based on an id prefix within a single namespace. It returns matching ids in a paginated form, with a pagination token to fetch the next page of results. This id list can then be passed to fetch or delete operations, depending on your use case.

Consider using the list method to avoid having to handle pagination tokens manually.

Examples:

>>> results = index.list_paginated(prefix='99', limit=5, namespace='my_namespace')
>>> [v.id for v in results.vectors]
['99', '990', '991', '992', '993']
>>> results.pagination.next
eyJza2lwX3Bhc3QiOiI5OTMiLCJwcmVmaXgiOiI5OSJ9
>>> next_results = index.list_paginated(prefix='99', limit=5, namespace='my_namespace', pagination_token=results.pagination.next)

Parameters:

prefix (Optional[str]) – The id prefix to match. If unspecified, an empty string prefix will be used with the effect of listing all ids in a namespace [optional]
limit (Optional[int]) – The maximum number of ids to return. If unspecified, the server will use a default value. [optional]
pagination_token (Optional[str]) – A token needed to fetch the next page of results. This token is returned in the response if additional results are available. [optional]
namespace (Optional[str]) – The namespace to fetch vectors from. If not specified, the default namespace is used. [optional]

Returns: ListResponse object which contains the list of ids, the namespace name, pagination information, and usage showing the number of read_units consumed.

Bulk Import¶

Index.start_import(uri: str, integration_id: str | None = None, error_mode: ImportErrorMode | Literal['CONTINUE', 'ABORT'] | str | None = 'CONTINUE') → StartImportResponse[source]¶

Parameters:

uri (str) – The URI of the data to import. The URI must start with the scheme of a supported storage provider.
integration_id (Optional[str], optional) – If your bucket requires authentication to access, you need to pass the id of your storage integration using this property. Defaults to None.
error_mode – Defaults to “CONTINUE”. If set to “CONTINUE”, the import operation will continue even if some records fail to import. Pass “ABORT” to stop the import operation if any records fail to import.

Returns:

Contains the id of the import operation.

Return type:

StartImportResponse

Import data from a storage provider into an index. The uri must start with the scheme of a supported storage provider. For buckets that are not publicly readable, you will also need to separately configure a storage integration and pass the integration id.

Examples

>>> from pinecone import Pinecone
>>> index = Pinecone().Index('my-index')
>>> index.start_import(uri="s3://bucket-name/path/to/data.parquet")
{ id: "1" }

Index.list_imports(**kwargs) → Iterator[ImportModel][source]¶

Parameters:

limit (Optional[int]) – The maximum number of operations to fetch in each network call. If unspecified, the server will use a default value. [optional]
pagination_token (Optional[str]) – When there are multiple pages of results, a pagination token is returned in the response. The token can be used to fetch the next page of results. [optional]

Returns:

Returns a generator that yields each import operation. It automatically handles pagination tokens on your behalf so you can easily iterate over all results. The list_imports method accepts all of the same arguments as list_imports_paginated

for op in index.list_imports():
    print(op)

You can convert the generator into a list by wrapping the generator in a call to the built-in list function:

operations = list(index.list_imports())

You should be cautious with this approach because it will fetch all operations at once, which could be a large number of network calls and a lot of memory to hold the results.

Index.list_imports_paginated(limit: int | None = None, pagination_token: str | None = None, **kwargs) → ListImportsResponse[source]¶

Parameters:

limit (Optional[int]) – The maximum number of ids to return. If unspecified, the server will use a default value. [optional]
pagination_token (Optional[str]) – A token needed to fetch the next page of results. This token is returned in the response if additional results are available. [optional]

Returns: ListImportsResponse object which contains the list of operations as ImportModel objects, pagination information,: and usage showing the number of read_units consumed.

The list_imports_paginated() operation returns information about import operations. It returns operations in a paginated form, with a pagination token to fetch the next page of results.

Consider using the list_imports method to avoid having to handle pagination tokens manually.

Examples:

>>> results = index.list_imports_paginated(limit=5)
>>> results.pagination.next
eyJza2lwX3Bhc3QiOiI5OTMiLCJwcmVmaXgiOiI5OSJ9
>>> results.data[0]
{
    "id": "6",
    "uri": "s3://dev-bulk-import-datasets-pub/10-records-dim-10/",
    "status": "Completed",
    "percent_complete": 100.0,
    "records_imported": 10,
    "created_at": "2024-09-06T14:52:02.567776+00:00",
    "finished_at": "2024-09-06T14:52:28.130717+00:00"
}
>>> next_results = index.list_imports_paginated(limit=5, pagination_token=results.pagination.next)

Index.describe_import(id: str) → ImportModel[source]¶

Parameters:: id (str) – The id of the import operation. This value is returned when starting an import, and can be looked up using list_imports.
Returns:: An object containing operation id, status, and other details.
Return type:: ImportModel

describe_import is used to get detailed information about a specific import operation.

Index.cancel_import(id: str)[source]¶

Cancel an import operation.

Parameters:: id (str) – The id of the import operation to cancel.

Records¶

If you have created an index using integrated inference, you can use the following methods to search and retrieve records.

Index.search(namespace: str, query: SearchQueryTypedDict | SearchQuery, rerank: SearchRerankTypedDict | SearchRerank | None = None, fields: List[str] | None = ['*']) → SearchRecordsResponse[source]¶

Parameters:

namespace (str, required) – The namespace in the index to search.
query (Union[Dict, SearchQuery], required) – The SearchQuery to use for the search.
rerank (Union[Dict, SearchRerank], optional) – The SearchRerank to use with the search request.

Returns:

The records that match the search.

Search for records.

This operation converts a query to a vector embedding and then searches a namespace. You can optionally provide a reranking operation as part of the search.

from pinecone import (
    Pinecone,
    CloudProvider,
    AwsRegion,
    EmbedModel
    IndexEmbed
)

pc = Pinecone(api_key="<<PINECONE_API_KEY>>")

# Create an index for your embedding model
index_model = pc.create_index_for_model(
    name="my-model-index",
    cloud=CloudProvider.AWS,
    region=AwsRegion.US_WEST_2,
    embed=IndexEmbed(
        model=EmbedModel.Multilingual_E5_Large,
        field_map={"text": "my_text_field"}
    )
)

# Instantiate the index client
idx = pc.Index(host=index_model.host)

# upsert records
idx.upsert_records(
    namespace="my-namespace",
    records=[
        {
            "_id": "test1",
            "my_text_field": "Apple is a popular fruit known for its sweetness and crisp texture.",
        },
        {
            "_id": "test2",
            "my_text_field": "The tech company Apple is known for its innovative products like the iPhone.",
        },
        {
            "_id": "test3",
            "my_text_field": "Many people enjoy eating apples as a healthy snack.",
        },
        {
            "_id": "test4",
            "my_text_field": "Apple Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces.",
        },
        {
            "_id": "test5",
            "my_text_field": "An apple a day keeps the doctor away, as the saying goes.",
        },
        {
            "_id": "test6",
            "my_text_field": "Apple Computer Company was founded on April 1, 1976, by Steve Jobs, Steve Wozniak, and Ronald Wayne as a partnership.",
        },
    ],
)

from pinecone import SearchQuery, SearchRerank, RerankModel

# search for similar records
response = idx.search_records(
    namespace="my-namespace",
    query=SearchQuery(
        inputs={
            "text": "Apple corporation",
        },
        top_k=3,
    ),
    rerank=SearchRerank(
        model=RerankModel.Bge_Reranker_V2_M3,
        rank_fields=["my_text_field"],
        top_n=3,
    ),
)

Index.search_records(namespace: str, query: SearchQueryTypedDict | SearchQuery, rerank: SearchRerankTypedDict | SearchRerank | None = None, fields: List[str] | None = ['*']) → SearchRecordsResponse[source]¶: Alias of the search() method.

Inference¶

Inference.embed(model: EmbedModel | str, inputs: str | List[Dict] | List[str], parameters: Dict[str, Any] | None = None) → EmbeddingsList[source]¶

Generates embeddings for the provided inputs using the specified model and (optional) parameters.

Parameters:

model (str, required) – The model to use for generating embeddings.
inputs (list, required) – A list of items to generate embeddings for.
parameters (dict, optional) – A dictionary of parameters to use when generating embeddings.

Returns:

EmbeddingsList object with keys data, model, and usage. The data key contains a list of n embeddings, where n = len(inputs). Precision of returned embeddings is either float16 or float32, with float32 being the default. model key is the model used to generate the embeddings. usage key contains the total number of tokens used at request-time.

Example:

>>> pc = Pinecone()
>>> inputs = ["Who created the first computer?"]
>>> outputs = pc.inference.embed(model="multilingual-e5-large", inputs=inputs, parameters={"input_type": "passage", "truncate": "END"})
>>> print(outputs)
EmbeddingsList(
    model='multilingual-e5-large',
    data=[
        {'values': [0.1, ...., 0.2]},
    ],
    usage={'total_tokens': 6}
)

Inference.rerank(model: RerankModel | str, query: str, documents: List[str] | List[Dict[str, Any]], rank_fields: List[str] = ['text'], return_documents: bool = True, top_n: int | None = None, parameters: Dict[str, Any] | None = None) → RerankResult[source]¶

Rerank documents with associated relevance scores that represent the relevance of each document to the provided query using the specified model.

Parameters:

model (str, required) – The model to use for reranking.
query (str, required) – The query to compare with documents.
documents (list, required) – A list of documents or strings to rank.
rank_fields (list, optional) – A list of document fields to use for ranking. Defaults to [“text”].
return_documents (bool, optional) – Whether to include the documents in the response. Defaults to True.
top_n (int, optional) – How many documents to return. Defaults to len(documents).
parameters (dict, optional) – A dictionary of parameters to use when ranking documents.

Returns:

RerankResult object with keys data and usage. The data key contains a list of n documents, where n = top_n and type(n) = Document. The documents are sorted in order of relevance, with the first being the most relevant. The index field can be used to locate the document relative to the list of documents specified in the request. Each document contains a score key representing how close the document relates to the query.

Example:

>>> pc = Pinecone()
>>> pc.inference.rerank(
        model="bge-reranker-v2-m3",
        query="Tell me about tech companies",
        documents=[
            "Apple is a popular fruit known for its sweetness and crisp texture.",
            "Software is still eating the world.",
            "Many people enjoy eating apples as a healthy snack.",
            "Acme Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces.",
            "An apple a day keeps the doctor away, as the saying goes.",
    ],
    top_n=2,
    return_documents=True,
)
RerankResult(
    model='bge-reranker-v2-m3',
    data=[{
        index=3,
        score=0.020924192,
        document={
            text='Acme Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces.'
        }
    },{
        index=1,
        score=0.00034464317,
        document={
            text='Software is still eating the world.'
        }
    }],
    usage={'rerank_units': 1}
)

Inference.list_models(*, type: str | None = None, vector_type: str | None = None) → ModelInfoList[source]¶

List all available models.

Parameters:

type (str, optional) – The type of model to list. Either “embed” or “rerank”.
vector_type (str, optional) – The type of vector to list. Either “dense” or “sparse”.

Returns:

A list of models.

Example:

pc = Pinecone()

# List all models
models = pc.inference.list_models()

# List models, with model type filtering
models = pc.inference.list_models(type="embed")
models = pc.inference.list_models(type="rerank")

# List models, with vector type filtering
models = pc.inference.list_models(vector_type="dense")
models = pc.inference.list_models(vector_type="sparse")

# List models, with both type and vector type filtering
models = pc.inference.list_models(type="rerank", vector_type="dense")

Inference.get_model(model_name: str) → ModelInfo[source]¶

Get details on a specific model.

Parameters:: model_name (str, required) – The name of the model to get details on.
Returns:: A ModelInfo object.

>>> pc = Pinecone()
>>> pc.inference.get_model(model_name="pinecone-rerank-v0")
{
    "model": "pinecone-rerank-v0",
    "short_description": "A state of the art reranking model that out-performs competitors on widely accepted benchmarks. It can handle chunks up to 512 tokens (1-2 paragraphs)",
    "type": "rerank",
    "supported_parameters": [
        {
            "parameter": "truncate",
            "type": "one_of",
            "value_type": "string",
            "required": false,
            "default": "END",
            "allowed_values": [
                "END",
                "NONE"
            ]
        }
    ],
    "modality": "text",
    "max_sequence_length": 512,
    "max_batch_size": 100,
    "provider_name": "Pinecone",
    "supported_metrics": []
}

Navigation

Related Topics

Pinecone¶

DB Control Plane¶

Indexes¶

Backups¶

Collections¶

Restore Jobs¶

DB Data Plane¶

Vectors¶

Bulk Import¶

Records¶

Inference¶