Link Search Menu Expand Document

This documentation remains in a beta state. It has content gaps and might contain bugs.

k-NN Index

knn_vector data type

The k-NN plugin introduces a custom data type, the knn_vector, that allows users to ingest their k-NN vectors into an OpenSearch index.

"my_vector": {
  "type": "knn_vector",
  "dimension": 4,
  "method": {
    "name": "hnsw",
    "space_type": "l2",
    "engine": "nmslib",
    "parameters": {
      "ef_construction": 128,
      "m": 24
    }
  }
}
Mapping Pararameter Required Default Updateable Description
type true n/a false The type of the field
dimension true n/a false The vector dimension for the field
method false null false The configuration for the Approximate nearest neighbor method
method.name true, if method is specified n/a false The identifier for the nearest neighbor method. Currently, “hnsw” is the only valid method.
method.space_type false “l2” false The vector space used to calculate the distance between vectors. Refer to here) to see available spaces.
method.engine false “nmslib” false The approximate k-NN library to use for indexing and search. Currently, “nmslib” is the only valid engine.
method.parameters false null false The parameters used for the nearest neighbor method.
method.parameters.ef_construction false 512 false The size of the dynamic list used during k-NN graph creation. Higher values lead to a more accurate graph, but slower indexing speed. Only valid for “hnsw” method.
method.parameters.m false 16 false The number of bidirectional links that the plugin creates for each new element. Increasing and decreasing this value can have a large impact on memory consumption. Keep this value between 2-100. Only valid for “hnsw” method

Index settings

Additionally, the k-NN plugin introduces several index settings that can be used to configure the k-NN structure as well.

At the moment, several parameters defined in the settings are in the deprecation process. Those parameters should be set in the mapping instead of the index settings. Parameters set in the mapping will override the parameters set in the index settings. Setting the parameters in the mapping allows an index to have multiple knn_vector fields with different parameters.

Setting Default Updateable Description
index.knn false false Whether the index should build hnsw graphs for the knn_vector fields. If set to false, the knn_vector fields will be stored in doc values, but Approximate k-NN search functionality will be disabled.
index.knn.algo_param.ef_search 512 true The size of the dynamic list used during k-NN searches. Higher values lead to more accurate but slower searches.
index.knn.algo_param.ef_construction 512 false (Deprecated in 1.0.0. Use the mapping parameters to set this value instead.) Refer to mapping definition.
index.knn.algo_param.m 16 false (Deprecated in 1.0.0. Use the mapping parameters to set this value instead.) Refer to mapping definition.
index.knn.space_type “l2” false (Deprecated in 1.0.0. Use the mapping parameters to set this value instead.) Refer to mapping definition.