Configure Index Partition

On this page

Syntax

Supported Values
Example

Important

The numPartitions option is available as a Preview feature.

For indexing, Atlas Search counts each document as a single index object when it isn't nested inside another document. For embedded documents, Atlas Search counts each embedded document as additional index objects depending on the number of levels of nesting. Atlas Search stops replicating changes for indexes larger than 2,100,000,000 index objects.

If you deployed Atlas Search on separate search nodes, you can increase the number of Atlas Search index objects by partitioning your index objects in to sub-indexes. By default, Atlas Search supports one partition per shard. Each partition supports up to 2 billion index objects. You can create up to sixty-four (64) sub-indexes by using the numPartitions option.

When you configure partitions for your index, Atlas Search automatically distributes the index objects between the sub-indexes in an optimal way. When you run queries against a collection with sub-indexes, Atlas Search scatters the queries to all the sub-indexes and gathers the search results and metadata to sort, merge, and return the results.

We recommend partitioning your index when:

Your index objects reach 50% of the total limit.
The number of documents in your collection reaches two billion.
Your index is in the STALE state because Atlas Search stopped replication.

When you configure sub-indexes or modify the number of sub-indexes, Atlas Search triggers a rebuild of your index.

If you have more than one sub-index in your cluster, you can't remove all the search nodes and migrate to a deployment model where both the mongod and mongot processes run on the same node.

Syntax

{
  "name": "<index-name>",
  "analyzer": "<analyzer-for-index>",
  "searchAnalyzer": "<analyzer-for-query>",
  "mappings": {
    "dynamic": <boolean>,
    "fields": { <field-definition> }
  },
  "numPartitions": <integer>,
  ...
}

Supported Values

The Atlas Search numPartitions option takes the following values:

1 - to create a single index, with no additional sub-indexes. This is the default value.
2 - to create up to two sub-indexes.
4 - to create up to four sub-indexes.
8 - to create up to eight sub-indexes.
16 - to create up to sixteen sub-indexes.
32 - to create up to thirty-two sub-indexes.
64 - to create up to sixty-four sub-indexes.

Example

The following index example uses the sample_mflix.movies collection to demonstrate how to configure up to 4 sub-indexes for the data in the collection. You can use the Visual Editor or the JSON Editor in the Atlas UI and other supported clients to create the index.

➤ Use the Select your language drop-down menu to set the client of the example in this section.

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--include \
--request POST "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes" \
--data '
  {
  "collectionName": "movies",
  "database": "sample_mflix",
  "name": "partitioned_index",
  "type": "search",
  "definition": {
    "analyzer": "lucene.standard",
    "mappings": {
      "dynamic": true,
    },
    "numPartitions": 4,
    "searchAnalyzer": "lucene.standard"
  }
}'

Create a file named indexDef.json similar to the following:
```
{
    "collectionName": "movies",
    "database": "sample_mflix",
    "definition": {
      "mappings": {
       "dynamic": true
      },
    },
    "name": "partitioned_index",
    "numPartitions": 4
  }
```
Run the following command to create the index.
```
atlas deployments search indexes create --file indexDef.json
```

Visual Editor

In the Atlas UI, select the following:

Search Type	Atlas Search
Index Name and Data Source	Index Name: `partitioned_index` Data Source: `sample_mflix.movies`
Configuration Method	Visual Editor

Click Next, then Refine Your Index.
Toggle Index Partitions to enable it.
Select 4 from the Number of partitions dropdown and click Save Changes.
Click Create Index to create the index.

JSON Editor

In the Atlas UI, select the following:
Search Type
Atlas Search
Index Name and Data Source
Index Name: partitioned_index
Data Source: sample_mflix.movies
Configuration Method
JSON Editor
Replace the default index definition with the following:
```
{
  "mappings": {
    "dynamic": true
  },
  "numPartitions": 4
}
```
Click Next, then Create Index to create the index.

db.movies.createSearchIndex(
    "search-index",
    { mappings: { dynamic: true }, "numPartitions": 4 }
)

using MongoDB.Bson;
using MongoDB.Driver;
// connect to your Atlas deployment
var uri = "<connection-string>";
var client = new MongoClient(uri);
var db = client.GetDatabase("sample_mflix");
var collection = db.GetCollection<BsonDocument>("movies");
// define your Atlas Search index
var index = new BsonDocument
{
  { "mappings", new BsonDocument
    {
      { "dynamic", true }
    }
  },
  { "numPartitions", 4 }
};
var result = collection.SearchIndexes.CreateOne(index, "partitioned_index");
Console.WriteLine(result);

import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
public class CreateIndex {
    public static void main(String[] args) {
        // connect to your Atlas cluster
        String uri = "<connection-string>";
        try (MongoClient mongoClient = MongoClients.create(uri)) {
            // set namespace
            MongoDatabase database = mongoClient.getDatabase("sample_mflix");
            MongoCollection<Document> collection = database.getCollection("movies");
            Document index = new Document()
                .append("mappings", new Document()
                   .append("dynamic", true)
                )
                .append("numPartitions", 4);
            collection.createSearchIndex("partitioned_index", index);
        }
    }
}

import { MongoClient } from "mongodb";
// connect to your Atlas deployment
const uri =  "<connection-string>";
const client = new MongoClient(uri);
async function run() {
  try {
    const database = client.db("sample_mflix");
    const collection = database.collection("movies");
    
    // define your Atlas Search index
    const index = {
        name: "partitioned_index",
        definition: {
            /* search index definition fields */
            "mappings": {
                "dynamic": true
            },
            "numPartitions": 4
        }
    }
    // run the helper method
    const result = await collection.createSearchIndex(index);
    console.log(result);
  } finally {
    await client.close();
  }
}
run().catch(console.dir);

from pymongo.mongo_client import MongoClient
from pymongo.operations import SearchIndexModel
def create_index():
    # Connect to your Atlas deployment
    uri = "<connectionString>"
    client = MongoClient(uri)
    # Access your database and collection
    database = client["sample_mflix"]
    collection = database["movies"]
    # Create your index model, then create the search index
    search_index_model = SearchIndexModel(
        definition={
            "mappings": {
                "dynamic": True
            },
            "numPartitions": 4
        },
        name="partitioned_index",
    )
    result = collection.create_search_index(model=search_index_model)
    print(result)

Back

4: Configure Synonym Mappings

6: Review Search Syntax