All About Partitioning In Azure Cosmos DB

Azure Cosmos DB provides you the opportunity to store a huge amount of that.
When there is a provision to store a huge amount of data if you are not thinking of storing the data in a proper manner then while querying the data from the huge amount, the performance will definitely go down.
Azure Cosmos DB provides you with the concept of partitioning where the items in a container are group into different partitions or you can also call those as unique subsets.

What is Partition Key?

Partition key plays an important role here. It is associated with each of the items inside a container and based on it’s value, the subsets or the partitions are formed. Another important point to remember here is every items inside a specific container have the same partition key.
The partition key is nothing but a JSON property that is responsible for distributing data among different partitions.
Behiend the scene, Partition key only decides where to place which document.
Another important point to remember here is, once you set the partition key, it’s not possible to change the Partition key again.
It is suggested to keep a partition key with many distinct values.

Partition Key Components

Partition key has two components and those are Path and Value.

Logical and Physical Partitions

There are two types of partitions related to Azure Cosmos DB partitioning and those are as below.

Logical Partition
Physical Partition

Logical Partition

The items present inside the containers are divided into logical partitions.
Logical partitions are again based on the Partition key which is associated with all the items inside the container.
Partition key value is same for all the items that are present in a logical partition.
As of now, a logical partition can have a limit to 20 GB.

Physical Partition

If you are thinking of the physical partitions, actually, behind the scene, these logical partitions are mapped to physical partitions and basically one or more than one logical partitions are mapped to one physical partition.
As of now, each physical partition can store up to 50 GB of data and can able to provide throughput up to 10,000 RU/s.
Once the physical partition limit reaches 50 GB, immediately, Azure Cosmos Db creates another brand new physical partition.

How Does Partition Work exactly?

Initially, one default partition gets created by Azure.
Then, when you are trying to insert a new document, based on the Partition key-value, Azure cosmos DB decides in which partition, it has to store the item.
In case, there are more items and it reached the maximum size of the partition, immediately, Azure creates a brand new physical partition and moves that particular logical partition to that brand new physical partition.
For better query performance, you can provide a partition key.

How to make sure you are choosing the best partition key?

We have already discussed, Partition key plays an important role here. Choosing the right partition key is one of the major concerns. Below are a few key suggestions that we should consider while choosing the best partition.

The best partition key is the one whose value doesn’t change.
The partition key must contains a wide range of distinct values.
It must have spread throughput across logical partitions.
As a filter, it should appear frequently in your queries for the best performance.
Don’t choose a partition key which doesn’t have a wide range of distinct values else the performance will go down.
For Example, if you are developing an application to host the Employee details. In that case, you can choose the employeeId as the partition key. Then you can map all the data related to particular employee to the employeeId partition key.
In case, you are trying to build a multi tenant application, in that case the best suggestion is to use the tenant-Id as the partition key.

How to create a large partition key in Azure Portal?

While creating a new container, you can create a large partition key in Azure Portal. Follow the below instruction.

Log in to the Azure Portal and navigate to the Azure Cosmos DB account.
Click on + Add container to create the new container.
On the New Container window, Fill all the details and then to create the large partition key, Make sure to select “My partition key is larger than 100 bytes” option.

If you don’t want to create a large partition key then make sure to uncheck the “My partition key is larger than 100 bytes” option.

How to create a large partition key in Azure Portal

create a large partition key using Azure Portal

4. Finally, click on the Ok button to create the container with the large partition key.

Cosmos DB Physical Partition Size Limit

Each physical partition can store up to 50GB of data as of now.

Cosmos DB Partition key Best Practices

Below are a few lists of best practices that you need to consider while choosing the partition key in Azure Cosmos DB.

An item ID in a container is one of the best choice as the partition key because it is unique and has a wide range of possible values and no chance of duplication.
You should choose a partition key that must contain a wide range of distinct values.
The value of the partition key shouldn’t change.
It should have spread throughput across logical partitions.
Choose the one that can be used with your queries as filter condition.
Primary key also can be used as one of the best choice for the Partition key.

Cosmos DB Multiple Partition Keys

An important point to consider here is, Azure Cosmos DB container can have only one partition key.

What is Synthetic Partition Key?

You can concatenate multiple property values to your Partition key and that is known as Synthetic Partition Key.

Let’s consider an example to understand more clearly what is Synthetic Partition Key exactly?

In the below example, You can use either use /productid or /date as the partition key. So you can partition your container based on either productid or date.

{
"productId": "xyz-328",
"date": 2020
}

Now, you can combine both productid or date properties and create one synthetic partition key as below.


{
"productId": "xyz-328",
"date": 2020
"partitionKey": "xyz-328-2020"
}

Cosmos DB Change Partition Key

You might be thinking Can you change the partition key in Cosmos DB? or How to change your partition key in Azure Cosmos DB.

The answer to your question is, technically, it is impossible to update your partition key in the existing container.

In case of migration meaing if you are trying to migrate from source to destination, If the existing collection has no dedicated partition key, you can create a new collection and you can select a different partition key at the destination.

You may also like following the below articles

CRUD Operations In Azure Cosmos DB C#

Wrapping Up

In this article, we have discussed All About Partitioning In Azure Cosmos DB, and along with that we have discussed the below topics.

What is Partition Key?
Partition Key Components
Logical and Physical Partitions
How to create a large partition key in Azure Portal?
How Does Partition Work exactly?
How to make sure you are choosing the best partition key?
Cosmos DB Physical Partition Size Limit
Cosmos DB Partition key Best Practices
What is Synthetic Partition Key?
Cosmos DB Change Partition Key

Hope you have enjoyed this article !!!

Rajkishore

I am Rajkishore, and I have over 14 years of experience in Microsoft Azure and AWS, with good experience in Azure Functions, Storage, Virtual Machine, Logic Apps, PowerShell Commands, CLI Commands, Machine Learning, AI, Azure Cognitive Services, DevOps, etc. Not only that, I do have good real-time experience in designing and developing cloud-native data integrations on Azure or AWS, etc. I hope you will learn from these practical Azure tutorials. Read more.