How To Use Named Entity Recognition In Text Analytics

Well, In this article we will discuss How To use Named Entity Recognition In Azure Cognitive Services Text Analytics API but before that, I feel we should know about What is Azure text analytics?.

What is Azure text analytics?

Azure text analytics API is the part of Azure cognitive services that helps you to perform different operations like Key phrase extraction, Language detection, Sentiment analysis, named entity recognition, etc easily. You will get a lot of requirements in these areas, Azure text analytics service can help a lot to quickly achieve these functionalities.

Azure text analytics supports many languages like Greek, Italian, Korean, Polish, Japanese, Russian, Swedish, Spanish, Turkish, Chinese, Danish, Dutch, English, etc.

The Azure Text Analytics API helps you takes the unstructured text and returns a list of entities, with links to
more information on the web. The Azure Text Analytics API supports the Named entity recognition (NER). Now, what if we will discuss a few lines about the Named entity recognition (NER).

What Is Named Entity Recognition (NER)?

Named Entity Recognition (NER) is also called Entity extraction or Entity Chunking or Entity Identification. You can consider the Named Entity Recognition (NER) is the process of identifying and evaluating the key entities or information in a text. An entity can be a keyword or a Key Phrase.

In other words, Named Entity Recognition (NER) is the ability to identify different entities in a text and categories them into different predefined classes.

Named Entity Recognition (NER) is also known as a form of natural language processing (NLP) that is a subfield of AI and responsible for analyzing the natural language i.e any language that has actually developed naturally.

How Named Entity Recognition (NER) Works?

Mainly the Named Entity Recognition (NER) performs the below two steps

Identify a named entity: In this process, the Named Entity Recognition (NER) identifies a word or a number of words that form an entity.
Categorize the entity: Once it identifies the entity using the first step, then it categorizes the entity into different predefined classes like Person, Organization, Time, Location, Event, Product, etc.

There are some use cases of Named Entity Recognition (NER), those are Speed Up the HR activities, Improve Response time of the Customer Support Department, Helps for the Content classification, Provides Better support for Health care, etc.

Named Entity Recognition versions and features

Features of Named Entity Recognition (NER)	Version 3.0	Version 3.1 Preview
Methods available for the single, and batch requests	yes	yes
Expanded entity recognition across different categories	yes	yes
Different endpoints for sending Named Entity Recognition (NER) requests.	yes	yes

Supported Entity Categories

Below are the entity categories that are returned by the Named Entity Recognition (NER). Basically, there are three categories that are

General
PII
Health

When the Endpoint is /entities/recognition/general, the entity category can be Person (Name of person), PersonType (Job types or roles), Location, Organization, Event (Sporting events), Product (Different physical objects of different categories), Skill (A capability, skill, or expertise), Address, PhoneNumber ( US and UE Phone numbers only), Email (Email addresses), URL (URLs to websites), IP, DateTime (Dateranges, Timeranges, Durations, Set, repeated times, etc), Quantity (Numbers, Percentages, Ordinal numbers, Ages, Currencies, Temperature, etc ).

REST API Request

While sending a REST API Request, You must have JSON documents in this format with ID, text, language. Another important point to note down here is Each JSON documents can contain up to 5,120 characters and can have up to 1,000 items (IDs) per collection and the collection needs to be submitted in the request body.

Request endpoints

The request endpoints URL for the Entity linking

https://<your-custom-subdomain>.cognitiveservices.azure.com/text/analytics/v3.1-
preview.2/entities/linking

Endpoint URL for the Named Entity Recognition (NER) (General)

https://<your-custom-subdomain>.cognitiveservices.azure.com/text/analytics/v3.1-
preview.2/entities/recognition/general

Endpoint URL for the Named Entity Recognition (NER) (PII)

https://<your-custom-subdomain>.cognitiveservices.azure.com/text/analytics/v3.1-
preview.2/entities/recognition/pii

To detect the health information in the text, yo can use the Endpoint URL as below

https://<your-custom-subdomain>.cognitiveservices.azure.com/text/analytics/v3.1-
preview.2/entities/recognition/pii?domain=phi

Example of NER request

Below is the example of a Named Entity Recognition (NER) request

{
"documents": [
{
"id": "1",
"language": "en",
"text": "I Love Azure Lessons."
}
]
}

Example 2

{
"documents": [
{
"id": "1",
"language": "en",
"text": "Check out the Azure Blogs at Azure Lessons."
}
]
}

The Azure Cognitive Services Text Analytics API is stateless and No data is stored in your account, and results are returned immediately as the response. All post requests return a JSON formatted response with the IDs and all the identified entity properties.

Example of a PII response

Below is the example of a PII response

{
"documents": [
{
"redactedText": "I Love Azure Lessons, you can contact on the below Email ID and Phone number",
"id": "0",
"entities": [
{
"text": "www.azurelessons.com",
"category": "URL",
"offset": 49,
"length": 20,
"confidenceScore": 0.8
},
{
"text": "8147568357",
"category": "Phone Number",
"offset": 81,
"length": 12,
"confidenceScore": 0.8
},
{
"text": "rajurims86@gmail.com",
"category": "Email",
"offset": 111,
"length": 27,
"confidenceScore": 0.8
}
],
"warnings": []
}
],
"errors": [],
"modelVersion": "2020-11-11"
}

Example of an Entity Linking response

Below is the example of an entity linking response

{
"documents": [
{
"id": "1",
"entities": [
{
"bingId": "f82345b08-106d-2664-7e5a-993u51u4dk7l",
"name": "sun light",
"matches": [
{
"text": "sun light",
"offset": 30,
"length": 9,
"confidenceScore": 0.4
}
],
"language": "en",
"id": "sun light",
"url": "https://en.wikipedia.org/wiki/Sunlight",
"dataSource": "Wikipedia"
},
{
"bingId": "5hyti8-86h1-3d61-7590-d15876e368r",
"name": "sun",
"matches": [
{
"text": "sun",
"offset": 62,
"length": 3,
"confidenceScore": 0.25
}
],
"language": "en",
"id": "Seattle",
"url": "https://en.wikipedia.org/wiki/Sun",
"dataSource": "Wikipedia"
}
],
"warnings": []
}
],
"errors": [],
"modelVersion": "2020-11-11"
}

Now the next step is we will create a new .NET Core application to use Named Entity Recognition In Text Analytics. But we should know the Prerequisites before moving to the actual functionality

Prerequisites

You must have a valid Azure Subscription or Azure Account. If you don’t have an Azure Account till now then Create an Azure Free Account now.
You need to install Visual Studio 2019 on your machine. If you don’t have Visual Studio 2019 installed in your Dev machine, Install Visual Studio 2019 now.
Create Azure Text Analytics Using Azure Portal and get the Key value of the Azure Text Analytics API.

Assuming that you have all the Prerequisites needed for the development activity, the first step is to Create the Azure Text Analytics API in the Azure Portal and copy the Key value of the Azure Text Analytics API.

Create Azure Cognitive Services Text Analytics API In Azure Portal

Follow the below steps to create Azure Cognitive Services Text Analytics API In Azure Portal.

Once you logged in to the Azure Portal, from the left side panel, click on the + Create a Resource button.

Create and configure Azure Text Analytics API

Search for the Text Analytics as show below

Or, For the same option, Once you logged in to the Azure Portal, you can directly Search for the Text Analytics on the global search and click on the search result Text Analytics under the Marketplace.

Check out How to Create Azure Cognitive Services Text Analytics API In Azure Portal for the next steps and Create the Azure Text API.

Assuming you have created the Azure Text Analytics API by following the above article. Now you can able to see below that I have created the Azure Text Analytics API

Develop Azure Cognitive Services Text Analytics API In Azure Portal

Now, as the next step, we need to copy the key1 value and the endpoint URL. To copy the key1 value, you will have to follow the below steps.

Click on the Keys and Endpoint option from the left navigation on the Cognitive services page that you have created above just now. Once you will click on the Keys and Endpoint option, you can able to see the KEY1 and KEY2 value. You can click on the Show Keys button to show the Key 1 and Key 2 values.

If you don’t want to use the existing Key1 or Key2 value then you can click on the Regenerate Key1 and Regenerate Key2 button to generate the new keys that you want to copy.

Now you can click on the copy button next to the Key1 value as highlighted below and keep it in a note pad as we are going to use this Key 1 value in the next steps.

How to detect Language With Azure Cognitive Service

Now the next step is Creating A .NET Core Application Visual Studio 2019.

Create A .NET Core Application Visual Studio 2019

Create a new .NET Core application using Visual Studio 2019.

Assuming that you have installed Visual Studio 2019 in your Dev machine, Let’s start the actual functionality. Follow the below steps to use Named Entity Recognition In Azure Cognitive Services Text Analytics API.

Open Visual Studio 2019 in your Local machine.

Click on the Create a new Project button on the Get started window.

The next step is choose the project template as Console App (.NET Core) and then click on the Next button.

Create A .NET Core Application Visual Studio 2019

On the Configure your new project window, provide the below details

Project Name: Provide a unique name for your Project.
Location: Provide the location in your local machine where you want to save your Project

custom entity extraction with text analytics

Finally, click on the Create button to create the .Net core application. Now the project will get created sucessfully with out any issue.

Now the next step is to add the Nugget package Azure.AI.TextAnalytics to the project. To add the Nugget package to the Project, Follow the below steps

Right click on the Project and then click on the Manage Nugget Packages as shown below

Click on the Browse tab, search for the Azure.AI.TextAnalytics Nugget package and then select the Nugget package and then click on the Install button and then on the Next window, click o the I accept button to accept the license agreement to install the Nugget Package.

How do you use a named entity recognition?

Now, the next step is to Open the program.cs file and add thefollowing using statements

using Azure;
using System;
using System.Globalization;
using Azure.AI.TextAnalytics;

Now the next step is to define to Private variable for the Key and Endpoint as below

 private static readonly AzureKeyCredential credentials = new AzureKeyCredential("<replace-with-your-textanalytics-key-here>");
        private static readonly Uri endpoint = new Uri("<replace-with-your-text-analytics-endpoint-here>");

The main method should contain the below code

static void Main(string[] args)
        {
            var client = new TextAnalyticsClient(endpoint, credentials);
            EntityRecognition(client);
            Console.Write("Press any key to exit.");
            Console.ReadKey();
        }

The next thing is, you need to create a method named as EntityRecognition and add the below lines of code.

 private static void EntityRecognition(TextAnalyticsClient client)
        {
            var response = client.RecognizeEntities("I had an excellent trip to GOA last Year.");
            Console.WriteLine("Named Entities:");
            foreach (var entity in response.Value)
            {
            Console.WriteLine($"\tText: {entity.Text},\tCategory: {entity.Category},\tSub-Category:{ entity.SubCategory}");
            Console.WriteLine($"\t\tScore: {entity.ConfidenceScore:F2},\tLength: {entity.Length},\tOffset:{ entity.Offset}\n");
            }

        }

Now the complete class file will look like below

using Azure;
using Azure.AI.TextAnalytics;
using System;

namespace TestAzureTextAnalytics
{
    class Program
    {
        private static readonly AzureKeyCredential credentials = new AzureKeyCredential("<replace-with-your-textanalytics-key-here>");
        private static readonly Uri endpoint = new Uri("<replace-with-your-text-analytics-endpoint-here>");
        static void Main(string[] args)
        {
            var client = new TextAnalyticsClient(endpoint, credentials);
            EntityRecognition(client);
            Console.Write("Press any key to exit.");
            Console.ReadKey();
        }

        private static void EntityRecognition(TextAnalyticsClient client)
        {
            var response = client.RecognizeEntities("I had an excellent trip to GOA last Year.");
            Console.WriteLine("Named Entities:");
            foreach (var entity in response.Value)
            {
            Console.WriteLine($"\tText: {entity.Text},\tCategory: {entity.Category},\tSub-Category:{ entity.SubCategory}");
            Console.WriteLine($"\t\tScore: {entity.ConfidenceScore:F2},\tLength: {entity.Length},\tOffset:{ entity.Offset}\n");
            }

        }
    }
}

Now, once you will run the project you will get the out put like below

Named Entities:
Text: trip, Category: Event, Sub-Category:
Score: 0.61, Length: 4, Offset: 18
Text: GOA, Category: Location, Sub-Category: GPE
Score: 0.82, Length: 3, Offset: 26
Text: last year, Category: DateTime, Sub-Category: DateRange
Score: 0.80, Length: 9, Offset: 34

You may also like following the below Articles

Wrapping UP

Well, In this article, we have discussed How To Use Named Entity Recognition In Text Analytics, What is Azure text analytics?, What Is Named Entity Recognition (NER)? How Named Entity Recognition (NER) Works? and we have also discussed Create Azure Cognitive Services Text Analytics API In Azure Portal and Create A .NET Core Application Visual Studio 2019. Hope You have enjoyed this article !!!