Online text entity extraction capability helps applications analyze massive text data

With the development of information technology , Many valuable knowledge hiding is distributed in massive data , It affects the efficiency of people acquiring knowledge , How to deal with the complicated unstructured text data has become a difficult problem .

In recent days, ,HMS Core Machine learning services 6.5.0 Version added Online text entity extraction capability , This ability can detect whether there is a date in the text 、 full name 、 Proper nouns and other entity information , And extract such entities , That is, the ability to automatically process unstructured natural language text data . for example , The application of the film and television industry often produces a large number of film reviews 、 Information, etc , Using the online text entity extraction capability, we can quickly extract structural information , Help build knowledge map , It is easy for users to understand clearly .

Besides , Text entity extraction capability is more used in question and answer system 、 Information index 、 Knowledge map construction and other fields .

Question answering system

Question answering system is an advanced form of information retrieval system , It can be used accurately 、 Simple natural language to answer user's questions . During the implementation of question answering system , Then we need to use the text entity extraction ability to identify the problem and the entity information in the knowledge base , And then through a variety of algorithm models to match the accurate answer .

Information index

Use online text entity extraction capabilities , You can name specific entity information as indexes and hyperlinks . For example, the proper nouns mentioned by users in their comments , You can generate hyperlinks , It is convenient for other users to search and understand relevant contents .

Knowledge map construction

The map of knowledge is made up of entities 、 A data structure consisting of relationships and attributes , That is, a knowledge base with directed graph structure , Text entity extraction capability is the bottom capability in the process of knowledge map construction , It plays an extremely important role . For example, build a music knowledge map , First, we need to extract singers from a large amount of text data 、 song 、 Lyrics 、 Film and television and other related information , Then build the knowledge map .

at present , The online text entity extraction capability of Huawei machine learning service supports a total of person names 、 money 、 Film name 、 Page links 16 Entity categories , It can be applied to different categories according to the actual semantic scenarios App in .

Integration steps

1. The development of preparation

For detailed preparation steps, please refer to Official website of Huawei developer Alliance .

2. Integration and configuration apigateway authentication

be based on apigateway The authentication mechanism of ：

"paths": {
"/entityExtract": { "post": { "operationId": "entityExtract",
"parameters": [{"in": "body",  "name": "req",  "required": true,
"schema": { "$ref": "#/definitions/NerEnterReq"  } },  {
"name": "X-Request-ID", "in": "header",  "required": true,
"type": "string"
}, {"name": "X-Package-Name",  "in": "header", "required": true,
"type": "string"  }, ……],
 "responses": {"200": {   "description": "response of 200",
"schema": { "$ref": "#/definitions/ResponseEntityNerBodyVo"}}}}}

3. Create an online text entity constructor

//  Create a language detector using a custom parameter configuration .
 MLRemoteNerSetting setting = new MLRemoteNerSetting.Factory()
                .setSourceLangCode("zh")
                .create();
MLRemoteNer ner = MLNerFactory.getInstance().getRemoteNer(setting);
 Extract text entities .

4. Get text entity extraction online

Asynchronous method sample code ：

ner.asyncEntityExtract(input).addOnSuccessListener(new OnSuccessListener<RemoteNerResultItem[]>() {
            @Override
            public void onSuccess(RemoteNerResultItem[] remoteNerResults) {
                //  Successful processing logic .
                if(remoteNerResults != null){
                    //  There are identification results 
                }else {
                    //   The recognition result is null 
                }
            }
        }).addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(Exception e) {
                //  Recognition failed , Get relevant exception information .
                try {
                    MLException mlException = (MLException) e;
                    //  Get error code , Developers can handle error codes , According to the error code for differentiated page prompt .
                    int errorCode = mlException.getErrCode();
                    //  Get error information , Developers can combine error codes , Fast location problem .
                    String errorMessage = mlException.getMessage();
                } catch (Exception error) {
                    //  Conversion error handling .
                }
            }
        });

Synchronization method sample code ：

try {
       RemoteNerResultItem[] remoteNerResults = ner.syncEntityExtract(input);
       //  Identify success logic 
       if(remoteNerResults != null){
       //  There are identification results 
       }else {
       //   The recognition result is null 
        }
   } catch (MLException mlException) {
       //  Failed processing logic .
       //  Get error code , Developers can handle error codes , According to the error code for differentiated page prompt .
       int errorCode = mlException.getErrCode();
       //  Get error information , Developers can combine error codes , Fast location problem .
       String errorMessage = mlException.getMessage();
   }

5. After completion , Release resources

if (ner != null) {
    ner.stop();
}

Learn more >>

visit Official website of Huawei developer Alliance
obtain Development guidance document
Huawei mobile service open source warehouse address ：GitHub、Gitee

Pay attention to our , The first time to understand HMS Core Latest technical information ~