Online text entity extraction capability helps applications analyze massive text data

With the development of information technology , Many valuable knowledge hiding is distributed in massive data , It affects the efficiency of people acquiring knowledge , How to deal with the complicated unstructured text data has become a difficult problem .

In recent days, ,HMS Core Machine learning services 6.5.0 Version added Online text entity extraction capability , This ability can detect whether there is a date in the text 、 full name 、 Proper nouns and other entity information , And extract such entities , That is, the ability to automatically process unstructured natural language text data . for example , The application of the film and television industry often produces a large number of film reviews 、 Information, etc , Using the online text entity extraction capability, we can quickly extract structural information , Help build knowledge map , It is easy for users to understand clearly .

Besides , Text entity extraction capability is more used in question and answer system 、 Information index 、 Knowledge map construction and other fields .

Question answering system

Question answering system is an advanced form of information retrieval system , It can be used accurately 、 Simple natural language to answer user's questions . During the implementation of question answering system , Then we need to use the text entity extraction ability to identify the problem and the entity information in the knowledge base , And then through a variety of algorithm models to match the accurate answer .

Information index

Use online text entity extraction capabilities , You can name specific entity information as indexes and hyperlinks . For example, the proper nouns mentioned by users in their comments , You can generate hyperlinks , It is convenient for other users to search and understand relevant contents .

Knowledge map construction

The map of knowledge is made up of entities 、 A data structure consisting of relationships and attributes , That is, a knowledge base with directed graph structure , Text entity extraction capability is the bottom capability in the process of knowledge map construction , It plays an extremely important role . For example, build a music knowledge map , First, we need to extract singers from a large amount of text data 、 song 、 Lyrics 、 Film and television and other related information , Then build the knowledge map .

at present , The online text entity extraction capability of Huawei machine learning service supports a total of person names 、 money 、 Film name 、 Page links 16 Entity categories , It can be applied to different categories according to the actual semantic scenarios App in .

Integration steps

1. The development of preparation

For detailed preparation steps, please refer to Official website of Huawei developer Alliance .

2. Integration and configuration apigateway authentication

be based on apigateway The authentication mechanism of ：

"paths": {"/entityExtract": { "post": { "operationId": "entityExtract","parameters": [{"in": "body",  "name": "req",  "required": true,"schema": { "$ref": "#/definitions/NerEnterReq"  } },  {"name": "X-Request-ID", "in": "header",  "required": true,"type": "string"}, {"name": "X-Package-Name",  "in": "header", "required": true,"type": "string"  }, ……], "responses": {"200": {   "description": "response of 200","schema": { "$ref": "#/definitions/ResponseEntityNerBodyVo"}}}}}

3. Create an online text entity constructor

//  Create a language detector using a custom parameter configuration . MLRemoteNerSetting setting = new MLRemoteNerSetting.Factory()                .setSourceLangCode("zh")                .create();MLRemoteNer ner = MLNerFactory.getInstance().getRemoteNer(setting); Extract text entities .

4. Get text entity extraction online

Asynchronous method sample code ：

ner.asyncEntityExtract(input).addOnSuccessListener(new OnSuccessListener<RemoteNerResultItem[]>() {            @Override            public void onSuccess(RemoteNerResultItem[] remoteNerResults) {                //  Successful processing logic .                if(remoteNerResults != null){                    //  There are identification results                 }else {                    //   The recognition result is null                 }            }        }).addOnFailureListener(new OnFailureListener() {            @Override            public void onFailure(Exception e) {                //  Recognition failed , Get relevant exception information .                try {                    MLException mlException = (MLException) e;                    //  Get error code , Developers can handle error codes , According to the error code for differentiated page prompt .                    int errorCode = mlException.getErrCode();                    //  Get error information , Developers can combine error codes , Fast location problem .                    String errorMessage = mlException.getMessage();                } catch (Exception error) {                    //  Conversion error handling .                }            }        });

Synchronization method sample code ：

try {       RemoteNerResultItem[] remoteNerResults = ner.syncEntityExtract(input);       //  Identify success logic        if(remoteNerResults != null){       //  There are identification results        }else {       //   The recognition result is null         }   } catch (MLException mlException) {       //  Failed processing logic .       //  Get error code , Developers can handle error codes , According to the error code for differentiated page prompt .       int errorCode = mlException.getErrCode();       //  Get error information , Developers can combine error codes , Fast location problem .       String errorMessage = mlException.getMessage();   }

5. After completion , Release resources

if (ner != null) {    ner.stop();}

Learn more >>

visit Official website of Huawei developer Alliance
obtain Development guidance document
Huawei mobile service open source warehouse address ：GitHub、Gitee

Pay attention to our , The first time to understand HMS Core Latest technical information ~