Corpus Create Annotation Schema By NER Lexical Resources

Select Option

To perform a new NER (Entity Recognition) based in Lexical Resources, start by loading a Corpus to the Clipboard.

Selecting the Corpus, you should right click over it and choose Corpus -> NER -> Lexical Resources

New Configuration or Load Configuration

A wizard will be presented to configure the process. The first step allows to select two options: Create a new process (New Configuration) or Load Configuration from an NER process that was already performed. To start a new process select New Configuration and press the Next button.

Resources Selection

In the next panel, select lexical resources. Here, dictionaries, lookup tables, rules sets and ontologies can be added to be used in the NER process. Selecting the respective tabs the user can select from existing resources organized by their types. When all lexical resources are selected, press Next.

Rule: Partial Match with Dictionaries

Using Rules, it is possible to associate some Rule annotations to Dictionary Terms (including only the Dictionaries selected in this step). For that purpose, you need to select the option Partial Match with Dictionaries on the Rules Tab as shown in the following figure.

'''Example'''

Dictionary Terms:
relA Gene [List Synonyms][List External Ids]
relB Gene [List Synonyms][List External Ids]
ppGpp Coumpound [List Synonyms][List External Ids]

Rule: 

AA(.*)?\b

If the rule is applied to to the following text segment: 

A AArelA gene for some organism interacts with AArelB.

'''''Results'''´´
* Without using this option
    relA ( Annotated by Partial Match Rule)
    relB ( Annotated by Partial Match Rule)

* Using Partial Match with Dictionaries
    relA ( Annotated by Partial Match Rule) + associated with dictionary term relA (List Synonyms and External Ids)
    relB ( Annotated by Partial Match Rule) + associated with dictionary term relA (List Synonyms and External Ids)

Select Class and Case Sensitivity

In the next panel, for each lexical resource previously selected you can filter for classes, i.e. select which classes will be associated to each resource. At this step, you can also define if the process will be case sensitive or not.

Pre-Processing

The Pre-Processing option allows you to define restrictions for annotations generated by the NER, as follows:

Selecting option No the processing continues without any Pre-Processing.

Stop Words

You can select a list of stop words (Lexical Words Set - Lexical Resources) to use in the NER algorithm. Stop words are important to prevent the algorithm from annotating common English words as entities and thus reduce false positive annotations. To activate this pre-processing option, select the Stop Words option and in the panel below select one Lexical Words Resource previously created. To continue with the NER configuration select Next

POS-Tagging

You can filter NER annotations by restricting the annotations to specific grammatical classes (nouns,adjectives,verbs,pronouns, etc). To select this pre-processing option select POS-Tagging an in the panel below select the tags representing classes to be considered in the annotation.

Normalization

In the last panel, you can select a Normalization option that allow to add white space between words delimiters and increase the entity recognition accuracy. This option changes the original offsets of the text.

The configuration is now complete once you press Ok. At all times you can cancel the operation clicking on the respective button.

Performing

NER operation will now start and a progress window will appear, indicating the execution of the operation. The NER operation will take a few minutes or hours depending on the number of documents, document size and size of the resources.

When the process ends, a new NER Process object will be added to the clipboard and can be visualized through the Corpus Process View.