Difference between revisions of "Corpus Create Annotation Schema By NER Lexical Resources"

From Anote2Wiki
Jump to: navigation, search
(→‎Rule: Partial Match with Dictionaries)
 
(36 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
__TOC__
 
[[Category:HOWTOs]]
 
[[Category:HOWTOs]]
  
 +
== Select Option ==
  
−
When there are one or more Corpus available in a clipboard, it is possible to execute an entity name recognition (NER).  '''Named Entity Recognition whit Lexical Resources''' is a native operation over Corpus(right clicking it - Clipboard).  
+
To perform a new NER (Entity Recognition) based in Lexical Resources, start by [[Corpora_Load_Corpus|loading a Corpus to the Clipboard]].  
  
−
Corpus -> NER -> Lexical Resources
+
Selecting the Corpus, you should right click over it and choose  '''Corpus -> NER -> Lexical Resources'''
  
−
[[Image:Corpus_Process_NER_ANote.png|center]]
 
  
 +
[[Image:Corpus_Process_NER_ANote.png|1500px|center]]
  
−
A wizard will be presented. This allows to configure the NER process. The first step is to select the A new NER Process Configuration or apply a older one . Click on ''new NER Process'' an continuing '''Next button'''.
 
  
−
[[Image:NER_ANote_Wizard1.png|center]]
+
== New Configuration or Load Configuration==
  
 +
A wizard will be presented to configure the process.
 +
The first step allows to select two options: Create a new process (''New Configuration'') or ''Load Configuration'' from an NER process that was already performed.
 +
To start a new process select '''New Configuration''' and press the '''Next''' button.
  
−
In the next step, a lexical resources must be selected for the NER. Here, dictionaries, lookup tables or rule set can be imported for process. After the lexical resources has been chosen continuing '''Next button'''.
 
  
 +
[[Image:NER_ANote_Wizard1.png|800px|center]]
  
−
[[Image:NER_ANote_Wizard1a.png|center]]
 
  
 +
== Resources Selection ==
  
 +
In the next panel, '''select lexical resources'''. Here, dictionaries, lookup tables, rules sets and ontologies can be added to be used in the NER process.
 +
Selecting the respective tabs the user can select from existing resources organized by their types.
 +
When all lexical resources are selected, press '''Next'''.
  
−
[[Image:NER_ANote_Wizard1b.png|center]]
 
  
−
In the next step, For each lexical resource the user must select the classes that pretend.
+
[[Image:NER_ANote_Wizard1a.png|800px|center]]
  
−
[[Image:NER_ANote_Wizard2.png|center]]
 
  
−
After all the configurations have been made, the '''Ok button''' has to be pressed. When the button is pressed, the NER operation will start and a small window will appear, indicating the execution of the operation. The NER operation will take a few minutes or hours.
+
[[Image:NER_ANote_Wizard1b.png|800px|center]]
  
−
When the process is finished, a new '''Process''' object will be added to the Corpus Process View.
+
 
 +
=== Rule: Partial Match with Dictionaries ===
 +
 
 +
Using Rules, it is possible to associate some Rule annotations to Dictionary Terms (including only the Dictionaries selected in this step).
 +
For that purpose, you need to select the option Partial Match with Dictionaries on the Rules Tab as shown in the following figure.
 +
 
 +
 
 +
[[File:RulesMatcing.png|center|800px]]
 +
 
 +
 
 +
<pre>
 +
'''Example'''
 +
 
 +
Dictionary Terms:
 +
relA Gene [List Synonyms][List External Ids]
 +
relB Gene [List Synonyms][List External Ids]
 +
ppGpp Coumpound [List Synonyms][List External Ids]
 +
 
 +
Rule:
 +
 
 +
AA(.*)?\b
 +
 
 +
If the rule is applied to to the following text segment:
 +
 
 +
A AArelA gene for some organism interacts with AArelB.
 +
 
 +
'''''Results'''´´
 +
* Without using this option
 +
    relA ( Annotated by Partial Match Rule)
 +
    relB ( Annotated by Partial Match Rule)
 +
 
 +
* Using Partial Match with Dictionaries
 +
    relA ( Annotated by Partial Match Rule) + associated with dictionary term relA (List Synonyms and External Ids)
 +
    relB ( Annotated by Partial Match Rule) + associated with dictionary term relA (List Synonyms and External Ids)
 +
 
 +
</pre>
 +
 
 +
== Select Class and Case Sensitivity ==
 +
 
 +
In the next panel, for each '''lexical resource''' previously selected you can '''filter for classes''', i.e. select which classes will be associated to each resource.
 +
At this step, you can also define if the process will be case sensitive or not.
 +
 
 +
 
 +
[[Image:NER_ANote_Wizard2.png|800px|center]]
 +
 
 +
 
 +
== Pre-Processing ==
 +
 
 +
The Pre-Processing option allows you to define restrictions for annotations generated by the NER, as follows:
 +
 
 +
Selecting option '''''No''''' the processing continues without any Pre-Processing.
 +
 
 +
 
 +
[[Image:NER_ANote_Wizard3_Without_Pre_processing.png|800px|center]]
 +
 
 +
 
 +
=== Stop Words ===
 +
 
 +
You can select a list of stop words (Lexical Words Set - Lexical Resources) to use in the NER algorithm.
 +
Stop words are important to prevent the algorithm from annotating common English words as entities and thus reduce false positive annotations.
 +
To activate this pre-processing option, select the '''''Stop Words''''' option and in the panel below select one Lexical Words Resource previously created.
 +
To continue with the NER configuration select  '''Next'''
 +
 
 +
 
 +
[[Image:NER_ANote_Wizard3_StopWords.png|800px|center]]
 +
 
 +
 
 +
=== POS-Tagging ===
 +
 
 +
You can filter NER annotations by restricting the annotations to specific grammatical classes (nouns,adjectives,verbs,pronouns, etc).
 +
To select this pre-processing option select '''''POS-Tagging''''' an in the panel below select the tags representing classes to be considered in the annotation.
 +
 
 +
 
 +
[[Image:NER_ANote_Wizard3_POS_Tagging.png|800px|center]]
 +
 
 +
== Normalization ==
 +
 
 +
In the last panel, you can select a '''Normalization option''' that allow to add white space between words delimiters and increase the entity recognition accuracy.
 +
This option changes the original offsets of the text.
 +
 
 +
 
 +
[[Image:NER_ANote_Wizard4.png|800px|center]]
 +
 
 +
 
 +
The configuration is now complete once you press '''Ok'''. At all times you can cancel the operation clicking on the respective button.
 +
 
 +
== Performing ==
 +
 
 +
NER operation will now start and a progress window will appear, indicating the execution of the operation.
 +
The NER operation will take a few minutes or hours depending on the number of documents, document size and size of the resources.
 +
 
 +
When the process ends, a new '''NER Process''' object will be added to the clipboard and can be visualized through the  [[Corpora_Load_Corpus|''Corpus Process View'']].

Latest revision as of 15:42, 12 June 2013

Select Option

To perform a new NER (Entity Recognition) based in Lexical Resources, start by loading a Corpus to the Clipboard.

Selecting the Corpus, you should right click over it and choose Corpus -> NER -> Lexical Resources


Corpus Process NER ANote.png


New Configuration or Load Configuration

A wizard will be presented to configure the process. The first step allows to select two options: Create a new process (New Configuration) or Load Configuration from an NER process that was already performed. To start a new process select New Configuration and press the Next button.


NER ANote Wizard1.png


Resources Selection

In the next panel, select lexical resources. Here, dictionaries, lookup tables, rules sets and ontologies can be added to be used in the NER process. Selecting the respective tabs the user can select from existing resources organized by their types. When all lexical resources are selected, press Next.


NER ANote Wizard1a.png


NER ANote Wizard1b.png


Rule: Partial Match with Dictionaries

Using Rules, it is possible to associate some Rule annotations to Dictionary Terms (including only the Dictionaries selected in this step). For that purpose, you need to select the option Partial Match with Dictionaries on the Rules Tab as shown in the following figure.


RulesMatcing.png


'''Example'''

Dictionary Terms:
relA Gene [List Synonyms][List External Ids]
relB Gene [List Synonyms][List External Ids]
ppGpp Coumpound [List Synonyms][List External Ids]

Rule: 

AA(.*)?\b

If the rule is applied to to the following text segment: 

A AArelA gene for some organism interacts with AArelB.

'''''Results'''´´
* Without using this option
    relA ( Annotated by Partial Match Rule)
    relB ( Annotated by Partial Match Rule)

* Using Partial Match with Dictionaries
    relA ( Annotated by Partial Match Rule) + associated with dictionary term relA (List Synonyms and External Ids)
    relB ( Annotated by Partial Match Rule) + associated with dictionary term relA (List Synonyms and External Ids)

Select Class and Case Sensitivity

In the next panel, for each lexical resource previously selected you can filter for classes, i.e. select which classes will be associated to each resource. At this step, you can also define if the process will be case sensitive or not.


NER ANote Wizard2.png


Pre-Processing

The Pre-Processing option allows you to define restrictions for annotations generated by the NER, as follows:

Selecting option No the processing continues without any Pre-Processing.


NER ANote Wizard3 Without Pre processing.png


Stop Words

You can select a list of stop words (Lexical Words Set - Lexical Resources) to use in the NER algorithm. Stop words are important to prevent the algorithm from annotating common English words as entities and thus reduce false positive annotations. To activate this pre-processing option, select the Stop Words option and in the panel below select one Lexical Words Resource previously created. To continue with the NER configuration select Next


NER ANote Wizard3 StopWords.png


POS-Tagging

You can filter NER annotations by restricting the annotations to specific grammatical classes (nouns,adjectives,verbs,pronouns, etc). To select this pre-processing option select POS-Tagging an in the panel below select the tags representing classes to be considered in the annotation.


NER ANote Wizard3 POS Tagging.png

Normalization

In the last panel, you can select a Normalization option that allow to add white space between words delimiters and increase the entity recognition accuracy. This option changes the original offsets of the text.


NER ANote Wizard4.png


The configuration is now complete once you press Ok. At all times you can cancel the operation clicking on the respective button.

Performing

NER operation will now start and a progress window will appear, indicating the execution of the operation. The NER operation will take a few minutes or hours depending on the number of documents, document size and size of the resources.

When the process ends, a new NER Process object will be added to the clipboard and can be visualized through the Corpus Process View.

Retrieved from "http://anote-project.org/wiki/index.php?title=Corpus_Create_Annotation_Schema_By_NER_Lexical_Resources&oldid=2161"