Difference between revisions of "Corpus Create Annotation Schema By NER Lexical Resources"
(→Select Class ( For each resource)) |
Anote2Wiki (talk | contribs) |
||
| Line 3: | Line 3: | ||
== Select Option == | == Select Option == | ||
| − | + | To perform a new NER (Entity Recognition) based in Lexical Resources, start by [[Corpora_Load_Corpus|loading a Corpus to the Clipboard]]. | |
| − | Selecting Corpus, | + | Selecting the Corpus, you should right click over it and choose '''Corpus -> NER -> Lexical Resources''' |
[[Image:Corpus_Process_NER_ANote.png|1500px|center]] | [[Image:Corpus_Process_NER_ANote.png|1500px|center]] | ||
== New Configuration or Load Configuration== | == New Configuration or Load Configuration== | ||
| − | A wizard will be presented. The first allows to select two options: Create a new process ('' New Configuration'') | + | |
| + | A wizard will be presented to configure the process. | ||
| + | The first step allows to select two options: Create a new process (''New Configuration'') or ''Load Configuration'' from an NER process that was already performed. | ||
| + | To start a new process select '''New Configuration''' and press the '''Next''' button. | ||
[[Image:NER_ANote_Wizard1.png|800px|center]] | [[Image:NER_ANote_Wizard1.png|800px|center]] | ||
== Resources Selection == | == Resources Selection == | ||
| − | In the next panel, the user must '''select lexical resources'''. Here, dictionaries, lookup tables, | + | |
| − | + | In the next panel, the user must '''select lexical resources'''. Here, dictionaries, lookup tables, rules sets and ontologies can be added to be used in the NER process. | |
| + | Selecting the respective tabs the user can select from existing resources organized by their types. | ||
| + | When all lexical resources are selected, press '''Next'''. | ||
[[Image:NER_ANote_Wizard1a.png|800px|center]] | [[Image:NER_ANote_Wizard1a.png|800px|center]] | ||
| Line 22: | Line 27: | ||
[[Image:NER_ANote_Wizard1b.png|800px|center]] | [[Image:NER_ANote_Wizard1b.png|800px|center]] | ||
| − | == Select Class | + | == Select Class and Case Sensitivity == |
| − | In the next panel, | + | |
| + | In the next panel, for each '''lexical resource''' previously selected you can '''filter for classes''', i.e. select which classes will be associated to each resource. | ||
| + | At this step, you can also define if the process will be case sensitive or not. | ||
[[Image:NER_ANote_Wizard2.png|800px|center]] | [[Image:NER_ANote_Wizard2.png|800px|center]] | ||
| − | |||
| − | |||
== Pre-Processing == | == Pre-Processing == | ||
| − | Pre-Processing option allows | + | The Pre-Processing option allows you to define restrictions for annotations generated by the NER, as follows: |
| − | |||
| − | |||
| − | Selecting option '''''No''''' the processing | + | Selecting option '''''No''''' the processing continues without any Pre_Processing. |
[[Image:NER_ANote_Wizard3_Without_Pre_processing.png|800px|center]] | [[Image:NER_ANote_Wizard3_Without_Pre_processing.png|800px|center]] | ||
=== Stop Words === | === Stop Words === | ||
| − | The user can select a list of stop words (Lexical Words Set - Lexical Resources) to | + | The user can select a list of stop words (Lexical Words Set - Lexical Resources) to use in the NER algorithm. |
| + | Stop words are important to prevent the algorithm from annotating common English words as entities and thus reduce false positive annotations. | ||
| + | To activate this pre-processing option, select the '''''Stop Words''''' option and in the panel below select one Lexical Words Resource previously created. | ||
| + | To continue with the NER configuration select '''Next''' | ||
[[Image:NER_ANote_Wizard3_StopWords.png|800px|center]] | [[Image:NER_ANote_Wizard3_StopWords.png|800px|center]] | ||
| Line 46: | Line 52: | ||
=== POS-Tagging === | === POS-Tagging === | ||
| − | + | You can filter NER annotations by restricting the annotations to specific grammatical classes (nouns,adjectives,verbs,pronouns, etc). | |
| + | To select this pre-processing option select '''''POS-Tagging''''' an in the panel below select the tags representing classes to be considered in the annotation. | ||
[[Image:NER_ANote_Wizard3_POS_Tagging.png|800px|center]] | [[Image:NER_ANote_Wizard3_POS_Tagging.png|800px|center]] | ||
| Line 52: | Line 59: | ||
== Normalization == | == Normalization == | ||
| − | + | In the last panel, you can select a '''Normalization option''' that allow to add white space between words delimiters and increase the entity recognition accuracy. | |
| + | This option changes the original offsets of the text. | ||
[[Image:NER_ANote_Wizard4.png|800px|center]] | [[Image:NER_ANote_Wizard4.png|800px|center]] | ||
| − | The | + | The configuration is now complete once you press '''Ok'''. At all times you can cancel the operation clicking on the respective button. |
== Performing == | == Performing == | ||
| − | NER operation will start and a | + | NER operation will now start and a progress window will appear, indicating the execution of the operation. |
| − | The NER operation will take a few minutes or hours depending number of documents, document size and | + | The NER operation will take a few minutes or hours depending on the number of documents, document size and size of the resources. |
| − | When process | + | When the process finishes , a new '''NER Process''' object will be added to the [[Corpora_Load_Corpus|''Corpus Process View'']]. |
Revision as of 00:05, 16 January 2013
Contents
Select Option
To perform a new NER (Entity Recognition) based in Lexical Resources, start by loading a Corpus to the Clipboard.
Selecting the Corpus, you should right click over it and choose Corpus -> NER -> Lexical Resources
New Configuration or Load Configuration
A wizard will be presented to configure the process. The first step allows to select two options: Create a new process (New Configuration) or Load Configuration from an NER process that was already performed. To start a new process select New Configuration and press the Next button.
Resources Selection
In the next panel, the user must select lexical resources. Here, dictionaries, lookup tables, rules sets and ontologies can be added to be used in the NER process. Selecting the respective tabs the user can select from existing resources organized by their types. When all lexical resources are selected, press Next.
Select Class and Case Sensitivity
In the next panel, for each lexical resource previously selected you can filter for classes, i.e. select which classes will be associated to each resource. At this step, you can also define if the process will be case sensitive or not.
Pre-Processing
The Pre-Processing option allows you to define restrictions for annotations generated by the NER, as follows:
Selecting option No the processing continues without any Pre_Processing.
Stop Words
The user can select a list of stop words (Lexical Words Set - Lexical Resources) to use in the NER algorithm. Stop words are important to prevent the algorithm from annotating common English words as entities and thus reduce false positive annotations. To activate this pre-processing option, select the Stop Words option and in the panel below select one Lexical Words Resource previously created. To continue with the NER configuration select Next
POS-Tagging
You can filter NER annotations by restricting the annotations to specific grammatical classes (nouns,adjectives,verbs,pronouns, etc). To select this pre-processing option select POS-Tagging an in the panel below select the tags representing classes to be considered in the annotation.
Normalization
In the last panel, you can select a Normalization option that allow to add white space between words delimiters and increase the entity recognition accuracy. This option changes the original offsets of the text.
The configuration is now complete once you press Ok. At all times you can cancel the operation clicking on the respective button.
Performing
NER operation will now start and a progress window will appear, indicating the execution of the operation. The NER operation will take a few minutes or hours depending on the number of documents, document size and size of the resources.
When the process finishes , a new NER Process object will be added to the Corpus Process View.
