Difference between revisions of "Evaluate RESchemas"
RRodrigues (talk | contribs) |
Anote2Wiki (talk | contribs) |
||
| Line 4: | Line 4: | ||
== Operation == | == Operation == | ||
| − | + | You can apply an evaluation operation comparing two RESchemas to calculate the annotation recall, precision and f-score. | |
| + | This will compare a RESchema taken as the gold standard (typically from a manually annotated corpus) and an RESchema to compare (typically from running an algorithm). | ||
| + | To start the evaluation process, right click on the RESchema Datatype to be selected as gold standard and select "Evaluate -> NER Schema". | ||
| + | |||
[[File:RESchema Evaluate.png|center|1500px]] | [[File:RESchema Evaluate.png|center|1500px]] | ||
| + | |||
== Select RESchema to compare == | == Select RESchema to compare == | ||
| − | A GUI will be launched to choose RESchema Datatype (in blue) to be compared with the | + | A GUI will be launched to choose the RESchema Datatype (in blue) to be compared with the gold standard selected on clipboard. |
| + | |||
[[File:RESchema Evaluate GUI.png|center|500px]] | [[File:RESchema Evaluate GUI.png|center|500px]] | ||
| − | NOTES: Only | + | NOTES: Only RESchema with the same Normalization process present in the Gold Standard will appear |
== Result == | == Result == | ||
| − | + | An information pop-up with the annotation recall, precision and f-score measures will appear after the evaluation process. | |
| + | |||
[[File:RESchema Evaluate Result.png|center|200px]] | [[File:RESchema Evaluate Result.png|center|200px]] | ||
Revision as of 16:23, 17 November 2014
Operation
You can apply an evaluation operation comparing two RESchemas to calculate the annotation recall, precision and f-score. This will compare a RESchema taken as the gold standard (typically from a manually annotated corpus) and an RESchema to compare (typically from running an algorithm). To start the evaluation process, right click on the RESchema Datatype to be selected as gold standard and select "Evaluate -> NER Schema".
Select RESchema to compare
A GUI will be launched to choose the RESchema Datatype (in blue) to be compared with the gold standard selected on clipboard.
NOTES: Only RESchema with the same Normalization process present in the Gold Standard will appear
Result
An information pop-up with the annotation recall, precision and f-score measures will appear after the evaluation process.
