Experimental results of note classification with different feature generation strategies – Part 3

In this posting, we present our last four strategies that add one by one the different binary local feature types.

Strategy 7 : input + non-weighted global features + a binary local feature (posspage)

Features M. et M. , « Aspects du droit dans marocain » , Droit et Société , , p. .STARTINITIAL POSSPAGE
Feature counts m.:2 et:2 ,:4 «:1 aspects:1 du:1 droit:2 dans:1 marocain:1 »:1 société:1 p.:1 .:1 strartintial:1 posspage:1
Numerical features 9:1 23:4 25:1 34:1 35:1 47:2 67:2 68:1 69:1 70:2 71:1 72:1 73:1 10004:1 10010:1
#Unique features 3939

The local feature ‘POSSPAGE’ is applied. Instead of counting the appearance of this feature in a note string, a binary value is used to mark its existence.

Total accuracy : 91.09

[Positive] Precision : 93.02 Recall : 95.42     [Negative] Precision : 84.31 Recall : 77.48

With this strategy, we obtain a small gain but significant enough for the fact that just one feature type is appended. Encouraged by this result, we continue to add the other types of local feature.

 

Strategy 8 : input + non-weighted global features + binary local features (posspage + weblink)

– Feature table is omitted –

Total accuracy : 91.30

[Positive] Precision : 93.28 Recall : 95.42     [Negative] Precision : 84.47 Recall : 78.38

The local feature ‘WEBLINK’ is applied. We get again a small gain.

 

Strategy 9:  input + non-weighted global features + binary local features (posspage + weblink + posseditor)

 – Feature table is omitted –

 Total accuracy : 91.30

[Positive] Precision : 93.28 Recall : 95.42     [Negative] Precision : 84.47 Recall : 78.38

The local feature ‘POSSEDITOR’ is applied but there is no change. It is perhaps because of the low frequency of this feature in the note data. Anyway we decide to keep this feature for the case of modifying learning data.

 

Strategy 10 : input + non-weighted global features + binary local features (posspage + weblink + posseditor + italic)

Features M. et M. , « Aspects du droit dans marocain » , Droit et Société , , p. .STARTINITIAL ITALIC POSSPAGE
Feature counts m.:2 et:2 ,:4 «:1 aspects:1 du:1 droit:2 dans:1 marocain:1 »:1 société:1 p.:1 .:1 strartintial:1 italic:1 posspage:1
Numerical features 9:1 23:4 25:1 34:1 35:1 47:2 67:2 68:1 69:1 70:2 71:1 72:1 73:1 10001:1 10004:1 10010:1
#Unique features 3942

Total accuracy : 94.78

[Positive] Precision : 95.77 Recall : 97.42     [Negative] Precision : 91.43 Recall : 86.49

In our final strategy, we applied the ‘ITALIC’ binary feature and this brings a large improvement, which is more than 3 points in terms of total accuracy. Especially, we obtain a great increase in both precision and recall of negative note. This result is reasonable because the italic feature usually appears with the title of article that is one of the main contributions of bibliographic reference.

In conclusion, we successfully generate a set of appropriate features that consists of three groups, input feature, local feature and global feature. With the final strategy, we obtain about 95 percent of micro-averaged precision for the note classification. This is 7 point better than the baseline strategy, which uses only input word features but is one of the most used approaches for text classification.



Citer ce billet
Young-Min Kim (2011, 21 septembre). Experimental results of note classification with different feature generation strategies – Part 3. BILBO - Automatic Annotation of Bibliographical References. Consulté le 29 mars 2024, à l’adresse https://doi.org/10.58079/m0gm

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

Ce site utilise Akismet pour réduire les indésirables. En savoir plus sur comment les données de vos commentaires sont utilisées.