Standardization of Wikipedia articles according to the lexical constancy of their introductions and body texts

Wikipedia is a prolific encyclopedia of unequal quality. To standardize it, qualitative categories classify certain articles. The length of the articles has been identified as one of the best predictor variables but is an insufficient criterion to standardize the cognitive accessibility of Wikipedia. This research measures the constancy of repetition of the vocabulary between introductions and the body of articles. Our reproducible methodology is largely inspired by supervised classification methods and similarity metrics. The programming interface of Wikipedia and the quanteda software library are exploited to collect and measure two quality categories: the positive category of the featured articles and the negative category of the articles needing rewrite. After idempotence tests, the K and Vm metrics are selected and applied to the texts. A complementary measure is formalized as the difference relating to independent measurements. Models of combinatorial properties are then evaluated. Decision trees give an overview. The performance of aggregated models of each metric is then compared from support vector machines (SVMs). The K and Vm metrics appear as better candidates than the length one to normalize Wikipedia but the metric K appears to be more discriminating.

Please let me know if you are interested in this (kind of) research and/or if you need a communication about it. Thanks for sharing !