They consider the universal strategy for correct answer search discovered by scientists from Brock University to be wrong.
Kazan scientists placed in doubt the development made by physicists from Brock University who had found out that answers in textbooks on Physics and Chemistry comply with statistic Benford`s law. Rational behavior strategy the scientists described in article, preprint of which was posted in the archive of the Cornell University. Briefly this work was outlined in blog Technology Review.
The Russian version was published by mass-media and this caused sharp criticism by Professor of the Department of General Physics, Dmitry Tayursky.
What is this hotly debated strategy about? The authors decided to apply their observations while guessing the answers to tasks from the textbooks on Physics and Chemistry. According to these observations in the majority of realistic data sets the digits that start from 1, 2 and 3 but not from 7, 8 and 9. During the analysis of the real data tests it was discovered that correct answers comply with the Benford`s law. In other words, the correct answers mostly start with 1 than with 9.
Comments by the Head of Department of Computing Physics and Physical Processes Modeling Anatoly Mokshin:
It is obvious that the regular tests set should meet the following criterion: tasks with correct answers should comply with random distribution providing that all tasks are characterized by equal interval of possible value and tasks amount is enough to perform the statistical analysis (more than 100 tasks). So called Benford’s law analyzes the opposite: it calculates the occurrence frequency of the first digit (in this case numbers of the correct tasks). This law is relatively applicable to multiple numbers. That`s why if you use results of this law during test completion and, hence, choose the answers “1”, “1x”, “1xx” and “2”, “2x”, “2xx” and it would be impossible to avoid opting “9”, “9x”, “9xx”.
Professor Tayursky also supports the idea that this search strategy doesn`t work. Break-in attempt of test collection on Physics with the help of Benford`s law gave no results. It is worth mentioning that successful strategy of completing tests can be mostly connected with low quality of tests themselves.
Senior researcher of the Department of Physics of Solid Body, Niazbek Useinov, considers these publications in mass media to be a new way for researchers to show their importance.
The Benford`s law
Benford's Law refers to the frequency distribution of digits in many (but not all) real-life sources of data. In this distribution, the number 1 occurs as the leading digit about 30% of the time, while larger numbers occur in that position less frequently: 9 as the first digit less than 5% of the time. Benford's Law also concerns the expected distribution for digits beyond the first, which approach a uniform distribution.