A Multi-criteria-Based Expert Modeling for Assessing the Textbook Quality

Abstract

This research considers the application of a multi-criteria approach to assess the quality of textbook content. The issues of determining evaluation criteria and organizing the work of experts are brought to the fore, as these factors directly affect the objectivity and efficiency of the evaluation process. The definition of criteria consisted of reviewing common mistakes and comments on the quality of textbooks, forming a preliminary and final list of criteria. The organization of the work of the experts consisted in the development of a platform with the introduction of a multi-criteria approach. As a result, 39 criteria were formulated and proposed, grouped into 10 main sections such as “Structure and organization”, “Content”, “Didactic aspects”, “Methodological aspects”, “Language level”, “Psychological and psycholinguistic aspects”, “Cultural and Value Aspects”, “ICT Integration”, “Differentiation Aspects”, and “Layout and Design”. The work of experts in the developed expert system is described, which helps in making a decision. The implementation of the multi-criteria approach in this expert system is the main contribution of the research work, which helps to improve the process of evaluating textbooks. The paper presents the results of the survey among 44 experts that helped to identify the most significant criteria which affect the evaluation a textbook. The paper presents the results of the survey among 44 experts that helped to identify the most significant criteria which affect the evaluation a textbook. Moreover, the base of allowed errors was formulated for the following types: for illustrations, QR codes, for texts, for the language of presentation, technical errors, content errors, errors for multimedia objects.