Quantitative and qualitative analysis. Methods for qualitative assessment and quantitative measurement of personal and professional development

Expert assessments and measurement methods.

An expert's assessment is a quantitative and/or qualitative measurement of the relevant indicator.

Methods for obtaining quantitative expert estimates

Direct Quantification

The expert indicates a specific numerical value or range of the desired parameter.

midpoint method.

It is used with a large number of alternative options.

First, the most (A) and least (I) preferred alternatives are selected. Next - an intermediate option - M, the assessment of which divides the segment A - Z in half.

Methods for obtaining qualitative expert assessments

Expert classification (determination of belonging to a group, variety, category - tea variety)

Ranking of alternatives - ordering of compared alternatives according to the degree of preference for a certain feature.

The method of expert curves is used to obtain forecasts based on the analysis of the dynamics of indicators characterizing the object of the experiment (graph, extrapolation). To prevent errors in extrapolation, based on the generalized opinion of experts, points are determined on the graph at which a change in the development trend of the indicator should be expected.

The tools of the above methods are qualitative and quantitative scales.

19. Methods for evaluating the qualities of experts and the formation of expert commissions.

court method.

The procedure is similar to a court session. Some of the experts are accusing, some are defending, and some are the jury, rendering the final verdict. Expert assessments are subjective, so the key issue is the selection of experts.

Evaluation of the qualities of an expert.

Expert- a specialist, a professional, whose assessments and judgments the decision maker considers it useful to take into account in the decision-making process.

The formation of the composition of the expert commission depends on:

specific decision-making situation;

Opportunities for the organizers of the examination to attract highly qualified specialists for work;

Opportunities for specialists to participate in the work of the expert commission.

There is no generally accepted unified methodology for assessing the qualities of an expert.

You can rely on:

Professional knowledge;

Experience and efficiency in the composition of expert commissions.

Methods for evaluating the qualities of an expert:

A priori– do not use information about the results of the expert's participation in previous examinations (self-assessment, mutual assessment, expert list method, documentary (questionnaire) method).

a posteriori- use information about the results of the expert's participation in previous examinations (the method of paired comparisons of objects that are most preferable for this expert, the method of deviation from the resulting group assessment).

Test- involves a special test.

The test must be specially designed for specific objects of peer review

The true values ​​of the estimated parameters (correct answers) should not be known to the expert

A scale should be developed to determine the accuracy of the assessments given by the expert.

The probability of randomly guessing the correct estimate should be very small.

The use of test methods makes it possible to evaluate such an important professional quality of an expert as the reproducibility of expert assessments. Conduct a series of similar tests and evaluate the results. The stability of the assessments indicates the professional competence of the expert.

Qualitative Methods(ethnographic, historical research as methods of qualitative analysis of local micro-societies, case study method, biographical method, narrative (narrative) method) - semantic interpretation of data. When using qualitative methods, there is no link between formalized mathematical operations between the stage of obtaining primary data and the stage of meaningful analysis. These are widely known and applied methods of statistical data processing.

However, qualitative methods include certain quantitative methods of collecting and processing information: content analysis; observation; interviewing, etc.

When making important decisions, a so-called “decision tree” or “goal tree” is used to select the best course of action from the available options, which is a schematic description of a decision-making problem. Structural schemes of goals can be represented in tabular and graphical ways. The graph method has a number of advantages over the tabular one: firstly, it allows you to record and process information in the most economical way, secondly, you can quickly create a development algorithm, and thirdly, the graph method is very visual. The goal tree serves as a basis for choosing the most preferred alternatives, as well as for assessing the state of the systems being developed and their interconnections.

Other methods of qualitative analysis are constructed similarly, including analogues of quantitative methods of factor analysis.

As rightly noted by D.S. Klementiev (21), the effect of qualitative methods of sociological research is possible only if ethical norms dominate in reflecting social factors. A sociologist, selecting information from a mass of all kinds of information, should not be limited only by his own preferences. In addition, when trying to answer the question about the actual state of affairs in the management environment, collecting specific information - empirical data, referring to the properties of the phenomenon under study, the sociologist should not operate with the generally accepted provisions of "common sense", "ordinary logic" or an appeal to the works of religious and political authorities. When compiling tests, the sociologist must avoid distortions that reflect not so much control as manipulation. And another fundamental norm for a sociologist is honesty. This means that a person, presenting the results of the study, even if they do not satisfy him, should neither hide nor embellish anything. The requirement of honesty also includes the provision of full documentation relevant to the case. You must be responsible for all information used by others to critically evaluate research methods and results. This is especially important to keep in mind in order to avoid the temptation to distort the information, which would undermine the credibility of the findings.

Quantitative Methods The study of the quantitative certainty of social phenomena and processes occurs with the use of specific means and methods. These are observation (not included and included), survey (conversation, questioning and interviewing), document analysis (quantitative), experiment (controlled and uncontrolled).

Observation as a classical method of natural sciences is a specially organized perception of the object under study. The organization of observation includes the determination of the characteristics of the object, the goals and objectives of observation, the choice of the type of observation, the development of a program and observation procedure, the establishment of observation parameters, the development of techniques for the implementation of results, the analysis of results and conclusions. When observation is not included, the interaction between the observer and the object of study (for example, a control system) is reduced to a minimum. When enabled, the observer enters the observed process as a participant, i.e. achieves maximum interaction with the object of observation, usually not revealing his research intentions in practice. In practice, observation is most often used in combination with other research methods.

Polls are selective and selective. If the survey is conducted covering the entire population of respondents (all members of a social organization, for example), it is called continuous. The basis of a sample survey is a sample population as a reduced copy of the general population. The general population is considered to be the entire population or that part of it that the sociologist intends to study. Sample - a set of people whom the sociologist interviews (22).

The survey can be conducted using a questionnaire or interview. Interview- is a formalized type of conversation. Interviews, in turn, are standardized, non-standardized. Sometimes they resort to telephone interviews. The person who conducts the interview is called the interviewer.

Questionnaire- Written survey. Like an interview, a survey involves a set of clearly formulated questions that are offered to the respondent in writing. Questions can be free-form (open-ended) or pre-set (closed-ended) where the respondent selects one of the suggested response options (23).

Questioning, due to its features, has a number of advantages over other survey methods: it reduces the time for registering respondents' answers due to self-counting; formalization of responses creates the possibility of using mechanized and automated processing of questionnaires; thanks to anonymity, it is possible to achieve sincerity in the answers.

In order to further develop the questionnaire, it is often used scale method applied. The method is aimed at obtaining quantitative information by measuring the attitude of specialists to the subject of examination on a particular scale - nominal, rank, metric. The construction of a rating scale that adequately measures the phenomena under study is a very difficult task, but the processing of the results of such an examination, carried out using mathematical methods using the apparatus of mathematical statistics, can provide valuable analytical information in quantitative terms.

Analysis method documents allows you to quickly obtain factual data about the object under study.

Formalized analysis documentary sources (content analysis), designed to extract sociological information from large arrays of documentary sources that are inaccessible to traditional intuitive analysis, is based on identifying some quantitative characteristics of texts (or messages). At the same time, it is assumed that the quantitative characteristics of the content of documents reflect the essential features of the studied phenomena and processes.

Having established the quantitative influence of the studied factors on the process under study, it is possible to build a probabilistic model of the relationship of these factors. In these models, the facts under study will act as a function, and the factors that determine it, in the form of arguments. Giving a certain value to these factors-arguments, a certain value of the functions is obtained. However, these values ​​will be true only with a certain degree of probability. To obtain a specific numerical value of the parameters in this model, it is necessary to process the questionnaire survey data in an appropriate way and build a multivariate correlation model on its basis.

Experiment as well as the method of questioning, it is a test, but unlike the first, it aims to prove one or another assumption, hypothesis. An experiment, therefore, is a one-time test for a given pattern of behavior (thinking, phenomena).

Experiments can be carried out in various forms. There are mental and "natural" experiments, dividing the latter into laboratory and field. A thought experiment is a special technology for interpreting the information received about the object under study, which excludes the intervention of the researcher in the processes occurring in the object. Methodologically, the sociological experiment is based on the concept of social determinism. In the system of variables, an experimental factor is singled out, otherwise denoted by an independent variable.

An experimental study of social forms is carried out in the course of their functioning, therefore, it becomes possible to solve such problems that are inaccessible to other methods. In particular, the experiment allows us to explore how the connections of a social phenomenon with management can be combined. It allows you to study not only individual aspects of social phenomena, but the totality of social ties and relationships. Finally, the experiment makes it possible to study the entire set of reactions of a social subject to a change in the conditions of activity (a reaction expressed in a change in the results of an activity, its nature, relationships between people, in a change in their assessments, behavior, etc.). Those changes that are produced in the course of the experiment may represent either the creation of fundamentally new social forms, or a more or less significant modification of existing ones. In all cases, the experiment is a practical transformation of a certain area of ​​control.

In general, the algorithmic nature of the quantitative method in a number of cases makes it possible to come to the adoption of “accurate” and reasonable decisions to a high degree, or at least to simplify the problem, reducing it to a step-by-step solution to some set of simpler problems.

The end result of any sociological research is the definition and explanation of patterns and the construction on this basis of a scientific theory that allows predicting future phenomena and developing practical recommendations.

Issues for discussion

1. What is the method of sociology of management?

2. What is the specificity of the methods of sociology of management?

3. List the classifications of management sociology methods known to you?

4. What is the difference between qualitative and quantitative sociological research methods?

5. Determine the essence of the interview, questionnaire, scaled assessment method, etc.

21 Klementiev D.S. Sociology of Management: Proc. allowance. - 3rd ed., revised. and additional - M.: Publishing House of Moscow State University, 2010. - P.124

22 Yadov V.A. Sociological research: Methodology, program, methods. - M., 1987. - S. 22-28.

23 Ilyin G.L. Sociology and psychology of management: a textbook for students. higher textbook institutions / G.L. Ilyin. - 3rd ed., erased. - M: Publishing Center "Academy", 2010. - S. 19.

Methods for measuring and evaluating personal and professional development

Plan

1. Quantitative and qualitative measurements.

2. The main directions for measuring personal and professional development.

3. Contradictions of personal and professional development.

4. Personal and professional development and its monitoring

Keywords: qualitative assessment, quantitative measurement, personal development, professional development, monitoring.

– qualitative assessment implies a non-numeric (given at the level of measurements of the name or order) assessment of significant personal and professional qualities;

– quantitative measurement produced on the basis of the development of special numerical scales (at the level of measurements of intervals or ratios) to assess the severity of significant personal and professional qualities;

– personal development in a narrow sense, this is the actual level of expression of significant personality traits as an integral systemic entity. At the same time, for various practical tasks, different in their generalization and level of personality quality turn out to be significant; in a broad sense, the dynamics of changes in significant personal qualities in a different time perspective. Methodically, to study personal development in a broad sense, the procedure for monitoring personal development is most often used;

– Professional Development- in a narrow sense - this is the actual level of expression of personality traits that are significantly related to her professional success; in a broad sense, this is a change over a certain period of time (for example, between two certifications) of personality traits associated with professional success. To assess such development, a professional development monitoring procedure may be applied;

– monitoring is a systematic procedure for comparable measurement of the level of expression of significant personal and professional qualities, carried out at regular intervals. It allows you to evaluate the dynamics of changes in the evaluated qualities.

Quantitative and qualitative measurements

Qualitative assessment and quantitative measurement in psychological research and practical applications, from a formal point of view, can be considered as special cases of a more general measurement procedure. As a first approximation, assigning numerical values ​​to represent properties can be called measurement . As a result of the measurement, in each case, a numeric value is associated with the measured property. However, this attribution is not arbitrary. It is necessary that the relationships between numerical values ​​correspond to those relationships that actually exist between the aspects of the object of measurement they measure. For the convenience of researchers, it is customary to distinguish four types of such correspondences, which are called measurement levels. There are the following levels of measurement (or scales for measurements):

Items (nominal);

Ordinal (ordinal);

interval;

Relations.

The first two levels of measurement are traditionally considered as a qualitative assessment, and the last two together form what is called the quantitative measurement of properties.

The higher the level, the greater the set of properties "useful" for the researcher and practitioner has the corresponding measuring scale. From this point of view, the lowest level of measurement is the nominal level, and the highest level is the relationship level. Traditionally, the development of measuring tools (for example, scales in psychological tests) goes from simple to complex. In terms of levels of measurement, this means that from the initial creation of measuring instruments operating at a nominal level, there is a gradual transition, as the features of the studied properties and their relationships with other properties are learned, first to the ordinal level of measurement, and then to the quantitative level of measurement, embodied in point scales. In practice, this trend is opposed by the ever-increasing complexity of measurement techniques, leading to an increase in the cost of their use and increased requirements for specialists using them.

Let us consider in more detail the features of the above measurement levels and specific examples of their use to assess the personal and professional qualities of employees.

Nominal dimension formalizes the classification procedure, but at the same time it has the smallest set of useful properties, giving only a set of discrete categories that make it possible to distinguish between different objects.

The simplest examples of the nominal level of measurement are related to the gender, nationality, party affiliation of the person being assessed.

In order to assess personnel, it is widely used matrix method, involving a comparison, for example, of a set of real and required qualities. The result of such a comparison is nominal in terms of the level of measurement and contains the assessments "corresponds - does not correspond". Note that there are more complex versions of this method that allow obtaining quantitative estimates of employees.

Execution Method- a technique that provides a qualitative description of what an employee has done for a certain period. It also allows you to get an assessment of activity at a nominal level (in terms of "did - did not do"). In addition, the method is sometimes used critical case(method incidents), within the framework of which an analysis of the behavior of the person being assessed in difficult or critical situations is carried out, and his failures in work can also be considered.

Ordinal dimension allows not only to break objects into classes, but also to arrange them. The most famous example of such a scale is the school mark (from 1 to 5).

To assess a latent one-dimensional property using an ordinal scale based on a set of external manifestations (which may or may not occur in relation to a particular object), a special algorithm for constructing the corresponding scales, called the Guttman scale*, has been developed. The prevalence of ordinal measurements for assessing professional development is illustrated by the traditional outcome of attestation procedures “corresponds to the position held” - “corresponds to the position held, subject to improvement in work” - “change of activity is recommended”, forming a typical ordinal scale.

* Yadov V.A. Sociological research. Methodology, program, methods. M., 1987. S.98-102.

Along with this, due to its simplicity, it is quite common, especially when assessing small work teams (up to 10 people), to classify in order, providing for the selection of the best and worst employees.

To do this, experts place employees according to each of the evaluated criteria (there may be several) in order, from the best to the worst. An individual final score can be defined as the sum of the rankings for all the evaluation criteria used. Thanks to this summation (although not entirely correct from the point of view of measurement theory), it becomes possible to move from ordinal to interval scales and give a final score in points.

Interval measurement- allows not only to classify or arrange objects, but also to say how much more or less of a measured property they are characterized in comparison with other objects. Measurement is based on the existence of some standard unit of measurement.

For example, income can be measured in rubles per year. The unit of measurement is the ruble. Another example is temperature measurement. There are different measurement systems (Celsius, Fahrenheit, Kelvin). Almost all standardized psychological tests used to assess professional and personal development result in scores, which allows them to be attributed to the interval level of measurement.

Very many interval scales are "natural". These are scales of age, income, weight. In addition, there is a technology for developing interval scales used to measure a latent indicator by its externally observable manifestations. Thurstone's Equal Interest Scale is the best known.

Equal interval scale Thurstone is used to determine the subjective attitude (often not explicitly realized) to a certain phenomenon, for example, attitude to work, based on a number of external signs. For this, the following steps are performed.

1. Develop at least 30 judgments of a positive and negative nature (these are external signs). For example, “work is the first vital necessity”, “you can’t even pull a fish out of a pond without difficulty”, “work is not a wolf, it won’t run away into the forest”, etc.

2. Select a group of experts (at least 50 people randomly selected from the study audience).

3. Each of the experts must sort the entire list of judgments into 11 classes. The number of classes may be different, it depends on the "fractionality" of the scale required by the researcher, which determines its accuracy. Class 1 includes judgments with the most positive attitude towards the object, class 11 - with the most negative attitude, and class 6 - with a neutral attitude.

4. Analysis of expertise is carried out:

the price (weight) of each judgment is determined on a scale of 11 intervals. So, if the opinions of 300 expert judges were used to determine the weight, then for one statement, for example, "work is not a wolf, it will not run into the forest," the marks can be distributed as follows (see Table 1).

Any changes in the conditions or results of the organization (industrial enterprise, trading company or bank), associated with one or another alternative management decision, can only be determined when there are corresponding indicators that can be measured. But we are talking not only about quantitative measurements in the usual sense for us, but also about qualitative assessments that make it possible to judge the dynamics of ongoing changes. Therefore, speaking of expert assessments, we understand them as quantitative or qualitative measurements of the relevant indicators.

Depending on the objectives of the examination, experts can assess the value of real estate, the expected level of inflation, changes in the exchange rate, the reliability of the issuer of the security, the rating of the bank, etc. The objects of assessment and the estimated indicators are very diverse. If at the same time the expert must measure the value of a quantitative indicator (give a quantitative estimate), he can indicate a specific numerical value or an interval in which it lies. If the expert must determine the value of an indicator that can only be assessed qualitatively (give a qualitative assessment), then he can use verbal-numerical scales for this.

Thus, expert information can be both quantitative and qualitative. Consider the most common methods for measuring it separately for each of these two cases.

Methods for obtaining quantitative expert estimates

Direct quantification. The expert directly indicates the numerical value of the indicator for the assessed object (alternative), for example, the cost of an apartment or land plot, the estimated market capacity, etc. If it is difficult to indicate a specific value of an indicator, the expert can determine the range in which this value is located.

midpoint method. This method is used when there are many alternatives to be evaluated. At the beginning, the expert quantifies the most preferable alternative - f(а1) and the least preferable - f(а2). Next, he must choose the third alternative a3, the estimate of which f(a3) is located in the middle between the values ​​of f(a1) and f(a2) and is equal to half of their sum. Then the expert determines the estimate of the alternative a4, the value of which - f(a4) - is located in the middle between the values ​​of f(a1) and f(a3), and the alternative a5 - f(a5), the value of which is the average value between f(a3) and f (a4), and so on, until all compared alternatives have been evaluated.

Methods for obtaining qualitative expert assessments

Method of expert classification. This method is appropriate to use when it is necessary to determine whether the alternatives being evaluated belong to established and accepted classes, categories, levels, varieties, etc., for example, the assessment of tea varieties by tasters or the definition of a hotel category. If the classes are set in advance, then the expert is sequentially presented with alternative options and he determines to which class this alternative belongs. If the classes are not established, then the expert is presented with alternative options in pairs and asked to determine whether they belong to the same class or to different ones. After that, the alternatives are offered for evaluation sequentially, and it turns out whether each of them can be attributed to one of the resulting classes, or whether a new class should be formed for it. The procedure is repeated until all alternatives are evaluated (classified).

Method of ranking alternatives. The essence of this method is to arrange the compared alternatives according to the degree of preference for a certain feature. If there are relatively few such alternatives (no more than 20), then the expert is presented with the entire set of alternative options, and he indicates the most preferable among them. Then the expert determines the most preferred option from the remaining ones. And so on, until all evaluated alternatives are ranked. If there are more options to be evaluated, then first they are divided into groups ordered by preference using the expert classification method, and then ranked within each group.

Method of expert curves. This method is used when the purpose of the examination is to develop forecasts based on an analysis of the dynamics of changes in indicators characterizing the object of examination, for example, GDP growth rates, inflation, unemployment, etc. The use of this method involves the construction, based on the available statistical data, of a graph that displays the trend in the dynamics of the analyzed indicator. Then, with the help of extrapolation, this trend continues into the future, which makes it possible to assess the value of this indicator in the forecast period. However, the use of extrapolation can lead to significant errors, because does not take into account the possibility of qualitative changes leading to a change in trend. Therefore, further, based on the generalized opinion of experts, the points are determined on the graph at which a change in the trend of the estimated indicator should be expected, which makes it possible to visualize possible alternative scenarios for the development of the situation.

Two types of scales correspond to the two groups of methods for obtaining expert assessments considered above:

scales of quantitative measurements;

scales of qualitative measurements.

quantitative scales

Absolute scale. If it is required to determine the number of seats in the audience or the number of workers required to perform a certain operation, then the number resulting from such measurements is uniquely determined and is unique. This is the feature of the absolute scale.

Relationship scale. Assumes the ability to make measurements in various systems of units. So, for example, if we evaluate the mass of an object in kilograms, we get one numerical value, in pounds - another, and so on. However, no matter what system of measurements is used, the ratio of the masses of any two objects will be the same and does not change when moving from one system to another. The same is observed when measuring the length of objects. These are typical examples of a ratio scale in which the ratios of numerical scores remain unchanged.

Interval scale. In the scale of intervals, when moving from one system of measurements to another, the ratios of the numerical values ​​of the evaluated alternatives are not preserved, but the ratio of the differences in the numerical assessments is preserved. An example of measurements in the interval scale is the determination of temperature, which, as you know, can be measured on different scales: Celsius, Fahrenheit, etc. Each time we will get different numbers, because. different scales use different reference points and different measurement units. But these different numerical estimates correspond to the same temperature of the object.

difference scale. If only the reference point changes when moving from one numerical system of measurement to another, such scales are called scales of differences. An example of quantitative estimates in the scale of differences is the systems of chronology, which differ in the starting point (from the birth of Christ, from the creation of the world, or from the resettlement of the prophet Mohammed from Mecca to Medina).

Qualitative measurements are much less rigorous, they are subjective and other types of scales are used for them, the main ones are the following.

Qualitative scales

Nominal scale (name scale). The essence of the measurements of the evaluated alternatives in this scale is their division into classes according to a certain attribute. All alternatives that fall into the same class must correspond to the same number. Examples of measurements in nominal scales can be the division of students of one course into groups, the division of products according to quality levels (grades), etc.

Ordinal scale. This scale is used for ranking, i.e. determining the order of preference for alternatives, the severity of a particular property, for example, importance, urgency, etc.

Quite often, during the examination, it becomes necessary to measure indicators, the assessments of which are obviously subjective. So, for example, the knowledge and experience of specialists are used to assess the degree of risk in the implementation of an investment project, the competitiveness of a certain type of product, the level of competence of a candidate for a vacant position, etc. In these cases, a special type of ordinal scales is used - verbal-numerical scales. Their peculiarity lies precisely in the fact that they allow you to measure the degree of intensity of any property that has a subjective character. The composition of the verbal-numerical scale includes a meaningful (verbal) description of the selected gradations and their corresponding (gradations) numerical values.

The Harrington scale, widely used in practice, shown in Table. 6.1.

Table 6.1.

Harrington scale

The numerical values ​​given in the table were obtained on the basis of a statistical analysis of a large amount of data, due to which the Harrington scale has universal application and can be used in appropriate modifications (for example, in the form of a score scale) to assess various qualitative indicators.

Summing up the study of expert methods, it should be noted that their significance lies in the fact that they enhance the element of collegiality in the process of making complex decisions and, using intuition and collective generation of ideas, allow finding new, original solutions to problems that cannot be reached with the help of only logical reasoning.

At the same time, it is rather difficult to establish clear boundaries between all the considered methods for developing and justifying decisions (modeling methods and expert methods), since the solution of the complex problems of modern management also requires the complex use of various logical, statistical, mathematical and heuristic techniques. Therefore, it is not just one, but the prevailing group of methods that forms this or that method. The areas of application of decision-making methods depend mainly on the nature of the problems being solved and the conditions for decision-making, which is reflected in Fig. 6.1.

conclusions

The essence of expert decision-making methods is to receive answers from specialists to questions posed before them, to process the information received using special logical and mathematical procedures, and to transform it into a form convenient for choosing the most preferred solution alternative.

Among the methods of expert assessments (which differ in the procedure for organizing the examination), the main ones are the commission method, the brainstorming method, the Delphi method and the scenario development method.

In the process of forming an expert commission, the problem of assessing the qualities of an expert arises - the need to take into account his professional knowledge, experience and work results in previous examinations.

Methods for assessing the qualities of an expert are divided into three groups: a priori (not taking into account the effectiveness of his participation in previous examinations); a posteriori (based on the real results of the participation of a specialist in the examination) and test (involving a special test for a potential expert).

Expert information can be both quantitative and qualitative. To obtain quantitative estimates, the main methods are the direct quantification method and the midpoint method. To obtain qualitative estimates, expert classification, ranking of alternatives and the method of expert curves are used. These two groups of methods for obtaining expert assessments also correspond to two types of scales: scales of quantitative measurements and scales of qualitative measurements.

Questions for self-examination

What does the concept of a priori expert evaluation methods include?

In what cases is it customary to use a posteriori methods of estimation?

What are the conditions for applying test methods for assessing the qualities of an expert?

What is the difference between quantitative and qualitative expert assessments?

What are the criteria for constructing qualitative scales?

The criteria used in the formation of the evaluation system must be measurable, i.e. it must be possible to evaluate any object of examination under consideration for each of the criteria.

In cases where the criterion characterizing an object cannot be measured objectively, we speak of subjective criteria, meaning, first of all, that there are no objective criteria to evaluate objects according to such criteria and it is necessary to develop special verbal-numerical scales.

Naturally, such criteria as "production volume", "production cost", "payback period" can be classified as objective. At the same time, such criteria as "goodwill" associated with the evaluation of intellectual property, "image of the company", "social significance of the project" and others can only be measured subjectively.

Many of the objective criteria relating to future periods can also often only be assessed subjectively. For example, the expected volumes of production that will become possible after the implementation of the project, the expected price of a unit of production, etc. often largely depend on expert estimates, which are subjective in the strict sense of the word.

Therefore, professionalism is so necessary in organizing and conducting the process of expert assessment, analyzing and processing the results of expert assessments used in the development and adoption of managerial decisions.

But the first step in the process of expert evaluation is the formation of an adequate evaluation system.

We discussed some of the requirements for the set of criteria included in the evaluation system.

The practical formation of a list of criteria is also largely an expert procedure. These can be 2-3-round examinations, when a previously formulated list of criteria is specified by experts.

When forming a set of criteria, attention should be paid to such points as a clear understanding of the meaning of each of the criteria by decision makers and experts.

Sometimes aggregation of criteria is appropriate. This can achieve both a reduction in the redundancy of criteria, especially in the case when there is a partial duplication of criteria, and a general reduction in the number of criteria, which is important for reducing the complexity of working with the evaluation system.

2. 3. Quantitative and qualitative expert assessments

Let us now dwell on the main methods of expert measurements - methods for obtaining expert assessments, which in many cases play a decisive role in making important management decisions.

Acquisition Methodsquantitative expert assessments

1. Direct quantification

Direct quantitative assessment is used both in the case when it is necessary to determine the value of the indicator measured quantitatively, and in the case when it is necessary to assess the degree of comparative preference of various objects.

In the first case, each of the experts directly indicates the value of the indicator for the assessed object. This may be a specific numerical value of the indicator for the assessed object, for example, the cost of a residential apartment; the price of a unit of production at which it can have a competitive demand; estimated market capacity; optimal production volume, etc.

If an expert finds it difficult to indicate a specific value of an indicator, he can indicate the range in which the value of the indicator being evaluated lies.

In the second case, when the relative preference of objects is assessed by one or another indicator, the quantitative assessment indicated by the expert determines the degree of their comparative preference.

It is necessary to agree in advance that, say, a higher value of the estimate corresponds to a more preferable alternative. Sometimes it is more expedient to quantify the relative preference of objects in points, using specially designed point scales.

2. Midpoint method

The method is used when there are a lot of alternative options. If we denote by f(a1) the assessment of the first alternative variant of the indicator value, relative to which the comparative preference of objects is determined, by f(a2) - the assessment of the second alternative variant, then further the expert is invited to select the third alternative variant a3, the assessment of which f(a3) is located in the middle between the values ​​f(a1) and f(a2) and is equal to f(a1) + f(a2)/2.

At the same time, it is advisable to choose the least and most preferred alternatives as the first and second alternatives.

Further, the expert indicates an alternative a4, the value of which f(a4) is located in the middle between f(a1) and f(a3), and an alternative option a5, the value of which f(a5) is located in the middle between the values ​​f(a1) and f(a4) .

The procedure is completed when the relative preference of all the alternatives involved in the examination is determined.

3. Method Churchman - Akof

The Churchman-Akoff method is used to quantify the comparative preference of alternative options and allows for adjustment of estimates given by experts.

The method assumes that the estimates of alternatives are non-negative numbers, that if alternative a1 is preferable to alternative a2, then f(a1) is greater than f(a2), and the assessment of the simultaneous implementation of alternatives a1 and a2 is f(a1) + f(a2)).

All alternative options are ranked by preference, and each of them is assigned quantitative estimates by the expert, as a rule, in fractions of a unit.

4. Lottery Method

According to this method, for any triple of alternatives a1, a2, a3, ordered in descending order of preference, the expert indicates such a probability p, in which the alternative a2 is equivalent to a lottery, in which the alternative a1 occurs with probability p, and the alternative a3 - with probability 1 - p.

Acquisition Methodsquality expert assessments

Sometimes the specifics of the objects of expert evaluation are such that experts find it difficult to give quantitative estimates of the values ​​of the evaluated indicators or the object as a whole, and in some cases such estimates are simply unjustified and do not allow obtaining sufficiently reliable expert information.

1. Expert classification

This method is advisable to use when it is necessary to determine whether the alternative options being evaluated belong to established and accepted classes, categories, levels, varieties, etc. (hereinafter referred to as classes).

It can also be used when the specific classes to which the evaluated objects should be assigned are not predetermined. The number of classes into which the evaluated objects are divided may not be predetermined. It can only be established after the completion of the classification procedure.