Evaluative Thought in A Theory of Design Thinking

Charles Burnette, PhD 2/24/2015 charlesburnette@comcast.net

Abstract

This paper describes evaluation as a component process for reaching decisions about the quality, quantity, and subjective interpretation of objects of purposeful thought. It locates evaluation as one of seven interacting modes of thought that address different aspects of any purposeful thought. The systematic framework of A Theory of Design Thinking allows judgment, assessment, and subjective values to be applied to intentions, objects, organizations, ideas, interactions, communications, products, processes, beliefs, learning, self, and society through a common framework. It achieves this by incorporating specialized evaluative components in the intentional frame used to organize thought or action about anything. Such thought is applied within a context of feelings, needs, and desires regarding a focal subject or situation.

Keywords: evaluation, judgment, assessment, value, objects of thought, feelings, modes of thought, goals, differences, purposeful thought, Design Thinking

Introduction

A Theory of Design Thinking is about purposeful thought that can improve a situation or how it 1

is understood. It proposes seven modes of intentional thought that collaborate within a shared framework. The frame for every mode incorporates all modes as components, including an Evaluative mode determining the quality, quantity, or value of the outcome of a purposeful thought. The affect, significance and utility of this value is then determined through the Reflective component of the Evaluative thought. Every component in the thought is activated by needs, desires, and feelings associated with that component. These activating factors arise through interpretation and expression of the focal object of a thought. Evaluative thought monitors, resolves, and confirms completion of an intentional thought and works to resolve problematic aspects, needs or desires that arise with it.

In the theory, the Evaluative mode of thought is understood to address any aspect of our experience, from subjective feelings to life itself, from the selection of the right problem to the

�1

appreciation of a painting, or the validity of a scientific experiment. This broad range of application necessitates that Evaluative thought be purposeful and selective about what is evaluated and what criteria are applied. Both purpose and selection regarding a focal subject result from feelings, beliefs, prior values, and useful knowledge generated through Evaluation of a focal subject or experience. Evaluation helps to adjust the subject and feelings, opinions and beliefs about it through the information it generates. Qualities, quantities, and subjective values are equally at home in it. Feelings and values, facts and constraints arise through lived experience and thought about it. Differences occur as people consider what concerns them and differential comparisons are needed. There is no inherent argument between art, science, and understanding.

Defining and Applying Evaluation

The quotes below are from the Introduction to Evaluation of the Web Center for Social Research Methods . They are included here to provide background for the reader. 2

The Web Center indicates that Evaluation is probably most frequently defined as “the systematic assessment of the worth or merit of some object”, and goes on to suggest that perhaps a better definition might be: “Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object.” “Both definitions agree that evaluation is a systematic endeavor and both use the deliberately ambiguous term 'object' which could refer to a program, policy, technology, person, need, activity, and so on…. The generic goal of most evaluations is to provide "useful feedback" to a variety of audiences including sponsors, donors, client-groups, administrators, staff, and other relevant constituencies.”

The Web Center also outlines goals of Evaluation in Social Research, first characterizing them as Formative evaluations that strengthen or improve the object being evaluated -- they help form it by examining the delivery of the program or technology, the quality of its implementation, and the assessment of the organizational context, personnel, procedures, inputs, and so on. Summative evaluations, in contrast, examine the effects or outcomes of some object -- they summarize it by describing what happens subsequent to delivery of the program or technology; assessing whether the object can be said to have caused the outcome; determining the overall impact of the causal factor beyond only the immediate target outcomes; and, estimating the relative costs associated with the object.

“Formative evaluation includes several evaluation types:

�2

Needs assessment determines who needs the program, how great the need is, and what might work to meet the need Evaluability assessment determines whether an evaluation is feasible and how stakeholders can help shape its usefulness Sructured conceptualization helps stakeholders define the program or technology, the target population, and the possible outcomes Implementation evaluation monitors the fidelity of the program or technology delivery Process evaluation investigates the process of delivering the program or technology, including alternative delivery procedures

Summative evaluation can also be subdivided: Outcome evaluations investigate whether the program or technology caused demonstrable effects on specifically defined target outcomes Impact evaluation is broader and assesses the overall or net effects -- intended or unintended -- of the program or technology as a whole Cost-effectiveness and cost-benefit analysis address questions of efficiency by standardizing outcomes in terms of their dollar costs and values Secondary analysis reexamines existing data to address new questions or use methods not previously employed Meta-analysis integrates the outcome estimate”

Four models of evaluation are also identified: “Scientific-experimental models …Taking their values and methods from the sciences -- especially the social sciences -- they prioritize on the desirability of impartiality, accuracy, objectivity and the validity of the information generated. Included under scientific-experimental models would be: the tradition of experimental and quasi-experimental designs; objectives-based research that comes from education; econometrically-oriented perspectives including cost-effectiveness and cost-benefit analysis; and the recent articulation of theory-driven evaluation.”

Management-oriented systems models. such as PERT, the Program Evaluation and Review Technique, and CPM, the Critical Path Method…the UTOS model where U stands for Units, T for Treatments, O for Observations and S for Settings; and the CIPP model where the C stands for Context, the I for Input, the first P for Process and the second P for Product. These management-oriented systems models emphasize comprehensiveness in evaluation, placing evaluation within a larger framework of organizational activities.

�3

Qualitative/anthropological models. “They emphasize the importance of observation, the need to retain the phenomenological quality of the evaluation context, and the value of subjective human interpretation in the evaluation process. Included in this category are … the various qualitative schools; critical theory and art criticism approaches; and, the 'grounded theory' approach of Glaser and Strauss among others.”

Participant-oriented models. “They emphasize the central importance of the evaluation participants, especially clients and users of the program or technology. Client-centered and stakeholder approaches are examples of participant-oriented models, as are consumer-oriented evaluation systems.”

Qualitative/anthropological models and Participant-oriented models are often used in design and other humanities. However, the models described do not build on a consistent, coherent, and unified framework as in A Theory of Design Thinking through which all of the objectives mentioned may be implemented. When designing, comparative evaluation is essential to discerning how to improve what is problematic.The modes of thought in the theory enable the generation, evaluation, and use of improved artifacts and services. They are also useful to reach other objectives in other circumstances. Any kind of evaluation may be implemented through the structure of A Theory of Design Thinking whether Formative or Summative or for personal or group purposes.

Rather than trying to describe the many applications of evaluation, the remainder of this paper will distinguish three categories through which to consider evaluation, outline a generic process of evaluative thinking, and describe how the components of the proposed theory apply to evaluation.

A Theory of Evaluative Thinking

Thought that enables empirical, subjective, quantitative, and qualitative assessment of the magnitude of difference between something and a standard, criteria, or object is essential to determining progress toward a goal. In A Theory of DesignThinking, Evaluative thought is the mode of thought through which expectations, experientially determined values, and feelings mark progress toward a goal. It is the cognitive agency through which monitoring, measurement, judgment, assessment, and valuation are accomplished. It is exercised whenever judgments, differential comparisons, measures, tests, and reflections are made to qualify, quantify,

�4

subjectively value, or understand the consequences of an expression, action, or change. Evaluative thought compares relevant information to goal criteria, standards, and targets. It relates abstract ideas to concrete outcomes to enable well-reasoned conclusions. It determines value within different modes of thought and integrates them. It recognizes and assesses feelings, assumptions, implications, and practical consequences; and it effectively communicates both progress and solutions. It both monitors progress and generates feedback to indicate if further thought is needed to satisfy a need or reach a goal. It encourages objective standards, the pursuit of excellence, and helps interpretation and expression to become clear and precise. It brings a satisfying end to an intentional thought, and determines the significance, utility, and value of the results. In sum, it fosters progress by providing experientially determined values for subjective reflection about their significance and usefulness to wellbeing, and the quality of life for the thinker, society, culture, and environment.

One thinks in an evaluative way by adopting an appreciative or critical stance toward a subject, situation, or object and then:

1. Determining the goals to guide thought about that subject, situation, or object; 2. Defining the standard, criteria, or objects to be used to determine if goals are achieved. 3. Organizing goal criteria in relation to factors in the situation to enable goal attainment. 4. Formulating a plan of action for evaluating the subject, situation, or object of thought. 5. Executing a differentiation that compares focal circumstances to goals regarding them. 6. Judging or Assessing the magnitude of difference to determine progress toward the goal; 7. Considering if goals have been reached, if not, revising the action plan for a better result.

A thinker can improve their evaluative thinking by applying the different modes of evaluative thought discussed below. For example: closer attention to needs and desires regarding a situation sharpens intentional focus on how to resolve them; careful definition of critical elements allows better use of them; exploration and analysis of potential relationships between elements enables well-reasoned models fitting intentions to circumstances; awareness of context and alternative ways to interpret and express a situation helps to synthesize a clear and appropriate expression of it; anticipating processing results through simulation, trial and error, or practice helps to reach outcomes more efficiently and effectively; using appropriate methods to match the entity to be judged or assessed to goal criteria regarding it can assure correct evaluation; exploring prior knowledge helps to map relevant knowledge and situated value to the focal situation and tighten assumptions, representations, and understandings regarding it. When goal criteria are satisfied information, practices, and expectations are reinforced in memory; if goals are not satisfied anomalies remain to motivate intentional thought to resolve them.

�5

In A Theory of Design Thinking a distinction is made between “abstract” analytical thinking pursued through Relational thought and “concrete” critical thinking undertaken through Evaluative thought. Success in analytical thinking is determined by the goodness of fit to a situation; success in critical thinking by the value of an outcome to the fulfillment of an intention. Effective thinkers interpret, apply, and evaluate the results of all component modes in an Evaluative thought, carefully judging or assessing not only the situated information available through them but also the intentions, thoughts, and feelings about that information. By evaluating a situation through all modes of thought thinkers avoid simplistic thinking and narrow views. They also apply the modes to interpret and evaluate the apparent intentions, knowledge, and behavior of others regarding a focal subject.

The Evaluative mode of thought is understood to be an integral aspect of thought addressing every aspect of experience. An Evaluative component exists in every mode of purposeful thought and all aspects of its subject are accessible to Evaluative thought. Every purposeful thought has a subject, situation, or object to evaluate. Objects of thought different from the focal object to which they belong may be considered if the thinker finds it useful to do so. The thinker may also take different points of view regarding how to evaluate a focal subject. For example, among possible objectives, the evaluation of a work of art may determine its market value, its significance to the career of an artist or their clients, its value as a cultural statement, the educational value to a student, or values related to making it. This broad range of application requires that such thought be purposeful and selective about what is to be evaluated, what criteria or standards are to be used, and how they are to be applied. Intentions and context for an evaluation must be declared. A focal subject must be expressed as an object of thought before its Evaluation is possible. Tentative goals must be declared before a situated object of thought may be evaluated. The aspects of that object to be evaluated and the criteria to be used in their evaluation must be indicated.

It is objective when it involves the comparison of some cognitive object to another cognitive object serving as a criterion. It is subjective when it involves comparison of some cognitive object to feelings and beliefs recalled from memory. Often there is a conventional, historic, cultural, or personal standard against which an object of thought is evaluated. The Evaluative component in every mode of thought enables it to operate across the entire range of experience, from feelings of uncertainty and doubt through facts supported by scientifically established evidence. An entire experience, or a thought, behavior or feeling within it can be selected for

�6

evaluation. Any change in meaning, usability, aesthetics, empirical fact, cultural norms, preference or belief can be addressed. An evaluation focuses on the expression and circumstances of a subject or situation to determine the magnitude of difference between them and criteria intentionally selected for their evaluation. Whether qualitative or quantitative, evaluation acts on a situated object of thought and responds to changes in its circumstances or expression. Prior evaluations of similar situations are subconsciously recalled from memory to interpret new situations and, if the interpretation using recalled information cannot be evaluated with existing criteria that have been established or recalled, the intentional component manages a new thought to reinterpret, reevaluate, and resolve the focal situation. If criteria are not met the outcome of Evaluative thought is fed back into a process of reflection, re-interpretation, re-expression, re-execution, and re-evaluation until the intended goal is reached.

Evaluative thought determines progress toward an intended goal or marks its attainment, significance, utility and affect. A major purpose of Evaluative thought is to determine if a need, desire, or intention regarding an established focal object has been resolved or fulfilled. Because emotional salience is assumed to motivate and prioritize a mode of thought, and feelings based on previous experiences are assumed to set its tone and direction, affective meaning of the outcome is implied. This outcome need not be tangible or even positive because evaluation of any thought, action, or experience is necessary to determine if its purpose has been achieved, situated needs are addressed, or no present or future use of the result can be imagined.

Marvin Minsky has written, “The difference engine scheme remains the most useful 3

conception of goal, purpose, or intention yet discovered. … The idea of a difference-engine embodies a representation of some outcome and a mechanism to make it persist until the outcome is achieved…. (It) must contain a description of a desired situation. It must have subagents that are aroused by various differences between the desired situation and the actual situation. Each subagent must act in a way that tends to diminish the difference that aroused it.” Jerry Diethelm refers to this as the “qualitative difference”. 4

In the proposed theory such a “difference engine” incorporates all modes of thought as sub-agents, is influenced by them, and is regulated through the Intentional, Evaluative and Reflective components of a thought. If an evaluation does not reach the desired conclusion, processing passes cyclicly through Reflective, Formative, Procedural, Evaluative, and Reflective components in a “Reflective loop” that recalls information from memory to revise Formative expressions of the focal object and improve the information, expression, and processing of the

�7

evaluation and move its result toward a goal until it is achieved, a new intention is motivated, or the thought is dismissed. If the goal is reached, Reflective thought ignores this recursive cycle and determines the significance and potential utility of the evaluative experience.

Appreciative thinking, design thinking, critical thinking, experiential thinking, and empirical thinking are five kinds of evaluative thinking. Appreciative thinking seeks positive outcomes. Design thinking focuses on improving the focal object. Critical thinking focuses on differences from a standard, criteria, or object. Experiential thinking focusses on the experience of realizing an intention, and Empirical Thinking focusses on assessment of grounded outcomes. With regard to critical thinking, the term "design criticism" carries a great deal of intellectual baggage from the fields of literary criticism and art history. "Situated critical thinking" in which the "criteria" that provide the basis for judgment and assessment are determined by intentions regarding a focal situation and knowledge relevant to it offers a more grounded approach. Failure to articulate and express all four factors (the focal subject, its context, background, and intention regarding it) results in a lack of critical focus and an inadequate frame of reference for either analytical or critical thinking and communication of their results. By producing results for both analytical and critical thinking regarding an experienced situation, evaluations contribute to change in a Formative expression by seeking correction or confirmation of information through reflective thought. Value is always relative and determined through comparison of change in a focal object to something that is felt, believed, or known to be right, needed or desired. Evaluations of all kinds are based on feelings, insights, judgments, events, measurements, or assessments of the magnitude of difference between changes in an expression of a focal subject and criteria specified through Intentional thought regarding it. Whether preconscious or conscious, judgment and assessment are intentionally directed and developed through the component processes in an Evaluative thought.

Concepts of Evaluation

It is useful to consider Evaluative thought through three general concepts; Judgment, Assessment, and Value. Many evaluative decisions are Judgmental, often fast, and intuitive. Assessment can involve monitoring, measurement, testing, accounting, estimating, and other tools for discriminating change, merit, efficiency, worth, fact, social position, cultural acceptance, feelings, and other variables grounded in the focal subject or situation. Value has similar range in that it can be tied to personal likes or dislikes, economics, social issue, policies, cultural practices and many more dimensions that might be selected for discrimination on a scale.

�8

Judgment A dictionary definition of Judgement is “ 5

1. The ability to make considered decisions or come to sensible conclusions 2. An opinion or conclusion. 3. A misfortune or calamity viewed as a divine punishment.

Malcolm Gladwell has popularized the speed, effectiveness, and ubiquity of subconscious judgement in "Blink, the Power of Thinking Without Thinking”. This speed and range in 6

subconscious thought is probably due to parallel processing by the brain. Subconscious thought is also known to access more information of greater diversity than conscious thought. As a 7

result, subconscious judgment is more prevalent in our thoughts than conscious, rational processes of evaluation against explicit criteria, standards, practices and goals. One can’t exercise judgment either subconsciously or consciously without focusing on a situation and generating an intention to evaluate it in whole or part. Nor can one do so without interpreting the focal situation in light of what is understood about what is to be evaluated. Judgment is informed through interpretation that references personal, social and cultural knowledge. It can be understood as a fast decision based on felt values regarding how well evaluative criteria have been met.

In A Theory of Design Thinking, immediate Judgment can be understood as subconscious evaluation (as in Gladwell’s Blink) that is then orchestrated and evaluated consciously through sequential, rational thought. Preconscious judgment is understood as a purposeful evaluation that may reach consciousness as an intuition or as a settled decision. Conscious judgment may follow an intuition or be a response to needs and desires arising from the focal subject, situation, or object. It may take a preconscious judgment and consciously deconstruct it, articulate its arguments and relations, analyze and organize them, express the proposed judgmental model, execute the plan of action called for, assess the outcome of this process, consider its appropriateness to intent and circumstances, and, when there is a result, its significance, and potential utility. Consciousness makes judgment explicit and rational, enabling language to communicate goals, the criteria that specify them, and the feelings invoked. Feelings associated with preconscious judgmental thought are adjusted as the thought becomes conscious and explicit. All modes of thought (and all ranges of emotion) are assumed to be involved in both preconscious and conscious judgment. Dehaene has stated “Based on what we now know, virtually all the brain’s regions can participate in both conscious and unconscious thought.” 8

�9

Daniel Kahneman has written "A judgment is said to be mediated by a heuristic when the individual assesses a specified target attribute of a judgment object by substituting a related heuristic attribute that comes more readily to mind. ...The noun refers to the cognitive process, and the adjective in ‘heuristic attribute’ specifies the substitution that occurs in a particular judgment." 9

This heuristic substitution of objects of thought that come more readily to mind facilitates fast judgment but is also a potential source of error that might stimulate more effortful conscious thought. Prediction, error detection, and reward systems in the brain that celebrate goal attainment usually operate subconsciously. Neuroscientists have identified the anterior cingulate cortex, a tiny area in the center of the brain having many dopamine neurons, as the source of a unique electrical signal known as "error related negativity”. Because of its central location and connection to the thalamus (which helps focus conscious attention) "the ACC helps control the conversation between what we know and what we feel." It also revises erroneous predictions to anticipate future judgments.

Different kinds of judgment apply to different modes of thought: judgment of success applies to intentional thought; semantic judgment applies to referential thought; organizational judgment to relational thought; aesthetic judgment to formative thought; judgment of efficiency to procedural thought; judgment of the magnitude of change and the effectiveness of the process to evaluative thought; while ethical and moral judgment is a matter for reflective thought. Opinion and preferences are considered to be judgments formed through Reflective thought. Biases and beliefs are cumulative results of reflective evaluations that color intentional judgments.

Assessment A dictionary definition of Assessment is: 10

“the evaluation or estimation of the nature, quality, or ability of someone or something: …”

Unlike the decisions of rapid judgment, in the proposed theory, Assessment is explicit, articulated, rational, and primarily a matter for conscious thought. It involves observation or monitoring, appropriate models and methods, useful representations, instrumental thought, methods, and techniques, empirical measurements, value determinations appropriate to the focal circumstances and intention, and contextually constrained interpretations of value. 11

Most practical evaluations involve measurements, tests, estimates, or predictions that require intention, rationale, and criteria to be articulated in purposeful language. Scientific evaluations

�10

are empirical assessments that would have no value without being reported. Evaluations of design are assessed through use, effects, and affections generated during and as a result of designing.

Wikipedia offers other examples of what assessment may refer to. “Educational assessment, the process of documenting knowledge, skills, attitudes, and beliefs Health assessment, a plan of care that identifies the specific needs of the client and how those needs will be addressed by the healthcare system Nursing assessment, the gathering of information about a patient's physiological, psychological, sociological, and spiritual status Psychiatric assessment, a process of gathering information about a person within a psychiatric or mental health service with the purpose of making a diagnosis Psychological assessment, an examination into a person's mental health by a mental health professional such as a psychologist Risk assessment, the determination of quantitative or qualitative value of risk related to a concrete situation and a recognized threat Tax assessment, value calculated as the basis for determining the amounts to be paid or assessed for tax or insurance purposes Vulnerability assessment, the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system Political assessment, assessment of officeholders for political donations” 12

However, some assessments are behavioral and may never get expressed in language, as when

one observes or performs a purposeful act such as hammering a nail, riding a bicycle, creating a

work of art, designing, or performing music. Flow is understood as conscious and emotional 13

awareness that one has met certain standards held by the performer that may not be expressed in

language. One evaluates both the behavior and its outcome without necessarily communicating

the evaluation even though one is conscious of making an evaluation and has the intention and

critical knowledge to do so. The evaluation of behavior involves comparing a situated process to

practiced criteria for that behavior in memory. Evaluating ones own behavior as it occurs

involves the evaluation of a situated process against goals for that behavior in memory whether

the patterns are constructed or physically experienced.

�11

A value judgment has been defined as an assessment of something as good or bad in terms of

one's standards or priorities. In the proposed theory, this use conflates the terms. 14

Value

A dictionary definition of value lists several interpretations; 15

“1. The regard that something is held to deserve. 2. The material or monetary worth of something. 3. The worth of something compared to the price paid or asked for it. 4. (Values) a person's principles or standards of behavior; one's judgment of what is important in life. 5. The numerical amount denoted by an algebraic term; a magnitude, quantity, or number. 6. Music: the relative duration of the sound signified by a note. 7. Linguistics: the meaning of a word or other linguistic unit. 8. The quality or tone of a spoken sound; the sound represented by a letter. 9. Art: the relative degree of lightness or darkness of a particular color.

In A Theory of Design thinking, value is determined by judging or assessing something in terms

of intentionally specified criteria for its evaluation. If elements within a focal situation are

valued, they are subject to intentions and goal criteria regarding it. Whether focused on a parent

frame or one of its components, value determination is always dependent on interpretation or

expression of the current focal object. This value may be one of feeling or empirically

determined fact. The determination of difference between a focal object and its desired state

helps to improve the focal object, but its subjective value, significance, or utility is determined

through Reflective evaluation that interprets value through knowledge of self, society, culture,

and similar circumstances.

The qualitative, or quantitative value of a focal subject is determined through the evaluative

component of an Evaluative thought as the magnitude of difference between a focal subject and

intentionally specified goal criteria regarding the evaluated aspect of it. A goal and its criteria

have intentionality regarding the focal situation that may be recalled, specified, or synthesized.

Criteria for both primary and component evaluation are set through their Intentional or

�12

Reflective modes of thought and interpreted through their Formative mode. The metric and

method for determining difference with the object being valued is carried by the criteria to be

applied. A metric is not always scalar or absolute. It may be relative, variable, or subjective.

Within this conception, qualia (a neuroscientific term), is understood as an emotionally weighted,

subjective feeling about an empirical value determined through evaluative thought. Formative

expressions obtain feelings of value this way. Qualia are experienced as holistic situated

phenomena that defy articulation but not memory. They are assumed to be neural networks that

manifest feelings about particular information in a focal situation. Qualia may be recalled by

mapping feelings about a focal object in memory to similar focal objects being evaluated. They

are criteria that may be sought or applied and experienced without articulating them. Quail are

essential to the evaluation of art, mysteries, or transcendent feelings. Neuroscience may

eventually be able to describe how qualia arise.

Judgments or Assessments determine the magnitude of difference between a focal subject or

situation and the criteria applied to it. They are discriminations of the difference between a state-

of-mind before and after its expression is changed by sensory input, reflective interpretation, or

Intentional thought. The resulting “value category memory”, to use Edelman’s term, is a highly 16

connected system of neural groups that manifest the sense of what has resulted in the brain from

an evaluation. Beliefs are among the cumulative results of such valuations. They may be

expressed as opinions or empirical valuations. (Opinions are evaluations based entirely on what

is believed about a situation. Empirical valuations are those determined through ) As applied or

summary objects in memory, opinions and beliefs may influence judgment through subconscious

interpretation of a focal situation. Alternatively, empirical valuations are those grounded in

experience of a situation.

Evaluation continues until the goals of an intention are met and its criteria satisfied, or the result

of the evaluative process is accepted or dismissed. Satisfaction of goal criteria ends an

intentional thought and the complete episode of evaluation is passed into memory (or a mediated

knowledge base such as a spread sheet, data base or worked example). Subjective value,

�13

significance and future utility is determined in memory. If goal criteria are never satisfied, or are

deferred, displaced or forgotten the magnitude of the difference that existed also passes into

long-term memory, introducing uncertainty into future Reflective thought on the subject.

“Valuation is a specialized process that calls upon evolved neural networks within a set of nuclei

called the basic ganglia (because they are located near the base of the brain)...They too can

operate totally outside our consciousness.” The neurological basis for Evaluative thought might 17

be in nuclei sending axons up and down the nervous system that Edelman has called “value

systems”. Value systems are “diffuse ascending neural networks that modulate synaptic

responses by releasing specific neuromodulators or transmitters in a broadside fashion. ……they

only constrain actions and perceptual categorization.” These nuclei affect large populations of 18

neurons simultaneously by releasing different neurotransmitters to affect both learning and

memory and control bodily responses necessary for survival. This seems consistent with the

functionality of Reflective evaluation in A Theory of Design Thinking. Following Edelman, the

theory also assumes that such thought, as all thought, involves “extensive encounters” between

neuron groups, and “reentrant circuits” that enable the brain to mediate a thought through higher

order "value–category memories" that result from the release of neurotransmitters through the

value systems of the brain. These circuits are assumed to influence the salience of a coherent

neural group for recall through Reflective thought and prioritization through intentional thought.

Trustworthiness

Evaluation is a process in which trust is required if the result is to be accepted. The potential for

error in judgment has been mentioned. The trustworthiness of assessment is equally suspect, if

not more so, as it depends on belief in the credibility, honesty, and competence of the evaluator

as well as the appropriateness of their purpose, the relevance of the information they consider,

their thoroughness in exploring it, the validity of their interpretation, their care in applying

appropriate methods, the objectiveness of their findings, and the knowledge and emotional

balance it provides. That is why the credibility of the evaluator, well understood intentions,

thoroughness, established methodologies, practices, forms of reporting that facilitate review, and

�14

review by knowledgeable peers are emphasized in scientific work. The seven components of

Evaluative thought mirror these concerns. They may be expressed as: Purpose and goals, Factors

and background, Hypothesis and approach, Experimental conditions, Methods and techniques,

Empirical outcomes, Significance and potential. Feelings associated with each mode of thought

assure that all considerations have an affective component.

Intentional evaluation

An intention to evaluate a focal situation or some aspect of it initiates and directs Evaluative thought. An object of thought must be instantiated and interpreted in some coherent context and an expression of that object must exist to have something for an intention to evaluate. For example, a logical proposition or algebraic statement can be evaluated because it is expressed in a formal language with defined vocabularies governed by rules of use that organize expression in formulaic terms grounded in prior knowledge. A poem or painting can be assessed according to intentionally selected criteria because they are “concrete expressions” with a specific context of related expressions and interpretations.

Intentional evaluation is the sub-mode of an Intentional thought that determines the goals regarding the focal situation that will be the basis of evaluation. Motivated by anomalies in a focal situation and its expression, it registers needs and desires and sets goals to resolve the anomalies and the criteria to be used for evaluating goal satisfaction. The modal subcomponents in the “intentional frame” of an evaluative thought are all intentional about the aspect of the focal object they address. They collaborate to construct and execute the evaluation that determines the magnitude of difference between the features of the focal situation being evaluated and the intentionally specified goal criteria regarding them. Weightings from emotional stimuli, feelings, and prior knowledge affect the salience of information in each mode of the thought and their synthesis in the formative expression of the object to be evaluated.

If the expression of a problematic situation is only partially resolved, or if the intention can’t be achieved through recursive processing within the Relective loop (from Reflective through Formative, Procedural, Evaluative, and Reflective evaluation) to revise the expression of the object of thought, another Evaluative intention and all of its component processes may be activated. Such a reframing of intent requires new goals, criteria, recognitions of data, modeling and analysis, expression, differential comparison, evaluation, and further reflective interpretation to reach the revised intentional goal.

�15

Referential evaluation The objects of Evaluative thought include expressions and features to be evaluated, goal criteria to be employed, and methods to be used in the evaluation. Features of the focal situation or its expression are coherent objects of thought that are not changed unless the results of evaluation do not satisfy the goal. Goals, criteria, and features of the focal situation are semantically meaningful, holistic objects of thought, that must have identity in memory to be recognized, selected, and recalled. To be recalled, these objects in memory are matched to similar objects in the focal situation, or are intentionally selected or specified. Their suitability for use in the evaluation is abstractly modeled and analyzed through Relational evaluation, synthesized and expressed through Formative evaluation, processed through Procedural evaluation; the results established through the Evaluative component, and the outcome interpreted through Reflective evaluation to improve it or to determine its significance and usefulness. Referential objects of all modes of thought; intentions, objects of thought, relationships, situated expressions, instrumental activity, empirical outcomes, and both episodic memories and the use of knowledge are subject to Referential evaluation to determine their relevance.

Relational evaluation Relational evaluation is the component of Evaluative thought that seeks to organize, structure, and analyze Referentially itemized objects of thought to suit the goals for a focal subject. It is focused on how information and evaluative criteria are structured to fit a focal situation. The Relational sub-mode of an evaluative thought structures discrete ohjects of thought recognized as potentially relevant through Referential evaluation. It structures these semantically meaningful items as arguments and relations in relationships, networks, and conceptual models that it explores, manipulates and analyzes to obtain the best fit to the goals of the intention, circumstances of the focal situation, and evaluative methods to be used. A Relational evaluation is constructive and propositional during modeling and deconstructive if applied to an already resolved expression.

Relational analysis compares abstract structures in order to differentiate, manipulate, and edit their elements and relationships to better fit an intention to the situation it addresses. It may involve a differential comparison to determine the best information or action to use, or editing and modification of prior models used in similar situations. It is applied to abstractly analyze fully constituted and grounded expressions for the purpose of critically evaluating them in their situated context. This conception emphasizes that evaluative thought focuses on an already structured, formulated, and expressed object of thought, grounded in the circumstances of a focal

�16

situation, that can be judged subconsciously or consciously and assessed in objective, factual terms. Formative Evaluation The Formative component of evaluation focuses on the expression of a focal situation, its features, and methods for evaluating them. These expressions may result from a synthesis of information in a conceptual model, from sensing and perceiving an experienced situation, or from Reflective interpretation. The purpose of evaluating these expressions is to judge or assess them or their constituent features when processing a differential comparison to the criteria specified for their evaluation. For example, measurement is a differential comparison in which an explicit scale is used to differentiate between an expression and a criterion regarding it. There can be no measurement without expressing a system of measurement (inches, pounds, units, feelings, etc.) an object to measure, a goal criterion, and a process for conducting the measurement of the object consistent with its subjective or objective properties and the purposes of the evaluation.

Characteristic forms of evaluative criteria include standards, rules of thumb, performance specifications, and threshold values. Characteristic forms of evaluative methods include judgments, measurements, and tests. Characteristic forms of outcomes include qualitative judgments such as important/secondary; acceptable/satisfying; like/dislike; or quantitative assessments such as number/amount; larger/smaller, positive/negative; true/false. Concepts of high/low; status, level, and worth can be of either type.

Although one may experience aesthetic pleasure in response to an ethically interpreted expression, aesthetic judgment or assessment is not ethical judgment or assessment because aesthetic evaluation is situation dependent, immediate, and felt, while ethical judgment is cumulative and based on knowledge and feelings acquired across experiences in different circumstances.

The thinker formulates expressions at any level in any mode. Since each mode expresses its own Formative evaluation regarding the focal situation of its parent mode, a Formative expression of an the parent Evaluative thought synthesizes the expressions of its seven sub-modes. These

�17

.

subordinate evaluative expressions are only accessible through the Intentional frame of the primary Evaluative thought to which they contribute their expressions. This hierarchy can be traversed to access the different evaluative thoughts through the primary framework> Evaluative mode> formative evaluation.

Any intentions, referential objects, conceptual models, expressions, processes, evaluations and reflections may be assessed against a focal situation by an external interpreter if expressed and communicated to them. The focal situation evaluated by the thinker may not, and in many respects will not, correspond to that recognized by an external interpreter. A satisfactory correspondence between internally generated expressions of meanings and those of an outside interpreter depend greatly on the predictive roles of shared experiences, a common language, the constraints of media and context, and shared processes or patterns of interpretation and behavior. To minimize misinterpretation, research on purposeful communication must take account of these factors and the context, circumstances, background and purpose of expression when evaluative thinking by another person is considered.

Procedural Evaluation Procedural Evaluation is the sub-mode of an Evaluative thought through which the application of methods, skills, and knowledge in evaluations are executed. It concerns the processes, methods, and techniques used to execute a plan of action for an evaluation that has been developed and expressed through the intentional, referential, relational, and formative modes of the Evaluative thought. This process begins by recalling, through Reflective evaluation, processes of evaluation prevviously used in situations similar to the current one and adapting them to the focal situation through Formative evaluation. It continues by intentionally considering the resulting expressions and adapting the focus, goals, criteria, and means of evaluation of a recalled process to fit the needs, desires, context, and circumstances of the present situation. Any features of the recalled thought may be intentionally adapted, accepted, or ignored, and preferences (or biases) may be applied. Procedural evaluation may call on Referential evaluation to retrieve information from other sources until appropriate procedural objects have been recalled. Relational evaluation then determines how these entities and factors are to be related to goal criteria to propose an evaluative process suited to the situated intention. The intended goal state of the focal situation and a plan for processing its evaluation is expressed through the Formative component of the Evaluative thought. Expressions from Procedural components in all sub-modes are integrated in this expression. The resulting plan of action is then activated to execute a differential comparison between the current focal expression and the goal criteria expressed regarding it. The procedural sub-component of Evaluative thought then

�18

determines the differential value for each goal criterion compared to the feature the criterion addresses. The Evaluative component then determines the overall value and forwards it to Reflective thought. Procedural evaluation is closely coupled with motor responses and actions of the mind, and body.

Evaluative component The Evaluative component of an Evaluative thought generates a value stating the magnitude of difference between some standard or criterion and the situated object of reference that has been intentionally selected and expressed for evaluation. The comparative difference between one object and another requires the specification of criteria, profile, or standard against which the magnitude of difference is to be measured. The intended goal, the objects of reference, the relationships between them, the expressive form to be evaluated in context, and the process by which variance is determined, must be part of this specification. It must check, index, and reconcile evaluative expressions of all sub-components of an Evaluative thought in order to determine if their goal criteria have been met through procedural evaluation. Monitoring, prediction, expectation, and completion, take place before the evaluative outcome is forwarded to the Reflective component of the thought. If all goal criteria are met the Evaluative component concludes thought and converts its result (judgment, measurement, preference, test, etc.) to Reflective memory where its significance and potential future use are determined for recall as needed. Its results do not change the focal situation. Alternatively, if goal criteria are not met, the Evaluative component records the progress attained for each feature, predicts the remaining “distance to the goal”, and forwards all information about unmet sub-goals to the Reflective component where it is mediated by relevant prior knowledge before being applied to change the Formative expression of the focal situation or the plan of action then executed through Procedural evaluation.This loop through Reflective thought to interpret and adjust a Formative expression can be an iterative recalibration process that stays close to the original goals or one that may be altered by a shift in intention or reinterpretation that draws on prior knowledge that may cause a problematic change to the focal situation.

Reflective evaluation The operations of Reflective evaluation were just partially described. Although decisions are made through intentional thought, and situated information is evaluated in all modes, Reflective evaluation regulates the process by revising the Formative plan of action without necessarily changing the intention or focal object. When goals are not satisfied, the Evaluative component forwards the resulting information to Reflective evaluation as feedback withy which to revise the

�19

plan of action, and continue to cycle through the Reflective loop until goals are reached. When goal criteria are met the Evaluative component concludes thought and converts information about it to long term memory. Its results (judgments, measurements, preferences, etc.) do not change the focal situation. Instead, Reflective evaluation determines the significance and potential use of information from the entire episode of Evaluative thought. It cumulatively assimilates evaluative information into personal, social, cultural, and historic memories, enabling complex value laden qualitative information to be introduced into the Reflective loop or recalled and interpreted for other purposes.

The athlete's or musician’s feelings of being “in the groove”, "self actualized" or in an optimal experience of "flow" manifests intentional behaviors often judged or assessed as they are 19

experienced. For example, an expert skier enthralled by the actual experience of meeting a challenge to their expertise is assumed to experience highly gratifying pleasure through optimal performance across all modes of thought. Successful, highly gratifying processes can also provide pleasure through the Formative procedural/evaluative/reflective loop. In such satisfying, fully immersive, yet critically evaluated experiences all modes of thought are assumed to be resolving and adapting concurrently; prior knowledge, intentional commitment, linguistic apprehension, combinatorial structure, mediating expression, procedural technique, and qualitative resonance are simultaneously reconciled - their projection onto the evaluative schema satisfies the intention framed even during the experience. The appreciation of music, art, and beauty also manifest the capacity to compare, scale, judge, measure, and resolve an experience as it happens.

To summarize the characteristics of Evaluative thought Determining value is the cognitive capacity associated with Evaluative thought; Evaluation of a focal situation is its intentional objective; Goal criteria are its defining elements; Correspondence its organizing principle; Focus its perceptual principle, Differentiation its operative principle; Magnitude its critical principle; and Pragmatic value its systemic principle.

Evaluative thinking of any kind and complexity can be built on these principles

�20

End notes

Burnette, C. Issues, Assumptions, and Components in A Theory of Design Thinking 1

www.independent.academia.edu/charlesburnette

The Web Center for Social Research Methods /Evaluation Research/Introduction to Evaluation 2

http://www.socialresearchmethods.net/kb/intreval.php

Minsky, M. 1985, 1986. The Society of Mind, NY: Touchstone/Simon & Schuster p793

Diethelm, J. 2015, personal correspondence4

New Oxford American Dictionary (Apple edition), definition of judgment5

Gladwell, M., 2005: Blink, Boston, Little Brown6

Deheane, S., 2014: Consciousness and the Brain: Deciphering How the Brain Codes Our 7

Thoughts, NewYork, Viking Penguin, location 341 of 7801, Kindle edition, amazon.com

Ibid, location 683 of 78018

Kahneman, D. December 8, 2002: Nobel Prize Lecture, Maps of Bounded Rationality; A 9

Perspective on Intuitive Judgement and Choice, Princeton University, Department of Psychology, Princeton, NJ 08544, USA and Google Scholar.

New Oxford American Dictionary (Apple edition), definition of assessment10

Burnette, C., 2015: Interpretation, Expression, and Interaction through Formative 11

Thought,www.independent.academia.edu/charlesburnette

http://en.wikipedia.org/wiki/Assessment 2.28pm, 1. 25 201512

Csikszentmihali, M.,1990: Flow: The psychology of optimal experience, New York, Harper 13

Collins

New Oxford American Dictionary (Apple edition), definition of value assessment14

New Oxford American Dictionary (Apple edition), definition of value15

Edelman, G., 2004: Wider Than The Sky; the phenomenal gift of consciousness, New Haven, 16

Yale University Press p 180

Dehaene,S., 2014: Kindle edition., location 1270 of 780117

Edelman, G., 2006: Second Nature; Brain science and Human Knowledge, New Haven,Yale 18

University Press, p 59

Csikszentmihali, M.,1990: Flow:19

�21