Using Psychophysiological Signals in a User-Adaptive Personal Assistant

Boris Brandherm,

Margeritta von Wilamowitz-Moellendorff, Tim Schwartz, Michael Schmitz

Saarland University, Computer Science

P.O. Box 15 11 50, 66041 Saarbrücken, Germany brandherm@cs.uni-sb.de

Holger Schultheis

University of Bremen, Computer Science, P.O. Box 330 440, 28334 Bremen, Germany,

schulth@informatik.uni-bremen.de

Abstract Since psychophysiological signals are continuously available and usually quickly reflect changes of the user’s state, they constitute an important source of information for adaptive assistance systems. Despite their potential benefits however, physiological information is frequently neglected in current adaptive systems. This may—at least partly—be due to the fact that physiological measures cannot be easily used for adaptation. Instead several steps have to be taken to be able to draw on the advantages of physiology. First, each measure has to be evaluated regarding its suitability to distinguish between user states. Second, the exact relationships between physiological measures and states need to be identified. Finally, psychophysiological information has to be integrated over time and with other sources of information. In the scope of the BAIR project all three sub problems have been tackled and the respective solutions have been combined to give a systematic approach for the utilization of physiological information in user-adaptive personal assistance systems. 1 Introduction One goal of the project BAIR is the automatic adaptation of the behavior of a mobile assistance system to a user's resource limitations in order to realize a presentation of instructions and information appropriate for a given situation. Specifically, our assistance system models the temporally variable user properties of cognitive load, time pressure, and affective states. Because these properties are not directly observable, they have to be estimated on the basis of indirect evidence. Such evidence can be found, for example, in the user's speech and motor behavior, in data from physiological sensors, and in knowledge about possible causes of resource limitations, such as the system's own behavior or the user's activities. From all of these indirect measures, physiological signals seem especially suitable for the goal at hand, since physiology (a) is continuously available, (b) usually changes quickly after the user state has changed and (c) is usually quite sensitive, i.e., even small changes in user state show up in changes of physiological signals (e.g., Kramer, 1991; Wilson & Eggemeier, 1991). However, some properties of physiological measures, such as their susceptibility to a lot of influences besides the user properties of interest, prohibit the straightforward use of raw signals. Instead thorough analysis of each individual physiological measure as well as subtle integration of the resulting information—over time and with information from other sources—is needed. More precisely, at least three distinct steps of analyzing/processing physiological signals necessary to ensure proper functioning of the adaptive mobile assistance system can be identified.

First of all, the utility of the individual measures has to be evaluated. As recent research by Schultheis & Jameson (2004) as well as Iqbal, Zheng & Bailey (2004) has shown, some measures may not be appropriate for all situations. Therefore, it is essential to find out which subset of all possible physiological measures (e.g., skin conductance, pupil size, and muscle tension) is best suited to detect the value of which user variable(s), i.e., whether and how signals and states are causally linked. The best way to verify the existence of such links is by conducting psychological experiments: In manipulating user states while monitoring physiological change systematic covariances of user states and physiological signals can be detected. Yet, to know that a link exists is not enough. In a second step it has to be examined how exactly the raw signal corresponds to the values of user variables. Generally, this requires relating—ideally uniquely—sets of physiological values (physiological measurements) to user states. Because of the multiplicity of values constituting one physiological measurement, such a relationship can not usually be discovered by just looking at the data. Rather techniques of machine learning or pattern recognition are needed to discover the potentially existent regularities, called patterns, in physiological values associated with distinct user states. Once the second step has been successfully completed complex physiological signals can be efficiently mapped to comprehensible descriptions of the user’s current state. In principle, it would be possible to base the adaptation of the mobile device solely on these descriptions. In doing so, however, not only additional information from other sources would be neglected but also erroneous adaptations due to inaccuracies in the mapping might occur. Accordingly, to achieve more reliable evidence of the user’s state it seems sensible to integrate physiological information with information from other sources (e.g., speech, motor behavior) as well as over time. One approach which is especially suited to realize such integration is that of Dynamic Bayesian Networks (DBNs). DBNs as commonly employed are however computationally expensive, prohibiting their use with current mobile devices. Consequently, a new form of representing and processing DBNs has to be developed in order to make this methodology available to mobile devices. In the context of the project BAIR a principled approach to address all three of the afore mentioned issues with respect to psychophysiological data has been developed. In the following a brief outline of each of the three processing steps is given: First a current psychological examination to evaluate certain measures in certain contexts and its results are reported (Section 2). Subsequently a pattern recognition approach to analyze physiological data is described (Section 3). In Section 4 a new method to represent and process DBNs which makes this methodology available to the use with mobile devices is introduced. Finally, some concluding remarks provide insight on open research questions and future work. 2 Empirical Evaluation of Psychophysiological Measures 2.1 Hypotheses, Experimental Design and Realization

For the implementation of the mobile assistance system two factors are of interest: time pressure and movement. Time pressure is one of the most common stressors in everyday life which restricts our cognitive capacity. The registration of movement by biosensors enables the mobile assistant to reason about the ongoing action of the user. The objective was to identify time pressure and movement on the basis of the psychophysiological patterns and enable a mobile assistant to analyze the user’s state online. In the experimental design four possible combinations of time pressure and movement as visualized in Figure 1 were tested.

No time pressure Time pressure

No

mov

emen

t

HR, EMG, EDA ↑

Mov

emen

t

HR, EMG ↑ (+ movement artifacts)

HR, EMG, EDA ↑ (+ movement artifacts)

Figure 1: Experimental design and hypotheses for movement and time pressure.

To simulate a usual business task, the processing of incoming mail was chosen. The 16 incoming letters, from an Assessment Center task, were divided into four sets, so for each combination four letters had to be executed. In the two conditions without movement, the subjects had to work with the letters as PDF-files in a computer based task. In the other two conditions the subjects were asked to distribute the sheets, after making notes and decisions, to the oddments trays at the other side of the office. During the conditions without time pressure, the subjects were free to spend as much time on the task as they like. When time pressure was induced, a clock ran from five minutes backwards and money was subtracted from the participants’ reward when they overran. 16 subjects (9 males, 7 females) were equipped by 4 sensors (EDA (electrodermal activity), ECG (electrocardiogram), and two EMG (electromyogram) sensors and one ground electrode) from the Varioport system. The EDA-sensor was applied to the more inactive hand. We used an EMG sensor to measure the muscle tension of the left lower arm in case the person was right-handed and vice versa. Another set of EMG electrodes was fixed to the corrugator supercilii, to measure the tension of the eyebrow muscle. From this signal we determine the quantity of eye-blinks by an algorithm. The sensor data was recorded with a sample rate of 256 Hz (256 samples per second). Each subject passed through all four conditions and had to fill out a questionnaire after each trial, to get a subjective appraisal of the induced pressure. The experiment took almost an hour. 2.2 Results

To examine the statistical significance of the mean value differences of the control sample, we used a multivariate analysis of variance (MANOVA) with repeated measures. To determine the influence of the factors time pressure (with grades: no time pressure/time pressure) and movement (with grades: no movement/movement) on the five independent variables EDA, EMG, HR (heart rate), PSD (power spectral density), and Blink, we used standardized data (obtained through a z-transformation). For lack of space we present the results of the sensors EDA and EMG only.

EDA. The evaluation of the EDA signal shows that the standardized data of the galvanic skin response in the computer condition averages slightly below zero (that is about 900mV). Only in the movement condition the EDA discriminates between time pressure and no time pressure of the subject. Under the condition “movement and time pressure”, the galvanic skin response raises at about twice the standard deviation and falls, under the condition „movement and no time pressure“, even more below zero than the average of both the computer conditions. The invariant test for the galvanic skin response under the condition „time pressure” is significant with F(1.15) = 5.547, p < 0.05. It is important to note that the galvanic skin response under the condition “movement without time pressure” averages on a lower value than in both computer conditions. This interaction effect was marginally significant at a value of 0.069. The effect is interesting in conjunction with the information the subjects gave about themselves. On average, the subjects felt the least loaded in the condition „movement without time pressure“. This gives some hint that there is some coherency between their perception of load and the value of the galvanic skin response.

0.3

0.2

0.1

0.0

-0.1

-0.2

-0.3

-0.4No movement Movement

No time pressure

Time pressure

Est

imat

ed M

argi

nal M

eans

of E

DA

0.3

0.2

0.1

0.0

-0.1

-0.2

-0.3

-0.4No movement Movement

No time pressure

Time pressure

Est

imat

ed M

argi

nal M

eans

of E

DA

1.4

1.3

1.2

1.1

1.0

0.9

0.8

0.7No movement Movement

Time pressure

No timepressure

Est

imat

ed M

argi

nal M

eans

of E

MG

1.4

1.3

1.2

1.1

1.0

0.9

0.8

0.7No movement Movement

Time pressure

No timepressure

Est

imat

ed M

argi

nal M

eans

of E

MG

Figure 2: Influence of movement and time pressure on the standard deviation

of EDA (left) and EMG (right).

EMG. To evaluate the EMG data, we examined the standard deviation of the signal, which is a measure for the strength of the activation. As can be seen in Figure 2, activity increases in the case of movement as well as in the case of time pressure. In the invariant test for the EMG sensor, the factor movement is significant with F(1.15) = 33.368, p < 0.01. Although a difference between time pressure and no time pressure is clearly visible, this effect was not significant. Most likely, however, a bigger sample size would have rendered this difference statistical reliable. Consequently, this indicator can be used to estimate the load of a user. 3 Pattern Recognition 3.1 Properties of Psychophysiological Measures

As already mentioned, one of the main advantages of psychophysiological—compared to other—measures is the continuous availability of the body which entails the continuous availability of information about the user. In order to use as much of this information as possible technical systems, measuring a certain physiological value several hundred times per second, have been developed. Consequently, for every physiological sensor a rather large number of values accumulate even over comparably short periods of time. If, for instance, psychophysiological

information is to be considered every second, any physiological measurement may comprise 200 – 500 values. Therefore, even in case the experimental evaluation of a measure (see Section 2) yields that the measure is indicative of the user’s state it is not obvious how exactly the several hundred values are related to the state. In other words, the experimental evaluation just shows that the measure contains information about the user’s state, but does not reveal how this information is represented by the measure. For example, an excessive cognitive load may indicated by the 300 values of one measurement all being high, by only every second of these values being high or even more complex patterns of values. Depending on their complexity it may be quiet difficult, if not impossible, to recognize these patterns with the naked eye. What is more, due to systematic and unsystematic variations of physiology not related to the user’s state (called noise), the relations between patterns and states may be partially obscured. Along with the large number of values constituting one measurement and the possible complexity of the patterns, this noise will in most cases render the identification of the relationship between measurements and states infeasible for a human. Instead, to solve this problem, certain machine learning or pattern recognition (see Duda, Hart & Stork, 2001, for a comprehensive overview) techniques have to be applied. 3.2 Theoretical Considerations

Generally, at least three types of pattern recognition can be distinguished: Template, syntactical and statistical pattern recognition. Since the latter is most suitable to analyze physiological data our approach to identify the relationship between psychophysiological measurements and user states has been based on this formalism. In statistical pattern recognition each measurement is viewed as a single object and each of the several hundred values of a measurement is one property or feature—the value of one property/feature, more precisely—of this object. Furthermore, it is assumed that the values of a measurement are probabilistically dependent on the state, in which they were measured, i.e., some patterns are more probable in some state than in others. If the aim is to determine the correct state with maximal probability, given a single measurement, one should, ideally, assume that state for which the probability to yield such a measurement is maximal. Implicitly, this optimal decision rule partitions the space of all possible measurements in distinct regions each associated with exactly one state. The borders of these regions, called decision boundaries, comprise those measurements for which more than one state with maximal probability exists. It is exactly those decision boundaries that pattern recognition techniques try to estimate in one or the other way. In other words, pattern recognition techniques try to extract from the psychophysiological measurements all the information inherent in them about the decision rule/boundaries regarding the user states. Accordingly, it seems reasonable to consider as much potential information (i.e., as many features of each object) as possible during estimation. As Trunk (1979) has shown, however, taking into account as many properties as possible is optimal only in theory. If the decision boundaries have to be estimated from a limited amount of data, considering one more property may actually lead to less accurate decision boundaries—a phenomenon termed curse of dimensionality. This paradoxical behavior results from the fact that taking into account one additional property not only potentially increases the utilized information, but also the space for which the decision boundaries have to be determined. In the extreme, the respective feature does not help in distinguishing between user states (i.e., bears no information), but nevertheless increases the amount of information to be estimated. Thus, there is a trade-off for considering additional features: Only if the amount of information relevant to distinguish states conveyed by a property exceeds a certain threshold should this feature be used during estimation. Consequently, pattern recognition should be performed with that subset of all available features, which constitutes the optimal trade-off between yielding relevant information and containing to many features. Unfortunately, however, determining such a subset is quiet difficult in itself.

Mainly, this difficulty stems from the fact that optimality may be arbitrarily distributed over the different subsets (cf. Cover & Van Campenhout, 1977). Although, for instance, two features F1 and F2 may allow the best estimation compared to all feature subsets with cardinality 2, adding a third feature F3 may yield a subset worse than any other with cardinality 3. Due to these difficulties a great number of methods to automatically select the best feature subset have been devised (Liu & Motoda, 1998, give an overview). For most of these feature selection techniques, however, it is advantageous or even necessary to know the cardinality of the feature subset to be selected—at least roughly—before starting the selection process. If the cardinality aimed at is chosen to high, the found subset will be suboptimal because of the curse of dimensionality. If, on the contrary, the cardinality is chosen to low, too few features will be used to exploit all the information contained in the measurements. Accordingly, the cardinality searched for should be the minimum number of features such that all relevant information can be represented by this number of features. As an approximation to this lower bound on the number of necessary features the intrinsic dimensionality of the psychophysiological data can be used, since the intrinsic dimensionality reveals the dimensionality of the smallest subspace in which the data can be represented without losing (too much) information. 3.3 A new approach

Ideally, then, pattern recognition to infer the user’s state from physiological signals should comprise three steps: First, intrinsic dimensionality is estimated. Second, based on the intrinsic dimensionality an optimal subset of features can be selected. Finally, using this subset, the decision boundaries are determined. Although the suitability of such a three step approach is well documented in the literature, to the knowledge of the authors, it has not yet been applied in any pattern recognition problem, let alone the analysis of psychophysiological data to infer cognitive and/or affective states. Usually, one particular pattern recognition technique is selected and applied to the data at hand. Even in the rare cases that feature selection methods are used their use is lessened by neglecting intrinsic dimensionality. Furthermore, all approaches of classifying physiological data so far have concentrated on one single pattern recognition technique to analyze their data. This is in clear contrast to theoretical results (cf. Schaffer, 1994) indicating that the usefulness of pattern recognition techniques depends on the data to be analyzed, i.e., there is no single technique which is, in general, more suitable than any other. Said differently, one method well suited for one kind of physiological data (e.g., pupil size) may be especially inappropriate for a different kind of physiological data (e.g., skin conductance). Therefore, it does not seem sensible to rely on a single pattern recognition method; in particular, if one wants to analyze several different psychophysiological signals. Accordingly, in the scope of the BAIR project a pattern recognition approach has been developed which not only realizes all three desirable steps of data analysis, namely estimating intrinsic dimensionality, selecting features, and estimating decision boundaries, but also combines several different techniques in every of these three steps. More precisely, the analysis of a data set starts with estimating its intrinsic dimensionality with a simple Principal Component Analysis, the nearest-neighbor estimation, and Sammon’s Mapping, respectively. The overall intrinsic dimensionality is determined by combining all three estimates. Based on this combined estimate, optimal feature subsets are selected through Sequential Floating Forward Search, First Choice Hill Climbing, and a variant of the Beam Search. Each of the three identified optimal feature subsets is then used to estimate the decision boundaries with a Support Vector Machine a variant of the Learning Vector Quantization, and a decision tree. In this way the most suitable method for each kind of physiological signals can be determined such that the resulting decision boundaries are least prone to inferring wrong user states. What is more, the whole approach is integrated by a graphical user interface with which the different

techniques can be easily selected and combined. Since also all necessary parameter values are preset with reasonable values taken from the literature or can be estimated, even system developers less familiar with pattern recognition should be able to achieve good results with the new approach. Nevertheless, the pattern recognition process can, if need be, parameterized to suit special needs. Thus, the new approach is not only easy to use, but at the same time remains flexible and powerful. Once pattern recognition has been performed, every new measurement can be mapped to a certain user state, i.e., the user’s state can be inferred given physiological measurements. Still, due to noise in the data and since the pattern recognition implemented is probabilistic in nature, the user state may not be determined correctly based on one single measurement. Therefore, several measurements have to be integrated over time as well as with additional information from additional sources to obtain reliable estimates of the current user state. How this can be achieved via dynamic Bayesian networks is discussed in the next section. 4 An Efficient Realization of Dynamic Bayesian Networks The evaluation of dynamic Bayesian networks (DBNs) places heavy demands on computational power and memory. Such an evaluation on a mobile system with limited processing power is therefore not straight forward. Our idea to address this problem is to generate optimized source code for a DBN, given evidence and query nodes. We present a tool that generates source code specialized to process particular queries to a DBN which enables them to run on mobile devices with low computing power. Furthermore, we demonstrate how the produced source code can be modified by the user, for instance, in order to model time dependent conditional probabilities (i.e., the quantitative part of the Bayesian network) or to perform sensitivity analyses. 4.1 Introduction

DBNs are a computational framework for the representation and the inference of uncertain knowledge that takes into account previous states as well as new evidence. They are an extension of Bayesian networks (BNs). A BN can be represented graphically as a directed acyclic graph (see Figure 3 (left)). The nodes of the graph represent the variables. The edges joining the nodes represent the dependencies among them. For each node, there is a conditional probability table (CPT) that quantifies these dependencies.

Figure 3: Time Slice Scheme 1 (left) and Time Slice Scheme 2 (right).

With a DBN, it is possible to model dynamic processes: Each time the DBN receives new evidence, a new time slice is added to the existing DBN. In Figure 3 two buildings blocks (so-called time slice schemes) are shown. The first scheme is used for instantiation of the first time slice and is simply a BN. The second scheme, which is used for each of the remaining time slices, is a BN plus a specification of the parent variables that are taken from the preceding time slice. Each time slice must satisfy the Markov assumption: The future is conditionally independent of the past given the current state. In principle, DBNs can be evaluated with the same inference procedures as normal BNs, but their dynamic nature places heavy demands on computation time and memory. Therefore, it is necessary to apply so called roll-up procedures that cut off old time slices without eliminating their influence on the newer time slices. These procedures may be exact or approximate. In order to compute a roll-up procedure, a probability table representing the belief state has to be maintained: a probability distribution over the state of the system at a given time. This belief state includes at least all of the nodes that are parents of nodes in the next time slice. The representation of this belief state as a probability table raises complexity problems: This table cannot in general be given a factorized representation without loss of information. One idea to realize inference via DBNs in a mobile scenario would be, to send the collected physiological data (e.g., over wireless LAN) to a server with enough computational power. But this solution has several disadvantages: Physiological data has a high and continuous data rate, which means that the wireless connection has to be stable and reliable. This cannot always be guaranteed. Besides this technical aspect, there are also concerns about the privacy of the user, since not every user wants to send her physiological data to the environment. Given these disadvantages, it would be favorable to run the computation on the user's mobile device. We developed an application that enables users to model DBNs and that generates source code (currently Java and C++) based on the polynomial representation of the DBN (see Darwiche, 2003 and Brandherm & Jameson, 2004) as we will describe in the next two sections. This application is called JavaDBN. Because the generated source code is optimized for the given network and the respective application, it can run on mobile systems with low computational power (e.g., an HP iPAQ PocketPC). Furthermore, a runtime guarantee can be given for the source code. 4.2 Polynomial Representation

A number of different procedures exist for evaluating DBNs. One approach is to transform the structure of the DBN into another structure (the so called junction tree), on which the calculations can be made. This transformation consists of complex graph conversions. From the junction tree a polynomial can be derived that represents the DBN. The belief values of the nodes—i.e., the vector that expresses how likely each possible value of the variable is, given the currently available evidence—can be calculated by evaluation (upward-pass) and differentiation (downward-pass) of this polynomial as follows: To every circuit node v two registers vr(v) and dr(v) are assigned. In the upward-pass the values of the vr(v) registers are computed and the circuit is evaluated. In the downward-pass the values of the dr(v) registers are computed and the circuit is differentiated. The technical background and much more elaborated algorithms can be found in Darwiche (2003). The polynomial can be simplified if the evidence and query nodes are known. The evidence nodes are in our case the nodes for which biosensors provide data (evidence). The query nodes are in our case the nodes which estimate the user’s time pressure and cognitive load. To calculate the belief values of the nodes in the DBN, only the (simplified) polynomial is of interest. The above mentioned complex graph transformations can be calculated off-line in a preprocessing stage on a normal PC.

4.3 JavaDBN

In JavaDBN a user can declare time slice schemes and their possible orders of instantiations as well as evidence- and query-nodes (see in Figure 3: Nodes A and B are query nodes, Nodes C and D are evidence nodes) of a DBN. Then a polynomial is determined that permits not only an exact roll-up of old time slices, but also a constant-space evaluation of the DBN. Instead of implementing some generic algorithm, which may allocate and delete data structures at runtime, we decided to put aside the separation of algorithm and polynomial, and to generate source code that is optimized for the respective polynomial. We implemented the afore mentioned simple algorithm for evaluation and differentiation of a polynomial such that the resulting source code remains human-readable and -understandable. All of the variables necessary for the computation are created on initialization of the application. No variables are created or disposed at run-time to avoid the activation of garbage-collection. Special functions to set evidence are provided for selected nodes. Specific methods for evaluation and differentiation of the polynomial were generated. They must be called in a certain order, which is provided by a function that has to be called in order to do the inference in the DBN. Afterward the belief values of interesting nodes can be read out and the roll-up can be executed with the corresponding methods provided by the source code. New evidence can be entered after a roll-up, such that new estimations can be computed, incorporating the impact of precedent (rolled-up) time slices without approximations. By putting aside the separation of inference algorithm and DBN representation, the structures on which the calculations occur get more transparent and can be influenced. This easily enables the user to change the source code. In doing so, he can realize features that would be hard or even impossible to achieve if the inference algorithm were precompiled, receiving the DBN as input. Constant CPT entries can be changed into variable entries through replacing arrays with method calls. This is necessary if, for example, CPT entries are unknown in advance or if they can change during runtime. A method call could, for instance, realize a database access (e.g., to read the CPT entries out of a user profile) or it could change the entries according to the time that has passed since the last instantiation. A sensitivity analysis makes sense if one wants to find out which CPT entries of which nodes in the BN have to be modeled very accurate and which nodes can rather be neglected. JavaDBN can also generate source code that can directly be used for such analyses. For the BN in Figure 4 (left) we parameterized the a priori probability of node A in order to determine the impact of changes of the a priori probability on the other nodes in the BN when no evidence is given. The results for values of the nodes B, C, D and E are visualized in Figure 4 (right). It shows that the belief value of each node is linearly depending on A1 and the CPT entries of Node A do not have to be very accurate for the nodes C and E whereas they have to be as accurate as possible for the other nodes in the BN.

Figure 4: Bayesian network (left) and sensitivity analysis (right).

5 Conclusions In this work, we have described how we analyze and process complex psychophysiological user data to obtain information about the current user state for a user-adaptive assistance system. Via experiments, we determined which physiological sensors in a given context determine our relevant states of pressure and movement. We further use a novel pattern recognition approach to specify the exact (inter-)relationship between physiological data and the discrete user states, which we finally use to model DBNs. These DBNs integrate this information over time and are able to include external information for a reliable estimation of the user state. Our implementation of the DBNs is able to run on mobile devices with low computing power, allowing us to deploy a self-contained system on a mobile, off-the-shelf device for a variety of applications. We are currently working on the integration of all three parts: Our experimental data has to be subjected to our methods for determining classifiers that interpret variables such as EDA, EMG, and PSD. The results of these classifiers need to be further processed by DBNs, which also have to be modeled in this process, whereas their structure and also their CPT entries do not only depend on the resulting classifiers but also on the experimental data. The final user-adaptive personal assistance system has to be evaluated formally with a similar experiment in a mobile context under real-life conditions as opposed to the laboratory setup of our initial experiment. 6 References Brandherm, B., & Jameson, A. (2004). An extension of the differential approach for Bayesian network inference to dynamic Bayesian networks. International Journal of Intelligent Systems, 19(8), 727-748. Cover, T.M., & Van Campenhout, J.M. (1977). On the possible orderings in the measurement selection problem. IEEE Transactions on systems, man and Cybernetics, 7(9), 657-661. Darwiche, A. (2003). A differential approach to inference in Bayesian networks. Journal of the Association for Computing Machinery, 50(3), 280-305. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification. New York: Wiley. Kramer, A. F. (1991). Physiological metrics of mental workload: A review of recent progress. In D. L. Damos (Ed.), Multiple-task performance (pp. 279-327). London: Taylor and Francis. Liu, H., & Motoda, H. (1998). Feature selection for knowledge discovery and data mining. Boston: Kluwer. Schaffer, C. (1994). A conservation law for generalization performance. In W. W. Cohen & H. Hirsh (Hrsg.), Proceedings of the Eleventh International Conference on Machine Learning (pp. 259-265). San Francisco: Morgan Kaufmann. Schultheis, H., & Jameson, A. (2004). Assessing cognitive load in adaptive hypermedia systems: Physiological and behavioral methods. In P. De Bra, W. Nejdl (Eds.), Adaptive Hypermedia and Adaptive Web-Based Systems (pp. 225-234). Berlin: Springer Verlag. Trunk, G. V. (1979). A problem of dimensionality: A simple example. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1(3), 306-307.