Identifying Challenges in Implementing Cognitive Systems
Engineering Analyses into Complex Systems Engineering
Jerry A Gordon
Embry Riddle Aeronautical University
Department of Human Factors and Systems
29 November 2010
Dr Shawn Doherty
Page 1
Table of Contents
INTRODUCTION..........................................................4
SYSTEMS ENGINEERING TEAM MENTAL MODELS....................................5FUNCTIONAL MODELING IN SYSML............................................7COGNITIVE SYSTEMS ENGINEERING............................................9THE FUNCTIONAL ALLOCATION PROCESS........................................11USING AND COMPARING MODELING SCHEMA......................................12DEFINING COMPLEXITY IN FUTURE PROBLEM DOMAINS.............................13PROBLEM STATEMENT.....................................................16
METHOD...............................................................17REQUIRED MATERIALS....................................................18SCENARIOS AND PARTICIPANTS..............................................19VARIABLES AND DATA COLLECTION...........................................20
References...........................................................21
Page 2
List of Figures
Figure 1 Function Block Diagram. This Figure illustrates
hierarchical nature of Functional Decomposition...........8
Figure 2 Example Activity Diagram used as EFFBD. This figure shows the decomposition of multiple levels................9Figure 3 Components of the Joint Cognitive System. This figuredepicts the differences in CSE and Technical perspectives..........................................................16Figure 4 Research Question and Hypothesis..................17
List of Tables
Table 1 -................................................14Table 2..................................................18
Page 3
Introduction
In recent years, Human Factors practitioners are
increasingly called to participate in complex system
development early in the development process. There are
certainly examples of systems which have involved
practitioners from both human and technical domains, and
this joint participation has improved the overall utility,
cost effectiveness and performance of these systems.
However, in other programs to where this participation has
been more difficult, and the overall value of including
human engineering not fully realized, the characteristics
leading to this failure are less well understood. In order
to explore these indicators for success, this thesis will
leverage mental model theory, and argue that graphical
models of a system developed by technical and human factors
practitioners are equivalent to some portion of the shared
team mental model. The central claim of the study is that
there will be a relationship between the characteristics
that make a system an open, complex system, and the degree
of agreement between concepts captured in the models
developed by technical and human factors practitioners.
The introduction starts with a justification for common
mental models between the technical and human factors
practitioners in the systems engineering team. Once the need
for common mental models is established, a basis for
measuring and comparing them must be developed. To that end,
the next sections start with traditional systems
engineering, and then describe the mechanisms employed by
the HF practitioners of interest here, “cognitive systems
engineers”. A description of the operational interface
between the two groups, “functional allocation”, follows
with a proposed mechanism to compare the two perspectives.
Since the overarching problem is tied to the development of
complex systems, to identify the challenges involved in
bringing CSE and traditional engineers together to develop
them, the introduction section concludes with a candidate
problem set and a summary of the overall goal of this
research.
Systems Engineering Team Mental Models
Empirically based science, as a process which moves from a
problem definition to a testable hypothesis, describes a set
of conditions, which when sensed, should yield predictable,
observable results. This relationship between sensed
conditions and expectation of future states is analogous to
Mica Endlsey’s truth model used in mental model research
involving situation awareness (Pew & Mavor, 1999). Similar
components for recognition of the situation and projection
of future states are present. Proceeding from the notion of
a “design as hypothesis” (Dekkar, Nice & Hoffman 2003), it
is possible to see the notion of a “system design” as a
critical component of the mental model of the team
engineering it.
A critical premise of this research is that engineering
teams, like any other team, rely on shared mental models to
conduct their work (Avnet, 2009). These mental models
involve both the understanding and tracking of
organizational processes used to manage the technical and
programmatic effort, as well as the system structure,
utility and behavior described by the system design as it
evolves towards final product. It is this understanding of
the system design which is the subject of the proposed
experiment.
The Object Management Group (OMG) was formed in the 1990’s
by three leaders in software engineering at IBM and proposed
the Unified Modeling Language (UML). The UML was originally
intended as an aid for software developers to exchange
design information, and in the ensuing years, it has matured
significantly. It has been adapted into the Systems Modeling
Premise #1 – Successful Performance of Human Factors and Technical practitioners working in concert on a system design team require congruence in their shared mental model of the state and purpose of the design.
Language (SYSML) as well, to capture the design elements of
not only software, but hardware, humans and other elements
of the system. The use of UML and SYSML provides a
semantically precise mechanism for capturing the elements of
design: utility, structure and behavior, as well as the
nature of the relationships between these elements
(Friedenthal, Moore & Steiner, 2008). These languages have
evolved sufficiently to enable systems design through a
“Model Based Systems Engineering” (MBSE) process, where the
subjective interpretation of the natural language in prose
specifications can be replaced with fixed relationships,
semantically precise graphical model elements and a system
of logical tools for validating the completeness and
consistency of these models. It is because of this richness
that this research assumes that validated UML/SYSML models
can be used to measure the mental models of design
practitioners working on a system.
Premise #2 – UML/SYSML graphical models are equivalent to the shared mental model of the current state and
The focus of this experiment is to compare the content of
design as it is understood by the technical and human
factors practitioners. However, there is an important
relationship between the flow of management and control
events (as they relate to team mental models) and the
creation of design artifacts. Classical systems engineering
(Kossiakoff & Sweet, 2003) describes this relationship as a
linear flow from utility (i.e. stakeholder requirements), to
behavior (i.e. functions) to structure (i.e. component
design). Complex systems engineering requires a non-linear,
iterative relationship, (Minai, Braha, & Bar-Yam, 2006) but
the basic artifacts of design still exist at some level of
maturity throughout.
Functional Modeling in SYSML
Systems engineering and software engineering originally
relied on a “structured” method, whereupon the system
functions were first decomposed and then allocated to
components. Structured methods would create a hierarchical
breakdown of system functions, which would then be allocated
to either hardware or software, and then to specific
components (Kossiakoff & Sweet, 2003). The Functional Block
Diagram (FBD) hierarchy used in structured methods is
depicted in figure 1.
Function 1.0
Function 1.1 Function 1.2 Function 1.3
Function 1.1.1 Function 1.1.2
Function 1.0
Function 1.1 Function 1.2 Function 1.3
Function 1.1.1 Function 1.1.2
Figure 1 Function Block Diagram. This Figure illustrates hierarchicalnature of Functional Decomposition
Current methods for systems engineering, evolving out of
software engineering, rely on Object Oriented (OO) methods.
Both Object-Oriented and Structured methods rely on the
notion of a “function” as a core construct defining system
behavior. Within UML and SYSML, the modeling language used
for object oriented methods, functions are defined within
activity diagrams. UML version 2.1 extended the earlier UML
activity diagram to include all of the information contained
within Enhanced Functional Flow Block Diagrams (EFFBD) used
by the earlier structured approaches (Friedenthal, Moore &
Steiner, 2008). UML activity diagrams still represent
decomposition with increased detail at each level, but they
convey more information as the relationship between
functions and data flows between functions, as well notions
of sequencing. An example EFFBD is depicted in figure 2.
Start CarPower From engine
Insert Key
Provide Start m otionDecouple Start signalfrom Engine
Transm it Start m otion to engine
Interlock signal
Torque couple
Level 1 decom position
Level 2 decom position
Figure 2 Example Activity Diagram used as EFFBD. This figure shows the decomposition of multiple levels
Cognitive Systems Engineering
Increasing levels of complexity in modern systems
development has led to the formation of a number of
disciplines and techniques to address this complexity.
Cognitive systems engineering (CSE) is a set of techniques
which draws on human factors methods and scientific
principles with the intent of addressing human issues in the
development of complex socio-technical systems directly
through the requirements process (Militello, Dominguez,
Lintern & Klein, 2010). CSE is a relatively new discipline,
and it continues to vie for acceptance as part of the
overall systems engineering process, although it has enjoyed
some early successes (Madni, 2010). A key portion of the CSE
process is the analysis of the human work involved in the
operations of the socio-technical system being designed.
CSE analysis techniques rely heavily on modeling, including
various approaches for describing the nature and purpose of
work, human performers of the work and information content
of the work space. These techniques evolved from the family
of Cognitive Task Analysis (CTA) techniques developed in the
1980’s by Jens Rasmussen and Kim Vicente to develop user
interfaces and training systems, among other uses (Jonassen,
D. Tessmer, M. & Hannum, W., 1999). One mechanism for
modeling cognitive work is the goals/means decomposition,
which describes a hierarchy of why work is performed tied to how
this work is performed (Jonassen et al, 1999).
One of the most recent approaches to CSE is the Applied
Cognitive Work Analysis (ACWA). While grounded heavily in
theory, the ACWA has evolved as a pragmatic answer to
addressing specific challenges in ensuring that complex
technical systems were developed with “affordances”; where
the work was designed to “intuitively fit” its users
(Hollnagel, 2003). A key construct of the ACWA process is
the development of a”functional abstraction network” (FAN).
This network describes the “goals-means decomposition” of
the workspace, and becomes the scaffolding upon which the
cognitive and information demands and characteristics of the
workspace are built. This goals-means decomposition is a
behavioral or functional description of the system, from the
perspective of the CSE/Human Factors practitioners
developing it.
One of the key elements of the workspace which are mated to
the FAN is the information and relationships requirements
(IRR). CSE practitioners have developed their own notations
for capturing the concepts of functional abstraction and
information exchange. However, this research assumes that
the combination of the elements within the FAN and the IRR
are functionally equivalent to the behavioral concepts
defined by the technical practitioners using UML and SYSML
modeling languages, and that a model translation algorithm
can be defined to capture FAN and IRR concepts as UML/SYSML
concepts.
The Functional Allocation Process
The functions identified during the functional analysis
phase of design (either the EFFBD of the technical
practitioners or the FAN of the CSE practitioners) must be
assigned to components that will satisfy them. In classical
systems engineering, this is a formal process that follows
only after functional analysis (through the requirements
definition process) is essentially complete. Similarly, in
the classic sense, human factors functional allocation
Premise # 3 – ACWA Functional and Information Requirements used by CSE practitioners can be mapped to equivalent concepts defined by the UML and SYSML meta-models.
proceeds from requirements to identify the impact on human
users, by mapping human task work to the software and
hardware functions that are supported by it. CSE, however,
in developing complex systems seeks to perform a “functional
synthesis” (Woods, D. & Hollnagel, E 2006), whereupon the
process is iterative with functional analysis, to more
correctly match the nature of (hardware and software) system
functions to human task-work in a more integrated whole that
optimizes the “total system” performance.
Thus the functional analysis and allocation process will
dictate in large measure what gets built, by defining what
it is expected to do. As the resultant hardware and software
is used by humans in a real-world operational setting, the
actual utility of the system and the capabilities it
provides becomes apparent. The process of systems
engineering evolved to try and focus on developing those end
state capabilities in a disciplined fashion, and CSE evolved
from the discovery that for systems of increasing
complexity, human concerns were becoming dominant in
dictating the utility and form of those final capabilities.
However, practical development of any real system requires
both sets of practitioners, and sustained performance of
this CSE/technical team requires that both sets develop and
maintain shared mental models of what is being built and why.
Using and Comparing Modeling Schema
One of the strengths of the UML is that it is a validated
internally consistent symbolic representation of the real
world. The validation of internal consistency was performed
by running a set of algorithms called Descriptive Logic (DL)
on the concepts, and relationships between them
(Nantajeewarawat, E. & Wuwongse, V. 2005). These algorithms
can demonstrate that the concepts in the language are
unambiguous and have natural start and stop points, as well
as prescriptive rules for combinations and compositions. The
same algorithms can be run against any model developed in
the language to determine that it is complete and
unambiguous as well. UML is implemented in a number of
toolsets provided by multiple vendors, and often include
model consistency checkers. UML models are intended to be
portable through multiple tools using the full definition of
the language by the use of Extensible Markup Language Meta-
Data Interchange (XMI) standards.
Similarly, Nantajeewarawat and Wuwongse have proposed a set
of metrics for comparing the consistency between two models
developed in the UML language, by using extensions of the DL
algorithms. Part of the consistency check s associated with
placement of concept characteristics in the various model
layers defined from the generic abstract to specific project
instances. For the sake of this research, the same tool
vendor will be used for all data development, so
inconsistencies in meta-model enhancements will be the same
for all datasets. However, a consistency check associated
with horizontal and vertical consistency; that concepts
defined in the model have the same parent/child and peer to
peer relationships, as well as semantic consistency, will be
key to understanding the mental model being measured in the
UML graphical model. This separate set of algorithms must
be developed to perform data analysis.
Defining Complexity in Future Problem Domains
Complexity is a term often used in engineering, but there is
little agreement as the exact definition. Similarly, while
there is recognition that there is a class of “complex”
systems which require qualitatively different approaches to
engineering that traditional systems engineering (Minai,
Braha Bar-Yam 2006), there a great deal of discussion on
specific criteria to classify a system as “complex”. A
generic abstract definition is that complexity is the
uncertainty involved in achieving (which may include proper
“understanding of”) stakeholder requirements (Suh, 2001).
There are a number of similar definitions that all stem from
interpretations of Shannon’s information theory.
Characteristics which present a more practical definition
from the chair of the International Council on Systems
Premise #4 – Descriptive Logic Algorithms run on UML/SYSML models can be used to measure the degree of similarity between the models
Engineering (INCOSE) Complex Systems Working Group (Sheard &
Mostashari, 2009), and the specific examples which will be
used for this research, is presented in table 1.
Table 1 - Mapping of Complex System Characteristics to Experiment System Definition
INCOSE Definition Interpretation for ExperimentComposed of autonomouscomponents High level of automated functions
Self organizing Co-located with users empowered to establishpriorities and missions
Emergent behavior - non linear
Difficulty in describing monotonic Measurements of Effectiveness (MOE) i.e. Q: what does it do? A: it depends
Adapt to environment Can be reconfigured in real timeIncrease in complexityover time Can add additional components in real time
A class of problems that is of topical interest and satisfy
all the characteristics of “complexity” is unmanned aircraft
systems (UAS). UAS are composed of remotely controlled
aircraft with varying levels of onboard automation (Sheridan
& Parasuraman, 2006), but those controllers are human
operators, who may utilize varying levels of automation in
their “ground control stations” (Cummings, Kirschbaum,
Sulmistras, & Platts, 2006). Moreover, UAS are designed to
carry a number of different payloads to perform a variety of
missions, and they have multiple levels of control which can
be passed between different human users and levels of
automation, in a large number of ultimate configurations.
During the past decade of operations in the Middle East, UAS
have gone from novelty item to a principal warfighting
platform. There number and utility is expected only to grow
in the future. CSE and related HF techniques are going to
be essential to the proper development, fielding, operations
and maintenance of these systems in the future. However,
this will require CSE to proceed from its current “initial
enthusiasm” (Madni, 2010) to a mature, repeatable discipline
with demonstrated Return on Investment (ROI). This will
require among other things, a proper understanding of the
characteristics of a system which will necessitate its use,
and, as the subject of this research, the characteristics of
those system development efforts which affect the success or
failure of introducing CSE principles in the large,
distributed teams that are often used to develop Defense and
Aerospace systems.
The impact of advances in automation on the human components
in systems is a well researched topic (Sheridan, &
Parasuraman, 2006). It has had an impact on the approaches
taken for functional analysis and allocation. The functional
allocation problem started with the “Machines and Better
at/Humans are Better at” approach that substituted machine
tasking for human tasking. It has migrated to recognition
that simple substitution is insufficient, and allocation on
that basis alone creates its own problems, such as
automation surprises (Woods & Hollnagel, 2006). For earlier
methods such as human centered design (Hollnagel, 2003).,
the technical and human factors perspectives still had the
human machine interface (HMI) as a common point of
reference, and it is reasonable to assume that the different
groups could use this common point to maintain consistent
mental models regarding the nature of the design; what they
were building and why. The requirements of a “joint
cognitive system”, which takes a qualitatively different
approach to understanding the nature of the system, still
has the HMI, but it is a final conclusion rather than a
starting point. In this case, the resultant understanding of
the system from the CSE perspective is likely to be very
different that the technical understanding. These elements
are depicted in figure 3.
Figure 3 Components of the Joint Cognitive System. This figure depicts thedifferences in CSE and Technical perspectives.
Hum an Contribution-cognitive and physical task work
M achine Contribution-Hardware and Software FunctionsOrganization as J oint Cognitive S ystem
Hum an M achine Interface
Social Network
Open System s Undergo Changes in Com position
Autom ation
CSE perspective starts here and works inward
System boundary
Problem Statement
The problem to be attacked in this research is trying to
understand the degree of shared understanding between
traditional engineers and CSE practitioners as they attack
the development of systems. It presumes that the use of
UML/SYSML tools to develop validated models of the system
under development can be used to measure the state of the
mental models of the two groups of practitioners. Using the
characteristics of complex systems defined previously in
table 1, the experiment will compare the degree of
consistency between the models developed separately by the
two sets of practitioners on two systems of similar size,
where one exhibits the characteristics of complexity, and
the other does not. The experiment will show that for the
non-complex system, the initial degree of consistency is
high. From this it can be assumed that putting the systems
engineers and CSE practitioners on a common team, that the
two models could be synchronized with relative ease. For the
complex system, this degree of initial consistency will be
low, and this discontinuity, exacerbated by the geographic
separation of distributed teams, may take so long or cost so
much to properly synchronize that the project budget and
schedule cannot sustain it. It is in these cases that the
proper value of CSE may not be realized, and cases such as
these will hamper its general acceptance and adoption. The
research question and hypothesis are listed in figure 3.
Figure 4 Research Question and Hypothesis
Method
The experiment will start with the creation of two notional
systems definitions. These systems will be based on two
battlespace deployments of UAS, one deployed on a ship for
use in maritime missions and one to operate in conjunction
with ground forces. These definitions will be modified to
represent the controlled levels of complexity, and will form
the basis for the design efforts which are being monitored.
Once the groups of participants are organized and trained,
the design efforts will be executed. The Systems engineering
Research Question Q1: Does the degree of complexity in a notional system definition affect the correlation between the functional architecture required to satisfy that system definition as understood separately by human factors and technologist practitioners.
H1: The level of agreement between the two team types will be lower in the complex case than in the closed case
and CSE teams will each have a domain appropriate toolset
for capturing the graphical models of their design, and this
same toolset will be used by teams of the same type
throughout. Following the design efforts, the CSE models
will be converted using the DR translator to UML/SYSML
models, equivalent to those used by the systems engineers.
The models will be validated, and extraneous concepts
deleted from the model repositories. The final models will
be compared for horizontal, vertical and semantic
compatibility on each level of complexity. Final results
will be analyzed and recorded.
Required Materials The systems engineering team will be provided the Enterprise
Architect toolset developed by Sparx Systems®. This toolset
will be used to capture a functional model of the system
definition using activity modeling. The system definitions
shall be captured in a requirements model provided to the
teams(although the definition will not be “requirements” in
the classical sense, requirement objects and block objects
are necessary for the model to pass the validation checker).
The CSE team will be provided a database tool for capturing
the elements of the Functional Abstraction Network and the
Information Relationship Requirements. The CORE® tool
developed by Vitech Corporation will also be provided to
provide a graphical aid for populating the database. The CSE
team will receive the same block and “requirements”
statements describing the system definition. Each team will
be provided access to a different UAS subject matter expert
who can answer technical questions. They will be located in
a laboratory or classroom environment for the duration of
the design exercise. Each design excursion will require
about 8 hours of work. Data reduction tools will include
SPSS to perform the analyses, the DL algorithms used to
validate the model, and a separately prepared test
instrument to perform text searches on the model components
to determine the values of the Dependent Variables (DV)
described in a follow-on section.
Scenarios and ParticipantsThe CSE and systems engineering (SYSENG) participants shall
be selected based on survey evaluations. The participants
should be technically competent but not highly experienced
in development of complex systems. Graduate Students who
have completed pre-requisite classes in systems engineering
and human factors analysis would be candidates. Each team
type will be split into two groups. The group size should be
even, with at least three members. The groups will be
separated to achieve a relatively uniform average and spread
for experience level. For counter balancing, each group will
receive different ordering of the complexity levels.
The scenarios are defined in the maritime and ground
domains. The system definitions will be created by a joint
team of HF and system engineering experts, to include
opportunities for approximately 40-60 leaf node functions
and at least three levels of decomposition. While artificial
constraints might be placed on the definition to make it a
“closed” system in this case, the participants are only to
address the open-system issues presented in the system
definition. The closed system shall have none of the
attributes listed in table 1. The highly automated system
will have at least 10% of its notional functions intended to
be automated. The open system will include desired
capabilities that present all of the features included in
table 1. The scenario presentations for each participant
group are summarized in table 2.
Table 2System Definition Scenario Presentation
participantgroup
BattlespaceDomain Complexity Level
OrderPresented
CSE 1 Maritime Closed 1CSE 2 Maritime Highly Automated 2CSE 1 Maritime Open/Complex 3CSE 2 Ground Closed 3CSE 1 Ground Highly Automated 2CSE 2 Ground Open/Complex 1SYSENG 1 Ground Closed 1SYSENG 2 Ground Highly Automated 2SYSENG 1 Ground Open/Complex 3SYSENG 2 Maritime Closed 3SYSENG 1 Maritime Highly Automated 2SYSENG 2 Maritime Open/Complex 1
The testing of the first hypothesis, looking for significant
difference (i.e. non agreement) between the models developed
by the CSE and SYSENG teams, is a between-groups analysis.
The testing of the second hypothesis, looking at the
relationship between the agreement of models developed by
the teams as the complexity level grows, is a within groups
analysis. The design is full factorial. Not every group will
see every combination of battlespace domain and complexity
level, but battlespace domain was varied to minimize
training and boredom residual effects, it is not a control
variable. The control group is the baseline “closed”
scenario as conducted for the four participant groups.
Variables and Data CollectionThe Independent Variable (IV) for the experiment is the
complexity of the system definition that is to me modeled.
It has three levels, as described above, closed, automated
and open. The Dependent Variables (DV) are the degrees of
agreement between the models developed by the CSE and SYSENG
teams. Three types of agreement are being evaluated:
DV1 Vertical agreement – degree to which concepts are
listed at the same level of abstraction such that
parent/child relationships are comprehended the same
way.
DV2 Horizontal agreement – degree to which concepts or
organized under the same parents within a given level,
such that peer to peer groupings are interpreted the
same way.
DV3 Semantic agreement – degree to which the same
concepts are decomposed
Semantic agreement is the most difficult to detect
automatically because small inconsequential changes between
teams may make ordinary text searches find “differences”. In
order to minimize these effects, a limited dictionary of
terms shall be provided to all team, strict non – verb –
object and tense rules will be used in the creation of
function definitions, and lower level concepts that
represent additional “non significant” decomposition levels
will be removed from the dataset. Two teams are used to
establish a baseline of natural variation in approach of
teams of the same type (i.e. SYSENG and CSE). For the level
comparison, agreement with either teams concepts will be
considered in-level agreement.
Even though there is only one IV, this study can evaluate
between groups effects because two different groups of each
practitioner type will execute models at the same level of
complexity. This is to evaluate whether the variability in
agreement due to level exceeds the natural variability
between teams of the same competency. Further studies might
examine the effects of other components of complexity
individually, but for this study, that was considered
infeasible. Automation was broken out as an individual level
to study as early work describing the “increased complexity”
of systems, the impact on systems design, and the need for
inclusion of human factors engineering, focused on
automation to the exclusion of the other aspects of “complex
open systems”.
References
Avnet, M. (2009). Socio-cognitive analysis of engineering systems design:
shared knowledge, process, and product. Unpublished Doctoral
Dissertation, Massachusetts Institute of Technology.
Retrieved 26 November, 2010 from
http://esd.mit.edu/people/dissertations/avnet.pdf
Cummings, M., Kirschbaum, A. , Sulmistras, A. , & Platts, J.
(2006). STANAG 4586 Human Supervisory Control Implications. Air and
Weapon Systems Dept, Dstl
Farnborough & the Office of Naval Research, Retrieved 26
November, 2010 from
http://web.mit.edu/aeroastro/labs/halab/papers/
STANAG_UVS.pdf
Dekkar, Nice & Hoffman (2003). From contextual inquiry to
designable futures: what do we need to get there?. IEEE
Intelligent Systems 18(2). 76-80. Retrieved from
http://www.ihmc.us/research/projects/EssaysOnHCC/DesignableF
utures.pdf
Friedenthal, S., Moore, A. & Steiner, R. (2008). A Practical
Guide to SYSML, The Systems Modeling Language. Ma. Morgan Kaufman
Press.
Hollnagel, E. (ed) (2003). Handbook of Cognitive Task
Design. NJ: Lawrence Erlbaum Associates.
Jonassen, D. Tessmer, M. & Hannum, W.(1999). Task Analysis
Methods for Instructional Design. NJ: Lawrence Erlbaum Assoc.
Kossiakoff & Sweet (2003). Systems Engineering Principles and Practice.
NJ: John Wiley and Sons
Madni. A. (2010). Integrating Humans with Software and
Systems: Technical Challebges and a Research Agenda. Systems
Engineering, The Journal of the International Council on Systems Engineering
13(3), 232-245.
Militello, l., Dominguez, C, Lintern, G. & Klein, G.(2010).
The Role of Cognitive Systems Engineering in the Systems
Engineering Design Process. Systems Engineering, The Journal of the
International Council on Systems Engineering 13(3), 261-273.
Minai, A. , Braha, D. & Bar-Yam, Y. (ed) (2006). Complex
Engineered Systems Science Meets Technology. NY: Springer.
Nantajeewarawat, E. & Wuwongse, V. (2005) Knowledge-Based
Inconsistency Detection in UML Models. In Chang, S.(ed) Handbook of
Software Engineering and Knowledge Engineering, Vol 3.
England: World Scientific Publishing.
Pew, R. & Mavor, A.(ed) (1999). Modeling Human and Organizational
Behavior. Washington DC: National Academy of Sciences
Press.
Sheard & Mostashari (2009). Principles of Complex Systems
for Systems Engineering. Systems Engineering, The Journal of the
International Council on Systems Engineering 12(4). 295-311.
Sheridan, T., & Parasuraman, R. (2006). Human-automation
interaction. Reviews of Human Factors and Ergonomics, 1, 89-
129.Parasuraman, R. (2000). Designing automation for human use: Empirical
studies and quantitative models. Ergonomics, 43, 931-951. Retrieved 26
November, 2010 from
http://archlab.gmu.edu/people/rparasur/Documents/Sheridan-
ParasuramanRevHFES06.pdf
Top Related