1 Using Converging Methods to Reveal Hidden Systems ...

1

Using Converging Methods to Reveal Hidden Systems-based Coaching Decisions and

Interventions in Sports to Improve Team Performance

Thesis

Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in

the Graduate School of The Ohio State University

By

Carmen Grande Pardo

Graduate Program in Industrial and Systems Engineering

The Ohio State University

2020

Thesis Committee

Dr. Michael F. Rayo

Dr. David D. Woods

2

Copyrighted by

Carmen Grande Pardo

2020

ii

Abstract

There is a large body of systems literature that studies and learns from adaptive

team performance in high-complexity domains (nuclear, healthcare, aviation). However,

there is little to no systems research on arguably the most popular team setting: organized

sports. Using converging methods, this study looked at (1) patterns of adaptation that are

revealed when using a resilience engineering lens to look at intra- and inter- game

coaching decisions, and (2) the strategies that coaches use to affect individual and system

performance on a collegiate basketball team. The specific concepts borrowed from

resilience engineering used to view this domain were common ground, mutual

directability, interpredictability, polycentric control and being poised to adapt, amongst

others. A resilience engineering lens was used due to the fact that it is known to be

critically important to other complex adaptive systems.

Decision-making and adaptations in sports are commonly thought to be targeted

towards each individual. Through structured interviews, this study found that there are

hidden triggers and decision-making processes that are system-focused rather than

individually-focused when coaches assess the need for an adaptation and execute it. In

addition, the existence of different time scales to implement and execute adaptations was

found to be a critical point that coaches exploit in order to increase the readiness to adapt

iii

during high tempo periods. Basketball has high varying tempos on multiple time scales,

from longest to shortest: practices and film sessions between games, halftime breaks,

timeout breaks, play breaks and in-play breaks. This study identifies various hidden

system level strategies that coaches use in order to achieve successful team performance

across the different time scales: increasing the levels of common ground, directability,

predictability, readiness to adapt and polycentric control. For example, this research

found that the development of coded terminology during low tempo periods can help a

system implement adaptations in very short timeframes during high tempo periods, which

is a key to successful performance. The paper will present a detailed explanation and

various examples of how these strategies translate to a basketball game. This research

calls into question much of the conventional wisdom assumptions about the importance

of individual traits (e.g. agility, speed, strength) and shows how often decisions are made

that reduce the overall team performance on these dimensions, but improve aspects of

team or system performance (e.g. coordination, readiness to adapt, predictability, etc.).

This paper will present the importance of considering these strategies and patterns in the

future when trying to advance the development and design of complex adaptive systems

that are resilient in face of decompensation.

iv

Dedication

Dedicated to my mom, dad and brother, Ángeles Pardo Izquierdo, José Grande

Ramos and Jorge Grande Pardo. Thank you for the endless advice and car rides to

practices, games and school. Thank you for encouraging me to learn a new culture while

chasing my basketball dreams. Thank you for being my biggest supporters to lean on and

for listening to all my crazy stories during these last five years through phone calls all the

way from the other side of the world. Thank you for teaching me how to work hard, be

compassionate and look out for others, and for encouraging me to learn more, discover

more and have more fun.

Also dedicated to my grandmother, Angelina Izquierdo Sánchez. Through your

sacrifice and your values, you have paved the path for us future generations to continue to

discover the world, learn and chase our dreams. None of this would have been possible

without you. You are dearly missed.

v

Acknowledgments

I would like to acknowledge and thank Dr. Michael F. Rayo for believing in me

since the very beginning and giving me the mentorship and opportunities that I needed to

grow as a person and in this field. His support, both on the court, in the classroom and in

my growth as a human being has been unmatched and is very much appreciated.

I would also like to thank and acknowledge Morgan E. Reynolds. Without her

advice, help and support I would not have been able to grow as much as I have grown

during the last four months. Throughout her life and career, she is going to make an

impact in so many people’s lives, and I am lucky to have been one of the first ones.

Special thanks to Morgan Fitzgerald, who has held us all together during our toughest

times. She has no limit to what she will achieve.

Special thanks to Cheyn S. Roux, for being my shoulder to cry on, my rock for

the past year and my home away from home. The future awaits for him and his support

and love are appreciated.

I would also like to acknowledge my fellow members and friends of the Cognitive

Systems Engineering Lab who have made many contributions along the way. Thank you

all for your support, the many laughs and all the fun we have had working together. How

you all have embraced this quasi-retired basketball player means the world. Special

vi

thanks to Dr. David D. Woods for his contributions, conversations and genuine support.

Thank you for also believing in me since the beginning.

I would like to also thank and acknowledge The Ohio State University Women’s

Basketball team for their continued collaboration and efforts. Thank you for giving me

the opportunity to represent this amazing university and further my education and

research. I would like to especially thank Kevin McGuff, Carrie Banks, Tamika Jeter,

Carla Morrow, Makayla Waterman, Ryan Murray and Beth Howe for their continued

participation and support during my research, as well as the endless basketball

conversations.

Lastly, I would like to thank my friends and teammates who have made The

United States my home for the last five years: Inés Mata, Alana Aguirre, Adreana Miller,

Savitha Jayaraman, Rachel Pereira, Leah Walker, Sydney Mitchell, John Paul Turner,

Sarah Swiderski, Erinn Casey, Molly and Tyler Bradfield, Helena Alonso, Beatriz

Chozas, Paula Vázquez, Josh Stewart, Joe Brown, Andrew Stroh, Daniel Montoliu and

Hannah Duszynski. They say that home is where the heart is. These people have my heart

and will always be home to me no matter where I am. Their continued friendship and

support through the years and different States is very appreciated.

vii

Vita

2016 – 2017……………………………… Summer Camp Basketball Coach, Ball State

University

2017……………………………………… B.S. Industry and Technology, Ball State

University

2013 -2019 ……………………………… Summer Camp Basketball Coach, Econvive

Association, Ribadeo, Asturias, Spain

2019 - present……………………………. Graduate Research Assistant, Cognitive

Systems Engineering Lab, Department of

Industrial and Systems Engineering, The

Ohio State University

Fields of Study

Major Field: Industrial and Systems Engineering

Major Field: Cognitive Systems Engineering

viii

Table of Contents

Abstract ............................................................................................................................... ii

Dedication .......................................................................................................................... iv

Acknowledgments............................................................................................................... v

Vita .................................................................................................................................... vii

List of Tables ...................................................................................................................... x

List of Figures .................................................................................................................... xi

Chapter 1. Introduction ....................................................................................................... 1

Chapter 2. Methods ........................................................................................................... 11

Phase I – Ethnographic Observations and Semi-Structured Interviews ....................... 12

Data Collection ......................................................................................................... 12

Participants ................................................................................................................ 18

Games ....................................................................................................................... 20

Data Analysis ............................................................................................................ 21

Phase II – Structured Interviews and Analysis ............................................................. 22

Interview Design Process .......................................................................................... 22

Interview Administration Process ............................................................................. 32

Participants ................................................................................................................ 33

Data Analysis ............................................................................................................ 34

Chapter 3. Results ............................................................................................................. 35

Maintaining and Repairing Common Ground .............................................................. 36

Selecting the Starting Five ........................................................................................ 36

Bringing the Starters Back ........................................................................................ 39

Establishing Conditions for Effortless Coordination .................................................... 40

Using Standardized Play Calls .................................................................................. 41

“They just play really good together” Players .......................................................... 42

ix

Increasing Practice Repetition of Targeted Groups of Players ................................. 42

Cues from Predictable Individuals (to Manage Fatigue) .......................................... 43

Increasing and Repairing Mutual Directability............................................................. 44

Calling a Timeout ..................................................................................................... 44

Player-to-player Feedback ........................................................................................ 45

Poised to Adapt ............................................................................................................. 45

Polycentric Control ....................................................................................................... 47

Multi-directional Feedback ....................................................................................... 47

Mental Model Realignment ...................................................................................... 48

Successful Goal Conflict Management ..................................................................... 49

Pushing Individual Boundaries ..................................................................................... 49

Chapter 4. Discussion, Limitations and Next Steps .......................................................... 51

Next Steps ..................................................................................................................... 55

Limitations .................................................................................................................... 56

Chapter 5. Conclusion ....................................................................................................... 58

Bibliography ..................................................................................................................... 59

Appendix A. Scouting Report Example............................................................................ 63

Appendix B. NCAA Official Game Statistical Report ..................................................... 69

Appendix C. Structured Interview Script.......................................................................... 97

x

List of Tables

Table 1 Participant Characteristics ................................................................................... 18 Table 2 Games Analyzed .................................................................................................. 20 Table 3 Key Points of Structured Interview Script ........................................................... 25 Table 4 Structured Interviews' Participants' Characteristics ............................................. 34 Table 5 Basketball Terms ................................................................................................. 37

xi

List of Figures

Figure 1 Timing of Converging Methods and Season Overview ..................................... 12 Figure 2 Game Time Breakdown Example Chart............................................................. 14 Figure 3 Team Goals Report Example.............................................................................. 17

1

Chapter 1. Introduction

When studying and designing complex adaptive systems, it is critically important

to be able to take a systems approach. In the field of Cognitive Systems Engineering, our

collective understanding of system performance has evolved over time from a classic,

individualistic, linear model to models of interdependent, dynamic, collaborative agents.

Starting in the 1980s, the more classic, linear models of performance were not able to

prepare us for the surprises that we kept encountering. The Heinrich’s (1931) Domino

Model, later substituted by other linear models such as the Swiss Cheese Model (Reason,

1990) were used as the main safety approaches in healthcare. However, those models

started to be inadequate to the upcoming complex adaptive working environments

(Hollnagel, Wears & Braithwaite, 2015).

The more classic, linear models have been denoted as the Old View, Bad Apple

Theory or Safety I approach (Dekker, 2014). In the Safety I approach, as few things as

possible are expected to go wrong. Systems are believed to be inherently safe and work is

believed to be done as imagined. When things do go wrong, the blame is placed

reactively on specific components, often the humans within the system (Hollnagel, Wears

& Braithwaite, 2015). The Safety I approach contends that removing and replacing

components from a system would fix the problem. However, that is usually not the case

and larger systemic problems are revealed as accidents happen (Dekker, 2014). As the

2

Safety I approach was unable to prepare us for the surprises that the more complex,

higher tempo, adaptive systems were encountering, a New View or Safety II approach

evolved to fill this gap. The Safety II approach understands that systems are not safe, and

that safety is created by people working together (Dekker, 2014). Failures are not unique

individual events, but rather outcomes that seldom happen. This New View suggests that

rather than looking at why something went wrong, we should understand why it usually

goes right. In addition, looking at a system’s performance variability rather than its

malfunctions could reveal design improvements in the future of safer complex adaptive

systems (Hollnagel, Wears & Braithwaite, 2015). This Safety II thinking is closely tied to

the prior development of resilience engineering in the early 21st century (e.g., Hollnagel,

Woods & Leveson, 2006).

Resilience engineering steers away from the linear models and tries to understand

how systems function, driven by a plan or strategy rather than by events (Hollnagel,

2009). Resilience engineering aims to develop resilient systems that manage adaptive

capacity to produce sustained adaptability over longer periods. Those systems are layered

networks that are part of layered networks themselves, which make them intricate and

harder to design for (Woods, 2015). At the same time, these layered networks constitute

Joint Cognitive Systems (JCSs). In order to understand system behavior, Cognitive

Systems Engineering looks at the work done in JCSs (Woods & Hollnagel, 2006). In any

JCS, collaboration allows the system to adapt and gracefully extend, reducing brittleness

at its design boundaries. According to Woods (2018), sustained adaptability refers to “the

ability to continue to adapt to changing environments, stakeholders, demands, contexts,

3

and constraints (in effect, to adapt to how the system adapts).” From sustained

adaptability, Woods derives the concept of graceful extensibility, which is the opposite of

brittleness. Graceful extensibility is the ability of a system to extend its capacity when

surprises arise that challenge the system boundaries (Woods, 2018).

To discover patterns in JCSs, systems and what they have adapted to should be

studied. As work in any JCS always occurs in the context of multiple parties and

interests, collaboration is required amongst agents in order to successfully adapt to the

changing and evolving situations (Woods & Hollnagel, 2006). However, collaboration is

not reduced to the individual agents and systems properties separately, but rather is about

the interdependencies and interactions of the individual agents that constitute a system.

Cognitive Systems Engineering uses a systems perspective to understand collaboration by

looking at both agent interactions and cross-scale interactions (Woods & Hollnagel,

2006). Each individual agent, in order to successfully be a team player and facilitate

interactions to collaborate, must possess sufficient common ground, be interpredictable

and be able to be directed and direct others (Klein et al., 2004). In JCSs, successful

collaboration across agents allows a system to adapt, and such adaptation reduces the

system’s brittleness and increases its ability to gracefully extend as conditions change and

challenge existing system environments (Woods, 2018).

Resilient systems’ ability to adapt is a critical component to their functioning. A

resilient system must be able to detect that something has happened, identify what has

happened, evaluate what response is necessary and respond with the necessary resources

so that the response solves the issue (Hollnagel, 2009). In order to design complex

4

adaptive systems, we should study patterns of adaptation, from which we can find new

ideas about how to engineer them (Parker et al., 2013). In high tempo, high risk, complex

environments, adaptation is also critical in order to avoid decompensation and stop the

disturbances that quickly cascade through a system before a response can be executed

(Woods, 2018). Resilient systems should have the ability to identify a bottleneck ahead

and generate capacity to adapt. As the systems become more complex, higher tempo

environments, identifying disturbances and deploying actions to resolve the issues that

grow and cascade through the different elements of a system becomes critical (Woods &

Branlat, 2011).

Locating deficiencies using hindsight has not helped explain the persistent

existence of these deficiencies, nor has it helped design better complex systems that

extend at their design boundaries and adapt when faced with challenges (Hollnagel,

Wears & Braithwaite, 2015). Different methods are needed in order to design complex

adaptive systems that are resilient and gracefully extend at their design boundaries

(Dekker, 2014). Throughout history, however, understanding patterns of adaptation in

nuclear (Woods, 1995), healthcare (Richard Cook, 1998) and aviation (Patterson et al.,

2004; or Fischer & Orasanu, 2000) settings did help us design more resilient systems.

Patterns of individual adaptations and patterns of systemic performance and their

interdependencies is what is helping us design more resilient systems. Since every

component of a system has interactions with every other component, we must look

beyond cause-effect relations (Hollnagel, 2009) and understand the different challenges

that happen at different scales across systems in high tempo environments with periods of

5

lower tempo activities (Woods & Hollnagel, 2006). During higher tempo environments,

systems are more likely to reach their design boundaries and need to be gracefully

extensible. During lower tempo periods, systems are usually away from their design

boundaries. System adaptation and performance is critical closer to the design

boundaries, but individual performance and adaptation prevails away from them (Woods,

2018). Learning how to design for both situations is critical to the future of systems as

they become more complex and face challenges they have never experienced before.

With all of the literature in psychology, human factors engineering, and cognitive

systems engineering devoted to teams, it is interesting to note that very little is devoted to

what is arguably the most visible team setting: sports. As we try to study JCSs to design

improved complex adaptive systems, we find that basketball is an accessible team sport

that allows for a natural laboratory to study complex system’s behavior and adaptations.

The properties of a team sport like basketball make it an interesting environment to

gather new insights in how to design complex adaptive systems. Basketball is a kind of

joint activity that gives us a unique lens that other systems won’t allow for since it is

adversarial, and it requires continuous assessment and adjustments to what the other team

is doing and what is happening on the court, creating a mix of who dictates the pace and

who attacks to create changes in the pace or the events that evolve. At the same time,

basketball is decomposable between games, meaning that it has low tempo reflection

periods between games, and is very accessible and observable. In addition, basketball has

high varying tempos on multiple time scales, from longest to shortest: practices and film

sessions between games, halftime breaks, timeout breaks, play breaks and in-play breaks.

6

This last characteristic of the joint activity that constitutes a basketball team makes it a

very interesting natural laboratory from which we can learn different strategies

surrounding adaptation, how to assess the need for one and when to execute it. It is

typically difficult to study adversarial and competitive environments (e.g., military,

corporate settings) due to the limitations of access to data and the lack of observability of

them while events are evolving. Basketball allows us to observe and study systems both

retrospectively and live. At the same time, basketball is unique because in order to keep

pace with the adaptations of the rival team, a team must make various adaptations as the

rival is adapting at the same time that they are. Basketball teams usually rely on game-

plans that they have planned out beforehand to follow during a game. However,

deviations always happen due to the reactions to what is happening on the court and what

the other team is doing to disturb their own system. Unlike aviation systems, basketball is

decomposable between games, which makes it easier to analyze and retrospectively

prime participants to study specific decisions and points in time. The fact that aviation is

not decomposable makes it difficult to study the different adaptation possibilities and

sensemaking in which a pilot may change the dynamics of a flight (Rayo, 2020).

Lastly, basketball is very accessible and observable to me, Carmen Grande Pardo.

Having been the captain point guard a year prior, my expertise, experience and

relationships with the coaches and players make me a suitable researcher to immerse

myself in an environment I was already familiar with. I am an insider in this setting,

which I am aware could be an issue and impede investigations if “over-rapport” occurs

(Miller, 1952). It is unique that I can be both a practitioner and a researcher in this

7

setting, an insider and an outsider. Being just an insider can hinder the ability to

objectively make ethnographic observations and could lead to gathering data that has

little to not objectivity and accuracy (Labaree, 2002). In order to maintain my outsider

perspective, I stated clearly and continuously my purpose and interests of being present

and put aside my personal interests and relationships to acquire the desired level of

insight and maintain rapport. By finding a balance between my outsider and insider

perspective, I am able to collect data that would not be shared with an outsider but do so

maintaining my research goals and without building over-rapport.

The systems lens implies that although individual performances of the different

players matter, the systemic implications and interrelations are critically important and

likely more impactful. Even though individuals need to be performing at their best to

increase system performance, a healthy system needs to be able to function even without

certain individuals. In addition, the optimization of the system is based on the

collaboration and interrelations of the players, not just their individual talents. The sport

of basketball could teach the field of Cognitive Systems Engineering the different

dynamics and properties that would lead to a higher leverage from a systemic approach

versus those properties that have a higher leverage from an individualistic approach, as

well as different strategies to assess the need for an adjustment and when to execute the

adjustment.

However, as stated above, there is not a considerable amount of work or systems

research around organized sports. The literature largely focuses on key components of

sport organizations and how individuals can predict their own performance. For example,

8

in the study by Winand, Rihoux & Zintz (2011) Combinations of key determinants of

performance in sport governing bodies, three points were observed to be common

amongst highly performing sport governing bodies: (1) innovative activities and an elite

training structure; (2) large size, innovative activities and a governance mix of

volunteer(s) and paid staff; and (3) small size and governance by one or two volunteers.

In a different current of literature, Jagacinski & Flach (2003) reveal patterns of influence

a person could exert on a dynamic system in order to achieve a certain goal through the

use of control theory. Even though there is a large body of literature surrounding sports

and their individual components, I find that there is a lack of systems research

surrounding organized sports teams. In order to improve our current systems, it is not

enough to focus solely on individual performance. Research should continue to bring

insight into the design of complex adaptive systems through the study of organized team

sports.

This study explores a current gap in the research: the relative efficacy or value of

systems-based interventions versus those focused solely on individual players. In

addition, I see that applying successful organized sports performing patterns to the design

of complex adaptive systems could be beneficial. Since there are no solid guidelines

about how to design a complex adaptive system, understanding the system dynamics and

patterns of adaptation of a basketball team could reveal different ways in which they

could be designed. Learning new environments and their systemic and individual

adaptations can teach us new ways of designing complex adaptive systems.

9

This study uses converging research methods to study the patterns of adaptation that are


coaching decisions. A resilience engineering lens helps translate what is being observed

in a basketball setting into applicable concepts of joint activity and resilience, allowing

me to learn from the sport of basketball and apply it to the design of resilient complex

adaptive systems in general. Through a resilience lens, this study looks at joint activity

(Woods & Hollnagel, 2006) between the different members of a basketball team, their

decision-making surrounding adaptations and the system attributes that play a part in

such decisions-making. This lens allows me to look at adaptations, specifically how

coaches assess the need to adapt, how coaches adapt, and when coaches implement the

adaptation. These adaptations involve the interaction of different agents at different levels

and times. In addition, this study looks at the effect of those decisions relative to

individual versus systemic adaptations.

We do not understand how complex systems adapt, especially adversarial team

sports. In addition, we do not understand the relative efficacy or value of system-based

interventions versus individual based interventions. In this study I plan on doing

ethnographic observations of the team during practices and home games, video analysis

of home and away games, semi-structured interviews around practices with the coaches,

knowledge elicitation sessions with the coaches and finally structured interviews with the

coaches. The structured interviews will be the center piece of this research, allowing me

to understand the decision-making of basketball experts. The ultimate goal is to answer

the questions of (1) what patterns of adaptation are revealed when using a resilience

10

engineering lens to look at intra- and inter- game coaching decisions, and (2) what

strategies do coaches use to affect individual and system performance?

11

Chapter 2. Methods

This study focused on answering the research questions by corroborating the

findings from multiple different converging data collection methods: direct observations

of games, practices, and coaching activities, ethnographic interviews, and analysis of

secondary data sources such as game statistics, video footage of games, and artifact

analysis of the coaches planning document (e.g., practice schedules, goals of the game,

etc.).

Between January 2020 and June 2020, my research occurred throughout two

different phases. The first one occurred during the months of January, February and

March of 2020. The second one occurred during the months of April, May and June of

2020. In the following sections I discuss in detail what methods I have used to complete

my research. A summary of the overlapping in time of those methods is portrayed in

Figure 1 below. The research was conducted during the second half of the season.

However, extensive documentation of non-conference past games was available and

occurring games were analyzed and attended, allowing me to follow the season and its

surprises as it unfolded.

12

Figure 1 Timing of Converging Methods and Season Overview

All the data gathered using the first set of converging methods during the Phase I

of the research was used to design a structured interview script, which was used during

the Phase II of the research. The results of the interviews done following such script were

then used as the center piece and main data source of this study. The practice

observations, game observations, game video analysis, semi-structured interviews,

knowledge elicitation sessions and the secondary statistical data review all were used to

design the structured interview script during Phase II and support the findings from those

interviews. The results from the interview questions are the center piece of the study.

Phase I – Ethnographic Observations and Semi-Structured Interviews

Data Collection

During this first phase, I used the fieldwork method of participant observations

(Russel, 1988), as well as direct and video observations and analysis. In addition, I

conducted semi-structured interviews with knowledge elicitation sessions and collected

13

different documents. All were intertwined and used to determine my main points of

focus.

Observations

To reveal patterns and adaptations throughout a period of time, observations were

made in person at seven of the nine home games between the months of January,

February and March of 2020. In person observations were done during thirty-six team

practices. Observations of nine away games and four neutral site games were made using

the App Hudl, which gathers video and documents of all the games and practices. The

game videos uploaded in the App were taken directly from the BTN+ Network, which

uses sideline, baseline and coach’s camera angles. The practice videos are filmed using

coaches angle only. On these videos, breaks, time-outs and long pauses of gameplay are

left out.

The focus was put on the ten games that were played against repeated opponents,

playing five different opponents twice. Studying the same opponent revealed and

highlighted changes that happened both during the first and second game and during the

practices leading up to the games. Fieldnotes were taken during games and practices.

Game time breakdown graphs were created (see Figure 2) to easily recognize the

tendencies of the game and highlight when runs happened. The main goal of the

observations was to find patterns and points of adaptation that a team focuses on during a

season. It is understood that covering all points of adaptation is unrealistic due to the

many possibilities that unfold throughout a season. The analysis started with a broad

perspective. Being open to all options was a point of emphasis during the first 12 games

14

of the observations. After the month of January, feedback from coaches and players was

used to translate our interest in adaptation and change into two main points of focus.

Moments in which something changed during a game and actions that triggered the

coaches to make a decision were the focus of the observations during the remainder of

the games. We then used such points to reveal different aspects of the patterns that a

season follows as well as practice and in-game strategy. Video footage of earlier games

was then reanalyzed based on the final set of target adaptations.

Figure 2 Game Time Breakdown Example Chart

15

Recruitment

My recruitment process consisted on talking to the head coach and three assistant

coaches, as well as four other people who are members of the coaching staff and the

twelve players who are part of the team. I emphasized the voluntary nature of

participating on the study, as well as the fact that their participation would neither

negatively nor positively impact their respective positions. I read the verbal consent form

and gave each of them my contact information. They verbally consented their

participation on the study throughout the months of January through June of 2020. The

names of the participants and references to non-participants included in this study are

coded to preserve confidentiality.

Knowledge Elicitation Sessions and Semi-Structured Interviews

Knowledge elicitation sessions and semi-structured interviews were combined

with the practices and games’ observations. A total of sixteen individual semi-structured

interviews were conducted with six different coaches. Very broad questions that elicited

the coaches and players talking were asked, and follow up questions such as “can you

talk more in depth about that?”, “why is that?” were asked to gather as much information

as possible and prevent my basketball knowledge from influencing their responses and

thoughts. Four semi-structured interviews were recorded on my personal laptop or phone

and transcribed using Descript software. During the remainder twelve I used my laptop

and phone to take notes.

By conducting semi-structured interviews, I obtained the participants’ point of

view on the situation being discussed, looked into how they made that decision, what

16

goals they had in mind, and what tradeoffs had to be made in order to make it effective. I

then used the observations and videos of games to prime the participants’ memories

during the knowledge elicitation parts of the sessions. I conducted a total of five

knowledge elicitation sessions with the coaches, one per opponent that the team had

played twice. I showed them selected clips of each game to prime their memory and get

their sensemaking process of specific decisions they made around points of change. I

took notes and recorded them using my laptop. Using video and verbal recollection of

game and practice scenes helped the participants recall the different situations and give

me their thoughts on them. Semi-structured interviews and knowledge elicitation sessions

were conducted around practice time to facilitate the coaches’ schedules. These sessions

discussed trends seen in practice and games, as well as the thoughts that coaches had

about them. By looking at sets of games that the team played twice, the strategies used to

adapt were highlighted, helping me understand how coaches assess and move forward

with different adaptations.

During all these sessions I used cognitive probes from the Critical Decision

Method for cognitive task analysis (Klein et al., 1989; Hoffman et al., 1998) to learn

more about or clarify the initial answers.

Secondary Statistical Data Review

Pdf files of scouting reports were consulted. The sections included in these files

are opponent per game stats and Ohio State Women’s Basketball per game stats,

opponent’s summary of offense and defense, opponent’s top three sets they like to run,

keys to win, opponent’s player breakdown with descriptions of how to prevent them from

17

doing what they like, opponent’s box score, opponent’s top scorers, 3-point shooters, free

throw shooters, and ball control, fast-scout offensive and defensive factors, recent games

with site, result and score, and finally team stat comparison. An example of this data set

can be found in Appendix A.

Post-game team goals reports were also consulted. Prior to the start of the season,

the team set team goals for every game. These team goals set a baseline of the

expectations and focus of the team. After each game, the statistics were compared and

uploaded to see what categories were met and which ones the team failed to accomplish.

An example of a team goals report can be found in the Figure 2 below.

Figure 3 Team Goals Report Example

18

Thirty-six team practices were attended at the practice and game facility in order

to understand the flow of the season and the changing focus of the coaches and the team.

Practice plans of every practice were obtained and analyzed to reveal the main focuses

and adaptations of each week based on results, observations and knowledge elicitation

sessions. Understanding how coaches were training to obtain a change of behavior by

implementing certain drills in practice in order to adapt was a main focus.

Participants

Twenty team members participated in an observation and/or interview. Of the

twenty, twelve were players ranging from freshman to senior, having one to four years of

experience playing at the collegiate level. The remainder eight were members of the

coaching staff. Their experience ranged from one to twenty-seven years of experience in

a coaching staff. Individual details about the participants can be found in Table 1 below.

All participants were informed and verbally consented their participation on the study

during the months of January through June of 2020.

Table 1 Participant Characteristics

ID Role Playing

Experience

(years)

Coaching

Experience

(years)

Participated In

C1 Graduate Assistant 5 1 Observation &

Interview

C2 Video Coordinator and

Head Coach Assistant

4 2 Observation &

Interview

*Continued

19

Table 1 Continued

C3 Assistant Coach 4 + 7

(WNBA)

15 Observation &

Interview

C4 Assistant Coach 4 17 Observation &

Interview

C5 Assistant Coach 4 12 Observation &

Interview

C6 Head Coach 4 27 Observation &

Interview

C7 Director of Basketball

Operations

4 7 Observation

C8 Director of Player Development 4 9 Observation

P1 Player 1 N/A Observation












20

Games

A total of twenty-two games were analyzed. Of the twenty-two, nine were home

games, nine were away games and four were played at a neutral site. The individual

details of each game are summarized in Table 2 below. The highlighted games were

opponents that the team faced twice, once at home and once away.

Table 2 Games Analyzed

ID Site Opponent W/L Analysis done

G1 Home 1 L Video

G2 Away 2 W Video

G3 Away 3 L Video

G4 Home 4 W Video

G5 Home 5 W Attended & Video

G6 Away 6 L Video


G8 Away 8 L Video

G9 Home 3 L Attended & Video

G10 Away 9 W Video

G11 Away 7 W Video



G14 Away 11 W Video


*Continued

21

Table 2 Continued

G16 Away 12 L Video

G17 Home 13 L Attended & Video

G18 Away 1 W Video

G19 Neutral

site

2 W Video

G20 Neutral

site

8 W Video

G21 Neutral

site

4 W Video

G22 Neutral

site

3 L Video

Data Analysis

Game film was analyzed using Hudl Software. Points of change in trends were

obtained by creating Game Time Breakdown charts (See Figure 1 above). Official

statistics from the NCAA were used to analyze the individual and team numbers over the

course of the twenty-two games (see Appendix B for detailed example of NCAA Game

Official Statistical Report). Practice plans were analyzed, breaking down the different

times devoted to practice different skills, such as passing, rebounding or transition, to

name a few. The time devoted to each skill was compared to the performing goals of the

games following each practice, as well as to the retrospective analysis of points of

improvement. The points of change found over the ten games highlighted above were

coded using Excel software, as well as the practices. Audio from the semi-structured

22

interviews was transcribed using Descript services. The transcriptions, field notes, and

memos were coded using Excel software. I used various sets of codes to reveal patterns in

the decisions and actions of the coaches. The first round of coding of the changes

described the adaptation that was made, what triggered it, what it resulted on and what

type of basketball adaptation it was. The second round of codes further abstracted the

initial code set, focusing on the connections between the system that a basketball team is

and the field of Cognitive Systems Engineering.

Finally, using everything that I had learned of how the system adapted, I entered

the Phase II of the research and designed a structured interview script to gather direct

information about the decision-making process of the coaches around adaptations.

Phase II – Structured Interviews and Analysis

Interview Design Process

Using everything I had learned around patterns of adaptation during the phase I of

my research project, a structured interview script of questions was put together. The

details of this structured interview can be found on Appendix C, and a summary of the

main points can be found on Table 3 below. The script contained different sets of data

used to prime the participants memory. Examples from practices, quotes that had been

heard during the phase I interview sessions, official game and season NCAA statistics of

the team, clips of different plays, game time breakdown charts and game goals reports

were used to probe and gather information about patterns of adaptation around coaching

decisions.

23

Interview Questions

The interview questions were crafted around the points that had been observed to

be when most of the adaptations happened. The interview consisted of three main

sections: general adaptation questions, timeout questions and lineup alteration questions.

The main points of the interview script are summarized in Table 3 below. The goal of the

first section was to understand the points in which coaches thought they were making the

most adaptations. The second and third sections were used to understand in detail the

sensemaking process of the coaches around timeouts and lineup alterations.

As an example of how the converging methods of Phase I helped put together the

questions of the Phase II interviews, we can look at the “general questions – slow

moment or run” part of the interview. The questions on this part are based on live game

observations, game video analysis, ethnographic observations and semi-structured

interviews conducted between the months of January and March of 2020. The game

observations and video analysis pointed towards various points in different games in

which runs and slow moments created a need to adapt. At the same time, the semi-

structured interviews with the coaches helped reveal and confirm that those points were

concerning and a trigger to implement an adaptation. The structured interviews were used

later to dig deeper in the sensemaking of the decisions surrounding these points. This is

just one example of how the set of converging methods influenced and were used to

design the structured interview manuscript.

24

Priming with Game Videos

Clips were carefully selected to prime the participants memory and help them

reflect back on their sensemaking process of specific situations. Counterexample clips of

situations in which the same action had occurred but the coaches made a different

adjustment were also selected in order to understand the differences between coaching

decisions.

25

Table 3 Key Points of Structured Interview Script

Topic Questions Priming

data

General

questions

• During games, how do you know when you need to

make a change?

• How do you determine when to make

changes?

• What types of changes do you make?

• Between games, how do you know when you need to

make a change?

• How do you determine when to make

changes?


• We have noticed that sometimes changes are directed

at the team, and sometimes changes are directed at an

individual player.

• Can you think of a situation in which an

individual player needed to be addressed?

• What happened?

• What was your goal when you

implemented this adjustment?

• Can you think of a situation in which it

needed to be a team adaptation?

• What happened?

• What was your goal when

implementing this adjustment?

N/A

*Continued

26

Table 3 Continued

General

questions -

slow

moment or

run

• There are some instances in which your

team is not working together, in the worst

case the other team goes on a run, best

case you guys are having a “slow”

moment, but the other team is too.

• Can you think of a time in which

the other team went on a run?

• What is your general

strategy when this

happens? Can you walk

me through it?

• Can you think of a time in

which you used a different

strategy? Can you walk

me through it?

• Can you think of a time in which your

team was having a “slow” moment?

• What is your general strategy

when this happens? Can you walk

me through it?


you used a different strategy? Can

you walk me through it?

• If those measures don’t end up working,

how do you try to re-adjust?

• Why is this your later option?

• In any of the above situations, what is

your ultimate goal?

Video of the team

going on a run and

having a slow

moment with official

NCAA stats of the

quarter in which it

happened

*Continued

27

Table 3 Continued

General

questions -

foul trouble

• What is your strategy for foul trouble?

For example, if a player gets two fouls in

the first quarter, what would be your

decision? If a player has three fouls in

the third, what would be your decision?

• What are the thresholds and

times for fouls?

• Do you ever break these

thresholds? Why or why not?

• In what circumstances do you

break these thresholds and times

you just explained to me?

Talking about games

that were given as

examples in the prior

question

Lineup

alterations

• Can you talk about what your lineup

strategy usually is throughout any game?

• How do you choose your

starters?

• How do you decide who your

sixth man is?

• What determines the time

in which they come in?

• Usually you only give

meaningful minutes to eight

players, why only eight?

• Can you tell me about a time in

which you used this strategy?

• Can you walk me through

it?

• Can you tell me about a time in

which you used a different

strategy?


it?

• What required you to use

a different strategy?

(1) Statistics of a

game in which they

played their usual

rotation and (2)

statistics of a game in

which they used a

different strategy vs.

the same opponent

*Continued

28

Table 3 Continued

Lineup

alterations -

fatigue and

injury

management

• Can you think of a time in which you

had to make a change because of a

player’s fatigue? How do you manage

fatigue with your players?

• What triggers you to give them

a break?

• How long is this break

usually?

• How does it vary

between different

players?


had to make a change because of a

player’s injury? How do you manage

such an incident?

• What types of adjustments do

you have in mind when a

situation like that arises so

unexpectedly?

• Do you practice any of

those adjustments?

How often?

Video of a player

getting injured

Lineup

alterations -

key player

management

• Can you think of a game in which a

player that usually plays a lot of

minutes does not see the court a lot?

what are the things that make you sit

someone like that for most of the

game?

• What makes you give them

another chance? Or not give it

to them?

• What are the strategies that

you use when trying to have

that player improve their

performance?

• In what instances do

you simply give up on

them during a game?

(1) Game statistics in

which a player plays

her average minutes.

(2) Game statistics in

which she plays

considerably less and

full season stats

*Continued

29

Table 3 Continued

Lineup

alterations -

small

lineup

• A lineup that we have observed

throughout this research on your team is a

small lineup in which you are playing four

guards and one post player. Can you think

of a time in which you decided to play a

small lineup?

• Can you tell me your general

strategy about putting a small

lineup on?

• Can you think about a time

in which you did it that

way?

• Can you walk me

through your

strategy for this?


which you did it

differently?

• Can you walk me

through your

strategy for this?

• When do you decide to put this on?

• Why do you decide to adjust?

• What are the tradeoffs that concern

you in this specific situation?

• What would cause you to avoid

using this lineup?

• What are the strengths of this

lineup?

• What are the weaknesses of this

lineup?

(1) Clip and stats of

a game in which a

small lineup is

subbed in after a run

from the other team.


a game in which a

small lineup is not

the strategy after a

run from the other

team

*Continued

30

Table 3 Continued

Lineup

alterations -

big lineup

• A lineup that we have observed throughout

this research on your team is a big lineup in

which you are playing three guards and two

post players. Can you think of a time in

which you decided to play a big lineup?

• Can you tell me your strategy about

putting a big lineup back on?

• Can you think about a time

in which you did it that

way?

• Can you walk me

through your

strategy for this?


which you did it differently?

• Can you walk me

through your

strategy for this?



• What are the tradeoffs that concern

you in this specific situation?

• What would cause you to avoid

using this lineup?

• What are the strengths of this

lineup?

• What are the weaknesses of this

lineup?


a game in which a

big lineup is subbed

in when the score is

tied. (2) Clip and

stats of a game in

which a big lineup

is not the strategy

when the score is

tied.

*Continued

31

Table 3 Continued

Lineup

alterations -

starter lineup

• A lineup that we have observed throughout this

research on your team is getting your starters

back on, which consists of three guards and two

post players.

• Can you tell me your strategy about

bringing the starters back on?

• Can you think about a time in

which you did it that way?

• Can you walk me

through your strategy for

this?


you did it differently?

• Can you walk me

through your strategy for

this?


• What are the tradeoffs that concern you

in this specific situation?

• What would cause you to avoid using

this lineup?

• What are the strengths of this lineup?

• What are the weaknesses of this lineup?

Clip of a

game in

which they

subbed the

starters back

in

*Continued

32

Table 3 Continued

Timeout

interventions

• What is your timeout strategy for any game?

• What is the usefulness of a timeout for you?

• Can you name the situations that would cause

you to immediately call a timeout?

• In each of such situations, what do

you usually talk about in the huddle?


used a timeout to address an

individual issue?


what happened?


used a timeout to address a team

adaptation?


what happened?

• Is there a structure that you follow on

timeouts?

• Does it vary? How?

• What are your goals on a timeout?

• What strategies do you use during a timeout

to refocus your players and get them to all be

on the same page?

• What do you think is the difference between

a timeout and halftime in terms of

effectiveness and execution afterwards?

N/A

Demographic

data

• How old are you?

• How many years have you been coaching?

• What are your past coaching and playing

experiences?

• What is your role on the team?

N/A

Interview Administration Process

Six interviews and one follow up interview were administered post-season over

Zoom video calls. The interviews were recorded using the “Record” option that the

33

software provides. Notes were taken under each interview question using my laptop and

the chat log option that Zoom provides. General questions were asked first and used to

determine what follow-up questions were needed. The videos were shown as the

interview flowed using a shared folder of clips in the Hudl App. Links of statistics were

sent over the Zoom chat to use as priming data. Conversations with previous participants

were used as data to provide examples of situations. Different participants needed

different data to remember the situations that I was asking questions about. I used their

responses to guide me through what needed to be administered and asked follow-up

questions on situations that I wanted more information about due to their value to

understanding the sensemaking process of coaching decisions. During all these sessions I

used cognitive probes from the Critical Decision Method for cognitive task analysis

(Klein et al., 1989; Hoffman et al., 1998) to learn more about or clarify the initial

answers.

Participants

Six members of the coaching staff participated in the structured interviews. Their

experience ranged from one to twenty-seven years of experience in a coaching staff.

Individual details about the participants can be found in Table 3 below. All participants

were informed and verbally consented their participation on the study during the spring

semester of 2020.

34

Table 4 Structured Interviews' Participants' Characteristics

ID Role Playing

Experience (years)

Coaching

Experience (years)

C1 Graduate Assistant 5 1

C2 Video Coordinator and Head

Coach Assistant

4 2

C3 Assistant Coach 4 + 7 (WNBA) 15

C4 Assistant Coach 4 17

C5 Assistant Coach 4 12

C6 Head Coach 4 27

Data Analysis

Audio from the interviews of Phase II was transcribed using Descript Software.

The interviews were coded using Excel software. I used various sets of codes to reveal

patterns of adaptation in the responses of the participants. The first round of coding of the

changes described the adaptation that was made using basketball terms. The second

round of codes further abstracted the initial code set, focusing on the connections

between the system that a basketball team is and the field of Cognitive Systems

Engineering.

The interview results are the center piece of the research and constitute subject

matter expert data that will be discussed and explained further in this paper.

This research followed Human Subjects protocols and was approved by the

university’s Institutional Review Board.

35

Chapter 3. Results

Bringing the new perspective of Cognitive Systems Engineering into basketball

has allowed me to use a systems lens to look at coaching decisions to reveal various

patterns of adaptation of complex systems that would have been overlooked by basketball

experts. My interviews revealed five themes that will be explained in their respective

sections. First, I found that a large number of coaching decisions were made in order to

maintain or repair common ground. Second, coaches made decisions in order to reduce

coordination costs across the team for practices and more importantly for games. Third,

decision-making revolved around increasing or repairing mutual directability amongst all

team members, usually targeting players. Fourth, I found that coaches used low-tempo

periods to be more opportunistic during high tempo periods; which is also known as

being poised to adapt (Woods, 2018). Lastly, coaches facilitated the implementation of

polycentric control to achieve distributed cognition across all agents. Polycentric control

systems are referred by Ostrom (2012) as those with multiple centers of power at

different levels, providing more opportunities to all agents and dispersing the decision

making of the authority between higher-authority agents and lower-authority agents.

Furthermore, I explain in an additional section the results obtained regarding

individually-directed adaptations. I find this data necessary in order to discuss the

different system dynamics. I found that individually-directed adaptations are still

36

necessary even in a system which predominantly makes decisions based on the group and

not just on an individual.

Maintaining and Repairing Common Ground

I found multiple coaching strategies related to maintaining and repairing common

ground that coaches believed were critical to successful performance. Common ground is

perhaps the most important basis for interpredictability (Clark & Brennan, 1991), which

refers to the pertinent mutual knowledge, mutual beliefs and mutual assumptions that

support interdependent actions in some joint activity. Common ground permits people to

use abbreviated forms of communication and still be reasonably confident that potentially

ambiguous messages and signals will be understood (Klein, Feltovich & Woods, 2004).

Examples of these strategies include decisions surrounding the starting five and subbing

the starters back in the game.

Selecting the Starting Five

During the interviews, coaches emphasized the importance of selecting a starting

five as a critical component of successful performance. Selecting the starting five is

important because these players will likely play the most minutes in the game: this squad

will have an oversize influence in determining the success or failure in the game.

Interestingly, the coaches reveal that how the different components (players) work and

interact together is more important than individual talent when they are making personnel

decisions. Coaches explicitly mentioned that they valued lineup efficiencies, plus/minus

statistics, consistency, psychological impact, who was playing well together, trends and

37

solid practice and game players as well as other external factors as important

characteristics when choosing the players that are going to be a part of the starting five

(C6, C2). These basketball concepts are summarized in Table 4 below.

Table 5 Basketball Terms

Basketball

Term

Definition

Lineup

efficiency

The game performance of different combinations of players is

tracked and the offensive and defensive efficiency of them is

considered to come up with a rating of each of the different lineups.

Determining how various lineups perform illustrates which

personnel groupings function best on a team (Khan, 20113).

Team Offensive

Efficiency

A team’s offensive efficiency is the amount of points it scores per

100 offensive possessions.

OE = (Points scored * 100) / Possessions

(Gallo, 2019)

Team Defensive

Efficiency

A team’s defensive efficiency is the amount of points it allows per

100 defensive possessions.

DE = (Points allowed * 100) / Possessions

(Gallo, 2019)

Plus/minus

statistics (or

player

efficiency)

In basketball, the most commonly used statistical benchmark for

comparing the overall value of players is called efficiency. It is a

composite basketball statistic that is derived from basic individual

statistics: points, rebounds, assists, steals, blocks, turnovers and shot

attempts. The efficiency stat, in theory, accounts for both a player's

offensive contributions (points, assists) and their defensive

contributions (steals, blocks), but it is generally thought that

efficiency ratings favor offense-oriented players over those who

specialize in defense, as defense is difficult to quantify with

currently tabulated statistics (“Efficiency (basketball),” 2020).

*Continued

38

Table 5 Continued

Consistency Refers to the predictable behavior and performance players have

individually and with the different combinations of players they play

and practice with. In this case, it refers to the ability to perform to the

expectations at a high level for most of the games and practices.

Psychological

impact

Coaches think about lineups, team functioning and the impact of them

in various ways. In this case, the psychological impact refers to the

effect that starting or being a bench player may have on certain

individuals. To mitigate the negative effects, coaches take into

consideration how these decisions may affect the performance of

certain players and use this information to maximize each player’s

capabilities.

Trends Coaches describe trends as lineups that work while a winning streak

is happening or starting off a game with a high performance and a

certain lineup (C6).

Repeatedly, coaches emphasized the critical importance of “experience and

consistency in practices and games” (C3) when choosing the starters. According to the

coaches, the common ground that these players develop during practices and games is a

critical factor to the success or failure of the team.

Coaches repeatedly reported that the player that they thought had the most

individual talent was not in the starting five, and she also did not play the most minutes.

In addition, she averaged the sixth more minutes played per game out of all the players.

Coaches almost unanimously agreed that decisions made both in game and during

practice were highly influenced by other factors such as predictability, compliance

behavior with team rules, common ground, ability to perform without getting fatigued for

longer periods of time, directability and defensive performance. These traits, which are

39

more predictive of system performance than of individual performance, were the

dominant determinants of choosing a starting five.

Bringing the Starters Back

These system-focused traits were also a strong influencer on when to bring the

starters back into the end of a game. Throughout the knowledge elicitation sessions, the

coaches revealed that one of the first in-game strategies they use to repair potential lost

common ground is to bring the starters back on the court. I have observed such strategy

across various games and during various points of the games. Usually, the starters will

get back on the court multiple times during a game, playing the most meaningful minutes.

This happens after halftime at the start of the third quarter and at the end of competitive

games. Even though I heard that there are a number of game constraints that make it

impossible for the starters to be together on the court all the time such as “fatigue, ball

screen defensive strategies, personnel characteristics or injuries” (C4) it was repeatedly

mentioned that in critical junctures the starting five was relied on to change game

dynamics, either to seize an opportunity or to mitigate further decompensation.

During the interviews, the coaches pointed that their trust level is higher for these

players, they “feel more at peace and most comfortable when they are on the floor” (C1),

and they believe they are the ones out of the whole roster who would come back from a

deficit or decompensation event during a game. “Whenever the second string is on the

floor and they are doing good, it’s a matter of time, something is going to go wrong”

(C1). The coaches believe the starters will be the ones to bring back up the level of

performance (C1, C2, C3). If the coaches “saw a team go on a run with one of [the non-

40

starters] in [the court], they were the first ones to come out” (C5). Coaches have

repeatedly stressed the importance of the predictability of the starters as the dominant

reason to keep them on the floor or bring them back in. The starters are believed to be the

players who have the highest levels of common ground, mutual predictability and mutual

directability amongst each other. As a result, putting the starters back on the court is used

as a strategy to repair the lost common ground across team members.

Establishing Conditions for Effortless Coordination

In addition to decisions regarding the starting five, strategies to improve

predictability on the floor permeated all facets of the game. Coaches repeatedly stressed

the importance of the predictability of the players as the dominant reason to keep them on

the floor. In addition, coaches referred to various strategies that help them increase or re-

establish the interpredictability across all team members. The strategies of using

standardized play calls, making substitutions to play those players who have higher levels

of interpredictability amongst each other, and increasing repetitions of groups of players

during practices and games were commonly mentioned during the interviews as

important points to improve system performance. In addition, coaches mentioned the

importance of reading physical cues from players on the court to avoid decompensation

due to fatigue as a key component of optimal team performance.

As a result, I separate the strategies for establishing conditions for effortless

coordination under the following themes: using standardized play calls, “they just play

really good together” players, increasing practice repetitions of targeted groups of players

and lastly cues for predictable individuals.

41

Using Standardized Play Calls

As an effort to coordinate during high tempo periods such as games, coaches

repeatedly mentioned the use of set plays on offense as one of the main resources to stop

decompensation. With the intent of giving the players “the exact things that they need to

focus on” (C3), coaches use a predictable, standard set of play calls. The players can

anticipate each other’s moves better when the coaches use this strategy and “get them to

run some type of offense” (C3). An example of a late game situation in which coaches

emphasized the need for mutual predictability to obtain a win of a game was reflected on

during the interviews:

“in a late game situation, it might all be about: hey, this is what we're

going to run right now, and you guys have to really lock in and focus

[...] on what you have to do here to make sure each person executes

accordingly” (C6).

The high tempo, high pressure situations were pointed to as critical points in

which coaches needed to take action to ensure interpredictability and coordination across

the players to minimize errors and successfully finish games. Using a play call is one of

the strategies to establish that coordination. Play calling is about shaping the conditions

of adaptations - some aspects are constrained, but some aspects of adaptations remain

open for some players. This is a reflection of the tension between standardization and

encouraging adaptation. The work of the individual parts themselves and the interaction

42

amongst them to ensure execution of play calls were repeatedly deemed as critical to

successfully finish out games.

“They just play really good together” Players

The second strategy that serves the purpose of establishing conditions for

effortless coordination is revealed when looking at pairs of players who “play good

together” (C4). Coaches referred to the use of two players who were mutually predictable

with each other as their resource “whenever they wanted a change of pace” (C4).

Coaches mentioned looking at “trends to see who was playing well together” (C6), and

based on their specific needs at a certain time they would put “those two in” (C4). The

interpredictability of those two players and the knowledge of what to expect out of their

performance were mentioned as two key points to implement a change to improve

performance.

Increasing Practice Repetition of Targeted Groups of Players

During G2 of conference play, due to injuries and fatigue management, the

coaches had to use a guard player at the four position. This position is usually reserved

for mobile post players. They refer to this as their small or fast lineup. Surprisingly, the

coaches liked how the team functioned with this lineup. However, none of the guards

who were going to get minutes at the four position knew the plays and basic offense as a

four player. As a strategy to improve predictability during games amongst the players on

their so-called small lineup, the coaches started to get selected guards “some reps at the

four” (C3). Coaches believe that increasing repetitions of certain groups of players during

43

practice will increase the coordination amongst them when they need them to play in a

game together. In addition, it will allow the guards to feel more comfortable and

confidently run the plays that are called. As discussed above, the coaches emphasized the

use of standardized play calls as a means to ensure predictability on the court. To make

sure that players know the offense, coaches repeatedly talked about giving more

repetitions in practice to groups of players that would see the court together during

games.

Cues from Predictable Individuals (to Manage Fatigue)

The coaches use “strategies not to let [the players] get too tired because then the

value that you are getting from them goes down” (C4). With the goal of “maximizing the

time that they are in (the court)” (C5), coaches pay attention to physical signs of fatigue

in order to sub that player out of the game before decompensation occurs and increase the

coordination on the court. In particular, coaches find it advantageous that one of the

players shows predictable signs of fatigue, specifically she “takes quick shots and is

taking longer to get back on defense” (C5). Coaches can hence take her out before

decompensation starts. Since decompensation can quickly cascade through the rest of the

players on the court, the ability to detect early signals of decompensation by using

predictable physical cues from players was highlighted by the coaches as a way to

increase coordination.

44

Increasing and Repairing Mutual Directability

Using a systems lens also revealed the need for mutual directability, which is the

ability to modify the actions of the other agents, across all system agents. Mutual

directability refers to deliberate attempts to modify the actions of the other partners as

conditions and priorities change (Klein, Feltovich & Woods, 2004). In a basketball team,

players possess directability across them and with the coaches. Coaches mentioned using

various strategies to repair or increase the directability between players. The two most

mentioned strategies were the use of a timeout and the establishment of safe

environments for players to direct each other by providing feedback.

Calling a Timeout

Throughout the interviews and video observations, the use of timeouts was

highlighted as one of the first resources coaches use when there is decompensation or a

risk of it. Timeouts were referred to as resources to “bring [the players] together again”

(C2), “realign people” (C2), to “make sure that everybody is up to speed on those

adjustments” (C5), getting “everybody on the same page” (C5), and to “talk about how

we are going to do some things moving forward that we want to emphasize” (C6).

Repeatedly, coaches described the use of timeouts in high tempo moments that were

needed whenever the adaptations put in place were not being executed on the court.

Coaches use timeouts whenever the information they are trying to convey is such that

“everybody needs to hear this message” (C5). The environment of a timeout was

described as one in which communication was clearer and goals could be set without the

45

noise that the games have while they are being played. Timeouts and the use of them

were repeatedly deemed as key components to ensure understanding and directability

from coaches to coaches, players to coaches, coaches to players and players to players.

Player-to-player Feedback

During the low tempo periods of the season, coaches created environments such

as film sessions and position meetings (e.g. point guards, guards and post players) in

which they encouraged the players to give each other feedback and “increase the

awareness of people” (C5) by pointing out aspects that each other could improve. The

ability to direct each other during low tempo periods improved game and practice

performance between the players that had been involved in the feedback sessions. As an

example, coaches noticed how after a player brought up that she was running the floor

and not getting the ball her teammates started to “look for more throw-ahead stuff” (C5).

These situations were highlighted by the coaches as key points to successful

performance.

Poised to Adapt

Coaches referred to the use of “shared terminology” (C6) as a needed trait across

the team when the game’s pace only allows for a high tempo, short timeframe adaptation.

The use of low tempo periods such as practices to be ready to adapt during high tempo

periods such as games was mentioned repeatedly, as well as observed during both games

and practices. Coaches talked about the various strategies that they use with the goal of

46

getting the team to be poised to adapt in the high tempo, short time frame situations that

occur during games.

To prepare during low tempo periods, coaches use strategies such as film analysis

(C1) to look at “what adjustments [they] didn’t make, which ones [they] made that still

were not the best adjustments” (C1), they use “discussions after the game about the stuff

[they] wrote on [their] notebooks” (C1), and they eventually implement changes “once

[they] go to practice after watching that film” (C1). This retrospective analysis that the

coaches do during low tempo periods sets the team up to be able to adapt during the high

tempo periods of the game. Ultimately, coaches can save resources “if it's something

where we have really good terminology that [the players] can pick up, then it's easier to

make that adjustment without having to call a timeout” (C6).

Developing such terminology requires time and effort. However, it was deemed

necessary for ball screen defense and overall defensive scheme changes, amongst others.

Coaches point out that adaptations could be made “on the fly” (C6) if there was good

terminology all across, emphasizing the defensive areas. As an example, coaches talked

about communicating a change in the ball screen defensive strategy mid-play. They

characterized shared terminology as “very important” (C6) in order to implement a

change without having to call a timeout. Establishing shared terminology during low

tempo periods was repeatedly talked about as a key to successfully performing in high

tempo periods such as games.

47

Polycentric Control

During the 2019-2020 season, an offensive scheme change occurred 10 games

into the 22-game conference part of the season. The players addressed the coaches with

some concerns and dislikes of the current offense. After receiving the feedback, the

coaches implemented minor changes to the offensive scheme that resulted in the team

winning 10 of the last 14 games of the season. The coaches deemed this change and

communication as a critical part of the successful performance of the team towards the

end of the season. I have found different reasons that explain why this shift was very

effective and led to good results. The coaches referred to the ability to give and receive

feedback from and to the players, the fact that “everyone was on the same page” and the

fact that the players got “a little bit of what they wanted” (C1) but the head coach was

able to keep some of the parts of the offense that he “had been doing for the past thirty

years” (C1) as the three key points to successfully changing the offensive scheme and

obtaining better results in the middle of the season. In the following paragraphs I explain

the further details of the three themes, which I have labeled as multidirectional feedback,

mental model realignment and successful goal conflict management.

Multi-directional Feedback

The offensive scheme change was rooted in the ability of the players to

communicate with and be heard by the coaches. The coaches believed that it was “really

useful when the players [were] giving dialogue back and forth” (C3). They believed that

this dialogue was not only useful for the coaches but that the players were able to

48

“understand what [the head coach was] seeing” (C3) as well. The coaches repeatedly

deemed as beneficial the ability that the players had to communicate their thoughts with

the coaches. The players were not feeling “comfortable enough to make those plays and

needed something more to help them be successful” (C6). The existence of feedback in

various directions and the ability to reflect and implement the changes needed were

characterized as critical components of the successful performance of the team towards

the end of the season.

Mental Model Realignment

According to the coaches, part of the reason why the new offensive scheme shift

worked so well was because it made the players' perspective of how the offense should be

run align with that of the coaches. Prior to the change, the players were saying that “they

were struggling when they had to rely so much on dribble drive [to score]” (C6). In the

players’ minds, the new additions to the offense were “sort of simplifying” (C6). They

made the players believe that “if [they] could be good at a little bit of post-up and a little

bit more ball screen motion, then [they] won’t even get to the dribble drive and have to

rely on that as much” (C6). The players “seemed to like [the new offense] better and said

that’s what they wanted” (C1). In addition, “they were not complaining and were actually

trying to do it, so we stuck with it” (C1). This shift realigned the mental models of

everyone on the team, described by the coaches as a part of the season that was critical

for the later successful performance of the team.

49

Successful Goal Conflict Management

During this time, the coaches had to successfully manage certain goal conflicts.

They referred to the ability to do so as one of the main reasons why the offensive shift

ended up being so successful for the team. The dribble drive offense that the players did

not like had been the head coach’s “offensive scheme for the thirty years that he had been

a coach” (C1). To manage these goal conflicts, the coaching staff decided to “give [the

players] a little bit of what they wanted” (C1), while still maintaining “what [the head

coach] had been doing for the past thirty years” (C1). The ability of both players and

coaches to give in and accommodate what each other wanted while still getting part of

what they wanted was described during the interviews as an important turning point in

the season that largely benefited the whole system.

Pushing Individual Boundaries

Even though the coaches most repeatedly referred to the previously discussed

system properties as the most important determinants of successful performance, they

also referred to certain individual attributes that had to improve at points of the season in

order to increase overall team performance. In some instances, coaches mentioned it was

important to individually evaluate and address “the effort of [the] players” (C1), consider

whether or not a player was “making [her] shots” (C1), consider “who is in foul trouble”

(C2), if a specific player is “giving up straight line drives” (C3) or if a player “turned the

ball over” (C3). These are some examples of places in which the coaches did not feel like

they “needed to stop everybody because really that one person [was] the only issue” (C5).

50

During the interviews, the system adaptations were emphasized as the most critical parts

to increase performance. However, it is worth noting that there are various instances in

which coaches believe only individuals need to be addressed to fix a problem, stop

decompensation or increase performance.

51

Chapter 4. Discussion, Limitations and Next Steps

In all the sports literature, the system-level attributes that are important in team

play are underappreciated. Throughout sports history, individual athletic qualities and

traits have been studied and deemed as critical to successful performance. Players with a

higher fast to slow twitch muscle fiber area ratio (Thorstensson et al., 1977), higher

vertical jumps or quicker 40-yard dash times are considered better recruits by the NFL

scouts (Park, 2016). Not only does this happen in American football, but overall in any

sport. The more athletic players tend to be considered better recruits and assets to a team.

According to the sports literature, optimal agility, speed, strength and explosiveness

amongst others are traits and talents that are important when building a basketball team

(El-Saleh, 2020; Ostojic et al., 2006; Zig & Lidor, 2009; Latin et al., 1994; Smith &

Thomas, 1991). At the same time, I found that the individual attributes of being a good

passer, shooter, dribbler, defender or ambidextrous were considered important when

making various decisions on who should be getting more playing time on the team.

However, I am seeing the importance of the coordinative abilities and attributes for

making coaching decisions. Using a system lens helped me unveil the systems thinking

that coaches have in place when trying to increase the performance of a team and take it

to its highest level possible. Contrary to common belief, systemic attributes were

52

constantly emphasized as the most critical points for successful performance over

individual attributes.

I find that there is a disconnect between what is valued by team managers, scouts

and coaches when determining the best individual players and what actually matters to

the coaches when they are in the midst of a season trying to take their team’s

performance to the highest level possible. I observe a major disconnect between how

players are rated and the impact they have on a team. Coaches place the most value on

those players who have the highest levels of common ground, mutual directability and

mutual predictability, as well as those who are consistent with their performance levels

and fit the scheme that coaches have in place instead of on those players deemed more

talented and athletic. Those players who are consistent are more predictable which allows

the coaches to know what to expect from them and plan around it. This is consistent with

the Cognitive Systems Engineering literature, as common ground, mutual predictability

and mutual directability have been observed to be key components for collaboration

within any Joint Cognitive System (JCS) (Woods & Hollnagel, 2006). I have observed

and gathered evidence that the same elements are necessary to maintain collaboration and

make a basketball team better.

In order to maintain the system properties above, I observed that coaches are

constantly making adjustments, both in-game and between games. These adjustments are

mainly directed towards the collective system and with the overall coordination of the

system in mind. Coaches talked about the system level attributes as the most important

points for successful performance, especially during critical points. During those points,

53

the main emphasis was in the ability to communicate amongst each other using shared

terminology. Coaches use low tempo periods such as practices and film sessions to

develop shared terminology with each other and with the players. This activity allows the

team to be poised to adapt during high tempo, short timeframe periods. In any system,

having shared terminology improves the readiness to adapt, the ability to implement an

adaptation in a reduced timeframe and the efficiency to convey a plan in a high tempo

situation. Having a shared terminology is directly related to having shared common

ground, mutual directability and predictability. It all helps understand each team

member’s mental model’s and how to alter them to adapt in a short amount of period.

The shortest periods of time in which an adaptation can be made are in between

plays and during a play. In order to make a change in such a short period of time, coaches

have to possess the ability to convey a plan in a high tempo situation in a short span of

time and the ability to relay that information effectively to the players for the adaptation

to be executed. The possession of shared terminology allows for the effective

communication of coaches amongst each other and with the players. The Capacity for

Maneuver for such a short period of time is very slim. The parameter of Capacity for

Maneuver (CfM) is defined by Woods (2018) as the amount of range that a unit has used

and the capacity that remains in it to handle upcoming demands. Shared terminology

allows for common ground and mutual directability, which reduces the time needed to

make agile changes that cascade through the team quickly.

Acquiring shared terminology is not trivial, but the need for it to quickly adapt

happens at multiple scales. In a study by Albolino, Cook & O’Connor (2005), the

54

sensemaking of ICU unit caretakers was studied to understand how they cope with

complexity and uncertainty, as well as how they manage to get good results:

Practitioners tradeoff the opportunity costs of formal, collective

sensemaking (sensemaking at intervals) against the value that this

preparation provides to sensemaking during high tempo work

(sensemaking on-the-fly).

This study provides just one example of how high tempo, high stake situations

require different levels of common ground and acquiring those may result in more lives

saved. Shared terminology increases the level of observability and directability of all

agents involved. Another example of the need for shared terminology to fit an adaptation

into a smaller space is reflected in the readiness in hospitals to respond to the

Coronavirus outbreak. The CDC and WHO are sending daily updates on measures and

information that the hospitals need to make or have. The hospitals are unable to keep up

with the updates from the larger organizations due to the lack of shared terminology and

common ground (Grimm, 2020; Allen, 2020). The larger organizations are unable to

convey what they need to convey on a daily message update and the hospitals themselves

are saturated, so they do not have enough time to figure out what the measures need to

be. This disconnection ultimately results in a lack of information in the hospitals about

the different needs that the COVID-19 patients may have, which can result in more

deaths or saved lives.

At the same time, I found that the existence of polycentric control that allowed for

multidirectional feedback at a critical point in the season was very relevant to the success

55

of this particular team. Different inputs from different stakeholders (players and coaches)

made a difference in the outcome of the season. Before effective feedback and changes

were implemented, the team had a 11-9 record (0.55 winning percentage). After the

feedback and change implementation happened, the team closed out the season with a 10-

3 record (0.77 winning percentage) in the last thirteen games of the season. Effective

coach-to-coach, coach-to-player, player-to-coach and player-to-player feedback were

shown to be critical attributes of a successful team functioning. Any organization or

system has higher-authority agents and lower-authority agents who interact with each

other and have different needs. I believe that organizations and systems who actively

support directability vertically and laterally can adapt to meet the needs in front of them

at the moment and maintain the common ground across all agents while positively

impacting the organization, opposed to those who make centrally controlled decisions.

Next Steps

I believe that it will be extremely valuable to direct our attention to the stated

above findings when designing and improving complex adaptive systems. Not only sports

teams can benefit from the perspective that a system lens brings, but also any

organization or Joint Cognitive System.

We have come a long way in the Cognitive Systems Engineering field in

understanding system performance and how to improve it, but there is so much more to

learn in order to design systems that are not brittle and that gracefully extend at their

design boundaries. Learning from any other systemic environments can help bring new

56

ideas to how cognitive agents best perform and how that translates into successful system

performance.

This research could be taken into other basketball teams to reveal similar and

additional patterns of adaptation and successful performance, as well as any other team

sports. Some of the adaptations observed throughout the research are unique to basketball

and some are unique to this season and this specific basketball team. The adversarial

properties of this system and what we have learned from it can be applied at some level

to any adversarial setting, such as any competitive corporation, including the military. In

addition, finding and implementing new strategies to increase the levels of common

ground, directability, predictability, readiness to adapt and polycentric control in any

complex adaptive setting could increase performance and reveal new ways in which

systems can successfully function.

Finally, I think it is worth noting that the ability of the coaches to know when

there is a need to adapt, how to implement the adaptation and when to do so is a decision-

making process that I have observed throughout my research. In the future, diving deeper

into this data with the aim of revealing patterns around assessing and executing

adaptations is a worthy direction to consider.

Limitations

I recognize that there are some limitations to our research. First, the video

analysis hides some of the things that can be revealed during live observations as well as

some of the behavior and communications between coaches and players on the bench and

during timeouts. Second, there are limitations to verbal reporting and how agents

57

perceive themselves doing work versus how they actually perform it. I mitigated those by

triangulating the findings between our observations, video review, game analysis and

structured interviews with knowledge elicitation sessions. Third, there are

methodological limitations to this research: due to restricted access, only one team has

been studied. In-depth interviews were conducted that required extended time and

analysis, leaving no room to study and contrast how various teams function. Although I

believe that this team is representative of other elite college basketball teams, and

therefore other elite teams in multiple sports, my findings may not generalize to all other

settings. I propose this as future research. Lastly, due to the many factors that go into

every game, it is difficult to determine one specific point as the final determinant of

successful performance and wins and losses. Because of this, I focused on the strategies

that coaches use to increase what they perceive as successful performance rather than

only considering an adjustment effective if it led to a win.

58

Chapter 5. Conclusion

The systems literature fails to research the most common system setting: team

sports. Using a set of converging methods, I looked at (1) patterns of adaptation that are


coaching decisions, and (2) the strategies that coaches use to affect individual and system

performance on a collegiate basketball team. My main findings pertain to common

ground, directability, interpredictability, readiness to adapt and polycentric control. First,

I found that coaches focus their decisions around increasing or repairing common ground,

directability and interpredictability across their team members. Contrary to common

belief, coaches value systemic properties over individual talent or athleticism in their

players. Second, I found that coaches direct efforts to develop shared terminology across

all team members during low tempo periods to increase their team’s readiness to adapt

during high tempo periods. Last, I found that coaches promote polycentric control

through encouraging multi-directional feedback to achieve successful team functioning.

In conclusion, coaches regularly use systems-focused strategies to increase the levels of

common ground, directability, predictability, readiness to adapt and polycentric control in

order to increase team performance. I believe that these strategies could all be used in the

future to improve and design any complex adaptive system, as well as reveal new ways in

which successful system performance can be achieved.

59

Bibliography

Albolino, S., Cook, R., & O’Connor, M. (2007). Sensemaking, safety, and cooperative

work in the intensive care unit. Cognition, Technology & Work, 9(3), 131–137.

doi: 10.1007/s10111-006-0057-5

Allen, J. (2020). New York hospital sends some 'borderline' COVID-19 patients home

with oxygen monitors. Reuters. Retrieved from

https://www.reuters.com/article/us-health-coronavirus-usa-home/new-york-

hospital-sends-some-borderline-covid-19-patients-home-with-oxygen-monitors-

idUSKCN21Q3F0

Charmaz, K., & Belgrave, L. (2012). Qualitative interviewing and grounded theory

analysis. The SAGE handbook of interview research: The complexity of the craft,

2, 347-365.

Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J.

M. Levine & S. D. Teasley (Eds.), Perspectives on socially shared cognition.

Washington: D.C.: American Psychological Association.

Cook, R. I., Woods, D. D., & Miller, C. (1998). A tale of two stories: contrasting views

of patient safety. The Foundation.

Dekker, S. (2014). The field guide to understanding 'human error'. Ashgate Publishing,

Ltd..

El-Saleh, M. S. (2020). The Psychological Characteristics of Basketball Players in UAE

Universities and Their Relationship to the Dimensions of Athletic Excellence.

Journal of Talent Development and Excellence, 12(3s), 1511-1524.

Efficiency (basketball). (2020, April 17). In Wikipedia. Retrieved from

https://en.wikipedia.org/wiki/Efficiency_(basketball)

Fischer, U., & Orasanu, J. (2000). Do you see what I see? Effects of crew position on

interpretation of flight problems.

Fram, S. M. (2013). The Constant Comparative Analysis Method Outside of Grounded

Theory. The Qualitative Report 2013.

60

Gallo, C. (2019). Offensive and Defensive Efficiency. CBB Stats Help. Retrieved from

https://cbbstatshelp.com/efficiency/offense-defense/

Glaser, B. G. , & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for

qualitative research. New York: Aldine.

Grimm, C. A. (2020). Hospital Experiences Responding to the COVID-19 Pandemic:

Results of a National Pulse Survey March 23–27, 2020. US Department of Health

and Human Services OoIG, ed. Washington, DC, 1-41.

Hoffman, R., & Crandall, B. Critical Decision Method. Human Factors.

Hollnagel, E. (2009). The four cornerstones of resilience engineering. Chapter 6-pages

117-134.

Hollnagel E., Wears R.L. and Braithwaite J. From Safety-I to Safety-II: A White Paper.

The Resilient Health Care Net: Published simultaneously by the University of

Southern Denmark, University of Florida, USA, and Macquarie University,

Australia.

Jagacinski , R. J., & Flach , J. (2003). Control Theory for Humans: Quantitative

Approaches To Modeling Performance. Mahwah, NJ: Erlbaum Associates.

Khan, E. (2013). Advanced NBA Stats for Dummies: How to Understand the New Hoops

Math. Bleacher Report. Retrieved from

https://bleacherreport.com/articles/1813902-advanced-nba-stats-for-dummies-

how-to-understand-the-new-hoops-math

Klein, G. A., Calderwood, R., & Macgregor, D. (1989). Critical decision method for

eliciting knowledge. IEEE Transactions on systems, man, and cybernetics, 19(3),

462-472.

Klein, G., Feltovich, P. J., & Woods, D. D. (2004). Common Ground and Coordination in

Joint Activity.

Klein, G., Woods, D. D., Bradshaw, J. M., Hoffman, R. R., & Feltovich, P. J. (2004). Ten

challenges for making automation a team player in joint human-agent activity.

19(6). doi: 10.1109/mis.2004.74

Labaree, R. V. (2002). The risk of ‘going observationalist’: negotiating the hidden

dilemmas of being an insider participant observer. Qualitative Research, 2(1), 97–

122. doi: 10.1177/1468794102002001641

61

Latin, R. W., Berg, K., & Baechle, T. (1994). Physical and performance characteristics of

NCAA division I male basketball players. The Journal of Strength &

Conditioning Research, 8(4), 214-218.

Miller, S. (1952). The Participant Observer and "Over-Rapport". American Sociological

Review, 17(1), 97-99. Retrieved May 31, 2020, from

www.jstor.org/stable/2088368

Ostojic, S. M., Mazic, S., & Dikic, N. (2006). Profiling in basketball: Physical and

physiological characteristics of elite players. Journal of strength and Conditioning

Research, 20(4), 740.

Ostrom, E. (2012). Polycentric systems: Multilevel governance involving a diversity of

organizations. Global environmental commons: Analytical and political

challenges in building governance mechanisms, 105-125.

Park, P. (2016). Does the NFL Combine Really Matter.

Park J, Seager TP, Rao PSC, Convertino M, Linkov I. (2013) Integrating risk and

resilience approaches to catastrophe management in engineering systems. Risk

Anal 33(3):356–367. https ://doi.org/10.1111/j.1539-6924.2012.01885 .x

Patterson, E. S., Roth, E. M., Woods, D. D., Chow, R., and Gomes, J. O. (2004). Handoff

strategies in settings with high consequences for failure: lessons for health care

operations. International Journal for Quality in Health Care, 16(2), 125–132.

https://doi.org/10.1093/intqhc/mzh026

Reason, J. (1990) The contribution of latent human failures to the breakdown of complex

systems. Philosophical Transactions of the Royal Society (London), series B. 327: 475-

484.

Russel Bernard, H. (1988). Research methods in cultural anthropology. Qualitative and.

Smith, H. K., & Thomas, S. G. (1991). Physiological characteristics of elite female

basketball players. Canadian journal of sport sciences = Journal canadien des

sciences du sport, 16(4), 289-295.

Thorstensson, A., Larsson, L. A. R. S., Tesch, P., & Karlsson, J. (1977). Muscle strength

and fiber composition in athletes and sedentary men. Medicine and science in

sports, 9(1), 26-30.

Winand, Mathieu & Rihoux, Benoît & Qualizza, David & Zintz, Thierry. (2011).

Combinations of key determinants of performance in sport governing bodies.

62

Sport, Business and Management: An International Journal. 1. 234-251.

10.1108/20426781111162657.

David, D. Woods. (1995). The alarm problem and directed attention in dynamic fault

management, Ergonomics, 38:11, 2371-2393, DOI: 10.1080/00140139508925274

Woods DD. (2015). Four concepts for resilience and the implications for the future of

resilience engineering. Reliability Engineering and System Safety,

http://dx.doi.org/10.1016/j.ress.2015.03.018i

Woods, D. D. (2018). Resilience is a Verb. In B. D. Trump, M.-V. Florin, & I. Linkov

(Eds.), IRGC resource guide on resilience (vol. 2): Domains of resilience for

complex interconnected systems. Lausanne: EPFL International Risk Governance

Center (IRGC).

Woods, D. D. (2018). The theory of graceful extensibility: basic rules that govern

adaptive systems. Environment Systems and Decisions, 38(4), 433–457. doi:

10.1007/s10669-018-9708-3

Woods, & Hollnagel. (2006). JCS - Patterns in Cognitive Systems Engineering.

Ziv, G., & Lidor, R. (2009). Physical attributes, physiological characteristics, on-court

performances and nutritional strategies of female and male basketball players.

Sports Medicine, 39(7), 547-568.

63

Appendix A. Scouting Report Example

69

Appendix B. NCAA Official Game Statistical Report

97

Appendix C. Structured Interview Script

We have observed two main points of adaptation throughout our data collection and

analysis. Lineup alterations and timeout interventions are the most used resources when

trying to increase resilience of a system and avoid decompensation/brittleness. This

interview will thus focus on getting information about the decision making for those two

adaptations. However, to avoid fixation and find more about other points of adaptation

the coaches may use as their resources, we plan on starting broad, asking follow-up

questions based on the responses we get, and using that to redesign future interviews. We

are going to focus on the five teams that were played twice (10 games total) during the

regular season of the Big Ten Conference against Purdue, Nebraska, Illinois, Maryland

and Minnesota.

General Questions

• During games, how do you know when you need to make a change?

• How do you determine when to make changes?


• Between games, how do you know when you need to make a change?

• How do you determine when to make changes?


98

• We have noticed that sometimes changes are directed at the team, and sometimes

changes are directed at an individual player.

• Can you think of a situation in which an individual player needed to be

addressed?

• What happened?

• What was your goal when you implemented this adjustment?

• Can you think of a situation in which it needed to be a team adaptation?

• What happened?

• What was your goal when implementing this adjustment?

• There are some instances in which your team is not working together, in the worst

case the other team goes on a run, best case you guys are having a “slow”

moment, but the other team is too.

• Can you think of a time in which the other team went on a run?

• What is your general strategy when this happens? Can you walk

me through it?

• Can you think of a time in which you used a different strategy?

Can you walk me through it?

• I will prime them by showing them a video of a run. I will also

show them the stats of that specific quarter and if needed the whole

game to help them get situated.

• OSU vs. Maryland - Run

99

https://www.hudl.com/watch/playlist/UGxheWxpc3Q1ZTdhN2JjNT

RiYTgzOTBiM2NlMzIxYjI=

• “In this video of the home game against Maryland, they are

completing a 2-15 run to end the quarter with a 23-8

advantage. Can you walk me through your strategy to

adapt to the situation?” Strategy: They put Rebeka,

Kierstan and Janai in, call a timeout, tried going small.

• https://ohiostatebuckeyes.com/wp-

content/uploads/2020/01/Ohio-State-vs.-Maryland-1-30-

20-1.pdf Maryland (2) stats

• OSU vs. Maryland first quarter stats:

https://www.hudl.com/watch/playlist/UGxheWxpc3Q1ZTdhN2JjNTRiYTgzOTBiM2NlMzIxYjI=


https://ohiostatebuckeyes.com/wp-content/uploads/2020/01/Ohio-State-vs.-Maryland-1-30-20-1.pdf



100

• Can you think of a time in which your team was having a “slow” moment?

• What is your general strategy when this happens? Can you walk

me through it?

• Can you think of a time in which you used a different strategy?

Can you walk me through it?

101

• I will prime them by showing video of a “slow” moment. I will also

show them the stats of that specific quarter at first and if needed

the whole game to help them get situated.

• OSU @ Illinois - Slow moment

https://www.hudl.com/watch/playlist/UGxheWxpc3Q1ZTdh

N2JjNTRiYTgzOTBiM2NlMzIxYjI=

• “In this video, there is under 4 minutes left of play in the

first quarter of your game against University of Illinois at

their place. The score is 11-11 and for these last minutes,

both teams combine for 9 total points with a final score of

16-15. The end of the quarter is pretty slow for Ohio State,

but it is also slow for Illinois. Can you walk me through the

strategy that you use to adapt to this situation?”


content/uploads/2020/02/Ohio-State-vs.-Illinois-2-6-20-

1.pdf Illinois (2) stats

• OSU @ Illinois first quarter stats:



https://ohiostatebuckeyes.com/wp-content/uploads/2020/02/Ohio-State-vs.-Illinois-2-6-20-1.pdf



102

• If those measures don’t end up working, how do you try to re-adjust?

• Why is this your later option?

• In any of the above situations, what is your ultimate goal?

• Think of the second Maryland game or the second Purdue game. What is your

strategy for foul trouble? For example, if a player gets two fouls in the first

103

quarter, what would be your decision? If a player has three fouls in the third, what

would be your decision?

• What are the thresholds and times for fouls?

• Do you ever break these thresholds? Why or why not?

• In what circumstances do you break these thresholds and times you just

explained to me?

Lineup Alterations

• Can you talk about what your lineup strategy usually is throughout any game?

• How do you choose your starters?

• How do you decide who your sixth man is?

• What determines the time in which they come in?

• Usually you only give meaningful minutes to eight players, why only

eight?

• Can you tell me about a time in which you used this strategy?

• Can you walk me through it?

• I will prime them using the OSU vs. Maryland game stats since we

have been talking about it. The timestamps of the subs are there

and can help them express their thoughts on the timing and

decisions made.

• https://ohiostatebuckeyes.com/wp-content/uploads/2020/01/Ohio-

State-vs.-Maryland-1-30-20-1.pdf OSU vs. Maryland full game

stats. “In this game, with 4:20 left in the first quarter, you take



104

Aaliyah, Braxtin and Maddison out and sub int Rebeka, Kierstan

and Janai. Can you walk me through this strategy?”

• Can you tell me about a time in which you used a different strategy?

• Can you walk me through it?

• What required you to use a different strategy?

• I will prime them using the OSU @ Maryland game stats since it is

the same team but a different strategy was used. The timestamps of

the subs are there and can help them express their thoughts on the

timing and decisions made.


State-vs.-Maryland-1-6-20-1.pdf OSU @ Maryland full game stats.

“In this game, with 4:37 left in the first quarter, you take Dorka

out and bring Rebeka in, with 3:27 left you take Maddison and

Jacy out and put Janai and Dorka in, and Bell comes in for Patty

with 3:00 left in the quarter. Janai, Braxtin, Kierstan, Dorka and

Rebeka are on the court. Can you walk me through the strategy

you used this time?”

• Can you think of a time in which you had to make a change because of a player’s

fatigue? How do you manage fatigue with your players?

• What triggers you to give them a break?

• How long is this break usually?

• How does it vary between different players?



105

• Can you think of a time in which you had to make a change because of a player’s

injury? How do you manage such an incident?

• What types of adjustments do you have in mind when a situation like that

arises so unexpectedly?

• Do you practice any of those adjustments? How often?

• I would prime them using the video “OSU @ Illinois - Dorka’s Injury”

https://www.hudl.com/watch/playlist/UGxheWxpc3Q1ZTdhN2JjNTRiYTgz

OTBiM2NlMzIxYjI= “Even though she was able to come back later, some

immediate adjustments had to be put in place in order to adjust to this new

situation. Can you walk me through the strategy?”

• Can you think of a game in which a player that usually plays a lot of minutes does

not see the court a lot? what are the things that make you sit someone like that for

most of the game?

• What makes you give them another chance? Or not give it to them?

• What are the strategies that you use when trying to have that player

improve their performance?

• In what instances do you simply give up on them during a game?

• https://ohiostatebuckeyes.com/wp-content/uploads/2020/02/Ohio-State-

vs.-Nebraska-2-2-20.pdf “Jacy only plays 18 minutes when she averaged

28 minutes per game this season on the game at Nebraska when you guys

went to overtime, can you walk me through the strategy of deciding to cut

down her minutes?



https://ohiostatebuckeyes.com/wp-content/uploads/2020/02/Ohio-State-vs.-Nebraska-2-2-20.pdf

https://ohiostatebuckeyes.com/wp-content/uploads/2020/02/Ohio-State-vs.-Nebraska-2-2-20.pdf

106


content/uploads/2020/03/OSUseasonstats030820-1.pdf Full 2019-

2020 season stats

• https://ohiostatebuckeyes.com/wp-content/uploads/2020/01/Ohio-State-

vs.-Maryland-1-30-20-1.pdf “Here are the OSU vs. Maryland stats since

we have been discussing this game and they are already primed on it. This

is a game in which Jacy plays higher minutes. Can you walk me through

this and why it was different than the game against Nebraska?”

Small Lineup

• A lineup that we have observed throughout this research on your team is a small

lineup in which you are playing four guards and one post player. Can you think of

a time in which you decided to play a small lineup?

• Can you tell me your general strategy about putting a small lineup on?

• Can you think about a time in which you did it that way?

• Can you walk me through your strategy for this?

• Can you think of a time in which you did it differently?




• What are the tradeoffs that concern you in this specific situation?

• What would cause you to avoid using this lineup?


https://ohiostatebuckeyes.com/wp-content/uploads/2020/03/OSUseasonstats030820-1.pdf

https://ohiostatebuckeyes.com/wp-content/uploads/2020/03/OSUseasonstats030820-1.pdf



107


• I will prime them by using a clip from the OSU @ Maryland game “OSU

@ Maryland - Small Lineup Not Working”


OTBiM2NlMzIxYjI= “In this clip, Maryland went on a run in the fourth

quarter and you sub in Kierstan at the four to have a small lineup on the

court, can you walk me through the strategy you followed here?”


State-vs.-Maryland-1-6-20-1.pdf OSU @ Maryland stats

• As an alternate strategy, they chose to go big when Nebraska went

on a run at the end of the second quarter during the game at their

place. Why did you decide to use a different strategy when they

were on a run?

• OSU @ Nebraska - Small Lineup (alternate strategy)

“Here Nebraska goes on a run and you guys are trying to

stay on the court with a small lineup but eventually you

play a big lineup, can you tell me what was different from

your type of adaptation on the game against Maryland?”

Big Lineup

• A lineup that we have observed throughout this research on your team is a big

lineup in which you are playing three guards and two post players. Can you think

of a time in which you decided to play a big lineup?





108

• Can you tell me your strategy about putting a big lineup back on?











• I will prime them by using a moment in which they subbed in a big lineup

“OSU @ Minnesota - Big Lineup”


OTBiM2NlMzIxYjI= “On your first game against Minnesota, with 3:30

left in the third quarter, the score is tied at 42 and you decide to sub in a

big lineup, can you walk me through the strategy of putting the big lineup

on?”


State-vs.-Minnesota-12-31-19-1.pdf OSU @ Minnesota stats

• As an alternate strategy, OSU is tied with Illinois on the game

against them at The Schott and the adaptation consists of calling a



https://ohiostatebuckeyes.com/wp-content/uploads/2019/12/Ohio-State-vs.-Minnesota-12-31-19-1.pdf

https://ohiostatebuckeyes.com/wp-content/uploads/2019/12/Ohio-State-vs.-Minnesota-12-31-19-1.pdf

109

timeout and playing a small lineup instead. Can you walk me

through this strategy and why you chose to do something different

with a tied score?

• OSU vs. Illinois - Big Lineup (other strategy)

Starter Lineup

• A lineup that we have observed throughout this research on your team is getting

your starters back on, which consists of three guards and two post players.

• Can you tell me your strategy about bringing the starters back on?










• I will prime them by using a moment in which they subbed the starters

back in “OSU vs. Nebraska - Starter Lineup”


OTBiM2NlMzIxYjI= “In this clip, you can see the end of a 13-2 Nebraska

run on the fourth quarter. OSU was taking quick and contested shots and



110

was turning the ball over against the zone due to lack of movement, so you

decide to put the starters back on by taking Kierstan out and putting

Miller in. Can you walk me through your strategy here?”

Timeout Interventions

• What is your timeout strategy for any game?

• What is the usefulness of a timeout for you?

• Can you name the situations that would cause you to immediately call a timeout?

• In each of such situations, what do you usually talk about in the huddle?

• Can you think of a time in which you used a timeout to address an

individual issue?

• Can you walk me through what happened?

• Can you think of a time in which you used a timeout to address a team

adaptation?

• Can you walk me through what happened?

• Is there a structure that you follow on timeouts?

• Does it vary? How?

• What are your goals on a timeout?

• What strategies do you use during a timeout to refocus your players and get them

to all be on the same page?

• What do you think is the difference between a timeout and halftime in terms of

effectiveness and execution afterwards?

111

Demographic data

• How old are you?

• How many years have you been coaching?

• What is your past coaching and playing experience?

• What is your role on the team?

1 Using Converging Methods to Reveal Hidden Systems ...

Documents

Transcript of 1 Using Converging Methods to Reveal Hidden Systems ...