User involvement throughout the innovation process in high-tech industries

Bosch-Sijtsema, P.M. & J. Bosch (2015) User involvement throughout the innovation process in high-tech industries. Journal of Product Innovation Management. 32 (5): 793-807. DOI: 10.1111/jpim.12233.

1

USER INVOLVEMENT THROUGHOUT THE INNOVATION PROCESS IN HIGH-TECH INDUSTRIES

Dr. Petra Bosch-Sijtsema* (corresponding author) Chalmers University of Technology Department of Civil and Environmental Engineering Division of Construction Management Sven hultinsgata 8, 412 96 Gothenburg, Sweden Phone: +46-31-7721945 [email protected]

Dr. Jan Bosch Chalmers University of Technology Department of Computer Science & Engineering Division of Software Engineering Gothenburg, Sweden [email protected]


2


Biography authors: Petra M. Bosch-Sijtsema Petra Bosch-Sijtsema is an associate professor at Chalmers University of Technology, Construction Management, Gothenburg, Sweden. She has a PhD in Management and Organization from Groningen University (Netherlands) and a licentiate degree from Lund University (Sweden). She has worked at universities in the Netherlands, Finland, Sweden and United States. Her research interests are in new ways of working, distributed and virtual work, knowledge work, ICT, innovation and knowledge creation, and collaboration within and between firms. She has published in journals like the Journal of Product Innovation Management, International Journal of Project Management, Group and Organization Management, Technology Analysis and Strategic Management, and the Journal of Knowledge Management. Jan Bosch Jan Bosch is professor of software engineering and director of the Software Research Center at Chalmers University Technology, Software Engineering in Gothenburg, Sweden. Earlier, he worked as Vice President Engineering Process at Intuit Inc. and led Open Innovation efforts and was head of the Software and Application Technologies Laboratory at Nokia Research Center, Finland. He has a PhD degree in Software Engineering from Lund University, Sweden. His research activities include open innovation, innovation experiment systems, compositional software engineering, software ecosystems, software architecture, software product families and software variability management. He has published significantly in major journals, writes and edits books, chairs conferences and is part of several program committees.


3


Abstract The feedback and input of users has been an important part of product innovation in

recent years. User input has been studied from different approaches and is applied through different methods in particular phases of the innovation process. However, these methods are not integrated to the whole innovation process and are used only in particular phases or on ad hoc basis. New developments in technology, social media, and new ways of working closer with customers have opened up for new possibilities for firms to gain user input throughout the whole innovation process. However, the impact that these new developments in technology offer for user input innovation in high-tech firms is unclear. Therefore, this study focuses on how high-tech firms collect and apply user feedback throughout the whole innovation process. The article is based on a comparative case study of eight cases in the high-tech industry, in which qualitative data collection was applied. The key contribution of the article is a conceptual framework on user data driven innovation throughout the innovation cycle. This framework gives insight into user involvement types and approaches to collect and apply user feedback throughout the innovation process. Key-words: User input, user involvement, innovation system, high-tech, new product and feature development

1 IntroductionIn the last couple of years, user or customer feedback has become increasingly important for product innovation. Customer integration or user integration is part of the open innovation paradigm (Chesborough, 2003), in which the innovation process has opened up to external knowledge (Enkel, Kausch and Gassmann, 2005). An important element in user integration is the need for information, which does not only come forth from marketing techniques to gather this data, but also from a cooperation between the firm and the user in terms of an intensive interaction (Rohrbeck, Steinhoff and Perder, Steinhoff and Perder, 2010). From this paradigm there are different approaches that have been developed for integrating external knowledge from customers and suppliers. One development has been towards involving users actively in the innovation process in which the customer initiates the starting activities for innovation, i.e., user innovation (Bogers, Afuah and Bastian, 2010; Kristensson, Gustafsson and Archer, 2004; Magnusson, 2009; Von Hippel, 1994; 2005; Voss, 1985). Other studies focus more on collaborating and gaining knowledge and feedback from users through interactions between manufacturers and their customers (Enkel, Kausch and Gassmann, 2005; Rohrbeck, Steinhoff and Perder, 2010). In this article the primary focus is on how firms can gain information and feedback from their users in the firm’s product development process. Several articles discuss different approaches companies can apply in order to gain input from their users in different phases of the innovation process. These processes can be more collaborative as in customer integration research (Enkel, Kausch and Gassmann, 2005; Schaarschmidt and Kilian, in press) or more focused on methods for a firm to collect user input in various ways (see Janssen and Dankbaar, 2008; Kaulio, 1998; Van Kleef, van Trijp and Luning, 2005). The input of users through different types of approaches is important for the development of new products. Many of these methods are applied in particular innovation stages, with a strong focus on the early conceptual phase (Schaarschmidt and Kilian, in press)


4

or on the deployment and sales phase (Van Kleef, van Trijp and Luning, 2005), but do not cover all innovation stages. Also, the different methods are not integrated to collect user data systematically over the whole innovation process. Furthermore, other researchers have warned for the risks of user integration when collaborating with users during the innovation process, in terms of loss of knowledge, dependency on customer views, misunderstandings between R&D and customers (Enkel, Kausch and Gassmann, 2005; Schaarschmidt and Killian, in press), as well as the limitations of listening - users do not always know what they want (Ulwick, 2006).

Some researchers mention the use of virtual customer or user integration (VCI) in which the Internet is the prime communication channel for either unidirectional or bidirectional communication (Rohrbeck, Steinhoff and Perder, 2010). User input for innovation is becoming more and more important now that technology is providing opportunities to gain access to input from larger groups of users. The use of the Internet can be less costly than other types of user feedback. In this study the primary focus is on the high-tech industry, as defined by Baruch (1997), that is working with products in which software is embedded, i.e., automotive, telecom, and IT industry. Within this industry, new developments in technology, social media, and new ways of developing software open up for new possibilities for companies to gain involvement and feedback from their users through both user feedback but also users’ actual behavior. In recent years, with the development of Web 2.0 technologies, social network systems and software-as-a-service (SaaS), more and more companies in the high-tech industry have started to work closer with their users through the Internet. This is for example increasingly the case for software-intensive embedded systems, including telecommunication and automotive systems, as these systems are increasingly becoming connected and the software in these systems can consequently evolve also after the system leaves the factory and is deployed at the user. The adoption of SaaS (Software as a Service) and cloud computing has shown a different approach to gain user input through frequent testing of the development process with the intent of minimizing R&D investment between user proof points, i.e., versions of the product that are tested with users (Bosch, 2012; Bosch and Eklund, 2012). Relatively new approaches like crowdsourcing (Janssen and Dankbaar, 2008; Poetz and Schreider, 2012) as well as experimentation and testing platforms (Davenport, 2009: Eisenberg and Quarto-von Tivader, 2008; Kohavi, et al., 2009) with larger groups of users are becoming more popular. Crowdsourcing makes use of informal communities of social networks centered around a specific topic or problem and through which individuals can contribute with ideas, suggestions, and solutions (cf. Janssen and Dankbaar, 2008; Poetz and Schreider, 2012).

For companies to be able to benefit from user input and feedback in various stages in the innovation process, it becomes important to find mechanisms to test and explore more ideas and concepts with users, and in the installed base test features, prototypes and products in real-time and at a much lower cost than earlier. This requires new behaviors at the business level, i.e. involving users in feature and product concept validation without an, initially clear, business model. Also, it requires changes to the R&D process, as users need to be involved much earlier and deeper in the R&D process. However, the impact these new developments in technology can offer for user data driven innovation in high-tech firms is unclear. This study focuses on how high-tech firms collect and apply user feedback throughout the whole innovation process. With help of eight case studies, the researchers develop a conceptual


5

framework concerning user data driven innovation that integrates different types of user involvement and data throughout the whole innovation process.

The article is structured as follows. The first section discusses the available literature concerned with incorporating user input in innovation. The second section focuses on the qualitative data collection and analysis approach of eight case studies. The findings of the cases are discussed in the section there after. In the discussion section the researchers conclude the article in which the findings are related to the literature and in which the theoretical and practical contributions are lifted up.

2 UserinputsupportinginnovationInvolving users in a company’s innovation process has been important for the development of new products. Over the years several studies have published how companies can gain from collaborating with users or gaining feedback and input from customers (e.g., Janssen and Dankbaar, 2008; Kaulio, 1998; Van Kleef, van Trijp and Luning, 2005; Veryzer, 1998). These approaches and strategies are both from R&D as well as from marketing research and focus on different stages in the new product development process, i.e., opportunity identification, development, testing, and launch of a product (Van Kleef, van Trijp and Luning, 2005). However, a large amount of literature focuses primarily on the conceptual and early development stages of innovation (Schaarschmidt and Kilian, in press; Van Kleef, van Trijp and Luning, 2005). An important part of user involvement during the innovation process is information. However, user information can be rather sticky information and transferring this type of information between the user and the product developer can become costly (Jeppesen, 2005; Von Hippel, 1994).

In the literature, different approaches are identified to gain user involvement or user input from approaches in which the main problem solving location is the firm or the user (Jeppesen, 2005). In this research the focus is on different approaches from collaborative to more passive user input approaches, however, the firm is the main problem solving location.

2.1 Listeningtoandcollaboratingwithusers

A number of different approaches to gain user input are discussed in literature, like ethnography (Rosenthal and Capper, 2006), focus groups, interviews, surveys, or product tests (see Janssen and Dankbaar, 2008; Kaulio, 1998; Van Kleef, van Trijp and Luning, 2005). In these approaches the firm is listening to customers more than collaborating with its users (Jeppesen, 2005). A more collaborative approach in which firms receive input from their users is interaction with lead users or leading edge customers (Lüthje and Herstatt, 2004; Von Hippel, 1986). These lead users provide use-experience in terms of domain and technical knowledge (Lüthje and Herstatt, 2004; Olson and Bakke, 2001), which is often used in the idea stage or fuzzy front end of new product development (Kristensson, Gustafsson and Archer, 2004). Another collaborative approach is used especially in the software industry, in which the user can become an active participant during innovation as a co-producer (Bogers, Afuah and Bastian, 2010). One example in the software industry that has gained popularity is the use of agile software development methods and techniques (Boehm and Turner, 2004) in which users are actively involved in the software design and collaborate with the development team.

Another approach supported by new developments in technology is the use of crowd sourcing design and development techniques (Janssen and Dankbaar, 2008; Poetz and Schreier, 2012), in which users are integrated in the design and development process and can


6

test new innovative designs and products with help of information technology. According to Janssen and Dankbaar (2008) user input contributes to a superior product, but they claim that more insight is needed in different techniques in order to obtain user input. Others mention that the capability to integrate diverse knowledge bases - internal and external integration - through problem solving can be the basis of learning processes in the organization (cf. Iansiti and Clark, 1994).

2.2 Testingandexperimentswithusers

Some of the techniques mentioned in literature have an element of experimentation in which particular ideas, concepts, and sometimes products are tested with users. These types of experiments or tests are used in research and development (R&D) as problem solving processes (Thomke, Von Hippel and Franke, 1998; Thomke, 2001), and are applied in technology driven research like the medical, automobile, engineering, and pharmaceutical industry (Thomke, 2003). Experimentation and testing is used for problem solving through a trial-and-error process in which each trial generates new insights to a problem (Lee, et al., 2004; Thomke, Von Hippel and Franke, 1998). Experiments are often carried out using simplified models of the test environment in order to reduce costs or control aspects of the environment. Experimentation behavior is critical to organizational innovation (Lee et al., 2004). Organization learns from the trial and error problem solving process in which both successes as well as failure provide learning. Lee et al. (2004) argues, however, that individuals avoid failure and seem to avoid experiments in which failure is likely. The problem solving approach through trial and error is often related to learning by doing (Eisenhardt and Tabrizi, 1995; Von Hippel and Katz, 2002) or learning in practice (Orlikowski, 2000) in which testing can support learning by practices to develop capabilities through trial and error and problem solving. From a learning perspective, frequent iterations of products in terms of prototypes or multiple design iterations, testing, and more frequent milestones are associated with faster product development (Eisenhardt and Tabrizi, 1995). Not only the number of designs increases and therefore the odds of success, but also learning by doing through multiple iterations are faster (Eisenhardt and Tabrizi, 1995). Furthermore, testing has been proven to be useful to speed up development (Gupta and Wilemon, 1990) as well as to accelerate understanding and reconceptualization of the product through trial and error (Eisenhardt and Tabrizi, 1995). With help of new technology and new development ways, tests and experiments with users are now possible and applied more and more (cf. Davenport, 2009).

The adoption of experimentation and testing with users is driven through three fundamental drivers in the industry. In the high-tech industry, the increasing adoption of cloud computing in favor of on-premise computing provides the companies developing solutions with an excellent platform for experimentation. In the software-intensive embedded systems industry, the increasing connectedness of these systems allows for the deployment of new software after the system has left the factory and is put in operation. In addition, the same connections are used to collect unprecedented amounts of diagnostic data from these systems, allowing for much more elaborate analysis of the performance of systems in the field (see Bosch and Ullrich, 2012). Finally, the third trend is the transition from a product to a service business, which means that telecom, automotive and other products are now offered as services to users rather than products. This is an important driver for deployment of new software in systems in the field, as the company that originally manufactured the system remains responsible for it during its economic lifetime. Through these changes in the high-tech and especially software intensive industry there are now possibilities to gain user


7

feedback not only in the early conceptual phases, but also after the product has been deployed, and firms can gain data concerning user’s actual behavior.

In the high-tech and especially software intensive industry, innovation through

experiments with users input is becoming more and more discussed (Davenport, 2009), primarily in the web2.0 and SaaS (Software as a Service) field. However, in this particular industry, these experiments are currently primarily performed on feature optimization. Feature optimization is often performed in controlled experiments that establish a causal relationship between changes and their influence on user-observable behavior (Kohavi, et al., 2009). Several companies have built their own testing platforms and teams for both online and offline testing, like eBay (Davenport, 2009), Google (Eisenberg and Quarto-vonTivader, 2008), and Microsoft (Kohavi et al., 2009). Experiments in terms of optimizing content of websites or online data are often discussed in literature (Agarwal et al., 2008; Eisenberg and Quarto-vonTivader, 2008; Kohavi et al., 2009). These experiments typically test an aspect of a website, such as an alternative workflow, changing the order of options, pricing models or the location of elements on the page. The experiment is developed to test whether the alternative version will improve some relevant business metric, such as the percentage of users signing up, the time a user spends on a page or the website or the relative number of purchases. The original version (A) and the alternative version (B) can then be presented to randomly selected users and the data relevant to the tested business metric is collected. Once a statistically relevant number of users has used either A or B, the experiment can be ended with a clear conclusion on the superior version. In these systems companies can gain user input to rather low costs through click systems, and A/B testing through collecting data on e.g., the amount of time a user spends using a feature, the relative frequency of feature selections, and the path that the user takes through the product functionality. The user is typically unaware about their involvement in experiments. In practice, however, the tests mentioned above performed online are often limited to optimizing narrow aspects of the front-end of the website through A/B testing but in connected, software-intensive systems experimentation, if applied at all, it is ad-hoc and not systematically applied.

3 MethodologyTo study how users’ feedback and user data is collected and supports innovation the

study applied a comparative case study analysis method (Eisenhardt, 1989) based on qualitative research methods. This analysis method was applied in two steps: First in-depth case studies were analyzed, and secondly the cases were compared. The researchers applied qualitative methods (interviews and observations) to study eight case examples of user input driven innovation in three different companies in the software, automotive and telecommunication industry. The cases researched are examples of different levels of maturity in the innovation cycle and have different scopes, i.e., feature optimization, new feature development, and new product development. The companies were all global Fortune 1000 companies and develop software or software intensive products. In total, the researchers held 49 semi and unstructured interviews. In one set of 12 interviews the researchers received input for three examples (cases A, C, and H). Case B is based on 8 interviews (several discussions were held with a senior manager); case D had 8 interviews, case E had 4 interviews, case F had 11 interviews, and case G had 6 interviews. Interviews were held with both managers (i.e., product development, project manager, head of innovation, and directors) as well as with a selection of the R&D team members. Furthermore, the researchers


8

followed these R&D teams in terms of observations, as well as participated in internal workshops and presentations, and collected internal secondary material. One of the authors had full access as an observer to most of the teams during the time they either applied the user input methods or while they developed new user input techniques. The researchers observed the development process and teams in their natural (physical) work setting (McDonald, 2005; Schultze, 2000) to understand how they work. Observations can provide unique insights into the day-to-day working practices because of its emphasis on the direct study of contextualized actions (McDonald, 2005).

All the qualitative data was systematically put through stages of naming data reduction; focused coding; and data display (Locke, 2001; Lincoln and Guba, 1985). Through open coding dominant themes were found that focused on the scope and maturity of the innovation, the input of the user in the innovation process, the techniques applied for user experiments, and the learning outcomes. The second step applied focused coding in these themes to extract passages related to these main themes as well as coding of subthemes (see example below). The third step was data display in which the researchers made lists and tables of emerging themes and monitored the internal cohesion of the codes. The coding was an iterative process between the two authors and both authors went trough several rounds of coding, and after each coding round the data was compared and discussed between the authors. An example of a set of codes on user input is mentioned in table 1. During this whole process the researchers found few discrepancies between the codes. The themes found in the codes were also related to the observational data. Conclusions drawn from the study were presented to some of the case studies for validation. Table 1: Example of codes Quotes (examples) Code: User input: (Case A) ‘In the past we did usability studies and got our (user) feedback through that, we watched the user application and came back and talked about it and adjusted it to broader changes. Nowadays in innovation jams the customer ideas are attacked (by users). It is a different approach, but the end result is that you get feedback.’ ‘Customers are engaged in discussions with: wow that would so great if you could build this...’ (Case D) ‘Customers get some kind of login account. Self service space. There the customer gets the ability to work with the Alpha.’

Conscious User input • Innovation phase • Technique Conscious User Input • Technique Conscious User Input • Technique

The case selection was performed in two steps. In an earlier study in which the authors looked at innovation processes the researchers studied case A and C and found that they showed different methods to collect user input. After this, the case selection focused on cases in which feedback from users and testing with users was applied on concepts, features, prototypes, or full products. The study has primarily focused on the high-tech industry. A high-tech firm is defined by Baruch (1997) as the following: the firm has a significant share of their organizational operations in their R&D budget, the area of activity is advanced technology, and the firm has a high proportion of academic and professional staff in their organization. The selected firms worked within the software, automotive and


9

telecommunication industry. Selection criteria that were applied were: application of user input, which supported product and service innovation. Section 3.1 discusses the eight cases in more detail.

3.1 Casestudies

Case A: Innovation Jams: The company is a Fortune 1000 company, building software products for mass-market deployment. The software development unit develops a product with a significant SaaS component. The company introduced a new agile approach to software development to overcome problems with alignment of user needs and the final product. The original process was rather hierarchically with little input from the engineers and the feedback loop between developed solutions and user feedback was the length of a full product release cycle, and this was too slow a cycle for learning. The new product development organization organizes full-day events for which 10-15 users are invited and all project management (PM) and product development (PD) staff, currently not full-time assigned to development projects, attends. Also, senior PM and PD management is present. These full-day events are called ‘Innovation Jams’. Typically, somewhere between 20 and 50 company staff attend the event. The users invited to the innovation jam are selected based on their potential interest in the areas highlighted by team members before the innovation jam event. During the innovation jam, the team fleshes out the user problem statement and starts to design a solution in terms of “mock-ups”. Based on user feedback, the team continues to iterate the design. At the end of the innovation jam, all teams present the ideas to a panel, consisting of the head of product management, head of product development and a selection of the users. The panel decides on a prioritization of the proposed solutions. Throughout the development, the user is present, gives feedback and test prototypes. Case B: Simulation: Case B is a Fortune 500 company in the automotive industry, building embedded, mobile systems with safety-critical elements. The company applies an agile software development approach with deeply embedded user feedback mechanisms in one of the user-facing subsystems as part of prototype development. In parallel to the traditional development of the mechanical and electronics parts, the team responsible for the software adopted an agile approach, releasing a production-quality version of the product every four weeks. Initially the versions were very light on features, but the team immediately started to collect user feedback by studying users operating the available features both in an available simulator as well as in retrofitted versions of the subsystem in earlier releases in the same product category. The feedback was used in every agile iteration, to select the next features to implement as well as to consider alternative implementation of already implemented features in case the feedback on a feature was below standard or the use of the feature as very different from the initially expected use.

Case C: Ethnographic user studies: One of our case study companies, a Fortune 1000 company, building software products, experienced challenges with the lack of understanding and user empathy of its engineering staff. In addition, it experienced a decreased "innovativeness" of its main products. In response, it institutionalized a program of ethnographic studies where small teams of product managers, user experience designers and engineers visit a user for a day. Different from traditional user visits, the team did not so much interview the user but rather followed the user during the day, with the intent of developing an improved understanding of the context of use of its products as well as the typical features used. The intent was to develop a degree of user intimacy and understanding that would result in superior implementation of already identified features as well as the development of viable hypotheses of innovative, valuable features to add to the products. The


10

program proved to be very successful and resulted in a situation where everyone in the organization involved in product development would spend at least one day per year with users. Case D: Feature Alpha: this case is about the introduction of new features in a mature product performed in a Fortune 1000 company that primarily builds software and services. In the case, rather than immediately driving for commercialization, the business unit decided that it was most important to maximize the user feedback. The team worked with a “Feature Alpha” in which lead users were given a free version of the latest stable release of the product extended with a new feature. The purpose was to collect user feedback of the initial version as well as constant evolutions of the feature implementation. The testing is concerned with evolving the feature in response to continuous user feedback. As the feature alpha contains mature software code, the user can use the feature alpha in their day-to-day operations. The user can also use the functionality associated with the concept, but with the understanding that the code is not yet of production quality and hence may experience stability and usability issues. The feature alpha allows the user to provide feedback on the concept in the context of their real business. Users are selected because of their specific interest in the functionality.

Case E: New user base: this case is about the introduction of a new online product solution to a new type of user, performed in a Fortune 1000 company that primarily builds software and services. In the case, rather than immediately driving for commercialization, the business unit decided that it was most important to maximize the user base and to achieve a significant network effect. Once the product was deployed (non-commercially) the team focused on growing the user base as rapidly as possible as a proxy for future business success. The team tracked the usage of the product and the satisfaction of the user through product instrumentation and in-product surveys and drove development based on active and passive user feedback.

Case F: Lead user development teams: This case is a Fortune 500 company working in the telecommunication industry. The teams work with large lead users for whom they build user requested features that extent the base product that all users receive. Recently the team has changed the interaction in the form of a dialogue during development of new features and finally products. The lead user is perceived as a more valuable source for product development for a larger user group. The development team has frequent development iterations in which prototypes or parts of features are tested and discussed with the user. Once a feature is developed enough for larger testing, the lead users gain the opportunity to access the particular feature in their existing product. Once the feature is matured with the lead user, the team verifies the general applicability of the feature through interactions with other users and, after typically small efforts, releases the feature for general availability for all users. Compared to traditional roadmap development, research at the company confirmed that the features developed by lead user development teams were delivered faster, had a better match with user needs and lead to higher user satisfaction.

Case G: Feature Optimization: This case is about user involvement in the final phase of product development. This case is from a Fortune 1000 software company, in which some of the SaaS properties of the product have already been commercially deployed for several years and have large user bases. Over the last years, the team responsible for one of these properties has adopted a vigorous experimentation approach using A/B testing. This final phase is for optimizing the features. The team has adopted a weekly cycle of tests where it will decide which experiments to run early in the week, take one or two days to develop alternative implementations of aspects of the system, deploy the solution and start the experiment towards the end of the week, collect data over the weekend and decide early the


11

week after which version (A or B) was more successful. Every week, the team runs dozens of experiments online and constantly improves the performance of the product and the user satisfaction. The user feedback is collected when users use the feature online and their click-behavior is saved.

Case H: Advertisement for gauging user interest: the case is from a Fortune 1000 company who tested out new concepts through advertisements delivered on a website, without actually developing any product or new feature. The user click-behavior is collected and analyzed to gain insight in the attractiveness of the concept. The advertising was applied when users entered an already deployed software product online, as well as through advertisements. The company experimented with different presentation forms, concepts for products, and features to determine the optimal value proposition and product conceptualization. The team experimented with different approaches. Initially, the team just served the ad both inside its mature product as well as on neutral websites such as Google and collected click-through behavior. Later, the team directed these users to a landing page where users would be asked to leave contact information as well as their preferences concerning new product and feature development. Finally, the team extended the approach with even demanding payment transaction before informing the user that the new product or feature was not yet available. Each additional step reduces the percentage and number of users that provide positive feedback, but on the other hand the signal that users send is stronger for every step.

In table 2 an overview of the cases and data collection per case is presented. Table 2: Overview of the case studies and data collection Case Industry Scope Innovation

phase No. of interviews

Other data collected

A. Innovation jams

Software New feature development

Pre-development

12 Workshops, presentation, observations

B. Simulation Automotive New product development

Pre-development

8 Secondary material, workshops

C. Ethnographic user studies

Software New feature & product development

Pre-development

12 Workshops, presentations

D. Feature Alpha

Software New feature development

During development


E: New user base

Software New product development

During development

4 Secondary material

F. Lead user development teams

Telecom New feature development

Commercial deployment

11 Workshops, secondary material

G. Feature Optimization

Software Feature optimization


6 Workshops, presentation, secondary material, observations

H. Ads for gauging user interest

Software New product development




12

4 FindingsThe following section discusses the main findings of the research. First, the article discuss different types of user involvement with the user feedback and data collection methods (section 4.1), secondly the study discusses different types of data that are collected through different techniques and in different innovation phases.

4.1 Userinvolvement

The researchers found two types of user involvement applied by the different case studies (see table 3). On the one hand users could be informed and consciously aware about the experiment, while on the other hand users could be uninformed and unaware that they were involved in the experiment process.

Conscious user input: Users were actively and consciously involved in the innovation process either in small groups, or through lead users. This conscious involvement of users was primarily performed in early development stages of the innovation process (in case A and B) in order to find out particular requirements, problems or ideas from the user.

In case A, the team invited 10-15 users in discussion and reflection sessions of a full day event. In these events, particular ‘problem areas’ of the users, which were stated or found earlier, are discussed and become a starting point for conversations and reviews with the users. During these events self-selected teams are formed, and the software teams select a user and user problems they want to work on. Based on the interviews, managers of this process mentioned that this immediate feedback and input from users caused the engineers to gain a better understanding of the users’ ideas, as well as causing them to focus on the right aspects of the solution immediately. The engineering team highly valued the user engagement and feedback. Engineers also mentioned that users were able to test and experiment features early on and discussed the positive responses from users on the developed solutions as validation and motivation for their work.

In case B users provided user stories on how they applied similar products and what they would prefer. These user stories were used as input for the next software sprints and feature developments. The case used a simulation in which users could apply the prototype for an infotainment system and could give feedback on the system.

In case C, every year engineers and project managers spend many hours visiting users in so called ‘ethnographic user studies’. The user was not aware of which features were observed during these visits, but the user was informed that they were part of the observation. An interviewee mentioned that: “this is partly to renew our connection and also to see what the user wants”.

In case E a larger user base was involved through online surveys with targeted questions towards the use of a particular feature the company wanted feedback on. The user involvement usually provided qualitative data for the R&D teams.

In Case F, the development team had close contact with the user units for large lead users. A frequent dialogue between the team and the user took place through the user unit on the requirements of new features, and in later stages on testing concepts and prototypes before the feature was offered online to the particular users. The engineers of the development team mentioned during the interviews that they gained a better understanding of the user requirements. Furthermore, the development team was also able to bring in their own knowledge of product development and sometimes suggested alternative solutions that were more suitable and user-friendly for the user than the solution requested by the user.


13

Uninformed user input: Next to the conscious user involvement primarily in early development stages, other cases applied tests and experiments with larger user sets in which users tested out features or products of which they were not consciously aware. This type of feedback and usage data is collected with different techniques.

In case C, the teams applied ‘ethnographic user studies’ to the customers home or office, but the user was not aware of which features were observed during these visits. Another approach was to collect data while the user is using the product or system, and this was evident in the cases B, D, E, F, G, and H. The later approach was often used when there was already a feature or product that could be used by users.

In case B (automotive industry) in addition to qualitative usage data, several kinds of other performance data are collected when users tested the products through the simulation. The case used a simulation in which users could apply the prototype for an infotainment system. The system was mounted in a prototype and showed two possible options to evaluate the layout of the start screen. In the tests, users were requested to perform some common task with the intent to measure which launcher worked best. The experiment showed that the on board system worked as well as the case collected user data which was analyzed later on and used in future sprints. In case D and E a new feature and new product were developed and a new market and user base needed to be set up. In order to gain user feedback, the team performed several experiments with online surveys (in which the users gave active feedback) as well as ‘labs’ website tests in which the users were not aware that their behavior was tested. In case E, the team worked with a separate website on which software prototypes and early versions of products are made available for the users free of charge. The primary purpose of this ‘labs’ website was to collect user feedback. Interviewees mentioned that a ‘labs’ website was perceived as valuable for firms that have an existing user base they can draw from for collecting feedback for new SaaS products that are under development. Existing, already deployed, products were in this case used to encourage users to try out the new product.

In case F a part of the data was collected through dialogues with lead users, while another part of the data was collected through user logs in the system. The users were unaware of the automated logs of data collection.

In case G, the user could experiment between multiple websites with different features, i.e., in this case two different versions A and B (AB-testing). The user was not aware that their clicking behavior was collected as data. The team has adopted a weekly cycle of experimentation where it will decide which experiments to run early in the week, take one or two days to develop alternative implementations of aspects of the system, deploy the solution and start the experiment towards the end of the week, collect data over the weekend and decide in the week after which version (A or B) was more successful. Every week, the team runs dozens of experiments and constantly improves the performance of the product and the user satisfaction.

Case H applied a non-informed input of users through a website with help of online ads as a mechanism to test concepts. The ad is presented on sites visited by potential users or inside existing products developed by the company. The goal of the advertisement is to evaluate the response (or click-through rate) of users. In addition, depending on the type of concept, users can even be asked to complete an order form including payment information before being informed that the product currently not available. According to the interviewees, this provides significant information about the attractiveness of the concept and even allows for testing different presentations of the concept through A/B testing. Advertising can be used to test both new products and new (cross-platform) features.


14

4.2 Typeofuserdataoverthewholeinnovationprocess

Next to the two different ways of how users were informed about their input to company’s innovation processes, it was found that there were different types of data that were collected over the course of the innovation process. These different types are discussed below and are related to the innovation phase in which they were primarily applied.

Pre-development phase: In the early pre-development innovation phases in which firms primarily worked with concepts and ideas instead of prototypes or features, much of the data collected came from close interaction with a smaller set of users, or even lead users. The users were actively and consciously involved in brainstorming on ideas, as well as prioritizing certain concepts (see cases A, C, and partly B). At this stage there is no working product or implementation and user input was only collected qualitatively through collaboration and dialogues (case A), user stories (case B) and observations of behavior (case C). The qualitative data was rich data in terms of understanding the context and background of the user. One example is mentioned by a member of case A: “I am not a process person and was hesitant in this part if this process would be helpful, since we knew the problems ... As soon as we got the users in the innovation jam, we found that ... a lot of topics we were worried about were discussed and users did not care about these. We got all this feedback in one day. This whole Q&A in one day with users had busted so many myths of the team on security issues, ideas etc. and they started validating all this in concepts. Engineers were just ‘wow’, in one day”. However, these techniques focus on only a limited number of users because of the amount of time needed to gain this rich user input.

During development: during the later innovation stage, different sets of data from users

were collected. In case D and E, features and product ideas with users are tested before the product is deployed onto the market. These teams employed different techniques to gain user feedback. In case D a selected group of users, those users who were interested in this particular feature, tested this feature in their own context (either own office, or home office) without being informed about what they were tested upon. The click-behavior and system use of different versions of the feature (AB-testing) was collected automatically and was applied as quantitative data. One member of case D mentioned: “the users are able to get some real code really early on. Users can use their own data and this helps in finding the right type of feedback. Our users had no problems with their data versions and could test the Alpha feature in a safe environment.” In case E online surveys on use and satisfaction were applied for which the company received quantitative feedback, but also qualitative feedback in the forms of suggestions and ideas in a crowdsourcing approach through labs’ websites. In case F data was collected through collaboration and dialogues with the lead users and led to more rich data concerning new feature development for these particular users. However, next to qualitative rich data, also quantitative log information and user behavior was collected automatically once users logged onto the system. This set of quantitative data comprised a large set of users. For example in case E, the uninformed user input helped the team to study new potential user bases through their tests and experiments for a low cost. One manager of case E mentioned: “instrumenting products with a data collection infrastructure allows for a very low-cost collection of data.”

In this particular innovation phase, the case companies used a combination of on the one hand qualitative rich user input through crowdsourcing and dialogues. While on the other hand the case companies also tested user behavior online through AB-testing or logs through which they received a larger scale of quantitative user behavior data.


15

Commercial deployment phase: Once the product was deployed and placed on the

market, companies could receive input through tests online. In case G and H, data from the users is collected in order to optimize certain features of the deployed product (case G) or as input for new development (case H). In case H in-product discovery and advertising were used as a mechanism to gauge user interest in alternative business models, new features and new products for cross-sell and up-sell. In both case G and H the users were uninformed and were not aware that their click-behavior online (AB-testing) was recorded. The quantitative data that was collected covered a very large set of users and their user-behavior (click-behavior), which to both case G and H increased the validity of the user input. Furthermore, the use of large-scale online user input data collection implied a low cost for the companies to collect user feedback based on actual behavior. The use of data feedback based on actual behavior instead of on opinions, discussions, or ideas also provided a different and more reliable set of data of a large set of users.

Concluding, most cases apply user input through conscious, qualitative and rich data

methods in the earlier phases of the innovation process in order to collect ideas, concepts, suggestions, and problems. While in the later phases of the innovation process, once there is a concept or partial product available, user input is more collected through other means. These methods are more relying on larger quantitative data sets of users who are unaware that they are participating in tests. Many of the case companies apply new technology in particular in the later innovation stages like AB-testing (case D and G), online surveys (case E), lab websites and crowdsourcing (case E), or online advertisements which could be perceived as a type of uninformed crowdsourcing (case H). In table 3 an overview of the different user input methods and their type of data is shown. Table 3: User input methods per case study Case Scope Innovation

phase User input Type of data Techniques for

user input A. Innovation jams

New feature development

Pre-development

Conscious Qualitative Collaboration and Dialogues

B. Simulation New product development

Pre-development

Conscious Uninformed

Qualitative Quantitative

User stories Simulation of behavior

C. Ethnographic user studies

New feature & product development

Pre-development


Qualitative Ethnographic study

D. Feature Alpha

Feature development

During development

Uninformed Quantitative AB/ multi-variety testing

E. New user base

New product development

During development


Quantitative Quantitative

Online surveys Lab websites- crowdsourcing

F. Lead user development teams

New feature development


Conscious Unformed

Qualitative Quantitative

Collaboration and Dialogues Log input

G. Feature optimization

Feature optimization


Uninformed Quantitative AB / multi- variety testing

H. Ads for gauging user interest

New product development


Uninformed Quantitative Ads in existing products on the web- type of crowdsourcing


16

5 DiscussionThe main contribution of this article is two-fold, (a) in the next section a conceptual framework for user data driven innovation throughout the innovation process is discussed and (b) the article presents implications of collecting and applying user input and feedback throughout the whole innovation process for high tech firms. Below these two main contributions are discussed.

5.1 Userdatadriveninnovationframework

Currently in the high-tech industry, in which a large focus has been on the use of software in their products, a number of methods and approaches are applied to gain user feedback and information throughout the whole innovation process. User input driven innovation methods become important across the entire scope of the innovation process in order to maximize the success rate of new innovations. From the case material it is found that companies are using new developments in technology to gain user data and test as many ideas as possible with users to drive user satisfaction and, consequently revenue growth. Some of these technologies are reported earlier in literature like the virtual user integration tools through the Internet (Rohrbeck, Steinhoff and Perder, 2010). However, the cases apply other types of technology as well next to just the Internet in order to collaborate or interact with their users. From marketing and R&D research it is known that many approaches exist to gain user input and information (Janssen and Dankbaar, 2008; Kaulio, 1998; Van Kleef, van Trijp and Luning, 2005; Veryzer, 1998), or to develop collaboration and interaction with users (Enkel, Kausch and Gassmann, 2005; Lüthje and Herstatt, 2004; Rohrbeck, Steinhoff and Perder, 2010). However, finding suitable techniques and approaches how firms can involve users and gain more user input throughout the whole innovation process has been more difficult (Janssen and Dankbaar, 2008) and many approaches have focused on the early innovation stages in terms of collaboration (Schaarschmidt and Kilian, in press) as well as in more virtual user integration (Rohrbeck, Steinhoff and Perder, 2010). Furthermore, due to new developments in information and communication technology and social technologies, companies can make use of different online or technology based techniques to gain feedback and input from users. Examples are methods of AB-testing and crowdsourcing, which could be viewed as experimentation or testing methods that are now applied to gain user input to support innovation. Below, figure 1 presents elements towards a conceptual framework based on the findings of the case study (see figure 1).


17

Figure 1: Conceptual framework for user data driven innovation

The conceptual framework consists of the following elements: (1) Innovation phase; (2) User involvement; (3) Type of data collected and (4) Amount of data.

Innovation phase: The data of the eight cases shows how R&D teams work with help of different techniques to gain feedback and input from users in different phases of the innovation process. Based on our eight cases it is found that user input through different approaches is not only applicable in particular stages of the innovation cycle, but can be applied through-out the whole duration of the innovation cycle, i.e., the pre-development, during development, as well as the commercial deployment stages (see table 3). Many approaches for user integration and input are applied in the early conceptual stages and some are applied in the deployment stages and sales (see Van Kleef, van Trijp and Luning, 2005), while fewer techniques are available in the stages of during development. Particularly, in the software industry the user has been viewed as an active co-creator in for example agile software development (Boehm and Turner, 2004; Bosch and Bosch-Sijtsema, 2011), or the user supports innovation with behavioral data through optimization experiments after product deployment (Agarwal et al., 2008; Davenport, 2009; Eisenberg and Quarto-vonTivader, 2008; Kohavi et al., 2009). However, few perceive the user as contributing with input throughout the whole innovation cycle. With help of the cases it is shown that user input can take place in all innovation cycles. Our framework supports both the more collaborative approaches that are applied in the early conceptual phases and slowly develops towards a more information and data collection approach through testing and experiments in the later phases of innovation. The combination of these different approaches might integrate the discussed difference in literature between collaborative user involvement originating from an open innovation perspective, in relation to data collection from a manufacture active perspective (see Rohrbeck, Steinhoff and Perder, 2010; Schaarschmidt and Kilian, in press).

User involvement: From the cases it is found that R&D teams apply both conscious and uninformed user feedback and input in different stages of the innovation process. In the

Qualita've*

Quan'ta've*

Small*size*of*rich*data*

Large*size*of*data*

Uninformed*

Conscious*

Customer)Type)data)

Amount)of)data)

Pre;development* Post;deployment*

Innova4on)phase)

Dialogues**(A)*

Simula'on*(B)*

Ethnographic*study*(C)*

AB;tests*(G)*Ads*in*exis'ng*products*online*(H)*

AB;tests*(D)*

Lead*user*dialogue*(F)*

Lab*website*crowdsourcing*(E)*

User*stories*(B)*

Logs*(F)*

Surveys*(E)*

During*development*


18

cases the researchers found support for conscious user involvement and input as techniques that can be applied in the earlier stages of product and feature development. Conscious user feedback is concerned with ethnographic studies, dialogues, review sessions, and surveys where the user is aware that they are providing feedback. Several of the more conscious techniques have been discussed in literature like ethnography, crowdsourcing design and development (Janssen and Dankbaar, 2008), focus groups (Van Kleef, van Trijp and Luning, 2005), lead users (Lüthje and Herstatt, 2004; Von Hippel, 1986), concept testing, and product and beta testing (Kaullio, 1998; Van Kleef, van Trijp and Luning, 2005). In the literature on feature optimization, uninformed user input is discussed (Agarwal et al., 2008; Eisenberg and Quarto-vonTivader, 2008; Kohavi et al., 2009) in which users participate unconsciously in different experiments of software features and products. However, the uninformed user input can only be used once there is a relatively mature implementation of the feature or product available and the company has access to a sizeable user base. The uninformed user input is collected through click-behavior of users.

Type and amount of data: The different techniques applied offer both qualitative and

quantitative input of users to the R&D teams. Qualitative and rich data was more collected during earlier innovation phases in order to gain more insight of users’ needs and contexts. This qualitative feedback was often based on dialogues or user stories and users were able to actively and consciously help shape the development of new functionalities and products. Quantitative and large-scale data sets were collected towards the final phases of the innovation process, when a product or feature had been partially or fully launched on the market and users would be able to use the particular feature and product. The large-scale set of data gave a different kind of feedback to the innovation process, but could provide more reliable real-behavioral data based on the click-behavior of users while using the system. However, the user is not actively involved in giving feedback, but feedback is automatically generated through online behavior. While more and more organizations are collecting this kind of quantitative data, discussions are arising in media about ethics and privacy of users behavior. This is an element that needs to be taken into account when focusing on collecting user feedback for innovation. Both qualitative and rich feedback of users as well as quantitative data trough surveys and tests are common in user input studies throughout all development phases (Van Kleef, van Trijp and Luning, 2005; Veryzer, 1998). However, through the use of new technology in the cases studied, the data is in earlier phases more connected to rich feedback, while in later phases larger scale data is automatically generated through the actual use and behavior of users.

5.2 Implicationsofuserinputfortheinnovationprocess

The new developments in technology in terms of SaaS, Web2.0, and social networks, support different and new techniques to gain user input with both rich data in the earlier phases and more actual click-behavior in later innovation phases. Based on our findings within the case studies and literature it is argued that with help of an integrated user data driven innovation system, high-tech firms with a strong software focus in their products are able to have frequent iterations of their products, continuous user input either qualitatively or quantitatively and the firm can learn faster by multiple iterations. These findings were clear from the case studies but are also confirmed by literature concerning experimentation (Eisenhardt and Tabrizi, 1995), learning by doing and practice (Orlikowski, 2000; Von Hippel and Katz, 2002), and speed up development through testing (Gupta and Wilemon, 1990). Furthermore, through the user input approaches it was found that firms gained a


19

deeper understanding of their users, markets, as well as their own products and way of working. These findings are confirmed in innovation literature in which user input is perceived as beneficial for a firm’s growth and profitability (Veryzer, 1998) as well as for developing knowledge and capabilities (Iansiti and Clark, 1994). With a focus on user input throughout the whole innovation cycle the researchers propose that firms can increase their speed of learning as well as develop a continuous learning cycle in which a continuous innovation iteration process is applicable. The learning that occurs through continuous user feedback throughout the innovation process changes the speed and iterations in the development process. The continuous innovation iteration process has implications for how firms organize their innovation process as well as proposes a different perspective on innovation. In such a continuous innovation iteration process, the final product is never finished, but is constantly optimized, developed and tested based on user feedback. A second implication of this user data driven innovation system is that the data collected is not only based on user’s perceptions but also on their actual behavior and therefore innovation is less prone to the drawbacks of customer input, i.e., dependency on customers’ view, the misunderstandings between firms and their users (Enkel, Kausch and Gassmann, 2005; Schaarschmidt and Killian, in press), the fact that users not always know what they want (Ulwick, 2006), and the costs involved of transferring sticky and rich information from customers to R&D (von Hippel, 1994). The overall goal of the continuous innovation iteration process is to maximize the return on product R&D investment. Based on the cases described in this article, this can be best achieved by minimizing the R&D investment between user proof points, through interviews, prototype and split tests, and data collection with users and that are used to confirm and support that R&D activity optimally, meets the needs of users, as well as meets the strategic business goals of the organization. One can observe an interesting parallel with technology or research driven innovation where innovation typically requires extensive experimentation and testing to confirm the desired qualities of a new or improved technology (see Lee et al., 2004; Thompke, Von Hippel and Franke, 1998; Thompke, 2001). However, the user is often not involved in these types of experimentations. The combination of both collaborative approaches and conscious user involvement for rich information in early phases, and the more uninformed user experiments and tests with large quantities of data on actual behavior in later innovation phases, can integrate the different perspectives lifted up in literature concerning user input and collaborative user involvement approaches (Jeppesen, 2005; Rohrbeck, Steinhoff and Perder, 2010; Schaarschmidt and Kilian, in press).

This article contributes to innovation literature with the development of a conceptual framework for different types of user-involvement and data that can be applied throughout the whole innovation process. As a practical contribution, this article is intended as a framework for R&D innovation managers to apply their resources to product and feature innovation in an effective way. Based on the case studies, R&D managers can maximize positive outcomes through different user feedback methods to develop new products and features that are appreciated by their users, by collecting data through both conscious and non-informed user input which can support optimization, and new feature and product development. In the cases the researchers noticed a number of implications of implementing a user innovation experiment system in the firm like an increase in development speed, decrease in cost, higher user satisfaction, and a change in team motivation and leadership. However, future research needs to study the implications of such a framework on an organizational level as well as developments of new business models that support innovation


20

systems based on continuous iteration cycles. The findings of the article predominately focus on the software, automotive, and telecommunications industry. However, our findings can be applied to any industry that employs software in its products and allows these to be connected to the Internet. Also, although the researchers focus on the user and user input in this article, the basic principles of continuous iteration cycles can be applied for other properties of the product or service besides user experience. For instance, the reliability of software intensive systems can be improved after the deployment of the product or system by experimenting with alternative implementations of the same functionality during operations.

Acknowledgements: We hereby would like to thank the case companies for their time and input in the article. Furthermore, we are grateful for the constructive comments and support of the editor professor Anthony Di Benedetto and the anonymous reviewers.

6 ReferencesAgarwal, D. Chen, B., Elango, P., Ramakrishnan, R., Motgi, N., Roy, S., and Zachariah, J. 2008. “Online

Models for Content Optimization.” Neural Information Processing Systems (NIPS). http://research.yahoo.com/node/2428.

Boehm B, Turner R. 2004. Balancing Agility and Discipline - A Guide for the Perplexed. US: Addison-Wesley. Bogers, M., Afuah, A., and Bastian, B. 2010. “Users as innovators: A review, critique, and future research

directions.” Journal of Management 36 (4): 857–875. Bosch, J., and Bosch-Sijtsema, P.M. 2011. “Introducing agile user-centred development in a legacy software

product line.” Software: Practice and Experience 41(8): 871-882. Bosch, J. 2012. “Building products as innovation experiment systems.” Pp. 27-39 in: Software Business. Lecture

Notes in Business Information Processing, edited by M.A. Cusumano, B. Iyer and N. Venkatraman. Berlin: Springer.

Bosch, J., and Eklund, U. 2012. “Eternal Embedded Software: Towards Innovation Experiment Systems.” Proceedings of International Symposium on Leveraging Applications, Crete, 15-18 October.

Chesbrough, H. W. 2003. Open innovation: the new imperative for creating and profiting from technology. Boston, MA: Harvard Business School Press.

Davenport, Thomas H. 2009. How to Design Smart Business Experiments. Harvard Business Review 2009, February.

Eisenberg, B., and Quarto-von Tivadar, J. 2008. Always Be Testing: The Complete Guide to Google Website Optimizer. Indiana: Wiley publishing.

Eisenhardt, K. M. 1989. “Building theories from case study research.” Academy of Management Review 14:532–550.

Eisenhardt, K.M., Tabrizi, B.N. 1995. “Accelerating adaptive processes: Product innovation in the global computer industry.” Administrative Science Quarterly 40(1): 84-110.

Enkel, E., Kausch, C., and Gassmann, O. 2005. “Managing the risk of customer integration.” European Management Journal 23(2): 203-213.

Gupta, A.K., Wilemon, D.L. 1990. “Accelerating the development of technology-based new products.” California Management Review 32(2): 24-44.

Iansiti, M., and Clark, K.B. 1994. “Integration and Dynamic Capability: Evidence from Product Development in Automobiles and Mainframe Computers.” Industrial and Corporate Change 3 (3): 557-605.

Janssen, K.L., and Dankbaar, B. 2008. “Proactive involvement of consumers in innovation: selecting appropriate techniques.” International Journal of Innovation Management 12(3): 511-541.

Jeppesen, L.B. 2005. “User toolkits for innovation: Consumers support each other.” Journal of Product Innovation Management 22 (4): 347-362.

Kaulio, M.A. 1998. “User, consumer and user involvement in product development: A framework and a review of selected methods”. Total Quality Management, 9(1): 141-149.

Kohavi, R., Longbotham, R., Sommerfield, D., and Henne, R.M. 2009. “Controlled experiments on the web: survey and practical guide.” Data Mining and Knowledge Discovery 18(1): 140-181.


21

Kristensson, P., Gustafsson, A., and Archer, T. 2004. “Harnessing the Creative Potential among Users.” Journal of Product Innovation Management 21(1): 4–14.

Lee, F., Edmondson, A.C., Thomke, S., and Worline, M. 2004. “The Mixed Effects of Inconsistency on Experimentation in Organizations.” Organization Science 15(3): 310-326.

Lincoln, Y.S. and Guba, E.G. 1985. Naturalistic Inquiry. Beverly Hills: Sage. Locke, K. 2001. Grounded theory in management research. London: Sage. Lüthje, C., and Herstatt, C. 2004. “The Lead User Method: An Outline of Empirical Findings and Issues

for Future Research.” R & D Management 34(5): 553–68. Magnusson, P.R. 2009. “Exploring the contributions of involving ordinary users in ideation of technology-

based services.” Journal of Product Innovation Management 26: 578-593. McDonald, S. 2005. “Studying actions in context: a qualitative shadowing method for organizational

research.” Qualitative Research 5: 455-473. Olson, E.L., and Bakke, G. 2001. “Implementing the Lead User Method in a High Technology Firm: A

Longitudinal Study If Intentions versus Actions.” Journal of Product Innovation Management 18 (6): 388–395.

Orlikowski, W.J. 2000. “Knowing in practice: Enacting a collective capability in distributed organizing,” Organization Science 13(3): 249-273.

Poetz, M. K., and Schreier, M. 2012. “The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas?” Journal of Product Innovation Management 29(2): 245–256.

Prahalad, C.K., and Ramaswamy, V. 2000. “Co-opting User Competence.” Harvard Business Review 78(1): 79–87.

Rohrbeck, R., Steinhoff, F., and Perder, F. 2010. “Souring innovation from your customer: how multinational enterprises use web platforms for virtual customer integration.” Technology Analysis and Strategic Management 22(2): 117-131.

Rosenthal, S.R., and Capper, M. 2006. “Ethnographies in the front end: designing for enhanced user experience.” Journal of Product Innovation Management 23: 215-237.

Schaarschmidt, M., and Kilian, T. in press. “Impediments to customer integration into the innovation process: a case study in the telecommunications industry.” European Management Journal, online: http://dx.doi.org/10.1016/j.emj.2013.04.004

Schultze, U. 2000. “A confessional account of an ethnography about knowledge work.” MIS Quarterly 24, 3-41.

Thomke, S.H. 1998. “Managing experimentation in the design of new products.” Management Science 44(6): 743-762.

Thomke, S.H. 2001. “Enlightened Experimentation: The New Imperative for Innovation.” Harvard Business Review 79(2): 67-76.

Thomke, S.H. 2003. Experimentation Matters: Unlocking the Potential of New Technologies for Innovation. Boston: Harvard Business School Press.

Thomke, S, Von Hippel, E., and Franke, R. 1998. “Modes of experimentation: an innovation process - and competitive - variable.” Research Policy 27: 315–332.

Ulwick, A.W. 2002. “Turn User input into innovation.” Harvard Business Review, January: 1-8. Van Kleef, E., Van Trijp, H.C.M., and Luning, P. 2005. “Consumer research in the early stages of new

product development: a critical review of methods and techniques.” Food Quality and Preference 16: 181-201.

Veryzer, 1998. “Key factors affecting customer evaluation of discontinuous new products.” Journal of Product Innovation Management, 15: 136-150.

Von Hippel, E. 1986. “Lead Users: A Source of Novel Product Concepts.” Management Science 32(7): 791–805.

Von Hippel, E. 1994. "Sticky information" and the locus of problem solving: Implications for innovation.” Management Science 40(4): 429-439.

Voss, C.A. 1985. “The Role of Users in the Development of Applications Software.” Journal of Product Innovation Management 2(2): 113–21.

West, J., and Gallagher, S. 2006. “Challenges of open innovation: the paradox of firm investment in open-source software.” R&D management 36(3): 319–331.

User involvement throughout the innovation process in high-tech industries

Documents

Transcript of User involvement throughout the innovation process in high-tech industries