Discovering automation level of software change request process from qualitative empirical data

Zeljko Stojanov University of Novi Sad, Technical faculty “Mihajlo Pupin”, Zrenjanin, Serbia

zeljko.stojanov@tfzr.rs

Abstract—This paper presents an approach in discovering automation level of software change request process from qualitative empirical data. In the study is investigated common practice in local very small software companies. Constructivist grounded theory was used as a research method suitable for discovering common practice and for creating explanations that are grounded in empirical data. In the research participated software experts from local very small software companies. Software experts were interviewed individually in their companies or participated in organized focus groups. The process of qualitative research that is used in the study is outlined, followed by the research findings. Analysis of empirical data reveals that very small software companies do not have automated software change request process, or have automated only parts of the process.

I. INTRODUCTION Software engineering research has been recently more

oriented towards real industrial problems and practice [1]. However, most of the software engineering’s research is rather directed toward providing prescriptions to practitioners what they ought to be doing, then to understanding what practitioners actually are doing [2]. A serious problem with research and practice in software engineering is related to understanding of the specific factors that cause tools and methods to be more or less cost-effective in the practice [3]. This further leads towards the gap between research and practice. The main reason for this gap is the fact that practitioners are deeply involved in today’s projects, while researchers often do not know what actually happen in practice. Practitioners will perceive researchers’ solutions as helpful only if researchers recognize that practitioners live in an imperfect world and understand issues that constrain their daily activities [4]. The gap between the state of the research and the state of the practice can be overcome through better understanding between researchers and practitioners. That means that researchers must place more emphasis on the problems identified in industry, while practitioners must frame their problems so that researchers can address the underlying scientific issues. According to Rainer [5], researchers need to strengthen collaborations with industry, because industry is both the most appropriate source of empirical evidence for developing understanding of the practice and the intended target for action. Higher quality, greater relevance of empirical studies, and better transfer of research results can be achieved by using the software industry as a laboratory [6]. Chiasson and Davidson [7] reported that the similar situation is in the field of information systems research where industrial practice is seldom considered.

The aim of empirical research is to explore, describe, predict, and explain natural, social, or cognitive phenomena through evidence based on observation or experience. According to Segal, understanding the nature of software engineering practice should be central to the discipline of empirical software engineering [8]. The common scenario for empirical research consists of the following steps [3]: formulating an hypothesis or question to test, observing a situation, abstracting observations into data, analyzing the data, and drawing conclusions with respect to the tested hypothesis. Quantitative studies, which are dominant in empirical software engineering, use this scenario in order to test hypotheses through experimentation and statistical analyses of the results [9]. Despite the dominant role of quantitative research methods, there are circumstances when they are inappropriate to comprehend the complexity that is embedded in software practice. Seaman stated [10]: “many in the industry recognize that software development also presents a number of unique management and organizational issues, or ‘people problems’, that need to be addressed and solved in order for the field to progress”. These situations that address software engineering as a social activity, thus emphasizing how software practitioners make sense of methods in their daily work, are well suited for qualitative research methods [11]. The findings from qualitative studies provide insight into the social dimensions that are an intrinsic part of software engineering practice.

Silverman [12] suggested the view of qualitative method analysis as how people do things, rather than how people see things. Qualitative research also provides results and knowledge that is an essential prerequisite for the generation and testing of theories and hypotheses and for interpreting the results of such empirical studies [10].

When the research goal is to investigate common industrial practice the one of suitable research methods is grounded theory that is regularly used in qualitative research. Grounded theory method consists of a set of systematic, but flexible, guidelines for conducting inductive qualitative inquiry with the main aim to construct a theory [13]. Grounded theory can also be used as qualitative research method, in the cases when the result of the research is not developed theory but explanation of the practice [14].

Because of its flexibility, grounded theory was in this research used as a method for exploring software engineering practice in very small software companies. This research used grounded theory research procedures proposed by Charmaz [14]. Charmaz proposed constructivist approach of grounded theory that is based

– 51 –

6th IEEE International Symposium on Applied Computational Intelligence and Informatics • May 19–21, 2011 • Timişoara, Romania

978-1-4244-9107-0/11/$26.00 ©2011 IEEE

on Corbin and Strauss work [15]. The main point in constructivist grounded theory is creative writing as a form of expression that has the potential to reveal how participants construct their worlds [16]. This approach is pragmatic and assumes that researcher has some knowledge of the studied phenomenon. This practically means that researcher may have many years of experience in the field and/or is familiar with the relevant literature. Preliminary literature review provides insight into the previous research and identification of gaps that should be investigated [14]. This can positively affect the research of the practice.

Software maintenance is typically perceived as being expensive and ineffective [17]. Therefore, for the investigation is selected software change request process as an essential sub-process in software maintenance [18]. In this paper is presented the process of investigating the automation level of software change request process in very small software companies. This implies a straightforward inductive and analytical process from things to thoughts that is fully supported by the technical procedures and analytic tools of grounded theory [19]. Research findings are grounded in qualitative data collected through semi-structured interviews with 15 software experts that work in local very small software companies.

II. BACKGROUND This research has background in the fields of grounded

theory research and common practice in very small software companies. Therefore, short reviews of these research fields are necessary for understanding and positioning this research.

A. Grounded theory Grounded theory emerged as a research method during

the research studies of sociologists Barney B. Glaser and Anselm L. Strauss. Their book The discovery of Grounded Theory, published in 1967, advocated development of theories from the research grounded in data through iterative inductive process [20]. The basic components of grounded theory defined by Glaser and Strauss are:

• Simultaneous data collection and analysis, • Construction of analytic codes, concepts and

categories from collected data, • Comparing the collected and analyzed data during

the each stage of the research, • Sampling data until no new concepts emerge

(saturation of developed categories is achieved), • Iterative development of theory based on

collected and analyzed data, • Writing memos to elaborate all aspects of the

research. In literature, the term grounded theory is related both to

research methodology and to the theory generated as a result of the research. The data for a grounded theory study can come from various sources, such as interviews, questionnaires with open-ended questions, documents, observations, audio and video material, newspapers, letters and books. In [21] are outlined procedures and canons for grounded theory research, as well as evaluative criteria that should be used to judge about the research.

Discovered grounded theory must meet the following criteria [20]:

• Close fit with the data, • Usefulness, • Conceptual density, • Durability over time, • Modifiability and • Explanatory power.

In the beginning, grounded theory was used in social sciences in disciplines such as sociology, psychology, education and medical research. Since 1980, it has been adopted in technical sciences, especially in the field of information systems research [22][23][24][25].

Grounded theory methods have recently gained attention in software engineering studies with the focus on human aspects of software engineering. These methods have been used to discover a theory about an investigated phenomenon from the practice [26]. Seaman advocated the usage grounded theory research methods to study software development and maintenance in real settings [10].

Grounded theory have been used in many areas of software engineering such as cognitive dimensions in learning software modeling [27], understanding of software process improvement [28][29], installation and use of an electronic process guide within a small-to-medium software development company [30], descriptive process modeling [31], understanding the social processes that influence software team performance and the influence software methods have on those processes [32], the impact of background and experience on software inspections [26], investigation of agile user-centered design in practice [33], or management of software change tasks using a state-of-the-practice IDE to a medium-sized open source software system [34].

B. Practice in very small software companies Small companies are dominant in economies across the

globe [35]. Laporte et al. [36] reported that in Europe, 85% of IT sector companies have between 1 and 10 employees. Organizations that have less than 25 employees are defined as very small enterprises (VSEs) [36][37][38]. The main challenges posed to software VSEs are [35]:

• Managing and improving their software processes,

• Dealing with rapid technology advances, • Maintaining their products, • Operating in a global software environment, and • Sustaining their organizations through growth.

The maturity levels of processes within the very small companies have broad range from very good processes to low level processes. Most of the processes are performed through an informal way and without any documentation. In many cases, processes are implemented in ad-hoc way.

Little published work is related to support processes for very small software organizations [38]. This is particularly true for software maintenance processes that are focused on maintenance activities after software delivery. Useful and successful software systems stimulate user-generated requests for change and improvements during the

Z. Stojanov • Discovering Automation Level of Software Change Request Process from Qualitative Empirical data

– 52 –

maintenance phase of life cycle [39]. Investigations and assessments of current perceptions and practice must be performed before launching any improvement activity [40]. These investigations and assessments should help in identifying potential areas and barriers for practice improvement.

This observation of literature that deals with the practice of very small software companies reveals that it is important challenge to study maintenance activities and processes in those companies. Software change request process is one of the essential processes in software maintenance phase because it is oriented towards solving problems and requests identified by companies’ clients. Therefore, software change request practice in small software companies requires continuous assessment and improvement.

III. RESEARCH METHODS In this research is used a tailored version of techniques

that are common practice in constructivist grounded theory research [14]. Two types of interviews were used in the research: focus groups and individual interviews [41][13]. All interviews were semi-structured with focused open-ended questions. Open-ended questions provide the opportunity to research participants to choose the terms with which they want to construct their descriptions. The purpose of interviews was to gather information about research topic through a set of focused open-ended questions. The general research goal of this study is to develop a broad and deep understanding of the practice, rather than a quantitative summary.

The modification of common research practice is in the method for collecting participants’ answers during interviewing. Participants’ answers were not tape-recorded. Instead of that, all participants were asked to write their answers on questions that were introduced and shortly discussed. In-depth descriptive fieldnotes [13, pages 341-343] about the context, participants and the process of research were written by the researcher as a complement for the collected qualitative data. Some quantitative data were extracted from the collected text and used as a complement to the qualitative data [42].

A. Research context In research participated software practitioners from

very small software companies in Zrenjanin (Serbia) and Timisoara (Romania). Experience levels of participants varied between two and twenty four years with an average of 7,47 years of experience in the practice. Two focus groups were organized during the research. The first focus group was organized with four participants in Business incubator in Zrenjanin. The second focus group with eight participants was conducted in Timisoara software business incubator. Focus groups’ participants were invited to participate in the groups through the announcements on the local incubators’ web sites. All software practitioners participated in the focus groups on the voluntary basis. The focus group in Zrenjanin was conducted in Serbian, while the focus group in Timisoara was conducted in English.

Other three software practitioners were individually contacted, and after that interviewed in their companies in Zrenjanin. Individual interviews were conducted in Serbian. All participants work in software companies that

develop software for both local clients and clients from other countries. Interviews were conducted from November 2009 to February 2010. The last two interviews were individual in participants companies.

B. Research procedure Interviews lasted between 45 and 60 minutes. The

research topic (software change request process) and the goal (exploration of current practice) were presented at the beginning of the interviews. After that, short discussion about the current practice on collecting and processing software change requests from customers was conducted. During these discussions were written fieldnotes. After the discussions, participants wrote answers on open-ended questions. During the writing responses on open-ended questions, participants were also allowed to ask questions that helped them to provide answers.

Collected empirical data were carefully read and coded with techniques presented by Charmaz [14, chapter 3]. Each piece of data was analyzed and compared with the previously analyzed data. Data collection process was completed when the clear repetitions within the data (data saturation) occurred [14, chapter 5]. That happened after the fifteenth interview. Memos were written during the whole research process. Writing memos is a method that is regularly used in grounded theory for analyzing data and codes, developing ideas and concepts [14, chapter 4]. Memos are intermediate step towards generating research reports, and they form the core of grounded theory. In this research, memos were used to develop ideas of basic concepts such as software change request process, software change request, specification approach and stakeholders.

C. Research findings In the research were used two sources of data: text from

user answers on open-ended questions, and fieldnotes recorded by the researcher during the interviews and in informal discussions with participants.

Common practice and problems were identified and derived from available empirical data during the coding process described by Charmaz [14, chap. 3]. Participant’s answers were first coded with initial coding technique. Further analysis of data and initial codes provides the basis for defining concepts and categories that describe common practice about SCR process in software companies. This analysis is realized as focused coding. Data collection process was completed after the fifteenth interview because of the clear repetitions within the data. The repetition within the collected data is called saturation [14, chap. 5][15, chap. 7]. This saturation within the data further leads to the identification of common characteristics and level of SCR process automation in very small software companies.

As a core category was identified ‘software change request process’. The core category represents the central phenomenon of the study and it emerged from among the categories already identified [21]. The other categories, such as request or stakeholder, stand in relationship to the core category. Categories were developed from memos, which are based on codes extracted from the text. Memos contain detailed descriptions of derived codes, concepts and categories. The automation level of the process is classified as a property of the core category. From the

– 53 –

6th IEEE International Symposium on Applied Computational Intelligence and Informatics • May 19–21, 2011 • Timişoara, Romania

collected data were also derived other properties of the core category, such as duration or exceptions.

The level of process automation in SCR specification practice is based on the answers provided by software experts on the question: Have you implemented any automated and/or controlled process for collecting and handling software change requests from your users? Provide short explanation. Sample answers on the stated question are:

Answer 1: A client submits a request usually by phone. Our team member records the request through an internal Web application. After that, the request is analyzed as soon as possible and assigned to a programmer. This assignment is in internal application represented as a task. After implementing the requested change, the evidence about the change is recorded in the application. There is no feedback to the client about the status of the request in the process. Some segment of our process should also be supported by internal application through development of additional modules.

Answer 2: Customers submit change requests by phone or by filling a web form on our web site. If the data submitted for change request are incomplete, the customer is contacted by phone or email to provide additional data. It usually takes 2-3 days to specify change request. The implementation of change depends on its complexity and feasibility. The steps in the process are not recorded internally.

These two answers denote cases were some kind of semi-automated process could be found. Other participants in the research provided similar answers with short descriptions of processes used in their companies.

Insight in the answers reveals that four experts have partially automated SCR process in their companies. This quantitative data is extracted by counting the number of collected data where it is noted the existence of partially automated process in the company. Other experts reported that there is no automated SCR process in their companies. The main characteristics of SCR process in observed very small software companies are:

• Process is combination of various approaches in collecting SCRs (phone call, email, direst contact, Web form on the company Web site),

• Process is adapted to the current case (user, client),

• Process is not automated or it is partially automated in few cases,

• Change implementation time depends mainly on the change complexity,

• Change approval depends on cost estimates and feasibility,

• In some cases required change will be implemented only if it could be implemented to a large group of clients,

• Usually there are many contacts with customer before a change is going to be implemented.

The main research finding is that the majority of local very small software companies does not have standardized and/or automated process for collecting and handling change requests. This is outcome of the fact that small software companies have small teams that are task oriented. Furthermore, these findings agree with the

discussion that can be found in literature about small software companies. Ribaud et al., stated that small software companies “cannot afford the resources – in number of employees, cost, and time - or see a net benefit in establishing software life-cycle processes” [38]. Because of the very low software engineering maturity level of small software companies, employees and management are typically oriented towards day-to-day business, and do not have time and resources to invest in strategic issues such as quality and process improvement [43]. Empirical evidence revealed from collected data in this study confirmed the state of the practice reported in literature about small software companies.

D. Ethical issues in the research Ethical issues related to researchers, participants, other

stakeholders, and all institutions affected by the research should be considered in empirical research that involves human subjects [44][45][46]. In the beginning of all research activities with participants, they were informed about the nature and consequences of the research in which they are involved [45]. This is realized through a document called ”Research Informed Consent” which is presented to all participants, and after that signed by the researcher and each participant. By signing the document participants agreed to participate in the research. Participants anonymity is preserved in two ways in the research: (1) they had possibility to sign ”Research Informed Consent” document with the personal name or just to write ”accept participation” or ”participant”, and (2) in ”Research Informed Consent” document is stated that participants names and the names of their companies will not be used in any published document that presents research findings [46][47]. On the other hand, we get agreement from all organizations that supported this research (business incubators) to use their names.

IV. VALIDITY OF THE STUDY In this research, validity is achieved by using prescribed

and well-entrenched grounded theory procedures and techniques proposed by Charmaz [14] and Corbin and Strauss [15]. Implementation of these procedures and techniques ensures that descriptive validity (accuracy of the collected data), interpretive validity (how well the researcher reports the participants meaning) and theoretical validity (theoretical constructions that the researcher brings to, or develops) proposed by Maxwell [48] are satisfied.

It should be noted here, that this paper reports only a segment of the whole research that is conducted in order to explore common maintenance practice in VSEs. Therefore, the research finding presented in this paper is not a fully developed grounded theory about the practice, but only its segment. Grounded theory research methods ensure correctness of qualitative research does not matter if the result of the research is developed grounded theory [14][15].

External validity is important quality attribute of empirical studies [49]. This validity corresponds to generalizability of qualitative studies proposed by Maxwell [48]. Without confirming the research findings of empirical studies with humans through additional external replications, they should be treated with caution [50]. The main characteristic of qualitative research is that it is concerned with the specific context and group of

Z. Stojanov • Discovering Automation Level of Software Change Request Process from Qualitative Empirical data

– 54 –

participants. Therefore, the findings of this study can be applicable to similar contexts and similar groups of people. Practically, the design of this study can be implemented in various research contexts where the research goal is to discover common practice by investigation of practitioners’ opinions. However, due to the specific business context in the region it is hard to draw any conclusions on how the results of this study can be generalized to other regions and countries without deeper understanding of their business circumstances.

V. CONCLUDING REMARKS The main research goal of the presented study is to

explore the automation level of software change request process in very small software companies. This process is selected as an essential process during the maintenance phase of software life cycle. It is important to note that this is just a snippet of the whole study that explores more aspects of software change request process in local very small software companies.

In the study was used constructivist grounded theory approach proposed by Charmaz [14]. Qualitative data were collected through individual interviews or organized focus groups with software professionals from local very small software companies. Analyzes of collected data revealed that local small software companies do not have well established and automated process for collecting and handling software change requests that originate from their clients. The process is rather tailored to a specific client or current situation. Only four practitioners reported that they implemented partially automated process in their companies. This evidence about the state of the practice is additionally supported by the random usage of many approaches for collecting change requests from clients (by phone, by email, via web site, direct contact).

These findings are outcome of the fact that small software companies have small teams that are task oriented, and have a little time to improve existing practice in maintenance activities. Having that as a state of the practice, the following research directions are challenging: improvements of the current practice through automation of maintenance processes, improvements of communication with clients, and closer cooperation with researchers from universities. In practice, these further researches should improve communication procedures in companies and with clients, and enable development of tolls or services that automate maintenance processes.

Important research direction is related to additional empirical research of the software change process in very small companies that will include response from their clients. Process investigation and assessment by using quantitative methods will also provide insight into the practice from another point of view. In that case, the picture of the practice will be more relevant, and triangulation will increase the validity of the research findings [51].

ACKNOWLEDGMENT The author would like to thank Bojan Ljutic director of

Business incubator in Zrenjanin and Radu Ticiu executive director of Software business incubator in Timisoara for their help and support in organizing focus groups. This research is financially supported by Ministry of Science and Technological Development, Republic of Serbia;

under the project number TR32044 “The development of software tools for business process analysis and improvement”, 2011-2014.

REFERENCES [1] A. Finkelstein and J. Kramer, “Software engineering: a roadmap”,

In Proceedings of the Conference on The Future of Software Engineering (ICSE '00), ACM, New York, NY, USA, 3-22, 2000, DOI: 10.1145/336512.336519.

[2] R. L. Glass, "Guest Editor's Introduction: The State of the Practice of Software Engineering," IEEE Software, Vol. 20, No. 6, pp. 20-21, 2003, DOI: 10.1109/MS.2003.1241361.

[3] D. E. Perry, A. A. Porter, and L. G. Votta, “Empirical studies of software engineering: a roadmap”, In Proceedings of the Conference on The Future of Software Engineering (ICSE '00), ACM, New York, NY, USA, 345-355, 2000, DOI: 10.1145/336512.336586.

[4] D. J. Reifer, "Is the Software Engineering State of the Practice Getting Closer to the State of the Art?," IEEE Software, Vol. 20, No. 6, pp. 78-83, 2003, DOI: 10.1109/MS.2003.1241370.

[5] A. Rainer, “The value of empirical evidence for practitioners and researchers”, In: Basili, V., Rombach, D., Schneider, K., Kitchenham, B., Pfahl, D., Selby, R. (Eds.), Empirical Software Engineering Issues. Critical Assessment and Future Directions, Vol. 4336 of Lecture Notes in Computer Science, Springer Berlin / Heidelberg, pp. 24–24, 2007.

[6] D. I. K. Sjoberg, T. Dyba, and M. Jorgensen, “The Future of Empirical Methods in Software Engineering Research”, In 2007 Future of Software Engineering (FOSE '07), IEEE Computer Society, Washington, DC, USA, 358-378, 2007, DOI: 10.1109/FOSE.2007.30.

[7] M. W. Chiasson and E. Davidson, “Taking Industry Seriously in Information Systems Research”, MIS Quarterly, Vol. 29, No. 4, pp. 591-605, 2005.

[8] J. Segal, “The Nature of Evidence in Empirical Software Engineering”, In Proceedings of the Eleventh Annual International Workshop on Software Technology and Engineering Practice (STEP '03), IEEE Computer Society, Washington, DC, USA, 40-47, 2003.

[9] H. Robinson, J. Segal, H. Sharp, “Ethnographically-informed empirical studies of software practice”, Information and Software Technology, Vol. 49, Issue 6, Qualitative Software Engineering Research, pp. 540-551, 2007, DOI: 10.1016/j.infsof.2007.02.007.

[10] C. B. Seaman, “Qualitative methods in empirical studies of software engineering”, IEEE Transactions on Software Engineering, Vol. 25, No. 4, pp.557-572, Jul/Aug 1999, DOI: 10.1109/32.799955.

[11] Y. Dittrich, M. John, J. Singer, B. Tessem, “For the Special issue on Qualitative Software Engineering Research”, Information and Software Technology, Vol. 49, Issue 6, Qualitative Software Engineering Research, Pages 531-539, June 2007, DOI: 10.1016/j.infsof.2007.02.009.

[12] D. Silverman, “Qualitative research: meanings or practices?”, Information Systems Journal, Vol. 8, Issue 1, Pages 3 - 20, 1998, DOI: 10.1046/j.1365-2575.1998.00002.x.

[13] Lisa M. Give, The SAGE Encyclopedia of Qualitative Research Methods, Sage Publications, Inc, Thousand Oaks, 2008.

[14] K. Charmaz, Constructing grounded theory: A practical guide through qualitative analysis, 1st edition, Sage Publications Ltd, London, UK, 2006.

[15] J. Corbin and A. Strauss, Basics of qualitative research: Techniques and procedures for developing grounded theory, 3rd edition, Sage Publications, Thousand Oaks, USA, 2008.

[16] J. Mills, A. Bonner and K. Francis, “The development of constructivist grounded theory”, International Journal of Qualitative Methods, Vol. 5, No 1, pp. 25-35, 2006.

[17] A. April, A. Abran, “A Software Maintenance Maturity Model (S3M): Measurement Practices at Maturity Levels 3 and 4”, Electronic Notes in Theoretical Computer Science, Volume 233, Proceedings of the International Workshop on Software Quality and Maintainability (SQM 2008), 27 March 2009, pp. 73-87, DOI: 10.1016/j.entcs.2009.02.062.

– 55 –

6th IEEE International Symposium on Applied Computational Intelligence and Informatics • May 19–21, 2011 • Timişoara, Romania

[18] A. Abran, P. Bourque, R. Dupuis, J. W. Moore, L. L. Tripp, Guide to the Software Engineering Body of Knowledge - SWEBOK, 2004th Edition, IEEE Press, Piscataway, NJ, USA, 2004.

[19] G. Thomas and D. James, “Reinventing Grounded Theory: Some Questions about Theory, Ground and Discovery”, British Educational Research Journal, Vol. 32, No. 6, pp. 767-795, 2006.

[20] B.B. Glaser and A.L. Strauss, The Discovery of Grounded Theory, Aldine Publishing Company, Chicago, USA, 1967.

[21] J. Corbin and A. Strauss, “Grounded Theory Research: Procedures, Canons, and Evaluative Criteria”, Qualitative Sociology, Vol. 13, No.1, pp. 3-21, 1990.

[22] W. J. Orlikowski, “CASE Tools as Organizational Change: Investigating Incremental and Radical Changes in Systems Development”, MIS Quarterly, Vol. 17, No. 3, pp. 309-340, 1993.

[23] J. Esteves, I. Ramos and J. Carvalho, “Use of grounded theory in information systems area: an exploratory analysis”, In Proceedings of the European Conference on Research Methods, pp. 129-136, 2002.

[24] R. Matavire and I. Brown, “Investigating the use of "Grounded Theory" in information systems research”, In Proceedings of the 2008 annual research conference of the South African Institute of Computer Scientists and Information Technologists on IT research in developing countries: riding the wave of technology (SAICSIT '08), ACM, NY, USA, 139-147. DOI=10.1145/1456659.1456676.

[25] C. Urquhart, H. Lehmann, M. D. Myers, “Putting the ‘theory’ back into grounded theory: guidelines for grounded theory studies in information systems”, Information Systems Journal, Vol. 20, Issue 4, pages 357–381, July 2010, DOI: 10.1111/j.1365-2575.2009.00328.x.

[26] J. Carver, “The Impact of Background and Experience on Software Inspections”, Empirical Software Engineering, Volume 9, Number 3, pp. 259-262, 2004, DOI: 10.1023/B:EMSE.0000027786.04555.97.

[27] G. Arcs, R. Razali, “Cognitive dimensions and grounded theory in learning software modeling”, Procedia - Social and Behavioral Sciences, Volume 1, Issue 1, World Conference on Educational Sciences, Nicosia, North Cyprus, 4-7 February 2009 - New Trends and Issues in Educational Sciences, pp. 1884-1888, 2009, DOI: 10.1016/j.sbspro.2009.01.331.

[28] G. Coleman, R. O'Connor, “Using grounded theory to understand software process improvement: A study of Irish software product companies”, Information and Software Technology, Volume 49, Issue 6, Qualitative Software Engineering Research, pp. 654-667, June 2007, DOI: 10.1016/j.infsof.2007.02.011.

[29] G. Coleman, R. O'Connor, “Investigating software process in practice: A grounded theory perspective”, Journal of Systems and Software, Volume 81, Issue 5, Software Process and Product Measurement, pp. 772-784, May 2008, DOI: 10.1016/j.jss.2007.07.027.

[30] L. Scott, L. Carvalho, R. Jeffery, J. D'Ambra, U. Becker-Kornstaedt, “Understanding the use of an electronic process guide”, Information and Software Technology, Volume 44, Issue 10, pp. 601-616, 2002, DOI: 10.1016/S0950-5849(02)00080-0.

[31] L. Carvalho, L. Scott, R. Jeffery, “An exploratory study into the use of qualitative research methods in descriptive process modeling”, Information and Software Technology, Volume 47, Issue 2, pp. 113-127, 2005, DOI: 10.1016/j.infsof.2004.06.005.

[32] S. Adolph, W. Hall and P. Kruchten, “Using grounded theory to study the experience of software development”, Empirical Software Engineering, n/a, doi: 10.1007/s10664-010-9152-6.

[33] Z. Hussain, W. Slany and A. Holzinger, “Investigating Agile User-Centered Design in Practice: A Grounded Theory Perspective”, HCI and Usability for e-Inclusion, Lecture Notes in Computer Science, Volume 5889/2009, 279-289, 2009, DOI: 10.1007/978-3-642-10308-7_19.

[34] J. Sillito, K. De Voider, B. Fisher, G. Murphy, "Managing software change tasks: an exploratory study," isese, 2005 International Symposium on Empirical Software Engineering, 2005.

[35] I. Richardson, C. G. von Wangenheim, "Guest Editors' Introduction: Why are Small Software Organizations Different?," IEEE Software, Vol. 24, No. 1, pp. 18-22, 2007, DOI: 10.1109/MS.2007.12.

[36] C.Y. Laporte, A. Renault, S. Alexandre, T. Uthayanaka, “The Application of ISO/IEC JTC 1/SC7 Software Engineering Standards in Very Small Enterprises”, ISO Focus, International Organisation for Standardisation, pp. 36-38, September 2006.

[37] N. Habra, S. Alexandre, J-M. Desharnais, C. Y. Laporte, A. Renault, “Initiating software process improvement in very small enterprises: Experience with a light assessment tool”, Information and Software Technology, Vol. 50, Issues 7-8, pp. 763-771, June 2008, DOI: 10.1016/j.infsof.2007.08.004.

[38] V. Ribaud, P. Saliou, R. V. O’Connor and C. Y. Laporte, “Software Engineering Support Activities for Very Small Entities”, Systems, Software and Services Process Improvement, Communications in Computer and Information Science, 2010, Volume 99, pp. 165-176, Springer-Verlag Berlin Heidelberg, 2010, DOI: 10.1007/978-3-642-15666-3_1.

[39] K. Bennett, V. Rajlich, “Software maintenance and evolution: a roadmap”, In Proceedings of the Conference on the Future of Software Engineering. ICSE '00, pp. 73-87, June 2000, Limerick, Ireland, DOI: http://doi.acm.org/10.1145/336512.336534.

[40] C. Y. Laporte, “Process Improvement and the Management of Change”, Proceedings - 4th IEEE Computer Society Workshop on Software Engineering Technology Transfer, pp. 213-216, 1994.

[41] D. W. Stewart, P. N. Shamdasani and D. W. Rook, Focus Groups: Theory and Practice Second Edition, SAGE Publications, London, UK, 2007.

[42] J. A. Maxwell, “Using Numbers in Qualitative Research”, Qualitative Inquiry, Vol. 16, No 6, pp. 475-482, 2010, doi: 10.1177/1077800410364740.

[43] E. Kamsties, K. Hormann and M. Schlich, “Requirements Engineering in Small and Medium Enterprises: State-of-the-Practice, Problems, Solutions, and Technology Transfer”, 1st Conference on European Industrial Requirements Engineering (CEIRE'98), pp. 40-50, October 1998, London, UK.

[44] J. E. Sieber, “Protecting Research Subjects, Employees and Researchers: Implications for Software Engineering”, Empirical Software Engineering, Vol. 6, Issue 4, pp. 329-341, 2001, DOI: 10.1023/A:1011978700481.

[45] C. G. Christians, “Ethics and politics in qualitative research”. In The Sage Handbook of Qualitative Research, N. K. Denzin and Y. S. Lincoln, Eds., Sage Publications, Chapter 6, 2005, pp. 139–164.

[46] N. G. Vinson, J. Singer, “A Practical Guide to Ethical Research Involving Humans”, In Guide to Advanced Empirical Software Engineering, F. Shull, J. Singer and D. I. K. Sjøberg, Eds., Springer London, 2008, Section II, 229-256, DOI: 10.1007/978-1-84800-044-5_9.

[47] K. M. Guenther, “The politics of names: rethinking the methodological and ethical significance of naming people, organizations, and places”, Qualitative Research, Vol. 9, No 4, pp. 411-421, 2009, doi:10.1177/1468794109337872.

[48] J. A. Maxwell, “Understanding and Validity in Qualitative Research”, Harvard Educational Review, Vol. 62, No. 3, pp. 279-300, 1992.

[49] B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam and J. Rosenberg, “Preliminary Guidelines for Empirical Research in Software Engineering”, IEEE Transactions on Software Engineering, Vol. 28, No. 8, pp. 721-734, 2002, DOI: 10.1109/TSE.2002.1027796.

[50] A. Brooks, M. Roper, M. Wood, J. Daly, and J. Miller, “Replication's role in software engineering”, In Guide to Advanced Empirical Software Engineering, F. Shull, J. Singer and D. I. K. Sjøberg, Eds., Springer London, Section III, 365-379, DOI: 10.1007/978-1-84800-044-5_14.

[51] L. Bratthall and M. Jørgensen, “Can you Trust a Single Data Source Exploratory Software Engineering Case Study?”, Empirical Software Engineering, Vol. 7, No 1, pp. 9-26, 2002, DOI: 10.1023/A:1014866909191.

– 56 –

Z. Stojanov • Discovering Automation Level of Software Change Request Process from Qualitative Empirical data