The global economy is becoming more challenging due to greater international competition, the pace of digital transformation and increased customer experience expectations. The importance of realizing benefits from IT-enabled investments is pressing; enterprises can no longer afford to continue wasting costly investments. Business environments are complex and changing rapidly, which makes it more difficult to realize benefits.1 Enterprises are challenged to adapt and respond to changing economic environments, new technologies, market opportunities, high-demand clients and government regulations. They need to be innovative, to deliver maximum business value and, at the same time, to do so in an effective way at the lowest possible cost. Therefore, enterprises must depend on their information systems, and big investments often are made in this domain to support these goals. Still, many enterprises are struggling to fully realize business benefits of their IT-enabled investments.
Benefits Management: What Is in a Name?
A benefit can be described as an advantage that occurs for a particular stakeholder or group of stakeholders.2 This can be extended to denote “a measurable advantage owned by a group of stakeholders incurred by changing the current state through project management mechanisms.”3 Benefits management, or benefits realization management, is a framework that formerly was used with the aim of increasing the success of IT projects.4, 5, 6 Benefits management is the set of processes that embeds the requirements of a business strategy into business as usual to ensure that projects, programs and portfolios create sustainable business value.7
Recent research conducted at Antwerp Management School in Belgium investigated which management practices could support an enterprise in benefit realization from IT-enabled projects. To obtain a robust answer to this question, an exploratory study of the literature was conducted to identify a set of management practices that could be helpful in this domain.
THE PRACTICES IDENTIFIED IN THE STUDY, COMBINED WITH INPUT FROM COBIT 5, CREATE A SOUND THEORETICAL OVERVIEW OF PRACTICES TO ENSURE EFFECTIVE BENEFITS REALIZATION IN IT-ENABLED PROJECTS.
In this context, management practices are defined as the working methods and innovations that managers use to improve the effectiveness of work systems and all instruments used to implement the concepts and ideas that ultimately support organizational processes.8 These practices will be viewed through the lens of enterprise governance of IT (EGIT):
EGIT addresses the definition and implementation of processes, structures and relational mechanism to enable both business and IT people to execute their responsibilities in support of business/IT alignment and the creation of value from IT-enabled business investments.9
Structures, such as organizational units, or certain roles or government bodies can support alignment in decision-making between business and IT executives. This decision-making process is supported by formalizing and institutionalizing procedures and monitoring them to ensure that daily behavior is consistent with existing policies. Relational mechanisms include collaboration and communication between business and IT managers on job rotations, colocation and informal meetings.
To enrich the study, the COBIT® 5 framework was investigated to add useful benefits management practices from a practitioner’s point of view.
According to ISACA®, “COBIT 5 is the only business framework for the governance and management of enterprise IT.” It “provides globally accepted principles, practices, analytical tools and models to help increase the trust in, and value from, information systems.”10
The practices identified in the study, combined with input from COBIT 5, create a sound theoretical overview of practices to ensure effective benefits realization in IT-enabled projects. Each of the identified benefit management practices is categorized as a structure, process or relation mechanism. Based on the relevant literature, all practices have clear definitions and are clustered in a broader theme. Figure 2 provides an overview of identified management practices.
During the second phase of the research, a group of business and IT executives involved in IT projects was sent a survey on the overview of practices. In total, 26 experts—mainly working as project managers, portfolio and program managers, business directors, consultants, and enterprise architects—were addressed, and 17 of them participated in the survey. They were asked to score the practices—categorized as structures, processes and relation mechanisms, according to the lens of EGIT—on two criteria: effectiveness (how much the practice contributes to the actual realization of the benefits) and ease of implementation (how easy it is to implement the practice).
To score the practices, a Likert scale was used. A Likert scale is:
...A set of statements (items) offered for a real or hypothetical situation under study. Participants are asked to show their level of agreement (from strongly disagree to strongly agree) with the given statement (items) on a metric scale.11
In this survey, participants could vary their answers from “1 – Very Low” to “5 – Very High.” The results were analyzed to determine which practices were perceived as the most effective, which were the easiest to implement and which scored the highest when both criteria were combined.
Practitioners Promote a Sound Benefits Management Approach
Based on the scores, all practices could be ranked on the criteria of effectiveness. In general, the experts scored all the practices rather high. Twelve out of the 30 practices had an average score greater than 0.80, which corresponds to a score of “4 – High” on the Likert scale. Another 17 management practices had an average score between 0.6 and 0.8, corresponding respectively with “3 – Neutral” and “4 – High” (figure 1).
The practice with the highest perceived effectiveness, an average score of 0.96, was S03, senior management support, in the program and project governance cluster. This means the respondents evaluated the sufficient support of senior executive management by allocation of resources and granting of necessary authorities for decision-making in projects as the practice that contributed most to the realization of benefits. S04, early assignment of managers and governance bodies, was perceived as the practice with the second-highest perceived effectiveness. Next was S02, with an average score of 0.87, which indicates that active stakeholder participation, cross-departmental collaboration, and stakeholder involvement in progress and project reviews also were considered crucial to benefit management. S05C, providing program and IT performance reports (functional and technical) with accurate data, also ranked high. It is noteworthy that the practices with the second-, third- and fourth-highest average scores all belong to the program and project governance cluster. Another finding is that the five management practices that the experts perceived as the greatest contributors are all structures. This suggests that the experts trusted most in structures for the actual realization of benefits. The top process and relational mechanisms share the seventh-highest average score of 0.84. In the second tier of the top 12, more processes in the benefits identification and benefits planning clusters appear.
As for effectiveness, the experts were asked to give a score for ease of implementation on a Likert scale from “1 – Very Low,” meaning very difficult to implement, to “5 – Very High,” meaning very easy to implement.
An initial conclusion is that the average scores for ease of implementation are lower compared to the average scores for effectiveness. The highest average score for ease of implementation is 0.76. Sixteen practices have an average score between 0.6 and 0.8, corresponding respectively with a “3 – Neutral” and “4 – High” on the Likert scale. For this criterion, the other 14 practices score between 0.4 and 0.6 on average, corresponding with “2 – Low” and “3 – Neutral” on the Likert scale. This implies that the experts believed some practices were rather difficult to implement (figure 3).
Perceived as easiest to implement was an ex aequo of two practices, S04 and S06. S04 is the early assignment of managers and governance bodies. S06 is a formal lessons learned meeting convened by a sponsor or project manager. S08, the transition to the organization in a formal handover, was also considered easy to implement. Next in the rankings is P07, business case development, which requires that the business case, including all outputs and benefits, be created and approved at the beginning of the project. That is not so hard to implement, according to the respondents. P012A, aligning project outcomes with business routines—such as altering the way people conduct their work or changing business processes through training, support, monitoring or outcome evaluation—was also considered relatively easy to implement.
ENTERPRISES THAT ARE LESS MATURE MAY FIND THAT IT IS NOT FEASIBLE TO IMPLEMENT ALL PRACTICES.
An overview of how the practices scored on both criteria—effectiveness and ease of implementation—is also provided. The score used for the perceived effectiveness and ease of implementation takes both individual scores into account equally. The argument for that is that not all enterprises have a high level of maturity regarding EGIT. Enterprises that are less mature may find that it is not feasible to implement all practices. They may have to make trade-offs between the efforts they would have to expend and the benefits they could gain. In some ways, this represents a cost/benefit analysis: The cost in this context refers to the ease of implementation and the benefit to the effectiveness of the practices.
The overview of the scores presented in figure 4 shows that most of the practices (26) have average scores between 0.6 and 0.8, corresponding with “3– Neutral” and “4 – High” on both criteria. S04, early assignment of managers and governance bodies, has an average score of 0.84 on both criteria, which means it was perceived as both highly effective and easy to implement. S08, transition to organization-formal handover, and S03, senior management support, also have average high scores of 0.79 and 0.78 respectively.
It should be noted that the highest scoring practices on both criteria are structures. Also, out of the 11 practices with the highest average score on both effectiveness and ease of implementation, eight also appear individually in the top tiers of effectiveness and ease of implementation. This shows that most of the practices that were perceived as the most effective by the experts also were perceived as relatively easy to implement. Figure 5 provides an overview.
A Case Study: Argenta
Based on the literature and survey of experts and using the lens of EGIT, 12 benefit management practices were selected for assessment in a case study. The case enterprise is Argenta, a midsize bank in Belgium that has made a lot of progress in portfolio management in recent years and also has taken the first steps in benefit management. In the initial phase, in-depth interviews were organized with the portfolio manager to cover in detail how the practices were applied within Argenta. Definitions found in the literature were used to facilitate a common understanding. These practices were then plotted on a maturity scale12—by the portfolio manager for most practices and by an enterprise architect for one. This step is important because if the maturity of certain practices within Argenta seem to be low or nonexistent, the evaluations on certain criteria, such as effectiveness, would be less accurate and meaningful. Figure 6 shows the ISACA maturity scale used in this exercise.
Source: ISACA®, COBIT® 2019, USA, 2018, www.isaca.org/resources/cobit
The portfolio manager assessed the maturity of 12 of the selected management practices based on the defined maturity model. The enterprise architect evaluated the practice related to the mapping of benefits and objectives to business changes and IT capabilities. Figure 7 shows the overall results.
DEFINITIONS FOUND IN THE LITERATURE WERE USED TO FACILITATE A COMMON UNDERSTANDING.
It can be concluded that Argenta has achieved a rather high level of maturity for most of the selected practices. The management practices related to governance topics are especially mature, and procedures in this area seem to be working effectively. They are measured and monitored by portfolio managers.
Practices related to the transition to the organization and the business integration of project outcomes are evaluated as defined on the maturity scale. Development of business cases also has reached the third maturity level of defined process. The defined level was assigned for these practices because although all processes are standardized and documented, some deviations might occur and remain undetected.
The other assessed practices were perceived to be repeatable but intuitive because they were not yet standardized, and the method of execution depended on the person involved. The single relational mechanism observed was assigned a lower maturity level because only a few cases could be listed as an example for this practice. The overall level of maturity, which clearly indicates that Argenta has some experience in implementing these practices, makes it relevant to analyze perceptions of effectiveness and ease of implementation.
In a second phase of the case study, the selected practices were assessed for effectiveness and ease of implementation by a panel consisting of the portfolio manager, the demand manager, the business manager and an enterprise architect. This was done using the same Likert scales, leading to scores from 1 to 5.13
Business case development was perceived as the most effective practice with a score of 0.8, followed by project governance (0.75). Early assignment of project managers and governance bodies, as well as senior managers support have the next highest scores, which can be explained by the fact that effectiveness is often related to rules enforcement and oversight by the same governance bodies, such as the steering committees (representatives from senior management level, sponsor, portfolio manager, and project or program manager), the core teams (representatives from middle management level for first escalations), the single point of contact appointed within each department, and the establishment of a phased life cycle approach for projects and programs.
Mapping the objectives and benefits to the business changes and IT capabilities and the development of long-term benefits in broader perspective by co-creation of stakeholders were relatively less effective. Business case development was perceived as the easiest to implement, compared to the other practices, but with an average score of only 0.5. Installing project governance scored equally based on this criterion. Other practices were evaluated as more difficult to implement, with relatively low scores between 0.2 and 0.4.
Next the perceived effectiveness and ease of implementation between the case study and the results of the expert survey are compared. Results are shown in figures 8 and 9. In general, it can be noted that all the assessed benefit management practices in the case study received lower scores on effectiveness. The difference in scores is rather large for most practices. P04, link with business changes and IT capabilities, was perceived as very effective by the experts, with a score of 0.82—but much less effective by Argenta, with a score of 0.25. The maturity of this process was assessed within Argenta as level 2—repeatable but intuitive, which indicates the enterprise has some experience in this area, making this finding relevant.
The scores for RM01, development of long-term benefits in broader perspective by co-creation of stakeholders, provide another example of high variance between the survey and the case study. Perceived effectiveness is 0.84 for the survey, while it is only 0.35 for Argenta, suggesting this practice was found relatively ineffective in reality. On the other hand, this relational mechanism existed only on an ad hoc basis within Argenta. The results for S05C are similar, with an average score of 0.86 from the experts, compared with 0.45 in the case study.
For some other practices, such as P07, business case development, the results for perceived effectiveness are comparable between the survey (0.82) and the case study (0.8). The maturity of this process within Argenta was evaluated as level 3—defined, which makes the result of this finding relevant. The results of S05B, project governance, are also in line, with a score of 0.82 compared with 0.75, and a high maturity assessment of this practice within Argenta at level 4—managed and measurable. For S04, early assignment of managers and governance bodies; S03, senior management support; and S02, active stakeholder involvement—all practices evaluated as very mature in Argenta—the variance in perceived effectiveness is also significant. Argenta perceived the effectiveness of these practices as closer to neutral than high or very high.
In general, although the assessed practices were perceived as very effective by the group of experts, the results of the case study seem less optimistic. Except for P07, business case development, and RM01, development of long-term benefits in a broader perspective by co-creation of stakeholders, the scores are significantly lower in the case study. It should be noted that these results are based on only one case study, implying that it is a first check of practice. Figure 9 shows the overview of results for ease of implementation for the survey compared with the case study.
IN GENERAL, ALTHOUGH THE ASSESSED PRACTICES WERE PERCEIVED AS VERY EFFECTIVE BY THE GROUP OF EXPERTS, THE RESULTS OF THE CASE STUDY SEEM LESS OPTIMISTIC.
In general, similar conclusions can be drawn for the ease of implementation comparison. Overall, the perception in the case study is that the practices were more difficult to implement. The biggest noticeable difference in scores is for S06, lessons learned deliberation. An explanation for this difference is that the participants in Argenta interpreted this practice not only as formally conducting lessons learned meetings with structured templates, but also as using this input in new initiatives, while that was not literally stated in the survey. For this reason, Argenta perceived the practice as very difficult to implement because it would involve project managers structurally using lessons learned from the past.
There is also a high discrepancy between the scores for P04, link with business changes and IT capabilities, which are 0.25 in the case study and 0.67 assigned by the experts. The highest scores given by Argenta are for P07, business case development, which is 0.5 (compared with 0.71 by the experts), and for S05B, project governance, which is 0.5 (compared with 0.66 by the experts). For all the practices, the scores attributed by the participants in the case study are significantly lower than the ones provided in the expert survey.
Conclusion
The focus of this research was to determine which management practices can support an enterprise in benefits realization in IT-enabled projects. Thirty relevant management practices in this area were found in the literature. The survey found that eight of those practices were perceived as the most effective and the easiest to implement. It seems an obvious choice for enterprises to invest first in those practices. However, the case study results are less optimistic. The assessed practices were perceived as less effective and a lot more difficult to implement.
Based on the results of both the expert survey and the case study, figure 10 provides a ranking of management practices based on the average scores.
According to the experts in the survey and within Argenta, developing business cases, having solid project governance, and making sure that project managers and governance bodies are assigned at an early stage of the project are the practices that contribute most to the realization of benefits. The other practices were also perceived as contributing to benefits realization. The choices enterprises will make depend on their situations and on their existing levels of maturity in these practices.
The research question was answered by providing a list of benefit management practices that can contribute to actively realizing benefits from IT-enabled investments. Further research should be done to investigate the success in benefit realization in enterprises that actively develop these practices.
THE ASSESSED PRACTICES WERE PERCEIVED AS LESS EFFECTIVE AND A LOT MORE DIFFICULT TO IMPLEMENT.
Endnotes
1 Bradley, G.; Fundamentals of Benefit Realization, The Stationery Office, UK, 2010
2 Ward, W.; E. Daniel; Benefits Management: Delivering Value From IS and IT Investments, Wiley, USA, 2006
3 Badewi, A.; “The Impact of Project Management (PM) and Benefits Management (BM) Practices on Project Success: Towards Developing a Project Benefits Governance Framework,” International Journal of Project Management, May 2016
4 Ashurst, C.; N. Doherty; “Towards the Formulation of a ‘Best Practice’ Framework for Benefits Realisation in IT Projects,” Electronic Journal of Information Systems Evaluation, 1 January 2003
5 Breese, R.; “Benefits Realisation Management: Panacea or False Dawn?” International Journal of Project Management, April 2012
6 Serra, C.; M. Kunc; “Benefits Realisation Management and Its Influence on Project Success and on the Execution of Business Strategies,” International Journal of Project Management, January 2015
7 Ibid.
8 IGI Global, “What Is Management Practices,” www.igi-global.com/dictionary/multi-agent-simulation-management-practices/17783
9 De Haes, S.; W. Van Grembergen; Enterprise Governance of Information Technology: Achieving Alignment and Value, Featuring COBIT 5, Springer International Publishing, Switzerland, 2015
10 ISACA®, COBIT®5: A Business Framework for the Governance and Management of Enterprise IT, USA, 2012, https://www.isaca.org/bookstore/cobit-5/wcb5
11 Joshi, A.; S. Kale; S. Chandel; D. K. Pal;“ Likert Scale: Explored and Explained,” British Journal of Applied Science and Technology, January 2015
12 ISACA, COBIT® 2019, USA, 2019, www.isaca.org/resources/cobit
13 Op cit Joshi et al.
Sam Opdenakker
Is a freelance program manager in Belgium. After having worked in consulting, he has been active in the banking and insurance industry for the last 13 years. Opdenakker is a bridge builder between business and IT and has taken up several roles including analyst, process manager, project manager and program manager. His specialization is to harmonize these two worlds to realize maximum value creation out of his projects.
Kim Maes, Ph.D.
Serves as deputy director of healthcare in communication and services, innovation, and continuous improvement at Vanbreda Risk and Benefits. He is a guest lecturer at Antwerp Management School (Belgium) and an affiliated research fellow with the department for information systems management at the University of Antwerp (Belgium). His expertise is oriented toward digital strategy and innovation, project and portfolio management, the use of business cases, and benefits management. His research has been published in books, conference proceedings and peer-reviewed journals such as the Information Systems Management, Electronic Journal Information Systems Evaluation, and International Journal of IT/Business Alignment and Governance.
Steven De Haes, Ph.D.
Is dean and professor of digital strategy and governance at Antwerp Management School and the University of Antwerp (Belgium). He is actively engaged in teaching and applied research in the domains of digital strategies, IT governance and management, IT strategy and alignment, IT value and performance management, IT assurance and audit, and information risk and security. His research has been published in international peer-reviewed journals and leading conferences. He has co-authored and edited several books, including Enterprise Governance of IT: Achieving Strategic Alignment and Value. He is also a speaker and facilitator in academic and professional conferences, and coaches organizations in their digital strategies, IT governance, alignment, value and audit/assurance efforts. He has been involved in the development of COBIT® as a researcher and co-developer. He can be reached at steven.dehaes@uantwerpen.be.