Changing IT service providers is never a simple undertaking. It is even more challenging when the organization making the change is responsible for processing meter reads and supporting the billing of more than four million customers on time-of-use rates. Such complexity necessitated a framework to help guide the search and contract process so the organization, in this case, turned to COBIT 5.
The Independent Electricity System Operator (IESO) balances the supply of and demand for electricity in Ontario (Canada) and then directs its flow across the province’s transmission lines. Working at the heart of Ontario's power system, the IESO connects all participants—generators that produce electricity, transmitters that send it across the province, retailers that buy and sell it, industries and businesses that use it in large quantities, and local distribution companies that deliver it to people's homes.
The IESO is also the Smart Metering Entity responsible for Ontario’s smart Meter Data Management and Repository (MDM/R). The MDM/R is the world’s first and largest smart meter management and processing shared service, supporting the meter-to-billing operations of more than 70 utilities. This critical, around-the-clock operation handles more than 100 million transactions per day and supports the billing of more than four million customers on time-of-use rates.
The IESO used COBIT 5 for the procurement of IT services, helping to accelerate the procurement process and improve the contract and how it is managed.
The Challenge of Changing Service Providers
As the contract with the existing service provider for operating the MDM/R system and infrastructure was nearing its end, the IESO undertook an open competitive procurement process to select a vendor to operate this critical, complex service under very demanding service levels. The operation of this service involves:
- Processing more than 100 million interval meter reads in a few hours each day
- Processing between 200,000 to 350,000 billing requests per day
- Processing more than 40,000 requests for consumption information per hour
- Delivering more than 2,300 reports per day
- Supporting five different Advanced Metering Infrastructure technologies and more than six different customer information billing systems
- Supporting 72 utilities with extensive stakeholder and governance requirements
- External audits of MDM/R functions and processes
To ensure that the existing MDM/R system transitioned to the successful respondent prior to the expiration of the current contract, the IESO needed to complete the competitive procurement and contracting process within a very tight time frame.
As part of the procurement process, it was necessary to specify the IESO’s requirements related to activities, roles, responsibilities and deliverables for operating the MDM/R to potential service providers.
Improving the RFP Process
The IESO looked at possible frameworks that could help it define requirements for operating the MDM/R for inclusion in a request for proposal (RFP). The framework had to include comprehensive coverage of all processes for governing and managing an integrated IT service. To satisfy this requirement, the IESO selected COBIT 5 for inclusion in the RFP.
The COBIT framework was used to specify the roles, responsibilities, deliverables and expected capability levels for each IT process. Most important, it specified the terms for both the IESO and vendors to clarify roles and avoid misinterpretation, using the COBIT 5 Responsible, Accountable, Consulted and Informed (RACI) matrix (see the example in figure 1).
Figure 1—Sample RACI From the IESO ContractLegend | |||||||
| Key Practice | Practice to Be Performed | Activities | Outputs/ Deliverables |
Deliver to IESO? | Vendor RACI | IESO RACI | Clarification of RACI (where applicable) |
| BAI06.01 Evaluate, prioritize and authorize change requests | Evaluate all requests for change to determine the impact on business processes and IT services, and to assess whether change will adversely affect the operational environment and introduce unacceptable risk. Ensure that changes are logged, prioritized, categorized, assessed, authorized, planned and scheduled. | 1. Use formal change requests to enable business process owners and IT to request changes to business process, infrastructure, systems or applications. Make sure that all such changes arise only through the change request management process. 2. Categorize all requested changes (e.g., business process, infrastructure, operating systems, networks, application systems, purchased/packaged application software) and relate affected configuration items. 3. Prioritize all requested changes based on the business and technical requirements, resources required, and the legal, regulatory and contractual reasons for the requested change. 4. Plan and evaluate all requests in a structured fashion. Include an impact analysis on business process, infrastructure, systems and applications, business continuity plans (BCPs) and service providers to ensure that all affected components have been identified. Assess the likelihood of adversely affecting the operational environment and the risk of implementing the change. Consider security, legal, contractual and compliance implications of the requested change. Consider also inter-dependencies among changes. Involve business process owners in the assessment process, as appropriate. 5. Formally approve each change by business process owners, service managers and IT technical stakeholders, as appropriate. Changes that are low-risk and relatively frequent should be pre-approved as standard changes. 6. Plan and schedule all approved changes. 7. Consider the impact of contracted service providers (e.g., of outsourced business processing, infrastructure, application development and shared services) on the change management process, including integration of organizational change management processes with change management processes of service providers and the impact on contractual terms and SLAs. |
1. Impact assessments 2. Approved requests for change 3. Change plan and schedule |
Y Y Y |
R | A | The vendor and IESO agree on a change management procedure that complies with the MDM/R Terms of Service and MDM/R Change Management Manual. The vendor will provide a summary of Informed changes (Business-As-Usual [BAU]) into the IESO service-desk tool. Business-as-usual items are vendor work going on behind the scenes to underlying infrastructure that are reported weekly as a single line item to the IESO with available details. |
| BAI06.02 Manage emergency changes | Carefully manage emergency changes to minimize further incidents and make sure the change is controlled and takes place securely. Verify that emergency changes are appropriately assessed and authorized after the change. | 1. Ensure that a documented procedure exists to declare, assess, give preliminary approval, authorize after a change and record an emergency change. 2. Verify that all emergency access arrangements for changes are appropriately authorized, documented and revoked after the change has been applied. 3. Monitor all emergency changes, and conduct postimplementation reviews involving all concerned parties. The review should consider and initiate corrective actions based on root causes such as problems with business process, application system development and maintenance, development and test environments, documentation and manuals, and data integrity. 4. Define what constitutes an emergency change. |
1. Post-implementation review of emergency changes | Y | R | A | The vendor and the IESO agree on a change management procedure that complies with the MDM/R Terms of Service, MDM/R Change Management Manual and requirements outlined in this document. |
| BAI06.03 Track and report change status | Maintain a tracking and reporting system to document rejected changes, communicate the status of approved and in-process changes, and complete changes. Make certain that approved changes are implemented as planned. | 1. Categorize change requests in the tracking process (e.g., rejected; approved, but not yet initiated; approved and in process; closed). 2. Implement change status reports with performance metrics to enable management review and monitoring of both the detailed status of changes and the overall state (e.g., aged analysis of change requests). Ensure that status reports form an audit trail so changes can subsequently be tracked from inception to eventual disposition. 3. Monitor open changes to ensure that all approved changes are closed in a timely fashion, depending on priority. 4. Maintain a tracking and reporting system for all change requests. |
1. Change request status reports (may be provided through Service Now tool) | Y | R | RA | The IESO will make available the Service Desk tool for tracking changes to the MDM/R for use by the vendor and the IESO. |
| BAI06.04 Close and document the changes | Whenever changes are implemented, update the solution and user documentation and the procedures affected by the change accordingly. | 1. Include changes to documentation (e.g., business and IT operational procedures, business continuity and disaster recovery documentation, configuration information, application documentation, help screens, and training materials) within the change management procedure as an integral part of the change. 2. Define an appropriate retention period for change documentation and pre- and postchange system and user documentation. 3. Subject documentation to the same level of review as the actual change. |
1. Change documentation | Y | R | RA | |
The IESO evaluated vendor responses to the RFP, including their demonstrated ability to meet the stated process requirements for governing and managing the operation of the MDM/R.
In finalizing the contract with the chosen RFP respondent, the vendor’s scope of service, obligations, responsibilities and deliverables for each process in the COBIT framework were clarified and embedded in the contract. Although this exercise involved a significant amount of effort from both the vendors’ and IESO’s teams, it helped ensure that both parties used a common, industry-recognized language for IT processes and practices.
Defining the New Contract Terms
This example illustrates how the objectives, activities, deliverables and responsibilities were defined for the BAI06 Manage changes process for the RFP and the IESO contract.
The RACI chart was used to develop a suggested assignment of level of responsibility for process practices to different roles and structures:
- R(esponsible)—Who is getting the task done?
- A(ccountable)—Who accounts for the success of the task?
- C(onsulted)—Who is providing input?
- I(nformed)—Who is receiving information?
BAI06 Process Description
All changes are managed in a controlled manner, including standard changes and emergency maintenance relating to business processes, applications and infrastructure. This includes change standards and procedures, impact assessment, prioritization and authorization, emergency changes, tracking, reporting, closure, and documentation. Changes can be identified at any time during the project or operational phase.
BAI06 Process Purpose
Fast and reliable delivery of change to the business and mitigation of the risk of negatively impacting the stability or integrity of the changed environment are enabled.
BAI06 Short-term Desired Capability Level
By the end of the transition phase, the business will have reached a level 3, established process (two attributes), capability level.1 The managed process is implemented using a defined process that is capable of achieving its process outcomes.
BAI06 Long-term Desired Capability Level
By three years following the transition phase, the business will have reached level 4, predictable process (two attributes), capability level. The previously described established process now operates within defined limits to achieve its process outcomes.
Conclusion
The COBIT framework helped the IESO to significantly improve clarity for defining process requirements and vendor obligations as it was a recognized framework, reducing the risk of misunderstanding or misinterpretation. The entire process, from having the RFP issued, responses evaluated and contract signed with the successful respondent, was completed in five months.
The framework also allowed the IESO to develop a transition strategy for the maturing of processes over the term of the contract. COBIT 5 continues to be used by the IESO and the vendor in the governance and oversight of the MDM/R.
COBIT has been a very useful tool in facilitating the agreement between the IESO and its vendor on a governance model and responsibilities, identifying and managing risk, and establishing targets for continuous improvement.
Przemek Tomczak, CISA, CA, CPA
Is the director of smart metering at Ontario’s Independent Electricity System Operator (IESO), overseeing the operations of the world’s first and largest smart meter management and processing shared service, supporting the meter-to-billing operations of Ontario’s local distribution companies. Prior to his current role, he led the IESO’s internal audit and risk management functions. Tomczak has extensive IT and business leadership experience, delivering program and transformation, consulting, outsourcing, and risk management initiatives. He has also held senior positions with Protiviti Consulting, Capgemini, Accenture, EMC and PricewaterhouseCoopers.
Endnotes
1 Capability level definitions are based on those used in the COBIT Assessment Programme Process Assessment Model (PAM), based on ISO/IEC 15504 part 2, which defines the measurement framework attributes at these levels.