Maximize Value, Adopt a Flexible Approach to Auditing Major Projects

For most organizations, project execution is the primary means of achieving business objectives—such as cost reduction, reorganization, regulatory compliance, expansion of product lines or entrance into new markets—and implementing new or significantly changed systems and processes. Inherent in each of these business objectives is the achievement of a particular value statement or vision—balancing opportunity with the organization’s risk capacity. However, the projects themselves are often costly and can be extraordinarily complex, and successful outcomes are far from guaranteed. Poorly executed projects can present significant risk to the organization, including operational impacts, loss of invested capital, failure to realize planned value, loss of reputation or regulatory findings that result in a negative consequence.

More than one-third of all projects fail to meet their original goals and business intent.¹ One study of 5,400 large-scale IT projects (initial budget > US $15 million) found:²

• On average, large IT projects run 45 percent over budget and 7 percent past deadline, while delivering less than half of the predicted value.
• 17 percent of large IT projects go so badly that they can threaten the very existence of the company.

The risk of cost and schedule overruns has been found to be highest for software projects and the overruns in large-scale projects have been found to be on par with those observed in large/complex construction projects.

Because of the significant risk major projects can present, internal audit (IA) is often engaged to provide them oversight and monitoring. However, developing a comprehensive audit plan for a large-scale, complex, multiyear initiative can be a major challenge. As a result, it is often tempting for IA to delay engagement early in the project life cycle, when uncertainty is highest. However, deferring engagement to later phases of the project life cycle (e.g., execution and delivery) limits IA’s opportunity to influence project outcomes because stakeholder influence and opportunity for risk management are highest at the beginning of a project (e.g., during conception and planning phases) (figure 1).

Efficient and effective auditing of major projects requires a flexible approach, where audit is actively engaged throughout the project life cycle. Such an approach, borrowing conceptually from Agile methodology, allows for detailed review activities to be iteratively refined throughout the project.

Agile Methodology

Agile methodology is an alternative to traditional project approaches (e.g., waterfall or sequential) (figure 2), typically used in software development. It helps project teams respond to unpredictability through incremental, iterative work cadences, known as sprints or iterations.

Agile methodology provides opportunities to assess the direction of a project throughout its life cycle. This is achieved through regular cadences of work, at the end of which teams must present a product increment. By focusing on the repetition of abbreviated work cycles, as well as the product yielded by each cycle, Agile methodology is described as iterative and incremental. In waterfall/ sequential projects, teams have only one chance to get each aspect of a project right. However, in an Agile paradigm, every aspect of the project (i.e., requirements, execution) is continuously revised throughout the project, which provides the opportunity to change course based on new information or changes in desired outcomes or business objectives (figure 3).

Because teams can execute at the same time they are planning, the phenomenon known as “analysis paralysis” is less likely to impede a team from making progress. Also, because a team’s work cycle is limited, it gives stakeholders recurring opportunities to calibrate deliverables. Instead of committing to a deliverable, teams are empowered to continuously reevaluate to optimize value. Following an Agile methodology preserves the relevance of a project’s deliverables because it helps to ensure that maximum value is created from project efforts.

Auditing Major Projects

Adopting a flexible approach to reviewing major projects (i.e., borrowing from the incremental/ iterative approach of Agile methodology) allows the audit team to engage early while maintaining flexibility regarding audit execution (e.g., scope of review and deployment of resources). Specifically, this approach to reviewing a major project includes:

• Risk assessment—This initial engagement with the project management team enables a high-level understanding of objectives and risk to be developed.
• Scope definition—Based on the risk assessment results, define the scope of audit activities to be completed.
• Scope alignment—Determine the sprint cadence and decompose the audit scope into sprints that are aligned with the project schedule.
• Sprint planning—Iteratively plan review activities based on past results, project progress and emerging risk/issues.
• Sprint execution—Iteratively review project governance and deliverables and report results.
• Final reporting—Validate that high-level scope was addressed, provide rationale for any areas not covered, and report all findings and items for follow-up.

As an example, the following describes an Agile auditing approach was utilized for a top 10 US bank’s software implementation project:

• Risk assessment was performed to identify the major risk from a project perspective (e.g., finishing on time and within budget) and from a business perspective (e.g., data integrity, regulatory requirements, customer impact).
• Risk assessment results were discussed with key stakeholders, and the scope of review was defined.
• The scope of review was aligned with the project schedule with the goal of executing reviews at times that allowed the project team to react to audit feedback.
• The audit team executed review activities (assessment of project management processes or project deliverables, depending on the stage of the project) and reported results to executive management in monthly sprints.
• As part of monthly reporting to executive management, IA reassessed progress against the defined scope, adjusted overall scope as needed and finalized planned activities for subsequent sprints.
• The audit team issued a final report in the form of a go/no-go assessment, which supported management’s decision to go live.

Scoping/Execution Considerations

A major project audit addresses two primary assurance objectives. The review determines if the project controls and processes implemented by management are effective to manage the project (e.g., scope, schedule, cost, risk). It also determines if the project deliverables and/or solution will be implemented with adequate quality and internal controls. That is, a comprehensive major project review emphasizes/includes coverage of both project life cycle and business process risk.

With respect to the project life cycle, it is useful to consider:

• Project governance—Assess oversight practices and structures, including project monitoring and decision making, scope/cost/schedule management, risk/issue management, and value delivery.
• Development life cycle—Analyze the appropriateness of the overall project plan, delivery approach, dependencies, critical path and resourcing model. Provide guidance on system development life cycle (SDLC) controls design. Identify risk related to project execution (e.g., requirements management, quality assurance/testing).
• Operational readiness—Assess plans to transition from a developmental to operational status, including the capability and maturity of ongoing support processes.

With respect to business processes, consideration should be given to:

• Solution design—Review the overall solution design and its alignment with improvement and automation imperatives of key business processes, as well as industry-leading practices. Assess process owners’ awareness and understanding of key design decisions and expected outcomes.
• Data quality and governance—Assess the design of data integration, interfaces and conversions, including project-specific quality-assurance procedures, as well as longer-term data maintenance/monitoring approaches.
• Internal controls—Assess the design and operation of key internal controls related to system functionality, business processes and access/security.
• Organization change enablement—Examine user adoption/enablement plans for the system and processes, including ongoing user support and training processes, process organization change, and process performance measurement.

It is valuable to perform reviews using a risk-based approach calibrated for the business risk, magnitude of change and complexity of the project being reviewed. Audit coverage (i.e., of the focus areas listed previously) and sprint cadence (i.e., frequency and duration of iterative review activities) should be customized to the objectives and risk of each project.

Both ongoing and iterative review activities should be included in the audit plan, with iterative review activities and reporting aligned with the project schedule (e.g., project phases and/or key milestones). The ongoing review activities serve as a means of actively monitoring the project and provide a foundation for planning and execution of the iterative review activities. For example, ongoing review activities could include project risk monitoring and assessment of project governance, and iterative review activities could include targeted review of governance capabilities, business and system requirements, testing plans and execution, and deployment readiness (e.g., procedures, controls and training). Of course, as each project is unique, each major project review requires a unique alignment of assessment activities.

Finally, for each project under review, a dedicated audit lead should be assigned to engage directly with project management. The audit lead’s role is to maintain a high-level understanding of the project (e.g., objectives, progress and emerging risk), define overall scope and sprint review activities, and ensure that appropriate resources (e.g., process auditors, IT auditors and subject matter experts) are involved at the right time. Following this approach, some staff members are engaged throughout the project life cycle, while others are engaged for only specific iterations and/or focus areas.

Conclusion

IA should adopt elements of Agile methodology for major project reviews.

Specifically, IA should limit the initial planning cycle in favor of high-level risk assessment and audit planning, followed by iterative refinement of the audit plan. This approach allows IA to engage earlier in the project life cycle, which is a key factor in value creation.

By actively participating early in the project management life cycle, IA can proactively address risk and support project delivery. Additionally, by taking an Agile approach to planning and executing review activities, IA can maximize value to the project management office (PMO) and executive management by providing timely and relevant observations and recommendations.

Endnotes

¹ Project Management Institute, Pulse of the Profession, March 2015, www.pmi.org/learning/pulse.aspx
² Bloch, M.; S. Blumberg; J. Laartz; “Delivering Large-scale IT Projects on Time, on Budget, and on Value,” McKinsey&Company, October 2012, www.mckinsey.com/business-functions/business-technology/our-insights/delivering-large-scale-it-projects-on-time-on-budget-and-on-value