Major Computer Systems in Mississippi’s State Agencies: A Review of Their Development and Implementation
Executive Summary
The majority of the state’s current computer system implementation projects have experienced revisions in estimated costs and completion dates. PEER sought to identify factors within the state’s process for developing and implementing new computer systems which could lead to systems costing over budget or taking longer than originally planned. The review included computer system projects in process with estimated costs greater than $1 million as of March 31, 1999, to identify possible weaknesses in how agencies manage projects throughout the process.
State law authorizes the Department of Information Technology Services (ITS) to protect the state’s interest in the development and acquisition of agencies’ computer systems. In practice, ITS’s roles and responsibilities vary according to the stage of development and implementation of a state agency’s computer system project. ITS has not fully exercised its authority to compel state agencies to use project planning and management procedures. It has assisted agencies in system planning and development on an as-needed basis.
There is no easy answer to the question of why computer system implementation projects often run over time and over budget. The causes are often complex and the pattern of problems is unique from project to project. However, through an analysis of the performance details of three large projects, PEER found that the primary condition associated with time, cost, and functionality problems was a failure to adhere to one or more of the generally accepted project management principles during system development and implementation. Agencies lacked fully defined project objectives and requirements, did not sufficiently review vendor experience and resources, failed to involve system users in designing and testing the system, failed to limit changes to the system once the project had commenced, failed to divide the project into manageable milestones, or did not engage in substantive quality assurance review.
Other factors which play an important role in successful computer system projects include proper communication among the project team members, adequate training of project staff, fostering project team morale, and building an atmosphere of teamwork between the vendor staff and the agency staff. Each of the projects PEER reviewed experienced difficulties with at least one of these factors.
Compounding the problem of lack of adherence to generally accepted project management principles, agencies’ lack of attention to accurate costing and cost reporting inhibits external oversight efforts. Agencies do not uniformly and periodically report cost information on large-scale computer projects to ITS, any other state agency, or to the Legislature. Also, current monitoring and reporting methods do not capture all personnel costs of a computer project. Without the uniform accumulation and reporting of segmented costs of large-scale computer system projects, the Legislature does not have all the information needed for decisionmaking.
Recommendations
Carnegie Mellon University’s Software Engineering Institute has developed a model framework for managing projects involving the development and implementation of software. ITS should consider this model, along with others, in developing management guidelines specifically designed to accommodate the needs of agencies, ITS, and the Legislature.
Under its authority to approve or disapprove contracts as specified in MISS. CODE ANN. §25-53-5, ITS should require as precondition to contract approval that agencies follow the promulgated guidelines and requirements in performing feasibility studies of proposed systems and in designing, developing, and implementing computer systems approved by the ITS board.
At a minimum, the ITS Board should require an annual independent quality assessment of each computer system project with a budget exceeding $1 million. The purpose of the annual independent quality assurance assessment is to have an independent review of the project to identify problems that could cause the project to be over budget or delay its implementation. For example, such problems could include poor quality of work by the vendor, lack of vendor or state staffing, poor communications in the resolution of problems, project team morale problems, or excessive change orders. This review should be conducted by ITS unless ITS staff are participating in the design, development, and implementation of the system, in which case an independent consultant should conduct the review.
The results of the annual independent quality assurance assessment, along with recommendations for addressing any problems noted in the project, should be reported to the ITS Board. The ITS Board should endorse recommendations it believes are needed to correct problems noted and, if problems persist, take aggressive action to ensure that such problems are addressed. Such action could range from refusal to approve further change orders on troubled projects to directing the ITS Executive Director to cancel a project vendor’s contract, if warranted.
The ITS Board should report these findings, along with any of its own recommendations, which could range from endorsing recommended solutions to canceling the project, to the Joint Legislative Budget Committee as part of the budgetary process.
As part of their appearance before the Budget Committee during annual budget hearings, executive agency managers should address project problems and recommendations noted by ITS and outline needed corrective actions.
•require agencies to capture all costs associated with a computer project, including expenditures from all sources;
•require agencies to report a project’s originally estimated cost, revised project cost (if applicable), and the amount spent as of the end of the most recent completed fiscal year;
•require agencies to report a project’s originally estimated completion date, revised completion date (if applicable), reasons for any delays, and actions to be taken by the agency to address any delays; and,
•require agencies to capture personnel resource costs by implementing a tracking system (recommended by ITS) to capture employee time dedicated to computer projects. This tracking system should, at a minimum, capture the number of hours which agency employees spend on the design, development, and implementation of a new computer system.
•the creation of separate funds for computer projects over $1 million in order to capture and track all related expenditures; and,
•agencies to report all costs relating to a computer project (including funds expended from all sources).