Examination and Study

This phase of the system development life cycle examines the results of the analysis, and using both the narratives and models, identifies the problems which exist as the result of misplaced functions, split processes or functions, convoluted, or broken, data flows, missing data, redundant or incomplete processing, and non-addressed automation opportunities.

The activities for this phase as illustrated in figure 17-1 are:

1. Identification of Current Environment Problems
2. Identification of problem causes
3. Identification of alternative solutions
4. Evaluation and Feasibility Analysis of each solution
5. Selection and recommendation of most practical and appropriate solution
6. Project Cost Estimation and Cost Benefit Analysis

Each system component must be examined and evaluated. Most evaluation is performed at the procedural level. Various techniques can be used during the evaluation process to uncover all the problems inconsistencies, and omissions in the existing system.

Data components are evaluated to determine whether the data gathered by the firm is:

1. Needed
2. Verified or validated in the appropriate manner
3. Useful in the form acquired
4. Acquired at the appropriate point, by the appropriate functional unit<
5. Complete, accurate and reliable
6. Made available to all functional areas which need it
7. Saved for an appropriate length of time<
8. Modified by the appropriate unit in a correct and timely manner
9. Discarded when it is no longer of use to the firm
10. Correctly and appropriately identified when it is used by the firm
11. Appropriately documented as to the type and mode of transformation when it does not appear in its original form
12. Appropriately categorized as to its sensitivity and criticality to the firm.

Each process, activity and procedure is evaluated to determine:

1. Whether all processes link together appropriately.
2. Are all processes being performed in the proper time-frame.
3. Are the resources necessary for the process available when needed
4. Are all proposed processes consistent with the job descriptions of those expected to perform them
5. Are all proposed processes consistent with the functional descriptions or charters of the organizations to which they have been delegated
6. Do the people charged with the responsibility for performance of the processes have the necessary authorities
7. Whether the supervisory personnel understand the processes in the same way that the persons actually performing them do.

The evaluation of the components seeks to achieve an end-to-end test of the analysis products, looking for inconsistent references, missing data or missing processes, missing documents or transactions, overlooked activities, functions, inputs or outputs, etc.

In many respects this evaluation should be conducted in conjunction with the other evaluation processes. The analyst and the user alike should be looking for "holes" in the document, both to determine "correctness" and completeness.

Level to level Evaluation

Depending upon the size and scope of the project, the analysis may have been conducted in levels. That is the analysis team may have first tried to achieve a broad-brush overview of the user function. This may have been followed by a more narrowly focused analysis of a particular function or group of functions, and working in successively finer and finer detail, finally arriving at the operational task level. This leveled approach usually works each level to completion before beginning the next lower level. Although in theory the work at the level under analysis should be guided by the work at the preceding (higher) level, in practice, this may not be true.

The reality is that these levels may be under concurrent analysis, or may be analyzed by different teams. The latter is especially in long running projects where staffing turnover, transfer and promotion continually change the make up of the analysis team.

Level to level evaluation and validation seeks to ensure that the information, perspective and findings at each level correspond, or at least do not conflict with, the information perspective and findings at each of the levels both higher and lower.

Carrier to Data Evaluation

Carrier to data evaluation focuses in on the data transactions to determine whether the data on the documents (data carriers) which enter the firm are passed consistently and accurately to all areas where it is needed.

It seeks to determine whether or not the firm is receiving the correct data, whether it understands what data it is receiving (i.e. what the transmitter of that data intended), and whether it is using that data in a manner which is consistent with its origin.

Since, with few exceptions, the firm collects data on its own forms, and with even fewer exceptions, the firm can specify what data it needs and in what form it needs that data, this level of evaluation, should compare the firm's use of the incoming data with the data received to ensure that the forms, instructions, and procedures for collection or acquisition, and dissemination are consistent with the data's subsequent usage.

Process to Process Evaluation

Process to process evaluation seeks to trace the flows of processes and determine whether the various processes are consistent with each other. That is if one process is expecting data from another, that they both have a common understanding of the data to be transferred. Process to process evaluation and validation seeks to answer the following questions:

1. Are the process time constraints similar between and across processes, and are all processes aware of those constraints?
2. Have all processes been identified?
3. Are there accompanying narratives for each process as a whole and for each component task?
4. Are all inputs, outputs, data storage and retrieval activities, and forms, reports and transactions for each process clearly identified and described?
5. Have all forward and backward references for these been clearly indicated and do those references have corresponding references in the sourcing or receiving process?

Zero-Based Evaluation

This evaluation approach is similar in nature to Zero-Based Budgeting, in that it assumes nothing is known about the existing environment. It is a "Start from Scratch" approach. All user functions processes and tasks are re-examined and rejustified. The reasons for each user activity are documented and all work flows are retraced.

Zero based evaluation should be undertaken where the analysis indicates that the systems, processes, activities and procedures are undergoing resystemization or reautomation. It should also be undertaken where the analysis indicates that the existing systems and automation may have been erroneous, or the environment may have changed sufficiently to warrant this start from scratch approach.

In any case, the analyst should never assume that the original reasons for data being collected or being processed are still valid. Each data transaction, and each processes must be examined as if it were being proposed for the first time, or for a new system. The analyst must ask:

1. Does this need to be done?
2. Are all the steps correct?
   1. And, are they all necessary?
3. Should this task or process be performed by the current unit?<
4. Does the work accomplished justify the resources being devoted to it?

For each report:

1. Is this report necessary?
2. If it is necessary, is it necessary in its current form?
3. To its current level of detail?
4. Should it be produced as frequently as it is?

For each process:

1. Is it still necessary?
2. Does it need to be performed as frequently as it is?
3. If manual should it be automated, and if currently automated, should it revert to manual
4. Do the procedures and standards which govern the process still apply, or should they be revised?
   1. Made simpler?
   2. Made more comprehensive?
5. Can it be combined with another similar processes?
6. Has it grown so complex that it needs to be fragmented into a larger number of more simplified processes?
7. Can the cost of the process be justified in terms of the benefit to the firm?
8. Does the volume of work expected justify the size of the organization or the resources devoted to it?

Problem identification

Although some of the problems with the existing system may have been known or suspected at the time of project initiation, the scope and extent of those problems, and in many cases the reasons for those problems, many not have been accurately known.

The analytical results not only represent the analysts understanding of current environment, and his diagnosis of the problems inherent in that environment, but also his understanding of the users unfulfilled current requirements and projections of his needs for the foreseeable future. The combination of these four aspects of the analysis will be used to devise a design for future implementation. Thus it is imperative that it be as correct and as complete as possible.

It is at this point that the analyst and user, should have the detailed analytical documentation before them, as well as, the results of the evaluation of that analysis.

During the interviews conducted during the functional, process, activity, task and data analysis phases, each user was asked questions which sought to identify one or more of the following processing problems:

1. Missing data
2. Inefficiencies in processing
3. Difficulties in data access
4. Cumbersome procedures
5. Procedures which did not reflect how activities were actually performed
6. Procedures which specified actions which were no longer necessary
7. Missing procedures

Users were asked to identify areas where responsibilities for activities were either not specified, or where responsibility for certain activities were accompanied by the appropriate authorities. Activities where responsibilities were split - where responsibility was shared across multiple functions - were identified.

Identical or highly similar activities which were performed in multiple areas were identified. Activities which appeared to be aimed at accomplishing the same purpose but which were performed in highly dissimilar manners were identified, and the causes for those dissimilarities were identified.

During the interviews, the users were also asked to enumerate processes and activities which were not currently being performed, and which could not be performed or could only be performed with great difficulty in the current environment, but which the users felt should be performed.

As the analysis progressed, the analysts should have been able to identify areas of opportunity. Areas of opportunity result from the synergy between data and process, data and data, process and data. Synergy which has created the ability to achieve a corporate mission or goal, or to provide a new service, or create a new product, or provide new information about the firm's products, competitors, customers or markets, but where that opportunity has not as yet been identified or taken advantage of.

System Design Requirements Document

The product of the Examination and Study phase is document which is usually referred to as a System Design Requirements Document, or a Client Requirements Document. This document classifies, catalogues, and enumerates all existing and identified problems and opportunities, by function, processes, activity, task and procedure. For each of these it lists the change requirements or specifications - the specifications and requirements for correcting the problems or the possible methods of taking advantage of the identified opportunities.

Simply stated, this is the verified documented list of all of places and ways where system and procedural changes have fallen behind the business activities which they were intended to support. It also documents all of the ways where the users anticipate changes to occur, or where changes which have occurred have created opportunities to take advantage of recognized synergism within and between business activities and among the resources of the business.

Prioritization of changes

Each item pair - problem and requirement or specification, or opportunity and proposed method or approach - is evaluated against all other pairs and prioritized. The prioritization may be performed by any one of a number of available methods such as:

1. A simple high, medium or low priority - reflecting perceived importance to the firm and the user
2. A simple scalar prioritization - where each item is assigned a number which places it in rank order such that the first item pair in the list is the most important, the second is next in importance, etc.
3. A complex scalar prioritization - where each item pair is assigned a degree of difficulty to implement, or a degree of importance, or both, and within the first a scalar prioritization. The result is a ranking of the most important items within another ranking method, i.e. Most important high priority item (assuming there are multiples)

The above methods (and variations thereof) are the most prevalent ones and while they accomplish part of the goal of analysis, and are necessary there are two other rankings which should be performed by both the user and the analysis and design team. These are dependency ranking and synergy or impact/benefit ranking.

Dependency ranking

Dependency ranking seeks to determine item dependencies, items which form the foundation for other items, or which items must be accomplished before certain others. This assumes that there are three types of changes which must be made during any new system design.

Foundation Changes - changes upon which all others are based. These include changes in charter, responsibility assignment or technology; Development of a new product or service, new technology; Selection of a new market; Finalization of corporate reorganization, Movement to new location, or modernization of existing location, etc.

Architectural And Procedural Changes - changes which once the foundation items are determined can be designed and developed. These are basic structural changes, and form the bulk of the difficult and usually time consuming system design tasks. These include changes in data gathering, validation, verification, storage and evaluation procedures. The also include all of the activities which aggregate and regroup those procedures to activities, the activities to processes, and the processes to systems.

Presentation, Display Or Cosmetic Changes - many changes which have been identified result from changes in user perception and perspective, or from increased user awareness of his environment and needs. These result in requests for changes in report form and content, faster access to existing data, or improved methods of accomplishing existing tasks. These changes usually do not require structural or architectural changes. Each of these changes while small in itself, when added to the vast number of other similar changes may require the largest part of the design team energy.

Synergy or impact/benefit ranking

Synergy or impact/benefit ranking is the most difficult task to perform in this phase but it is also the one where major benefits to the firm exist. Although it is termed a ranking, in reality it is the examination of each item or combination of items (the items in this case being proposed changes or modifications) to assess;

1. The number of specific, recognizable, direct or indirect benefits which are expected to be achieved from this change
2. The extent of the positive impact, that is:
   1. The number of other changes which could be accomplished if this change is made
   2. The number of other changes which would not be necessary if this change is made
   3. The number of other changes which would be made easier if this change were made
3. The number of additional (and unanticipated) things which could be accomplished or goals which could be achieved if this change or combination of changes were made
4. If this change in combination with one or more other changes, if all made at the same time would increase any of the above (synergy)

Cost benefit ranking

The Examination and Study phase usually includes a cost benefit analysis of each proposed change, and an evaluation of the costs and benefits of each change with respect to its various rankings and prioritization. The item by item cost benefit analysis provides another (and sometimes the most definitive) method of evaluating the proposed change specifications and requirements.

Cost Benefit Analysis

The Examination and study phase is completed when a cost benefit analysis is developed for the system design phase for the selected approach.

A system design project will require the expenditure of a certain level of resources, both in terms of manpower and other assets. In effect the firm is buying a product, the completed work and the accompanying document, from each phase. In order for user and data processing management to assess the relative value of each of these products, in advance of their being contracted for, many firms require a separate justification document, called a cost benefit analysis. This document provides, in estimate form, the costs for producing the product, and the benefits which the firm can expect to accrue as a result of that effort.

In the early stages the project has little form or definition, thus early cost benefit documents are predominantly based upon estimates. As more and more information is developed these cost benefit documents become more and more detailed. and more and more accurate.

Cost Benefit Analysis (CBA) is used in two ways. The first being to develop estimates for project budgetary planning and later actual costs for project monitoring purposes. This is usually done at each phase of the development project. This allows Data Processing management to assess whether the costs of the project can be justified by the potential benefits to the user and the firm. The second is to develop user costs and benefits which will result if the project is implemented. This allows user management to plan future budgets and to estimate when the costs will be incurred. Although the costs in the first CBA contain only project costs, the costs in all succeeding analyses should also contain separate projected user costs for implementation and on-going operations.

The first cost benefit analysis is usually performed at the project initiation phase and is used by user and DP management to allocate a budget for the first phase. At the end of the problem identification and analysis phase a second cost benefit analysis is normally generated. Since work has already been performed, and the scope and direction of the project are fairly clear, this document can be based upon the concrete information as to costs and benefits. The development alternative selected will allow hardware and software costs to be added, as well as user documentation and training costs to be estimated. At the end of the Proposed Environment Analysis phase user costs should be well defined and fairly accurate numbers generated.

During the problem identification and analysis phase, it is usually necessary to perform a separate CBA for each alternative being evaluated. These alternative CBA documents can assist in making the selection between multiple viable alternatives.

Project Development Costs

All costs which can be expected as a result of thus unit of work should be estimated and itemized. Cost figures are usually generated by the project manager and the systems analyst assigned to the project. Generally, these costs will be broken out into several categories:

The first, and perhaps the most important category is that of direct manpower costs - these costs are developed from the project plan for this phase. From that plan, should be extracted the number of people, at each level, and the number of periods (hours, days, weeks, months, etc.) that each person will be working on the project.

A per period costs should be assigned on any one of the following bases:

1. Standard cost - applicable to all personnel at all levels,
2. Level by level costs - applicable to all personnel at each given level
3. Per person costs - based upon the compensation for each person (this is only used where there are no company policy restrictions on publication of personnel compensation)
4. The per period cost for each person should be multiplied by the number of periods each person is expected to work on the project and the results for all assigned personnel totaled. For the best results, each person, the person's level, the costs assigned to that person, and the number of periods the person is expected to work on the project should be detailed, either in the body of the analysis or as a separate appendix.

When developing these costs, be sure to include

1. Any benefits additions, payroll added costs, and vacation and holiday costs to be assigned to the project.
2. Indirect manpower costs - these include project management, staff and administrative services which may be needed to support the project personnel. In this category would be project managers, area managers, secretarial, filing, etc. Costs. These may be calculated in the same manner as direct costs, estimated, or taken as a percentage of the total development department administrative costs.
3. Travel costs - if the project requires personnel to travel between locations, or if personnel must be transported to a central location, all costs of travel, accommodations, board, entertainment, and other travel related expenses should be estimated. In lieu of an estimate, a budgetary maximum may be used.
4. Printing and graphics costs - the development of charts, graphs, diagrams, and printing the final report (usually in multiple copies) should be estimated.
5. Space costs - if staff must be assembled from diverse locations, it may be necessary to assign office space for them to work in during the project's duration. These costs should be calculated from the firms space budget costs and added to the project cost totals.
6. Machine costs - these include estimates of the costs of any computer time necessary to run analyses, simulations, print machine stored documentation, analyze machine stored files, analyze existing user applications, create duplicate copies of user reports, etc.
7. Supplies costs - these include costs of paper, copier supplies, binders and covers for reports, pencils, pens, other stationary supplies, any special materials which may be necessary for the project.
8. Training costs - these include any special training required by project personnel in terms of familiarization with specific business functions, special software, special machinery, general education, etc.
9. Special software costs - these include the acquisition of any specialized software for either mainframe, mini, or microcomputers which will be necessary for the project.
10. All costs in each of these categories should be totaled and the sum becomes the total project cost.

User and Project Implementation Costs

Aside from the costs to be incurred in the project itself, there may be ongoing costs to be incurred when the project is completed and implemented. These costs fall into the same basic categories as project costs, with the difference that the costs are developed for the user area staff, etc.

In addition to the above costs, the following might be included:

1. Special user training costs - including all costs for training user personnel in the use of the new system, procedures, etc.
2. Special or new user documentation and user forms - including all new forms and documents, manuals, procedural documents, etc. which might be needed by the user as a result of the new system being implemented.
3. Special user hardware - including user owned computers, terminals, printers, Personal Computers, communications devices, security devices, etc.
4. Special user software license or acquisition costs - including additional copies of PC software, multiple site licenses for mainframe and minicomputer locations, on-going training costs.

Benefits

Benefits are usually more difficult to quantify, and most benefits are shown in estimated form, especially during the early stages of a project. In many cases, these benefits numbers are obtained form the user requesting the project.

They are determined by quantifying the costs of the existing environment and problems, or by estimating the savings from rectifying these problems. Benefits are usually broken out into two sub-categories - Direct Benefits and Indirect Benefits.

Direct Benefits - are those whose value can be calculated or estimated in monetary terms. These include savings or reductions in:

1. Direct or indirect manpower costs - these cost savings are calculated in the same manner as development direct and indirect manpower costs. These benefits are obtained from the reduced level of effort, or reduced level of staff which can be expected to be achieved if the project is successful.
2. Supply costs - these cost savings are calculated by estimating the reductions in supplies usage (paper, stationary, etc.) which can be expected to be achieved if the project is successful.
3. Machine Costs - these cost savings are calculated by estimating the reductions in computer usage which can be achieved if the project is successful. This benefit area may also include reduced rental or lease costs on machinery being upgraded as a result of this project.
4. Travel costs - these cost savings are calculated by estimating the reductions in travel and entertainment costs which can be expected to be achieved if the project is successful.
5. Interest or penalty costs - in many cases inefficient operations result in the company having to borrow money to finance its operations, or to incur penalties for late payments, etc.

The benefits here are estimates of those reductions of such costs which might be achieved through more efficient processing if the project is successful.

In some cases the benefits may be achieved through actual increases in revenue, or faster collections, or more rapid turnover, or increased productivity. Some examples of benefits to be looked for, and estimated, in this area are:

1. More rapid processing of customer invoices decreases the time between shipment and billing thus decreasing the accounts receivable period and increasing the cash flow.
2. More rapid identification and payment of vendor invoices offering discounts allowing the firm to reduce its purchasing costs.
3. Faster access to information allowing employees to process more transactions in the same period of time.
4. Reductions in work due to reducing the number of processing steps or resulting from simplified procedures
5. Improved editing or validation procedures which reduce errors.

Indirect benefits - are those which can't be quantified, or assigned a monetary value, but which nonetheless result in desirable outcomes from the project. These might include:

1. Access to previously inaccessible or missing information<
2. Faster access to information
3. More flexible processing, or additional processing capability or ease<
4. Additional functionality
5. Standardization of information
6. Improved communications or reporting procedures
7. Improved, clearer or more accurate reports or screens
8. Reports providing more detail, or reports removing unneeded detail.
9. Improved understanding of the basic functional or processing interactions within the firm.
10. Improved documentation of the basic functions and processing within the firm.

A cost benefit analysis may be as short as a single page or may cover many pages. In format it is similar in style to a standard budget, with costs being equated to the expense side, and benefits being equated to the income side. All cost items should be subtotaled by category and an overall total cost figure computed. All benefits should be subtotaled by category and a total benefit figure computed. In more complete cost benefit analyses, both the costs and the benefits would be "spread" across the project or phase time frame, with the costs and projected benefits.

The data requirements of the operational level units, while extensive, rarely change since they are contingent upon fixed sources of input.

Additional analysis techniques

Many other techniques are used during the Evaluation and Study process to gather and analyze data, some of which are shown in Table 17.1, Examination and Study Techniques.  The set of tools used during any given project depends upon the approach selected.

Table 17.1. Examination and Study Techniques

Risk Identification And Mitigation

All projects have risk associated with them. All change, even the most trivial have associated risk (the least being that the change won’t achieve the desired result.)  Risk management is the combination of processes and procedures for managing areas of risk associated with a project, or with the changes proposed by the project. Risk management will:

1. Identify the areas of risk and prioritize them.
2. Identify the constituents risk factor that contribute to the potential occurrence of each risk.
3. Document procedures for monitoring the risk factors and for reducing the potential occurrence of each risk.
4. Identify contingency procedures for each area of risk, as appropriate.

Four steps of risk management:

Risk Identification.  This activity identifies and evaluates the risk and its source.  The risk evaluation should include identification of metrics for evaluating progress and determination of the risk impact (e.g., cost, schedule, and/or technical performance, business efficiency, product safety, customer perception, legislative and regulatory impact. ).

Risk Assessment. This activity consists of risk analysis, qualification, and quantification.  The effect of the risk on the system, the seriousness of the risk, the priority (high, medium, low) of the risk, and the size of the risk should be determined.

Risk Containment. A containment plan should be developed to mitigate each risk.  Included in the containment plan should be the identification and method of collection of metrics to be used for evaluating progress in containing the risk, the progress evaluation schedule milestones and reviews, and responsibility for evaluating performance against plan.  One or more approaches or contingency plans for containing the risk should be identified. Each individual responsibility for containment of each risk and should be documented.

Risk Closure. Authority to determine that a risk has been closed should be identified. Each area of risk should be clearly documented and the status of the progress toward containing or eliminating each risk should be regularly documented clearly indicating both open and closed items.

As risk conditions are eliminated or as the containment activities are completed and considered closed, they should be removed from the open item and logged in a history file with appropriate documentation reflecting how they were eliminated or closed, who performed those activities and who authorized closure, and when.

Risk Issues

Potential risk issues associated with changing any business processes include:

1. Selection of the wrong process to redesign
2. Failure to develop an adequate change management strategy
3. Failure to properly measure and map the process
4. Inability to act as a communication facilitator and team builder
5. Maintaining a rigid organizational structure
6. Failure to meet tight schedules.