Effects of prior automation:
“Legacy Systems”

Almost every organization has some degree of automated support. The degree of this automated support varies from organization to organization and depends on its resources, its level of sophistication and its requirements. This prior automation is almost always the result of the efforts of a previous development team, whether they were employee, contractor, or vendor.   These previously automated systems are referred to by the generic name of “legacy systems.”  Although we usually think of legacy systems as automated systems, they can be wholly manual, partially manual; and partially automated, or wholly automated.  New systems, the results of the system analysis and redesign process, produce reautomation.

Introduction

Although we refer to analysis as part of the system design process, it should more appropriately be thought of as a part of the system redesign process. Normal business change cause procedures and resource usage to change over time.  All procedures and resources operate within an existing system which while appearing to be stable and unchanging is in reality undergoing constant gradual and sometimes radical change. Moreover, although business systems may appear to be inflexible, they are in fact very informal, flexible, tenuous and limited in nature.

When the number of changes to an individual procedure reach critical mass (that is, new changes can no longer be applied, or the user can no longer function effectively) the individual procedure must be changed.  When the number of individual procedures which must be changed reaches a critical mass all the procedures and the framework within which they operate must be changed.  When the change encompasses all of the procedures, the resources used by them, and the framework within which they operate, we are changing the system.

One of the reasons which make procedural changes difficult or impossible, is that the system within which they operate is not flexible enough or cannot provide the resources which are needed for their operation.  In most cases unless the procedural changes are mostly cosmetic, they must be accompanied by some system modification.  That is the system must be redesigned to a certain extent.  This need for redesign initiates the analysis of the current environment.

Although the effects of gradual procedural change and the need for more extensive procedural change can be tolerated by the firm for relatively long periods of time before critical mass is reached, there is much less tolerance for gradual data change. When data requirements change those changes by and large must be implemented as soon as possible. However, whereas procedural change can be developed and implemented rather quickly and with low impact on the firm, the data change takes time and has a substantial impact on the firm. This is true whether the changes required are for new data or for changes to existing data.

Data change is similar to changes in language, and in fact as we shall describe in later chapters, there is a strong parallel between language and data.  Changes in meaning of words can make existing texts obsolete, or meaningless. The lack of the appropriate words to describe something or some event renders communication ineffective. Use of the wrong word can have extremely negative effects in any communication situation.

In very similar ways, and for very similar reasons changes in the meaning, or use of data can render information, or business communication ineffective.  As with words, the meaning of data changes over time, often subtly, gradually and unintentionally.  While this happens with all data in whatever form it is stored, its effects are most strongly felt when it is computer stored data that changes.

Different redesign environments

There are substantial similarities between all redesign efforts, but there are also substantial differences.  These differences are apart from the obvious applications or functional area differences which make payroll different from order entry, and inventory, different from accounts receivable.  These differences are those which are due to the effects of prior automation (or the lack of it), and the level and extent to which automation will be used in the redesigned system.

These differences occur most strikingly where data is concerned. Conversion from manual to automated data record keeping, from card media to tape and disk media, and from files stored using basic operating system file storage and access methods to files store and managed by data management software known as data base management systems, or simply as data management systems.   There are even differences which occur when data is transferred from one data base management system to another, or when data is allocated between several different data base management systems.

Additional considerations occur when data and processing is moved from the centralized or mainframe envionment to the decentralized or distributed environments, and still others are introduced when data and processing are migrated still further to microcomputers (also called personal computers or workstations). Each of these entails different decisions and requires a different set of criteria to be analyzed when determining where and how data should be collected, maintained and stored.

However these differences not only occur when data storage and processing mechanisms are changed, but also when data philosophy changes (usually based on some new business or competitive need).

Reasons for Initiating a System Redesign Project

Aside from the need to apply the accumulated backlog of business changes, redesign projects may be initiated for a variety of other reasons. These reasons may be attributed to:

1. a change in the basic aspects of the user's functional role,
2. a change in company strategic objectives,
3. a need for increased performance from the automated systems,
4. a need for more direct and immediate access to the firm's automated files,
5. a desire to upgrade the system to take advantage of more current technology, or<
6. a need to clean up the system.

These reasons reflect the procedural needs. In addition, and in many cases more importantly, there are strong data reasons for initiating such projects. Almost all of any given firm's business procedures have been developed to acquire, maintain or disseminate data, or its derivative - information. The techniques used to analyze and redesign procedures have been developed and refined over time. These techniques may be data directed or process directed, but all focus on the end product of procedural development. Many of these techniques however, treat data as a by-product, or an afterthought. This is true even of the newer "engineering" based techniques and CASE based techniques.

These approaches to redesign generate data models because data models are called for and they link, or associate the components of the data model with the process components. In many cases however the data model itself is ignored after that, and there is no methodological or support techniques for developing the data model itself or analyzing it for validity, accuracy or completeness. In fact, many of these approaches begin with a top-down generated data model, and then proceed to compete it from the bottom up by populating it with know data items, or data items suggested or requested by the user participants in the redesign process.

Regardless of whether the scope and magnitude of the functional, process, activity and procedural changes is fairly narrow or wide reaching in almost all cases the data changes is substantial, or has the potential for being substantial. In some cases, aside from streamlining and updating the procedures of the system, there may be only minor changes to either the data architecture or in additional business processing capability aimed at providing more meaningful or usable data to the firm. Very few of these projects look at data opportunities, or evaluate existing data to determine whether it was correct in the first place. In other words, data in existing files very rarely gets discarded or dropped from the files of the firm.

Aside from the variety of reasons for a project being undertaken, we must also recognize that the technical base differs from project to project. This technical base reflects the differences in current user processing environment technology, the current level of user automation, and current residence on the maturity curve. Because of these differences in current user processing technology and user automation, and because the goals of any system redesign project normally include a) movement along the maturity curves, b) an increase in the level of system integration, and c) an increase in the level of information which can be obtained about business operations, systems redesign project activity must reflect or at least be cognizant of the level of technology and automation of the current environment.

These automation levels can be categorized into three types which reflect both their existing level of technology and the technology of the redesigned system. These categories are:

1. basic systems - wholly manual
2. intermediate systems - partially manual and partially automated
3. mature systems - fully automated

From a redesign perspective, each of these three automation levels involve different, and yet similar work. The work is similar in that the development activities, which are involved in each, follow the same general phases and approach. They are different in that each of the starting or current environments that the system redesigner must use as a base have substantially different characteristics.

The following is an overview of the redesign activities in each category of system.

A. The Basic system environment - wholly manual procedures

In a very few cases today, the current system is manual and the redesign team has determined that the redesigned system should remain manual. This is the simplest environment, from the redesigner's viewpoint in that all the procedural components of the environment are overt as is the framework. That is, they are clearly visible for observation and validation.

Data records are usually paper and stored in massive cabinets using a variety of filing systems, and these systems are characterized by extensive redundancy of data. That is data records are generated on multi-part forms and each copy is disseminated to different areas of the firm, often changing independently of each other. These purely manual systems tend to be small, containing relatively few procedures.  The procedures themselves however may be exceedingly complex. These procedures tend to fall into identifiable categories:

1. procedures which involve the handling of physical items, such as materials or original documents.
2. procedures which involve the operation of machinery
3. procedures which involve high degrees of human interaction, or human judgment
4. procedures which are highly complex and apply to very few cases, which have to be modified for each case.

In manual systems, all work will be performed by user personnel, who work directly with physical files, forms, documents and materials. The proposed processing of these forms and documents, the work flows, the individual steps can be simulated and modeled relatively easily. The creation of work flows, data flows, and physical material flows form the heart of the system redesign.

In the manual environment, there is a direct, in most cases one-to-one correspondence between user task and procedure. Aside from the obvious use of machinery, this is one of the primary differences between manual and automated system.

From a system redesign perspective, all systems are concerned with what are, or once were, essentially manual operations. In fact, it is helpful during the redesign process to view all the activities of the user as if they were still being performed by hand. This allows the redesigner to review, in detail, each task being performed, each data operation, and each data movement, and each data carrier. (A data carrier is a piece of paper, a form, a report, a worksheet, a transaction, etc.

The redesigner's task in the manual environment is to rewrite the procedures which govern task performance and the work flows to streamline the processes, reduce redundant processing, and rearrange the tasks so as to ensure more orderly processing. The redesigner must also ensure that the forms, documents and reports which both enter and leave the user area contain all necessary data.

The procedures for each task, and each task step, must be placed in perspective within the user area and the user's surroundings to determine its appropriate definition, position, and level and mode of performance.

The redesigner must ensure that each new procedure conforms to existing standards and that the procedures are collected and appropriately indexed, cross-referenced and documented.

Any new or revised standards and procedures which are included in manual systems must clearly define the processing sequence for the task to be performed, the rules which govern their performance, and the responsibilities and authorities of the person or persons to whom they will be assigned. The redesigner must also revise any user enabling documents such as job descriptions, unit charters, and mission statements. In addition the redesigner must develop new input forms, control procedures, monitoring procedures and reports. The redesigner must also develop new or revised work and data flows diagrams.

Procedure content and form

Each procedure must include standards of performance. Standards of performance describe the quality (and sometimes the quantity) of the work to be performed, the time-frame within which the work should be performed, the amount of time which should be devoted to each unit of work, and the number of exceptions which can be tolerated.

Each procedure should include responsibility and authority information. This information describes who is responsible for performing the work described in the procedure, and what authorities are vested with the person or persons performing the work.

Each procedure should include descriptions of the security and control aspects of the work. All steps to ensure the security of the materials and resources used in the work should be described, and each resource should be classified as to its security level, storage, use and disposal criteria. All controls which are included in the procedure should be clearly described and should show how the control is enforced. Controls can be simple financial controls, such as the addition of various balancing steps, to more complex batch preparation and verification steps. Controls can be applied to materials, documents, reports, and other physical materials, which must be counted and accounted for.

Each procedure should include information which shows how it relates to other procedures. All materials, data and information sources which are external to the procedure should have their source documented, including the source of each form, report, document, or other information source. All materials which have their source in the procedure or which are processed by the procedure and which must be passed to another procedure should have their destination clearly indicated.

It should be noted that while some user information processing systems are and will remain wholly manual, even in these cases they may interact with, and either receive data from, or pass data to systems which are partially or completely automated. In these cases, while these other systems might remain unaffected by the new manual system, the redesigner must examine the impact on them, and my have to tailor the new manual procedures where these interfaces exist to conform to what is being delivered to or received from the automated systems.

B. Intermediate systems - partially manual and partially automated

For many currently manual systems the determination is made by both the user and technical members of the redesign team that some of the current activities, in whole or in part, can be cost effectively augmented by automation. In other cases, the current system is already partially automated, and the new redesign will maintain or probably increase the level of automated support.

One major difficulty in this mode arises when a normally integrated, or straight line, procedure must be partitioned into manual and automated segments. The system redesigner must always consider the impact on the timing and execution of the manual segments when making the determination as to how to apply automation to those segments to be automated. The redesigner must always look to the larger, and more global user process within which the automated segment fits when making these implementation determinations. These framework or global user process considerations affect implementation choices, such as hardware and software choices, batch versus on-line execution, and other similar technical selections.

The analysis phase and the subsequent examination and study phase will have identified those current manual procedures where it is feasible to substitute automated processing for manual processing and where such automation is suitable. For each suggested area of automation, the redesigner must break each process, each activity, and each task down into its component steps, and determine if the rules for performing the step are sufficiently well defined and if there are a sufficiently low level of exceptions to allow the procedures to lend themselves to machine automation.

The redesigner's product for this type of project closely resembles that produced for a strictly manual system redesign. However, here the redesigner must also develop new, input forms suitable to an automated environment, file content requirements, for on-going master, transaction and reference files, report layouts and a processing flow which intermixes the original, and unmodified manual processes, new manual processes and new automated processes.

When preparing the redesign for procedures where segments will be automated and others not, or where new automation is being applied, the redesigner must prepare supporting plans for "conversion" procedures. Conversion procedures are those which must be developed for single, or in some cases on-going use, and which are normally used to prepare data in manually maintained files for ultimate storage in machine readable for.

In mixed mode environments (partially manual and partially automated) a decision must be made as to whether that data is to be kept in wholly automated form or in both automated and manual form. In the first case, wholly automated files, the redesigner must make provisions for procedures which produce and distribute file content reports, edit, validation, maintenance activity, and reference reports which would be needed to support the manual segments. In the second case, the redesigner must make provisions for procedures all of the above, and for additional procedures which must be performed to ensure that the contents of the manual and automated versions of the users files are coordinated.

The redesigner must also make a determination as to the costs involved in the automation process, project schedules, and hardware and software recommendations.

C. Mature systems - fully automated

Although we like to consider that some user systems are completely automated, in reality no system is completely automated. Mature systems are in reality those where all procedures which are capable of being automated cost-effectively have been automated. These systems still have many manual procedures, but whereas in the prior environments automated procedures supported the essentially manual character of the system, here the manual procedures support the essentially automated character of the system.

In today's business environment, most firms of any size have many user areas with heavy levels of automation. Many in fact have progressed from concentrating system redesign efforts on initial automation to redesign efforts which concentrate on re-automation.

In these user areas most of the existing processes and procedures are either totally or partially automated. Automated procedures and the manual procedures which support them, are the most complex, most highly integrated and the least visible (or overt), of any business procedures, and thus are the most difficult and time-consuming to change.   They are also the least amenable to change. Consequently they usually lag furthest behind the on-going business changes.

The nature of information processing technology is such that in many of these systems, changes which would be trivial in a manual environment, become monumental in the automated environment. Interestingly enough in many cases it is changes in data requirements definition or format which has the greatest impact on automated systems, and it is these which appear to be the primary impetus behind many re-automation redesign efforts.

Effects of Prior Automation

The effects of this prior automation on the re-automation redesign efforts must be fully understood by the redesign team. This prior automation impacts all forms and documents currently used by the business area users, all automated and supporting manual procedures, and the structure and contents of all data files used by the automated system. The technology used to construct these automated systems, may now be outdated or inefficient, or worse obsolete. Business changes which occurred since these systems were originally developed, may have completed changed the user requirements or business environment which the system was originally redesigned and developed to support.

The difficulty of changing automated systems is such that it is usually easier to twist and contort the user processing to fit the system that it is to change the system to fit user business processing needs. As the business changes mount, the contortion or twisting in the user area is such that when the tension is released by the initiation of a new redesign effort, the emergent processing requirements bear little if any resemblance to the system being replaced. Procedural forms, documents, reports, and file contents which were the result of some prior redesigner's efforts and may no longer reflect the current information or data needs of the firm.

The natural user procedural processing flows themselves may have become unnatural, to the extent that they were modified to reflect the intrusion of less-than-effective automated processing sequences, minor attempts to repair systems which no longer function properly, and the normal contorting of business procedures which occurs when individual task procedures are consolidated to accommodate efficient and integrated automated processing. These automated processing flows may have been structured to accommodate the needs of the then prevalent technology, rather than the needs of the business and as a result the user processing was changed to accommodate the needs of automation.

Each of the individual task procedures both automated and manual, the process flows which have been established to govern and control the procedures, and the documents, transactions and data files all must be re-examined in the light of the current business environment, the current business processing needs and the currently available and usable technology. Because of the combined effects of the changes necessitated by business changes, those necessitated by technology changes, and those which result from the natural pressures to move forward along the maturity curve, increase the levels of integration among the firms processing systems, and the needs of management to have increased access to and availability of business management information (control information, monitoring information, etc.) the most common decision by the redesign team is to scrape the existing system entirely in favor of a new and more streamlined processing flow, new procedures, new data files, new documents, forms and reports, etc.

These re-automation projects are very difficult, complex and time-consuming, because the replacement systems are almost always larger, more complex, more global (more integrated), more technologically sophisticated, and have more capability than the system or in most cases systems which they replace.

Forms of Re-Automation

Re-automation project activities may take the following forms:

1. Basic changes - Simple procedural rewrite

The procedural rewrite is one of the simplest aspects of re-automation. Procedural rewrite does not major framework or architectural changes in the redesign of the current business system and thus entails very little structural change to the user business processing capability. Procedural rewrite normally results in the exchange of the old version of procedural component with a new version - a component for component substitution.

Procedural rewrite does not involve change to the inter-procedural processing flows nor does it add to system complexity. It is simply a clean up of the procedural processing within the existing flows usually in a more up-to-date language or with more streamlined logic within the automated procedures themselves. Because the opportunity exist, may of these efforts incorporate those business changes which can be accomplished within the context of the original system structure and with existing system resources.

In the process of the procedural rewrite, minor changes may be made to existing file structure, data content consistency may be cleaned up, report layouts, screens, and transactions may change to a minor extent, but generally speaking the changes are predominantly cosmetic in nature and as such file and report content, the availability and accessibility of data to management, and system business support capability usually do not increase. There is usually little of any impact on the user area. A procedural rewrite may eliminate processing anomalies ("bugs"), remove unused, obsolete, or outdated code and may add minor new data presentation capability to the system. The emphasis here is on minor.

Procedural rewrites are usually done for the benefit of, and at the initiation of, the information processing area, specifically for either the system maintenance personnel who need a "cleaner" system to maintain, or the data processing operations personnel who have requested operational changes to streamline the processing, or to make it more perform more efficiently.

2.  More complex changes - procedural enhancement

Procedural enhancement has may characteristics in common with procedural rewrite. One of the main one being that it usually requires minimal of any framework or architectural changes to the system. Procedural enhancement may result in increased user processing capability, increased user access to existing data files or the introduction of new procedures into the system framework.

These types of changes require all of the activities which need to be performed at the procedural rewrite level, and additional activities to assess the impact of the proposed changes on an existing system and to make those architectural, framework or other procedural changes (automated and manual support) which are needed to enable the additional or enhanced procedures.

These procedural enhancement projects, usually leave the majority of the system framework intact. They also leave intact, and usually unchanged, the bulk of the system resources. The framework may be modified slightly to reflect the new or expanded capability of the existing procedural components as well as to accommodate newly added procedural components. Other procedural components may have to be slightly modified to support the added capability and to provide the needed interface capability with the new and revised procedures.

The requests for procedural enhancement normally originate with the user, although they may originate with the development team itself.

There are numerous reasons for this level of system modification requests, among them are changes to the business environment, user requested additional capability, correction of erroneous processing, and user requested refinements, or cosmetic changes, to the existing system. In most cases these changes are relatively easy to make (that is easy in relation to a full-blown system redesign project) and incorporate a sufficient level of business changes and new capability. These types of changes are normally sufficient in stable user areas (areas which are not subject to a high degree of change), and can normally be accomplished within a short period of time.

Of these, the procedural enhancements which originate from alterations to the business environment are the most difficult to incorporate into an existing system design, followed closely by those which add new processing capability. These implementation difficulties arise because these types of changes not only require new procedural redesign but also selective modification of the original system design to enable the procedural changes to be made. Procedural enhancement which is necessitated by the need to correct erroneous processing, by the need to add user requested refinements, or other cosmetic changes, usually require little in the way of system redesign modification.

The redesign efforts required by business environment changes and additions of new processing capability can be almost as extensive as those which were required in the original systems development even though many system components and large parts of the system framework remain untouched or are only changed in minor ways.

The most difficult types of procedural enhancements are those which also require changes in underlying structure, format or contents of the system data and reference files or the system database

Procedural changes which require additions or modifications to portions of the system files or database, may not only effect the system being changed, but any other system or application which uses the same files or database, and in particular those which use the same data records or data elements.

In some cases, the desired or needed procedural enhancement will require data which can only be procedurally captured by a different, unrelated, application. The "chain reaction" or "cascading" of changes can increase the scope and impact of the initial request by orders of magnitude. When database technology has been used a means to achieve system integration, this integration by its nature causes a greater interdependence between what might otherwise be separate systems. The greater the level of integration or interdependence between systems, the greater the potential impact of any change to the integration mechanism.

These data driven changes, when the files or database must be modified can have major impact. The more extensive the use of the database, the more thorough and painstaking the system modification efforts that are required. Minor changes in format or content definition may require changes in all procedures which use the modified data items. These include data acquisition, data editing and validation, reporting, retrieval or other data access or presentation facilities may be impacted by these data structure, content, or processing logic changes. Although the individual procedural changes may be minor in each case, the cumulative effort to make these changes may severely impact the time needed to effect the changes.

3.  Most complex - complete system redesign and redevelopment

The complete system redesign and redevelopment project is the most comprehensive and difficult type of re-automation and thus is the most difficult and comprehensive of any of the variants of system redesign project. It involves:

1. the procedural modifications and redesign of a the "new" development project
2. the additional activities of data file, and database conversion, and
3. reintegration of the newly redesigned system into the global family of systems under which the firm operates.

System redesign and redevelopment requires a start from scratch redesign, which must at the same time acknowledge

1. the presence, capabilities, and limitations of the existing system;
2. the needs and desires of user areas who have lived with an increasingly restrictive system, and who have a increasing backlog of business, cosmetic and ease-of-use changes to be included in the next version of the system;
3. the pressures and lure of newer and more attractive hardware and software technology;
4. and pressures from management to provide more useful and timely information, more service, more flexibility and more accessibility to information, to provide more integrated information, more monitoring and control capabilities

A project of this magnitude is usually undertaken when

1. the backlog of pending changes reaches critical mass;
2. when a single radical change has occurred, such as corporate takeover, or the introduction of a new product or product line by either the firm or its competitors;
3. the firm undergoes a radical reorganization, or restructuring, or radical change in senior management. This internal restructuring may have been as a result of a shift from a centralized to a decentralized environment, or as a result of the addition of major new lines of business, or of a change in orientation of the firm, such as from an account to a customer orientation, or from a sales force based organization to a direct mail marketing organization;
4. the capability of the existing system "runs out of steam" - that is, it can no longer process the growing volume of information;
5. the organization's information processing technical staff and management makes a determination to replace the existing hardware and/or software technology base with a newer, more sophisticated, and usually radically different technology base, which is expected to offer increased processing flexibility and capability and which is expected to be easier to change.

Any single reason may be the trigger for such a redesign effort, but more often than not once started, the project takes on a life of its own. The growing integration of systems brings many users under the same systems "umbrella" and thus there are many interested areas of the firm seeking to incorporate as much of their backlog of business changes into the new system as possible. Given the complexity of the issues which affect such a project, it is not surprising that the size of these types of projects mushroom to tremendous proportions.