IBM Redbook Modernization of IBM i Applications

Which generation of users need instructions about how to use a mouse?

For the IBM i community, the need to modernize applications is now an urgent requirement. The world of business applications has become a fast-moving, ever-changing environment. The web, mobile computing, and the cloud have quickly followed the impact of graphical interfaces. The community is facing an ever-changing environment and it must ensure that applications can be easily enhanced and maintained when dealing with these new requirements.

The choices for modernizing an application are either to replace it or to re-engineer it. As to which of these two options is better depends on the application and why the modernization process is being undertaken. No matter which option is the correct solution, there is still a requirement to understand the modernization process and the re-engineering process that is adopted or rejected.

Traditional applications are often large and difficult to modify. Modernizing an application does not mean that the application must be replaced or rewritten in a new language. Modernization means re-engineering an application. This might include changing any combination of the interface, business logic, and database. The goal of modernization is to make it easier to change and maintain your application in the future and provide the correct interface for your users.

The reality is that, over the years, companies already have made a heavy investment in applications that satisfy business needs. Although business requirements might be changing, the current business needs must be satisfied. The challenge is to keep and reuse those core business processes while reengineering, refactoring, and enhancing the rest of the application to meet these demands.

Perception. 5250 is considered old and GUI or mobile is considered new. New applications must be better, right? Often, a new UI can dramatically change the community view.

This chapter describes the driving forces behind modernization and some of the commonalities behind modernization projects. It covers the following topics:

•      What is modernization
•      The need for modernization
•      The benefits of modernization

Tip: The only mistake is to do nothing.

1.1 What is modernization

Modernization means different things to different people. Modernization might mean one or more of the following items:

•      Creating a user interface
•      Refactoring a database
•      Making major changes to code
•      Making minor changes to code
•      Making no changes to code
•      Integrating new applications
•      Any permutation or combination of the preceding items

Modernization can be anything from screen scraping an application to converting it to a tiered or service-oriented architecture (SOA).

Modernization can consist of any permutation and combination of using an independent software vendor (ISV) solution, redesigning a database, redesigning and recoding application programs, coding new programs, using new programming languages, and writing new applications. Modernization is a broad term with many different aspects to consider. This goal of this publication is to provide a holistic view of modernization.

1.2 Approaches to modernization

The developer’s job is much easier if there is one simple roadmap for modernization. Unfortunately, such is not the case.

Many factors determine the wanted approach to modernization:

•      Budget.
•      Management support.
•      Why the modernization process is being undertaken. What permutation of interface, database, and application code must be modernized?
•      Resource. The challenges that are faced by a hundred-developer shop are different from those faced by a ten-developer shop, which are, in turn, different from those faced by a two-developer shop.
•      What is the state of the current application? How easy is it going to be to change that code?

Any one of these items can be broken into multiple bullet points.

Because there are different requirements, different starting points, different levels of support, and different levels of resource, there are many different approaches to modernization.

Although there are many approaches, one of the key challenges is to ensure that the chosen approach is fluid. One must walk the line between control and flexibility and develop the ability to embrace what is new while maintaining a working core.

At the outset, choosing a modernization path seems like a daunting task. If one accepts that the path changes when the process starts, it is much easier to get started.

2      Modernizing IBM i Applications from the Database up to the User Interface and Everything in Between

1.3 The need for modernization

The need for modernization is prompted by a business need; this often means that users prompt the need for modernization. Although you might know that, from a technical point of view, an application should be modernized, it is rare that you have the resources and necessary financing without the requests and support of users and management.

Users are now more technologically savvy. User demand is not just for solutions. Users are also demanding how they want the solutions delivered.

Business demands lead to modernization requirements in three areas:

•      Database
•      Interface
•      Application code

Although there are many variations, the following examples should sound familiar.

1.3.1 The need for database modernization

Here are some common scenarios that can lead to database modernization:

•      The company decides to replace Query/400 with IBM DB2 Web Query.
•      The database must be modernized to make it more accessible to users and to ensure that the use of software such as DB2 Web Query does not adversely affect the performance of the system.
•      Users start to use a new PC analysis tool that extracts information from the database. The tool vendor recommends replicating the data onto another platform because it does not look like a database to them. The database must be refactored to represent it in a format more familiar to a database administrator (DBA).
•      A new application is written over an existing database. The new application uses database features (for example, constraints and triggers) that are not implemented in the existing database. Implementing these features might also affect the existing application.
•      An existing application is rewritten. Modernizing the database reduces the complexity of the programs in the new application.

1.3.2 The need for interface modernization

The need for new or enhanced interfaces is probably the most often quoted reason for modernization. These are some common scenarios that can lead to interface modernization:

•      A company invests in new applications that use a graphical user interface (GUI). Users are switching between GUI and traditional 5250 interfaces for different applications. They prefer a common interface, and a GUI is their preferred interface.
•      There are now generations of users who have never encountered a 5250-style screen before joining the company. To these users, a GUI is the interface of choice and a 5250-style screen is considered old-fashioned.
•      Users who have been using an application for a number of years request that a new GUI interface be put in to place, often because of poor screen design. The users are happy with the application, but not with the way it looks. Often, when users are happy with the 5250-style design, they do not want a new interface until new users who have not experienced the 5250-style design come along.
•      The CEO wants monthly reports as a single graph on their smartphone.

1.3.3 The need for program modernization

Here are some common scenarios that can lead to program modernization:

•      A new web-based interface that replicates an existing 5250-style application is being developed. For example, an order entry application is being created so customers can place their own orders without having to call them in. The new application must use the same business logic as the existing application. Unfortunately, the business logic is firmly embedded in the 5250 screen programs.
•      Because of changes that are being made to the database or the interface, changes must be made to the application programs.
•      An application has four horrendous, gigantic programs that contain the core business logic of the application. Any change (even a minor one) to one of these programs is a massive undertaking that requires a detailed knowledge of the programs and extensive testing. It is time to simplify your maintenance cost.
•      The programmers with the detailed knowledge of the application programs are nearing retirement. It is better if the code is in a format that is easier for a recently trained programmer to understand. It is time to use modern programming styles and tools.
•      Customers must access existing information through a web service as opposed to existing interfaces.

1.4 The benefits of modernization

There are many benefits to be gained from modernization:

•      A better interface
•      A better database
•      Easier to maintain applications
•      More flexible applications. New business requirements are easier to implement.
•      Integrated applications. It is easier to integrate with other applications, platforms, and interfaces.
•      It is easier to find developers who can maintain modern code.
•      A modernized application can give a company a competitive edge. The ability to make rapid changes to the application means that the company can respond quickly to business changes.
•      Continue to use the existing investment. The fact that an application might benefit from an update to its interface does not mean that the application is not working and fulfilling a business need.

1.5 Where are you starting from

As with all journeys, the length of the journey and the challenge of the terrain to be traversed depends on where you start.

The advantage and disadvantage of IBM i is that applications are compatible with earlier versions. It is possible for a program that was written on the System/38 in 1980 to run (with no recompilation) on one of today’s IBM Power Systems. The benefit is that, as hardware was replaced and improved, applications did not need to be upgraded or changed. Unfortunately, the same applications are written using code and coding techniques that might have been leading edge twenty plus years ago but fall short of today’s requirements.

You need to determine the following items:

•      Does the application have a well-structured relational database?
•      Was the application originally written for System/34, System/36, System/38, or one of the many flavors of IBM AS/400, IBM eServer™ systems, IBM iSeries, or IBM i?
•      What language is the application written in? Is it COBOL, COBOL ILE, RPG II, RPG III, RPG IV, Java, C, some other language, or some permutation or combination of two or more languages?
•      Is the application using the IBM Integrated Language Environment® (ILE)?
•      Are the application programs well-structured?

1.6 Laying the ground rules

Starting a modernization project can be a daunting prospect and it is easy to be overwhelmed by the options and amount of information that is available. This is the stage at which most people falter: it is easier to stick with what you know as opposed to embracing what is new.

There is also the challenge of having to maintain the existing application while it is being modernized.

The scope of a modernization project, the importance of different options, and the structure of a project depend on the resources that are available and the goal for the result. There are enormous differences between the choices that are available to a single-developer shop and a hundred-developer shop. Usually, you must be selective about what is modernized and how modernization is achieved.

There might be more benefit from modernizing code as opposed to starting with the database or vice versa. Choices about the usage of new languages depend on the availability of training or programmers with the required skills. The need for a defined change management process varies greatly depending on how many programmers are working on the project. There many terms and conditions that apply.

It can be helpful if there is a single plan that can be applied to all modernization projects, but such is not the case. The process is different for every shop and every application.

Although every modernization project is different, there are many ground rules and circumstances that are common to all of them.

Here are the basic principles to apply to all modernization projects:

•      Do not be afraid to fail.
•      There is a need for education and training.
•      Developers must use the correct tools and they must learn how to use them.
•      There is a need for change management, which is change management in a development environment as opposed to a maintenance environment.
•      Try an unofficial project. Go through the modernization process with just a couple of programs. Perform a proof of concept (PoC).
•      Take small steps. There is much to learn.
•      Put a documentation process in place. Use wikis.
•      Determine the rules and disciplines for the modernization process. At the beginning of the process, the rules and disciplines are guidelines, which evolve into firmer processes.
•      Do not adhere to the same practices you have been using for years. Use agile development principles to deliver smaller projects that build on each other.
•      You need to do it right the first time. Otherwise, it can add to the expense.
•      Consider a “tactical” solution that can build upon your strategic goal.

1.7 What a modern application is

A modern application is one where modifications or enhancements to the application have a minimum effect. For example, changes to the interface should not require changes to the business logic.

The nature of a modern application is that the design of the database and programs is such that the requirement for testing is minimized. This approach allows for rapid development and easier, less complex maintenance.

Here are some of the features to look for in a modern application:

•      A tiered design that separates the application into separate components: interface, business logic, and database.
•      A well-designed and tuned database that implements integrity.
•      Code flexibility, which allows for a flexible means of developing new interfaces and changing business processes.
•      Code reuse, which means that the same procedures are used everywhere. It does not mean that code is copied and pasted to multiple programs.
•      Code maintainability, which means that there is no duplicate code and that the structure of the programs is simpler and easier to understand. Maintenance of these programs has fewer potential side effects.

1.8 Roadblocks

Apart from the obvious requirements for financing and resources, there are many conditions that can inhibit the modernization process. Based on the experience of the team, here are some of the common roadblocks that can hinder, if not derail, the modernization process:

•      Developers do not use the correct tools. Developers are comfortable with what they know.

Sometimes, it is easier to stay within the comfort zone of using familiar tools as opposed to facing the chore of learning how to use new tools.

•      Developers do not know what they do not know. Developers who have no experience with certain technologies or techniques make decisions about how they are used, which is why a PoC is so important.
•      Unlearning old habits. Many developers have been observing the same practices for many years. It can be a challenge to change these habits and embrace new ones.
•      Not getting adequate training. It is not enough to install a tool and say “start using it” or to instruct developers that they should use a new technique when they do not really understand why they should use it or how to use it properly.
•      There is so much to learn that developers feel overwhelmed before they start. Everything must be done in small steps.
•      The whole process of modernization can be frightening. At the outset, there are so many unknowns that developers want to retreat to what they know best.
•      Setting unrealistic targets. Targets are often set without understanding what is required to achieve them.
•      Lack of management support. This is why it is important to start small. The sooner you can demonstrate gains, the easier it is to get management support.

1.9 The journey begins

Whichever of the many modernization paths on which you are about to set out, the journey is full of challenges, achievements, and interesting diversions. It is a journey that constantly offers new horizons.

Tip: Modernization is a journey, not a destination.

Path to modernization

This chapter provides a modernization path that you can follow regardless of the modernization approach that you choose. It also provides guidelines to help define where you should start your modernization path and the definitions of some of the most common modernization approaches that are available.

2.1 Where are you starting

Before you start a modernization effort, it is important to do an honest evaluation of your software assets to define your strategy. Even if your software is working, it might need to be modernized. There are important software quality attributes to consider beyond the functionality, such as maintainability, flexibility, usability, and performance. For this evaluation, consider the following steps:

1.  Build an inventory of your core applications:

–  Business purpose

Define the business purpose of the application from a business perspective. Understand which business processes are supported by the application and how critical each one is.

–  Primary stakeholder

Determine who the stakeholders are that you should consider in an eventual modernization project. This list reminds you how important this project is for different people, not just the IT experts.

–  Technical expert

You must clearly understand who the technical personnel are that maintain the applications. These experts know many small details about the application that can be critical in the project.

2.  Evaluate each application against the following criteria:

–  User interface flexibility

–  Maintainability

–  Performance

–  Ease of integration

–  Available skills

–  Business value

3.  Define your modernization goals:

–  Better maintainability

–  New user interface

–  Improved code readability

–  Better database access methods

4.  Prioritize the inventory according to the goals.

After you complete these steps, you can start working. The modernization activities involve different aspects of the application:

     Database

     User interface

     Integration with other systems

     Business logic

You are just starting, so do not attempt to plan the entire modernization project now. Your modernization plans change over time and are refined along the way. Find a small piece that you can start with. Focus on small pieces that over time can lead to significant things.

2.2 What are your modernization goals

Depending on your modernization goals, there are several approaches that are available. What you must keep in mind is that there is not a “one size fits all” approach to the modernization process. This section introduces some of the most common approaches in application modernization, which are easily adaptable with most modernization goals.

2.2.1 Replacement

In this modernization approach, the original system is replaced by a new system. The new system can be custom-made or a commercial, off-the-shelf product. This technique can be appropriate if the modernization costs are high, if a modernization project is under a different approach, or if the existing application cannot meet the business objectives.

There can be many risks in this approach that you should consider. For example:

     There is no guarantee that the new system contains the same functions or performance as the old system, which might a long history of changes to make it customized for the business needs. This situation probably is going to cause some degradation of the service at some level.

     The IT personnel that are involved in maintenance of the old system need training to learn about the new system. This training is definitely not optional.

     New applications can change your backup and recovery methodologies and must be considered.

2.2.2 Reengineering

Reengineering is the modernization approach that transforms the original application into a new form to improve its original characteristics, such as functionality, performance, flexibility, UI, and maintainability with less risk than replacement.

The main difference between replacement and reengineering is that, in the latter approach, you take into account the existing application as a constraint of the steps to be taken in the modernization process. You cannot ignore the old application. Usually, the reengineering approach is a two-phase process:

     Reverse engineering

     Forward engineering

Reverse engineering

Reverse engineering is the extraction of the high-level processes and structure of the original code into an easily understandable representation. The understanding of existing code that is gained in this phase is vital to the reengineering process. Given the complexity of many of the applications that must be modernized and the lack of existing functional documentation, this process can be difficult. So, there are many tools that can help you in this process.

Here are the key steps that take place in this phase:

1.  Generate a structural representation of the code that helps you easily understand it.

2.  Start mapping the structural representation of the code to business functions, which helps you identify reusable business rules that can be extracted for modularity.

3.  Construct an architectural representation of the application so you can start understanding the application as a whole.

Reverse engineering tasks are repeated several times during the modernization process. Do not worry if you are finding it hard to understand the application. As the modernization process advances, your comprehension of the original system increases.

Forward engineering

Forward engineering, also known as “refinement”, is the creation of the new system from the high-level representation of the original code.

Here are the key steps that take place in this phase:

1.  Construct an architectural representation of the wanted application.

2.  Define the functional transformation from the original to the new application.

3.  Write the new code.

In this process, you should apply preferred practices in terms of design, code, technologies, and tools. Later in this book, you find many tools, techniques, and practices that you should consider while constructing the new application. There are many tools to help with existing program understanding along with tools to help you write code in a modern manner.

2.2.3 Refacing

Refacing is the reengineering of only the user interface of the application. The main objective of this approach is to improve usability and flexibility from the user point of view without having to restructure the underlying code of the application. Refacing is a technology that has been available for a long time. Some solutions are better than others. The one significant advantage of a refacing solution is time to market. Using a refacing technology, you can potentially move a 5250 “green screen” application to a web or mobile application in a matter of several days. This does not mean that you must stop at refacing. This approach can be used as a stepping stone or first step approach. This method can give you valuable time if you are considering a reengineering approach.

Types of refacing

There is more than one way to reface an application. Depending on the degree to which they are going to be applied, some refacing techniques are only superficial and do not require changes in the source code. Other techniques involve minimal changes in the code that handles the UI, but without going into the business logic or database access.

     Screen scraping

This type of refacing focuses on converting traditional 5250 terminal-based applications into window-based interfaces, web interfaces, or mobile based interfaces. Screen scraping usually works by emulating a user on the terminal. While they are connected, they emulate keystrokes, process the display output, extract the data, and generate a new window. The user sees the new UI while the screen scraping does the intermediation. These technologies are targeted at reading the 5250 data stream and modifying what the user sees based on some merged content that can change or control layout, color, and much more. This is a great approach for applications where you no longer have access to the source code.

     Open Access (OA) based UI refacing

For interactive programs, you can also use OA to transform your application. The level in which OA acts is different from a screen scraper. The 5250 datastream is bypassed and the I/O buffer of the RPG program is directly assigned to the OA handler. The I/O buffer can be processed by field name or through a data structure for each format. This offers more flexibility and possibilities over the control and the extensions of your refacing approach. You can find more about OA in 10.2, “IBM Rational Open Access: RPG Edition” on page 467. Using the OA solution enables you to potentially control the UI in a more direct manner from your existing RPG code and pass new data directly from your new UI to your back-end RPG code.

Benefits of refacing

As with every modernization approach, refacing can bring you many benefits in the short term, including the following ones:

     Quick start.

With this approach, you can start and finish modernizing quickly. You do not have to understand the code or change the business logic, which can be helpful when finding executive support for more detailed modernization projects, such as complete refactoring.

     Focus only on the UI.

With refacing, you do not need to understand the underlying code of the application. If you use a screen scraping solution, the original application can be just a black box for you. If you go with OA, you change only the display file specification (a keyword in the F-spec). This can reduce the risk of breaking parts of the application that you do not understand.

     Start moving to other modernization approaches.

Soon you realize that you must start changing other aspects of the application to make it more maintainable. The OA approach opens many possibilities to consider when you are ready to start an integrated modernization effort. Often, OA can be a starting point to a more comprehensive modernization effort.

Show great improvement quickly.

Risks

You have learned the benefits of the refacing approach, but there are many risks that are associated with this approach. You can give a “facelift” to the application and make it look modern to the users. However, the application is still the same inflexible and hard-to-maintain piece of software.

From a performance point of view, the application performance might be affected because it was not originally designed with the new user interface in mind. To perceive real benefits, the UI modernization must be accompanied with a redesign of the underlying code, where you can optimize it accordingly.

Refacing an application does not take into account the design of the UI. The flow and how a user might want to interact with an interface from a mobile or web perspective might be different from how the application is interacted with by using the green screen flow. For example, using a green screen approach might require you to select a menu to see more data, and on the web UI a simple select or drop-down menu might be a much better approach. Understanding the UI flow and interaction requires you to continue down the modernization path.

Refacing is more likely to be a temporary or tactical solution. Remember to clarify this situation with management because from their perspective they might think that the application is modernized and that it does not require extra effort.

2.2.4 Refactoring

Refactoring is a variation of reengineering that restructures a code base, improving its internal structure without adverse effects to its external functionality. Refactoring takes a “do no harm” approach, meaning among other things that applications interacting with the modernized system do not have to be changed at all.

The “do no harm” philosophy is ensured through extensive testing during the modernization process. Tests should be automated as much as possible. Ensure that the tests cover all aspects of the application.

Without a full set of tests, the risks of the refactoring increase because you cannot ensure that the new system has the exact external behavior as the original system and that you are not introducing bugs to the system.

When to use the refactoring approach

Refactoring is the preferred approach when your modernization goals include the following items:

•      Reducing maintenance costs
•      Increasing the clarity of the code
•      Revitalizing the application and enabling it to adapt better to business needs
•      Reducing the risk of affecting others by modernizing an application
•      Keeping the application in the same platform and language as the original
•      Using the existing skills of your current programming force
•      Allowing user interface flexibility (It is easier to update the application business logic to be accessed by either different or multiple UI technologies.)

Some refactoring is often necessary regardless of the modernization approach that you choose. This book describes a modernization model that guides the modernization project. Some of the activities that are described in this model are more related to the refactoring approach. You must evaluate whether your project needs these activities.

2.3 How to modernize

This section provides general principles that must be considered in any modernization effort, regardless of the approach that you choose. It also provides a model to guide you through a modernization project. Following a model and a set of predefined rules helps you to achieve a mature modernization process. You should adapt this model to meet your needs.

2.3.1 Introduction

Modernization is a discovery process and you must learn a lot. But even so, it is important to follow a model that can guide your way. There will be more than just one modernization project ahead, so prepare to record your journey so others can repeat your success and avoid your mistakes.

This section provides you with a model that can help you understand what you must do to modernize an application. There are some general principles that are applicable not only to this model but to any custom process that you define.

Modernization is not constrained to specific languages or platforms. There are thousands of software applications that must be modernized, and you are not the only one who has needed to undertake a modernization project. These principles and this model are applicable no matter what your situation.

The model that you are about to learn is not “carved in stone”. You should take the parts that are applicable to your modernization approach and your specific situation and adapt them.

There is plenty of room for improvement and creativity. Adopt the model and improve it any way that you need. Make sure that others can repeat it and improve it themselves so that you have a mature process that can reduce risks and uncertainty.

2.3.2 Modernization model

This section presents the modernization model that is proposed in this book. You are introduced to the principles that guide the modernization model and a prescriptive set of phases that you can follow to achieve your modernization goals.

A model or a methodology

Before you learn about the modernization model, it is necessary to have a clear understanding of what is a model and what is a methodology. Sometimes these terms are mistakenly used as synonyms. A model defines an approach and is a scheme to do things. A model defines what you need to do, not how to do it. On the other side, a methodology defines every activity that must be done to achieve every step of the model. A methodology can be thought of as the implementation of a model.

This book defines a modernization model instead of a methodology. Modernization can be approached in different ways and it might be hard to track every new technique, but a model can be time-proof. It is your responsibility to adapt this model to your specific situation, keeping in mind the principles, guidelines, and phases that every modernization effort should have.

The model

Modernizing applications is more than just following steps and performing some activities. It implies a set of principles that you must remember. The modernization model that is proposed defines this set of principles and also a set of phases to achieve your modernization goals.

Chapter 2. Path to modernization      15
The modernization model is shown in Figure 2-1. As you can see, the model defines an iterative modernization cycle that is repeated as many times as you need. Every iteration contributes to the main objective: build a modern application.

After every iteration, you should stop and reassess the situation. You can change your modernization approach or any specific technology that you selected. The key point is that you can see results early and you can stop as soon as your modernization goals are achieved.

In the following section, you see the definition of the phases that compose the model:

     Phase 0: Prepare

This phase deals with the planning that you need for the iteration. You must select the part of the application that you are going to modernize and define how you are going to measure your progress. Every iteration must have planning activities so you can effectively adapt to the immediate situation.

     Phase 1: Experimentation

You must try new things in each iteration. You must understand the technologies that are available in the market and determine whether they fit your situation. This phase is about experimenting.

     Phase 2: Off and running

This phase is about doing the real work and applying what you experiment with to achieve the goals of the iteration.

     Phase 3: Evaluate and expand, future thinking

This phase involves analyzing the work that is done so far, including all previous iterations. In this phase, you can see how to introduce new technologies to enable greater flexibility in the modernized application. This phase is dedicated to analyzing how to expand the current functions of the application.

     Phase 4: Deployment

You must deliver the software as soon as possible. Every cycle ends with working software. You must carefully select the deployment strategy to reduce risks.

2.3.3 General principles

This section provides some vital principles that you should follow during the modernization project. The application of these principles can help you with the modernization project, especially when you must face specific unexpected situations and make important decisions.

Iterative and incremental process

The modernization process can be thought of as an iterative and incremental process. It is iterative because the whole project can be separated into smaller projects that are easier to manage and faster to finish. It is incremental because with each finished iteration, you make the application more modern, one component at a time. This is also called an agile approach.

Sometimes, people that are involved in a modernization project try to follow a “big-bang” approach and end up being overwhelmed or frustrated. So, with each modernization project, remember to divide and conquer.

Keep researching and reading

Things change. What is cutting edge technology today can be obsolete in a matter of a few months. To keep yourself modern, make it a habit to read and research modernization theory and technologies. Read about the latest tendencies and try them. Go to seminars and conferences. Talk to your colleagues and look for successful experiences (or failures) about applying modernization approaches or technologies.

Modernization is not something that only you are dealing with. It affects many companies, platforms, and programming languages. Find time to experiment with technologies. Create proof of concepts to both learn and to prove the technology in your environment.

Automating what you can

Occasionally in a modernization project, you find yourself doing the same task over and over again. Think about how you can automate these steps. There are always better ways to do things. Look for commercial modernization tools or create them yourself.

Testing, testing, testing

Tests are the only way to ensure that you are not breaking anything that was not already broken. There are plenty of different testing approaches that you can use in your project. Here are two of the most common ones:

     Regression tests

This type of test focuses on comparing the results of an application against a

pre-established base line, in this case, the original application. One key aspect of this type of test is that you do not need to understand the system before testing it. This can help when you are starting your modernization project and your comprehension of the code is not complete.

     Automatic unit tests

With unit tests, you can test a small part of the application. Unlike regression tests, unit tests require more understanding of the code that you are testing. So, start creating automatic unit tests as soon as possible. Unit tests must be designed in a self-configured way, meaning that you should not need to configure anything before running the tests. Also, the units must be able to run fast. You might need to run them after every compilation.

All of these approaches can complement each other, so figure out a way to use them in your modernization project and automate them. Remember, you should be able to run your tests fast and often; it is the only way to feel more confident while changing your code.

Do not overanalyze

Modernization is about experimentation. Sometimes fear can derail your efforts, so do not be afraid and do not overanalyze things. Start working as soon as you can.

The old code that you are modernizing causes surprises every occasionally, no matter how much you analyze it. Remember the iterative and incremental nature of the process and plan and analyze in the same way.

Being agile

Modernization projects must be agile, so focus on avoiding activities and documentation that do not add value to the process. Keep in mind the following agile principles:

     Deliver working software frequently.

Plan your iterations in a way that you can deliver working software fast and frequently. Your executives must believe in the value behind a modernization project, so do your best to deliver value to them in early stages of the process. A true agile project is broken down into two-week iterations. You are continually and regularly delivering progress. However, do not get hung up on breaking things into the smallest units possible. The purpose is to be flexible and deliver small incremental updates.

     Focus on the software over documentation.

Avoid any documentation that does not add value to the modernization process. Look for dynamic documentation that can be easily maintained and reused for different purposes. There are many content management systems that can help you create collaborative and dynamic documentation. Using a tool such as a wiki can be useful because it allows the user community to contribute to the documentation effort.

     Simplicity is essential

Keep things as simple as possible. Resist the temptation to add features that might be used in the future but are not needed immediately. Unnecessary complexity is a source of bugs that can damage your process.

     Give continuous attention to technical excellence.

Do not forget that modernization is about improving. Focus on applying better practices and technologies. Become an advocate of excellence. Promote preferred practices and transmit your knowledge. You must help others to understand the importance of doing things correctly.

     Business people and developers must work together.

Build good communication lines between developers and business experts. The application that you are modernizing is what supports the business. Do not underestimate the contributions that non-IT personnel can make to the project.

Always use modern tools

Stop using obsolete tools. Modern tools help you do things faster and better than before. Modern tools are valuable resources that can help accomplish amazing tasks. Do not let your current tools stop your modernization journey.

Naturally, you are more productive with tools with which you are familiar. Do not mistake this advantage of your familiarity as proof of the superiority of the tools. Changing tools require a period of adjustment before their productivity gains are realized. With time (and often it is not much time), modern tools allow you to be even more productive than before. It is vital to attract new talent to the organization, and new developers feel comfortable with tools that resemble the tools that they use. “Green screen” development tools can give the impression that you are using obsolete technology, so avoid using them. Many developers see significant productivity improvements, normally 20 – 50%, depending on each developer, by switching from a green-screen based development environment to a modern integrated development environment (IDE).

2.3.4 Phase 0: Prepare

Applications are like living beings. As they grow, processes, people, and other systems start to build around them. There is more than code in a modernization project and you must prepare accordingly.

The main objective of this phase is to prepare you for the modernization effort ahead of you. You need a clear, panoramic view of what the application means to the different stakeholders who are involved in the project.

Building the business case

Before you begin the project, you should have the application inventory prioritized. Now, you must create the business case. A business case can determine whether a project is

cost-effective. The goal is to convert your needs into something that your manager can understand in their terms and make a business-based decision.

A business case contains the following parts:

     Problem statement

Ensure that this part describes the inadequacies, inefficiencies, and weaknesses of the current application. Put together numbers and figures that explain the situation to the decision makers and help them see the importance of the modernization project.

     Solution

This is the high-level description of the solution for the problem. You should include the following items:

–  Current state-of-the-art technologies.

–  An overview of the proposed solution.

–  A preliminary cost and schedule proposal.

You can also include a brief description of the modernization model that is going to guide the project.

     Risks

You should do an honest assessment of the risk that you might have to face. This assessment helps you prepare yourself before any unexpected situations happen. Also, a good risk analysis helps you gain credibility by enumerating the things that might affect the scope of the project.

     Benefits

Ensure that you specify the benefits of the modernization project in a quantifiable way. It is recommended to include several scenarios:

–  Pessimistic scenario: In this scenario, you should specify what are the minimum benefits of the modernization project. Ensure that you indicate what risks are more likely to happen and how you are planning to overcome these situations.

–  Optimistic scenario: In this scenario, you can specify what are the maximum benefits of the modernization project. Be sure to set realistic quantifiable benefits. Avoid exaggerating, which can affect your credibility if something goes wrong.

     Metrics

You must define how you are going to measure your progress. Each iteration should be evaluated against these metrics. This is not only something that is important to your manager; it also helps you adjust your plan.

     Stakeholders

Identify all of the stakeholders of the project. Make sure that you include all the relevant people, from developers to business users. Having input from all the affected stakeholders ensures that you remember the importance of the application to your business.

     Examples

Here is where proof of concept comes into play. If you can show a small, successful example of what you are attempting to accomplish, it helps boost the business case.

The business case is essential for the modernization project. Without it, it is doubtful that you can obtain the funding that is required and keep it throughout the project’s life. It is vital that the business case is complete. Otherwise, your modernization effort can be hampered even before you change a single line of code.

Building the team

Modernization projects always involve several people. It is vital that you define who is going to be part of your team at an early stage. Besides the developers that do the actual work, the modernization team is also composed of non-IT personnel (for example, the users). Do not underestimate their contributions or commitment to the project.

Make sure that you define the following roles at a minimum:

     Modernization team lead

This person is in charge of directing the team. Make sure that the person who is going to be assigned to this role is an advocate of modernization. It must be a pragmatic and open-minded person who is not afraid of change and passionate about continuous improvement.

     Modernization engineer

The modernization engineers are in charge of applying the modernization approach to the application. Make sure to choose the correct people for this role and keep them focused on the main objectives. During the modernization process, it is natural to want to fix everything immediately. Engineers must remember that modernization is an iterative process.

     Business logic expert

The business logic expert usually is a non-IT person. They can be final users or someone whose business process is supported by the application. This role is essential for the project. They can give you guidance about understanding the reasons behind specific functionalities of the application.

     Application maintenance personal

This is the person who has experience maintaining the application and modifying it according to the new requirements. This role can explain most of the design and coding decisions.

Do your best to maintain the team’s cohesion and promote team work. Sometimes, the application maintenance personnel feel threatened by the modernization project. Make sure that you convince them of the importance and value of the project.

Developing the plan

Do not try to anticipate every little detail. Things most likely will change, so define a high-level roadmap of the project. Then, do a detailed plan for only one or two iterations. Use the roadmap as you perform your plan to keep your project on track.

This plan should define your immediate objectives for the iteration and the tasks that are needed to accomplish them. Make sure that your plan includes a deliverable at the end of the iteration.

Developing standards and processes

At first, you experiment frequently with the modernization process, but remember to document what does and does not work for you. During this process, you must define standards to ensure that the modernization team does things in a predictable way. As soon as possible, define a set of rules and standards that aim for quality. Be prepared to modify your standards as you and your team find improved techniques.

Always avoid reinventing the wheel

If there is a good standard in the industry, analyze it. If it fits your needs, adopt it. You are not the only one trying to modernize applications. Many things that you need already are researched and implemented by someone else.

2.3.5 Phase 1: Experimentation

This phase is a good point to try new technologies. Given that this is an experimentation phase, you perform many tests. Most of the things that you are going to code are not going to be in the final product.

The following sections describe some of the activities that you should consider in this phase.

Doing a proof of concept

You must know which technologies are best for your project. The preferred way to determine this goal is by creating a proof of concept (PoC). You might find obstacles and need to do some rollbacks, but the PoC helps you and your teammates gain experience.

If you are planning to select a commercial solution, it is likely that you are going to need your manager’s approval. Do not rely solely on marketing figures and charts. Show them real working software. Take advantage of no-cost trials.

Remember to document what you learn in this phase. Make sure that you involve your stakeholder with the PoC. They can provide valuable insight to help make sure that you are going in the correct direction.

Looking for simpler projects

While you are learning and getting used to the modernization process, avoid selecting the most complex application. There should be applications that are important for the business and do not add unnecessary risks to the process.

With every success, gain more skills and confidence that enable you to modernize more critical and complex applications.

Experimenting

You are changing code, changing the UI, and changing the architecture. This is just testing and you are not going to deploy it to the production environment. The ideas and experiences that you gain in this activity help you work better on the new application and to understand the underlying code.

Reassessing

This whole experimentation process is going to broaden your thinking. Many ideas that you had are going to change. Do not be afraid to reassess your modernization approach if you think that there is a better way to do it.

Defining the measurement criteria

In this phase, you gain more knowledge about the application. Define the mechanisms that you are going to use to measure your progress. You are going to report regularly to your manager, and they need to see your progress. So, define your metrics according to the selected application and modernization approach.

2.3.6 Phase 2: Off and running

This phase focuses on doing real work. Now that you have a panoramic view of the available technologies and techniques, start modernizing the application.

Keep in mind the iterative and incremental nature of the modernization process, so start step-by-step in a way that makes sense for your situation. Divide the application into small units that you can modernize one at a time, but do not lose the whole view of the application.

The following sections describe some of the activities that you should consider in this phase.

Building your sandbox

Many modernization projects run in parallel with other business projects, so it is important that you build a sandbox that is specific for your project, where you can be sure that you are not affecting someone else’s work and that no one else is affecting your work.

When you design and build your sandbox, remember to consider the following elements:

     Objects

     Data

     Tests

Objects

Copy all of the application objects to your sandbox, including, at the least, data areas, programs, service programs, user spaces, data queues, and any other object that your application is going to need to run in an isolated environment. To help you build the sandbox, create the application structure diagrams, which can be generated with automated tools.

Data

In addition to the executable objects, make sure to have all of the tables that interact with your application. Isolate the data to allow changes to be made without affecting other projects.

You should develop a method for restoring your sandbox data to its initial state in an automated fashion, which helps you run repeated test scenarios and automate testing. Always journal your sandbox data. Journals are indispensable when debugging your changes.

Tests

You need a complete set of tests to ensure that your modernization work is going in the planned direction. Your sandbox should contain a test set that you can run multiple times and easily. Without tests, any change can be unrecoverable.

In the first iterations, you must have a set of regression tests. As the project advances, your understanding of the application increases, enabling you to write specific unit tests.

Make sure that the test set covers all parts of the application that you are changing to ensure the “do no harm” approach.

Consider the following techniques for code understanding and apply them to your situation accordingly:

     Compilation of existing documentation

It is likely that there is no documentation of the original application. Do your best to compile every piece of documentation that is available. Some useful documentation might include the following items:

–  Deployment and configuration notes

–  Incident history

–  Test cases

     Code reading sessions

The most basic technique to understand the code is to read the code. You must read the code that you are working on with your team, and look for repeated code and logic. Initially, reading the code does not appear to be helpful, but after you become familiar with it, things become clearer.

Ask the experts about the business functions that are implemented in the code and keep in mind those functions while reading the code. Sometimes it is easier to understand the underlying business process than the code that implements it.

     Debugging sessions

Debugging the application with specific tests can help you understand the application and how every module works. The usage of modern tools, such as IBM Rational Developer for i, makes the debugging sessions more productive.

     Static analysis

This is the analysis of the source code or the compiled object without running it. Usually, the static analysis has the following components:

–  Usage of code analysis tools

There are several tool providers that have specialized tools to review and help you analyze your monolithic programs. The Rational ARCAD Power Pack provides the Observer tool, which can help analyze code.

–  Extraction of business rules

Business rules refer to the business logic that is embedded in the code. Many tools help you extract this type of data from the code, which can help you understand the functionality that is implemented in the program.

–  Dependency analysis between components

Refers to the identification of relationships between the modules of the application. The diagram that is generated can help you understand how one component integrates with the whole application.

–  Internal structure diagram

Most legacy applications are large, monolithic programs. To easily understand the flow inside these programs, some tools can generate call diagrams between internal subroutines and subprocedures, such as the ARCAD Observer tool.

–  Complexity and maintainability metrics

Complexity and maintainability metrics can help you diagnose the section of code on which you are working and measure your progress. They can also help you estimate the amount of work that is needed to modernize.

These types of analysis are most easily done through the usage of automated tools.

     Dynamic analysis

This type of analysis is performed by tracing the running of the programs in a specific environment. There are many uses of the results of the dynamic analysis:

–  Code coverage

Traditional applications usually contain “dead code”. This is code that is not run but is still in the program. To identify code that is no longer needed, you can use a tool that helps you calculate how much code is covered with a set of transactions and how much code is possibly dead.

–  Dynamic call diagrams

The preferred way to understand how the program works is by monitoring its behavior while running. Dynamic analysis tools usually provide data that depicts the execution flow of a program, subroutines, and subprocedures.

To be effective, the dynamic analysis requires a good set of tests to produce an interesting result.

Modernizing

When your sandbox is prepared and you have a better understanding of the application, you can start doing modernization work. Here are some of the common tasks that you can do in this stage:

     Cleaning the code

Cleaning code is the removal of elements that make the code hard to understand. Here are some of the things that must be considered when you clean your code:

–  Removal of duplicated code.

–  Conversion to a more readable form. In the context of RPG, this means converting it to free format.

–  Removal of unused variables and procedures.

     Restructuring the code

After you clean the code, you can extract related business logic of service programs that can be reused in different parts of the application. There are several good tools that are available to help you identify and restructure your code. Here are some tasks that you can do in this step:

–  Extraction of similar logic into reusable components.

–  Encapsulation of global variables.

–  Conversion of subroutines to procedures.

–  Renaming of variables and subprocedures to more meaningful names.

–  Writing self-documenting code.

     Repackaging

Every module must contain only functionally related subprocedures and every service program must contain only functionally related modules. You must separate monolithic modules and organize the procedures into new modules. Every module must have one responsibility.

     Modernizing the database

Data base modernization is a complete modernization model by itself. Later in this book (Chapter 9, “Database re-engineering” on page 351), you find a methodology that guides you through the process for this modernization.

Testing

After you modernize the application, make sure that all of your tests are successful. If your tests are not successful, you should not continue with the modernization process. You must ensure that everything is working as before and that you are not breaking existing functions.

2.3.7 Phase 3: Evaluate and expand, future thinking

The main objective of this phase is to maximize some of the benefits of the current state of the art technologies that are applied to your application.

You have prepared your environment and you have modernized the application. Now, it is time to evaluate the progress of your work. If this evaluation indicates that you should step back in the process, do not worry; you might have to do this step multiple times.

If the evaluation shows that you have accomplished your goals, it is time to start expanding the value of the application and prepare it for the future.

The following sections describe some of the activities that you should consider in this phase.

Measuring

You must determine whether you are achieving the modernization objectives. If your main goal is to reduce complexity, this step must reflect a reduction of the complexity. If you must improve the user experience, you should measure it in some way. The key point is that you keep control of the objectives and the progress to achieve these goals.

Your measurement should indicate adjustments that should be made to the project.

Improving the architecture

Now, you can consider optimizing the application architecture. You might want to start separating the application in layers or tiers and to include more components, such as a new UI replacement layer. This step focuses on the application as a whole.

Integrating

One of the common modernization goals is to integrate the application in to an enterprise architecture. Often, it is necessary to enable the application for SOA or to make it more flexible for usage with other enterprise integration technologies.

Bringing in new technologies

If there is any new technology that can help you improve the application, you can include it in this step. Carefully analyze it to keep it from becoming a future obstacle.

Always aim for standard tools, technologies, and well-tested solutions. Evaluate the user community around them and the support of the official application maintenance personnel.

These new technologies can become obsolete quickly, so wrap them in a way that helps you replace them easily if necessary.

2.3.8 Phase 4: Deployment

Now, it is time to deliver the results of your work. However, the components that you deliver should not be considered a final product. With each iteration, you can improve pieces of software even if you already delivered them. The key point here is that you do your best to start deploying as soon as you can to show the value of your modernization effort.

The following sections describe some of the activities that you should consider in this phase.

Documenting the application

It is likely that the new application is different from the old one. You must create effective documentation that explains the new application to the people that do maintenance and the users and stakeholders of your applications.

Take advantage of any available resource that can help you document. Use class, sequence, and deployment diagrams to help people understand the application. Most types of Unified Modeling Language (UML) diagrams are designed for object-oriented languages, but you can adapt them easily to your situation. The advantage of using a standard type of diagram is that many people can understand them easily, which makes you seem more professional.

Your documentation must add value. So, avoid documentation that goes directly to the shelf. The usage of content management systems (for example, MediaWiki, WordPress, and Joomla) can help you create simple and collaborative documentation.

The documentation must be easily maintained. There are many tools that can generate documentation from the code or from the compiled objects. Use them (or create them) if possible.

Training the people

Depending on the modernization approach, you might have to train the users, the application maintenance personnel, or both.

The documentation that you previously built should be used as the primary reference for the trainees. Make sure that this documentation can be accessed easily and that it contains all the necessary information for the users and for the IT experts.

If the new application is more difficult to understand than the original, this is a good indication that the modernization plan needs improvement. Evaluate the new system from different points of view.

Defining your deployment strategy

Just as there are many modernization approaches, there is more than one way to deploy the new application. Make sure that both approaches are aligned. Here are some of the most common deployment techniques:

     Big-bang approach

This approach for deployment replaces the entire application at one time. This approach is used with modernization projects that must solve an immediate problem affecting critical services. However, the associated risk is higher and must be carefully managed.

When you follow this type of approach, it is necessary to maintain the old application and the new application in parallel. The issue is that changes in the new system should be reflected in the old one and vice versa. This situation can lead to much work and effort.

     Incremental approach

Also known as “Phased-out”, in this approach, sections of the application are

re-engineered and added incrementally, which allows you to deliver results and identify errors more quickly.

When you follow this approach, you can change only one component at a time without considering the application as a whole. Thus, the modernization process might take longer to finish.

     Evolutionary approach

Similar to the incremental approach, the evolutionary approach replaces sections of the original application with modernized parts. The difference is that in this approach, the sections to replace are selected based on their functions and not the structure of the application. This situation means that the developers put forth extra effort to identify functional parts that are separated in the original application. To help you understand the differences in these approaches, consider Table 2-1, which contains a brief comparison of the three approaches.

2.3.9 Repeat as necessary

After you finish the process, you must go back and repeat the process as many times as needed. In every iteration, a new component is modernized and you are closer to your goals.

Not all the phases require the same amount of effort in every iteration. At the beginning, you probably must invest more effort in the preparation and experimentation phases. However, as the project advances, these phases are shorter. You see how the phases adapt during the successive iterations.

Evaluate your progress in every iteration and adjust your path accordingly. While going through all the phases, adhere to the principles of modernization and adjust the process to your situation.