D a t abo rou gh

Analysis and Documentation

with X-Analysis

» Analyze Application Graphically

» Automatic System Documentation

Richard Downey
Stuart Milligan

W HITE PAPER
Preface

Developing tools services for analyzing and re-engineering RPG

applications for the last 20 years, has given Databorough a
unique view of the very large and complex world of legacy
applications running on System i, iSeries and AS/400. In 2005,
IBM and Databorough published an IBM Red book “Modernizing
and Improving the Maintainability of RPG Applications Using X-
A n a l y s i s Ve r s i o n 5 . 6 ” . T h i s p a p e r e x p a n d s o n t h e r e c e n t l y
released white paper 'Modernizing RPG/COBOL System i
applications using X-Analysis 8' incorporating new concepts and
methods for application analysis and documentation of a RPG
applications. Contact info@databorough.com for a copy of the
wh i t e p a p e r, t h e R e d b o o k a n d t r i a l s o f t wa r e .
Table of Contents
Executive Summary...................................................................................................................................1
Introduction................................................................................................................................................2
Why Design Recovery is difficult..............................................................................................................3
Analysis, Documentation & Application Subdivision...............................................................................5
Understanding Design & Function More Easily...................................................................................5
Producing Static Documentation Automatically...................................................................................9
Summary..................................................................................................................................................12
 Analysis & Documentation with X-Analysis

Executive Summar y

The knowl ed ge a nd in for mati on cont ai ned in an orga nizati on's busi ness soft war e is
vitall y imp orta nt and very val ua ble but oft en this i nfor mat io n coverin g the
operati on, metrics, an d d esign of the software is tan tal izing l y out of reach. W ithout
this knowle dg e, ma int ena nce an d chan ges to the syste m are n ot as efficie nt or
eff ective as th e y cou ld be , an d the risk of fail ure or pr ob lems increases
exponent ia ll y the lar ger t he en ha ncement re quir ed. This could l ea d to a par al ysis
where chan ges can't be ma de du e to a lack of conf id ence in t he outco me.

Accurat e a nd current inf ormati on abo ut an entir e syste m can gre atl y i mprove the
productivit y of you r IT staff, an d re duce mai nte na nce costs b y e li min ati ng the need
to research, cat alo g an d asse mbl e the infor ma tio n ma nu all y for each service
request, or mo der nizati on pro ject.

Existi ng Syst em i 1 ap plicat io ns have some fairl y consiste nt and disti nct
characteristics th at mark th em as costl y a nd pot ent ial l y hi gh-risk:

1. Appl icati ons te nd to be larg e a nd compl ex

2. Litt le or n o d ocument atio n
3. Ori ginal Design ers no lo nger avai lab le
4. They h ave b ee n d evelo ped over man y years
5. Monol ith ic Progr ammin g Mod el
6. Writt en in obsol ete lan gu ag es

Points 1 thr ou gh 3 can l arge l y b e ma na ged more effective l y b y investin g in a

product like X-Ana l ysis to bot h recover the desi gn of the app licat ion , a nd provide
highl y pro ductive an al ys is to oli ng to compe nsate f or the comple xit y of the
applicat ion .

Inconsiste nt pr ogr ammin g stan dar ds an d d esigns, sig nif icant amoun ts of re du ndant
and dupl icate d code , a nd an increasi ngl y costl y de ma nd for glo bal l y di min ishing
legacy devel opment skil ls, are the results of p oin ts 4, 5 an d 6.

This paper wil l ill ustrate h ow the X-An al ysis carries out ana l ysis and aut omatic
documentati on wh ich del ivers dee p insigh ts of yo ur s ystems.

1 Note - for consistency throughout this document we use System i to refer to the family of computers
that grew out of IBM’s System/ 38 over the last twenty years namely the AS/ 400 , iSeries and latterly the
S y s t e m i a n d I B M i o n P o w e r. W e h a v e p i c k e d S y s t e m i a s i t i s t h e c u r r e n t n a m e u s e d b y I B M t h o u g h
most readers (and the author) will likely use the earlier terms.

1
 Analysis & Documentation with X-Analysis

5 Introduction

As we have see n gat heri ng kno wl ed ge a bou t Syst em i a pp licati ons is not a
straightfor wa rd task for to da y's ge nera tio n of busi ness an al ysts a nd devel opers. To
ill ustrate that p oin t an d to ful l y u nd erstand the pr obl em d omain we wi ll lo ok at Why
Design Recov er y is diffi cult 2 b y worki ng thr oug h th e prob le ms th at X-An al ysis
solves in bui ldi ng its rep ositor y of desi gn recovery infor ma tio n.

In situatio ns whe re deve lo pers are no t famili ar wi th a s yste m or its d ocument ati on
is inadequ ate, the s ystem's source code beco mes th e o nl y rel ia ble source of
inf ormation. Unfort una tel y, source cod e h as much mor e d etai l t han is ne ed ed just
to understan d th e s yste m, a lso it disperses or obscures h ig h-leve l constructs that
woul d ease th e syst em's un derstan di ng. X-An al ysis ai ds s ystem und erstan ding by
identi f yi ng recurrin g progr am fea tures, classif yi ng th e s yste m modu les b ased on
their purpose an d usa ge patt erns, and an al yzi ng de pe nde ncies across the mo dul es.

This anal ys is provi des de tai le d d esign inf ormati on abo ut the ent ire system,
accessibl e to no n RPG/COBOL experts, and be easi l y up dat ed to i ncorpor ate
ongoi ng chang es in the base s ystem.

W hatever th e busin ess n eeds driving comp an ies to mo der nize their ap plica tions,
most want to e nsure that the busi ness log ic an d fu nctio nal desi gn are preserved to
varying degr ees as th ese ar e core assets.

Design Recovery of a n a ppl icati on can b e br oken down int o a few l ogica l steps or
stages tha t represe nt a g ener ic a da pta ble a ppr oach to a n y a ppl icati on
modernizati on proj ect:

An al ysis, Docume ntatio n, Ap p lic ation Subd ivi sion – This t yp e of anal ysis
repr esents the most commo n use of th e X-Ana l ysis tool across the worl d. On top of
very powerf ul cross-re fere ncing fu nction al it y, gr aph ical, narr atives or a combinati on
of both, ar e use d to abstract and describ e th e s yste m in a simpl e a nd int uitive way,
even for n on- RPG/COBOL experts. Th e leg acy ap plicat io n can be completel y
documented usin g mod ern dia gra mmi ng stand ards such as UML, Entit y
Relat ionship Diagr ams, Syst em Flo w Diagr ams, and Structur e Char ts etc.
Furthermore, th e le gac y syst em can b e a uto mat icall y sub divid ed into ap pl ication
areas so that effective s yste m overview & i nterf ace di agra ms can be ge ner ated.
The compl ete a ppl icati on docu me ntat io n can the n be o utp ut to a vari et y of third
part y desi gn to ols such as Ratio na l, MS Visio, MS Word, etc. – ind eed an y tool
capable of imp orti ng XML or DDL is supp orte d.

2 If your colleagues or the project sponsors don’t appreciate the difficulty of modernization get them to
read this section.

2
 Analysis & Documentation with X-Analysis

Reco vering a n Ap pl ic ation de sign – [T his is b e yo nd t he scope of th is d ocument

so we won ’t go into det ail h ere.] This advance d level of an al ysis extracts model
inf ormation from the existin g a ppl icatio n. X-Anal ys is uses its o wn an al ysis
repositor y, p lus p atter n search ing al gori th ms, to derive rel atio na l d ata models,
extr act busin ess rules, b uil d UML Activit y/Use/Case Diagr ams, a nd lo gical screen
flows. Onl y re levan t desig ns n ee d be use d as a b ase specificati on for new
devel opers to rewr ite th e ap plica tio n. Th e structur ed, reposit or y- based format of
these extr acted desi gns, make it possibl e, to pr ogr ammat icall y re use th em for
rebuil ding the core of a ne w a ppl icati on. Th is can b e do ne wit h purp ose-b uilt tools,
wi th X-Modern ize or a combi nati on of bot h.

Redevelop ing Using a Recov ered Ap pli cati on Desig n – [T his is be yon d the
scope of this docu me nt so we won ’t g o int o det ail here, if you req uire more
inf ormation pl ease cont act Dat ab orou gh f or the mod ernizat io n wh ite pap er.] This
star ts wi th dat abase mo der nizati on using th e recovere d data mode l. The designs
for the view, contr oll er, an d b usiness rul e lo gic ar e a lso e xtracte d a nd reused in
modern frameworks such as Hib erna te, a nd wit h ne w JSF/Facel ets a nd Java bean
components. This opt io n makes it possibl e to pr ogra mmatica ll y re-factor the
existing ap plica tio n i nto mo der n, consu mab le assets and arti facts for devel opers to
use for a s ystem re bui ld. The o bjective is to pr oduce cle an, wel l structured,
industr y sta nd ard code rather th an messy s ynt ax conversions wit h u n-main tai nable
code.

Why Design Recovery is difficult

From the p oint of view of the user of X-An al ysis t his process of bui ldi ng the cross-
refer ence reposit or y an d d erivin g t he mod els h ap pens auto ma ticall y! It's just t here
and happe ns t ypica ll y as par t of th e instal lat ion process - tho ugh it can be
triggered aga in lat er o n if req uire d. However it is wo rth takin g some ti me to
understand this process and t o see wh at h app ens, ho w th e mo de l is constructed
and the re lat ionsh ips in ferre d.

If you thi nk for a mi nut e of a t ypical Syst em i a pp licati on it is l ikel y to consist of a

mi x of RPG progr ams, DDS files a nd me mb ers for displa y files, dat abase fil es and
logical vie ws, newer s ystems ma y h ave these in tersperse d wi th SQL scrip ts b ut the
sum of kno wl edg e in th at s ystem, how it works an d int eracts amo ngst its various
elements is cont ain ed wi thi n t hose source fil es an d compil ed obj ects - the issue is
retri eving tha t kno wl ed ge effici entl y.

To understa nd and ful l y ap precia te the pro ble ms X-Ana l ysis solves just consi der
the process you wou ld have to und ertake you rself if yo u want ed to discover how a
system op erat es or make chan ges to it. As a simpl e example for part of your
applicat ion yo u have a customer d etai ls scree n wit h n o d edicat ed pl ace for an
email address an d mobi le ph one nu mb ers, th e s ystem has a da pte d itse lf to the
int ernet ag e as man y Syst em i apps have d on e b y making use of .extra. a nd .n otes.
ad- hoc fi elds. The syste m has cope d b ut it h as be en time consumin g to retri eve
these deta ils wh en re quir ed f or market ing purp oses. Bu t ther e is n ow a bud get to
correct this and start to lo ok at mo dern izing th e a ppl icatio n and maki ng the
functi onalit y ava ila bl e to mor e areas o f th e b usiness.

3
 Analysis & Documentation with X-Analysis

You woul d prob ab l y first start b y looki ng at th e pro gra m an d displ a y fi les that
handl e the displa y and ma int enance of th e custo mer infor ma tio n, from that you
woul d discover the dat ab ase ta bles/f iles i nvolved.

At this poi nt from a simplistic poi nt o f vie w yo u have the necessar y inf ormati on to
make the chan ges a nd the y ar e pr oba bl y n ot that difficult - ad d new fi el ds for email
and mobi le ph one to the dat ab ase ta bl es or re na me the existing on es the n modif y
the progra m a nd displ a y fil es accord in gl y¡¦ b ut yo u'r e prob abl y thinki ng wh at about
the rest of the s ystem? W hat else uses th at tab le? Is the disp la y fil e used
anywher e e lse? So th e cha nge has more aspects th an wo ul d first app ear these are
just a few of th e q uesti ons we have to a nswer :

• Scope and imp act of the chan ge - ho w ma n y pro grams and ta bles ar e e ffected?

• Datab ase chang es - do we a dd n ew fiel ds or just rena me t he fi elds an d preserve

the stat us q uo? Do we kno w th ose fie lds wer e on l y use d for emai l and mobil e
phone da ta?

• Datab ase integrit y - Fie lds d estine d for a d-hoc dat a like 'extra inf orma tion' and
'not es' are un likel y t o h ave an y valid atio n or to be even req uire d so if mi grat ing
the existi ng valu es t o ne w fie lds we can't simp l y cop y it over some cle ansing wi ll
be req uire d.

The pr ocess of ga in ing the kno wl edg e to a nswer these qu estio ns ma y n ot be all
that straig htf orward, par ticul arl y if the syst ems ar e comple x or th e p eop le trying to
answer the m ar e n ew to the ap plicat io n, s yste m or pla tfor m.

To assess the scop e a nd impact of th e cha nge you nee d to fi nd out wh ich pr ogr ams
use the fi les/ta bl es aff ected , this can be ver y lab ori ous :

✔ Go thr oug h a ll source fil es in PDM,

✔ opt ion 25 to searchin g
✔ the n F1 3 to re pea t
✔ press ent er ,
✔ type i n yo ur search ter m
✔ review resu lts ...

...and that's just th e first enq uir y! Depe ndi ng on the comp le xit y a nd histor y of your
systems you ma y have do ubts that you wer e lo okin g at al l of the source or the
lat est versi on.

Looking int o Dat ab ase in tegr it y ma y we ll thro w up ite ms like the screen shot bel ow:

4
 Analysis & Documentation with X-Analysis

We have a number of diff ere nt formats of email addr ess an d some e xtra ne ous text ,
simil arl y on th e ph one number l ist t here is te xt an d a variet y of la you ts. Fin all y we
have the in evitab le result of usi ng ad- hoc fi elds wit h n o vali dat io n or on screen
gui dance - tr anspose d d ata mo bil e i n e ma il and vice- versa.

Hopeful l y this sectio n ra ised aware ness of th e prob le ms arou nd chan gin g and
modernizing S yste m i ap plicat io ns, the issu es wi th fin din g o ut the necessary
inf ormation and ho w seemi ng l y stra ig htfor wa rd issues can be time consu mi ng and
problemat ic. X-Anal ys is is desig ne d and opti mize d to make the d esign recover y
process as stra ig htfor wa rd as possibl e as th e rest of t his concepts gui de wi ll
ill ustrate.

Analysis, Documentation & Application Subdivision

X-Anal ys is bu ilds a ver y det ail ed rep ositor y over an entir e app licati on. The
repositor y maint ai ns all inf ormati on ab out ap plicat ion ob jects, th eir rela tio nships
and all necessar y inf ormatio n to o bta in de tai le d inf orma tio n fro m each object
across the e ntir e s ystem. 2 0 ye ars of on go ing deve lop me nt, over th ousa nds of
AS/400/iS eries/S yst em- i app licati ons wr itte n in a ll varia nts of RPGII/ 400/IV,
COBOL, and CL, has pr oduce d a n u nmatche d capa bi lit y to extract ever yth ing about
an appl icati on fro m ob ject righ t do wn to ind ividu al vari abl es. The re positor y is bui lt
aut omatical l y usi ng a sing le comma nd, a nd in iti all y coll ects all obj ect relat ed
inf ormation, but the n p arses ever y source member in the specifie d s yste m and
ever y source li ne ma ppi ng the cont ext ual inf ormati on of each vari abl e i n the
system. A certa in amo unt of lo gical abstracti on processi ng the n takes place whil e
bui lding the re positor y to accoun t for some of the i dios yncr asies typic al in a n RPG
applicat ion . Th is inclu des constructs such as varia bl e pr ogra m calls, fil e overr ides,
pref ixi ng an d re na min g i n RPG. Th e re posit or y thus re prese nts a map of ho w the
ent ire app licat ion fu nctions right do wn to in divid ual varia bles.

Understanding Design & Function More Easily

For eff icien t famili arizati on o f an a ppl icatio n's structur e an d ge nera l functi on, an
abstracti on a bove th e source code comb in ed with obj ect-to-o bject re lati onal

5
 Analysis & Documentation with X-Analysis

inf ormation is requ ired . A few simple bu t rich types of col or-code d, graphical
diagrams can reve al th e dat a flo w a nd archi tectur e of in divid ual obj ects or par ts of
an entir e syste m. Th is is combi ne d wi th a uto mat icall y der ived descript ions in the
form of Pse udo narra tives e ith er in th e di agra ms or wh il e bro wsin g source code.
The dr ill- do wn , go- an ywh er e-fro m-an ywhe re, i nter active nat ure of t hese int erfaces
in the X-Ana l ysis clie nt pr ovides a un iq ue appr oach to infor ma tio n assimi lat ion,
all owing an a na l yst to ga ther infor ma tio n at hi gh l evel or ver y d etai le d in an
effici ent and i ntu itive mann er. Th e ap plica tio n abstracti on is raised one l evel above
implementat io n. T his insta ntl y removes comple xit y cause d b y the idi osyncrasies of
diff erent lan gua ge versi ons a nd codin g pr actices, t yp ical in larg e l egac y
applicat ions devel ope d over ma n y yea rs.

Here is a bri ef descripti on of some of th ese d iagr ammat ic constructs and views:

Structure Chart Diagr am - A Structur e Ch art Dia gra m (SCD) Displ a y gives a
graphic re prese ntat ion of ho w the contro l p asses fro m o ne pro gra m to an other
program wit hi n the a pp licati on. This foll ows th e call structure d own th e complet e
stack. The d ia gram also revea ls dat a i npu t o bjects an d also a uto mat icall y d erives a
summarized d escri ptio n of e ach of the obj ect in the dia gra m. Color- codi ng also
reveals imp orta nt functio na l asp ects such as u pd ates, prints, and disp la ys, which
hel p the user to zone in on commonl y, soug ht af ter det ails.

Figure 1: Structure Chart Diagram for a Program

Data Flow Diagra m - A Data Flo w Dia gram (DFD) is a grap hical represe ntat io n of a
program/ob ject wh ere used, showin g th e files a nd pro gra ms accessed by the
subject obj ect. It is also color-cod ed and shows bot h fl ow of da ta at a hi gh obj ect
level, and con textua l inf ormati on a bou t the specif ic varia bl es/para me ters passed
bet ween ob jects.

6
 Analysis & Documentation with X-Analysis

Figure 2: Data Flow Diagram for a Program

Program Structure Chart - A Progr am Structure Chart gra ph icall y d ispla ys the
sequence o f calls i n th e progr am. Th e call coul d b e to execute a Subro uti ne /
Progr am / Mo du le / Service Pro gra m. F or d etai ls, ref er to X-Ana l ysis User Manual.

Figure 3: Program Structure Chart for a Program

7
 Analysis & Documentation with X-Analysis

Overvi ew Structure Chart - The Overvie w Structure Chart gives a snapshot of an

applicat ion . It displ a ys all the e ntr y p oi nts to the a ppl icati on, a nd th en th e structure
chart f or e ach of these entr y poi nts.

Figure 4: Overview Structure Chart for complete application

RPG as Ps eudo Code- W ith a singl e click, RPG can be viewed as a form of
structur ed Eng lish or Pseud o cod e. Mne mo nics. are substitut ed wi th
fil e/fi eld/varia ble te xts an d consta nts or liter als.

8
 Analysis & Documentation with X-Analysis

Figure 5: RPG to Pseudo code with a single click

Producing Static Documentation Automatically

Inter active an al ysis via a gra ph ical clien t is g ener al l y th e most in tuit ive ma nner in
which t o a nal yz e a syst em, but th ere is ofte n a req uire me nt for vari ous t yp es of
static inf ormati on in the form of struct ured docu me ntat io n. Examples of th is are
proj ect document ati on, aud iti ng in formati on, testing instructio ns, and customer
suppor t docume nta tio n (such as wit h ISV suppl ie d b usiness software). X-Anal ysis
produces a number of these out puts i nclud in g:

Data Flow Chart in MS Visio - An y in teractive di agr am prod uced b y X-Ana l ysis 8
in the clien t, can be a uto mat icall y export ed i nstant l y to MS Visio with a sin gle click.
In additi on to th is, an RPG or COBOL pr ogra m can be pro duce d as a dat a fl ow
chart int eractivel y whi le bro wsi ng th e source from wit hin X-Anal ys is. If t he RPG
program is in Pseu do Co de mode , the Dat a Flo w Char t wi ll use th e narr atives from
the Pseud o cod e. T his en ab les no n-s ystem i techno lo gists an d a nal ysts to
assi mil ate i nfor mat ion at a det ai led level of t he ap pl icatio n wi th out a n y d ep endenc y
on RPG or COBOL experts.

9
 Analysis & Documentation with X-Analysis

Figure 6: DFD exported to MS Visio

Lists and Resu lts sets – An y source, o bject, or i mp act-an al ysis resu lt l ist can be
directl y exp orte d to for mat ted MS Excel or Word wit h a singl e click wh il e usi ng the
client .

MS Word Project Docume ntatio n Wizard – W ith the use of a simpl e wizar d,
documents that mi ght take we eks to pr oduce ma nu all y, all ow th e user to select any
of the grap hical di agr ams, l ists, flo wcharts, ann ota tio n a nd busin ess rules
summaries gen erat ed int eractivel y b y the client int erface , can be collat ed int o a
singl e document wi th conte nts an d i nde x. This can be do ne for a sing le obj ect, an
applicat ion are a (exp lai ne d b elo w), a list of ob jects, or an entir e syste m. Any of
these docu men ts can th en be ed ite d a nd distrib ute d as re qu ired .

Dividin g S ys tem s into App li cation Ar e as

Entir e l egac y ap plica tio ns are oft en too larg e to effectivel y compre hen d or eff ect
whol esale chan ge. For this re ason it is of ten necessar y or h el pful to sub d ivide a
system into ap plica tio n ar eas. The reasons and specificat ions for th ese may
change wit h ti me to o. X-An al ysis provid es facilit ies for subd ividi ng an ap plicat ion
area into grou ps of obj ects that me et user d efin ed selecti on crit eria. Th ese criter ia
mi ght be based on fu nction or even ge ner ic n ame. X-Ana l ysis then uses the
sophisticat ed cross-refer ence inf ormatio n and Data Mo de l rel ati onshi ps to include,
aut omatical l y a ll rel ate d el ements such as pro gra ms, d ispla ys, or files in the

10
 Analysis & Documentation with X-Analysis

applicat ion area . App licati on ar eas filt ers can th en b e used thr ou gh th e X-Anal ysis
Solution Se ts to vie w, docu me nt or re-e ng ine er as opp osed to i ndivi du al obj ects.

Figure 7: Application Area diagram for System Overview

The Appl icatio n Are a dia gra m in X-An al ysis is i nter active an d b y clicki ng on
diff erent par ts of yo ur s yste m you can see the re lat ionsh ips bet we en e ith er all
parts or just the are a yo u've clicked on an d th e ar eas it rel ates t o.

11
 Analysis & Documentation with X-Analysis

10 Summary

Compr ehensive, accurat e, an d current d ocument atio n of a l egac y app licati on

improves qua lit y, pro ductivit y an d red uces risk, for an y mai nte na nce, mo der nizati on
or rebui ld IT pr oj ect. Th e risk associa ted with ma int ain in g l arge comp le x l egac y
applicat ion , wit h a rapi dl y d imi nishi ng set of le gac y skills, can be lar gel y miti gat ed
by access to such document ati on.

X-Anal ys is provi des qu al it y an al ysis a nd docu men tat ion fe atur e. 2 0 ye ars of
devel opme nt eff ort, ensur es t hat virtu all y an y le gac y a ppl icatio n can be ana l yzed
and automat icall y d ocume nte d.

Richard Dow ney and Stuart Milligan

© Databorough

12