Weekly Status Report

Database Support Team

Date: April 21st 2014 April 27th 2014

Agenda
1 2 3 4 5 6 7 8 9 10 11 12 Last Week Incident Summary Weeks Highlights People Recognition Last Week Incident Summary Weekly incident count Weekly SLA compliance High priority incidents details Mitigation Change Details Problem Records Additional & adhoc activities Work Request / Activity list
strictly confidential, confidential, internal, public

Operational Highlights

Weeks Highlights
One High Priority incident resolved by DB Support last week IM0001932943/P2 (Oracle) . DBA Supported two P2 Priority Incidents IM0001929189 (Informix), IM0001929611 (DB2). 7 P3 and 145 P4-P6 incidents resolved this week within SLA. Problem tickets resolved 5; Problem ticket RCA Submitted 3; Problem ticket update sent to PM 2. 1 High risk in My SQL, 1 Medium Risk in Oracle, 2 low risk in DB2 and 1 low risk MS SQL changes successfully implemented this week. MS-SQL 10 Completed out of 39; Informix 1 completed out of 6; Oracle 17 completed out of 52; DB2 100% completed for FMO with recommended patch.

Patching: Revised Patching schedule updated. New patching schedule published & raised respective changes for Informix DB servers. Oracle: PTOPIC standby sync issue -- PTOPIC standby sync issue has been resolved manually. permanent solution has been applied for it. PCRD02 standby sync issue -- Root cause analysis is in progress. WR 3255 P14 Nutmeg 3 - Excalibur Database Copy access Completed Space uplift activity for ABCS Completed successfully. SMC-24 new process testing is done. SRP project ORC started. DB2: Need to move DB2 pre-prod digital backups from old backup server tsnv0039 to new one tsukrv0139 This has been completed with C000020460 change There is some time amendment in terms of backup schedule for EDEN FMO databases under the change(C000018417) has been completed successfully Informix: Table restore on discsoc database was implemented successfully. MySQL: TOOLKIT release on 4/17/2014 was successfully implemented(under C000020129).

strictly confidential, confidential, internal, public

Challenges/Ongoing Issues from this week

Weeks Highlights

Informix: Informix estate monitoring problem case needs to be presented to Ian/ Andy. Offshore DB team sent the Position statement to Prasad & onsite team for review. Changes created to resolve monitoring alerts are not getting approved in time even after providing all information. Where the situation is getting critical we are implementing the changes under P3 ticket with TSM approvals. Patching changes delayed. OVO Monitoring cannot be setup. We need to setup our own monitoring. Addition of Fragments on two tables in JT was postponed twice last week since it was not fully approved. It is scheduled on 28/04/2014.. Another change on table reorg on cenmain was postponed since incorrect RFC was attached. It is scheduled for 30/04/2014 01:00:00. MS SQL: Unknown MS SQL count is reduced from 200+ servers to 113 servers. End report need clarification Discussed with Ian/Andrew. Not yet clarified. Still team is following up onsite folks. Next week call need to be setup with TSL DBAs for more information. Oracle: BDA Tool Access issue with TICKS card. Aston Request pending got ASTON certificate for some of the team members and others are in progress. WIW requests and ticks card requests are pending for new team members. This can be highlighted as a risk. Because Access has been revoked for released resources. Due to this cannot use current bandwidth efficiently. ETU FMO performance issue.It not yet clear whether it is in scope or not. Need to create updated patching plan as per new PSU release by oracle. PCRD02 standby sync issueneed confirmation whether to proceed with full restore. Patching: PIVR(TSM did not approve) and PSOA database patching Cancelled. (another SLT change was scheduled).

Next week hot topics

PTOPIC switchover is in plan. Excalibur post go-live change reschedule for 30th April. Excalibur CMO database decommission request is planned. Change C000019477 has been scheduled for TOOLKIT application to Increase MAX_CONNECTION parameter for Toolkit Database. ORC phase for SRP project from MS SQL area has been scheduled for next week.

Appreciations/Certifications
N/A

People Recognition

strictly confidential, confidential, internal, public

Last Week Incident Summary

Database Executive Summary
Inflow
Number of Incidents received Number of Incidents transferred Number of Incidents closed Number of Incidents closed within SLA Number of Incidents closed within SLA (Mitigated) Number of Incidents open Number of Incidents open Older than 25 days SLA Compliance (Mitigated) / CKPI Compliance
P1 P2 P3 P4 P5 P6

0 0

1 0

7 0

48 0

76 0

21 0

0 0 0 0 0 N/A

1 1

7 5

48 48 0 11 0 100%

76 76 0 42 4 100%

21 21 0 14 1 100%

0 0 0 100%

2 5 0 71%

Weekly incident count

Incident Resolved Count - Weekly Week 3rd Mar 9th Mar 10th Mar 16th Mar 17th Mar 23rd Mar 24th Mar 30th Mar 31st Mar 6th Apr 7th Apr 13th Apr 14th Apr 20th Apr 21st Apr 27th Apr P1
1 0 0 0 0 0 0

P2
2 0 0 0 2 0 0

P3
19 16 13 6 4 16 6

P4
59 61 50 39 29 43 64

P5
99 104 82 108 80 80 104

P6
38 33 29 32 6 25 70

48

76

21

Count 218 214 174 185 121 164 244 153

strictly confidentia, confidential, internal, public

Last 10 weeks dta

Weekly SLA compliance

% Mitigated SLA Compliance
P1 Week
Actual al Mitigate ated Actual l

P2
mitigate ated Actual al

P3
mitigated ed Actual

P4
mitigate gated d
100% 100% 100% 97% 100% 95%

P5
Actual l mitigated ted Actual al

P6
mitigated ted

3rd Mar 9th Mar 10th Mar 16th Mar 17th Mar 23rd Mar 24th Mar 30th Mar 31st Mar 6th April 7th Apr 13th April 14th Apr 20th Apr

N/A N/A N/A N/A N/A N/A

N/A N/A N/A N/A N/A N/A

N/A N/A N/A N/A 100% N/A

N/A N/A N/A N/A 100% N/A

100% 100% 100% 100% 100% 94%

100% 100% 100% 100% 100% 100%

100% 100% 100% 97% 100% 93%

100% 100% 100% 97% 100% 96%

100% 100% 100% 100% 100% 100%

100% 100% 100% 100% 100% 92%

100% 100% 100% 100% 100% 100%

N/A
N/A

N/A
N/A

N/A
100%

N/A
100%

83%
71%

100%
100%

100%
100%

100%
100%

99%
100%

99%
100%

100%
100%

100%
100%

21st Apr 27th Apr

Last 10 weeks data

strictly confidential, confidential, internal, public

High priority incidents details

High Priority Incidents : IM0001932943 /P2(Oracle) Summary: Archival error on WFM database. Details: WF application purging was scheduled which generated lots of redo on database . Resolution : Archive mount point has been changed where enough space is available, also P3 is raised for backup team to ensure archive backup works with effective speed. Business Impact: business being unable to manage resource correctly causing advisors to be incorrectly assigned to queues and showing incorrect adherence states which will affect service levels and advisor bonus. Service Affected: WFM Current Status: Resolved Incident raised at: 27/04/2014 08:22 Incident reported at: 27/04/2014 08:35 Incident resolved at: 27/04/2014 08:57 Actions taken: 27/04/2014 08:35 IM accepted, Tech bridge opened 27/04/2014 08:37 Technical bridge opened with Application team, DBA & IM . 27/04/2014 08:39 Archival issue found on wfmprod database due to high volume of archive log generation. 27/04/2014 08:45 As a work around, archive mount point was changed and confirm to application team to restart the work. 27/04/2014 08:57 IM/application team confirm resolution of incident. 27/04/2014 10:15 ECR C000021334 which was raised for purging activity completed . Time Spent: 20 Min on call. 90 Min for monitoring as precautionary measure. RISK : No low priority incident was triggered by Tivoli/HP Ovo hence will raise incident for their investigation as well.
strictly confidential, confidential, internal, public

Mitigation
Mitigation plan
Incident ID INM Close Group Name INM Resolution Code
SLA EE#13(SLA failure beyond our control - incidents breached when re-assigned - Internal TSL Issue)
IM0001910156 CSS-SK.SM.DBA-SK SLA EE#13(SLA failure beyond our control - incidents breached when re-assigned - Internal TSL Issue) P3 No P3

Priority

Breached by Cognizant Domains? (Yes/No)

No

IM0001909182

CSS-UK.SM.DBA

strictly confidential, confidential, internal, public

Change Details
Change implemented
Domain Change ID Start Time End Time Pre change activity Efforts Post Change activity Brief description Success Impact of Change

DB2 DB2

C000020950 C000020954

25/04/14 10:00:00 25/04/14 10:00:00

25/04/14 15:00:00 25/04/14 15:00:00

30 min 30 min

180 min 180 min

0 min 0 min

BAU | EE | CUST | Eden - EE Staging

Success

Low Risk Low Risk

BAU | EE | CUST | Eden TMUK Success Staging UNIX - SECURITY PATCH Using Bladelogic patch on EEUKRP0345 Success

C000020710 MySQL Oracle C000021211

24/04/14 22:00:00

25/04/14 06:30:00

0 Min 30 Min

30 Min 120 Hrs

0 Min 0 Min

High Risk Medium Risk Low Risk

4/25/2014 23:00

4/27/2014 6:00 25/4/2014 13:00:00

MS-SQL

C000020114

24/4/2014 13:00:00

30 Min

30 Min

Success SLT - SOA DB migration Execute the provided script Success on RAPROD01 in the FILE 20 Min MATCH database

strictly confidential, confidential, internal, public

Problem Records
Summary Top issues

strictly confidential, confidential, internal, public

Additional & adhoc activities

Summary

Additional & Adhoc activities S.no Reference Number Date Description Effort Spent

Work Request / Activity list

Summary In Progress: 8 Early Engagement: 10 Waiting for formal request :5 Go live this week: N/A ORC: 4

Projects planned for Go Live next week: N/A Project names :N/A New WR/Project request received: N/A

strictly confidential, confidential, internal, public

THANK YOU!