1. Terminology

Term 

Definition

PMR  

“Problem Management Record”. Problem reported by a customer - 
not necessarily a product bug.

客户报上来的问题，不一定是产品的bug/defect。

APAR       

“Authorized Program Analysis Report”. 
Once it is confirmed that issue reported by the customer 
in a PMR is a bug, a customer visible bug report is created as an APAR

如果确定客户报上来的PMR是一个bug的话，那么就会产生一个客户可见的APAR出来。

CRM        

“Customer Relationship Management”. 
An entry in the WebCRM system corresponding to a customer problem or question.

CRM is used for communication between L2 and L3 teams about
 customer reported issues.  

WebCRM is the tool that is used.

MDV

Maintenance Delivery Vehicle - code patch that 
is delivered to the customers containing code fixes. 
MDV can be a Fix Pack/Interim Fix/Limited Availability Fix or a Test Fix.

MDV是包含了修改的代码的Patch给客户的。
MDV可以是fix pack，iternim fix，LA fix或者Test fix。

2. What is an APAR and when are APARs created?

IBM customers are very familiar with the term APAR which is an acronym for "Authorized Program Analysis Report". APARs are externally defined as "bugs in the product code that require a fix". Every time that a defect is found in our code (either by external customers, or internally by IBMers) which will impact our customers, an APAR document must be created so that customers can be alerted to the problem and can determine which Fix Pack will ultimately include the fix for this bug. For a PMR reported problem, when there is clear evidence that the reported problem is a defect (code or documentation), a field APAR will be created once Level 2 and Level 3 have reached collective agreement. If a Developer encounters a problem or new function which he/she decides that customers are likely to encounter, then an APAR must be created for the defect. These are internal discovered APARs.

3. PMR生命周期

Defined below is the workflow for the resolution of PMRs by Level 2 (L2)- 1.    Customer contacts the support team to report a problem seen with the product 2.    Support engineer (L2) will collect all relevant information from the customer a.    Version of Product b.    Platform on which the problem was seen c.     Database server used and the version d.    WAS version e.    MQ f.     Problem encountered 3.    Based on the description of the problem the support engineer will determine the severity of the problem and try to nail down the component in which the problem occurs. 4.    Support engineer will open a PMR for the problem in RETAIN. 5.    L2 will carry out the initial analysis and try to narrow down the problem 6.    If a viable solution is not available to provide to the customer and if the L2 feel a consultation with L3 is needed then the L2 will report the problem to L3 via the CRM.  The CRM is created using the WebCRM tool. 7.    Based on the component against which the CRM is created appropriate L3 team members receive e-mail notification of the CRM entry creation.  At this point the CRM is in "L3 assistance pending" state. 8.    L3 engineer will accept the CRM and then moves the CRM to "L3 investigating" state. 9.    L3 engineer works on the issue at hand and provides updates on the issue using the WebCRM tool.  At this point they will either leave it in the "L3 investigating" state or move it to "Waiting for L2 action" when more details are needed from L2. 10.   If the CRM is in the "Waiting for L2 action" state then L2 either provides a response to the queries posed by L3 in the CRM or sets the state to "Waiting for Customer Action" when more data is needed from the customer. 11.   Once the data/information is available from the customer the L2 updates the CRM with it and then moves the CRM to "L3 investigating" state. 12.   See section 2.2 below for Workflow for Level 3 (L3) 13.   After all the investigation done by the L3 either a solution is provided to the customer (no code change) and CRM entry is closed out or the L3 agrees that there is a defect in the product and requests the L2 to create an APAR and sets to status to "Waiting for L2 action" and L2 creates the APAR and sets it back to "L3 investigating" state. 14.   For APARs that are created the SPoRT which is a bridge tool will poll RETAIN and create defects in CQ for the APARs. 15.   The APARs are triaged to determine which release the fix will be shipped. Please refer to the Types of Maintenance deliverable section 5. 16.   When the CQ defect corresponding to the APAR is fixed the L3 engineer goes back to the CRM and sets its state to "Pending Closure" 17.   If it is an interim fix then the package is determined, README containing instructions for install is prepared, fix is tested and then the fix is uploaded to the ftp site (submit.boulder.ibm.com) by the L3 manager. 18.   L3 manager will communicate to the L2 engineer about the availability of the fix. 19.   After the fix is delivered either via a Fix Pack release or via an interim fix the CRM/PMR is closed.

4. Systems used by the Support team

RETAIN RETAIN(REmote Technical Assistance Information Network) is the customer facing issue tracking system, customer issues are logged as PMRs. The IBM IM support organizations use an application called RETAIN (Remote TEchnical Assistance Information Network) for logging and tracking customer issues. The RETAIN system contains records related to customer reported problems and the solution (or fix) for it. The product support teams open a Problem Management Report (PMR) to track data related to the problem, like nature of problem, problem severity, expected resolution, etc. in RETAIN. Resolution provided for the problem may or may not cause a change in the product. If change is needed in the product then the change is termed as a ‘Defect Oriented Problem’ and an ‘Authorized Program Analysis Report (APAR)’ record is created to track the solution provided for the problem.  Additionally, a ‘Program Temporary Fix (PTF) ’ record may be created to track a temporary fix to be sent to the customer until the solution is made generally available in the next fixpack or release. WebCRM WebCRM is the web-based CRM system that is used for communication between L2 and L3 teams about customer issues. This is NOT a customer facing system. ClearQuest ClearQuest is the internal system used to track product issues. Issues may be reported internally by QA or are reported by customers. ClearQuest is a customizable Defect and Change tracking system, mainly useful for dynamic software development environment. ClearQuest allows software development groups to create, modify and track the Change Requests, Bug Reports and Feature Enhancements. The ClearQuest database may be accessed from either a thick client which may be installed on a users Windows, Linux, or UNIX machine, or through a ClearQuest web client interface. SPoRT SPoRT(Service Process on Rational Tools) is the bridge tool between Retain and ClearQuest. SPoRT shares the ClearQuest schema for storing of APAR information. For every APAR created on Retain, a new equivalent APAR entry and a corresponding CQ defect is created automatically.

5. Type of Maintenance Deliverables

通常Patch就可以理解为以下等fixpack，分为四种fixpack。 1. Test Fix (TF) A temporary fix that is supplied to specific customer(s) for testing in response to a reported problem 2. Interim Fixpack A certified fix deliverable that is generally available to all customers between regularly scheduled fix packs, refresh packs, or releases Interim fixpack is a generic fixpack that is provided between 2 fixpack releases or between a MOD release and a fixpack release.  There can be any number of interim fixpack releases between 2 fixpack/MOD releases.  The interim fixpack is applicable to all customers.  It can contain fixes for 1 or more customer APARs or internal APARs.  Interim fixpack is labeled as V.R.M-IF001, V.R.M-IF002 etc. iFix是两个fixpack release之间或者一个fixpack release与MOD release之间的fixpack（之间可以有很多个版本的iFix）。它可以提供给所有客户的。它包含了1个或多个客户的APAR或者IBM内部开的APAR。iFix版本号通常表示为V.R.M-IF001, V.R.M-IF002。 3. Limited Availability Interim Fix (LA) An interim fix that is offered on a limited basis. For any of several reasons, the L3 team wants to limit availability of this maintenance delivery vehicle Limited Availability fixpack release is provided to a specific customer.  Limited Availability fixpack is labeled as V.R.M-LA001, V.R.M-LA002 etc.  LA fixpacks are not cumulative. LA fixpack release是提供给一个特定的客户的。LA fixpack版本号通常表示为V.R.M-LA001, V.R.M-LA002。 4. Fix Pack (FP) Official maintenance release published on a periodic basis to the IBM support web site. 注：Hotfix可以是上面的任意一种，只要是客户比较急需要我们ship出去的都是一个hotfix