1. Terminology

 Term  Definition PMR   “Problem Management Record”. Problem reported by a customer - not necessarily a product bug. 客户报上来的问题，不一定是产品的bug/defect。 APAR        “Authorized Program Analysis Report”. Once it is confirmed that issue reported by the customer in a PMR is a bug, a customer visible bug report is created as an APAR 如果确定客户报上来的PMR是一个bug的话，那么就会产生一个客户可见的APAR出来。 CRM         “Customer Relationship Management”. An entry in the WebCRM system corresponding to a customer problem or question. CRM is used for communication between L2 and L3 teams about  customer reported issues.  WebCRM is the tool that is used.   MDV Maintenance Delivery Vehicle - code patch that is delivered to the customers containing code fixes. MDV can be a Fix Pack/Interim Fix/Limited Availability Fix or a Test Fix. MDV是包含了修改的代码的Patch给客户的。 MDV可以是fix pack，iternim fix，LA fix或者Test fix。

2. What is an APAR and when are APARs created?

IBM customers are very familiar with the term APAR which is an acronym for "Authorized Program Analysis Report". APARs are externally defined as "bugs in the product code that require a fix". Every time that a defect is found in our code (either by external customers, or internally by IBMers) which will impact our customers, an APAR document must be created so that customers can be alerted to the problem and can determine which Fix Pack will ultimately include the fix for this bug. For a PMR reported problem, when there is clear evidence that the reported problem is a defect (code or documentation), a field APAR will be created once Level 2 and Level 3 have reached collective agreement. If a Developer encounters a problem or new function which he/she decides that customers are likely to encounter, then an APAR must be created for the defect. These are internal discovered APARs.

3. PMR生命周期

Defined below is the workflow for the resolution of PMRs by Level 2 (L2)- 1.    Customer contacts the support team to report a problem seen with the product 2.    Support engineer (L2) will collect all relevant information from the customer a.    Version of Product b.    Platform on which the problem was seen c.     Database server used and the version d.    WAS version e.    MQ f.     Problem encountered 3.    Based on the description of the problem the support engineer will determine the severity of the problem and try to nail down the component in which the problem occurs. 4.    Support engineer will open a PMR for the problem in RETAIN. 5.    L2 will carry out the initial analysis and try to narrow down the problem 6.    If a viable solution is not available to provide to the customer and if the L2 feel a consultation with L3 is needed then the L2 will report the problem to L3 via the CRM.  The CRM is created using the WebCRM tool. 7.    Based on the component against which the CRM is created appropriate L3 team members receive e-mail notification of the CRM entry creation.  At this point the CRM is in "L3 assistance pending" state. 8.    L3 engineer will accept the CRM and then moves the CRM to "L3 investigating" state. 9.    L3 engineer works on the issue at hand and provides updates on the issue using the WebCRM tool.  At this point they will either leave it in the "L3 investigating" state or move it to "Waiting for L2 action" when more details are needed from L2. 10.   If the CRM is in the "Waiting for L2 action" state then L2 either provides a response to the queries posed by L3 in the CRM or sets the state to "Waiting for Customer Action" when more data is needed from the customer. 11.   Once the data/information is available from the customer the L2 updates the CRM with it and then moves the CRM to "L3 investigating" state. 12.   See section 2.2 below for Workflow for Level 3 (L3) 13.   After all the investigation done by the L3 either a solution is provided to the customer (no code change) and CRM entry is closed out or the L3 agrees that there is a defect in the product and requests the L2 to create an APAR and sets to status to "Waiting for L2 action" and L2 creates the APAR and sets it back to "L3 investigating" state. 14.   For APARs that are created the SPoRT which is a bridge tool will poll RETAIN and create defects in CQ for the APARs. 15.   The APARs are triaged to determine which release the fix will be shipped. Please refer to the Types of Maintenance deliverable section 5. 16.   When the CQ defect corresponding to the APAR is fixed the L3 engineer goes back to the CRM and sets its state to "Pending Closure" 17.   If it is an interim fix then the package is determined, README containing instructions for install is prepared, fix is tested and then the fix is uploaded to the ftp site (submit.boulder.ibm.com) by the L3 manager. 18.   L3 manager will communicate to the L2 engineer about the availability of the fix. 19.   After the fix is delivered either via a Fix Pack release or via an interim fix the CRM/PMR is closed.

4. Systems used by the Support team

RETAIN RETAIN(REmote Technical Assistance Information Network) is the customer facing issue tracking system, customer issues are logged as PMRs. The IBM IM support organizations use an application called RETAIN (Remote TEchnical Assistance Information Network) for logging and tracking customer issues. The RETAIN system contains records related to customer reported problems and the solution (or fix) for it. The product support teams open a Problem Management Report (PMR) to track data related to the problem, like nature of problem, problem severity, expected resolution, etc. in RETAIN. Resolution provided for the problem may or may not cause a change in the product. If change is needed in the product then the change is termed as a ‘Defect Oriented Problem’ and an ‘Authorized Program Analysis Report (APAR)’ record is created to track the solution provided for the problem.  Additionally, a ‘Program Temporary Fix (PTF) ’ record may be created to track a temporary fix to be sent to the customer until the solution is made generally available in the next fixpack or release. WebCRM WebCRM is the web-based CRM system that is used for communication between L2 and L3 teams about customer issues. This is NOT a customer facing system. ClearQuest ClearQuest is the internal system used to track product issues. Issues may be reported internally by QA or are reported by customers. ClearQuest is a customizable Defect and Change tracking system, mainly useful for dynamic software development environment. ClearQuest allows software development groups to create, modify and track the Change Requests, Bug Reports and Feature Enhancements. The ClearQuest database may be accessed from either a thick client which may be installed on a users Windows, Linux, or UNIX machine, or through a ClearQuest web client interface. SPoRT SPoRT(Service Process on Rational Tools) is the bridge tool between Retain and ClearQuest. SPoRT shares the ClearQuest schema for storing of APAR information. For every APAR created on Retain, a new equivalent APAR entry and a corresponding CQ defect is created automatically.