General Testing Instructions
(1) Orienting yourself to your assigned project areas of the program and how far along our projects are is a good place to start. These projects represent different sections or features of the program, and for many projects the plan is for each of us to be responsible for our entire project area with individual assignments coming as they are ready. It may also be helpful to get a sense for some of the projects that haven't been implemented yet, as we will be testing around these parts of the program.
(2) Please contact Winston or the Project Manager (Tina for GiftWrap, Ann for PGM) if you want to do any horse-trading of projects in your court with other testers. You can also put in bids on the projects that will be coming later (haven't been assigned yet).See Winston or Nancy Fazzina if your machine has not been set up to run the test version of the software.
(3) There will be 3 types of assignments - Intest, Thotest, and SoloTest. The main difference between SoloTest and Intest is that we plan to put the Intest items also through a Thotest while SoloTest items will go straight to Fintest without a second look. Thotest is a second review by a different person than did the Intest and won't start until a critical mass of Intest has been successfully completed. If you are both a programmer and a tester, you are ineligible to test parts of the application that you programmed.
Use of Problem Tracker for Testing
(1) How to pull up and print your assignments. Click on the Query button, then select the Inbox query and click Preview. Then at the bottom for the report layout select either Assigned/Status if you want a list or Full Log if you want the full story. When printing, select Table for Assign/Status and Text for Full Log. Other useful queries are Projects and Assignments. Beware that the list on the Home page when you first log in is not complete and that the Inbox report form does not include all the items on the Full Log form. Remember to log on as yourself if you are using one of the testing lab machines so that you are not logging stuff under the previous tester's name.
(2) What standard of review to give your assignment. In general, this will depend on the testing state of the item in question.
Solotest - These are generally smaller items that we feel only need to be tested once, as opposed to going through a Intest by one tester and a Thotest by another tester. Test it rigorously as you will be the last person to look at it.
Intest - These are items will be tested again in Thotest. For the initial test, can generally just test on local machine, unless feature is specific to another operating system or systems. Tester should look for any problems, and also that design works as intended and clients would expect.
Thotest - These items should have already been put through an Intest, and this should be the last look. For the thorough test, should put the feature though all its paces and check on each platform being separately tested. At this point we are interested foremost whether the feature works as designed, and less in redesign of the feature.
Fintest - This indicates that items have passed successfully through QA. Generally, we don't assign these to individual testers - instead we do assign some number of hours of final testing.
(3) How to route assignments. Note that while you can route items from query results screens, it is faster to close this screen first to avoid query refreshes, particularly if routing a number of items.
If the testing assignment works, tester should use task Complete/Approve to move automatically to the next state. Generally don't need to add any testing notes, as "tested OK" is implied by approving.
If the testing assignment doesn't work and the whole thing should be sent back, tester should use task Send Back/Reject. The testing notes should be annotated with timestamp, initials, and description of the specific problem.
In many instances, a new item should be logged rather than rejecting the whole assignment, and you should continue testing the original assignment. If there is nothing more to test and you are waiting to get a separately logged item back before approving, you can use the edit button to change the substatus to "Pending" to indicate you are waiting for someone else. A good rule of thumb is to not reject Intest or Thotest assignments and instead log any problems as new items. Some examples of when you should log separate items rather than just reject the original assignment:
(A) Stuff that a tester finds in one part of the program while testing another. For example, a tester is assigned reports but finds a problem on the person information screen. We don't want the tester to reject the report, and we don't want the tester to spend a lot of time chasing down who was assigned the person screen, where it is in the process, etc.
(B) Stuff being proposed for future updates. Sometimes, the person creating these can send them straight to Open although it's also useful for the project leader to have a look in case it is relevant to the current release.
(C) Items that don't stop a tester from continuing to test but that programmers can work on while testing continues. (So as to reduce the number of times that big items get routed back and forth, and reduce the turnaround time for testing and fixing a particular problem.)
(D) Parts of problem reports that we decide to table without tabling the original item.
(E) Problem reports where one part can be fixed now but another part will have to wait till later in this update.
(F) Tester finds something related but not limited to their assignment. That is, tester finds problem while testing one report, but discovers same problem also appears on other reports. We wouldn't want to have to track down who was assigned all the other reports, and even if they were assigned to the same person, we probably wouldn't want to send them all back for a Fixit particularly if there are other aspects of the reports that do work or that can continue to be tested.
(4) How to log new items. To add a new item into the database, click the Add icon from the toolbar at the top of the page. You will be presented with a screen of data elements to fill in about the item. The following is a list of the data elements with a description of what might be entered for the items. Some of the fields are required and you cannot add the item without entering that information. Other fields are not required, but you should take a shot at providing the information.
Product – (Required) Pull down menu, select the appropriate Product (GiftWrap, GiftCalcs, PGM) for the item being entered. Double check this as if you select the wrong Product, your newly added item will be lost.
Project – Text field, enter the project name (i.e. PGM 4.5, Custom Reports) assigned to this item. For each Product release there will be an approved list of project names that must be entered exactly. If you don’t know the correct project name or are unsure as to which project a particular item belongs, please leave this field blank. This information will most likely be entered AFTER the item has been reviewed by Winston or Bill and not during the entry process.
Title – (Required) Text field, enter a title for the item, be as descriptive as possible. For example, Update the copyright dates on all pages or DCN: add new narrative follow up questions. A good title will both summarize the issue in a way that will be identifiable and differentiate from other items, example of poorly written title: Error in CashTrac.
Date Logged – Date field, will default to current date.
Description – (Required) Long text field, enter some description of what the item is in plain English, giving as much detail as necessary. For example, “The current copyright date is out of date on all pages. Replace the copyright date in all location on page. if you attach screen shots of error messages, please summarize in the description so that a person reading can get the gist of the problem without having to open the attachment.
Plan/Instructions – Long text field, enter a plan for implementing the item. This can be either a plain English description or a more technical one. For example, “Collect user id and password from clients entering the site in a login in screen, check the entry against the client database” or “Use ASP page to build the dates and location page from a database stored on the server. Programmers can add details/follow up to the plan including information about how they programmed the item. Generally, it’s a bad idea to attach design documents because of the difficulty of editing attachments if there are later changes. Better practice is to include a directory path to a design document saved on ouur network.
Project Priority This will default to None and should be left alone for the Project Manager or Development Manager to assign later.
State – Pull down menu, this will default to New and should be left alone unless you have permission to move the item directly into another State.
SubStatus – Pull down menu, this will default to None and should be left alone unless you have permission to indicate a valid Substatus for the item. Substatus will only be used in specific instances, so the majority of the time it should be left at None when a new item is being added.
Assigned To – Pull down menu, this will default to State Manager when adding and should be left alone unless you have permission to assign the item to another person.
Category – Pull down menu, this will default to None but you should try to select the Category that matches the type of item you are entering.
Request Type – Pull down menu, this will default to None, but you can select the appropriate Request Type for the item if you know it. It's okay to leave this field blank to be filled in letr when the item is reviewed. There are four Request Types, and: Bug, Enhancement, Issue, Maintenance and definitions are below:
Bug – Confirmed problem or error in released version (not just under development)
Enh – Enhancement, new feature, functionality or improvement
Issue – Item that needs attention, edits, changes, graphic updates
Main – Maintenance item, ongoing edits, changes (i.e. discount rate update)
IBE Number – This is for items being moved over from IBE (our old system) and generally should be left blank.
Testing Instructions – This is generally filled in by the Project/Dev/QA Manager or by one of the programmers. If there is something specific you want to be sure is tested, you can enter it when adding a new item.
Reported By – Pull down menu, this will default to the person who is logged in when the item is entered. You can change this if it is not you.
Make Visible to..- Selection list, should default to Users and be left alone.
Annotate Buttons – There are Annotate Buttons next to the long text fields. If you are editing an item and wish to add/append information to one of these fields (description/plan) you should hit the annotate button first which will insert a line indicating the date/time/user name below which the additional information can be added.
(5) How to attach screen shots, etc. For error messages, it’s very useful to paste a screen capture into a Word document, which can then be attached to a Problem Tracker entry.The Add screen does not allow for attachments. However, once you have added an item the Edit button. The Edit screen contains an Edit Attachments button that will enable you to attach a file. Note that you should still fill in a good description for the item as described above.
Use of Problem Tracker for Design
(1) Indentifying items for consideration. During the first part of the design process, items to be considered should be moved to Selected and given a substatus of None. Typically, there will be items already in Selected left over from the last release, although their substatus may need to be changed. This may also involve a review of all items in Open, as well as items in New logged since the last release.
(2) Sorting items in Selected. To facilitate design committee meetings, the items in Selected should be sorted by Winston, Bill, or the Product Manager into three groups using the substatus field. Then lists can be printed from Problem Tracker for review by the design committee, and once a decision is made can be routed as described below. The three groups are:
ToDo – items to include in the new release
Table – items to table for future consideration
Discuss – items that need to be discussed by the design committee
(3) Routing items to be tabled. Items to be tabled can go to one of three places
Open – no special consideration for next release, will be mixed in with lots of items already in Open
Selected, Pending – high consideration for next release, don’t mix with all items in Open
Out – not a problem, no longer relevant, doesn’t need to be part of future review of Open items
(4) Routing items ToDo. The ToDo items can be routed to Design, Program, or Fixit as appropriate and assigned to the appropriate person.
(5) Ongoing review of New items. After the initial design process is complete, there will be a continuing need to review new items being logged by testers, clients service representatives, etc. These should not all be routed to Selected, as that is now reservered for items that will get high consideration in the next release and won’t be looked at for the current release.
(6) How to attach screen shots, etc. For error messages, it’s very useful to paste a screen capture into a Word document, which can then be attached to a Problem Tracker entry. The Add screen does not allow for attachments. However, once you have added an item just click the Edit button, which by default will bring up the item you just added. The Edit screen contains an Edit Attachments button that will enable you to attach a file.
Use of Problem Tracker for Programming
(1) General. Programmers typically get items in Design, Program, or Fixit. While the item is being worked on it may be useful to change the Assign To from one programmer to another. Or the main programmer can keep the item on his or her list and coordinate the work of the other programmers involved. If during the Design you actually program it, first change the State to Program or Fixit, as appropriate, then Approve it – this way it will be automatically routed to build.
(2) Design. Don’t use Approve or Reject - report back to Winston or Bill when the item is ready to be moved. Can set substatus to Table if you think the items should be tabled or Discuss if you think the implementation should be discussed before proceeding.
(3) Program & Fixit. Approve will send to Build. Don’t use reject – report back to Bill or Winston if you think something else should be done with the item. You can set the substatus to Table if you think the items should be tabled or to Discuss if you think the implementation should be discussed before proceeding.
(4) Filling in the Plan/Instructions. Enter any information that will be helpful to understanding the implementation. If the item is large or part of a larger group of items, a separate design document may be useful. Generally speaking it is not a good idea to attach design documents that may have to be edited. It’s better to save the design document on out
(5) Writing Testing Notes. It’s very helpful to write testing notes to guide testers. For example, you may want to identify particular aspects of the new feature that you are most concerned about not working. Or you might identify conditions that should be checked, side effects that should be watched for, and other parts of the application that could be affected.
Project Milestones
A typical medium or large typical project will have the following milestones. Smaller projects may compress milestones.
Start of design
Scope complete
Start of programming
Data structure complete
Design complete
Start of Intest
Code complete
Start of Thotest
End of Intest
End of Thotest
Start of Fintest
End of Fintest – Gold
After Gold comes production - finalization of accompanying materials, duplication, kitting, etc.