JavaHelp Quality Assurance Test Plan and Tests

06/11/11 12:57:35

(Updated for V1.1.1)

Overview

This is the test plan for JavaHelp V1.1.1. It is a comprehensive document to insure the quality of the bundles that are released on java.sun.com as well as the Solaris packages that are distributed for the Solaris installation CD. As such it includes additional test requirement for java.sun.com JavaHelp web pages.

Bundles and Solaris packages are initially tested after a release engineer has successfully completed a build. They are not forwarded to java.sun.com or to the java consolidation until all the tests have been successfully completed. Once available on java.sun.com or in the Solaris Operating System the packages are retested.

External Testing

JavaHelp V1.1.1 was provided in beta form and has been released on the web since September 2000. At the time of it's web release there were no P1 or P2 bugs filed against JavaHelp. There is currently one P2 bug filed against V1.1.1.

Overall

The main bundles to test are the "fool-proof" bundles."Fool-prof" bundles have a JRE included. The "small" bundles will be tested only by content and by short tests on a few platforms since a "fool-proof" bundle is based on the small bundle. The Solaris packages will be tested as a main bundle

Each bundle or package will have at least one of the following tests executed.

Test Description
Documentation Review Full review of the product Documentation
TCK Test JavaHelp Technical Compatibility Kit for JavaHelp 1.0 Specification
Contents Verification Verification of a bundles/packages contents
Installation Verification Verification package installs correctly
Full Functionality A complete test of functionality of JavaHelp via the demos
Short Functionality A shorten version of the Full Functionality tests
Web Pages A comprehensive test of the Web pages

The tests executed on a given package are described in Testing Configurations.

Bundles and Packages

The release of JavaHelp V1.1.1 comes in 4 different packages, as indicated below:

Bundles for JH V1.1.1
Name Platform Approx Size JRE included
javahelp1_1_1-solsparc-rt.bin Solaris/SPARC 23,356,630 1.2.2
javahelp1_1_1-win-rt.exe win32 17,056,357 1.2
javahelp1_1_1-unix-no_rt.bin Unix platform 8,291,388 None
javahelp1_1_1-win-no_rt.exe win32 8,672,334 None

The JavaHelp V1.1.1 Solaris install packages comes in 4 different packages, as indicated below. The packages are designed to be installed against the Java1.2 and Java2 Solaris packages.

JH V1.1.1 Solaris Packages
Name Description Size
SUNWjhrt JavaHelp runtime 407,552
SUNWjhdev JavaHelp developer tools 391,680
SUNWjhdoc JavaHelp documentation 2,944,512
SUNWjhdem JavaHelp demos 5,427,712

Testing Configurations: Priorities

Descending Priority Order
OS Install Bundle TYPE
N/A javahelp1_1_1-win-no_rt.exe

javahelp1_1_1-unix-no_rt.bin

Documentation Review

TCK Test

Win95 javahelp1_1_1-win-rt.exe Full Functionality
Solaris 2.7 javahelp1_1_1-solsparc-rt.bin Full Functionality
NT4.0 javahelp1_1_1-win-rt.exe Full Functionality
Win98 javahelp1_1_1-win-rt.exe Full Functionality
Solaris 2.8 Solaris install packages Contents Verification

Installation Verification

Full Functionality

Solaris 2.9 Solaris install packages Contents Verification

Installation Verification

Full Functionality

Any Solaris 2 javahelp1_1_1-unix-no_rt.bin Contents Verification

Installation Verification

Any Win32 javahelp1_1_1-win-no_rt.exe Contents Verification

Installation Verification

Any win32 javahelp1_1_1-win-rt.exe Contents Verification

Installation Verification

Any Solaris

javahelp1_1_1-solsparc-rt.bin Contents Verification

Installation Verification

N/A N/A Web Pages

Documentation Review

Proof read manuals for technical accuracy. All examples and procedures will be tested.

Technical Compatibility Kit Tests

Execute the JavaHelp TCK tests against a given installation.

Contents Verification

Verify that doc/welcome/content.html and doc/welcome/CONTENS.html matches what is in the release.

Installation verification

Including installation instructions, product documents, and Release Notes.

(Review documentation from a user's perspective)

Verifying "JRE (aka VM)" Installation on Solaris

  1. Install the JRE-full release (InstallAnywhere script)
  2. Verify the version of the JRE installed as:
  3. <installdir>/jre/bin/java -fullversion

    java full version "JDK-1.2-V"

  4. Test demo programs activation.
  5. Test jhindexer.

Verifying "JRE (aka VM)" Installation on Win32

  1. Install the JRE-full release (InstallAnywhere script)
  2. Verify the version of the JRE installed from the MSDOS prompt as:
  3. <installdir>/jre/bin/java -fullversion

    java full version "JDK-1.2-V"

  4. Test demo programs activation for installed program group.
  5. Test jhindexer.

Verifying NoVm Installation

Do this in one win32 and one Solaris machine.

  1. Verify the version of the JRE installed as:
  2. java -fullversion

    java full version "JDK-1.2-V" [or 1.1.8]

  3. Install the NoVM release.
  4. Adjust CLASSPATH and PATH as XXXX. Verify as YYYY.
  5. Execute short tests.

Do a content difference between the small and the full releases. Changes should be limited to:

  1. jre/ directory is missing
  2. README_small file is new

Verifying Solaris Package Installation

Do this in on Solaris 8 update 5 and Solaris 9.

  1. Verify the version of the JRE installed as:
  2. java -fullversion

    java full version "JDK-1.2.2-W"

  3. Install the four JavaHelp Solaris packages.

    Verify proper installation against:

  4. Test demo programs activation for installed program group.
  5. Test jhindexer.

Full Functionality Tests

These tests are a complete test of the functionality of JavaHelp. They rely heavily on the JavaHelp demos as the testing mechanism.

  1. Test hsviewer. (short)
  2. Test hsviewer. (full)
  3. Test apiviewer.
  4. Test IdeDemo.
  5. Test merge.
  6. Test object.
  7. Test jhindexer.
  8. Rebuild IdeDemo

Short Functionality Tests

These tests are an abbreviated test of the functionality of JavaHelp. They rely heavily on the JavaHelp demos as the testing mechanism. Short Functionality tests are executed as a sanity check that the installation proceeded correctly. They test only a minimal amount of the functionality. In all cases the Full Functionality test has already been executed on the platform and it is only necessary to retest with the short test.

  1. Test hsviewer. (short)
  2. Test apiviewer.
  3. Test jhindexer.

Short Test for HSViewer (JavaHelp User's Guide)

This tests that the main information available to the users in the release is available. It also tests that the basic JavaHelp machinery is OK.

  1. Start HSViewer with no arguments: you should get the JavaHelp User's Guide HelpSet up. There are three navigators: TOC, Index and Search
  2. Try all the entries in the JHUG TOC navigator.
  3. Collapse/expand some nodes.
  4. Check its contents, including LICENSE and COPYRIGHT.
  5. Verify that FORWARD and BACKWARD arrows work as desired (should not be active until there is something that can be done with them).
  6. Try some entries in the JHUG Index navigator.
  7. Try some queries in the JHUG Search navigator.
  8. Click on some returned values
  9. Exit hsviewer

Long Test for HSViewer

This tests that different ways of locating and accessing at HelpSet information are working. Most of these tests could be done using the "Merge" application, but this application came quite late, so we are sticking to this mechanism.

The welcome documentation lists a number of other helpsets that can be tried, some of them are given as JAR files, some of them as directories. Each entry should be tested. Here is the list:

welcome.hs doc\welcome
HolidayHistory.hs demos\hsjar\holidays.jar
HolidayHistory.hs demos\hs\holidays
IdeHelp.hs demo\hsjar\idehelp.jar
api.hs demos\hsjar\apidoc.jar;demos\lib\classviewer.jar
HolidayHistory.hs demos\lib\jhsearch.jar

Go to File|Open Page and look at some .html page.

Testing APIViewer

This tests the extension mechanism for navigators. It also tests that the users can navigate through the API documentation.

  1. You should get API viewer.
  2. There may be an issue with the vertical separator between the Navigators and the content.
  3. Select a class, select some methods/constructors
  4. Repeat for a few classes and interfaces.
  5. Test with Index navigator
  6. Search on the Full-Text Search.
  7. Exit APIViewer

Testing IdeDemo

This tests Context-Sensitive Help, Embedded Help, multiple independent Help viewers.

  1. Launch IdeDemo. There should be one Frame split into two panes. The Frame has a toolbar.
  2. Select Help->Demo JDE - Help. After a few seconds the HelpViewer on the IdeHelp HelpSet will show up. Check that the TOC is alive by collapsing and compressing some; check that the search database is working by querying on "breakpoints"; there should be around 25 hits.
  3. Close the viewer.
  4. Try again; it should appear much faster.
  5. Now try Help->API Reference. Should open a new viewer. Check that it is alive by expanding and collapsing the TOC.
  6. Close the viewer.
  7. Click on the "?" button. The cursor should change to one including a ?. A known bug in Solaris makes this happen only the first time; it does not happen at all on win32. Now click on the Save Icon in the toolbar; the Help viewer for IdeHelp will show on the JDE Main Toolbar.
  8. Now try Window->Class inspector. A TOC-like Navigator (actually the ClassViewer Navigator) without the Viewer will show in the Desktop pane. Select a class, select a method. The content will be shown in the tabbed lower pane.
  9. Now select the Search Navigator. Type in the phrase compiling a java source file, you should get a hit.
  10. File->Exit. Click on Help. Should show help.
  11. File->Exit again. Click OK and exit.

Testing Merge

This tests Merging HelpSets. Also it provides an interface to locating HelpSets and the such.

  1. Start Merge.
  2. Click on Locate HelpSet. This will locate HolidayHistory.hs
  3. Click on AddHelpSet
  4. Click on show MasterHelp. This should show a HelpSet viewer that has TOC, Index, and Search corresponding to the "master" helpset, plus the holidays.
  5. On the TOC, expand and contract some nodes.
  6. Select some nodes. Beware of selecting the intro of the HolidayHistory since there is a bug somewhere that raises an exception.
  7. Select the Index, do the same.
  8. Select the Search, search for holidays; search for helpset
  9. Back to the main Window, change the HelpSet name to say "IdeHelp.hs". Locate the Help Set, then add it.
  10. The HelpSet viewer should show how the HelpSet for IdeHelp is added dynamically.
  11. Collapse, expand, and select an entry in the new portion of the TOC and in the old one.
  12. Back to the main Window, select the holidays helpset from the list; then click on "Show Selected HelpSet".
  13. A new HelpSet viewer will be shown with the content only of the Holidays helpset. Navigations on both viewers should be independent.
  14. Exit Merge

Testing Object

This tests components within HTML content.

  1. Start Object.
  2. Click on Text Flyover Demo. This will bring up the flyover demo
  3. Place the cursor over the work object
  4. Click on Popup Window Demo in the TOC. This will bring up the popup demo
  5. Click on the Help Button. The HTML text appears in a popup.
  6. Exit Object

Testing JHIndexer

This tests building a Full Text Search.

  1. cd to doc/jhug directory
  2. Delete the directory JavaHelpIndex if it exists.
  3. Execute jhindexer with the command

    "../../javahelp/bin/jhindexer -verbose -db JavaHelpIndex2 .".

  4. Test the Index with the following command

    "../../javahelp/bin/jhsearch JavaHelpIndex2"

Testing Building IdeDemo:

This tests that the users can rebuild the IdeDemo example.

  1. See instructions in demos/README.

Web page Verification

Home page

Product Marketing owns this

Release

Product Marketing owns this. I don't remember where the "in-progress" pages live. These are the installed pages:

Things to check


1. Install Bundles come with Swing1.1beta. Using a different Swing requires intervention.

2. Any Solaris will do. The contents is also tested implicitly with every full test.