Testing Best Practices for Application Packaging

Here is one of the best testing processes for application packaging. By doing all these tests there won't be a chance to miss anything.

Project Build and Validation Test

This test is performed on the development system using Wise Windows Installer Editor.

It is necessary to perform an MSI validation on the package. Internal consistency evaluator (ICE) errors and warnings that are reported during the validation process should be resolved.

Installation Test

The installation test initializes any run time errors during the installation.

Run the repackaged installation.

If errors occur, record the errors and number of the Windows Installer errors.

On your development system use editor to fix the errors.

Do the validation test again, if errors occur open in editor and solve the errors.

When the installation completes successfully the test in completed.

Production Functionality Test

The product functionality test verifies how the application works after installing with repackaged installation (MSI).

Verify that each shortcut created by the repackaged installation works fine.

Verify the main menu items of the application that works as specified in the smoke test or acceptance test. If not provided any test document then perform the regular application test by opening the application, clicking on Open, New, Save, Save As, Help etc...

Check file that associations have been set up correctly by launching a file with any of the extensions which belong to that application.

Record any errors with this test and resolve it by opening the application in Editor. (For this kind of errors please do the Legacy difference test.)

Uninstallation Test

This test verifies clean uninstallation and error free uninstallation.

Uninstall the repackaged installation, if an error occurs, record and open the editor and solve it. (Files, Folder, regkey, shortcuts not uninstalling etc..)

Re-image the test build and install the application again.

And uninstall the repackaged application again.

Ensure the uninstalltion completes successful.

Reboot the machine and check for any startup errors or windows functionality errors.

Legacy Differences Test

To perform this test you need software like Installrite or Picturetaker.

Installrite software is free and it can be downloaded from http://www.epsilonsquared.com/installrite.htm

The legacy difference test captures any files or registry data installed by legacy installation (.Exe) but which are missing from the repackaged installation (.MSI).

This test is useful for finding the problems if the application is not working as expected.

This is to be performed on a clean machine.

Install the repackaged application (MSI) first.

Open the Installrite software to take initial capture of the system status.

Select (Click on) Create a manual "snapshot" of your PC

Click to view.

Run legacy installation (Exe).

Open the Installrite software again and take a second snapshot, this will capture the changes between the repackaged installation and legacy installation.

Then click on the "Perform an "analysis" of the changes since the last snapshot"

Click to view.

Save the report.

Click to view.

Click OK to Continue

And the report will be opened automatically, if its not opened then Go to Review installations, the report of the changes will be opened.

Click to view.

If you click on Review Installation then the report will be opened

Click to view.

By this report you can easily identify the missing data.

Like this if any file or registry key is missing then Open the project in editor and update the repackaged installation with any missing file or registry keys etc...

After finishing, save and compile the application, then the changes happen to the to project.

Validate and correct any errors.

Uninstall the application and do the Legacy differences test again.

Application Clean-up Test

Applications clean up test check if the repackaged installation leaves behind any unexpected data, files registry keys after uninstallation, or deletes or modifies any data that is not related to the application.

Usually files that are in a component marked as permanent, for example components with a destination of system Folder are normally marked as permanent, should remain on the target system after uninstallation.

Open Installrite select (Click) Create a manual "snapshot" of your PC to capture the initial system status.

Click to view.

Click on Create a manual "Snapshot" of your PC.

Then...

Install the repackaged application.

Launch all the shortcuts.

Uninstall the repackaged application.

Open Installrite again.

Click on the "Perform an 'analysis' of the changes since the last snapshot" to capture the changes effected by uninstallation.

Click to view.

Click on Perform an "Analysis" of the changes since the last "Snapshot".

Save the report.

Click to view.

Give a name and click on OK.

Click to view.

Click on Review Installation.

Then it will show the changes done by the application to the system or to other applications etc...

Click to view.

Then it will shows the changes.

If needed Clean up and modify the repackaged installation in editor.

Save and compile the MSI.

Revalidate and correct any errors.

Re-image the test machine and repeat the test to check unexpected data remain on the target system after uninstallation.

Different User Test

The purpose of the difference user test is to check that the application functions normally, when a different user logged on to the system. This test can also identify problem with an application running from an account with limited privileges.

Create a new user account on the test machine.

Install application as an admin, but do not launch the application.

Log off and log on as a new user.

Check that each shortcut starts the application correctly.

Fix any errors using editor.

Save and compile MSI package.

Revalidate and correct any new errors.

Re-image the test machine and repeat the test.

Advertisement Test

The advertisement test checks that the application functions normally when installed using the windows installer advertisement (On demand Installation) feature. Ideally, these tests are performed on a clean machine.

Advertise the repackaged application by running it from the command line running msiexec with /jm switch.

Msiexec /jm path to .msi.

Execute the application shortcut from the programs folder in the start menu.

Check that the application installs and starts correctly.

Correct any errors using editor.

Save and compile the MSI package.

Revalidate and correct any errors.

Re-image the test machine and repeat the test.