Dry Running Test Scripts

Vault Validation Management allows test script authors to request that users dry run a specific test script to verify its accuracy prior to submission for pre-approval and test script execution. From the user interface, assigned dry run executors can provide comments on the test script and its test steps, identifying scripting errors or suggesting improvements. Upon completion of the dry run, test script authors are prompted to review comments, resolve or reject feedback, and make adjustments to the test script as part of the authoring process.

The Test Dry Run Interface is not available by default and requires an Admin to configure the Test Script object to accommodate the execution of dry runs.

About the Test Dry Run Interface

When you perform the Start Dry Run action on a test that is in the In Authoring or Dry Run Authoring Review lifecycle states, the Test Dry Run Interface becomes available to the assigned executor.

Test Header

This section includes the Test Script ID if available or the Name with the Title of the test script being authored. If the test name and title are too long to display, you can hover over the header to view the full name and title.

The navigation panel on the left side of the interface allows you to select either the Step View or Comments View.

Step View

Similar to the Test Authoring Interface, this view displays sections, setup steps, and execution steps of the test script. Click on a step or section in this panel to navigate directly to it in the content panel. You can collapse and expand sections to efficiently navigate through large numbers of test steps.

Comments View

This view displays comments captured during the dry run. From this view, you can add a comment for the Test Script or select a specific test step in the content panel to add a comment. Once a comment is saved, it is stored in the Comments View and you must click View more to expand the comment. Click the ellipses at the top of a saved comment to Edit, View Details, or Delete.

Click the ellipses at the top of the Comments View to Expand All or Collapse All comments.

Click the filter icon to filter the comments by their status. Click Apply to set the filter options, or Reset to remove them.

Content Panel

Similar to the Test Script Execution Interface, the content panel in the lower right portion of the interface contains the Setup Steps and Execution Steps which you will dry run. The content panel displays up to 25 steps per page.

To add a comment for a specific test step, select the test step in the content panel and populate the comment from the Comments View.

Status & Action Bar

The top right of the interface includes a next step locator icon which takes you to the next incomplete step in the Test Script.

The Complete button is available at all times and Vault does not require you to complete all test steps in order to complete the dry run.

Performing a Dry Run

To dry run a test step as the assigned executor:

  1. Navigate to a Test Script record in the In Dry Run lifecycle state.
  2. From the Test Dry Run Interface, execute test steps in any order, providing attachments and verdicts as needed. Vault does not require you to complete all test steps in order to complete the dry run.
  3. Add comments where necessary.
  4. Click Complete to finish the dry run.

The Test Script transitions to the Dry Run Complete state, where the author can review the comments from the dry run(s). The lifecycle state of all comments transitions from Draft to Submitted.

Adding Comments

To add comments during a dry run:

  1. From the Comments View, select a Type from the drop-down. Alternatively, select a specific test step from the content panel to create a comment for that step.
  2. Optional: If you are creating a comment for a test step, select a Step from the drop-down of existing test steps. If you select a test step from the content panel, Vault automatically populates this field.
  3. Optional: If applicable, select a Prompt.
  4. Select a Category to provide context for the comment. For example, Scripting Error.
  5. In the text box, provide a comment.
  6. Optional: Select the upload icon to provide an attachment for the comment.
  7. Click Save.

Reviewing a Dry Run

When the assigned executor completes a dry run, the dry run Test Script enters the Dry Run Complete state. Vault displays read-only comments in the Test Authoring Interface from the dry run(s). The source Test Script may be in the In Authoring or Dry Run Authoring Review state. From the source Test Script, the author can review all saved comments in the Comments View and Mark as Resolved or Mark as Rejected using the provided buttons. From the content panel, edit each test step’s title, prompts, settings, or requirements as needed and according to the dry run comments.

Click Complete to complete your authoring and review or click Start Dry Run to execute another dry run for this test script. Once the dry run review is complete, the Test Script can proceed to pre-approval, and the dry run is captured in a section on the source Test Script record.

To use the Test Dry Run Interface, users require a security profile and a permission set which grants these minimum object and field-level permissions:

  • Validation Test Script Comment: Read, Create, Edit, Delete
  • All fields on the Validation Test Script Comment object: Read
  • Test Script: Read
  • Test Script Dry Run? and Source Test Script for Dry Run fields on the Test Script object: Read
  • Source Test Script for Dry Run Steps and Source Test Step for Dry Run fields on the Test Step object: Read
  • Source Prompt for Dry Run field on the Additional Prompt object: Read