Use Visual Diff Thresholds
If you’ve defined a similarity threshold for visual diffs, Tugboat sets a “Pass” or “Fail” status on visual diffs where that threshold is defined. That status displays in the Tugboat Dashboard, and it reports to the linked git provider - for example, GitHub.
You may want to do a few things with visual diff thresholds:
- View pass/fail results in a linked git repository
- View pass/fail results in the Tugboat Dashboard
- Approve or Decline visual diffs
- Re-run visual diff similarity calculations
If you’re looking for documentation on how to configure visual diff thresholds, see: Configure Visual Diffs -> Set thresholds for pass/fail
View visual diff pass/fail results in a linked git repository
Pass results in a linked git repository
When you define a similarity threshold, visual diffs show as a “Pass” in the linked git provider if they generated successfully, and the visual diffs meet or exceed your similarity threshold.
When you don’t define a similarity threshold, successfully generating visual diffs shows as a “Pass” in the linked git repository. In this case, the “Pass” status doesn’t indicate anything about the results of the visual diffs - only that they’ve been created.
Fail results in a linked git repository
If you define a similarity threshold and the visual diffs don’t meet it, you’ll see the visual diffs as a fail in the linked git repository.
When you don’t define a similarity threshold, visual diffs shows as a “Fail” in the linked git repository when they don’t generate successfully. In this case, the “Fail” status doesn’t indicate anything about the results of the visual diffs - only that the process failed and needs to be debugged.
View visual diff results from a linked git repository
If you want to view the visual diff results, you can click the Details link to go directly to the visual diffs in the Tugboat Dashboard.
View visual diff pass/fail results in the Tugboat Dashboard
There are two ways to view the visual diff results:
- Click the Details link from the pull request in the git repository.
- View visual diffs from within the Tugboat Dashboard.
In this view, you’ll see a list of all the visual diffs that have been defined for the Preview at various breakpoints (Desktop, Tablet, Mobile), along with a similarity percentage for each visual diff.
If you have defined a threshold, and that threshold is not met, you’ll see an X and a message that the diff is “Less than {Your Threshold} similar”, indicating the visual diff is currently in a “failed” status.
From here, you can click into the screenshot at each breakpoint to view the visual diff for that breakpoint.
Approve or Decline visual diffs
When you have clicked into a screenshot at a specific breakpoint to view the visual diff for that breakpoint, you can manually change the visual diff status to Approve or Decline. You might want to do this to block or unblock merging the pull request.
If you set all visual diffs to Approve, the visual diff status in the linked git repository will change to a “Pass”. This will unblock merging a pull request. You might want to do this if all of the changes on the page are expected given the work in the pull request, even though the changes fall outside the normal visual diff similarity threshold.
If you set any visual diff to Decline, the visual diff status in the linked git repository will show as a “Fail.” Depending on the repository settings, this may block merging a pull request. You might want to do this if you spot a change in a visual diff that represents a bug or requires developer rework.
To approve or decline a visual diff:
- Follow our instructions to view visual diffs, clicking into the breakpoint where you want to change the Pass/Fail status.
- Click the Approve or Decline link.
You’ll see the visual diff results change to a message saying: “This visual diff has been approved.”
Once all of the visual diffs are approved, you’ll see visual diffs in a linked git repository change to a “Pass” status. Alternately, once any visual diff is declined, you’ll see visual diffs in a linked git repository change to a “Fail” status.
Re-run visual diff similarity calculations
Use the Regenerate option to generate a new set of visual diffs, and re-run the visual diff similarity calculation. You can regenerate a single visual diff, or an entire set of them.
To regenerate a single visual diff:
- Follow our instructions to view visual diffs, clicking into the breakpoint where you want to re-run the similarity calculation.
- Click the Regenerate link.
To regenerate a set of visual diffs:
- Click into a Preview that has finished building.
- Scroll down past the Services and Lighthouse Reports, and you’ll see the Visual Diffs pane.
- Click the Regenerate link.
