AppDash
AppDash
VibeTestSupportAbout
Sign In
Marketplace Review

Reviewer Integration Testing Guide

VibeTest by AppDash · Version 1.0 · Linked from the Jira panel reviewer footer and the Marketplace review ticket.

VibeTest lets teams write tests in plain English or BDD format inside Jira, then execute them with a local AI agent and stream the results back to Jira — no CI pipeline required. This guide walks an Atlassian Marketplace reviewer through verifying the app, covering both the no-desktop panel path and the full end-to-end flow.

1 · What VibeTest does

ComponentWhere it runsRole
Forge app (this listing)Atlassian cloud (Forge-hosted)Stores tests and run results; dispatches jobs via Jira web triggers; presents the issue panel and the project settings page.
AppDash AI Link (free desktop app)The user’s own macOS machinePolls Jira via long-poll web triggers; passes test definitions to a local AI agent; streams step results and screenshots back to Jira.
AI agent (Claude Code or BYO CLI agent)The user’s own machineExecutes the test steps. AppDash operates no sub-processor — the user’s own agent runs the tests.
Key architectural fact

The Forge app has no third-party backend. There are no remotes, no external.fetch, no configurable-http-endpoint — the only egress is the bridge returning each test job to the user’s own machine, never to a server AppDash operates. The desktop companion inbound-polls Jira using web triggers — Jira never initiates a connection to the user’s machine, and each poll is authenticated per pairing over HTTPS. Because that bridge uses a dynamic web trigger, the app is not Runs-on-Atlassian eligible — and it doesn’t need to be: the trust story is the privacy posture, not a badge.

2 · What you need

To verify the panel UI only (no desktop)

Nothing extra — just a Jira Cloud site with VibeTest installed. The panel renders useful state and functional test editors even without the desktop companion.

To verify full end-to-end test execution

  1. A macOS machine (the desktop companion is macOS-only in this release).
  2. Claude Code installed: claude.ai/download (or any compatible CLI AI agent — see §4).
  3. AppDash AI Link desktop app: appdash.ai/download.
  4. A Jira Cloud site with VibeTest installed (your review site).
  5. The review credentials / test-site details supplied on the review ticket.
Credentials for the review ticket

The review ticket should include a test Jira site URL and any credentials needed to install and use the app in the review environment. If credentials are missing, contact the submitter via the ticket before beginning functional testing.

3 · Verification path A — panel UI without the desktop

This path confirms the app presents useful state and does not crash when the desktop companion is absent — the minimum bar for any reviewer.

  1. Open any Jira issue → click the “Tests” panel.The Welcome to VibeTest banner renders, plus a setup checklist (“To run tests, you’ll need: …”) and two test-creation cards: BDD Scenario and Free-Form Task. Pass if the banner and cards render without error.
  2. Click “BDD Scenario”.The BDD editor opens. You can type Given / When / Then steps; the form has Save and Cancel buttons. Pass if the editor renders and accepts input.
  3. Save the BDD test, then click “Run”.The panel shows a run card with status errored and a message indicating no paired desktop. This is correct behaviour — the error is informative, not a crash. Pass if the error state renders and the panel does not crash.
  4. Click “Free-Form Task” and save a plain-English test.Same behaviour — the editor opens, saves successfully, and the run shows the no-desktop error. Pass.
Panel-only verdict

If these steps pass, the panel UI is functional without the desktop. The app “performs as described” for users who have the panel installed but haven’t yet downloaded the companion — consistent with what the listing describes.

4 · Verification path B — full end-to-end execution

This path verifies that test runs execute, stream results, and return pass/fail verdicts to Jira. It requires a macOS machine and the desktop companion (§2).

  1. Install the desktop companion.Download AppDash AI Link from the download page. Open the .dmg, drag to /Applications, and launch.
  2. Install Claude Code (or configure a BYO agent).Install Claude Code from claude.ai/download and run claude --version to confirm it is on your PATH; the desktop app auto-detects it on launch. BYO CLI agents are supported via the desktop app’s Agents settings.
  3. Pair the desktop with your Jira site.In Jira: open any project → Project settings → VibeTest → Pair a device. A pairing code appears. In the desktop app, click Add Jira connection and enter the code. The Jira settings page then shows the device as paired.
  4. Run a simple BDD test end-to-end.Open a Jira issue → Tests panel → create a BDD test with a trivial scenario (e.g. Given I open a browser / When I navigate to example.com / Then the page title contains “Example”) and click Run. The run card updates in real time — steps appear with running / passed / failed status, a verdict renders when complete, and screenshots attach to the Jira issue. Pass if the verdict appears and the steps render correctly.
Timing note

The desktop app long-polls Jira (every 2–5 seconds), so after clicking Run there may be a 3–10 second delay before the desktop picks up the job. The panel live-polls and updates automatically — you do not need to refresh.

5 · Security & privacy summary

QuestionAnswer
Does the app call any third-party backend?No. There are no remotes, no external.fetch, and no configurable egress in manifest.yml — and no server AppDash operates. The only egress is the dynamic web triggers (the bridge) returning each test job/result to the user’s own paired desktop, authenticated per pairing over HTTPS. GDPR reporting uses privacy.reportPersonalData(), a first-party Forge helper — not external fetch.
Does Jira connect to the user’s machine?No. The desktop companion polls inbound via Forge web triggers (standard HTTPS to Atlassian). Jira never initiates a connection to the user’s machine.
Who runs the AI agent?The user’s own machine, using the user’s own AI agent and API key. AppDash operates no sub-processor for test execution.
What personal data is stored?Atlassian accountIds are stored in Forge KVS (issue association, device pairing, run history, audit log). The app implements the Atlassian Personal Data Reporting API (report:personal-data scope) — data is reported weekly and erased on GDPR close/uninstall requests.
Are secrets (API keys) stored in Jira?No. Secret values never leave the desktop; only secret names reach Forge (for autocomplete in the BDD editor). Values live in the macOS Keychain on the user’s machine.
What scopes does the app request?read:jira-work, write:jira-work, storage:app, read:jira-user, and report:personal-data (5 total). No admin scopes. The granular write:comment:jira / write:attachment:jira narrowing is not yet available — those scopes are Beta-state and forge lint requires the classic write:jira-work, which the app justifies at submission.
Runs-on-Atlassian — not eligible, by design

forge eligibility -e development against the real code reports HAS_EGRESS_WEBTRIGGER_MODULE=true: the Forge↔desktop bridge’s bridge-connect is a dynamic web trigger that long-polls and returns each test job to the off-platform desktop. Only static web triggers are RoA-eligible, and a per-run job dispatch cannot be static — so this is permanent and architectural. Off-platform execution on the user’s own Mac is the product. The trust story is the privacy posture: no third-party backend, no remotes, no external.fetch, all data stored only in Forge KVS (data residency stays pinnable), and execution + credentials on the user’s own machine. See the security model and privacy policy for more.

6 · Third-party dependencies

DependencyTypeRequired?Notes
AppDash AI LinkDesktop app (free)Required for test executionDistributed via GitHub Releases. macOS only. No account required.
Claude CodeCLI AI agent (Anthropic)Default agent; BYO supportedRequires an Anthropic API key. Any Claude Code-compatible CLI agent can be substituted in the desktop app’s Agents settings.
The app is useful without both dependencies

A reviewer (or end user) without the desktop companion can still open the panel, write tests in the BDD editor, and see the structured test definitions. The execution path (clicking Run) returns an informative error when no desktop is paired — it does not crash the panel.

7 · Contact & escalation

For questions during review, use the review ticket’s comment thread. For urgent issues or to request a live walkthrough, contact AppDash support or use the Marketplace partner contact on the listing.

The review ticket should include:

  • A test Jira site URL with the app pre-installed.
  • Credentials or an invite for the review account.
  • A test desktop + agent environment (optional — required only for path B).
  • A link to the GitHub Releases page for the desktop companion.
We’re here to help

If anything is unclear or the verification steps don’t match what you see, flag it in the review ticket. We respond within one business day and can arrange a screen-share walkthrough if needed.