Skip to main content

QA Process

MX follows this QA process for code changes to our mobile app:

  1. A developer completes a code change. Each code change:
    • Runs through unit tests maintained collectively by the developers.
    • Includes test instructions for QA.
  2. Another developer peer-reviews the code change.
  3. If the code change passes peer-review, it’s submitted to a QA engineer for mobile QA pre-merge testing to verify that:
    • Test instructions are clear and cover relevant user stories.
    • The change affects the intended change.
    • The code doesn't introduce unintended changes.
  4. If the code change passes mobile QA pre-merge testing, the developer merges the code change into the master branch.
  5. The same QA engineer triggers our build server to compile the app, including the new code. That build server process runs a suite of automated sanity and CRUD tests.
  6. If the build compiles successfully, the QA engineer reruns the tests to ensure behavior isn't changed by the branch context.
  7. QA performs a final end-to-end testing of the app on Android and iOS to ensure there are no new bugs or discrepancies across platforms.
    • Any bugs found during this process, whether directly related to the new code change or not, are discussed among QA, developers, and product to determine whether the bug’s severity and lift to fix mean that it must be addressed before continuing, or if it should be surfaced to the client and/or client consultant to determine priority.
  8. Test builds may be delivered to the client for review. The client then reports any issues using the in-app bug reporter, which submits issues directly to an intake repository where QA will see:
    • Details automatically included to aid in replicating the issue, such as the build ID, OS version, hardware model number, and limited logs from the app preceding the client user activating the bug reporter.
    • Additional info the client user can opt to include, such as screenshots, reporter name, and any helpful commentary.
  9. The app build is submitted to the store, by MX or the client, for publishing. Path QA Process

MX follows this QA process for Path changes that affect our mobile app:

  1. A developer completes a code change. Each code change includes test instructions for QA.
  2. Another developer peer-reviews the code change.
  3. If the code change passes peer-review, it's promoted from the developers’ sand environment to a QA-purposed environment where it's submitted to QA engineer testing to verify that:   
  • Test instructions are clear and cover relevant user stories.
  • The change affects the intended change.
  • The code doesn't introduce unintended changes.
  1. If the code change passes tests in the QA environment, the developer merges it into the integrations environment, where code changes can’t yet affect real world users, but can more accurately reflect our test connections to client integrated services.
  2. Repeat the tests from step 3.
  3. If the code change passes tests in the integrations environment, the developer merges it into the production environment for final verification.
  4. Repeat the tests from step 3.
  5. If the code change passes the previous tests, we rely on internal error monitoring, test users, and (as a last resort) external users to notify us of any relevant issues.