Conformance Test Suite for data holders

Conformance Test Suite process 

The Australian Competition and Consumer Commission (ACCC) manages the Conformance Test Suite, which is a key part of the Consumer Data Right on-boarding process. 

The Consumer Data Right Conformance Test Suite confirms the technical conformance of your production-ready software using a range of test scenarios targeting specific areas.

The Conformance Test Suite has two modes allowing for either provider (also known as participant) type, data holder and data recipient, to perform relevant tests. If you are a data holder, the tests provide a simulated data recipient and a simulated Register to support the test scenarios. You test in isolation against the simulated providers and the simulated Consumer Data Right Register, so you don’t interact with live consumer data.

The diagram below shows how data holders each interact with the Conformance Test Suite. 

The Conformance Test Suite is not a testing tool to assist you during the development of your software (see Participant tooling), rather, it is available to you during on-boarding, once registered as a data holder and before you become activated on the Register.

The Conformance Test Suite tests your conformance with the Consumer Data Standards before entering into the Consumer Data Right system. You should have a production-ready brand before undertaking the Conformance Test Suite.

Completing the Conformance Test Suite

The Conformance Test Suite is an automated testing suite and can be completed within an hour if all provider configurations are completed correctly and no errors are encountered. It is important to ensure your solution has been adequately tested before executing the Conformance Test Suite to minimise the test completion period.

If errors are encountered, the completion period can be lengthy (for example, days or weeks) as errors need to be diagnosed and resolved, and your solution made ready for retesting.

Conformance Test Suite (CTS) test scenarios for Data Holders 

The Conformance Test Suite for data holders comprises of a number of test scenarios that are crucial for ensuring compliance. These test scenarios include:

1. Discovery Document: This scenario tests the ability for a Data Holder to demonstrate that they can return a valid Open ID Discovery Document, adhering to CDS standards.
2. Ensure Infosec Endpoints Using MTLS: This scenario tests the ability for a Data Holder to validate that they can correctly handle an invalid certificate such as missing, self- signed, expired, or revoked client certificate received from CTS simulated Accredited Data Recipient.
3. Ensure SSA Validation: This scenario tests the ability for a Data Holder to validate that they can correctly handle an invalid or unapproved SSA received from CTS simulated Accredited Data Recipient.
4. Create Client Registration: This scenario tests the ability for a Data Holder to demonstrate that they can register the CTS simulated Accredited Data Recipient’s software product.
5. First Consent: This scenario tests the ability for a Data Holder to verify the consent flow with the CTS simulated Accredited Data Recipient utilising common API. The Data Holder receives a call from the CTS simulated Accredited Data Recipient to the ‘Get Customer’ endpoint with the Access Token as part of the consent.
6. Holder Of Key Resource Requests: This scenario tests the ability for a Data Holder to validate their compliance to the Holder of Key (HoK) mechanism when accessing MTLS secured endpoints. This includes validating authorised and unauthorised token types.
7. Second Consent: This scenario tests the ability for a Data Holder to verify the consent flow with the CTS simulated Accredited Data Recipient by granting two consent arrangements for a single Accredited Data Recipient (multiple CDR Arrangement IDs). The Data Holder receives a call from the CTS simulated Accredited Data Recipient to the 'Get Customer' endpoint with the Access Token as part of the consent.
8. Data Recipient Initiated Arrangement Revocation: This scenario tests the ability for a Data Holder to validate the arrangement revocation correctly when a customer withdraws their consent from the Accredited Data Recipient. To confirm this, the Data Holder will receive an arrangement revocation from the CTS simulated Accredited Data Recipient.
9. Amending Existing Consent:  This scenario tests the ability of a Data Holder to replace an existing consent arrangement with CTS simulated Accredited Data Recipient and correctly respond to a token request with an invalid refresh token.
10. Data Recipient Initiated Token Revocation: This scenario tests the ability for a Data Holder to validate the correct treatment of the withdrawal of a token, verifying that a Data Holder can receive a token revocation request from the CTS simulated Accredited Data Recipient.
11. Data Holder Initiated Arrangement Revocation: This scenario tests the ability for a Data Holder to validate that when a customer withdraws their consent from the Data Holder, the Data Holder handles the arrangement revocation correctly. To verify this, the Data Holder will send an arrangement revocation to the CTS simulated Accredited Data Recipient.
12. Ensure Client Assertion Data in Requests: This scenario tests the ability for a Data Holder to demonstrate that they can respond with appropriate bad request error(s) from CTS simulated Accredited Data Recipient. This may include missing or a poorly formed access token request while exchanging their Authorisation Code. An incorrect Client Assertion may contain wrong or missing AUD (audience), SUB (subject), ISS (issuer) or ALG (algorithm) in client assertion, or an expired Client Assertion, or a wrong signature, or wrong client assertion type in Client Assertion.
13. Retrieve and Update Client Registration: This scenario tests the ability for a Data Holder to demonstrate that they can modify and return the client registration when requested by CTS simulated Accredited Data Recipient’s software product.
14. Removed Software Product: This scenario tests that a Data Holder fulfils its responsibilities when CTS simulated Accredited Data Recipient’s Software Product status is changed to ‘Removed’.

The Conformance Test Suite: version history and guidance page provides in-depth information on each of these test scenarios.