Conformance Test Suite: version history and guidance

Data Holders - 14 Scenarios

1. Discovery Document
2. Ensure Infosec Endpoints Using MTLS
3. Ensure SSA Validation*
4. Create Client Registration
5. First Consent
6. Holder Of Key Resource Requests
7. Second Consent
8. Data Recipient Initiated Arrangement Revocation
9. Amending Existing Consent
10. Data Recipient Initiated Token Revocation
11. Data Holder Initiated Arrangement Revocation
12. Ensure Client Assertion Data in Requests
13. Retrieve and Update Client Registration
14. Removed Software Product

This test plan has several changes from version 5.0.1, as summarised below:

• When CTS makes a Pushed Authorisation Request, it will now send sector specific scopes.
• Resolved internal issues to improve stability of the product.
• Improved validation in scenario ‘Ensure Infosec Endpoints using MTLS’ for the ‘missing client certificate’ test step.

Accredited Data Recipients - 4 competencies

1. Dynamic Client Registration

2. Establishing consent

3. ADR to DH consent arrangement revocation

4. DH to ADR consent revocation

This test plan has been updated to conform with version 1.22.1 of the Consumer Data Standards, notably the FAPI 1.0 Phase 3 obligations. As part of these changes the test plan has:

• Added support for the conformance testing of the Authorisation Code Flow (ACF)
• Added support for JARM

Accredited Data Recipients - 4 competencies

1. Dynamic Client Registration
2. Get Accounts
3. ADR to DH consent arrangement revocation
4. DH to ADR consent revocation

This test plan has several changes from version 3.5.0, as summarised below:

• No new scenarios or competencies
• Support all non-retired Register API versions, including the Get Data Holder Brands Statuses endpoint

Data Recipients – Flexible Test plan - 1 Scenario, 4 core competencies.

1. Dynamic Client Registration (DCR)
2. Consent (using Get Accounts)
3. ADR to DH consent arrangement revocation (DR to DH)
4. DH Initiated Revoke Consent Arrangement (DH to DR)

Released on 30 March 2022

After the ADR has been activated in the CTS ecosystem, CTS waits for the ADR to engage on 4 core competencies to determine if the ADR’s software product can conform to the CDS and the Register Design. The 4 test competencies can be completed in any order.

Data Recipients - 9 scenarios

1. Dynamic Client Registration (DCR)
2. Once-Off Consent (Get Accounts)
3. Ongoing Consent (Get Accounts)
4. Once-Off Consent (Get Transactions)
5. Ongoing Consent (Get Transactions)
6. Revoke Consent Arrangement (DR to DH)
7. DH Initiated Revoke Consent Arrangement (DH to DR)
8. Token Revocation (DR to DH)*
9. PAR Extend Consent*

Released on 9 September 2021

No New Scenarios

CTS Test Scenarios have been updated to validate the usage of standardised Error Codes, where Participants have adopted  the Standardised Error Handling requirements, as per CDS 1.10+.

Where the Participant has adopted the Standardised Error Handling Requirements, CTS will validate that their responses for specific CTS induced failure conditions (e.g. InvalidArrangement etc.) conform to CDS 1.10+ requirements - which includes validation of HTTP Status Codes, Error schema and the usage of correct Standard Error code itself.

Where Participants have not started using the standardised Error codes in their responses, the CTS validation will revert to the existing logic as per Test Plan 3.3.

* Scenarios 8 and 9 are considered optional for data recipients in line with CDS 1.11.1

Data Recipients - 9 scenarios

1. Dynamic Client Registration (DCR)
2. Once-Off Consent (Get Accounts)
3. Ongoing Consent (Get Accounts)
4. Once-Off Consent (Get Transactions)
5. Ongoing Consent (Get Transactions)
6. Revoke Consent Arrangement (DR to DH)
7. DH Initiated Revoke Consent Arrangement (DH to DR)
8. Token Revocation (DR to DH)*
9. PAR Extend Consent*

No new scenarios

• Registration can be made with the CTS DH with phase 2 scopes (noting that the CTS DH doesn’t support most resource end points at this stage)
• The CTS will accept CDR Standardised Errors for early adopters, but will not validate their contents. The CTS still supports application-specific errors
• New ADR Flexible Testing beta test plan – contact CDRTechnicalOperations@accc.gov.au to express your interest in participating in the beta, or for more information

* Scenarios 8 and 9 are considered optional for data recipients in line with CDS 1.11.1

Data Recipients - 9 scenarios

1. Dynamic Client Registration (DCR)
2. Once-Off Consent (Get Accounts)
3. Ongoing Consent (Get Accounts)
4. Once-Off Consent (Get Transactions)
5. Ongoing Consent (Get Transactions)
6. Revoke Consent Arrangement (DR to DH)
7. DH Initiated Revoke Consent Arrangement (DH to DR)
8. Token Revocation (DR to DH)
9. PAR Extend Consent

No additional scenarios

Modifications have been made to be compliant with the following two Register design changes described in Release notes v1.5.0:

• Added “profile” to the registration SSA response
• Added “profile” and “request_object_signing_alg” to the registration response provided from CTS back to the ADR

One user to many software products

The CTS implemented a feature which will allow users with the ‘Authorised CTS Tester’ role in the RAAP to access all CTS instances under their organisation(s) e.g. brands. When the user logs into the CTS participant UI, they will be able to pick a brand or software product from the CTS user interface, and they will be navigated to that instance.¹

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

The Data Holder test plan 4.3.0 updates the ‘Ensure Holder of Key for Resource Requests’ scenario to be sector agnostic. This test plan will allow data holders to demonstrate conformance with version 1.24.0 of the Consumer Data Standards.

This test plan will use Common APIs instead of sector specific resource APIs.

The CTS Simulated ADR will now call the Data Holder’s Get Customer endpoint instead of the Get Accounts & Get Energy Accounts endpoints.

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

This test plan version has been updated to conform with version 1.23.0 of the Consumer Data Standards. As part of these changes the Data Holder test plan has:

• Removed id_token encryption parameters from Dynamic Client Registration request for Authorisation Code Flow
• Updated validation of the ID Token during authorisation

The following Register API versions have been retired in this version of the test plan:

• Get Software Product Statuses V1
• Get Data Recipient Statuses V1
• Get Data Recipients V1 & V2

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

This test plan version has been updated to conform with version 1.23.0 of the Consumer Data Standards. As part of these changes the Data Holder test plan has:

• Removed id_token encryption parameters from Dynamic Client Registration request for Authorisation Code Flow
• Updated validation of the ID Token during authorisation

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

The following API versions have been retired and are no longer supported by CTS:

• Get Software Product Statuses V1
• Get Data Recipient Statuses V1
• Get Data Recipients V1 & V2

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

This document covers the guidance related to the Data Holder (DH) Test Plan (TP) version 4.1.0.

This test plan has been updated to conform with version 1.21 of the Consumer Data Standards, notable the FAPI 1.0 Phase 3 obligations. As part of these changes Data Holder test plan has:

• Added conformance testing for the Authorisation Code Flow (ACF)
• Added support for JARM
• Removed id token encryption in Authorisation Code Flow (ACF)
• Removed conformance testing for OIDC Hybrid Flow

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

This test plan has been updated to include bug fixes for the following scenarios:

2. Concurrent Consent

7. Amending Existing Consent

Data Holders - 12 Scenarios

1. Dynamic Client Registration

2. Concurrent Consent

3a. Get Software Product Status Register Polling

3b. Get Data Recipients Register Polling

4. Ensure Infosec Endpoints Using MTLS

5. Ensure Holder of Key for Resource Requests

6. Ensure Client Assertion Data In Token Request

7. Amending Existing Consent

8. Removed Software Product

9. Data Holder Initiated Revocation

10. Data Recipient Initiated Revocation

11. Data Recipient Initiated Token Revocation

This test plan has been built as a sector agnostic test plan to suit the needs of current and future sectors. Updated to conform with CDS standards 1.16.0 & FAPI 1.0 Phase 2.
Rationalised scenarios and the following have been removed or merged:

1. Discovery Document validation
2. Reactive Software Product
3. Register PUT GET
4. Amending Account for an Existing Consent Scenario with PAR
5. Consent Software Statement Assertion with Sector Identifier URI

Data Holders - 17 scenarios

1. Discovery Document Validation
2. Dynamic Client Registration (DCR)
3. Concurrent Consent
4. DH Initiated Revocation
5. DR Initiated Revocation
6. Removed Software Product
7. Reactivate Software Product
8. Replace Existing Consent with PAR Scenario
9. DR Initiate Token Revocation
10. Register PUT GET
11. Get Software Product Status Register Polling
12. Get Data Recipients Register Polling
13. Ensure Client Assertion Data In Token Request
14. Amending Account for An Existing Consent Scenario with PAR
15. Ensure Holder of Key
16. Ensure Infosec Endpoints Using MTLS Authentication with X509
17. Consent Software Statement Assertion with Sector Identifier uri

No new scenarios

Updates on CTS Register endpoints to be consistent with CDR Register endpoints, no operational process change. Details of CTS Register endpoint changes are listed in the CTS technical guidance:

Updated DCR endpoint – from CTS CDR Register via JWKS Endpoint to CTS CDR Register via the SSA JWKS Endpoint  

Data Holders - 17 scenarios

1. Discovery Document Validation
2. Dynamic Client Registration (DCR)
3. Concurrent Consent
4. DH Initiated Revocation
5. DR Initiated Revocation
6. Removed Software Product
7. Reactivate Software Product
8. Replace Existing Consent with PAR Scenario
9. DR Initiate Token Revocation
10. Register PUT GET
11. Get Software Product Status Register Polling
12. Get Data Recipients Register Polling
13. Ensure Client Assertion Data In Token Request
14. Amending Account for An Existing Consent Scenario with PAR
15. Ensure Holder of Key
16. Ensure Infosec Endpoints Using MTLS Authentication with X509
17. Consent Software Statement Assertion with Sector Identifier uri

No new scenarios

CTS Test Scenarios have been updated to validate the usage of standardised Error Codes, where Participants have started adopting the Standardised Error Handling requirements, as per CDS 1.10+.

Where the Participant has adopted the Standardised Error Handling Requirements, CTS will validate that their responses for specific CTS induced failure conditions (e.g. invalid Software Product, Invalid Consent etc.) conform to CDS 1.10+ requirements - which includes validation of HTTP Status Codes, Error schema and the usage of correct Standard Error code itself.

Where Participants have not started using the standardised Error codes in their responses, the CTS validation will revert to the existing logic as per Test Plan 3.3.

Data Holders - 17 scenarios

1. Discovery Document Validation
2. Dynamic Client Registration (DCR)
3. Concurrent Consent
4. DH Initiated Revocation
5. DR Initiated Revocation
6. Removed Software Product
7. Reactivate Software Product
8. Replace Existing Consent with PAR Scenario
9. DR Initiate Token Revocation
10. Register PUT GET
11. Get Software Product Status Register Polling
12. Get Data Recipients Register Polling
13. Ensure Client Assertion Data In Token Request
14. Amending Account for An Existing Consent Scenario with PAR
15. Ensure Holder of Key
16. Ensure Infosec Endpoints Using MTLS Authentication with X509
17. Consent Software Statement Assertion with Sector Identifier uri

No new scenarios

• Each scenario has been modified to support November 2021 obligations for Phase 2 for non-major ADIs, including registration requests, which will be made using phase 1 and 2 scopes:
  • openid
  • profile
  • bank:accounts.basic:read
  • bank:accounts.detail:read
  • bank:transactions:read
  • bank:regular_payments:read
  • bank:payees:read
  • common:customer.basic:read
  • common:customer.detail:read
  • cdr:registration
• The CTS will confirm the non-disclosure of phase 2 consumer data* to inactive software products
• The CTS will confirm the successful disclosure of phase 2 consumer data* to an authorised client
• The CTS will accept CDR Standardised Errors for early adopters, but will not validate their contents. The CTS still supports application-specific errors

*Phase 2 consumer data will be sourced from the following end points: Get Account Detail, Get Direct Debits for Account, Get Bulk Direct Debits, Get Direct Debits for Specific Accounts, Get Scheduled Payments for Account, Get Scheduled Payments Bulk, Get Scheduled Payments for Specific Accounts, Get Payees, Get Payee Detail, Get Customer Detail

Data Holders - 17 scenarios

1. Discovery Document Validation
2. Dynamic Client Registration (DCR)
3. Concurrent Consent
4. DH Initiated Revocation
5. DR Initiated Revocation
6. Removed Software Product
7. Reactivate Software Product
8. Replace Existing Consent with PAR Scenario
9. DR Initiate Token Revocation
10. Register PUT GET
11. Get Software Product Status Register Polling
12. Get Data Recipients Register Polling
13. Ensure Client Assertion Data In Token Request
14. Amending Account for An Existing Consent Scenario with PAR
15. Ensure Holder of Key
16. Ensure Infosec Endpoints Using MTLS Authentication with X509
17. Consent Software Statement Assertion with Sector Identifier uri

No new scenarios

Each scenario has been modified to be compliant with the following two Register design changes described in Release notes v1.5.0:

• Added the definition that profile is part of the registration SSA scope
• Added the definition that software_roles is now optionally part of the registration response

Testing related to the “request_object_signing_alg” may be considered at a future date.

One user to many brands

The CTS implemented a feature which will allow users with the ‘Authorised CTS Tester’ role in the RAAP to access all CTS instances under their organisation(s) e.g. brands. When the user logs into the CTS participant UI, they will be able to pick a brand or software product from the CTS user interface, and they will be navigated to that instance.¹

Data Holders - 14 scenarios

• Discovery Document Validation
• Dynamic Client Registration (DCR)
• Concurrent Consent
• DH Initiated Revocation
• DR Initiated Revocation
• Removed Software Product
• Reactivate Software Product
• Replace Existing Consent with PAR Scenario
• DR Initiate Token Revocation
• Register PUT GET
• Get Software Product Status Register Polling
• Get Data Recipients Register Polling
• Ensure Client Assertion Data In Token Request*^
• Amending Account for an Existing Consent Scenario with PAR*

Two new scenarios*

^Conducts suite of bad requests against the token endpoint, to test error returned from DH

Data Recipients - 9 scenarios

• Dynamic Client Registration (DCR)
• Once-Off Consent (Get Accounts)
• Ongoing Consent (Get Accounts)
• Once-Off Consent (Get Transactions)
• Ongoing Consent (Get Transactions)
• Revoke Consent Arrangement (DR to DH)
• DH Initiated Revoke Consent Arrangement (DH to DR)
• Token Revocation (DR to DH)
• PAR Extend Consent

Data Holders - 12 scenarios

• Discovery Document Validation
• Dynamic Client Registration (DCR)
• Concurrent Consent
• DH Initiated Revocation
• DR Initiated Revocation
• Removed Software Product
• Reactivate Software Product
• Replace Existing Consent with PAR
• Token Revocation
• Register PUT GET
• Get Software Product Status Register Polling
• Get Data Recipients Register Polling

No new scenarios

However support for Register design 1.3.0 (new optional fields in the registration request, plus additional new OpenID scope)

Data Recipients - 9 scenarios

• Dynamic Client Registration (DCR)
• Once-Off Consent (Get Accounts)
• Ongoing Consent (Get Accounts)
• Once-Off Consent (Get Transactions)
• Ongoing Consent (Get Transactions)
• Revoke Consent Arrangement (DR to DH)
• DH Initiated Revoke Consent Arrangement (DH to DR)
• Token Revocation (DR to DH)
• PAR Extend Consent

No new scenarios

However support for Register design 1.3.0 (new optional fields in the registration request, plus additional new OpenID scope)

Data Holders - 12 scenarios

• Discovery Document Validation
• Dynamic Client Registration (DCR)
• Concurrent Consent
• DH Initiated Revocation
• DR Initiated Revocation
• Removed Software Product
• Reactivate Software Product
• Replace Existing Consent with PAR
• Token Revocation
• Register PUT GET
• Get Software Product Status Register Polling
• Get Data Recipients Register Polling

Data Recipients - 9 scenarios

• Dynamic Client Registration (DCR)
• Once-Off Consent (Get Accounts)
• Ongoing Consent (Get Accounts)
• Once-Off Consent (Get Transactions)
• Ongoing Consent (Get Transactions)
• Revoke Consent Arrangement (DR to DH)
• DH Initiated Revoke Consent Arrangement (DH to DR)
• Token Revocation (DR to DH)
• PAR Extend Consent