...
- Do we currently have all the right tools that we want (and preferably are we cooperating on tool development and not duplicating where appropriate)?
- Have we defined all the tests that we want in the right contexts (e.g. test for eduGAIN compliance vs local federation compliance vs general SAML compliance)?
- Tools vs instances, what do we want, where?
- Who is running the test?
- How are we promoting these to users? Can people find the tools right now?
- What happens when a service instance flags something as red?
- What reports are being delivered to the testers? Can these be standardised / combined into a larger grade report? Something like the AAF report? https://aaf.edu.au/wp-content/uploads/2015/04/AAF_example_sum_report.pdf
...