There are currently already several eduGAIN checking tools that examine the quality of an eduGAIN federation’s metadata, the connection status of their IdPs or the attribute release status of their IdPs. The results of these checks are public but only few federation operators seem to regularly consult the results. 
The eduGAIN Support team plays a pivotal role in trying to make federation operators and sometimes also entities aware of some of these issues, but as this is done manually and it is tedious work, they typically only act in case of complaints (e.g. UK/InCommon metadata import filters rejecting entities), which means not not all entities are engaged with and it is done so only irregularly.

It would be an improvement if all federation operators would receive a monthly email summarizing the state of their federation.  Therefor this activity proposes to create a tool that aggregates the combined result of all these checks into 1 comprehensive infographic per entity. This infographic should also show how well the entity is doing in relation to its peers. At the same time it should provide pointers to documentation on how to actually improve. A second infographic will generate an overview for federation operators, summarizing the per entity findings. This page should also compare their own federation with their peers.
The tool will put this information readily into the hands of the federation operators allowing them to decide if and how to pass it on towards the entities in their federation.

In addition, there is work ongoing to check for status of various entity categories, like CoCo, R&S and SIRTFI. This is currently happening in separate teams. Rather then reinvent the wheel and create yet another tool, we should investigate joining efforts.

Example infographic (not on entity status, but just for example: https://www.surf.nl/files/2019-03/gebruik_surfconext_2018.pdf)

• Collect the well know issues that need checking an reporting on w/ Lukas and members of eduGAIN support team and fedOPS generically
• Collect the various information sources (existing checking tools)  we have for creating these informational pages
• Design a layout for the two infographics
• Create a tool (and possible required backed checks)  to produce the infographic (Note
• Enable periodic emailing of infographics to fedOPS

• We may need to add some new checks
• We should discuss if such tools can be closely integrated with the existing eduGAIN technical pages
• We should engage with the eduGAIN support team to establish if they want additional checking and notification to take place
• We should collaborate with e.g. the team working on SIRTFI/CoCo checking
• We should engage with some people from e.g. marcoms to help us work on the layout and presentation of the infographics
• We should discuss this with the baseline expectation work in REFEDs

• Overall this would increase the awareness of which issues a federation has, it would increase the motivation to fix them and it thus would increase the overall eduGAIN service quality.
• By automating some of these task, we relieve the burden of the eduGAIN support team
• Also, federation operators would become aware of some of the existing eduGAIN checking tools, which would be beneficial also for their local federation

• Check tools too difficult to integrate with
• FedOps would still not act/care

This activity is done when:

• We have collected a list of things to include in the reporting and the sources for this information are understood
• We created a comprehensive design for the infographics
• We have a tool to create the infographics
• We have done outreach on the availability of this tool to relevant stakeholders

The activity was successfully completed after a total of two iterations. A prototype was developed that meets the expectations of the stakeholders. The tool, in future called "eduGAIN Reporting", was handed over to the eduGAIN Task. In order to develop this further, an eduGAIN reporting development team was founded. The created prototype is used for additional tests with the users and replaced by a productive system a few months later. The following resources were produced and delivered:

• Source Code
  • Ansible deployment
  • Web client
  • Server API
• Documentation
  • README in GitLab per repository
  • User documentation
  • Open issues
• Infrastructure
  • Production VM hosted by GÉANT
  • Test VM hosted by KIFU
  • Incubator domain: edugain-reporting-beta.incubator.geant.org