You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Current »

The table below attempts to show some of the existing monitoring tools that are available for SAML entities and federations, what they focus on and what results they deliver. The idea is to look at where we are currently delivering tools, how we present them, where we can combine efforts and, most importantly, if we can combine results in a convenient and coherent report for entities.

Questions to ask:

  1. Do we currently have all the right tools that we want (and preferably are we cooperating on tool development and not duplicating where appropriate)?
  2. Have we defined all the tests that we want in the right contexts (e.g. test for eduGAIN compliance vs local federation compliance vs general SAML compliance)?
  3. Tools vs instances, what do we want, where?
  4. Who is running the test?
  5. How are we promoting these to users? Can people find the tools right now?
  6. What happens when a service instance flags something as red?
  7. What reports are being delivered to the testers? Can these be standardised / combined into a larger grade report?  Something like the AAF report? https://aaf.edu.au/wp-content/uploads/2015/04/AAF_example_sum_report.pdf
Check TypePurposeREFEDSeduGAINWiderWhen RunReport GivenComments
SAML Deployment Profile checksTo check compliance against SAML deployments in given contextsFedLab: SAML2Int (code)

eduGAIN metadata validator (service) - tests against the eduGAIN Metadata Profile for federation metadata

SAML2Int? - no test run against the SAML2Int SHOULD

FEDERATION

Fedlab: SAML2Int (code)

TestShib (service)

Testing during deployment process by IdPs and SPs.???metadata validator a different audience, not entity focused.
SAML Configuration CheckTo test specific elements of the way of SAML deployment is configured and whether it is operationalN/A

eduGAIN metadata validator (service)

eduGAIN Connectivity Check (service)

USER CENTRIC SITE

Fedlab: MCCS - Metadata Monitoring Service (code)Testing during service operation to flag operational issues. Connectivity check gives red / yellow / green warnings. Currently no action taken when flagged. 
Verify Entity CategoriesTo verify that entities are meeting requirements as laid out in entity categories (mostly R&S and CoCo at this stage).

FedLab: Entity Check (code)

Need R&S monitor?

CoCo Monitor (service)

BOTH

eduGAIN Attribute Release Check (in development - service)

BOTH

FedLab: Entity Check (code)Testing during service operation or testing when setting up an entity categoryCoCo shows a red / yellow / green flag.  Sends automated email to SP admin when it turns red. Can be used by entities or by a service operator (eduGAIN, federation). 
Check Attribute ReleaseTools to check that IdPs are releasing attributes / what attributes are being releasedN/AN/A

SWITCH Interfederation Attribute Check (service)

Foodle has a built in page which highlights what is being released (service)

In service.SWITCH tool gives a report showing fail / pass and shows other entities that have passed.  "Fail" is difficult in some contexts as the result might actually be correct for the implementation - only works locally.Difficult to get right as the IdP might be deliberately chosing not to release attributes to the SP.
Test IdP / Access CheckCheck to see if an SP works with a test IdPN/A

eduGAIN Access Check (service)

USER CENTRIC - SP

TestShib (service)

Feide OpenIdP and Metadata Edit (shutdown as of 1-Jan-2016)

Testing during deployment process by IdPs and SPs.?? 

Metadata Explorer

Human readable metadata and metadata searchMET

eduGAIN Entities (service)


BOTH

SMEV (service)

Pyff (service and code)

WAYF.dk Cantina (service)

 

General overview of metadata at any given time

Shows human readable metadata and reports against them.

Current implementations probably right for each environment.

Different than the other tools listed above

IsFederated

Checks to see if an organisation is federatedN?A

eduGAIN isFederated


USER CENTRIC  - SP


N/A 

Shows if a specific domain is using any given federation.

Different than the other tools listed above
  • No labels