By default DSX Discovery Service lists all eduGAIN IdPs for user to select from, which is a list of several thousand IdPs. Showing all IdPs often is not desirable. The DSX Discovery Service allows to filter out IdPs so that only the relevant IdPs are shown in the list. This allows an SP admin to create a list (or several lists) specifically targeting the user base of his SP. If for nothing else, the filtering should be used to hide IdPs not meant to be shown for 'normal' end user i.e. IdPs having entity category http://refeds.org/category/hide-from-discovery.

Creating the filter using the DSX DS Filter Generator

The filter is a base64 encoded JSON object that defines the rules for including or excluding IdPs. The easiest way to generate this is by using DSX Filter Generator.

https://dsx.edugain.org/filter

The filter generator can create two types of filters, you may filter entities based on their SAML entity categories or based on IdP entityID values (or both).

Allow and Deny lists of Entity Categories

You may create both, allow and deny, lists of entity categories.

If an "allow" list is defined, only IdPs matching at least one of the categories are visible, all others are hidden.
If a "deny" list is defined, all IdPs matching any of the categories on it are filtered out and thus are hidden.

Allow or Deny list of IdPs

You may create allow or deny list of IdPs.

If an "allow" list is defined, IdPs that are not on this list are filtered out and thus are hidden
If a "deny" list is defined, all IdPs on it are filtered out and thus are hidden

This example shows how to list all IdPs not tagged as hidden. This is the main use case to setup list of all eduGAIN IdPs.

Navigate to filter generator on https://dsx.edugain.org/filter

Click open the 'select entity categories' - accordion. You should now see all the possible entity categories to choose from in a grey box.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.31.28.png

Move http://refeds.org/category/hide-from-discovery to red box.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.31.48.png

At the bottom of the page you should see the resulting filter and its plaintext version.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.32.03.png

Show all IdPs not belonging to category http://refeds.org/category/hide-from-discovery

This example shows how to list only IdPs tagged with Research and Scholarship entity category.

Navigate to filter generator on https://dsx.edugain.org/filter

Click open the 'select entity categories' - accordion. You should now see all the possible entity categories to choose from in a grey box.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 8.34.04.png

Move http://refeds.org/category/research-and-scholarship to green box. Move also http://id.incommon.org/category/research-and-scholarship.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 8.34.41.png

At the bottom of the page you should see the resulting filter and its plaintext version.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 8.35.11.png

Show only IdPs having atleast one of the categories http://refeds.org/category/research-and-scholarship and http://id.incommon.org/category/research-and-scholarship.

This example shows how to list a specific set of IdPs.

Navigate to filter generator on https://dsx.edugain.org/filter.

Click open the 'select individual IdPs' - accordion. You should now see all the possible IdPs to select from.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 9.01.33.png

Make sure that the radio - button 'Selected IdPs will be visible' is checked. Select the entities you want the user to pick the IdP from.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 9.01.59.png

At the bottom of the page you should see the resulting filter and its plaintext version. Note that by picking individual IdPs you will easily end up with a large filter and you need to set it by reference.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 9.02.36.png

Show only IdPs https://idp.aalto.fi/idp/shibboleth, https://birk.wayf.dk/birk.php/wayf.au.dk and https://shibboleth.aber.ac.uk/shibboleth-TST.

Merging Two Filter Types

When both entity categories and IdP list filters are used together, the list shown comprises of IdPs not filtered out by categories filter with possible additions of IdP allow list or possible deductions of IdP deny list. The assumption is that SP creates the base of the rules using categories filtering and then possible exceptions to those rules by Allow/Deny lists of IdPs.

This example shows how to list all IdPs tagged with Research and Scholarship entity category and a specific set of IdPs.

Navigate to filter generator, https://dsx.edugain.org/filter.

Click open the 'select entity categories' - accordion. You should now see all the possible entity categories to choose from in a grey box.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.48.22.png

Move http://refeds.org/category/research-and-scholarship to green box. Move also http://id.incommon.org/category/research-and-scholarship.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.49.04.png

Click open the 'select individual IdPs' - accordion. You should now see all the possible IdPs to select from.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.49.18.png

Make sure that the radio - button 'Selected IdPs will be visible' is checked. Select the "Aalto" entity (not belonging to mentioned categories at the time of the writing).

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.49.29.png

At the bottom of the page you should see the resulting filter and it's plaintext version.

eduGAIN > Filtering IdPs > Screen Shot 2017-08-23 at 12.49.51.png

Show https://idp.aalto.fi/idp/shibboleth and any IdP having at least one of the categories http://refeds.org/category/research-and-scholarship and http://id.incommon.org/category/research-and-scholarship.

Applying the Filter in the Discovery Request

The filter generated with the tool is appended as an HTTP GET parameter to the discovery request. It can be set either by value or by reference. The maximum length for discovery request query parameters by some browsers is 512 bytes. If the filter is exceeding this value one should the filter by reference.

Filter by value - filter

https://dsx.edugain.org/wayf.php?filter=eyJhbGxvd0hvc3RlbCI6dHJ1ZSwiYWxsb3dIb3N0ZWxSZWciOnRydWV9Cg==

<SSO
    discoveryProtocol="SAMLDS"
    discoveryURL="https://dsx.edugain.org/wayf.php?filter=eyJhbGxvd0hvc3RlbCI6dHJ1ZSwiYWxsb3dIb3N0ZWxSZWciOnRydWV9Cg==">
    SAML2 SAML1
</SSO>

'default-sp' => array(
    'saml:SP',
    'entityID' => 'https://sp.example.com/simplesaml/',
    'idp' => NULL,
    'discoURL' => 'https://dsx.edugain.org/wayf.php?filter=eyJhbGxvd0hvc3RlbCI6dHJ1ZSwiYWxsb3dIb3N0ZWxSZWciOnRydWV9Cg==',
    'privatekey' => 'example.key'
),

Filter by reference - efilter

https://dsx.edugain.org/wayf.php?efilter=www.example.com/filter

where contents of www.example.com/filter would be a plain text document containing the filter, for example:

eyJhbGxvd0hvc3RlbCI6dHJ1ZSwiYWxsb3dIb3N0ZWxSZWciOnRydWV9Cg==

<SSO
    discoveryProtocol="SAMLDS"
    discoveryURL="https://dsx.edugain.org/wayf.php?efilter=www.example.com/filter">
    SAML2 SAML1
</SSO>

'default-sp' => array(
    'saml:SP',
    'entityID' => 'https://sp.example.com/simplesaml/',
    'idp' => NULL,
    'discoURL' => 'https://dsx.edugain.org/wayf.php?efilter=www.example.com/filter',
    'privatekey' => 'example.key'
),

More complex filters

The filter language supports scenarios that are not currently covered by the filter generator, but that can be manually constructed. Similarly, it is possible to programmatically generate your own filters by referencing a script hosted at the efilter location. In both cases, you create a filter by generating an appropriate JSON object and then base64 encoding it.

A good starting point is to get the filter as close as possible to your needs using the filter generator at https://dsx.edugain.org/filter (e.g. honouring hide-from-discovery). You can then take the resulting JSON object show as the "Human readable form of filter" and further customise it further to your needs.

It is possible to filter IdPs by their registration authority (the federation that they come from). This may be useful when you want to list only the IdPs from a specific subset of eduGAIN that are not already identified by an entity category (for instance, only from certain countries).

The registration authority is usually specified as a URL identifying the federation operator. You can determine the correct one either from the federation's entry on the eduGAIN technical site or by examining metadata for the <mdrpi:RegistrationInfo registrationAuthority=""> element.

For instance, the South African Identity Federation uses a registration authority of https://safire.ac.za. To list only IdPs from this federation, you'd need to generate a JSON filter object like this:

{"ver":"2","allowFeeds":{"eduGAIN":{"allowRA":["https://safire.ac.za"],"denyEC":["http://refeds.org/category/hide-from-discovery"]}}}

which would then be base64 encoded to produce a filter:

eyJ2ZXIiOiIyIiwiYWxsb3dGZWVkcyI6eyJlZHVHQUlOIjp7ImFsbG93UkEiOlsiaHR0cHM6Ly9zYWZpcmUuYWMuemEiXSwiZGVueUVDIjpbImh0dHA6Ly9yZWZlZHMub3JnL2NhdGVnb3J5L2hpZGUtZnJvbS1kaXNjb3ZlcnkiXX19fQ==

You can also remove IdPs from a specific federation in a similar way, by using the denyRA keyword.