The IdP/SP testbed is a modular collection of trust and identity software products. It provides the capability to select existing products to easily deploy and integrate them without requiring technical knowledge of the individual products. This allows the automated set-up of a complete test environment for the execution of integration tests.
Architecture Overview
The architecture of the testbed is based on the use of self-containing containers (Docker) for the software products (IdP, SP, Proxy). These containers are maintained by a product maintainer and made available at a location of their choice (e.g. Docker Hub). The key component of the testbed is a centrally managed repository (GitHub) that contains the configurations of all products, i.e. deployment and test instructions. A user only has to clone the central repository and can then select the required software products and have it deployed in a target environment (e.g. local or cloud). In addition, the testbed provides a way to run automated web-based tests using Selenium, which are tailored to the selected software products.
This approach makes the administration of the testbed very lightweight and gives developers and other maintainers the opportunity to make their own products easily available.
Roles
The following roles are involved in operating testbed components.
Product Maintainer
The maintainer is responsible for managing a single software product in the testbed. This includes the creation of a Docker container and its configuration according to the testbed specification.
The maintainer can either be the official developer of the product or any other trusted community members who is willing to take on this task.
Testbed Administrator
The administrator is the owner of the testbed GitHub repository. He is responsible for the general testbed configuration and the generic tests. The administrator can add new products to the testbed by creating corresponding subdirectories and passing them on to the maintainer.
Testbed User
The end user of the testbed. The testbed is freely available as an open source application.
The expected target group are developers of T&I software products or (federation) operators who want to perform an integration test.
Activities
The following activities describe the actions necessary to manage and use the testbed.
Manage the testbed
[01] package product
The maintainer packages his product in a container according to the Docker Specification.
[02] provide config
The maintainer provides a configuration file that contains all the information needed to deploy the Docker container.
[03] accept config
The Testbed Admin checks the configuration.
If the product does not yet exist in the testbed, the admin needs to create a suitable directory to grant access to the maintainer.
[04] update testbed
The product configuration is added to the testbed and is now available for use.
[08] manage
The testbed admin makes necessary changes to the general configuration, the testbed deployment script or the generic tests, if required.
Use the testbed
[05] fetch config
The user receives the complete testbed configuration by cloning the GitHub repository.
[06] deploy config
The user selects the desired software products and versions in the configuration.
[07] fetch product
Using the deployment script provided, the configured testbed can be deployed into any environment that is able to run Docker.
Testbed requirements
The architecture is based on docker. The SAML software to be tested should be available as self-contained docker images. However, we also need to make sure that they can interfederate with each other.
The requirements of this are the following.
Domain Name Resolution: There needs to be a mechanism for discovering what IP addresses got assigned to individual docker images. The docker images should have a local domain name that should be resolved. The initial proposal is to generate a hosts file and have the docker images use their pre-allocated slots.
Metadata Exchange: each docker image should make its metadata available for download on a pre-arranged endpoint. Furthermore, each docker image should know about the metadata of the other images. Therefore they either need to use a pre-arranged MDX service or they need to have some pre-arranged way for taking up metadata, i.e., via a mounted folder specifically for this purpose
The logs should be made available from outside of the docker images for investigation. For better debugging it is advised that debug mode is used.
Any IdPs should support a pre-arranged set of test users' credentials and released attributes, in order to support automated testing.
Test Automation
The testbed offers the capability to define and execute automated web-based tests using Selenium.
Generic test cases
The testbed provides a set of generic test cases to be used with every product in the testbed (Testbed supported test cases). These include standard authentication flows which are supported by every IdP/SP product, i.e. they can be used out of the box.
Testbed configuration
The testbed contains a general Selenium configuration that describes the generic test cases. The tests are using a standard flow, which expects a specific page sequence and defined HTML field identifiers (e.g. for username and password). The field identifiers are dynamic and can be replaced by including a products specific configuration file, which is provided by the product maintainer.
Product specific configuration
Every product directory should contain a configuration file that specifies the HTML field identifiers used by the product.
Product specific test cases
Each product may define its own test cases to test specific features. These test cases can be defined using the Behave/Selenium specification of the testbed and added to the product directory. They are then executed if the product is deployed in the testbed.
