The FaaS Operations Team uses this document to adapt newly created instances of FaaS machines (cloned from the then current "template" machine) and make it specific for an NREN, using the information from FaaS instances configuration parameters.
Table of Contents
1. FaaS template instance
VM template.faas.geant.net is the FaaS template instance that holds the deployment of software stack and setup used for FaaS. Each customer instance should be cloned from the template instance.
The FaaS software stack consists of:
CentOS 6 GNU/Linux OS
Apache httpd web server w/ mod_ssl
- PHP 5.4 packages from the IUS Community Project
Jagger web application, root directory
MySQL database server used by Jagger
Postfix mail server used by Jagger
Pyff (Python SAML Metadata Aggregator), installed in
pyff pipelines (feed configurations) are in
FaaS custom scripts, placed in
Passphrase/PIN for HSM access, placed in
Shibboleth Service Provider (optional, to allow federated logins into the federation registry itself)
This guide describes configuration changes that need to be done for each instance cloned from the template instance and software update procedures.
Configuration steps that should be performed only on the production instances (and not on training or test) are labeled with the symbol .
2. Firewall configuration
All users should use HTTPS in order to communicate with the server, so the machine is visible from the public network only on HTTPS. Here is the list of allowed services:
- TCP port 443 opened to everyone
- TCP port 80 opened to everyone
- TCP port 25 should be open to the provided smarthost or if there is no smarthost to everyone
- TCP port 1792 opened to se-tug-hsm1.sunet.se
- TCP port 443 opened to everyone
- SSH access allowed only from PSNC management IP addresses
For outbound traffic TCP port 80 could be limited to http://mds.edugain.org/ and TCP port 443 could be removed for production use, but since
pip download packages via http/https we should leave these ports open.
For HSM access, we are currently using only se-tug-hsm1.sunet.se, but another HSM appliance in Denmark will be available at some point, allowing for geo-redundant access to the HSM service. So, when that happens we’ll need to update firewall configuration.
3. System changes
Change the root password on each new instance:
4. Software stack configuration
4.1. Apache web server
If you are using this deployment guide for creation of training instances, then skip the sections 4.1.1 and 4.1.2. For production instances you will need to follow the procedure for getting a new certificate since these instances will not use template wildcard certificate.
4.1.1. Creating TLS certificate
Based on the certificate information from the FaaS instances configuration parameters page, create a new private key and CSR. (For the exact values of supported/desired key size and hashing algorithm check FaaS instances configuration parameters.)
Replace the secret key in
/etc/pki/tls/private/webserver-key.pemwith the newly created one:
- Send a copy of the CSR to firstname.lastname@example.org. FaaS team will then forward the CSR to the customer for requesting the certificate.
After receiving the certificate from FaaS team, rename the certificate to webserver-cert.pem, replace the server certificate in
/etc/pki/tls/certs/webserver-cert.pemwith the new one and set appropriate permissions:
- Set up the certificate chain:
For NRENs that are using TCS, generate certificate chain by running the following command:
After you run
makefollow the further instructions for the available targets (arguments).
NRENs using other CAs (i.e., not TCS) should supply us with the appropriate certificate chain (or documentation). Rename the certificate chain to chain.pem, replace the certificate chain file in
/etc/pki/tls/chain.pemwith the new one and set appropriate permissions:
- In case of a CA without any intermedate CAs (e.g. MREN) simply comment out the
4.1.2. Testing web server TLS
Restart web server and test with the following command, replacing
HOSTNAME with the publicly visible server name also contained in the certiticate itself, and the referenced
CAfile with a copy of the root CA certificate for the CA used (e.g. TCS in the example below):
This should produce the output:
4.1.3. Apache httpd configuration
/etc/httpd/conf.d/ssl.conf file and set the
ServerName in the eponymous directive.
4.2. MySQL database
- Change the password for the mysql root user
as root user run mysqladmin command to reset password. You can use
pwgen(1)to generate new password, e.g.:
- write new password to file
/root/.my.cnfand make sure it’s still only readable by root (chmod 0600).
- Change password for database user
Generate and note down another password, e.g. using
pwgenagain (as done for the MySQL root account)
login to mysql database as root user, no password required after following the above steps:
issue UPDATE statement to modify user table, pasting in the freshly created/generated password:
This password also needs to be configured within Jagger later.
4.3. Postfix configuration
If customer provides us with a smarthost for email notifications, change
/etc/postfix/main.cfso that it points to the smarthost:
Make sure the smarthost accepts connections on port 25:
Restart postfix and test email delivery by sending an email to yourself:
Email headers on the recieved message should reveal that this email traversed the NREN's smarthost successfully.
- If email delivery is not working, check log files for error message and send it to FaaS development team.
- In file
- In file
set default registration authority (the unique identifier of the SAML metadata registrar, i.e, of the federation):
set list of languages in two-letter ISO 639-1 codes with single quotes and separated by commas, for example array ('en', 'rs'). By default should be the official languages of the country/region served by the NREN, plus English:
set the default language in two-letter ISO 639-1 code. Should be one of the official languages of the country/region served by the NREN.
set permission who can do translation of Jagger UI and into what language (optional):
Depending on the language (e.g. Lithuanian) you need to change permissions for
rr_lang.phpfile in the appropriate language directory (e.g.
/opt/rr3/application/language/lt) to allow
apacheuser to write to this file.
You need to set
/etc/php.inifile (the main reason to use PHP > 5.3 here).
- change the site logo:
- upload new logo into
refernce the file name of the new logo in the
site_logovariable in the config_rr.php file:
- upload new logo into
- In file
change password for
rr3user(password is already set in step 2 of the MySQL database section):
- In file
Change SMTP sender:
Since JaggerMailer is running under control of supervisor process, restart supervisord to apply changes:
Test it's running successfully (pid and uptime will vary, of course):
4.5. HSM access
For training instances skip this section, as training machines only use SoftHSM-protected keys.
Create key pair to secure HSM connections:
Send the created certificate (
/usr/safenet/lunaclient/cert/client/FILENAME.pem) to email@example.com (Cc'ing firstname.lastname@example.org) and wait for confirmation that the client certificate has been deployed.
Change user and file system permissions for
When you have confirmation from the NDN NOC, verify the connection to the HSM:
4.6. Pyff configuration
/opt/faas/pipelines/edugain-upstream.fd, replacing all
CHANGE_*strings with provided values (cf. FaaS instances configuration parameters):
/opt/faas/pipelines/federation-downstreams.fd, replacing all
CHANGE_*strings with provided values:
4.7. Cron configuration
After configuring pyff pipelines you should do the following:
Open cron file for editing (as user
Activate (uncomment) all cron jobs for user root after the line “# FaaS”:
This will cause SAML metadata for the "production" and eduGAIN (upstream) "federations" to be refreshed and re-signed daily at the given hour (here: 1:10 and 1:15 a.m. local time – note that FaaS instances have UTC as local time!), even when no changes to the local metadata occured.
md-watch.pyscript will only re-sign metadata if local metadata changes within Jagger have occured. By default this script is run every 10 minutes via cron, but this can be adjusted to taste. (Running more often will cause changes to be picked up/published sooner.)
4.8. Shibboleth SP configuration (optional)
Many Jagger deployments (and all training and test instances) won't federate their SAML federation registry right away (or ever). So initially there is no need to configure the Shibboleth SP, as its own SAML metadata would need to be registered into Jagger first, as well as any IDPs to log in to Jagger with.
Regenerate key material for SAML SP:
, replacing all CHANGE_* strings with provided values:
Test configuration (might complain due to missing/incorrect SAML metadata from Jagger, if nothing has been put into Jagger previously):
Ensure shibd starts after reboots:
5. Log files
Maintain system and application logs for audit trails. Following logs should be kept:
- Standard system logs: messages, cron, secure etc.
- Logs of specific services run from FaaS: web server, web applicaiton, pyff, shibboleth, mysql, postfix.
All logs should be rotated weekly, where at least 4 weeks sets should be kept, or as otherwise mandated by the respective laws under which the infrastructure is operated.
Location of log files:
- Standard system logs (messages, cron, secure etc.): /var/log/
- Apache logs: /var/log/httpd/
- Web application: /var/log/rr3/
- Pyff logs: /var/log/faas
- Shibboleth logs: /var/log/shibboleth/ and /var/log/shibboleth-www/
- Mysql logs: /var/log/mysqld.log
- Postfix logs: /var/log/maillog
Backup is only important for production instances; there is no need to back up training instances as those do not contain real data and are generally rather short-lived by nature.
For each VM:
Create backup of
/var/local/mysqlbackupdirectories, recursively, on a daily basis. Backed up files and directories should be kept two weeks at minimum. Ideally at least the last 2 generations of backed-up files should be kept (i.e., the current one and the previous one).
Create a [hot] copy of the VM on a monthly basis. The last two images should be kept for quick disaster recovery of the configured system.