1. There are no references to other sites or branding or contacts or even to eduGAIN.

True statement. No branding at all. Deliberately kept simple to minimize the risk of programming mistakes and vulnerabilities. I'd argue that the fact that the web site is running TLS and lives within the edugain.org domain space attests to its association with eduGAIN, but of course, it is debatable whether that is enough.

2. The text suggests you are timing a response and ranking us.   I do have an issue with this because someone aimlessly clicking a link in an email without checking with colleagues or checking it's validity, is a potential information security risk to organisation.  Quick responses are not necessarily the best, and also we prioritise calls as I'd expect any service desk to, and responding to an automated check would figure much lower than a real security incident.   Similarly, they could be categorised as spam by less experienced staff or someone who may not be aware of this security challenge, particularly as our calls to security@ are part of a wider ticketing system.

Ranking: True statement. The wording is possibly or even probably poor. We probably should not use the phrase "rank" anywhere. We took this from similar campaigns in other environments of collaborating CERT/CSIRT's. In fact this "scoring" will stay anonymous and will only be used to discuss desirable reaction times with the community in which this challenge was run. Nevertheless it has proven to add a gamification component to it :-) You will only get your own scoring. The results of other teams will only be used to check if we have an issue with the registered contact addresses, and, or if the foreseen communication methods do not work as expected.

3. There was no prior notice on any channels that my team or I use.

Too bad! The notices have been sent to your eduGAIN delegate and deputy through the eduGAIN Steering Group mailing list. Next time we'll make sure that the information arrive to the right people.

4. Maybe sharing something that says you will get an email "like this" beforehand would be useful.

Good point and it is exactly what we've done, but once again only with the eduGAIN Steering Group mailing list.

5. I got an out of office message into our helpdesk from one of the members of the abuse@edugain.org team.   I guess you are using an email forward to deal with abuse@edugain.org.  However, getting out of office message is not a great situation for service addresses.

True again, this is a miss-configuration at our end, its of course useless if team members have an autoreply for a team address configured in their personal mail settings.
In fact this campaign is also about detecting these kind of flaws and we deem to have a baseline which can be used in a follow up campaign to check the improvements