...

The first thought that came to mind: since we're running several Linux servers already on IPv6-only, would it be possible to run a Windows server also exclusively on IPv6?

Last year my intern Joris Claassen has proved that it was possible to run a basic VMware VSphere cluster on IPv6 only, and that included a Windows box as well. So things look bright.

While Windows Server 2012 has been released recently, we can not use it (yet), because the latest version of Windows that our current VMware setup (ESXi 4.1) supports as guest OS is Server 2008 R2.

...

Basics

IPv6-only VLAN

A new dedicated VLAN was created on our Cisco 3750X for IPv6-only systems, so that building, testing and configuring would not interfere with any production networks. Configuring this VLAN with only IPv6 allows to use simpler IP This IPv6-only VLAN allows us to start from scratch using only two IPv6 Access Control Lists (ACLs), and start from scratch:

interface Vlan9
 description IPv6_only_Servers
 no ip address
 no ip proxy-arp
 ipv6 address 2001:610:148:BAD::1/64
 ipv6 nd prefix 2001:610:148:BAD::/64
 ipv6 traffic-filter ipv6_servers2only_servers_out in
 ipv6 traffic-filter ipv6_only_servers2_servers_in out
end

I started out with IPv6 ACLs that disallow everything by default, then started configuring the server, monitor the log files for any dropped traffic, and then open up specific thingsgradually opened up the necessary things.

Addressing

Obviously, the first thing to do is to uncheck IPv4 in the interface configuration . For  For IPv6 addressing I choose autoconfigured EUI64, no privacy extensions. The reason is that this is a server, and it will not be used for any web browsing activities. Static addresses also help configuring the (empty) IP IPv6 ACLs. The idea is that during configuration and testing the IPv6 ACLs will be constructed, based on stuff that does not work. Eventually, when everything works, it might be an option to use privacy extensions.

Also, I disabled all tunnelling stuff (ISATAP, Teredo, etc). Combined script:

...

The "Register this connection's addresses in DNS" option has been deselected, because this causes DNS registration requests to go out, which we do not want. Eventually things look like this:

NTP

At the time of writing, the default time server that is used by Windows 2008 R2 to sync its clock, time.windows.com, unfortunately  unfortunately is not (yet) reachable over IPv6.

But again , SURFnet comes to the rescue, because several of their NTP servers are IPv6-enabled. I picked chime3.surfnet.nl, which, according to the web interface, appears to be a Meinberg NTP server. Windows digs it all-right:

...

To manage the server, Remote Desktop Protocol (RDP) is used, which supports IPv6 without any configuration.

...

Microsoft Update

Unfortunately the Microsoft Update servers are also available only on IPv4 .

I set-up a limiting HTTP proxy server that listens on IPv6, and added a number of regular expressions to allow Windows to download updates.

As can be seen from the lists, several other URLs also need whitelisting, such as CRLs and OCSPs.

Details of the proxy and the Windows configuration process are described on a separate page.

Exact Software

I phoned up Exact Software Netherlands to see if Exact Globe would support IPv6, but the help desk could not provide me with a definitive answer. So then, time for some testing. 

Exact Globe consists of a network share containing an installer, and an SQL Server. Clients map the network drive and install the software on their computer.

File sharing

Windows 2008 R2 does SMB (File sharing) over IPv6-only without any problems:

Image Removed

Windows Internet Name Service (WINS) does not work with IPv6. But because this was a piece of junk anyway, nothing is lost here.

SQL Server

Once installed, the Exact software then initiates a connection to TCP port 1433 of an SQL Server. In our case this is the same server.

The old (Windows 2003) server runs SQL Server 2000, but starting from release 404 (July 2012), the Exact software does not support this any more. So I had to upgrade to SQL Server 2008 R2.

This supports IPv6 just fine. All users in accounting all run Windows 7, so the operating system itself shouldn't be a problem, but I was a bit wary of potential client side application issues. While testing I quickly found out that the Exact application was using ODBC, which did support IPv6 right away, so the application works on IPv6 .

Delta software

This is proprietary software made by Dutch company BCS HRM en Salarisadministratie B.V..

Recently we bought a server license, and after some research I found out that is made up of a network share with files, and a database component.

The network share can do IPv6 as mentioned earlier.

Sybase SQL AnyWhere

The database is a Windows Service, based on a file called dbeng11.exe. It's not stated as such in the BCS documentation, but this is an instance of SQL AnyWhere Personal Server:

Image Removed

The BCS support desk had no clue what IPv6 was, so I did some research myself.

There is a nice and very clear PDF that explains SQL AnyWhere networking essentials. According to this document, SQL AnyWhere supports IPv6 from version 10, which is from 2006. Version 11 and 12 support it explicitly. These docs came in handy because they list some special connection options that I needed.

Since we only have one instance of SQL Anywhere, which is running in a different network than the clients, I figured it didn't make much sense to implement the broadcast/multicast stuff that is needed for the service discovery. Instead, I hardcoded the server FQDN, which worked fine. After opening up TCP port 2638 the software worked 

Crypto

I wasn't really sure if the SQL Server traffic was plain text or not, and after running Wireshark it turned out that is wasn't...

Because we are part of the TERENA Certificate Service, we have access to 'free' SSL certificates from SURFnet.

Generating key material

I did this on a host with OpenSSL. Also possible on the Windows host but I will write about that later.

openssl req -new -keyout server.key -out server.csr -subj /CN=hayek.terena.org/

I submitted the signing request to the SURFnet web site, and after a few hours, and jumping through the Domain Control Validation hoops, I got a signed certificate (cert-11988-hayek.terena.org.pem) back, and the chain, which consists of 3 certificates concatenated into one file (chain-11988-hayek.terena.org.pem).

Then combined all certificates into one file, and created a PFX from these materials:

cp server.pem all.pem
cat chain.pem >> all.pem
openssl pkcs12 -export -inkey server.key -in all.pem -out server.pfx

Adding the key and certificates

Copy this file to the Windows server, and run mmc, then add the Certificates snap-in.

When it asks for who to manage certificates, select an account that the SQL Server has access to.

In our case that was the "Local System" account, and we were running the snap-in as Administrator, so all is well.

Expand: Console Root -> Certificates (Local Computer) -> Personal.

Right-click -> All Tasks -> Import.

Now navigate to the pfx file and import it. Include all extended properties.

I kept Mark this key as exportable unchecked, as I already have the key material in PEM format in a different place.

We don't need this, and any malicious export attempt will be more difficult this way.

Configuring MS SQL Server to use the certificate

Run the SQL Server Configuration Manager, expand the SQL Server Network Configuration, and right-click Protocols for MSSSQLSERVER (or whatever your instance is called).

On the Certificate tab you should be able to see your certificate.

In my case nothing would show up 

According to How to: Enable Encrypted Connections to the Database Engine (SQL Server Configuration Manager), the name of the certificate must be the fully qualified domain name (FQDN) of the computer. The TCS certificates we use can only contain a valid FQDN as the Subject's Common Name (CN), so this was correct. I checked permissions and those seemed to be OK as well.

It turned out that the server did not have a Full Computer Name yet... duh.

After fixing that the certificate showed up in the SQL Server Network Configuration. Then I forced encryption, checked again with Wireshark and indeed no more plain text queries 

Image Removed

RDP Crypto

Since we now have a nice server certificate in the Windows Certificate Store, I figured I might as well use it to secure more services that run of this machine. The most obvious service is of course the channel by which the machine is managed: Remote Desktop Protocol (RDP). Based on the docs for Remote Desktop Services (RDS), this service does support SSL (TLS1.0). The installed certificate can be configured with the Remote Desktop Session Host Configuration. While at it, I also selected to only use SSL (TLS 1.0), and High Encryption level:

Image Removed

Using Wireshark it is again easy to spot that the session is encrypted.

Then I noticed one thing. Up until now I would always get this warning upon connecting to a Windows server:

Image Removed

After installing a real certificate, this warning isn't shown any more, and a nice little lock is shown on the RDP status bar, which shows the certificate information if clicked:

Image Removed

Service monitoring

All of this stuff is very nice, but it's no good without monitoring.

We run a Nagios 3.x instance in a remote data center, and ideally I'd want to monitor all services that this hosts runs.

Host alive

Modified version of the default host alive check. Excerpt from /etc/nagios3/commands.cfg:

# Changed to include IPv6 flag 
define command{
        command_name    check-host-alive6
        command_line    /usr/lib/nagios/plugins/check_ping -H '$HOSTADDRESS$' -6 -w 5000,100% -c 5000,100% -p 1
}

Disk usage

This can be done with the standard check_disk_smb_host_user, because that relies on the smbclient binary, which is part from the samba-3.4.7 package, which supports IPv6.

Microsoft SQL Server

FreeTDS library patch uitleggen

SSL certificate expiration date

According to Microsoft, Windows 8 will use IPv6, if available, to download Windows Updates. They will works with Content Delivery Networks (CDNs) to extend IPv6 support later. So at this time, for Windows Server 2008 R2, the update servers are accessible only through IPv4.

As a solution I set-up a dual-stack limiting HTTP proxy server, and added a number of regular expressions to allow Microsoft Updates to happen.

Details of the proxy and the Windows configuration process are described on a separate page.

As can be seen from the lists, several other URLs also need whitelisting, such as CRLs and OCSPs.

Windows Activation

Same story here. Activating Windows is done by several HTTP POSTs to URLs in the go.microsoft.com domain, which is not (yet) accessible through IPv6:

Since the proxy server was already in place, I just needed to add this domain to the proxy whitelist, and then the Windows activation succeeded.

Secunia CSI

This tool monitors installed software for vulnerabilities. We have the corporate version CSI 6. This is a service that registers itself at Secunia to download patch lists, upload results, etc.

Unfortunately the host that is used for communication (agent.csi6.secunia.com)is only reachable via IPv4.

But when configured to use the HTTP proxy everything works as expected and CSI is able to phone home again .

Access from IPv4 networks

Users that need to access this server from IPv4 networks (teleworking etc) use Cisco Secure Mobility Client to connect to our Cisco ASA, which gives out IPv6 addresses.

Exact Software

I phoned up Exact Software Netherlands to see if Exact Globe would support IPv6, but the help desk could not provide me with a definitive answer. So then, time for some testing. 

Exact Globe consists of a network share containing an installer, and an SQL Server. Clients map the network drive and install the software on their computer.

File sharing

Windows 2008 R2 does SMB (File sharing) over IPv6-only without any problems:

Image Added

Windows Internet Name Service (WINS) does not work with IPv6. But because this was a piece of junk anyway, nothing is lost here.

SQL Server

Once installed, the Exact software then initiates a connection to TCP port 1433 of an SQL Server. In our case this is the same server.

The old (Windows 2003) server runs SQL Server 2000, but starting from release 404 (July 2012), the Exact software does not support this any more. So I had to upgrade, preferably to the latest supported version, which is SQL Server 2012. Because such an upgrade needs an intermediate step, I decided it was OK to use SQL Server 2008 R2 for now.

This supports IPv6 just fine. All users in accounting run Windows 7, so the operating system itself shouldn't be a problem, but I was a bit wary of potential client side application issues. While testing I quickly found out that the Exact application was using ODBC, which did support IPv6 right away, so the application worked on IPv6 .

Delta software

This is proprietary software made by Dutch company BCS HRM en Salarisadministratie B.V. and is made up of a network share with files, and a database component.

The network share can do IPv6 as mentioned earlier.

Sybase SQL AnyWhere

The database is a Windows Service, based on a file called dbeng11.exe. It's not stated as such in the BCS documentation, but this is an instance of SQL AnyWhere Personal Server:

Image Added

The BCS support desk had no clue what IPv6 was, so I did some research myself.

There is a nice and very clear PDF that explains SQL AnyWhere networking essentials. According to this document, SQL AnyWhere supports IPv6 from version 10, which date back to 2006. The docs for version 11 and 12 state is even more explicitly with some examples. These docs came in handy because they list all the connection options, which I needed.

Since we only have one instance of SQL Anywhere, which is running in a different network than the clients, I figured it didn't make much sense to properly implement the service discovery. Instead I hardcoded the server FQDN, which worked fine. After opening up TCP port 2638 the software worked 

Crypto

I wasn't really sure how safe this SQL Server traffic was, but running Wireshark revealed plain text SQL queries on the wire . I think it's good practice to encrypt traffic containing personal data, even though it's inside our own network. SQL Server 2008 R2 supports SSL encryption, and because we get "free" SSL certificates through SURFnet (which is part of the TERENA Certificate Service), we should be good to go!

I created the cryptographic materials on one of our Ubuntu systems because those already have OpenSSL installed. All of this is also perfectly possible on Windows, but you do need some extra bits such as Microsoft Visual C++ 2008. I just didn't want to pollute my Windows system with extra software that is already installed elsewhere in our network.

openssl req -new -keyout server.key -out server.csr -subj /CN=hayek.terena.org/

I submitted the signing request to the SURFnet web site, and after jumping through the Domain Control Validation hoops, I got a signed certificate (cert-11988-hayek.terena.org.pem) back, and the chain, which consists of 3 certificates concatenated in one file (chain-11988-hayek.terena.org.pem).

Then I combined all certificates into one file, and created a PFX from it:

cp cert-11988-hayek.terena.org.pem all.pem
cat chain-11988-hayek.terena.org.pem >> all.pem
openssl pkcs12 -export -inkey server.key -in all.pem -out server.pfx

After copying the PFX file to the Windows server, I ran mmc and added the Certificates snap-in. When it asked for who to manage certificates, I selected the "Local System" account, because that's what the SQL Server was configured to run as.

Expand: Console Root -> Certificates (Local Computer) -> Personal.

Right-click -> All Tasks -> Import. Navigated to the pfx file and imported it, including all extended properties.

I kept "Mark this key as exportable" unchecked, as I already have the key materials in a different place. I figured that since it is not needed on this machine, we might as well disable it, so any future malicious export attempts will be impossible (or at least more difficult).

Configuring MS SQL Server to use the certificate

Run the SQL Server Configuration Manager, expand the SQL Server Network Configuration, and right-click Protocols for MSSSQLSERVER (or whatever your instance is called).

On the Certificate tab you should be able to see your certificate.

In my case nothing would show up 

According to this Howto, the name of the certificate must be the fully qualified domain name (FQDN) of the computer. Since the TCS certificates we use can only contain a valid FQDN as the Subject's Common Name (CN), this had to be correct. I checked permissions and those seemed to be OK as well.

It turned out that the FQDN is called Full Computer Name in Windows, and indeed that was still set to a single label:

Image Added

To change this I had to add our domain name at Primary DNS suffix of this computer:

Image Added

I wasn't aware that Windows did this sort of checks, but I think it makes sense because it prevent you from inadvertently using a wrong certificate .

After fixing the FQDN the certificate showed up in the SQL Server Network Configuration.

Then I forced encryption:

Image Added

checked again with Wireshark and indeed no more plain text queries 

RDP Crypto

Since we now have a nice server certificate in the Windows Certificate Store, I figured I might as well use it to secure more services that run off this machine. The most obvious service is of course the channel by which the machine is managed: Remote Desktop Protocol (RDP). Based on the docs, RDP does support SSL (TLS1.0). The installed certificate can be configured with the Remote Desktop Session Host Configuration. While at it, I also selected to only use SSL (TLS 1.0), and High Encryption level:

Image Added

Using Wireshark it is again easy to spot that the session is encrypted.

Previously I would always get this warning upon connecting to a Windows server:

Image Added

After installing a real certificate, this warning isn't shown any more, and a nice little lock is shown on the RDP status bar, which shows the certificate information if clicked:

Image Added

IPv6 service monitoring with Nagios

All of this stuff is very nice, but what good is a service when it is not being monitored?

We already run a Nagios-3.2 instance on an Ubuntu system in a remote data centre, and my goal is to monitor as much services on my new Windows host as possible.

Host alive

The check-host-alive test is a basic test from the nagios-plugins-basic package, and is based on check_ping, which can do IPv4 and IPv6, and defaults to IPv4. It is defined in /etc/nagios-plugins/config/ping.cfg.  Interestingly, there are checks to deal with dual stack monitoring:

####
# use these checks, if you want to test IPv4 connectivity on IPv6 enabled systems
####
 
# 'check-host-alive_4' command definition
define command{
    command_name    check-host-alive_4
    command_line    /usr/lib/nagios/plugins/check_ping -H '$HOSTADDRESS$' -w 5000,100% -c 5000,100% -p 1 -4
    }

Since our host's FQDN only has a AAAA DNS records, the default check is OK.

Disk usage

This can be done with the standard check_disk_smb_host_user, because that relies on the smbclient binary, which is part from the samba-3.4.7 package, which supports IPv6.

Microsoft SQL Server

There is no Nagios plugins in any Ubuntu package that can check the status of SQL Server, but Nagios Exchange does list a number of them. They are written in various languages (Perl, Python, PHP), and they all rely on FreeTDS, a set of libraries for Unix and Linux that allows programs to natively talk to Microsoft SQL Server and Sybase databases. The bad part is that this library does not support IPv6, hence none of the plugins work with my server .

After asking around on the FreeTDS mailing list, a guy called Peter Deacon wrote a patch that adds IPv6. I did some testing with Perl and after forcefully installing a newer version of DBD::Sybase I was able to successfully connect to my SQL Server .

With the FreeTDS library now supporting IPv6 connections, I went to the Nagios Exchange and picked check_mssql_health, which can do rather elaborate checks on all sorts of SQL Server properties.

SSL certificate expiration date

Now that the server has an SSL certificate, it should be monitored so that the admins get a warning when it is about to expire. The way I usually do this is by using a Nagios plugin to monitor a service that uses the certificate, and then pick the certificate expiration option of the plugin. For example the check_smtp plugin has this option:

 -D, --certificate=INTEGER
    Minimum number of days a certificate has to be valid.

The common way to test this sort of stuff is with OpenSSL's s_client, but unfortunately that does not support IPv6. Off then to GnuTLS.

Connecting to SQL Server on port 1433 yielded nothing, the handshake wasn't establisched, so that's a dead end for now. Connecting to RDP on port 3389 however happens with a nice TLS 1.0 handshake 

bofh@nagios:~$ gnutls-cli hayek.terena.org -p 3389 --x509cafile /etc/ssl/certs/ca-certificates.crt
Processed 142 CA certificate(s).
Resolving 'hayek.terena.org'...
Connecting to '2001:610:148:bad:250:56ff:fe86:9:3389'...
- Certificate type: X.509
 - Got a certificate list of 3 certificates.
 - Certificate[0] info:
  - subject `C=NL,O=TERENA,OU=CFO,CN=hayek.terena.org', issuer `C=NL,O=TERENA,CN=TERENA SSL CA', RSA key 2048 bits, signed using RSA-SHA, activated `2013-01-23 00:00:00 UTC', expires `2016-01-23 23:59:59 UTC', SHA-1 fingerprint `513523a823b8d5c15c30a80f2772e58d826605ba'
 - Certificate[1] info:
  - subject `C=NL,O=TERENA,CN=TERENA SSL CA', issuer `C=US,ST=UT,L=Salt Lake City,O=The USERTRUST Network,OU=http://www.usertrust.com,CN=UTN-USERFirst-Hardware', RSA key 2048 bits, signed using RSA-SHA, activated `2009-05-18 00:00:00 UTC', expires `2020-05-30 10:48:38 UTC', SHA-1 fingerprint `3a881764472b6441ddb3afdd47c6b8b76ee7ba1d'
 - Certificate[2] info:
  - subject `C=US,ST=UT,L=Salt Lake City,O=The USERTRUST Network,OU=http://www.usertrust.com,CN=UTN-USERFirst-Hardware', issuer `C=SE,O=AddTrust AB,OU=AddTrust External TTP Network,CN=AddTrust External CA Root', RSA key 2048 bits, signed using RSA-SHA, activated `2005-06-07 08:09:10 UTC', expires `2020-05-30 10:48:38 UTC', SHA-1 fingerprint `3d4b2a4c64317143f50258d7e6fd7d3c021a529e'
- The hostname in the certificate matches 'hayek.terena.org'.
- Peer's certificate is trusted
- Version: TLS1.0
- Key Exchange: RSA
- Cipher: AES-128-CBC
- MAC: SHA1
- Compression: NULL
- Handshake was completed

This indicates that it possible to monitor the certificate the same way as with HTTPS web sites. I tested this and it works 

Concluding, this is what the monitoring looks like in the end:

Image Added

 RDP HTTPS check