I was wondering if it would be possible to run parts of the TERENA Secretariat office network on IPv6 only.
Our office has a /24 IPv4 and a /48 IPv6 network since 2002, and all our public services are available on IPv4 and IPv6.
Since we have native IPv6 connectivity in our office, it seems that the first step would be to remove IPv4 from services that are only used internally.
This page keeps track of progress, bugs, and issues.

BTW, this is not the first time this has been tried out. For instance Arkko & Keranen did some interesting work in 2010.

Jump to:

Milestones

nameserver 2001:610:148:dead::4
nameserver 2001:610:158:98d::42
domain terena.org

Coda 2 drops support for IPv6

One of my users that joined our company recently needed to do some web editing stuff on her Apple computer. Other users have good experience with the Coda editor from panic.com. So I ordered and installed it from the Apple Store. Hadn't done this earlier but seems convenient. It was the new version 2 - whatever.

For some reason she could not connect to our web server. Checked what was wrong and found out that version 2 dropped support for IPv6:

Copied the latest old version (1.7.5) and that worked just fine.

Remote support with IPv6-only VNC

While looking for a cross-platfrom remote support solution that does IPv6, I ended up using commercial software this time: RealVNC Enterprise Edition. This does some other handy things as well, such as AES encryption, system authentication, and an Accept/Reject dialog on the client.

By default it listens on both IPv4 and IPv6 but it can be easily configured to listen only on IPv6:

I use this to remotely support people. One important issue: you do have to know the network address before you can connect to it . There are two ways, users are either:

• In the office. All our computers autoconfigure themselves with the modified EUI64 interface identifier, which is based on the interface's MAC address. This means that computers will have a predictable address in the office. This means that you can assign easy to remember hostname to them 'laptop-alice-wifi', 'laptop-alice-wired', etc. 
• Out of the office. People working from remote need to set up their Cisco AnyConnect VPN connection. Once that is done, they have IPv6 connectivity, bypassing any NAT boxes they might be behind. They either need to tell me their network address, or I can look it up on our VPN server.

An alternative for locating machines would be dynamic DNS, but I have not found an end-user-friendly and easy-to-administer cross platform solution for that - yet.

Microsoft AutoUpdate does not work in an IPv6-only environment

2011-03-28. Upon installing a Mac with OSX 10.7 and the applications we use, I connected it to our IPv6-only network. Everything went fine, until I tried to update Microsoft Office for Mac 2011:

Because the network does NAT64/DNS64, this must mean that AutoUpdate does some weird IPv4-only stuff. After connecting it temporarily to the dual stack network, I did get the updates immediately.

I created a couple of tcpdumps, and it looks like the AutoUpdate tool has some hardcoded logic that makes it only use A records for download.microsoft.com, which of course fail:

Interestingly, had the tool queried for a AAAA record, it would have worked:

administrators-macbook-pro:~ root# host -t aaaa download.microsoft.com
download.microsoft.com is an alias for download.microsoft.com.nsatc.net.
download.microsoft.com.nsatc.net is an alias for main.dl.ms.akadns.net.
main.dl.ms.akadns.net is an alias for intl.dl.ms.akadns.net.
intl.dl.ms.akadns.net is an alias for dl.ms.georedirector.akadns.net.
dl.ms.georedirector.akadns.net is an alias for a767.ms.akamai.net.
a767.ms.akamai.net has IPv6 address 2001:610:148:ffff:b0b0::58dd:d830
a767.ms.akamai.net has IPv6 address 2001:610:148:ffff:b0b0::58dd:d83a

Boo!!!

Ubuntu servers not ready

Trying to debug OpenDNSSEC packages on one of my IPv6-only systems but unfortunately some bumps in the way. The system keyserver.ubuntu.com does not have IPv6 connectivity, so adding GPG keys using apt-key does not work:

root@svn:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7C1A977926535DB3
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys 7C1A977926535DB3
gpg: requesting key 26535DB3 from hkp server keyserver.ubuntu.com
gpgkeys: HTTP fetch error 7: Failed to connect to 91.189.89.49: Network is unreachable
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0

Waiting for response at https://answers.launchpad.net/launchpad/+question/189094

Once manually copy/pasted they key, the next hurdle is ppa.launchpad.net:

Reading package lists... Done
W: Failed to fetch http://ppa.launchpad.net/pkg-opendnssec/ppa/ubuntu/dists/lucid/Release.gpg  Could not resolve 'ppa.launchpad.net'
W: Failed to fetch http://ppa.launchpad.net/pkg-opendnssec/ppa/ubuntu/dists/lucid/main/i18n/Translation-en_US.bz2  Could not resolve 'ppa.launchpad.net'
W: Some index files failed to download, they have been ignored, or old ones used instead.

Added bug https://bugs.launchpad.net/launchpad/+bug/942571 for that...

Samba

Our servers run Ubuntu 10.04 LTS, which ships with Samba 3.4.7, and this has proper IPv6 support. There is no global switch to make it listen to a specific IP version, but is not too difficult to make it listen to IPv6 only.
You need to tell it to only bind to a specific (hard coded) interface address in smb.conf:

bind interfaces only = yes
interfaces = 2001:610:148:beef::132/64

This machine is a server but sits in the Office network, so it was needed to disable autoconfiguration to prevent is from accumulating multiple IPv6 addresses (never a good idea for servers).
Contrary to what you would expect, setting /proc/sys/net/ipv6/conf/eth0/autoconf to 0 does not do the trick. You have to explicitly state the interface. So I ended up with this /etc/network/interfaces:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet6 loopback
        pre-up ip addr del 127.0.0.1/8 dev lo

# The primary network interface
auto eth0
iface eth0 inet6 static
        address 2001:610:148:beef::132
        netmask 64
        gateway 2001:610:148:beef::1 dev eth0
        pre-up echo 0 > /proc/sys/net/ipv6/conf/eth0/autoconf

MacOSX Lion & Cisco AnyConnect

Late July 2011 Apple released version 10.7 of their OSX operating system, named Lion. This version has several major IPv6 related improvements, the most important I think is DHCPv6 support. This means that it is now possible to successfully run a Mac in an IPv6 only environment without any configuration.
Unfortunately for us AnyConnect has a serious bug on Lion, namely that there is no default gateway being configured for IPv6 upon connection.
Since we have several IPv6 only services these days, this is a true show stopper
What makes it worse it that the smbd in Lion has IPv6 support, and because our Windows 7 computers already support SMB via IPv6, this means that we could make our Samba server IPv6 only. But since AnyConnect does not work, this is not (yet) an option...
The issue has been reported already to Cisco and is filed as CSCts11510 (login required).
Rumour has it that a fix is available soon, so let's just keep our fingers crossed!
Update: As of 29 September 2011, AnyConnect 3.0.4235 fixes the problems! Now all my users can have IPv6 again from everywhere

Specific issues

Name resolution

To avoid name resolution problems, it was sometimes necessary to copy to the legacy 127.0.1.1 entries to ::1 in the /etc/hosts file:

127.0.0.1       localhost
127.0.1.1       ldap.terena.org ldap

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback ldap.terena.org ldap

127.0.0.1

On IPv6 only hosts, there is no IPv4 address configured on the network interface, so obviously all communications will go via IPv6.
There is still an IPv4 address (127.0.0.1) sitting on the loopback interface lo. It's doesn't hurt but it should not be there if the host were to be truly IPv6 only
I could not find anywhere in /etc where this address get added.
Since I can prevent it from getting added, I removed it after it gets added, by hacking configuring /etc/network/interfaces:

# The loopback network interface
auto lo
iface lo inet6 loopback
        pre-up ip addr del 127.0.0.1/8 dev lo

This seems to work fine, only now ntp dumps core, but that has been fixed as of 17 April 2011.

Skype

Skype does not support IPv6 at all  FAIL!!!! Please  VOTE FOR IPv6 (<- this link appears to be dead as of January 2012, and the original poll is gone it seems... wonder why that happened...)

I've set-up a poll on http://www.change.org/petitions/skype-add-ipv6-support-to-skype , please vote there so Skype can be persuaded to add IPv6 support

FYI the first request for IPv6 enabled Skype date back to 2004!!

Cisco Catalyst 3750

This switch (WS-C3750G-24TS-E) does not support IPv6 access lists on VLANs.

2011-10 Replaced with a WS-C3750X-48T-S and a WS-C3750X-48PF-S.

This one does IPv6 access lists on VLANs, but I had to use sdm templates for this. In hindside the old one might have supported it as well.

But because that died after powering it off to take it out of service, no way to find that out anymore 

Cisco AP1200

These access points do not support IPv6. Need replacing anyway. The AIR-AP1142N-E-K9 could be a drop-in replacement. Also does N.

December 2011: after successful participation in the Eduroam-as-a-Service from SURFnet/Wentzo wireless, these accesspoints were not needed any more.

I donated them to the System & Network Engineering master education of the University of Amsterdam.

Sharp AR-M276

This copier/printer does not support IPv6 at all.
Could not retrieve e-mail addresses for 'scan to email' after LDAP server went IPv6 only. Hack Work-around: manually put addresses in.
Our big Sharp MX2600n has IPv6 support, so we should get rid of this clunker on the first occasion.

Google Mini

This box does not support IPv6. Needs replacing in 2011 anyway, but don't forget to check!!

Axis Q1755 web cam

Can be configured to do IPv6, but only PING works
Investigate further.

2011-12-20 Reset the cam and started with fresh config, now works fine over IPv6.

Sharp MX-2600N printer

After enabling IPv6 on our Sharp MX2600N printer, the network stack actually works, but only a couple of services are running IPv6:

root@expat:~# nmap -6 2001:610:148:beef::134

Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-01 10:31 CET
Interesting ports on 2001:610:148:beef::134:
Not shown: 996 closed ports
PORT    STATE SERVICE
21/tcp  open  ftp
23/tcp  open  telnet
515/tcp open  printer
631/tcp open  ipp

Nmap done: 1 IP address (1 host up) scanned in 2.29 seconds

This is in stark contrast to what runs on IPv4:

root@expat:~# nmap --system-dns 192.87.30.134

Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-01 10:35 CET
Interesting ports on sharp-mx2600n.terena.org (192.87.30.134):
Not shown: 991 closed ports
PORT      STATE SERVICE
21/tcp    open  ftp
23/tcp    open  telnet
80/tcp    open  http
443/tcp   open  https
515/tcp   open  printer
631/tcp   open  ipp
5900/tcp  open  vnc
9100/tcp  open  jetdirect
50001/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 2.37 seconds

Now I'm trying to find out how to print using IPP from Windows 7.

Ecdysis

Some tests indicate the Ecdysis works well.
Also, they presented at our own conference last year
Take into consideration!

PECL radius

IPv6 doesn't work. Needed for TERENA web site. Update 2011-03-01: by upgrading Pear_Auth, Pear Live_User was able to use LDAP (via IPv6), without the Radius overhead.

MySQL

MySQL at this moment does not support IPv6 connections, but the development versions seem to support it (sort of). There are tools to make it work, such as https://twiki.cern.ch/twiki/bin/view/EGEE/IPv6CARE.
Not A Problem Here: we have only one host running MySQL, and that will be phased out in the future any way.

apt-get

security.ubuntu.com does not work, so no security updates. Workaround: use local mirror nl.archive.ubuntu.com for security updates.

Pear Net_Socket

Does not like IPv6 addresses, wrote patch.

ntp

• ntp.ubuntu.com does not work. We are able to use SURFnet's chime3.surfnet.nl and chime4.surfnet.nlhowever:

  visser@svn:~$ ntpq -pn
       remote           refid      st t when poll reach   delay   offset  jitter
  ==============================================================================
  +2001:610:508:11 .GPS.            1 u  360 1024  377    4.499   -0.340   0.483
  *2001:610:0:800b .PPS.            1 u  192 1024  377    4.746    1.525   0.089
  

• dumps core without IPv4 loopback address. This has been fixed in 1:4.2.4p8+dfsg-1ubuntu2.1 .

Confluence and JIRA LDAP auth via IPv6

Authentication on dual stack LDAP servers does not work. Using an IPv6-only hostname does work. For us this works, because our LDAP server is IPv6 only.

Radiator

Cannot use IPv6 LDAP server. Filed support ticket at Open.com.au. Fixed as of 2011-02-12. Also make sure to add flags to any custom perl hooks:
my $ldap = Net::LDAP->new('ldap://ldap.terena.org',inet6=>1);.
We use Radiator for our Eduroam set-up. Our instance (radius.terena.org is connected upstream to SURFnet's radius servers.
I worked together with SURFnet to do some IPv6 debugging, and since 12 April 2011 the TERENA-SURFnet radius connection is using only IPv6
In the next months we will be participating in a Eduroam-as-a-service pilot. This would mean our current (IPv4 only) Cisco AP1200's would not need to connect any more.
That would leave our Cisco ASA5505s as the only IPv4 radius clients.
If Cisco fixes that, then radius.terena.org can become IPv6 only as well.

Listmanager

This is a custom email list manager, running on Erasmus. 2 lists were doing queries to ldap.terena.org. Unfortunately the Net::LDAP in Ubuntu Hardy (libnet-ldap-perl) is too old and does not recognise the inet6 paramater. Hacked Fixed by copying /usr/share/perl5/Net/LDAP.pm from a Lucid box.

Nmap

Nmap only recognizes IPv6 resolvers by specifying "--system-dns"

Mtr

Name resolution does not work with IPv6 only name servers

Exact Globe 2003

This financial software package runs on Windows 2003 Server, which supports IPv6.
It also requires Microsoft SQL Server, and the version we run (SQL Server 2005) seems to speak IPv6 as well.
So based on that it looks like it could work.
However, after more close inspection it does not look too encouraging:

• The update process involves logging in and downloading patches from https://update.exactsoftware.com and https://eportal.exact.nl, which are IPv4 only.
• The main Exact website is IPv4 only.
• All the different sub sites are IPv4 only:
  • https://customers.exact.com/
  • https://partners.exact.com/
  • https://employees.exact.com/
• Searching for "ipv6" on the Exact website yields 0 results.
• Phoning Exact customer support if their software will work with IPv6: "What? IPv6? What kind of operating system is that?" +1 for this, but still FAIL.

Having a software package on a dedicated Windows server, with a dedicated SQL server is quite some overhead, so I was interested in their new web based product Exact Online.
The Exact Online web site (surprise surprise) can't be reached via IPv6. But if everything is just running through HTTP(S), then a NAT64/DNS64 solution might make things work.

WinSCP

Upgrade to 4.2.8 or later to get IPv6 going.

EMS PostgresQL/MySQL manager

Tunneling via SSH does not work. Native Postgres connections work, so the bug must be in sshfwd.dll.
Confirmed by EMS, but not yet fixed.
Work-around was to not use SSH tunneling anymore, but directly connecting to the database server. Have set up proper rules in pg_hba.conf.
Since the cryptographic shield of SSH was now gone, I have configured all non-local entries in pg_hba.conf to force SSL, such as:

# Erasmus
hostssl all all 2001:610:148:dead::2/128    password

TurtoiseSVN

Works, but some weird things: I had some repositories checked out with TurtoiseSVN, using my SSH keys from Putty/Pageant. Any actions on the repository started to have a really long delay after switching off IPv4 on the subversion server. Fixed after using the right repository URL format, in my case using the Putty session name instead of the host name. This session has everything set properly already. In my case the hostname is svn.terena.org, and the PuTTY session name is svn.

Windows Remote Desktop Client

The "Remote Desktop client" in Windows 7 (mstsc.exe) has some weird behavior. An RDP connection to a Windows 7 computer using a hostname that only has a AAAA record takes 11 seconds. mstsc.exe does an A query first, gets back a No such name, then wait 11 seconds, then asks for and receives the AAAA record, and then immediately connects. |

FWbuilder

We use this excellent tool to create and install iptables/ip6tables scripts on our Ubuntu hosts.
FWBuilder is the tool for easy, understandable, object based firewalls.
We have a commercial license because I wanted the Windows version, but I think it's worth every penny. Linux versions are GPL though.
There is a bug in the batch installer that makes it choke on IPv6 only hosts, but that is fixed as of 2011-04-27, in build 3532.
Friendly developer, and very responsive. Has fixed numerous corner cases that I have run into.

Mac OS X

DNS

2011-07-20 OSX Lion finally supports DHCPv6 Now autoconfiguration can work!

CIFS

CIFS client on Mac OS X does not support IPv6. Unable to file bug report due to lame web site ("An error has occurred. Please report the error to Apple Inc. by emailing the error detail to devbugs@apple.com.").
CIFS via IPv6 works in Lion. After upgrading all the Macs to Lion I can make our Samba server IPv6-only

CyberDuck

CyberDuck does not work with IPv6 hostnames. Use either literal IPv6 address, or IPv6-only host name godzilla.ipv6.terena.org.
Semi-fixed in 4.0, but the default is to use IPv4 addresses when an address has both: http://trac.cyberduck.ch/ticket/2539#comment:15

So in order to use Cyberduck in TERENA, you'd have to change the defaults:

defaults write ch.sudo.cyberduck connection.dns.ipv6 true

To Do

• VMware ESXi, Vcenter, VCB. These SHOULD work on IPv6. Curious if they really work on IPv6 only...
• Reconfigure Nagios checks so that either IPv6, IPv4, or both are monitored.