I was wondering if it would be possible to run parts of the TERENA Secretariat office network on IPv6 only.
Our office has a /24 IPv4 and a /48 IPv6 network .
Test to see if it would be possible to run parts of the TERENA Secretariat office network on IPv6 onlysince 2002, and all our public services are available on IPv4 and IPv6.
Since we have native IPv6 connectivity in our office, it seems that the first step would be to remove IPv4 from services that are only used internally.
This page keeps track of progress, bugs, issues with this transition.
Milestones
and issues.
BTW, this is not the first time this has been tried out. For instance Arkko & Keranen did some interesting work in 2010.
Jump to:
Table of Contents | ||||
---|---|---|---|---|
|
Milestones
Date |
|
| ||
---|---|---|---|---|
2012-08-02 | Remote support only on IPv6 | VNC on client computers runs in IPv6 only mode. | ||
2012-01-13 | All database are running exclusively on IPv6 | Today I killed the last two instances of MySQL, which ran on our ticket system JIRA and this Confluence wiki, and replaced them with PostgreSQL. Now we only run PostgreSQL databases, that are all accessible over the network exclusively through IPv6. | ||
2011-12-20 | Axis IP camera now IPv6 only | The RTMP stream on http://www.terena.org/webcam originates from our media server media.terena.org, which is dual stack. That server pulls the stream from our AXIS Q1755 camera with host name axis.terena.org, which is now IPv6 only. Removed IPv4 address and A record, adjusted access-lists. | ||
2011-12-05 | iPhones and iPads can now use IPv6 from everywhere | Added Cisco AnyConnect mobile license to our our ASA5505, so mobile devices can have IPv6 for everywhere. | ||
2011-10-31 | Our primary file server runs Samba on IPv6 only | Removed IPv4 address 192.87.30.132 | ||
2011-04-12 | Radius communication is IPv6 | All Eduroam Radius traffic between TERENA and the Radius servers at SURFnet is done using IPv6 only. | ||
2011-03-09 | All linux servers run PostgreSQL on IPv6 only | Configure " | ||
2011-03-08 | HP Printer IPv6 only | Upgraded Laserjet 4250 with new print server, removed IPv4 address 192.87.30.138 and A record for | ||
2011-02-28 | Primary directory server | Removed IPv4 address 192.87.30.30 | ||
2011-02-07 | Nagios web interface only reachable on IPv6 | Configure | ||
2011-02-07 | SVN repository | Removed IPv4 address 192.87.30.29 and A record | ||
2011-02-01 | ||||
| All linux servers use IPv6 resolvers | Only statically configured IPv6 addresses in | ||
| All linux managed by SSH on IPv6 only | Configure | ||
| All linux servers run Postfix on IPv6 only (except public mail serverlisted MXs) | Remove IPv4 address addresses from | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="58399f33-7bb7-4005-91a4-715cc6187116"><ac:plain-text-body><![CDATA[ | | Nagios runs only on IPv6 | Configure | ]]></ac:plain-text-body></ac:structured-macro> |
| Entire host IPv6 (removed IPv4 address and A record) | | ||
| All PostgreSQL server use IPv6 | Should be configured on all hosts (web, mail, dev, etc). To do. |
Misc issues
...
status
...
description
...
...
...
NAT64
...
...
...
DNS64
...
...
...
Cisco 3750 does not support IPv6 access lists on VLANs
...
Needs replacing in 2011 anyway. New box might support NAT64?
...
...
Google Mini does not support IPv6
" | ||||
2011-01-31 | All linux servers managed by SSH via IPv6 only | Configure " | ||
2011-01-30 | All linux servers use only IPv6 resolvers | Only list IPv6 addresses in
| ||
2010-10-26 | VPN supports IPv6 | A new VPN setup: Cisco AnyConnect. Clients get an IPv4 and an IPv6 address from the office pools, so they can access all services via IPv6 |
Tinyproxy
Some of our systems have extra "security needs", and they are not allowed to initiate outgoing connections by default. This means that IP ACLs are used so that they can only reach neccessary services (SMTP gateway, DNS resolvers, NTP etc).
Because those hosts do need access to some web sites (mostly for software updates), we use a proxy server to allow them access to those domains.
If you have an IPv6-only host that only needs access to some outside HTTP resources, then this approach kills two birds with one stone:
- Many services are run on CDNs such as Akamai, which renders IP ACLs into a nightmare. A proxy solves this by allowing domains/URLs.
- Some services are only accessible via IPv4 (Microsoft Update, hostupdate.vmware.com, Secunia.com). A dual stack proxy does the protocol translation. If those web sites were the only problems on the IPv6-only system, this is just what you need, and you can avoid using additional complex systems such as NAT64/DNS64.
Because we do not need any caching, but only the access restriction part, I choose tinyproxy because it is very light weight and simple.
The only downside is that the tinyproxy that sits in Ubuntu 12.04 does not listen on both IPv4 and IPv6 at the same time
2013-01-04 Shame on me... I didn't test properly , but tinyproxy does work on both protocols
I assumed that this result:
netstat -tlnpvw | grep tinyproxy
tcp6 0 0 :::8888 :::* LISTEN 3946/tinyproxy
meant that it didn't listen on v4... but I was wrong.
Listen ::
#This will accept connections on IPv6, but also on IPv4: IPv4-mapped IPv6 addresses are used:
#CONNECT Jan 04 15:29:13 [23566]: Connect (file descriptor 6): host.terena.org [::ffff:192.87.30.2]
Listen 0.0.0.0
# This will listen on IPv4 only
Listen 2001:610:148:dead::666
# This will listen only on the specified IPv6 address. Not nice, but workable.
Coda 2 drops support for IPv6
One of my users that joined our company recently needed to do some web editing stuff on her Apple computer. Other users have good experience with the Coda editor from panic.com. So I ordered and installed it from the Apple Store. Hadn't done this earlier but seems convenient. It was the new version 2 - whatever, let's go.
For some reason she could not connect to our web server. "Could not connect to server", "Not connected.":
Checked what was wrong and found out that version 2 dropped support for IPv6 , which renders it useless in our IPv6 only environment.
Copied the latest old version (1.7.5) and that works just fine. Questions about this on the Coda support list confirmed by fears.
Hopefully they add back IPv6 support, perhaps with a configuration switch. Until that time we won't upgrade.
As of December 2012, Coda is at version 2.0.7, and still no IPv6 support.
Remote support with IPv6-only VNC
While looking for a cross-platfrom remote support solution that does IPv6, I ended up using commercial software this time: RealVNC Enterprise Edition. This does some other handy things as well, such as AES encryption, system authentication, and an Accept/Reject dialog on the client.
By default it listens on both IPv4 and IPv6 but it can be easily configured to listen only on IPv6:
I use this to remotely support people. One important issue: you do have to know the network address before you can connect to it . There are two possibilties:
- In the office. All our computers autoconfigure themselves with the modified EUI64 interface identifier, which is based on the interface's MAC address. This works out of the box for Mac OSX 10.7/10.8, but on Windows you have to manually configure this (once). Note that this does not have to clash with privacy extensions; those can (and should, IMHO) still be disabled. This means that computers will have a(n extra) predictable address in the office, so you can assign easy to remember hostname to them, such as 'laptop-alice-wifi', 'laptop-alice-wired', etc. Note that this does not have to interfere with.
- Out of the office. People working from remote need to set up their Cisco AnyConnect VPN connection. Once that is done, they have IPv6 connectivity, bypassing any NAT boxes they might be behind. They either need to tell me their network address, or I can look it up on our VPN server.
An alternative for locating machines would be dynamic DNS, but I have not found an end-user-friendly and easy-to-administer cross platform solution for that - yet.
Microsoft AutoUpdate does not work in an IPv6-only environment
2011-03-28. Upon installing a Mac with OSX 10.7 and the applications we use, I connected it to our IPv6-only network. Everything went fine, until I tried to update Microsoft Office for Mac 2011:
Because the network does NAT64/DNS64, this must mean that AutoUpdate does some weird IPv4-only stuff. After connecting it temporarily to the dual stack network, I did get the updates immediately.
I created a couple of tcpdumps, and it looks like the AutoUpdate tool has some hardcoded logic that makes it only use A records for download.microsoft.com, which of course fail:
Interestingly, had the tool queried for a AAAA record, it would have worked:
Code Block |
---|
administrators-macbook-pro:~ root# host -t aaaa download.microsoft.com
download.microsoft.com is an alias for download.microsoft.com.nsatc.net.
download.microsoft.com.nsatc.net is an alias for main.dl.ms.akadns.net.
main.dl.ms.akadns.net is an alias for intl.dl.ms.akadns.net.
intl.dl.ms.akadns.net is an alias for dl.ms.georedirector.akadns.net.
dl.ms.georedirector.akadns.net is an alias for a767.ms.akamai.net.
a767.ms.akamai.net has IPv6 address 2001:610:148:ffff:b0b0::58dd:d830
a767.ms.akamai.net has IPv6 address 2001:610:148:ffff:b0b0::58dd:d83a |
Boo!!!
Ubuntu servers not ready
Trying to debug OpenDNSSEC packages on one of my IPv6-only systems but unfortunately some bumps in the way. The system keyserver.ubuntu.com does not have IPv6 connectivity, so adding GPG keys using apt-key does not work:
Code Block |
---|
root@svn:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7C1A977926535DB3
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv-keys 7C1A977926535DB3
gpg: requesting key 26535DB3 from hkp server keyserver.ubuntu.com
gpgkeys: HTTP fetch error 7: Failed to connect to 91.189.89.49: Network is unreachable
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
|
Waiting for response at https://answers.launchpad.net/launchpad/+question/189094
Once manually copy/pasted they key, the next hurdle is ppa.launchpad.net:
Code Block |
---|
Reading package lists... Done
W: Failed to fetch http://ppa.launchpad.net/pkg-opendnssec/ppa/ubuntu/dists/lucid/Release.gpg Could not resolve 'ppa.launchpad.net'
W: Failed to fetch http://ppa.launchpad.net/pkg-opendnssec/ppa/ubuntu/dists/lucid/main/i18n/Translation-en_US.bz2 Could not resolve 'ppa.launchpad.net'
W: Some index files failed to download, they have been ignored, or old ones used instead.
|
Added bug https://bugs.launchpad.net/launchpad/+bug/942571 for that...
Samba
Our servers run Ubuntu 10.04 LTS, which ships with Samba 3.4.7, and this has proper IPv6 support. There is no global switch to make it listen to a specific IP version, but is not too difficult to make it listen to IPv6 only.
You need to tell it to only bind to a specific (hard coded) interface address in smb.conf
:
Code Block |
---|
bind interfaces only = yes
interfaces = 2001:610:148:beef::132/64
|
This machine is a server but sits in the Office network, so it was needed to disable autoconfiguration to prevent is from accumulating multiple IPv6 addresses (never a good idea for servers).
Contrary to what you would expect, setting /proc/sys/net/ipv6/conf/eth0/autoconf
to 0 does not do the trick. You have to explicitly state the interface. So I ended up with this /etc/network/interfaces
:
Code Block | ||||
---|---|---|---|---|
| ||||
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet6 loopback
pre-up ip addr del 127.0.0.1/8 dev lo
# The primary network interface
auto eth0
iface eth0 inet6 static
address 2001:610:148:beef::132
netmask 64
gateway 2001:610:148:beef::1 dev eth0
pre-up echo 0 > /proc/sys/net/ipv6/conf/eth0/autoconf
|
MacOSX Lion & Cisco AnyConnect
Late July 2011 Apple released version 10.7 of their OSX operating system, named Lion. This version has several major IPv6 related improvements, the most important I think is DHCPv6 support. This means that it is now possible to successfully run a Mac in an IPv6 only environment without any configuration.
Unfortunately for us AnyConnect has a serious bug on Lion, namely that there is no default gateway being configured for IPv6 upon connection.
Since we have several IPv6 only services these days, this is a true show stopper
What makes it worse it that the smbd in Lion has IPv6 support, and because our Windows 7 computers already support SMB via IPv6, this means that we could make our Samba server IPv6 only. But since AnyConnect does not work, this is not (yet) an option...
The issue has been reported already to Cisco and is filed as CSCts11510 (login required).
Rumour has it that a fix is available soon, so let's just keep our fingers crossed!
Update: As of 29 September 2011, AnyConnect 3.0.4235 fixes the problems! Now all my users can have IPv6 again from everywhere
Specific issues
Name resolution
To avoid name resolution problems, it was sometimes necessary to copy to the legacy 127.0.1.1
entries to ::1
in the /etc/hosts
file:
Code Block |
---|
127.0.0.1 localhost
127.0.1.1 ldap.terena.org ldap
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback ldap.terena.org ldap
|
127.0.0.1
On IPv6 only hosts, there is no IPv4 address configured on the network interface, so obviously all communications will go via IPv6.
There is still an IPv4 address (127.0.0.1) sitting on the loopback interface lo
. It's doesn't hurt but it should not be there if the host were to be truly IPv6 only
I could not find anywhere in /etc
where this address get added.
Since I can prevent it from getting added, I removed it after it gets added, by hacking configuring /etc/network/interfaces
:
Code Block |
---|
# The loopback network interface
auto lo
iface lo inet6 loopback
pre-up ip addr del 127.0.0.1/8 dev lo
|
This seems to work fine, only now ntp dumps core, but that has been fixed as of 17 April 2011.
Skype
Skype does not support IPv6 at all FAIL!!!! Please VOTE FOR IPv6 (<- this link appears to be dead as of January 2012, and the original poll is gone it seems... wonder why that happened...)
I've set-up a poll on http://www.change.org/petitions/skype-add-ipv6-support-to-skype , please vote there so Skype can be persuaded to add IPv6 support
FYI the first request for IPv6 enabled Skype date back to 2004!!
Cisco Catalyst 3750
This switch (WS-C3750G-24TS-E) does not support IPv6 access lists on VLANs.
2011-10 Replaced with a WS-C3750X-48T-S and a WS-C3750X-48PF-S.
This one does IPv6 access lists on VLANs, but I had to use sdm templates for this. In hindside the old one might have supported it as well.
But because that died after powering it off to take it out of service, no way to find that out anymore
Cisco AP1200
These access points do not support IPv6. Need replacing anyway. The AIR-AP1142N-E-K9 could be a drop-in replacement. Also does N.
December 2011: after successful participation in the Eduroam-as-a-Service from SURFnet/Wentzo wireless, these accesspoints were not needed any more.
I donated them to the System & Network Engineering master education of the University of Amsterdam.
Sharp AR-M276
This copier/printer does not support IPv6 at all.
Could not retrieve e-mail addresses for 'scan to email' after LDAP server went IPv6 only. Hack Work-around: manually put addresses in.
Our big Sharp MX2600n has IPv6 support, so we should get rid of this clunker on the first occasion.
Google Mini
This box does not support IPv6. Needs replacing in 2011 anyway, but don't forget to check!!
...
Axis Q1755 web cam
Can be configured to do IPv6, but only PING works
Investigate further.
2011-12-20 Reset the cam and started with fresh config, now works fine over IPv6.
Sharp MX-2600N printer
After enabling IPv6 on our Sharp MX2600N printer, the network stack actually works, but only a couple of services are running IPv6:
No Format |
---|
root@expat:~# nmap -6 2001:610:148:beef::134
Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-01 10:31 CET
Interesting ports on 2001:610:148:beef::134:
Not shown: 996 closed ports
PORT STATE SERVICE
21/tcp open ftp
23/tcp open telnet
515/tcp open printer
631/tcp open ipp
Nmap done: 1 IP address (1 host up) scanned in 2.29 seconds
|
This is in stark contrast to what runs on IPv4:
No Format |
---|
root@expat:~# nmap --system-dns 192.87.30.134
Starting Nmap 5.00 ( http://nmap.org ) at 2011-03-01 10:35 CET
Interesting ports on sharp-mx2600n.terena.org (192.87.30.134):
Not shown: 991 closed ports
PORT STATE SERVICE
21/tcp open ftp
23/tcp open telnet
80/tcp open http
443/tcp open https
515/tcp open printer
631/tcp open ipp
5900/tcp open vnc
9100/tcp open jetdirect
50001/tcp open unknown
Nmap done: 1 IP address (1 host up) scanned in 2.37 seconds
|
Now I'm trying to find out how to print using IPP from Windows 7.
Ecdysis
...
...
Windows 2003 + MS SQL 2005 + Exact Globe 2003 != ??
...
Check if this works...
Some tests indicate the Ecdysis works well.
Also, they presented at our own conference last year
Take into consideration!
Linux issues
PECL radius
IPv6 doesn't work. Needed for TERENA web site. Update 2011-03-01: by upgrading Pear_Auth, Pear Live_User was able to use LDAP (via IPv6), without the Radius overhead.
MySQL
MySQL at this moment does not support IPv6 connections, but the development versions seem to support it (sort of). There are tools to make it work, such as https://twiki.cern.ch/twiki/bin/view/EGEE/IPv6CARE.
Not A Problem Here: we have only one host running MySQL, and that will be phased out in the future any way.
Update 2013-03-22: We were forced to use MySQL for some stuff, but the version in Ubuntu 12.04 appears to have IPv6 support . I configured it to listen to IPv6 only:
Code Block |
---|
bind-address = '::' |
...
apt-get
security.ubuntu.com
does not work, so no security updates. Workaround: use local mirror nl.archive.ubuntu.com
for security updates.
...
Update 2013-03-12: This has been fixed : https://lists.ubuntu.com/archives/foundations-bugs/2013-March/140037.html
Pear Net_Socket
Does not like IPv6 addresses, wrote patch.
...
ntp
...
. We are able to use SURFnet's
chime3.surfnet.nl
andchime4.surfnet.nl
however:Code Block visser@svn:~$ ntpq -pn remote refid st t when poll reach delay offset jitter ============================================================================== +2001:610:508:11 .GPS. 1 u 360 1024 377 4.499 -0.340 0.483 *2001:610:0:800b .PPS. 1 u 192 1024 377 4.746 1.525 0.089
...
...
- This has been fixed in 1:4.2.4p8+dfsg-1ubuntu2.1
.
Confluence and JIRA LDAP auth via IPv6
...
Workaround: use IPv6 only hostname: ldap.ipv6.terena.org
Authentication on dual stack LDAP servers does not work. Using an IPv6-only hostname does work. For us this works, because our LDAP server is IPv6 only.
...
Radiator
Cannot use IPv6 LDAP server.
...
Filed support ticket at Open.com.au. Fixed as of 2011-02-12. Also make sure to add flags to any custom perl hooks:
my $ldap = Net::LDAP->new('ldap://ldap.terena.org',inet6=>1);
.
We use Radiator for our Eduroam set-up. Our instance (radius.terena.org
is connected upstream to SURFnet's radius servers.
I worked together with SURFnet to do some IPv6 debugging, and since 12 April 2011 the TERENA-SURFnet radius connection is using only IPv6
In the next months we will be participating in a Eduroam-as-a-service pilot. This would mean our current (IPv4 only) Cisco AP1200's would not need to connect any more.
That would leave our Cisco ASA5505s as the only IPv4 radius clients.
If Cisco fixes that, then radius.terena.org
can become IPv6 only as well.
Listmanager
This is a custom email list manager, running on Erasmus. 2 lists were doing queries to ldap.terena.org
. Unfortunately the Net::LDAP in Ubuntu Hardy (libnet-ldap-perl
) is too old and does not recognise the inet6 paramater. Hacked Fixed by copying /usr/share/perl5/Net/LDAP.pm
from a Lucid box.
Nmap
Nmap only recognizes IPv6 resolvers by specifying "--system-dns
"
Mtr
Name resolution does not work with IPv6 only name servers
Windows issues
Exact Globe 2003
This financial software package runs on Windows 2003 Server, which supports IPv6.
It also requires Microsoft SQL Server, and the version we run (SQL Server 2005) seems to speak IPv6 as well.
So based on that it looks like it could work.
However, after more close inspection it does not look too encouraging:
- The update process involves logging in and downloading patches from https://update.exactsoftware.com and https://eportal.exact.nl, which are IPv4 only.
- The main Exact website is IPv4 only.
- All the different sub sites are IPv4 only:
- Searching for "ipv6" on the Exact website yields 0 results.
- Phoning Exact customer support if their software will work with IPv6: "What? IPv6? What kind of operating system is that?" +1 for this, but still FAIL.
Having a software package on a dedicated Windows server, with a dedicated SQL server is quite some overhead, so I was interested in their new web based product Exact Online.
The Exact Online web site (surprise surprise) can't be reached via IPv6. But if everything is just running through HTTP(S), then a NAT64/DNS64 solution might make things work.
...
WinSCP
Upgrade to 4.2.8 or later to get IPv6 going
...
.
EMS PostgresQL/MySQL manager
Tunneling via SSH does not work. Native Postgres connections work, so the bug must be in sshfwd.dll
.
Confirmed by EMS, but not yet fixed.
Work-around was to not use SSH tunneling anymore, but directly connecting to the database server. Have set up proper rules in pg_hba.
...
Mac OS X issues
...
...
Autoconfiguring name servers does not work for Mac OS X
...
Macs need manually configured name servers, boo!
conf
.
Since the cryptographic shield of SSH was now gone, I have configured all non-local entries in pg_hba.conf
to force SSL, such as:
Code Block |
---|
# Erasmus
hostssl all all 2001:610:148:dead::2/128 password
|
TurtoiseSVN
Works, but some weird things: I had some repositories checked out with TurtoiseSVN, using my SSH keys from Putty/Pageant. Any actions on the repository started to have a really long delay after switching off IPv4 on the subversion server. Fixed after using the right repository URL format, in my case using the Putty session name instead of the host name. This session has everything set properly already. In my case the hostname is svn.terena.org
, and the PuTTY session name is svn
.
Windows Remote Desktop Client
The "Remote Desktop client" in Windows 7 (mstsc.exe
) has some weird behavior. An RDP connection to a Windows 7 computer using a hostname that only has a AAAA record takes 11 seconds. mstsc.exe
does an A query first, gets back a No such name, then wait 11 seconds, then asks for and receives the AAAA record, and then immediately connects. |
FWbuilder
We use this excellent tool to create and install iptables/ip6tables scripts on our Ubuntu hosts.
FWBuilder is the tool for easy, understandable, object based firewalls.
We have a commercial license because I wanted the Windows version, but I think it's worth every penny. Linux versions are GPL though.
There is a bug in the batch installer that makes it choke on IPv6 only hosts, but that is fixed as of 2011-04-27, in build 3532.
Friendly developer, and very responsive. Has fixed numerous corner cases that I have run into.
Mac OS X
DNS
2011-07-20 OSX Lion finally supports DHCPv6 Now autoconfiguration can work!
CIFS
...
CIFS client on Mac OS X does not support IPv6
...
. Unable to file bug report due to lame web site ("An error has occurred. Please report the error to Apple Inc. by emailing the error detail to devbugs@apple.com.")
...
.
CIFS via IPv6 works in Lion. After upgrading all the Macs to Lion I can make our Samba server IPv6-only
CyberDuck
...
CyberDuck does not work with IPv6 hostnames
...
. Use either literal IPv6 address, or IPv6-only host name godzilla.ipv6.terena.org
...
Misc
.
Semi-fixed in 4.0, but the default is to use IPv4 addresses when an address has both: http://trac.cyberduck.ch/ticket/2539#comment:15
So in order to use Cyberduck in TERENA, you'd have to change the defaults:
Code Block |
---|
defaults write ch.sudo.cyberduck connection.dns.ipv6 true |
To Do
...
...
Axis Q1755 web cam
...
Can be configured to do IPv6, but only PING works
...
- VMware ESXi, Vcenter, VCB. These SHOULD work on IPv6. Curious if
...
- they really work on IPv6 only...
- Reconfigure Nagios checks so that either IPv6, IPv4, or both are monitored.
...