| CARVIEW |
With the Google IPv6 statistics showing the number of IPv6 connections to Google services approaching 10%, it’s perhaps an opportune time to sign off for the year with the wish that 2016 will be the year that IPv6 deployment goes mass market.
In fact according to Google, IPv6 deployment already saw a 60% increase in 2015, but with the IPv4 address pool essentially exhausted in the ARIN region, there are now clear signs of IPv6 being widely deployed in North America and parts of Latin America. Over in Europe there are also significant success stories with Belgium and Germany leading the way, with other major deployment programmes in the pipeline in the UK, France and the Netherlands.
However, the Asia-Pacific and African regions lag significantly in IPv6 deployments, which is especially concerning in the Asia-Pacific, a region experiencing rapid Internet growth but with a very limited supply of remaining IPv4 addresses. It’s therefore worth again pointing again to RIPE/APNIC Labs article explaining how IPv6 gives comparable performance to IPv4, but without the limitations or difficulties in obtaining address space.
So if you have some spare time over the holiday season, please do look at our Start Here page to understand how you can get started with IPv6.
With this, the Deploy360 team wishes everyone Happy Holidays and a very IPv6 deployable New Year!
]]>
Akamai recently published its third quarterly report for 2015 on ‘The State of the Internet‘. As a major content delivery network provider, Akamai is able to collate substantial amounts of data on many metrics including connection speeds, network availability, traffic patterns, and IPv6 adoption.
Whilst the number of unique IPv4 addresses connecting to Akamai continues to increase, there’s substantial evidence this trend is starting to level off as available IPv4 addresses become more scarce. Indeed, the statistics for IPv4 show small declines in some countries like the US, Germany and India which suggests implementation of IPv4 conservation measures such as Carrier Grade NAT, or increased IPv6 adoption.
Belgium remains the country with the highest percentage (34%) of content requests made over IPv6, despite a 8.4% quarterly drop. Switzerland, the United States, Peru and Germany follow with between 17 and 20% traffic, although with the exception of Peru, all saw a decrease from the previous quarter. On the upside, Greece and Estonia saw a sizeable increases in their IPv6 traffic with 37% and 20% respectively.
Cable and mobile providers continue to push IPv6, particularly in the US (Comcast Cable, AT&T, Verizon Wireless, T-Mobile and Time Warner Cable), Belgium (TELENET), Peru (Telefonica del Peru), Switzerland (Swisscom), Germany (Deutsche Telekom and Kabel Deutschland) and Portugal (Sapo). BT has also announced plans to IPv6 enable 100% of its UK network by the end of 2016.
Finally, the report notes that ARIN exhausted its pool of IPv4 addresses in July 2015 and was forced to wait list thirteen requests until it received additional address space from IANA and recovered addresses from other organisations. Whilst the other Regional Internet Registries do still have address space remaining, the substantial transfer activity during the quarter suggests that organisations are increasingly concerned about the availability of address and were taking action to ensure continuity of their business.
It’s clear that organisations really do now need to be actively deploying IPv6, so please take a look at our Start Here page to understand how you can get started transitioning your networks, devices and applications!
]]>
In short: It’s doable and your government should support it!
Over 4 years ago, the Go6 Institute started a discussion with the Slovenian government about the idea for a Slovenian IPv6 roadshow project. This would include a web portal with basic technical and deployment information about the IPv6 protocol, as well as one-day basic IPv6 workshops that would be free for everybody to attend, but more importantly, outside of the capital city (Ljubljana).
The motivation behind this initiative was the fact that the majority of IPv6 related conferences, meetings and workshops were usually held in Ljubljana as the density of Internet experts in that area is higher than elsewhere. But is using this as a reason to organize IPv6 events there fair? We thought not, and this resonated with Internet experts living outside the Ljubljana region.
“If the mountain won’t come to Muhammad, then Muhammad must go to the mountain…”
People from busy IT departments, enterprises, operators and other parts of industry are involved with their work and don’t have the time and resources to travel for several hours to join a workshop about something they are not entirely sure they need to know about. Of course, usually after joining a meeting or workshop they realize how important the knowledge is and change their mind, but before this happens, they have hesitations about whether it is a good use of their time.
The aim was to change this mentality by bringing an IPv6 workshop to their city that is free to attend, and to encourage them to see IPv6 as part of their future professional expertise and required knowledge.
The Slovenian government always stressed the importance of providing education and learning opportunities to everyone in the country, so we saw the perfect opportunity to partner this vision with increasing knowledge and awareness of IPv6 awareness. We therefore need to thank the Ministry of Education, Science and Sport and Arnes, the Slovenian NREN for supporting this pilot project.
This project was announced in September 2015, and needed to be completed before the funding expired at the end of 2015. We decided to divide the project into two parts – developing a web portal with IPv6 information in Slovenian, as well as organising two IPv6 workshops in Nova Gorica and Maribor – two cities at diametrically opposite ends of the country.
The web portal now proudly lives at https://ipv6.si/ and is still under development, but the content is nearly complete and should be available next week. The aims is to gather together IPv6 information in Slovenian to become a reference point for any citizen needing to understand and deploy IPv6 in their networks and services. Our wish is to continue to develop the portal – adding new protocols, tutorials and workshop materials over time.
IPv6 workshop in Nova Gorica
The two IPv6 workshops took place this week on 15 December 2015 in Nova Gorica, and 17 December 2015 in Maribor. The first had over 80 participants, whilst the second had around 40 participants, so both can be considered a great success. Arnes also recorded the proceedings of the Maribor workshop which will appear on https://ipv6.si/ shortly.
IPv6 workshop in Maribor
Participants were highly interested in learning about IPv6 and Matjaž Straus Istenič and Luka Manojlovič introduced them to the IPv6 protocol, how to start, and the importance of gaining a lot of experience in order to be able to run trouble-free networks and services
The workshops also covered why we need to implement IPv6, addressing and making address plans, accompanying protocols, ICMPv6 services, and the usage of different auto configuration mechanisms such as SLAAC and DHCPv6. More advanced topics included DNS and IPv6, privacy and traceability in IPv6, a look at IPv6 security issues, and transitional mechanisms like A+P (MAP), 6to4, NAT64 and others.
One of participants said after the workshop: “If you guys had not organised this workshop, I would not have learnt about IPv6 to the extent that the whole thing would catch my attention and interest to start testing and experimenting with it. Quite simply, I have too much other work and would not have been able to reprioritize my other assignments to start learning about IPv6 from the Internet. I live and work nearby and in a single day I learned the basics of IPv6, lost the fear of an unknown new protocol, and actually obtained enough knowledge to encourage me to start playing with it at home. When I get comfortable with it, I’ll start thinking about implementing it at work…”
From the feedback received, it’s clear this initiative was welcomed as a good to way to encourage people to deploy IPv6. We therefore plan to talk our government and NREN about continuing this activity next year, as the web site is nearly ready, workshop material has been prepared, and a team well trained in its use.
We’d like to thank everyone who made this pilot project happen, including the Ministry of Education, Science and Sport; Arnes; IZUM Maribor; Šolski center Nova Gorica; Go6 Institute, the Internet Society; and other everyone else that helped make the workshops a bit better.
We’re also interested in hearing if there are similar initiatives in other countries, and would encourage other governments to support the deployment of IPv6 in this way.
Jan Žorž opening the workshop
Luka Manojlovič and Matjaž Straus Istenič at full IPv6 swing
Davor Šoštarič, IZUM CEO and event host – co-opening the IPv6 workshop in Maribor
Local guest insert: David Brezovšek from University of Maribor talking about DHCPv6
Luka Manojlovič, Microsoft MVP during his part of the workshop action
Learning, learning, learning…
Matjaž Straus Istenič, a.k.a. Dr “Sixter” during his IPv6 rock’n’roll
It’s worth drawing attention to the Study and Measurements of the RPKI Deployment. This is a recently published thesis analysing the deployment of RPKI and the quality of the data, but is also worth reading for its comprehensive documentation of routing incidents, the problems they can cause, and mitigation measures that can be implemented.
The analysis reveals that the global percentage of IPv4 address space covered by a Route Origin Authorisation (ROA) was 6.03% in September 2015, although this figure varies widely between the RIR regions. The RIPE NCC and LACNIC lead the way with 18.67% and 13.87% respectively, AfriNIC comes close to the average at 5.31%, but ARIN registers just 1.98% and APNIC even further behind with just 0.40% .
Perhaps more interestingly though, an authentication analysis undertaken between March 2012 and September 2014 revealed issues with the registration of many RPKI resources, as well as a couple of RIR repositories. However, whilst the percentage of invalid RPKI-covered prefixes in 2012 was as high as 21%, this progressively dropped to just over 7% by September 2015 which indicates a decrease in problems as RPKI deployments has risen.
It’s also interesting to note that even where invalid prefixes were found, most of them were covered by another valid or not found prefix. This suggests that dropping invalid prefixes from the routing table may be less problematic than previously thought by network operators.
More Information
For more information on Securing BGP, please do look at our Start Here page to understand how you can get started transitioning your networks.
]]>
Let’s Encrypt enters public beta today, which means anyone can now sign-up for free digital certificates. Let’s Encrypt is a new trusted Certificate Authority (CA) that aims to bring down the costs of configuring secure servers in order to increase overall deployment of TLS.
This initiative offers more than just free certificates though, as it also supports automation to make the business of obtaining and managing certificates significantly less complex, whilst encouraging more frequent (90 day) renewal to limit damage from key compromise and mis-issuance. This is achieved through the Automated Certificate Management Environment (ACME) which offers a standards-based REST API allowing the agent software to authenticate that a server controls a domain, request a certificate, and then install it on a server without human intervention.
Over 26,000 certificates were issued during the closed beta trial, so the system has already been extensively tested in the wild. The CA is supported by fourteen sponsoring organisations with an interest in promoting encryption as the norm.
For more information on TLS, please do look at our Start Here page to understand how you can get started transitioning your networks, devices and applications!
]]>
The Deploy360 team is sad to the hear of the passing of Rob Blokzijl yesterday. Rob was a hugely influential character in the Internet community, having founded and then chaired RIPE until last year, but was also instrumental in the formation of a number of other important Internet bodies. It’s surely no coincidence that Amsterdam has become a major centre for the Internet, which can no doubt be attributed to Rob’s ability to bring different parties together in common goals.
Olaf Kolkman has posted his thoughts over on Internet Technology Matters, but I first met Rob in 1997 at TERENA Technical Committee meetings in the days when the RIPE NCC was still a ‘project’ of TERENA, albeit three times bigger than its parent. I not only found him good and humorous company, but always seeming to have time for everyone regardless of stature. He later proved to be good and supportive counsel whilst I was running the Domain Name Registration Forum at RIPE meetings in the period 2000-2001.
The R&E networking and RIPE communities somewhat diverged after this, so I had much less contact with RIPE and Rob in the years afterwards. However, I was pleased to be invited as a representative of TERENA to his investiture into the Dutch Order of Oranje-Nassau in 2010, which I think demonstrated his ability to remember everyone he met!
Perhaps though, this extract from “Exploring the Internet” by Carl Malamud about an early RIPE meeting provides a better testimony to his achievements…
The meeting was about to start, so I gathered my badge and one each of the dozens of handouts and went into a university-style lecture hall. Down at the bottom of the hall sat three serious-looking RIPE officials facing a room of 50 or so equally serious-looking engineers.
Rob Blokzijl was going through a long agenda, reviewing action items from the previous meeting and making announcements. I propped my jet- lagged eyes open with toothpicks and started leafing through the materials I had picked up.
RIPE was formed as a sort of anti-organization, a reaction to the total ineffectiveness of other groups in setting up a pan-European Internet. At the time RIPE was formed, there had been several years of thrashing while people tried to figure out how to make OSI into something real.
Meanwhile, several organizations with work to accomplish had been putting together TCP/IP networks. An Internet was being formed, but on an ad hoc basis. Networks were touching and accidents were starting to happen.
“What kinds of accidents were happening?” I naively asked Rob the next day while we were hiding in his office during a slack time in the working group sessions.
“Routing loops, black holes, the usual,” Rob said. To try and solve the problems, Rob had convened a group of six people from the real networks in Europe. Nuclear physics was the lead player, so in addition to Rob there were representatives from CERN in Geneva and the Italian physics community. The Italians were operating the only 2 Mbps international link in Europe at the time. To supplement the physicists, representatives from NORDUnet and EUnet also came.
At first, the six tried to work through the Réseaux Associés pour la Recherche Européenne (RARE), the association of research networking groups in Europe. RARE wouldn’t touch this issue, though. The six researchers were operating TCP/IP networks and RARE was exclusively and officially an OSI group.
Borrowing a phrase from Daniel Karrenberg of CWI that the “time was ripe for networking in Europe,” an extremely lightweight organization was founded in September 1990 with the mission of helping to ensure cooperation among European IP networks. Rob Blokzijl formally announced the formation of the group in RFC 1181, a very carefully worded one-page document.
In Europe, careful wording is crucial and I’ve seen more than one group spend several hours wordsmithing seemingly innocuous phrases. Rob proudly pointed out the triumphs in the RFC, such as the sentence that “RIPE serves as a focal point for other common activities of the participants related to IP networking.”
Apparently meaningless to the untrained eye, this sentence is rife with subtleties meant to steer a course through the shoals of politics. RIPE is a focal point, not an operational unit. It worries about only its own voluntary participants and is not trying to impose solutions on others. It deals only with IP networking and is thus not advocating against OSI but merely serves as a forum for those who have already chosen.
Another working group was devoted to the issue of routing coordination. Routing was the single biggest technical issue at the meeting, and it is in this area that RIPE made its biggest contribution. In the U.S., there was a single backbone which used the EGP protocol to announce available routes to the second-tier regional networks. The backbone was centrally managed, so all the regionals got fairly consistent information. In Europe, there was no backbone, so it was pretty much guaranteed that conflicting routing information would be propagated, leading to routing loops and black holes. RIPE took a look at the map of Europe and it became clear that four sites acted as the main hubs.
These four sites, in Stockholm, Amsterdam, CERN, and Bologna, served as the centers for large numbers of secondary networks. Stockholm was the center of NORDUnet and Bologna was the center of the Italian network. Amsterdam and CERN had huge numbers of links going out to different countries.
The four sites agreed to take the lines running from Stockholm to Amsterdam to CERN to Bologna and turn them into a de facto European backbone. The routers at the four sites were combined into a single routing domain, thus presenting a consistent view of the world. To make the backbone a reality, all four sites agreed that transit traffic could traverse their links.
This was certainly not a centrally managed, fault resilient, highly designed backbone, but it did the job of connecting the European Internet. Since the four main sites also had links to the U.S., the European Internet was well connected to NSFNET and the rest of the world.
In a centrally managed backbone environment like NSFNET, low-level networks that wish to make their availability known to the world (and thus enable communications with said world) simply have to inform the NIC or NOC running the backbone. If the low-level network doesn’t fill out the appropriate forms or jump through the appropriate hoops, the network is simply not announced and thus is effectively isolated. It would be as if a road existed into a country but there was no indication of the road on any map.
In Europe, maintaining the name space was a trickier situation. RIPE came up with a novel plan to keep information up to date. A European WHOIS service was put into place containing network names, domain names, IP addresses, and the like.
The incentive to keep this information up to date was quite simple. The four backbone sites used the WHOIS database to load their routers. Sites that didn’t keep information up to date were simply not announced over the de facto backbone.
RIPE was a simple, effective tool for coordination and instruction, all done on a shoestring. The informal nature of the group and the advocacy of the heretical TCP/IP was too much for the networking establishment and RIPE came under heavy attack at first from officially sponsored groups like RARE.
To understand why RARE would object, one has only to remember that TCP/IP continues to be referred to by many as “DoD IP.” In contrast, OSI, developed with very heavy participation from European academics and PTTs, was the open way to do networking.
While RIPE was under attack, it was also clear that IP networking was gaining a strong hold in Europe. In one of those bizarre twists of European politics, RIPE became an official RARE “activity.” Not an official working group, mind you, with the connotations of European Commission financial support, official voting by one participant per country, and a chairman appointed by the RARE Council of Administration. Simply an activity.
Under the merger and acquisition agreement, RIPE would continue as an independent group and would elect its own chairman. RARE got to bring the renegades under its umbrella and unity once again reigned, at least on the surface.
Do you have an idea for a new way to use DNSSEC or DANE to make the Internet more secure? Have you recently installed DNSSEC and have a great case study you can share of lessons learned? Do you have a new tool or service that makes DNSSEC or DANE easier to use or deploy?
If you do, and if you will be attending ICANN 55 in Marrakech, Morocco (or can get there), we are now seeking proposals for the ICANN 55 DNSSEC Workshop that will take place on Wednesday, 9 March 2016. Anyone is welcome to send in a brief (1-2 sentences) description of what you would like to talk about to:
The deadline is Monday, 14 December 2015.
Any ideas related to DNSSEC or DANE are welcome. To provide some suggestions, the full Call for Presentations is included below with a list of different ideas. You can also view the agenda of the recent ICANN 54 DNSSEC Workshop in October in Dublin to get a sense of what we talk about at these events.
These DNSSEC Workshops are great ways to bring ideas to the wider DNSSEC community. All sessions are recorded as well so that people get a chance to view them later.
If you are doing anything interesting with DNSSEC or DANE, I’d strongly encourage you to submit a proposal!
The full call for participation is below…
The DNSSEC Deployment Initiative and the Internet Society Deploy360 Programme, in cooperation with the ICANN Security and Stability Advisory Committee (SSAC), are planning a DNSSEC Workshop at the ICANN 55 meeting on 09 March 2016 in Marrakech, Morocco. The DNSSEC Workshop has been a part of ICANN meetings for several years and has provided a forum for both experienced and new people to meet, present and discuss current and future DNSSEC deployments. For reference, the most recent session was held at the ICANN meeting in Dublin, Ireland on 21 October 2015. The presentations and transcripts are available at: https://meetings.icann.org/en/dublin54/schedule/wed-dnssec.
At ICANN 55 we are particularly interested in live demonstrations of uses of DNSSEC or DANE. Examples might include:
* Email clients and servers using DNSSEC, OPENPGPKEY, or S/MIME for secure email.
* Tools for automating the generation of DNSSEC/DANE records.
* Services for monitoring or managing DNSSEC signing or validation.
* Tools or services for using DNSSEC/DANE along with other existing protocols and
services such as SSH, XMPP, SMTP, S/MIME or PGP/GPG.
* Innovative uses of APIs to do something new and different using DNSSEC/DANE.
* S/MIME and Microsoft Outlook integration with active directory.
Our interest is to provide current examples of the state of development and to show real-world examples of how DNSSEC and DANE related innovation can be used to increase the overall security of the Internet.
We are open to presentations and demonstrations related to any topic associated with DNSSEC and DANE.
If you are interested in participating, please send a brief (1-2 sentence) description of your proposed presentation to dnssec-marrakech@isoc.org by **Monday, 14 December 2015**
Examples of the types of topics we are seeking include:
1. DNSSEC activities in Africa
For this panel we are seeking participation from those who have been involved in DNSSEC deployment in Africa and also from those who have not deployed DNSSEC but who have a keen interest in the challenges and benefits of deployment. In particular, we will consider the following questions: Are you interested in reporting on DNSSEC validation of your ISPs? What can DNSSEC do for you? What doesn’t it do? What are the internal tradeoffs to implementing DNSSEC? What did you learn in your deployment of DNSSEC? We are interested in presentations from both people involved with the signing of domains and people involved with the deployment of DNSSEC-validating DNS resolvers.
2. Potential impacts of Root Key Rollover
Given many concerns about the need to do a Root Key Rollover, we would like to bring together a panel of people who can talk about what the potential impacts may be to ISPs, equipment providers and end users, and also what can be done to potentially mitigate those issues. In particular, we are seeking participation from vendors, ISPs, and the community that will be affected by distribution of new root keys. We would like to be able to offer suggestions out of this panel to the wider technical community. If you have a specific concern about the Root Key Rollover, or believe you have a method or solution to help address impacts, we would like to hear from you.
3. Implementing DNSSEC validation at Internet Service Providers (ISPs)
Internet Service Providers (ISPs) play a critical role by enabling DNSSEC validation for the caching DNS resolvers used by their customers. We have now seen massive rollouts of DNSSEC validation within large North American ISPs and at ISPs around the world. We are interested in presentations on topics such as:
* Can you describe your experiences with negative Trust Anchors and operational realities?
* What does an ISP need to do to prepare its network for implementing DNSSEC validation?
* How does an ISP need to prepare its support staff and technical staff for the rollout of DNSSEC validation?
* What measurements are available about the degree of DNSSEC validation currently deployed?
* What tools are available to help an ISP deploy DNSSEC validation?
* What are the practical server-sizing impacts of enabling DNSSEC validation on ISP DNS Resolvers (ex. cost, memory, CPU, bandwidth, technical support, etc.)?
4. The operational realities of running DNSSEC
Now that DNSSEC has become an operational norm for many registries, registrars, and ISPs, what have we learned about how we manage DNSSEC? What is the best practice around key rollovers? How often do you review your disaster recovery procedures? Is there operational familiarity within your customer support teams? What operational statistics have we gathered about DNSSEC? Are there experiences being documented in the form of best practices, or something similar, for transfer of signed zones?
5. DANE and DNSSEC application automation
For DNSSEC to reach massive deployment levels it is clear that a higher level of automation is required than is currently available. There also is strong interest for DANE usage within web transactions as well as for securing email and Voice-over-IP (VoIP). We are seeking presentations on topics such as:
* What tools, systems and services are available to help automate DNSSEC key management?
* Can you provide an analysis of current tools/services and identify gaps?
* Where are the best opportunities for automation within DNSSEC signing and validation processes?
* What are the costs and benefits of different approaches to automation?
* What are some of the new and innovative uses of DANE and other DNSSEC applications in new areas or industries?
* What tools and services are now available that can support DANE usage?
* How soon could DANE and other DNSSEC applications become a deployable reality?
* How can the industry use DANE and other DNSSEC applications as a mechanism for creating a more secure Internet?
We would be particularly interested in any live demonstrations of DNSSEC / DANE application automation and services. For example, a demonstration of the actual process of setting up a site with a certificate stored in a TLSA record that correctly validates would be welcome. Demonstrations of new tools that make the setup of DNSSEC or DANE more automated would also be welcome.
6. When unexpected DNSSEC events occur
What have we learned from some of the operational outages that we have seen over the past 18 months? Are there lessons that we can pass on to those just about to implement DNSSEC? How do you manage dissemination of information about the outage? What have you learned about communications planning? Do you have a route to ISPs and registrars? How do you liaise with your CERT community?
7. DNSSEC and DANE in the enterprise
Enterprises can play a critical role in both providing DNSSEC validation to their internal networks and also through signing of the domains owned by the enterprise. We are seeking presentations from enterprises that have implemented DNSSEC on validation and/or signing processes and can address questions such as:
* What are the benefits to enterprises of rolling out DNSSEC validation? And how do they do so?
* What are the challenges to deployment for these organizations and how could DANE and other DNSSEC applications address those challenges?
* How should an enterprise best prepare its IT staff and network to implement DNSSEC?
* What tools and systems are available to assist enterprises in the deployment of DNSSEC?
* How can the DANE protocol be used within an enterprise to bring a higher level of security to transactions using SSL/TLS certificates?
8. Hardware Security Modules (HSMs) use cases and innovation
We are interested in demonstrations of HSMs, presentations of HSM-related innovations and real world use cases of HSMs and key management.
In addition, we welcome suggestions for additional topics.
If you are interested in participating, please send a brief (1-2 sentence) description of your proposed presentation to dnssec-marrakech@isoc.org by **Monday, 14 December 2015**
We hope that you can join us.
Thank you,
Julie Hedlund
On behalf of the DNSSEC Workshop Program Committee:
Mark Elkins, DNS/ZACR
Cath Goulding, Nominet UK
Jean Robert Hountomey, AfricaCERT
Jacques Latour, .CA
Xiaodong Lee, CNNIC
Luciano Minuchin, NIC.AR
Russ Mundy, Parsons
Ondřej Surý, CZ.NIC
Yoshiro Yoneya, JPRS
Dan York, Internet Society
Last week, we highlighted a presentation given by Geoff Huston at RIPE 71 where he discussed the performance and reliability of IPv6. His conclusions were that if you can establish a connection, then IPv4 and IPv6 appear to have comparable performance, although currently the odds of establishing the connection still favour IPv4 for various reasons.
RIPE Labs have followed-up on this presentation by publishing a more detailed article on IPv6 performance on their website. As well as explaining the test methodology in more detail, it makes some interesting observations about the reliability of tunnelled connections, but also the rise in the use of Unicast IPv6. More generally, it reinforces the recommendation in RFC 7526 that 6to4 should be obsoleted.
For more information on what this means for you, please do look at our Start Here page to understand how you can get started transitioning your networks, devices and applications!
]]>Since the announcement of the Let’s Encrypt project, we’ve been enthusiastic about the idea of having an open and transparent Certification Authority (CA) that could show other CA’s that it’s possible to run a certificates business and keep it free for everyone!
So, after the Closed Beta programme was announced, we managed to get involved and secure our web servers with their certificates. Firstly, we had a look at their ACME client that’s needed if you want one of their certificates, as the only way to request a domain certificate is through interaction with a back-end API.
Firstly you need to know that a separate certificate is required for every domain and subdomain you want to secure. For example you need separate certificates for go6.si, www.go6.si, mail.go6.si, etc…
When you install the ACME tool on your system, you have a few options – or you can use Apache mode that configures your Apache web server automatically, talks to the Let’s Encrypt system to obtain an “auth token”, and publishes it on your website. It then tells the remote system to check for that file and whether it has the right content, and if that check goes through, the system will transfer the signed certificates and install them on your system. In case you are not running Apache (and for now just on Debian systems), you have other options such as Standalone mode that does a similar thing. However, since it sets up its own web server on port 80 in order to authenticate, you need to shut down your primary web server for a period of time which is not entirely ideal.
We chose Webroot mode that does the same authentication as the other methods, except that you need tell the ACME client where your webroot directory is. Then when you request a certificate for a domain, it obtains the authentication file from the remote system and puts it in your web server root directory so the letsencrypt.org system can access it and verify that you really control the server for that domain.
Here’s a quick how-to from their beta announcements post:
————-
If you’re running Apache on a recent Debian-based OS, you can try the Apache plugin, which automates both obtaining and installing certs:
./letsencrypt-auto --apache --server https://acme-v01.api.letsencrypt.org/directory \ --agree-dev-preview
To obtain a cert using a “standalone” webserver (you may need to temporarily stop your exising webserver) forexample.com and www.example.com:
./letsencrypt-auto certonly -a standalone \ -d example.com -d www.example.com \ --server https://acme-v01.api.letsencrypt.org/directory --agree-dev-preview
To obtain a cert using the “webroot” plugin, which can work with the webroot of any webserver software:
./letsencrypt-auto certonly -a webroot --webroot-path /var/www/example \ -d example.com -d www.example.com \ --server https://acme-v01.api.letsencrypt.org/directory --agree-dev-preview
Note: Currently the webroot plugin can only obtain certs for several domains simultaneously if they share a webroot.
To receive instructions for the (fairly complex) process of obtaining a cert from Let’s Encrypt by manually providing proof you control a domain:
./letsencrypt-auto certonly -a manual -d example.com \ --server https://acme-v01.api.letsencrypt.org/directory --agree-dev-preview
————-
So, after experimenting a bit with the whole thing, the command that worked for us was:
./letsencrypt-auto certonly --debug --webroot --webroot-path /var/www/html \ -d go6.si --server https://acme-v01.api.letsencrypt.org/directory \ --agree-dev-preview /etc/init.d/httpd restart
Why is –debug there? Because their client does not (sort of) support Python 2.6 anymore. This is the default on CentOS 6.x systems and upgrading it creates a high probability that you’ll break many things on your server. So if you end up having to deal with Python 2.6, the letsencrypt client will not do anything other than try to update the dependancies on your system and exit with a message that Python 2.6 is not supported. If you really mean it, you should add that –debug switch to the script. Well okay, so be it, debug it is 
go6.si secured
After obtaining the initial certificate, I had to fix the content on the server https://go6.si/ as some pictures were referenced with https:// at the beginning of the URL and the verification system began complaining that some content was not secured. Well, now it is and the lock on the left of URL glows green.
So, the first step went quite smoothly and was not that hard at all, but with the understanding that the certificate was only valid for 90 days, we wanted to make the renewal automatic. However, how can we make it automatic if the client requires dialogs for entering email during the initial procedure?
Well, we tried to run it again with same parameters, but it started asking questions and after some research we found out that adding –renew to the script automagically skips all the questions and becomes perfect for automation. So, now we have our cron system ready to renew that certificate on the first day of every month at 00:55, fingers crossed
During the process we encountered some small issues and questions that we’re still awaiting an answer to. One of the issues was that you can’t currently obtain a certificate for a server on an IPv6-only network as Let’s Encrypt seems to have some issues with the data centers where they have their system installed, although they say they’re working to rectify this.
Let’s Encrypt uses DNSSEC validating resolvers (unbound) and this ensures a quite high level of protection during the authentication process for DNSSEC signed zones. For non-signed zones, they are querying geographically sparse DNS resolvers and comparing the results – if results are not consistent, an attack is happening and validation fails. They also plan to rotate the resolving servers, so somebody could not build a sparse enough attack to poison all their DNS servers.
What we’d also hope is that Let’s Encrypt will put a big sign on on every corner of their webpage saying “Please sign your domain with DNSSEC and make the verification for certificate delivery more resilient to attacks, and by the way, your domain and services would be much more secure this way…” in order to encourage people to deploy DNSSEC. This is in the interests of everyone, and we already showed how deploying DNSSEC, signing domains and doing a KSK rollover is not difficult.
Our Conclusion: Let’s Encrypt is a great idea for running a CA that issues free certificates, and their certificate just works. The tools and client do need to a bit of polishing and made a bit more automatic so that every sysadmin will be able to install it with apt-get/yum, run it, and then state which domain they wish to secure.
Our next test will be to obtain a Let’s Encrypt certificate for our mail server sitting on mx.go6lab.si. Theoretically it should work, but let’s see next week when that subdomain will be whitelisted on the Let’s Encrypt system.
Important notice: The Let’s Encrypt Open Beta program starts up on 3 December 2016 and then more domains will be whitelisted in a much shorter time. Don’t be afraid to try it out!
]]>
The RIPE 71 meeting is happening this week in Bucharest and each day we’ll be highlighting the presentations and activities related to the Deploy360 technologies.
Wednesday and Thursday were primarily devoted to RIPE NCC and RIPE policy matters, but there was also some interest for Deploy360 in the RIPE IPv6 and DNS Working Groups.
To start, we absolutely have to highlight the presentation by Geoff Huston on IPv6 Performance. This attempts to answer the questions about reliability of IPv6 connections in terms of how often TCP connections succeed, and whether IPv6 is actually slower than IPv4. Using a script embedded in an online advert that generates a set of URLs to fetch, this allows servers to determine reliability and RTTs, with measurements taken in both 2011 and 2015.
The first set of measurements taken in 2011 reveal astonishingly high failure rates for TCP connections over IPv6; around 40% on average. However, this was during the period when Teredo and 6to4 were still common and suggests either firewall configurations blocking incoming connections, or overloading/reliability issues with outgoing relays.
Matters seem to have improved in 2015, with substantially lower failure rates of just over 4% which also showed progressive improvement over the course of the year. Teredo has almost disappeared, but 6to4 still accounts for 27% of IPv6 connections and has a 9% failure rate. Unicast IPv6 hovers around 2%, although this compares with just 0.2% for IPv4.
Looking at the speed measurements, the comparative data from 2015 also shows significant improvement over 2011. In fact, for IPv6 unicast it is faster about half of the time, and 70% of all unicast cases are within 10 ms RTT of IPv4, although this decreases significantly with 6to4.
The conclusions are that if you can establish a connection then IPv4 and IPv6 appear to have comparable performance, although currently the odds of establishing the connection still favour IPv4. This might point to the various transition mechanisms employed, and further encourage the deployment of native IPv6.
It’s also worth highlighting a couple of IPv6 case studies presented in the IPv6 Working Group. First of all the presentation from Timo Hilbrink about IPv6 deployment at XS4All, a Dutch ISP that has enabled IPv6 for all subscribers since June 2014 and currently has 175,000 active IPv6 users. Then also the presentation from Oskari Rasi about IPv6 deployment at DNA Finland, the largest cable operator in that country that in 2015 has launched IPv6 support in their mobile, cable and DSL products.
And last but not least from the IPv6 Working Group, check out the presentation by Sander Steffann on DHCPKit which aims to solve the issues of dynamically assigned static IPv6 prefixes. Existing DHCPv6-PD servers did not work as well as desired, so a new DHCP server framework has been developed to handle this.
Over in the DNS Working Group there were also several items of interest. There was a good presentation from Chris Baker on iOS and OSX preferences for IPv6. In July 2015, Apple announced that iOS 9.x and OSX El Capitan 10.11.x would prefer using IPv6 by default, and Dyn have undertaken some real world tests on DNS resolution. This led to some interesting observations about the different underlying topologies of IPv4 and IPv6, and some anomalies that led to IPv6 queries being sent to servers in different continents.
Following on were a couple of presentations from NLnet Labs on work into ways to deliver DNSSEC information to end hosts that have customer premises equipment blocking it. They had enabled them to raise the number of successful DNSSEC queries to almost 80%, so please do take a look at the presentations on a Discovery Method for a Validating Stub Resolver, and DNSSEC for Legacy Applications.
Finally, you should be aware of the ICANN plans for the forthcoming Root Zone Key Signing Key rollover. This specifically relates to RFC 5011 which specifies a means for automated updating of DNSSEC trust anchors, but there are some concerns about how operators choose to follow the recommendations and whether existing tools fully or partially support these. ICANN is therefore raising awareness of this issue and building a list of parties who might be affected in order to best plan how to undertake the rollover and when.
All the presentations and videos from these sessions can be found on the RIPE 71 website.
]]>