For activating the secure connection lock symbol for a web site, a certificate is required. The certificate can be purchased from an ISP or activated from Let’s Encrypt for free.
Many web site administrators start with the free option – perhaps to check if it is easy to deploy – but the process flow tends to grab the attention of a curious person (including myself) and eventually, plenty of time is used to get the free certificate properly activated. Some web admins feel the process is a nightmare, whereas others regard it as a normal Linux configuration project. Installing a Letsencrypt certificate is a project of its own, but what if you encounter problems when it is time to renew the certificate? Here are a few tips to get started with the debugging project.
We had successfully installed Letsencrypt certificates on a number of web sites in 2q2018. All certificates were set to renew automatically. Everything went smoothly until early January 2019. The Letsenrypt certificate renewal script sent an email notification informing that renewal for one of the sites will fail. Our first thought was: nothing has changed at our server – what the heck was going on? Nonetheless, the notification was useful: it gave us enough time to explore the problem.
“Your certificate (or certificates) for the names listed below will expire in 9 days (on 29 Jan 19 22:23 +0000). Please make sure to renew your certificate before then, or visitors to your website will encounter errors.”
Since none of us happens to be a certificate, SSL, or encryption specialist, we had to rely on DuckDuckGo, Qwant and Startpage for searching pointers to potential solutions. Fortunately, Let’s Encrypt has an active community where experts and non-experts exchange their experiences.
An unexpected error occurred:
The request message was malformed :: Unable to update challenge :: authorization must be pending. Please see the logfiles in /var/log/letsencrypt for more details.
We spent half a day trying to find tips from other web admins who had similar symptoms with certificate renewal, but none of their solutions did help us. Since it was a complex, technical case, we were not surprised.
We modified our Apache web server configuration, ran manual commands suggested by web admins who had similar problems, made sure the webroot directory was correct, studied Letsencrypt log files on the web server, but nothing helped.
We kept searching and testing suggested solutions – until we hit the jackpot.
Somewhere in a discussion thread someone casually mentioned Let’s Debug (letsdebug.net). It is an online tool for analyzing problems with a site’s certificates. It was exactly what we wanted. Letsdebug is simple to use, and above all, it can point you to the right direction. Just type your web site address into the field and run the test.
The service returns a human readable – and understandable by a human – report indicating problems with a web site.
The analysis of Letsdebug pointed us to right direction. It couldn’t determine exactly the correct reason for certificate renewal failure – we didn’t expect it to – but it was close enough so that we could quickly identify and correct the problem.
In our case, we had modified the DNS record of the problematic web site. The web site ran just fine, but the Letsencrypt certificate renewal script didn’t agree with our DNS settings. We have the certbot-auto script running as a cron job.
We corrected the DNS record (with valuable help from Travis at Godaddy), and that was it. Problem solved.
Our big thanks goes to the people behind Let’s Debug with wishes that they can continue maintaining and developing the service. The code for Let’s Debug is available in Github.