Problem Statement

Today I found my whole site is down for several hours due to the expiration of the ssl (Secure Sockets Layer) certificate issued by Let's Encrypt and thanks to the DNS (Domain Name System) and CDN (Content Delivery Network) service and more importantly, the Always Online™ function provided by Cloudflare which reminded me to renew the certificate by clearly stating the situation on an (unfriendly) error page.

This shouldn't happen since I've already deployed the auto-renewal procedures according to some official instructions which now is proved to be ‘un-robust’.

Current Methods

So again I did some research on how to renew the certificate and found many also had the same problem as mine. Here I just skip all the tedious as well as miserable tries I've made whose results are always the same saying that

Failed authorization procedure.
benny.hk (http-01): urn:acme:error:unauthorized
:: The client lacks sufficient authorization
:: Invalid response from http://benny.hk/.well-known/acme-challenge/... 

It kept shouting even I've tried manually allow the 80 and 443 port for the purpose of domain ownership validate communication. And I've paused the CDN service from Cloudflare.

Solution

Then I was thinking maybe the existing long Nginx configuration file is the bad guy who's getting in the way. So I replace it with the almost default one containing just several following lines. (Modified file is called ghost under /etc/nginx/sites-available)

server {
listen 80;
server_name benny.hk www.benny.hk;
root /var/www/ghost;

location ~ ^/.well-known {
    root /var/www/ghost;
    }

location ~ /.well-known/acme-challenge/ {
    allow all;
    }
}

Then apply the following cmd to restart the Nginx service.

sudo systemctl restart nginx

After that, try again the Webroot method loaded from /opt.

cd ..
cd /opt
./certbot-auto certonly

It throws out the question like this:

How would you like to authenticate with the ACME CA?
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

According to the suggestions from forums on the internet, choose 2.

The question continues:

Please enter in your domain name(s) (comma and/or space separated)  (Enter 'c'to cancel): 

I typed benny.hk www.benny.hk with a space in between.

There displays:

Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for benny.hk
http-01 challenge for www.benny.hk

Select the webroot for benny.hk:

1: Enter a new webroot

Press 1 [enter] to confirm the selection (press 'c' to cancel):

Of course 1 is the only option.

It keeps asking as if I've never set it up before:

Input the webroot for benny.hk: (Enter 'c' to cancel):

I typed /var/www/ghost, actually this location is quite flexible, it's okay if you type somewhere else but usually I'll use this one as the first time when I set it up.

Select the webroot for www.benny.hk:

1: Enter a new webroot
2: /var/www/ghost

Select the appropriate number [1-2] then [enter] (press 'c' to cancel):

All right, stupid answer, of course 2.

Then it was the show time!

Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/benny.hk/fullchain.pem. Your cert will 
expire
   on 2017-10-08. To obtain a new or tweaked version of this
   certificate in the future, simply run certbot-auto again. To
   non-interactively renew *all* of your certificates, run
   "certbot-auto renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

You see what!! “Congratulations!”. Awesome!
Then I restore the long Nginx configuration file back (/etc/nginx/sites-available/ghost) and restart the Nginx service again.

sudo systemctl restart nginx

Result

My site is alive again. No more words.

Reference

  1. How To Secure Nginx with Let's Encrypt on Ubuntu 16.04
  2. Stackoverflow Question on Certbot not creating acme-challenge folder