Not able to generate a Let's Encrypt Certificat - nginx

I am using Laravel Forge to manage my servers and websites. So generating SSL certificates via Let's Encrypt is also done vie Forge. Somehow one of my domains throws me an error (see attached).
This domain is running on a server which holds several other domains. The nginx configuration is exactly the same.
The application is a Laravel app running on Laravel Octane.
Error:
2022-06-13 10:41:26 URL:https://forge-certificates.laravel.com/le/1441847/1663342/ecdsa?
env=production [4653] -> "letsencrypt_script1655109686" [1] Cloning
into 'letsencrypt1655109686'... Note: switching to
'91cccc0c234e4decf0a19595fa19a6f306788032'.
You are in 'detached HEAD' state. You can look around, make
experimental changes and commit them, and you can discard any commits
you make in this state without impacting any branches by switching
back to a branch.
If you want to create a new branch to retain commits you create, you
may do so (now or later) by using -c with the switch command. Example:
git switch -c
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to
false
HEAD is now at 91cccc0 ensure newline before new section in
openssl.cnf ERROR: Challenge is invalid! (returned: invalid) (result:
["type"] "http-01" ["status"] "invalid"
["error","type"] "urn:ietf:params:acme:error:connection"
["error","detail"] "111.222.333.444: Fetching
http://my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw:
Timeout during connect (likely firewall problem)"
["error","status"] 400
["error"] {"type":"urn:ietf:params:acme:error:connection","detail":"111.222.333.444:
Fetching
http://my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw:
Timeout during connect (likely firewall problem)","status":400}
["url"] "https://acme-v02.api.letsencrypt.org/acme/chall-v3/119151352296/awZDUg"
["token"] "_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw"
["validationRecord",0,"url"] "http://www.my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw"
["validationRecord",0,"hostname"] "www.my-domain.de"
["validationRecord",0,"port"] "80"
["validationRecord",0,"addressesResolved",0] "111.222.333.444"
["validationRecord",0,"addressesResolved",1] "2a01:4f8:141:333::84"
["validationRecord",0,"addressesResolved"] ["111.222.333.444","2a01:4f8:141:333::84"]
["validationRecord",0,"addressUsed"] "2a01:4f8:141:333::84"
["validationRecord",0] {"url":"http://www.my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw","hostname":"www.my-domain.de","port":"80","addressesResolved":["111.222.333.444","2a01:4f8:141:333::84"],"addressUsed":"2a01:4f8:141:333::84"} ["validationRecord",1,"url"] "http://www.my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw"
["validationRecord",1,"hostname"] "www.my-domain.de"
["validationRecord",1,"port"] "80"
["validationRecord",1,"addressesResolved",0] "111.222.333.444"
["validationRecord",1,"addressesResolved",1] "2a01:4f8:141:333::84"
["validationRecord",1,"addressesResolved"] ["111.222.333.444","2a01:4f8:141:333::84"]
["validationRecord",1,"addressUsed"] "111.222.333.444"
["validationRecord",1] {"url":"http://www.my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw","hostname":"www.my-domain.de","port":"80","addressesResolved":["111.222.333.444","2a01:4f8:141:333::84"],"addressUsed":"111.222.333.444"}
["validationRecord",2,"url"] "http://my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw"
["validationRecord",2,"hostname"] "my-domain.de"
["validationRecord",2,"port"] "80"
["validationRecord",2,"addressesResolved",0] "111.222.333.444"
["validationRecord",2,"addressesResolved",1] "2a01:4f8:141:333::84"
["validationRecord",2,"addressesResolved"] ["111.222.333.444","2a01:4f8:141:333::84"]
["validationRecord",2,"addressUsed"] "2a01:4f8:141:333::84"
["validationRecord",2] {"url":"http://my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw","hostname":"my-domain.de","port":"80","addressesResolved":["111.222.333.444","2a01:4f8:141:333::84"],"addressUsed":"2a01:4f8:141:333::84"} ["validationRecord"] [{"url":"http://www.my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw","hostname":"www.my-domain.de","port":"80","addressesResolved":["111.222.333.444","2a01:4f8:141:333::84"],"addressUsed":"2a01:4f8:141:333::84"},{"url":"http://www.my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw","hostname":"www.my-domain.de","port":"80","addressesResolved":["111.222.333.444","2a01:4f8:141:333::84"],"addressUsed":"111.222.333.444"},{"url":"http://my-domain.de/.well-known/acme-challenge/_bL98lTvqMOeJG-NCeLzl2Z3VWUm7EJBa1i6IEBDuLw","hostname":"my-domain.de","port":"80","addressesResolved":["111.222.333.444","2a01:4f8:141:333::84"],"addressUsed":"2a01:4f8:141:333::84"}] ["validated"] "2022-06-13T08:41:47Z")

I've finally found the solution. Laravel Forge does not support IPv6 out of the box. So you either have to configure Forge to use IPv6 as well or remove all AAAA records pointing to the server managed by Laravel Forge.

Related

Mosquitto: Starting in local only mode

I have a virtual machine that is supposed to be the host, which can receive and send data. The first picture is the error that I'm getting on my main machine (from which I'm trying to send data from). The second picture is the mosquitto log on my virtual machine. Also I'm using the default config, which as far as I know can't cause these problems, at least from what I have seen from other examples. I have very little understanding on how all of this works, so any help is appreciated.
What I have tried on the host machine:
Disabling Windows defender
Adding firewall rules for "mosquitto.exe"
Installing mosquitto on a linux machine
Starting with the release of Mosquitto version 2.0.0 (you are running v2.0.2) the default config will only bind to localhost as a move to a more secure default posture.
If you want to be able to access the broker from other machines you will need to explicitly edit the config files to either add a new listener that binds to the external IP address (or 0.0.0.0) or add a bind entry for the default listener.
By default it will also only allow anonymous connections (without username/password) from localhost, to allow anonymous from remote add:
allow_anonymous true
More details can be found in the 2.0 release notes here
You have to run with
mosquitto -c mosquitto.conf
mosquitto.conf, which exists in the folder same with execution file exists (C:\Program Files\mosquitto etc.), have to include following line.
listener 1883 ip_address_of_the_machine(192.168.1.1 etc.)
By default, the Mosquitto broker will only accept connections from clients on the local machine (the server hosting the broker).
Therefore, a custom configuration needs to be used with your instance of Mosquitto in order to accept connections from remote clients.
On your Windows machine, run a text editor as administrator and paste the following text:
listener 1883
allow_anonymous true
This creates a listener on port 1883 and allows anonymous connections. By default the number of connections is infinite. Save the file to "C:\Program Files\Mosquitto" using a file name with the ".conf" extension such as "your_conf_file.conf".
Open a terminal window and navigate to the mosquitto directory. Run the following command:
mosquitto -v -c your_conf_file.conf
where
-c : specify the broker config file.
-v : verbose mode - enable all logging types. This overrides
any logging options given in the config file.
I found I had to add, not only bind_address ip_address but also had to set allow_anonymous true before devices could connect successfully to MQTT. Of course I understand that a better option would be to set user and password on each device. But that's a next step after everything actually works in the minimum configuration.
For those who use mosquitto with homebrew on Mac.
Adding these two lines to /opt/homebrew/Cellar/mosquitto/2.0.15/etc/mosquitto/mosquitto.conf fixed my issue.
allow_anonymous true
listener 1883
you can run it with the included 'no-auth' config file like so:
mosquitto -c /mosquitto-no-auth.conf
I had the same problem while running it inside docker container (generated with docker-compose).
In docker-compose.yml file this is done with:
command: mosquitto -c /mosquitto-no-auth.conf

Proxy authentication using wget on cygwin

My institute recently installed a new proxy server for our network. I am trying to configure my Cygwin environment to be able to run wget and download data from a remote repository.
Browsing the internet I have found two different solutions to my problem, but no one of them seem to work in my case.
The first one I tried was to follow these instructions, so in Cygwin:
cd /cygdrive/c/cygwin64/etc/
nano wgetrc
at the end of the file, I added:
use_proxy = on
http_proxy=http://username:password#my.proxy.ip:my.port/
https_proxy=https://username:password#my.proxy.ip:my.port/
ftp_proxy=http://username:password#my.proxy.ip:my.port/
(of course, using my user and password)
The second approach was what was suggested by this SO post, so in my Cygwin environment:
export http_proxy=http://username:password#my.proxy.ip:my.port/
export https_proxy=https://username:password#my.proxy.ip:my.port/
export ftp_proxy=http://username:password#my.proxy.ip:my.port/
in both cases, if I try to test my wget, I get the following:
$ wget http://www.google.com
--2020-01-30 12:12:22-- http://www.google.com/
Resolving my.proxy.ip (my.proxy.ip)... 10.1XX.XXX.XX
Connecting to my.proxy.ip (my.proxy.ip)|10.1XX.XXX.XX|:8XXX... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
2020-01-30 12:12:22 ERROR 407: Proxy Authentication Required.
It looks like if my user and password are not ok, but I actually checked them on my browsers and my credentials work just fine.
Any idea on what this could be due to?
This problem was solved thanks to the suggestion of a User of the community AskUbuntu.
Basically, instead of editing the global configuration file wgetrc, I should have created a new .wgetrc with my proxy configuration in my Cygwin home directory.
In summary:
Step 1 - Create a .wgetrc file;
nano ~/.wgetrc
Step 2 - record in this file the proxy info:
use_proxy=on
http_proxy=http://my.proxy.ip:my.port
https_proxy=https://my.proxy.ip:my.port
ftp_proxy=http://my.proxy.ip:my.port
proxy_user=username
proxy_password=password

module.run not executing in state.highstate, but works with state.sls

I'm attempting to re-run a state from another state. I'm not using watch or watch_in etc b/c i want it to run each time. I configure all my nginx virtual hosts and then at the end another state runs called nginx-certs the relevant portion is here:
nginx-frontend:
module.run:
- name: state.sls
- mods:
- nginx-frontend
During the highstate i see the state_id is executed but has no comments, nor shows it reruns that state, it just completes as Result: True. I can then jump to the salt master and run
sudo salt webserver state.sls nginx-certs
and when it hits nginx-frontend, it does reload all of the virtual hosts, putting the new cert in the config.
I'm curious why this does not run in the highstate.
I have attempted ll sorts of different variations of the simple block outlined above. This one works, but not in the highstate, which is my goal to fix.
If you wonder why i do it this way, all certificates for production and staging terminate at HAProxy and nginx only serves up 80/http1 81/h2, but when building out dev servers i want to assign the cert directly to the server as it will be public facing. I need to build out the virtual hosts first to get port 80 open which is used for lets-encrypt. Then when the cert is available, update the nginx vhosts listen directive and cert paths.
From what I understand: you have one server which you want temporarily configured with Nginx on port 80, then generate its certificate with letsencrypt, then change Nginx configuration to be on port 443.
What you can do is:
have one state which installs and configures Nginx to listen on port 80
have another state with installs/configures/runs letsencrypt
a third state which configures Nginx as you want it to be at the end [1]
you just include them in salt to be run in the specific order like
# custom_nginx.sls
include:
- temp_nginx_on_port_80
- letsencrypt_cert
- nginx
[1] for this I think its better to use formula like the one from the community https://github.com/saltstack-formulas/nginx-formula/ and configure it with pillar data. Obviously if you use it for step 3, you won't be able to use for step 1 (or at least I don't see right now how)

How to set up a secure connection between Filbeat and Elasticsearch using SSL

I'm unable to setup an SSL connection between Filebeat and Elasticsearch.
My knowledge is lacking when it comes to SSL. I'm using X-Pack to generate a certificate using the certutil command. bin/xpack/certutil ca generates a certificate authority under the name elastic-stack-ca.p12.
Then
$ bin/x-pack/certutil cert --ca elastic-stack-ca.p12
Which I believe creates a certificate signed by that CA. This results in the file elastic-certificates.p12. From here I'm clueless.
I tried testing to see if the certificates work by setting up a HTTPS connection to ES.
I put
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /path/to/elastic-certificates.p12
xpack.security.http.ssl.certificate: /path/to/elastic-certificates.p12
xpack.security.http.ssl.certificate_authorities: [ "/path/to/elastic-stack-ca.p12" ]
However, this brings up quite a few errors one of them being
caught exception while handling client http traffic, closing connection
When I add the https IP and the CA in Kibana it fails to connect with ES.
I would like to know how to successfully set up https. Also how can a SSL connection be established between two servers. One containing Filebeat, but no X-Pack and the receiving server with ES on it alongside X-Pack installed.
After adding those SSL settings in your elasticsearch.yml, you also need to add the password to the Elasticsearch keystore and truststore. You should've set a password when you ran the certutil command. You can do that with:
$ echo password | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin xpack.security.transport.ssl.keystore.secure_password
$ echo password | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin xpack.security.transport.ssl.truststore.secure_password
Make sure you restart Elasticsearch after making these changes.

How to generate fixed url with ngrok

I am working on facebook messenger.
Facebook app only accept one url for webhook but ngrock is generating new URL every time. Now I am unable to test my app because of webhook URL changed.
UPDATE May 2020
Serveo is up and running again! No installation, no signup!
All you need to do is to run this:
ssh -R <unique subdomain>:80:<your local host>:<your local port> serveo.net
like
ssh -R youruniquesubdomain:80:localhost:8000 serveo.net
UPDATE January 2020
Since there are some issues with Serveo and localtunnel, I want to share with you another free ssh-based self-hosting service: Localhost.run
Unfortunately, it does not provide unique subdomains but it is ssh-based so you do not have to install additional applications. Still waiting for Serveo coming back.
UPDATE April 2018
I've found Serveo just now! And it is totally incredible!
UPDATE November 2017
Probably, it is not the best option for you but I started using localtunnel instead of ngrok.
An installation and run flow is very simple:
npm install -g localtunnel
lt --port <your localhost port> --subdomain youruniquesubdomain
Then I can go to my http://youruniquesubdomain.localtunnel.me
That's it!
No more free subdomain support from ngrok.....pls have a error as below
Tunnel session failed: Only paid plans may bind custom subdomains.
Failed to bind the custom subdomain 'arvindpattartestfb.ngrok.io' for the account 'arvccccc'.
This account is on the 'Free' plan.
Upgrade to a paid plan at: https://dashboard.ngrok.com/billing/plan
ERR_NGROK_313
You need to set up auth token. You can find it here https://dashboard.ngrok.com/auth. (W̶o̶r̶k̶s̶ ̶w̶i̶t̶h̶ ̶f̶r̶e̶e̶ ̶v̶e̶r̶s̶i̶o̶n̶,̶ ̶n̶o̶ ̶n̶e̶e̶d̶ ̶t̶o̶ ̶p̶a̶y̶ it's now paid feature, see ngrok pricing).
Then you can use it like this:
ngrok http 80 --subdomain yoursubdomain
Neither localtunnel.me or Serveo are working for me right now so I created a temporary solution that works for some use-cases including mine (react-native local development): using the ngrok npm package one can save the generated ngrok url into a json file, and that file can be read for any other app.
First make sure to install ngrok using npm install ngrok then use this node script:
const ngrok = require('ngrok');
const fs = require('fs').promises;
(async function() {
const url = await ngrok.connect(3000);
const api = ngrok.getApi();
let data = await api.get('api/tunnels');
data = JSON.parse(data);
let dict = {'domain': data.tunnels[0].public_url}
await fs.writeFile("config.json", JSON.stringify(dict));
console.log("saved " + data.tunnels[0].public_url);
})();
Then from your app you may read it using code similar or equal to:
const backend = require('./config.json').domain;
For DHIS2 local installation, I did this on the terminal on ubuntu server.
Make sure your web is running on the specified port. Mine was on 8080.
ssh -R dani.serveo.net:80:localhost:8080 serveo.net
The beauty of this serveo.net is you can reuse the same hostname prefix url before serveo.net as many time as you want, even if power goes off or internet diconnection.
Staqlab tunnel is providing domain for free. Its works great but need a binary to be downloaded from there website. Using this service for month without any hassle
In 2022 (almost 2023) pagekite.me works for me.
It is very similar to ngrok, and requires the installation of pagekite.py (and, obviously Python).
After installation click on pagekite.py opens pagekite shell.
Run command: 8080 subdomain.pagekite.me
I noticed that no one mentioned how to have static ngrok urls, which was the main question about.
A way to do it is to edit the ngrok.yml file, which is located at
Linux: "~/.config/ngrok/ngrok.yml"
MacOS (Darwin): "~/Library/Application Support/ngrok/ngrok.yml"
Windows: "%HOMEPATH%\AppData\Local\ngrok\ngrok.yml"
You can have content such as:
version: "2"
authtoken: valid_auth_token
tunnels:
first-app:
addr: 3000
proto: http
hostname: yourfixedngrok_id1.ngrok.io
second-app:
addr: 8000
proto: http
hostname: yourfixedngrok_id2.ngrok.io
This will help you expose multiple ports, and have a persisted url for each of which based on the value you set for hostnames.
After that, you run your ngrok using this command:
ngrok start --all
Though its not a solution but take it as workaround, I had the same problem while testing. What i did is keep the ngrok running with my http port, so my ngrok url is not changing. but I frequently changing and restarting my server for testing and debugging.

Resources