ESP HTTP connection issue : on connection callback function not getting called - http

I am using custom build for NodeMCU with following modules: cjson, file, gpio, http, net, node, ow, tmr, websocket, wifi.
conn = nil
conn=net.createConnection(net.TCP) -- 30 seconds timeout time of serv
conn:on("connection", function(conn, payload)
print("Server DB Connected, sending event") -- This line is not getting print on esplorer.
conn:send ( <some-data>)
end)
conn:connect(80,"mysite.com")
conn:close()
For debugging i tried using following code :
net.dns.resolve("www.google.com", function(sk, ip)
if (ip == nil) then
print("DNS fail!")
else
print("Resolved: " .. ip)
end
end)
But above is giving output " DNS fail! "
Site is working perfectly fine along with server.
A normal GET request to the same site using putty is returning correct data.
Please help me resolving the issue.

Related

HTTPS data retrieval error

I have an ESP12 with NodeMCU firmware installed (modules: http, tls, etc.).
Whenever I try to use http.get with an https link, I receive the following error:
> http.get("https://pastebin.com/raw/cAB83eF1", nil, function(code, data)
>> if (code < 0) then
>> print("HTTP request failed")
>> else
>> print(code, data)
>> end
>> end)
> HTTP client: Disconnected with error: -114
HTTP client: Connection timeout
HTTP client: Connection timeout
where as when I tried the same pastebin.com link without https, I am able to receive data and the HTTP status code.
> http.get("http://pastebin.com/raw/cAB83eF1", nil, function(code, data)
>> if (code < 0) then
>> print("HTTP request failed")
>> else
>> print(code, data)
>> end
>> end)
> 200 test
Quoting from the docs of http.get:
Each request method takes a callback which is invoked when the
response has been received from the server. The first argument is the
status code, which is either a regular HTTP status code, or -1 to
denote a DNS, connection or out-of-memory failure, or a timeout
(currently at 10 seconds).
Since the docs say valid codes are -1 and HTTP codes, how can I find out what is -114?
Edit1: After building the firmware with debug option ON. I get the following error:
> http.get("https://pastebin.com/raw/cAB83eF1", nil, function(code, data)
>> if (code < 0) then
>> print("HTTP request failed")
>> else
>> print(code, data)
>> end
>> end)
> client handshake start.
please start sntp first !
please start sntp first !
client handshake failed!
Reason:[-0x7200]
HTTP client: Disconnected with error: -114
HTTP client: Connection timeout
HTTP client: Connection timeout
=rtctime.get()
1501394767 272709
>
As per the log statement Reason:[-0x7200] 0x72 hex is -114 decimal which gets printed later Disconnected with error: -114.
Also sntp.sync() has already been performed as seen from the output of rtctime.get(). Even if the SNTP has been synced why is the message please start sntp first ! printed?
Also later in the log, after the -114 message, it displays Connection timeout message. How can I change the timeout for https connections? any pointers to C/header files?
Note: I do have the git repo and docker image installed on my Linux machine for building the firmware.
PS: I need https because I'll be making my ESP12 communicate with a RESP API hosted on the cloud.
This is a short and unsatisfactory answer: you hit https://github.com/nodemcu/nodemcu-firmware/issues/1707.

OpenResty rewrite_by_lua vs. rewrite_by_lua_file - syntax errors

I am using OpenResty Nginx, and have a snippet of Lua script in the Nginx conf. (sample below, but truncated for brevity). This works fine when it is in the nginx.conf file.
rewrite_by_lua '
-- Load Memcached module
local memcached = require "resty.memcached"
local memc, err = memcached:new()
if not memc then
ngx.log(ngx.ERR, "Failed to instantiate memc: ", err)
return
end
'
However, if I extract this script in to a file, memcached.lua and then include it in the nginx config with rewrite_by_lua_file, I am unable to restart nginx and get notifications about syntax errors such as:
unexpected "," in /etc/nginx/sites-enabled/memcached.lua
unknown directive "--" in /etc/nginx/sites-enabled/memcached.lua
If I remove the ngx.log line completely the syntax error disappears.
How come the syntax is valid in the nginx conf but not when I extract it to a separate file? I've read somewhere about bytecode but not sure if that is related?
EDIT
Here is the full output of the .lua file. This exact code works if it's embedded directly in to the nginx.conf file via rewrite_by_lua
cat /etc/nginx/sites-enabled/memcached.lua
-- Define Memcached key
local memcached_key = "saas_" .. ngx.var.host
-- Load Memcached module
local memcached = require "resty.memcached"
local memc, err = memcached:new()
if not memc then
ngx.log(ngx.ERR, "Failed to instantiate memc: ")
end
-- Set Memcached timeout
memc:set_timeout(1000)
-- Connect to Memcached daemon
local ok, err = memc:connect("127.0.0.1", 11211)
if not ok then
ngx.log(ngx.ALERT, "Failed to connect to Memcached: ", err)
end
-- Query for branch name in Memcached
local branch, flags, err = memc:get(memcached_key)
if err then
ngx.log(ngx.NOTICE, "Failed to get account from Memcached: ", err)
end
-- If branch not found in Memcached, ask MySQL instead
if not branch then
-- Load MySQL module
local mysql = require "resty.mysql"
local db, err = mysql:new()
if not db then
ngx.log(ngx.ALERT, "Failed to instantiate MySQL: ", err)
return
end
-- Set MySQL timeout
db:set_timeout(1000) -- 1 sec
-- Connect to MySQL daemon
local ok, err, errno, sqlstate = db:connect{
path = "/var/run/mysqld/mysqld.sock",
database = "dbname",
user = "dbuser",
password = "dbpass" }
if not ok then
ngx.log(ngx.ALERT, "Failed to connect to DB: ", err, ": ", errno, " ", sqlstate)
return
end
-- Define subdomain string by extracting it from the http hostname
local subdomain = string.match(ngx.var.host, "([%w-]+).")
local quoted_subdomain = ngx.quote_sql_str(subdomain)
-- Execute an SQL query to get the branch name based on the subdomain
local sql = "SELECT branch FROM saas_account WHERE account = " .. quoted_subdomain
local res, err, errno, sqlstate = db:query(sql, 1)
if not res then
ngx.log(ngx.ALERT, "Bad result from DB: ", err, ": ", errno, ": ", sqlstate, ".")
return
end
-- Check we got a result
if not res[1] then
ngx.log(ngx.NOTICE, "No docroot was returned for subdomain: ", subdomain)
ngx.var.branch = "chaletops"
-- Assign the result from MySQL to the Nginx variable
else
ngx.var.branch = res[1].branch
-- Store the result from MySQL back in to Memcached with a lifetime of 1 hour (3600 seconds)
local ok, err = memc:set(memcached_key, ngx.var.branch, 3600)
if not ok then
ngx.log(ngx.WARN, "Failed to set branch in Memcached: ", err)
-- return
end
end
-- Save MySQL KeepAlive connection
local ok, err = db:set_keepalive(10000, 100)
if not ok then
ngx.log(ngx.ALERT, "Cannot set MySQL keepalive: ", err)
return
end
ngx.log(ngx.NOTICE, "Loading branch from MySQL: ", ngx.var.branch)
-- Else we did get a result from Memcached
else
-- Assign the result from Memcached to the Nginx variable
ngx.var.branch = branch
ngx.log(ngx.NOTICE, "Loading branch from Memcached: ", ngx.var.branch)
end
-- Save Memcached KeepAlive connection
local ok, err = memc:set_keepalive(10000, 100)
if not ok then
ngx.log(ngx.ALERT, "Cannot set Memcached keepalive: ", err)
-- return
end
SOLUTION
TL:DR - put the .lua file outside of your Nginx configuration directory.
After being asked to post the exact error messages, I had to recreate the problem. For reference and Google searchers the error messages are like this:
2015/12/17 10:04:02 [emerg] 6903#0: unexpected "," in /etc/nginx/sites-enabled/memcached.lua:18
This time I copied the Lua code out of my nginx.conf and put it in a file called memcached.lua and saved it in my project directory. I then added the following include to my nginx.conf in place of the Lua script:
rewrite_by_lua_file /path/to/my/project/memcached.lua;
Immediately it starts working!
Previously I had included it by symlinking it from my project directory in to /etc/nginx/sites-enabled/memcached.lua (assuming it would be better for it to actually live alongside the .conf that includes it) and then just used rewrite_by_lua_file memcached.lua; as it was relative the .conf file.
So I initially wondered if it was because the rewrite_by_lua_file function could not follow symlinks or something, but then it dawned on me that it was nothing to do with the Lua include statement, but OBVIOUSLY (now!) because of the symlink being in /etc/nginx/sites-enabled so Nginx was automatically trying to load the file as though it was a normal Nginx configuration file.
This is because Nginx (on Ubuntu/Linux Mint) is configured to import all config files using the path /etc/nginx/sites-enabled/* in to it's main config.
I think the reason I made this simple error is that on later versions of Ubuntu with Apache, a similar process happens but with Apache (which I am much more used to) you have an include path of /etc/nginx/sites-enabled/*.conf - i.e. it specifically requires a .conf file extension, which is why I incorrectly assumed that Nginx would not try to include my .lua file.

Golang TCP error wsarecv

I am writing an app which listens tcp connection (see this example) . When a tcp connection disconnected I got error read tcp ip_server.:port1->ip_client:port2: wsarecv: An existing connection was forcibly closed by the remote host.
I expected error EOF and timeout(for conn.SetReadDeadline()) and tried to catch error with this code:
if err != nil {
log.Println("getting error from listener")
// I thought, listener can continue work another cases
if neterr, ok := err.(net.Error); ok && neterr.Timeout() || err == io.EOF {
log.Println("Closing connection...")
break // connection will be closed
}
}
Anyone knows about this error? Do you know how to catch this error and when this error will occured ? Thanks in advance !
The usual cause of this error is that you sent data over a connection which had already been closed by the peer. In other words, an application protocol error.

How to use Lua with Redis?

I am using Lua+nginx with OpenResty bundle. But problem comes when I try to connect Lua script with Redis, I'm not able to make successful connection. I have gone through a lot of links and blogs, But in the end it always fails. This is the snippet of code which I am trying.
server {
location /test {
content_by_lua '
local redis = require "resty.redis" // **Problem in code "Not able to require "resty.redis""**
local red = redis:new()
red:set_timeout(1000) -- 1 sec
-- or connect to a unix domain socket file listened
-- by a redis server:
-- local ok, err = red:connect("unix:/path/to/redis.sock")
local ok, err = red:connect("127.0.0.1", 6379)
if not ok then
ngx.say("failed to connect: ", err)
return
end
}
}
Assuming "not able to require" means you are getting module 'resty.redis' not found message with a list of paths, the error indicates that you are missing the module. You need to check the paths that are listed and make sure that resty/redis.lua is in one of those folders. You will find that file in lua-resty-redis-<version> folder in your OpenResty installation.

Local network pinging in python

Does anyone know how to use python to ping a local host to see if it is active or not? We (my team and I) have already tried using
os.system("ping 192.168.1.*")
But the response for destination unreachable is the same as the response for the host is up.
Thanks for your help.
Use this ...
import os
hostname = "localhost" #example
response = os.system("ping -n 1 " + hostname)
#and then check the response...
if response == 0:
print(hostname, 'is up!')
else:
print(hostname, 'is down!')
If using this script on unix/Linux replace -n switch with -c !
Thats all :)
I've found that using os.system(...) leads to false positives (as the OP said, 'destination host unreachable' == 0).
As stated before, using subprocess.Popen works. For simplicity I recommend doing that followed by parsing the results. You can easily do this like:
if ('unreachable' in output):
print("Offline")
Just check the various outputs you want to check from ping results. Make a 'this' in 'that' check for it.
Example:
import subprocess
hostname = "10.20.16.30"
output = subprocess.Popen(["ping.exe",hostname],stdout = subprocess.PIPE).communicate()[0]
print(output)
if ('unreachable' in output):
print("Offline")
The best way I could find to do this on Windows, if you don't want to be parsing the output is to use Popen like this:
num = 1
host = "192.168.0.2"
wait = 1000
ping = Popen("ping -n {} -w {} {}".format(num, wait, host),
stdout=PIPE, stderr=PIPE) ## if you don't want it to print it out
exit_code = ping.wait()
if exit_code != 0:
print("Host offline.")
else:
print("Host online.")
This works as expected. The exit code gives no false positives. I've tested it in Python 2.7 and 3.4 on Windows 7 and Windows 10.
I've coded a little program a while back. It might not be the exact thing you are looking for, but you can always run a program on the host OS that opens up a socket on startup. Here is the ping program itself:
# Run this on the PC that want to check if other PC is online.
from socket import *
def pingit(): # defining function for later use
s = socket(AF_INET, SOCK_STREAM) # Creates socket
host = 'localhost' # Enter the IP of the workstation here
port = 80 # Select port which should be pinged
try:
s.connect((host, port)) # tries to connect to the host
except ConnectionRefusedError: # if failed to connect
print("Server offline") # it prints that server is offline
s.close() #closes socket, so it can be re-used
pingit() # restarts whole process
while True: #If connected to host
print("Connected!") # prints message
s.close() # closes socket just in case
exit() # exits program
pingit() #Starts off whole process
And here you have the program that can recieve the ping request:
# this runs on remote pc that is going to be checked
from socket import *
HOST = 'localhost'
PORT = 80
BUFSIZ = 1024
ADDR = (HOST, PORT)
serversock = socket(AF_INET, SOCK_STREAM)
serversock.bind(ADDR)
serversock.listen(2)
while 1:
clientsock, addr = serversock.accept()
serversock.close()
exit()
To run a program without actually showing it, just save the file as .pyw instead of .py.
It makes it invisible until user checks running processes.
Hope it helped you
For simplicity, I use self-made functions based on socket.
def checkHostPort(HOSTNAME, PORT):
"""
check if host is reachable
"""
result = False
try:
destIp = socket.gethostbyname(HOSTNAME)
except:
return result
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(15)
try:
conn = s.connect((destIp, PORT))
result = True
conn.close()
except:
pass
return result
if Ip:Port is reachable, return True
If you wanna to simulate Ping, may refer to ping.py
Try this:
ret = os.system("ping -o -c 3 -W 3000 192.168.1.10")
if ret != 0:
print "Host is not up"
-o waits for only one packet
-W 3000 gives it only 3000 ms to reply to the packet.
-c 3 lets it try a few times so that your ping doesnt run forever
Use this and parse the string output
import subprocess
output = subprocess.Popen(["ping.exe","192.168.1.1"],stdout = subprocess.PIPE).communicate()[0]
How about the request module?
import requests
def ping_server(address):
try:
requests.get(address, timeout=1)
except requests.exceptions.ConnectTimeout:
return False
return True
No need to split urls to remove ports, or test ports, and no localhost false-positive.
Timeout amount doesn't really matter since it only hits the timeout when there is no server, which in my case meant performance no longer mattered. Otherwise, this returns at the speed of a request, which is plenty fast for me.
Timeout waits for the first bit, not total time, in case that matters.

Resources