I'm trying to send logs into datadog using rsyslog. Ideally, I'm trying to do this without having the logs stored on the server hosting rsyslog. I've run into an error in my config that I haven't been able to find out much about. The error occurs on startup of rsyslog.
omfwd: could not get addrinfo for hostname '(null)':'(null)': Name or service not known [v8.2001.0 try https://www.rsyslog.com/e/2007 ]
Here's the portion I've added into the default rsyslog.config
module(load="imudp")
input(type="imudp" port="514" ruleset="datadog")
ruleset(name="datadog"){
action(
type="omfwd"
action.resumeRetryCount="-1"
queue.type="linkedList"
queue.saveOnShutdown="on"
queue.maxDiskSpace="1g"
queue.fileName="fwdRule1"
)
$template DatadogFormat,"00000000000000000 <%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% - - - %msg%\n "
$DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.logs.datadoghq.com
*.* ##intake.logs.datadoghq.com:10516;DatadogFormat
}
First things first.
The module imudp enables log reception over udp.
The module omfwd enables log forwarding over (tcp, udp, ...)
So most probably - or atleast as far as i can tell - with rsyslog you just want to log messages locally and then send them to datadog.
I don't know anything about the $ActionSendStreamDriver tags, so I can't help you there. But what is jumping out is, that in your action you haven't defined where the logs should be sent to.
ruleset(name="datadog"){
action(
type="omfwd"
target="10.100.1.1"
port="514"
protocol="udp"
...
)
...
}
I'm newish to LUA and want to practice some LUA scripting using nginx/openrestry.
Is there a workflow where I can use docker that runs openresty, and link my laptops filesystem to my docker container such that when I make a change to my lua script, I can quickly reload openrestry server so I can quickly see my lua changes take effect?
Any help or guidance would be appreciated.
You can disable Lua code cache — https://github.com/openresty/lua-nginx-module#lua_code_cache — add lua_code_cache off inside the http or server directive block. This is not actually “hot reload”, it's more like php request lifecycle:
every request served by ngx_lua will run in a separate Lua VM instance
You can think of it as if the code is hot–reloaded on each request.
However, pay attention to this:
Please note however, that Lua code written inlined within nginx.conf [...] will not be updated
It means that you should move all your Lua code from the nginx config to Lua modules and only require them:
server {
lua_code_cache off;
location /foo {
content_by_lua_block {
-- OK, the module will be imported (recompiled) on each request
require('mymodule').do_foo()
}
}
location /bar {
content_by_lua_block {
-- Bad, this inlined code won't be reloaded unless nginx is reloaded.
-- Move this code to a function inside a Lua module
-- (e.g., `mymodule.lua`).
local method = ngx.req.get_method()
if method == 'GET' then
-- handle GET
elseif method == 'POST' then
-- handle POST
else
return ngx.exit(ngx.HTTP_NOT_ALLOWED)
end
}
}
}
Then you can mount your Lua code from the host to the container using --mount or --volume: https://docs.docker.com/storage/bind-mounts/
I am using custom build for NodeMCU with following modules: cjson, file, gpio, http, net, node, ow, tmr, websocket, wifi.
conn = nil
conn=net.createConnection(net.TCP) -- 30 seconds timeout time of serv
conn:on("connection", function(conn, payload)
print("Server DB Connected, sending event") -- This line is not getting print on esplorer.
conn:send ( <some-data>)
end)
conn:connect(80,"mysite.com")
conn:close()
For debugging i tried using following code :
net.dns.resolve("www.google.com", function(sk, ip)
if (ip == nil) then
print("DNS fail!")
else
print("Resolved: " .. ip)
end
end)
But above is giving output " DNS fail! "
Site is working perfectly fine along with server.
A normal GET request to the same site using putty is returning correct data.
Please help me resolving the issue.
I am using OpenResty Nginx, and have a snippet of Lua script in the Nginx conf. (sample below, but truncated for brevity). This works fine when it is in the nginx.conf file.
rewrite_by_lua '
-- Load Memcached module
local memcached = require "resty.memcached"
local memc, err = memcached:new()
if not memc then
ngx.log(ngx.ERR, "Failed to instantiate memc: ", err)
return
end
'
However, if I extract this script in to a file, memcached.lua and then include it in the nginx config with rewrite_by_lua_file, I am unable to restart nginx and get notifications about syntax errors such as:
unexpected "," in /etc/nginx/sites-enabled/memcached.lua
unknown directive "--" in /etc/nginx/sites-enabled/memcached.lua
If I remove the ngx.log line completely the syntax error disappears.
How come the syntax is valid in the nginx conf but not when I extract it to a separate file? I've read somewhere about bytecode but not sure if that is related?
EDIT
Here is the full output of the .lua file. This exact code works if it's embedded directly in to the nginx.conf file via rewrite_by_lua
cat /etc/nginx/sites-enabled/memcached.lua
-- Define Memcached key
local memcached_key = "saas_" .. ngx.var.host
-- Load Memcached module
local memcached = require "resty.memcached"
local memc, err = memcached:new()
if not memc then
ngx.log(ngx.ERR, "Failed to instantiate memc: ")
end
-- Set Memcached timeout
memc:set_timeout(1000)
-- Connect to Memcached daemon
local ok, err = memc:connect("127.0.0.1", 11211)
if not ok then
ngx.log(ngx.ALERT, "Failed to connect to Memcached: ", err)
end
-- Query for branch name in Memcached
local branch, flags, err = memc:get(memcached_key)
if err then
ngx.log(ngx.NOTICE, "Failed to get account from Memcached: ", err)
end
-- If branch not found in Memcached, ask MySQL instead
if not branch then
-- Load MySQL module
local mysql = require "resty.mysql"
local db, err = mysql:new()
if not db then
ngx.log(ngx.ALERT, "Failed to instantiate MySQL: ", err)
return
end
-- Set MySQL timeout
db:set_timeout(1000) -- 1 sec
-- Connect to MySQL daemon
local ok, err, errno, sqlstate = db:connect{
path = "/var/run/mysqld/mysqld.sock",
database = "dbname",
user = "dbuser",
password = "dbpass" }
if not ok then
ngx.log(ngx.ALERT, "Failed to connect to DB: ", err, ": ", errno, " ", sqlstate)
return
end
-- Define subdomain string by extracting it from the http hostname
local subdomain = string.match(ngx.var.host, "([%w-]+).")
local quoted_subdomain = ngx.quote_sql_str(subdomain)
-- Execute an SQL query to get the branch name based on the subdomain
local sql = "SELECT branch FROM saas_account WHERE account = " .. quoted_subdomain
local res, err, errno, sqlstate = db:query(sql, 1)
if not res then
ngx.log(ngx.ALERT, "Bad result from DB: ", err, ": ", errno, ": ", sqlstate, ".")
return
end
-- Check we got a result
if not res[1] then
ngx.log(ngx.NOTICE, "No docroot was returned for subdomain: ", subdomain)
ngx.var.branch = "chaletops"
-- Assign the result from MySQL to the Nginx variable
else
ngx.var.branch = res[1].branch
-- Store the result from MySQL back in to Memcached with a lifetime of 1 hour (3600 seconds)
local ok, err = memc:set(memcached_key, ngx.var.branch, 3600)
if not ok then
ngx.log(ngx.WARN, "Failed to set branch in Memcached: ", err)
-- return
end
end
-- Save MySQL KeepAlive connection
local ok, err = db:set_keepalive(10000, 100)
if not ok then
ngx.log(ngx.ALERT, "Cannot set MySQL keepalive: ", err)
return
end
ngx.log(ngx.NOTICE, "Loading branch from MySQL: ", ngx.var.branch)
-- Else we did get a result from Memcached
else
-- Assign the result from Memcached to the Nginx variable
ngx.var.branch = branch
ngx.log(ngx.NOTICE, "Loading branch from Memcached: ", ngx.var.branch)
end
-- Save Memcached KeepAlive connection
local ok, err = memc:set_keepalive(10000, 100)
if not ok then
ngx.log(ngx.ALERT, "Cannot set Memcached keepalive: ", err)
-- return
end
SOLUTION
TL:DR - put the .lua file outside of your Nginx configuration directory.
After being asked to post the exact error messages, I had to recreate the problem. For reference and Google searchers the error messages are like this:
2015/12/17 10:04:02 [emerg] 6903#0: unexpected "," in /etc/nginx/sites-enabled/memcached.lua:18
This time I copied the Lua code out of my nginx.conf and put it in a file called memcached.lua and saved it in my project directory. I then added the following include to my nginx.conf in place of the Lua script:
rewrite_by_lua_file /path/to/my/project/memcached.lua;
Immediately it starts working!
Previously I had included it by symlinking it from my project directory in to /etc/nginx/sites-enabled/memcached.lua (assuming it would be better for it to actually live alongside the .conf that includes it) and then just used rewrite_by_lua_file memcached.lua; as it was relative the .conf file.
So I initially wondered if it was because the rewrite_by_lua_file function could not follow symlinks or something, but then it dawned on me that it was nothing to do with the Lua include statement, but OBVIOUSLY (now!) because of the symlink being in /etc/nginx/sites-enabled so Nginx was automatically trying to load the file as though it was a normal Nginx configuration file.
This is because Nginx (on Ubuntu/Linux Mint) is configured to import all config files using the path /etc/nginx/sites-enabled/* in to it's main config.
I think the reason I made this simple error is that on later versions of Ubuntu with Apache, a similar process happens but with Apache (which I am much more used to) you have an include path of /etc/nginx/sites-enabled/*.conf - i.e. it specifically requires a .conf file extension, which is why I incorrectly assumed that Nginx would not try to include my .lua file.
I am using Redis with Django project which is running on nginx and i am creating connection by code
red = redis.Redis("localhost", port=6397, db=5, socket_timeout=2)
Now by using Openresty i am fetching cache data from Redis using lua into nginx.conf file i am able to create connection
local redis = require "resty.redis"
local red = redis:new()
red:set_timeout(1000) -- 1 second
local ok, err = red:connect("10.0.0.161", 6379)
Here in nginx.conf file i am not able to understand how can define db value .
I tried local ok, err = red:connect("10.0.0.161", 6379, {db=5) but it is not working .
Please help me.
Just use select once connected:
red:select(5)