pulling table with redis on lua - nginx

I'm running LUA on Nginx. I decided to fetch some variables via Redis. I am using a table on lua. It is like this;
local ip_blacklist = {
"1.1.1.1",
"1.1.1.2",
}
I'm printing on Nginx;
1.1.1.1
1.1.1.2
I want to keep the values here on Redis and not on Lua. My redis : http://prntscr.com/10sv3ln
My Lua command;
local ip = red:hmget("iplist", "ip_blacklist")
I'm printing on Nginx;
{"1.1.1.1","1.1.1.2",}
Its data does not come as a table and functions do not work. How can I call this data like local ip_blacklist?

https://redis.io/topics/data-types
Redis Hashes are maps between string fields and string values
You cannot store a Lua table as a hash value directly. As I understand, you have stored a literal string {"1.1.1.1","1.1.1.2",} using RedisInsight, but it doesn't work that way.
You can use JSON for serialization:
server {
location / {
content_by_lua_block {
local redis = require('resty.redis')
local json = require('cjson.safe')
local red = redis:new()
red:connect('127.0.0.1', 6379)
-- set a string with a JSON array as a hash value; you can use RedisInsight for this step
red:hset('iplist', 'ip_blacklist', '["1.1.1.1", "1.1.1.2"]')
-- read a hash value as a string (a serialized JSON array)
local ip_blacklist_json = red:hget('iplist', 'ip_blacklist')
-- decode the JSON array to a Lua table
local ip_blacklist = json.decode(ip_blacklist_json)
ngx.say(type(ip_blacklist))
for _, ip in ipairs(ip_blacklist) do
ngx.say(ip)
end
}
}
}
Output:
table
1.1.1.1
1.1.1.2

Related

FTP_INCORRECT_HOST_KEY in N/SFTP Module

While creating the connection from NetSuite to SFTP using N/SFTP module i'm facing an error states:
"FTP_INCORRECT_HOST_KEY","message":"Provided host key does not match
remote server's fingerprint."
I have tried checking with my server team but no hope. Can any one suggest me how to resolve this or how can i get an authorized finger print host key from server.
I have tried with Suitescript 2.0 module (N/SFTP) with the help of the tool mentioned below.
https://ursuscode.com/netsuite-tips/suitescript-2-0-sftp-tool/
/**
*#NApiVersion 2.x
#NScriptType ScheduledScript
*/
define(['N/sftp', 'N/file', 'N/runtime'],function(sftp, file,runtime) {
function execute(context)
{
var myPwdGuid = "Encrypted password by GUID";
var myHostKey = "Some long Host key around 380 characters";
// establish connection to remote FTP server
var connection = sftp.createConnection({
username: 'fuel_integration',
passwordGuid: myPwdGuid, // references var myPwdGuid
url: '59.165.215.45',//Example IP
directory: '/sftproot/TaleoSync',
restrictToScriptIds : runtime.getCurrentScript().id,
restrictToCurrentUser :false,
hostKey: myHostKey // references var myHostKey
});
// specify the file to upload using the N/file module
// download the file from the remote server
var downloadedFile = connection.download({
directory: '/sftproot/TaleoSync',
filename: 'Fuel Funnel Report_without filter.csv'
});
downloadedFile.folder = ;
downloadedFile.save();
context.response.write(' Downloaded "Fuel Funnel Report_without filter" to fileCabinet');
}
return {
execute: execute
};
});
I expect to create a connection between SFTP and NetSuite to down a file from SFTP and place it to NetSuite file cabinet.
A couple of things:
restrictToScriptIds : runtime.getCurrentScript().id,
restrictToCurrentUser :false,
Are not part of the createConnection signature. Those should have been used when you created a Suitelet to vault your credential.
However the hostkey complaint may be dealt with by using ssh-keyscan from a linux box.
ssh-keyscan 59.165.215.45
should replay with the server name then ssh-rsa then a long base64 string. Copy that string so it ends up in myHostKey and set the hostKeyType to RSA.

Teradata and Java server Connection

I am trying to connect to JAVA server using Teradata UDF, here is my code below.It uses HOST as "localhost" and PORT(integer) as "9091" and all the parameters such as PEM file location etc have been provided, but the function always exits from the catch block.I know that exit(0) is not allowed in Teradata UDF's, I have used it just for the sake of debugging.The same code connects to the server absolutely fine in case ORACLE UDF. Please tell which ports can be used for communication in case of Teradata, or whether an SSL connection to server is even supported in Teradata.
The server accepts argument of type request and returns result ,both of type vector <struct>.
Response CplusplusClient::startClient(DeTokenizationRequest request) {
boost::shared_ptr<TSSLSocketFactory> factory(new TSSLSocketFactory());
factory->loadPrivateKey(PRIVATE_KEY.c_str());
factory->loadCertificate(CERTIFICATE.c_str());
factory->loadTrustedCertificates(TRUSTED_CERTIFICATE.c_str());
factory->authenticate(true);
boost::shared_ptr<TSSLSocket> socket = factory->createSocket(HOST,PORT);
boost::shared_ptr<TTransport> transport(new TBufferedTransport(socket));
boost::shared_ptr<TProtocol> protocol(new TBinaryProtocol(transport));
XSecurityServiceClient client(protocol);
Response result;
try {
transport->open();
client.requested(result, request);
transport->close();
} catch (TException& tx) {
exit(0);
}
return result;
}
}
Changed the Port to 9092 and it worked

Send request with body with openresty lua-resty-http module

I am trying to send request via http module lua-resty-http. How I can send request with body data.
I have tried this
hc:connect("127.0.0.1", 82)
dates = ngx.req.get_post_args()
local hc = http:new()
result, errors = hc:request{
path = requrl,
method = "POST",
body = dates,
headers = {
["Host"] = "localhost",
},
}
Basically I am trying to send a lua table to another server location. And how to capture on that lua table location.
I'd appreciate a detailed explanation.
ngx.req.get_post_args() returns a table of key, value pairs. The body argument for the http client's request function must be in a format supported by OpenResty's cosocket send API. This means either a string, or array like table holding strings.
If you want to send a lua table with an HTTP request then you'll need a way to encode it to a string. A common approach is using JSON, and you can do this with the bundled cjson library:
local json = require "cjson"
local dates = ngx.req.get_post_args()
hc:request {
body = json.encode(dates),
...
}

How to safely handle raw (file) data in Java?

An image gets corrupted while being retrieved (through HTTP) and then sent (through HTTP) to a database. Image's raw data is handled in String form.
The service sends a GET for an image file, receives response with the raw image data (response's body) and the Content-Type. Then, a PUT request is sent with the aforementioned request's body and Content-Type header. (The PUT request is constructed by providing the body in String) This PUT request is sent to a RESTful database (CouchDB), creating an attachment (for those unfamiliar with CouchDB an attachment acts like a static file).
Now I have the original image, which my service GETs and PUTs to a database, and this 'copy' of the original image, that I can now GET from the database. If I then `curl --head -v "[copy's url]" it has the Content-Type of the original image, but Content-Length has changed, went from 200kb to about 400kb. If I GET the 'copy' image with a browser, it is not rendered, whereas, the original renders fine. It is corrupted.
What might be the cause? My guess is that while handling the raw data as a string, my framework guesses the encoding wrong and corrupts it. I have not been able to confirm or deny this. How could I handle this raw data/request body in a safe manner, or how could I properly handle the encoding (if that proves to be the problem)?
Details: Play2 Framework's HTTP client, Scala. Below a test to reproduce:
"able to copy an image" in {
def waitFor[T](future:Future[T]):T = { // to bypass futures
Await.result(future, Duration(10000, "millis"))
}
val originalImageUrl = "http://laughingsquid.com/wp-content/uploads/grumpy-cat.jpg"
val couchdbUrl = "http://admin:admin#localhost:5984/testdb"
val getOriginal:ws.Response = waitFor(WS.url(originalImageUrl).get)
getOriginal.status mustEqual 200
val rawImage:String = getOriginal.body
val originalContentType = getOriginal.header("Content-Type").get
// need an empty doc to have something to attach the attachment to
val emptyDocUrl = couchdbUrl + "/empty_doc"
val putEmptyDoc:ws.Response = waitFor(WS.url(emptyDocUrl).put("{}"))
putEmptyDoc.status mustEqual 201
//uploading an attachment will require the doc's revision
val emptyDocRev = (putEmptyDoc.json \ "rev").as[String]
// create actual attachment/static file
val attachmentUrl = emptyDocUrl + "/0"
val putAttachment:ws.Response = waitFor(WS.url(attachmentUrl)
.withHeaders(("If-Match", emptyDocRev), ("Content-Type", originalContentType))
.put(rawImage))
putAttachment.status mustEqual 201
// retrieve attachment
val getAttachment:ws.Response = waitFor(WS.url(attachmentUrl).get)
getAttachment.status mustEqual 200
val attachmentContentType = getAttachment.header("Content-Type").get
originalContentType mustEqual attachmentContentType
val originalAndCopyMatch = getOriginal.body == getAttachment.body
originalAndCopyMatch aka "original matches copy" must beTrue // << false
}
Fails at the last 'must':
[error] x able to copy an image
[error] original matches copy is false (ApplicationSpec.scala:112)
The conversion to String is definitely going to cause problems. You need to work with the bytes as Daniel mentioned.
Looking at the source it looks like ws.Response is just a wrapper. If you get to the underlying class then there are some methods that may help you. On the Java side, someone made a commit on GitHub to expose more ways of getting the response data other than a String.
I'm not familiar with scala but something like this may work:
getOriginal.getAHCResponse.getResponseBodyAsBytes
// instead of getOriginal.body
WS.scala
https://github.com/playframework/playframework/blob/master/framework/src/play/src/main/scala/play/api/libs/ws/WS.scala
WS.java
Here you can see that Response has some new methods, getBodyAsStream() and asByteArray.
https://github.com/playframework/playframework/blob/master/framework/src/play-java/src/main/java/play/libs/WS.java

How to get the user IP address in Meteor server?

I would like to get the user IP address in my meteor application, on the server side, so that I can log the IP address with a bunch of things (for example: non-registered users subscribing to a mailing list, or just doing anything important).
I know that the IP address 'seen' by the server can be different than the real source address when there are reverse proxies involved. In such situations, X-Forwarded-For header should be parsed to get the real public IP address of the user. Note that parsing X-Forwarded-For should not be automatic (see http://www.openinfo.co.uk/apache/index.html for a discussion of potential security issues).
External reference: This question came up on the meteor-talk mailing list in august 2012 (no solution offered).
1 - Without a http request, in the functions you should be able to get the clientIP with:
clientIP = this.connection.clientAddress;
//EX: you declare a submitForm function with Meteor.methods and
//you call it from the client with Meteor.call().
//In submitForm function you will have access to the client address as above
2 - With a http request and using iron-router and its Router.map function:
In the action function of the targeted route use:
clientIp = this.request.connection.remoteAddress;
3 - using Meteor.onConnection function:
Meteor.onConnection(function(conn) {
console.log(conn.clientAddress);
});
Similar to the TimDog answer but works with newer versions of Meteor:
var Fiber = Npm.require('fibers');
__meteor_bootstrap__.app
.use(function(req, res, next) {
Fiber(function () {
console.info(req.connection.remoteAddress);
next();
}).run();
});
This needs to be in your top-level server code (not in Meteor.startup)
This answer https://stackoverflow.com/a/22657421/2845061 already does a good job on showing how to get the client IP address.
I just want to note that if your app is served behind proxy servers (usually happens), you will need to set the HTTP_FORWARDED_COUNT environment variable to the number of proxies you are using.
Ref: https://docs.meteor.com/api/connections.html#Meteor-onConnection
You could do this in your server code:
Meteor.userIPMap = [];
__meteor_bootstrap__.app.on("request", function(req, res) {
var uid = Meteor.userId();
if (!uid) uid = "anonymous";
if (!_.any(Meteor.userIPMap, function(m) { m.userid === uid; })) {
Meteor.userIPMap.push({userid: uid, ip: req.connection.remoteAddress });
}
});
You'll then have a Meteor.userIPMap with a map of userids to ip addresses (to accommodate the x-forwarded-for header, use this function inside the above).
Three notes: (1) this will fire whenever there is a request in your app, so I'm not sure what kind of performance hit this will cause; (2) the __meteor_bootstrap__ object is going away soon I think with a forthcoming revamped package system; and (3) the anonymous user needs better handling here..you'll need a way to attach an anonymous user to an IP by a unique, persistent constraint in their request object.
You have to hook into the server sessions and grab the ip of the current user:
Meteor.userIP = function(uid) {
var k, ret, s, ss, _ref, _ref1, _ref2, _ref3;
ret = {};
if (uid != null) {
_ref = Meteor.default_server.sessions;
for (k in _ref) {
ss = _ref[k];
if (ss.userId === uid) {
s = ss;
}
}
if (s) {
ret.forwardedFor = ( _ref1 = s.socket) != null ?
( _ref2 = _ref1.headers) != null ?
_ref2['x-forwarded-for'] : void 0 : void 0;
ret.remoteAddress = ( _ref3 = s.socket) != null ?
_ref3.remoteAddress : void 0;
}
}
return ret.forwardedFor ? ret.forwardedFor : ret.remoteAddress;
};
Of course you will need the current user to be logged in. If you need it for anonymous users as well follow this post I wrote.
P.S. I know it's an old thread but it lacked a full answer or had code that no longer works.
Here's a way that has worked for me to get a client's IP address from anywhere on the server, without using additional packages. Working in Meteor 0.7 and should work in earlier versions as well.
On the client, get the socket URL (unique) and send it to the server. You can view the socket URL in the web console (under Network in Chrome and Safari).
socket_url = Meteor.default_connection._stream.socket._transport.url
Meteor.call('clientIP', socket_url)
Then, on the server, use the client's socket URL to find their IP in Meteor.server.sessions.
sr = socket_url.split('/')
socket_path = "/"+sr[sr.length-4]+"/"+sr[sr.length-3]+"/"+sr[sr.length-2]+"/"+sr[sr.length-1]
_.each(_.values(Meteor.server.sessions), (session) ->
if session.socket.url == socket_path
user_ip = session.socket.remoteAddress
)
user_ip now contains the connected client's IP address.

Resources