I want to add a fetch call to the initial load of index.html on a local app. This data is otherwise loaded server-side, but for this particular case I need to make an http call in dev.
I am having difficulty proxying the webpack-dev-server with the fetch added to the DOM.
The proxy is working correctly after my app is instantiated and I use Axios to make http calls to /api, but on init, the proxy is still serving from localhost instead of the target endpoint.
This is a bit puzzling to me - why might the proxy work in JS post-load but not on init?
devServer: {
contentBase: '/some/path',
port: 9000,
https: true,
open: true,
compress: true,
hot: true,
proxy: { '/api/*': [Object] }
},
Script in index
<script>
(async function() {
const data = await getData();
addDataset(data);
async function getData() {
return fetch('/api/my/endpoint').then(
(response) => response.data.result,
);
}
function addDataset(data) {
var el = document.getElementById('root');
const parsed = JSON.parse(data);
Object.entries(parsed).forEach((entry) => {
const [k, val] = entry;
el.dataset[k] = JSON.stringify(val);
});
}
})();
</script>
Error
400 Bad request
Request URL - https://localhost:9000/api/my/endpoint
Related
I am using http-proxy to forward all requests in NextJS, I want to handle the self response data with selfHandleResponse option. Specifically, I want to do something like set a some data to cookie, etc...) in Next.js API Routes, but if I set selfHandleResponse to true, none of the webOutgoing passes are called, that resulted in my request never being responded. What should I do? Please help me, I have researched but no hope :(
My code below:
const handleProxyInit = (proxy) => {
proxy.on('proxyRes', (proxyRes, req, res) => {
let body = [];
proxyRes.on('data', function (chunk) {
body.push(chunk);
});
proxyRes.on('end', function () {
body = Buffer.concat(body).toString('utf8');
// what should I do here?
});
});
};
httpProxyMiddleware(req, res, {
...,
selfHandleResponse: true,
changeOrigin: true,
onProxyInit: handleProxyInit,
});
I am using fastify with next.js and I need to include tracing (requestId is the problem so far). What I am doing right now is creating a fastify onRequest hook and generating a requestId value and setting it in request object (could be as a request header as well). What I want is to get access to this request object for two reasons:
In logger object (pino in this case, I want to include the requestId in all custom server-side logs).
In all request that needs to be made to other services need to include the requestId in headers.
Maybe I am missing something trivial and I'm not doing it the best way.
HERE SOME SNIPPETS
This how I am generating the reqId
const fastify = fastifyFactory({
logger, // logger configuration (Pino instance with custom configuration, see below)
genReqId: () => {
return Math.random()
.toString(36)
.slice(-6);
}
});
pino instance
const pino = require('pino');
const logger = pino({
messageKey: 'message',
prettyPrint: true,
changeLevelName: 'severity',
useLevelLabels: true,
base: {
serviceContext: {
service: 'web'
}
},
level:'info'
});
module.exports = {
logger
};
This is a plugin to gets the reqId generated and setting it to a query property within request object
const tracing = function tracing(fastify, opt, next) {
fastify.addHook('onRequest', (req, res, nextRequest) => {
const { id } = req;
const logger = fastify.log.child({ reqId: id });
req.query.reqId = id;
fastify.log = logger; //overrides the current fastify logger to include the reqId in all custom logs
nextRequest();
});
next();
};
tracing[Symbol.for('skip-override')] = true;
module.exports = tracing;
I have no problem when using fastify.log.info(...) because how logger is overrided in each request, it will include the reqId as a child log. The problem is that I want to create a generic logger to use at any part and fastify logger is not available in React components (for example to write logs at getInitialProps). Another important think is tha I need to include this reqId in all request I send to other services (ex: when fetching data), this is why I tried to store this value in request object but need to get it.
Starting from a project build with:
npx create-next-app --example custom-server-fastify custom-server-fastify-app
And changing the server.js with:
const Next = require('next')
const Fastify = require('fastify')
// your pino config
const fastify = Fastify({
logger: {
level: 'info',
prettyPrint: true,
changeLevelName: 'severity',
useLevelLabels: true,
base: {
serviceContext: {
service: 'web'
}
}
},
genReqId: () => { return Math.random().toString(36).slice(-6) }
})
// your plugin
const aPlugin = function yourPlugin (fastify, opts, next) {
fastify.addHook('onRequest', (request, reply, next) => {
request.log.info('hello')
const { id } = request
request.query.reqId = id
next()
})
next()
}
aPlugin[Symbol.for('skip-override')] = true
fastify.register(aPlugin)
[.... other generated code]
const port = parseInt(process.env.PORT, 10) || 3000
[.... other generated code]
fastify.get('/*', (req, reply) => {
console.log('-------->', req.id, req.query.reqId) // both your id is ok
return app.handleRequest(req.req, reply.res).then(() => {
reply.sent = true
})
[.... other generated code]
})
Then:
npm run dev
# another console
curl http://localhost:3000/
It will print out:
[1558441374784] INFO : Server listening at http://127.0.0.1:3000
serviceContext: {
"service": "web"
}
> Ready on http://localhost:3000
[1558441405416] INFO : incoming request
serviceContext: {
"service": "web"
}
reqId: "2i810l"
req: {
"method": "GET",
"url": "/",
"hostname": "localhost:3000",
"remoteAddress": "127.0.0.1",
"remotePort": 57863
}
req id ----> 2i810l
--------> 2i810l 2i810l
[ event ] build page: /
[ wait ] compiling ...
[1558441406171] INFO : request completed
serviceContext: {
"service": "web"
}
reqId: "2i810l"
res: {
"statusCode": 200
}
responseTime: 753.012099981308
So I think the misunderstanding is in the request object that is the Fastify request and not the Node.js "low level" request object, that could be accessed with request.req.
Moreover, running fastify.log = logger; is dangerous because it means that each request override and create a new logger and change the logger for the fastify instance, this is not safe, and as shown it is not necessary.
If you want more child logger (per route prefix per example) I suggest exploring/using the onRegister hook.
EDIT:
Now the custom hook print:
[1558443540483] INFO : hello
serviceContext: {
"service": "web"
}
reqId: "zjuhw2"
I attempting to use ASP and vb.net to run a node.js file that returns a port number from Bungie's API. The issue I am having is that I can use this application locally, but not if I were to publish and deploy the code. The way I do this in Visual Studio is open the Package Manager Console, type in "node proxy.js," and copy the returned value (proxy.js is the node file I'm executing). Then I'd take that port number and add it into my code.
Obviously, this isn't the ideal way to execute this file. So, my question is: is there a way I can execute this file from the code behind in VB.NET? Or, is there a way that I can take my node file and execute it in VB? Here is the proxy.js file I'm working with:
var https = require('https');
var http = require('http');
var BUNGIE = {
host: 'www.bungie.net',
port: 443
};
function copyHeaderFrom(source) {
return function (target, k) {
if (typeof target[k.toLowerCase()] === 'undefined') {
target[k] = source[k];
}
return target;
};
}
http.createServer(function (req, res) {
var outboundData = {
method: req.method,
host: BUNGIE.host,
port: BUNGIE.port,
path: req.url,
headers: req.headers
};
console.log('req.headers.cookie:', req.headers.cookie);
outboundData.headers.host = BUNGIE.host;
console.log('outbound request ========================');
console.log(outboundData);
if (outboundData.method === 'OPTIONS') {
res.writeHead(200, {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Headers': outboundData.headers['access-control-request-headers'] || ''
});
return res.end();
}
https.request(outboundData, function (bungieRes) {
var initialHeaders = {
'access-control-allow-origin': '*',
'origin': outboundData.headers.origin
};
res.writeHead(
bungieRes.statusCode,
Object.keys(bungieRes.headers).reduce(copyHeaderFrom(bungieRes.headers), initialHeaders)
);
bungieRes.pipe(res);
}).end();
}).listen(process.argv[2], function () {
console.log('Bungie Proxy Server running on port %s', this.address().port);
});
I am using node-http-proxy. However, in addition to relaying HTTP requests, I also need to listen to the incoming and outgoing data.
Intercepting the response data is where I'm struggling. Node's ServerResponse object (and more generically the WritableStream interface) doesn't broadcast a 'data' event. http-proxy seems to create it's own internal request, which produces a ClientResponse object (which does broadcast the 'data' event) however this object is not exposed publically outside the proxy.
Any ideas how to solve this without monkey-patching node-http-proxy or creating a wrapper around the response object?
Related issue in issues of node-http-proxy on Github seems to imply this is not possible. For future attempts by others, here is how I hacked the issue:
you'll quickly find out that the proxy is only calling writeHead(), write() and end() methods of the res object
since res is already an EventEmitter, you can start emitting new custom events
listen for these new events to assemble the response data and then use it
var eventifyResponse = function(res) {
var methods = ['writeHead', 'write', 'end'];
methods.forEach(function(method){
var oldMethod = res[method]; // remember original method
res[method] = function() { // replace with a wrapper
oldMethod.apply(this, arguments); // call original method
arguments = Array.prototype.slice.call(arguments, 0);
arguments.unshift("method_" + method);
this.emit.apply(this, arguments); // broadcast the event
};
});
};
res = eventifyResponse(res), outputData = '';
res.on('method_writeHead', function(statusCode, headers) { saveHeaders(); });
res.on('method_write', function(data) { outputData += data; });
res.on('method_end', function(data) { use_data(outputData + data); });
proxy.proxyRequest(req, res, options)
This is a simple proxy server sniffing the traffic and writing it to console:
var http = require('http'),
httpProxy = require('http-proxy');
//
// Create a proxy server with custom application logic
//
var proxy = httpProxy.createProxyServer({});
// assign events
proxy.on('proxyRes', function (proxyRes, req, res) {
// collect response data
var proxyResData='';
proxyRes.on('data', function (chunk) {
proxyResData +=chunk;
});
proxyRes.on('end',function () {
var snifferData =
{
request:{
data:req.body,
headers:req.headers,
url:req.url,
method:req.method},
response:{
data:proxyResData,
headers:proxyRes.headers,
statusCode:proxyRes.statusCode}
};
console.log(snifferData);
});
// console.log('RAW Response from the target', JSON.stringify(proxyRes.headers, true, 2));
});
proxy.on('proxyReq', function(proxyReq, req, res, options) {
// collect request data
req.body='';
req.on('data', function (chunk) {
req.body +=chunk;
});
req.on('end', function () {
});
});
proxy.on('error',
function(err)
{
console.error(err);
});
// run the proxy server
var server = http.createServer(function(req, res) {
// every time a request comes proxy it:
proxy.web(req, res, {
target: 'http://localhost:4444'
});
});
console.log("listening on port 5556")
server.listen(5556);
I tried your hack but it didn't work for me. My use case is simple: I want to log the in- and outgoing traffic from an Android app to our staging server which is secured by basic auth.
https://github.com/greim/hoxy/
was the solution for me. My node-http-proxy always returned 500 (while the direct request to stage did not). Maybe the authorization headers would not be forwarded correctly or whatever.
Hoxy worked fine right from the start.
npm install hoxy [-g]
hoxy --port=<local-port> --stage=<your stage host>:<port>
As rules for logging I specified:
request: $aurl.log()
request: #log-headers()
request: $method.log()
request: $request-body.log()
response: $url.log()
response: $status-code.log()
response: $response-body.log()
Beware, this prints any binary content.
I am developing a node.js proxy server application and I want it to support HTTP and HTTPS(SSL) protocols (as server).
I'm currently using node-http-proxy like this:
const httpProxy = require('http-proxy'),
http = require('http');
var server = httpProxy.createServer(9000, 'localhost', function(req, res, proxy) {
console.log(req.url);
proxy.proxyRequest(req, res);
});
http.createServer(function(req, res) {
res.end('hello!');
}).listen(9000);
server.listen(8000);
I setup my browser to use HTTP proxy on localhost:8000 and it works. I also want to catch HTTPS requests (ie. setup my browser to use localhost:8000 as HTTPS proxy as well and catch the requests in my application). Could you please help me how can I do that?
PS:
If I subscribe to upgrade event of httpProxy server object I can get the requests but I don't know how to forward the request and send response to client:
server.on('upgrade', function(req, socket, head) {
console.log(req.url);
// I don't know how to forward the request and send the response to client
});
Any helps would be appreciated.
Solutions barely exist for this, and the documentation is poor at best for supporting both on one server. The trick here is to understand that client proxy configurations may send https requests to an http proxy server. This is true for Firefox if you specify an HTTP proxy and then check "same for all protocols".
You can handle https connections sent to an HTTP server by listening for the "connect" event. Note that you won't have access to the response object on the connect event, only the socket and bodyhead. Data sent over this socket will remain encrypted to you as the proxy server.
In this solution, you don't have to make your own certificates, and you won't have certificate conflicts as a result. The traffic is simply proxied, not intercepted and rewritten with different certificates.
// Install npm dependencies first
// npm init
// npm install --save url#0.10.3
// npm install --save http-proxy#1.11.1
var httpProxy = require("http-proxy");
var http = require("http");
var url = require("url");
var net = require('net');
var server = http.createServer(function (req, res) {
var urlObj = url.parse(req.url);
var target = urlObj.protocol + "//" + urlObj.host;
console.log("Proxy HTTP request for:", target);
var proxy = httpProxy.createProxyServer({});
proxy.on("error", function (err, req, res) {
console.log("proxy error", err);
res.end();
});
proxy.web(req, res, {target: target});
}).listen(8080); //this is the port your clients will connect to
var regex_hostport = /^([^:]+)(:([0-9]+))?$/;
var getHostPortFromString = function (hostString, defaultPort) {
var host = hostString;
var port = defaultPort;
var result = regex_hostport.exec(hostString);
if (result != null) {
host = result[1];
if (result[2] != null) {
port = result[3];
}
}
return ( [host, port] );
};
server.addListener('connect', function (req, socket, bodyhead) {
var hostPort = getHostPortFromString(req.url, 443);
var hostDomain = hostPort[0];
var port = parseInt(hostPort[1]);
console.log("Proxying HTTPS request for:", hostDomain, port);
var proxySocket = new net.Socket();
proxySocket.connect(port, hostDomain, function () {
proxySocket.write(bodyhead);
socket.write("HTTP/" + req.httpVersion + " 200 Connection established\r\n\r\n");
}
);
proxySocket.on('data', function (chunk) {
socket.write(chunk);
});
proxySocket.on('end', function () {
socket.end();
});
proxySocket.on('error', function () {
socket.write("HTTP/" + req.httpVersion + " 500 Connection error\r\n\r\n");
socket.end();
});
socket.on('data', function (chunk) {
proxySocket.write(chunk);
});
socket.on('end', function () {
proxySocket.end();
});
socket.on('error', function () {
proxySocket.end();
});
});
Here is my NO-dependencies solution (pure NodeJS system libraries):
const http = require('http')
const port = process.env.PORT || 9191
const net = require('net')
const url = require('url')
const requestHandler = (req, res) => { // discard all request to proxy server except HTTP/1.1 CONNECT method
res.writeHead(405, {'Content-Type': 'text/plain'})
res.end('Method not allowed')
}
const server = http.createServer(requestHandler)
const listener = server.listen(port, (err) => {
if (err) {
return console.error(err)
}
const info = listener.address()
console.log(`Server is listening on address ${info.address} port ${info.port}`)
})
server.on('connect', (req, clientSocket, head) => { // listen only for HTTP/1.1 CONNECT method
console.log(clientSocket.remoteAddress, clientSocket.remotePort, req.method, req.url)
if (!req.headers['proxy-authorization']) { // here you can add check for any username/password, I just check that this header must exist!
clientSocket.write([
'HTTP/1.1 407 Proxy Authentication Required',
'Proxy-Authenticate: Basic realm="proxy"',
'Proxy-Connection: close',
].join('\r\n'))
clientSocket.end('\r\n\r\n') // empty body
return
}
const {port, hostname} = url.parse(`//${req.url}`, false, true) // extract destination host and port from CONNECT request
if (hostname && port) {
const serverErrorHandler = (err) => {
console.error(err.message)
if (clientSocket) {
clientSocket.end(`HTTP/1.1 500 ${err.message}\r\n`)
}
}
const serverEndHandler = () => {
if (clientSocket) {
clientSocket.end(`HTTP/1.1 500 External Server End\r\n`)
}
}
const serverSocket = net.connect(port, hostname) // connect to destination host and port
const clientErrorHandler = (err) => {
console.error(err.message)
if (serverSocket) {
serverSocket.end()
}
}
const clientEndHandler = () => {
if (serverSocket) {
serverSocket.end()
}
}
clientSocket.on('error', clientErrorHandler)
clientSocket.on('end', clientEndHandler)
serverSocket.on('error', serverErrorHandler)
serverSocket.on('end', serverEndHandler)
serverSocket.on('connect', () => {
clientSocket.write([
'HTTP/1.1 200 Connection Established',
'Proxy-agent: Node-VPN',
].join('\r\n'))
clientSocket.write('\r\n\r\n') // empty body
// "blindly" (for performance) pipe client socket and destination socket between each other
serverSocket.pipe(clientSocket, {end: false})
clientSocket.pipe(serverSocket, {end: false})
})
} else {
clientSocket.end('HTTP/1.1 400 Bad Request\r\n')
clientSocket.destroy()
}
})
I tested this code with Firefox Proxy Settings (it even asks for username and password!). I entered IP address of machine where this code is runned and 9191 port as you can see in the code. I also set "Use this proxy server for all protocols". I run this code locally and on VPS - in both cases works!
You can test your NodeJS proxy with curl:
curl -x http://username:password#127.0.0.1:9191 https://www.google.com/
I have created a http/https proxy with the aid of the http-proxy module: https://gist.github.com/ncthis/6863947
Code as of now:
var fs = require('fs'),
http = require('http'),
https = require('https'),
httpProxy = require('http-proxy');
var isHttps = true; // do you want a https proxy?
var options = {
https: {
key: fs.readFileSync('key.pem'),
cert: fs.readFileSync('key-cert.pem')
}
};
// this is the target server
var proxy = new httpProxy.HttpProxy({
target: {
host: '127.0.0.1',
port: 8080
}
});
if (isHttps)
https.createServer(options.https, function(req, res) {
console.log('Proxying https request at %s', new Date());
proxy.proxyRequest(req, res);
}).listen(443, function(err) {
if (err)
console.log('Error serving https proxy request: %s', req);
console.log('Created https proxy. Forwarding requests from %s to %s:%s', '443', proxy.target.host, proxy.target.port);
});
else
http.createServer(options.https, function(req, res) {
console.log('Proxying http request at %s', new Date());
console.log(req);
proxy.proxyRequest(req, res);
}).listen(80, function(err) {
if (err)
console.log('Error serving http proxy request: %s', req);
console.log('Created http proxy. Forwarding requests from %s to %s:%s', '80', proxy.target.host, proxy.target.port);
});
The node-http-proxy docs contain examples of this. Look for "Proxying to HTTPS from HTTPS" at https://github.com/nodejitsu/node-http-proxy The configuration process is slightly different in every browser. Some have the option to use your proxy settings for all protocols; some you need to configure the SSL proxy separately.