In response to a http request to an endpoint I have to basically make some computations, db calls, etc. and return the response to the requester. Out of the computations some of them can be made after the response is being sent and for my case it makes sense to return the response as fast as possible.
From what I've seen in ASP.NET once you return the response the whole request-response cycle is over. Is what I need achievable with basic code without using a job queue system or anything like that? I've looked into using Task and fire an async operation before sending the response back, but from what I've been reading it can be dangerous, the system might terminate the process randomly and such.
I'm very much used to node/express and I'm trying to do the equivalent of:
router.get('/myroute', executeStuff, returnResponse, executeStuff2);
// initial processing
executeStuff = (req, res, next) => {
someAsyncOperation()
.then(() => next())
.catch(...);
}
// send response after necessary stuff is done
returnResponse = (req, res, next) => {
res.send(responseBody);
next();
}
// do the after response processing, side effects, etc.
executeStuff2 = (req, res, next) => {
someAsyncOperation()
.then(() => {
// end of the chain
})
.catch(...);
}
Related
I have written a handler function inside my nextjs page/api folder;
handler(req, res) {}
Am using #influxdata/influxDb-client as mentioned in the documentation. Am using
from(queryAPI.rows(query).pipe(....).subscribe(next(value)=> {results.push(value}, complete(console.log(results); res.status(200).json(results)}
Am getting all the query value, once the observable is completed. it works most of the time.
Am pushing the intermediate results in the next part of the subscriber and trying to send the results back to client in the complete part of the subscriber. I want the request handler to wait till i get all my values from influx DB query in the complete part of the subscriber and can send the value back to client..
But the issue "Handler function will not Wait till the observable is completed". Handler function returns, before the observer gets completed. Am getting error: API resolved without sending a response...
I get all the values only when the observer is completed.
I don't know how to handle the scenario.
How can I make the handler function wait until the observable is completed?
I found the solution for the same
I used new Promise() with await, added my observable inside this promise and resolved the promise on Complete of the subscribe.
Code will look like the following :
export async function handler (req, res) {
const results=[];
await new Promise((resolve, reject) => {
from((queryAPIs.rows(query))
.pipe(map(({values, tableMeta}) => tableMeta.toObject(values)))
.subscribe(
{
next(object) => {results.push(object)}
complete() => { resolve (results) }
error(err) => { reject (err) }
});
res.status(200).send(results);
}
}
I thought that event.passThroughOnException(); should set the fail open strategy for my worker, so that if an exception is raised from my code, original requests are sent to my origin server, but it seems that it’s missing post data. I think that’s because the request body is a readable stream and once read it cannot be read again, but how to manage this scenario?
addEventListener('fetch', (event) => {
event.passThroughOnException();
event.respondWith(handleRequest(event));
});
async function handleRequest(event: FetchEvent): Promise<Response> {
const response = await fetch(event.request);
// do something here that potentially raises an Exception
// #ts-ignore
ohnoez(); // deliberate failure
return response;
}
As you can see in the below image, the origin server did not receive any body (foobar):
Unfortunately, this is a known limitation of passThroughOnException(). The Workers Runtime uses streaming for request and response bodies; it does not buffer the body. As a result, once the body is consumed, it is gone. So if you forward the request, and then throw an exception afterwards, the request body is not available to send again.
Did a workaround by cloning event.request, then add a try/catch in handleRequest. On catch(err), send the request to origin using fetch while passing the cloned request.
// Pass request to whatever it requested
async function passThrough(request: Request): Promise<Response> {
try {
let response = await fetch(request)
// Make the headers mutable by re-constructing the Response.
response = new Response(response.body, response)
return response
} catch (err) {
return ErrorResponse.NewError(err).respond()
}
}
// request handler
async function handleRequest(event: FetchEvent): Promise<Response> {
const request = event.request
const requestClone = event.request.clone()
let resp
try {
// handle request
resp = await handler.api(request)
} catch (err) {
// Pass through manually on exception (because event.passThroughOnException
// does not pass request body, so use that as a last resort)
resp = await passThrough(requestClone)
}
return resp
}
addEventListener('fetch', (event) => {
// Still added passThroughOnException here
// in case the `passThrough` function throws exception
event.passThroughOnException()
event.respondWith(handleRequest(event))
})
Seems to work OK so far. Would love to know if there are other solutions as well.
I'm having a hard time wrapping my brain around this pattern I am trying to implement so I'm hoping the stack overflow community might be able to help me work through a solution to this.
Currently I use redux-thunk along with superagent to handle calls to me API and syncing it all up with redux
An example of this might look like
export const getUser = (id) => {
return (dispatch) => {
const deferred = new Promise((resolve, reject) => {
const call = () => {
API.get(`/users/${id}`)
.then((response) => response.body)
.then((response) => {
if (response.message === 'User found') {
serializeUser(response.data).then((response) => {
resolve(response);
});
} else {
reject('not found');
}
}).catch((err) => {
handleCatch(err, dispatch).then(call).catch(reject)
});
}
call()
});
return deferred;
};
};
In the case where the server comes back with a 200 and some data I continue on with putting the data into the store and rendering to the page or whatever my application does.
In the case I receive an error I have attempted to write a function that will intercept those and determine if it should show an error on page or in the case of a 401 from our API, attempt a token refresh and then try to recall the method...
import { refreshToken } from '../actions/authentication';
export default (err, dispatch) => {
const deferred = new Promise((resolve, reject) => {
if (err.status === 401) {
dispatch(refreshToken()).then(resolve).catch(reject)
} else {
reject(err);
}
})
return deferred;
};
This works, however, I have to add this to each call, and it doesn't account for concurrent calls that should not attempt to call if there is a refresh in progress.
I've seen some things in my research on this topic that maybe redux-saga could work but I haven't been able to wrap my brain around how I might make this work
Basically, I need something like a queue that all my API requests will go into that is maybe debounced so any concurrent requests will just be pushed to the end and once a timeout ends the calls get stacked up, when the first call gets a 401 it pauses the queue until the token refresh either comes back successful, in which case it continues the queue, or with a failure, in which case it cancels all future requests from the queue and sends the user back to a login page
The thing I would be worried about here is if the first call in the stack takes a long time, I don't want the other calls to then have to wait a long time because it will increase the perceived loading time to the user
Is there a better way to handle keeping tokens refreshed?
I'm using rxjs v5.4.3, redux-observable v0.16.0.
in my application, I'd like to achieve below:
an user has auth token, and refresh token to regenerate auth token.
the user requests with auth token. (by emitting REQUEST action)
if it failed, request regenerating auth token with refresh token.
if refreshed, emit TOKEN_REFRESHED action to update auth token, and do not emit REQUEST_FAILURE.
if refreshing failed, emit REQUEST_FAILURE
after refreshing(and updating auth token reducer), retry requesting using the refreshed auth token.
if request succeeded, emit REQUEST_SUCCESS, and if failed, emit REQUEST_FAILURE.
I'd like to achieve like:
const fetchEpic = (action$: ActionsObservable<Action>, store: Store<IRootState>) => action$
.ofAction(actions.fetchPost)
.mergeMap(({ payload: { postId } })) => {
const { authToken, refreshToken } = store.getState().auth;
return api.fetchPost({ postId, authToken }) // this returns Observable<ResponseJSON>
.map(res => actions.fetchSuccess({ res })) // if success, just emit success-action with the response
.catch(err => {
if (isAuthTokenExpiredError(err) {
return api.reAuthenticate({ refreshToken })
.map(res => actions.refreshTokenSuccess({ authToken: res.authToken });
.catch(actions.fetchFailure({ err }))
// and retry fetchPost after re-authenticate!
}
return Observable.of(actions.fetchFailure({ err }))
})
}
is there any solution?
There are many ways to do it, but I would recommend splitting off the reauthentication into its own epic to make it easier to maintain/test/reuse.
Here's what that might look like:
const reAuthenticateEpic = (action$, store) =>
action$.ofType(actions.reAuthenticate)
.switchMap(() => {
const { refreshToken } = store.getState().auth;
return api.reAuthenticate({ refreshToken })
.map(res => actions.refreshTokenSuccess({ authToken: res.authToken }))
.catch(err => Observable.of(
actions.refreshTokenFailure({ err })
));
});
We'll also want to use something like Observable.defer so that each time we retry, we look up the latest version of the authToken:
Observable.defer(() => {
const { authToken } = store.getState().auth;
return api.fetchPost({ postId, authToken });
})
When we catch errors in fetchEpic and detect isAuthTokenExpiredError we return an Observable chain that:
Starts listening for a single refreshTokenSuccess, signalling we can retry
Just in case the reauthing itself fails, we listen for it with .takeUntil(action$.ofType(refreshTokenFailure)) so that we aren't waiting around forever--you might want to handle this case differently, your call.
mergeMap it to the original source, which is the second argument of the catch callback. The "source" is the Observable chain before the catch, and since Observables are lazy, when we receive the refreshTokenSuccess action it it will resubscribe to that chain again, effectively be a "retrying"
Merge the above chain with an Observable of an reAuthenticate action. This is used to kick off the actual reauth.
To summarize: the Observable chain we return from catch will first starting listening for refreshTokenSuccess, then it emits reAuthenticate, then when (and if) we receive refreshTokenSuccess we will then "retry" the source, our api.fetchPost() chain above the catch that we wrapped in an Observable.defer. If refreshTokenFailure is emitted before we receive our refreshTokenSuccess, we give up entirely.
const fetchEpic = (action$, store) =>
action$.ofType(actions.fetchPost)
.mergeMap(({ payload: { postId } })) =>
Observable.defer(() => {
const { authToken } = store.getState().auth;
return api.fetchPost({ postId, authToken });
})
.map(res => actions.fetchSuccess({ res }))
.catch((err, source) => {
if (isAuthTokenExpiredError(err)) {
// Start listening for refreshTokenSuccess, then kick off the reauth
return action$.ofType(actions.refreshTokenSuccess)
.takeUntil(action$.ofType(refreshTokenFailure))
.take(1)
.mergeMapTo(source) // same as .mergeMap(() => source)
.merge(
Observable.of(action.reAuthenticate())
);
} else {
return Observable.of(actions.fetchFailure({ err }));
}
});
);
These examples are untested, so I may have some minor issues but you hopefully get the gist. There's also probably a more elegant way to do this, but this will at least unblock you. (Others are more than welcome to edit this answer if they can decrease the complexity)
Side notes
This creates the slight potential for infinite retries, which can cause nasty issues both in the person's browser or your servers. It might be a good idea to only retry a set number of times, and/or put some sort of delay in between your retries. In practice this might not be worth worrying about, you'll know best.
You (or someone else reading this later) may be tempted to use .startWith(action.reAuthenticate()) instead of the merge, but be mindful that a startWith is just shorthand for a concat, not a merge, which means it would synchronously emit the action before we have started to listen for a success one. Usually that isn't an issue since http requests are async, but it's caused people bugs before.
So I would like to do something like:
app.On_All_Incoming_Request(function(req, res){
console.log('request received from a client.');
});
the current app.all() requires a path, and if I give for example this / then it only works when I'm on the homepage, so it's not really all..
In plain node.js it is as simple as writing anything after we create the http server, and before we do the page routing.
So how to do this with express, and what is the best way to do it?
Express is based on the Connect middleware.
The routing capabilities of Express are provided by the router of your app and you are free to add your own middlewares to your application.
var app = express.createServer();
// Your own super cool function
var logger = function(req, res, next) {
console.log("GOT REQUEST !");
next(); // Passing the request to the next handler in the stack.
}
app.configure(function(){
app.use(logger); // Here you add your logger to the stack.
app.use(app.router); // The Express routes handler.
});
app.get('/', function(req, res){
res.send('Hello World');
});
app.listen(3000);
It's that simple.
(PS : If you just want some logging you might consider using the logger provided by Connect)
You should do this:
app.all("*", (req, res, next) => {
console.log(req); // do anything you want here
next();
});
You can achieve it by introducing a middleware function.
app.use(your_function) can be of help. app.use with accept a function that will get executed on every request logged to your server.
Example:
app.use((req, res, next) => {
console.log("req received from client");
next(); // this will invoke next middleware function
});
Express supports wildcards in route paths. So app.all('*', function(req, res) {}) is one way to go.
But that's just for route handlers. The difference is that an Express route handler is expected to send a response, and, if it doesn't, Express will never send a response. If you want to do something without explicitly sending a response, like check for a header, you should use Express middleware. app.use(function(req, res, next) { doStuff(); next(); }