I have been trying to implement an SSE stream with Koa for hours now but got the following error when trying to send a message to my client after initializing the connection.
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
Here's how I set up my SSE:
Client-side:
const source = new EventSource("http://localhost:8080/stream");
this.source.onmessage = (e) => {
console.log("---- RECEIVED MESSAGE: ", e.data);
};
// Catches errors
this.source.onerror = (e) => {
console.log("---- ERROR: ", e.data);
};
Server-side (Koa):
// Entry point to our SSE stream
router.get('/stream', ctx => {
// Set response status, type and headers
ctx.response.status = 200;
ctx.response.type = 'text/event-stream';
ctx.response.set({
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
});
// Called when another route is reached
// Should send to the client the following
ctx.app.on('message', data => {
ctx.res.write(`event: Test\n`);
ctx.res.write(`data: This is test data\n\n`);
});
});
The error comes when we call ctx.res.write once a message is received.
Why is my stream ended although nothing explicitly is doing it?
How may I send a message through the stream with Koa?
Koa is entirely promise based and everything is a middleware.
Every middleware returns a promise (or nothing). The middleware chain is effectively 'awaited' and once the middleware returns, Koa knows the response is done and will end the stream.
To make sure that Koa doesn't do this, you have to make sure that the chain of middlewares don't end. To do this, you need to return a promise that only resolves when you're done streaming.
A quick hack to demonstrate would be to return a promise that doesn't resolve:
return new Promise( resolve => { }});
Related
I have written a handler function inside my nextjs page/api folder;
handler(req, res) {}
Am using #influxdata/influxDb-client as mentioned in the documentation. Am using
from(queryAPI.rows(query).pipe(....).subscribe(next(value)=> {results.push(value}, complete(console.log(results); res.status(200).json(results)}
Am getting all the query value, once the observable is completed. it works most of the time.
Am pushing the intermediate results in the next part of the subscriber and trying to send the results back to client in the complete part of the subscriber. I want the request handler to wait till i get all my values from influx DB query in the complete part of the subscriber and can send the value back to client..
But the issue "Handler function will not Wait till the observable is completed". Handler function returns, before the observer gets completed. Am getting error: API resolved without sending a response...
I get all the values only when the observer is completed.
I don't know how to handle the scenario.
How can I make the handler function wait until the observable is completed?
I found the solution for the same
I used new Promise() with await, added my observable inside this promise and resolved the promise on Complete of the subscribe.
Code will look like the following :
export async function handler (req, res) {
const results=[];
await new Promise((resolve, reject) => {
from((queryAPIs.rows(query))
.pipe(map(({values, tableMeta}) => tableMeta.toObject(values)))
.subscribe(
{
next(object) => {results.push(object)}
complete() => { resolve (results) }
error(err) => { reject (err) }
});
res.status(200).send(results);
}
}
I thought that event.passThroughOnException(); should set the fail open strategy for my worker, so that if an exception is raised from my code, original requests are sent to my origin server, but it seems that it’s missing post data. I think that’s because the request body is a readable stream and once read it cannot be read again, but how to manage this scenario?
addEventListener('fetch', (event) => {
event.passThroughOnException();
event.respondWith(handleRequest(event));
});
async function handleRequest(event: FetchEvent): Promise<Response> {
const response = await fetch(event.request);
// do something here that potentially raises an Exception
// #ts-ignore
ohnoez(); // deliberate failure
return response;
}
As you can see in the below image, the origin server did not receive any body (foobar):
Unfortunately, this is a known limitation of passThroughOnException(). The Workers Runtime uses streaming for request and response bodies; it does not buffer the body. As a result, once the body is consumed, it is gone. So if you forward the request, and then throw an exception afterwards, the request body is not available to send again.
Did a workaround by cloning event.request, then add a try/catch in handleRequest. On catch(err), send the request to origin using fetch while passing the cloned request.
// Pass request to whatever it requested
async function passThrough(request: Request): Promise<Response> {
try {
let response = await fetch(request)
// Make the headers mutable by re-constructing the Response.
response = new Response(response.body, response)
return response
} catch (err) {
return ErrorResponse.NewError(err).respond()
}
}
// request handler
async function handleRequest(event: FetchEvent): Promise<Response> {
const request = event.request
const requestClone = event.request.clone()
let resp
try {
// handle request
resp = await handler.api(request)
} catch (err) {
// Pass through manually on exception (because event.passThroughOnException
// does not pass request body, so use that as a last resort)
resp = await passThrough(requestClone)
}
return resp
}
addEventListener('fetch', (event) => {
// Still added passThroughOnException here
// in case the `passThrough` function throws exception
event.passThroughOnException()
event.respondWith(handleRequest(event))
})
Seems to work OK so far. Would love to know if there are other solutions as well.
I was reading the MDN docs, and got confused why event.respondWith can have a fetch object returned? Isn't the actual request initiator expecting a response instead of a fetch?
addEventListener('fetch', event => {
// Prevent the default, and handle the request ourselves.
event.respondWith(async function() {
// Try to get the response from a cache.
const cachedResponse = await caches.match(event.request);
// Return it if we found one.
if (cachedResponse) return cachedResponse;
// If we didn't find a match in the cache, use the network.
return fetch(event.request);
}());
});
The actual request initiator is not expecting a response. It is expecting a promise that resolves into a response. The MDN docs say exactly that:
The respondWith() method of FetchEvent prevents the browser's default
fetch handling, and allows you to provide a promise for a Response
yourself.
You are not returning a fetch object here when you call fetch(event.request). You are calling the fetch method which returns a promise that resolves into a response.
You can return any promise that resolves to a response here, like so:
return new Promise(function(resolve, reject) {
setTimeout(function() {
resolve(/* { Fake Response Object } */);
}, 1500);
});
After having implemented SSE with Express I wanted to do the same with Koa like so:
const Koa = require('koa');
const Router = require('koa-router');
const app = new Koa();
const router = new Router();
router.get('/stream', (ctx, next) => {
ctx.set({
'Access-Control-Allow-Origin': '*',
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'Keep-Alive',
});
const id = new Date().toLocaleTimeString();
ctx.res.write(`id: ${id}'\n`);
ctx.res.write(`data: CONNECTION ESTABLISHED)}\n\n`);
next();
});
app.use(router.routes());
app.listen(8080, () => {
console.log('Listening on port 8080');
});
And for my client, in a React component's constructor:
constructor(props) {
super(props);
this.state = {
source: new EventSource("http://localhost:8080/stream"),
};
}
But for some reason, I received the following error message client-side:
Firefox can’t establish a connection to the server at http://localhost:8080/stream.
Even though my client's request to /stream does go through (but no answer is sent back).
What could be causing this problem with the SSE connection?
I have a Koa server listening on a given port, a route to catch the initial GET request with the correct header data and yet it fails.
A major problem here is that Koa will 'end' the HTTP response as soon as all middleware have run.
This happens immediately after the function ends, of if the function returns a promise, when the promise has resolved. To keep a connection open and circumvent Koa's response handling, you need to make sure that the function 'never ends', and the best way to do that is to simply return a promise that does not resolve.
You're effectively taking over the response handling and stopping Koa from doing so. At this point you can start doing stuff with the socket.
I'm not sure if a more appropriate way exists to handle this in Koa, but this is how I've solved this in the past.
I have the following code (simplified for this post) - assume an initial call to onStart().
Running this works fine. If I lose the internet connection I get the net::ERR_INTERNET_DISCONNECTED error (as expected) but the polling stops.
Clearly I am not handling any errors here as that is where I'm getting stuck. I'm not clear where I handle those errors and how? Do I need to call startPolling() again?
I need the polling to continue even if there is no internet connection, so that on re-connection data is updated. Any advice please?
onStart() {
this.startPolling().subscribe(data => {
// do something with the data
});
}
startPolling(): Observable<any> {
return Observable
.interval(10000)
.flatMap(() => this.getData());
}
getData() {
var url = `http://someurl.com/api`;
return this.http.get(url)
.map(response => {
return response.json();
});
}
Thanks in advance.
If you know the error happens because of this.http.get(url) then you can add catch() operator that lets you subscribe to another Observable instead of the source Observable that sent an error notification.
getData() {
var url = `http://someurl.com/api`;
return this.http.get(url)
.catch(err => Observable.empty())
.map(response => {
return response.json();
});
}
This will simply ignore the error and won't emit anything.