Close stream after all data has been recieved - http

I am trying to make an HTTP request (I realise there is an http package that would probably make this easier but I want to learn how to do it with dart:io). My code below successfully prints out index.html from example.com, except the program runs forever, I assume because it is continues listening for data from the response. How do I stop the listener once all the data has been received from the stream? I assume I need to pass a handler to the onDone argument but I'm not sure what. I tried calling response.detachSocket() as per below, as that was the only logical seeming thing I could find, but it didn't work. If it's not obvious, I'm not totally sure what I'm doing here so any explanation would be greatly appreciated.
import 'dart:convert';
import 'dart:io';
main() async {
var client = new HttpClient();
var request = await client.getUrl(Uri.parse("http://www.example.com/"));
var response = await request.close();
response.transform(utf8.decoder).listen((data) {
print(data);
}, onDone: () => response.detachSocket());
}

You never close your HttpClient. Close it after you're done with it. The stream subscription will close itself when it's done, the onDone handler is a convenient way to run code when a stream completes, it's not necessary to use it to close the stream.
import 'dart:convert';
import 'dart:io';
main() async {
var client = new HttpClient();
var request = await client.getUrl(Uri.parse("http://www.example.com/"));
var response = await request.close();
response.transform(utf8.decoder).listen((data) {
print(data);
}, onDone: () => response.detachSocket());
client.close();
}
There is also no need to detach the socket.
You could even call client.close earlier with the force parameter set to false, which is the default:
main() async {
var client = new HttpClient();
var request = await client.getUrl(Uri.parse("http://www.example.com/"));
client.close();
var response = await request.close();
response.transform(utf8.decoder).listen((data) {
print(data);
});
}

Related

Post request to normal route, not using API middleware

Below my pages directory i have a few routes (for example "/product/details").
I'm using getServerSideProps() to have the page render server side.
How can i send a POST request containing data in the body to this page directly?
The idea would be that i can do something like this:
export async function getServerSideProps(postData) {
return {
props: {postData.body},
}
}
I've tried console logging "postData". I can see that the post request headers are being sent, but the request body is missing.
Thanks
Edit:
I'm doing the posting using Postman, and i'm sending a raw body of type JSON containing a single key:value. But as i said, the page doesn't seem to receive the posted data.
Here is a code snippet for how i'm sending a post request to a route using puppeteer:
const page = await puppeteerConnection.newPage();
await page.setRequestInterception(true);
await page.once('request', (request) => {
let data = {
'method': 'POST',
'postData': JSON.stringify(jsonData),
'headers': {
...request.headers(),
'Content-Type': 'application/json'
},
};
request.continue(data);
page.setRequestInterception(false);
});
await page.goto('pathToNextJSRoute');
getServerSideProps() accepts a context parameter (which you've named postData in your example) and one of the keys in that object (req) contains the request body you're looking for. It arrives as a readable stream of byte data, though, so you'll need to convert it first:
const streamToString = async (stream) => {
if (stream) {
const chunks = [];
for await (const chunk of stream) {
chunks.push(Buffer.from(chunk));
}
return Buffer.concat(chunks).toString("utf-8");
}
return null;
};
export async function getServerSideProps(context) {
let data = null;
if (context.req.method === "POST") {
const body = await streamToString(context.req);
data = JSON.parse(body);
}
console.log(data);
return {
props: { data },
};
}

How to deal with slow loading pages in next js

I have a page which requires making an HTTP request to an API which might take more than 10 seconds to respond, and my host limits me to 10 second executions.
Is there a way that I can load a temporary page or something and then asynchronously load the rest of the data? I'm currently doing this:
export async function getServerSideProps({ params }) {
const res = await fetch(`${process.env.API_USER}name=${params['name']}`)
const videos = await res.json()
const tag_res = await fetch(`${process.env.API_TAG}author=${params['name']}`)
const tags = await tag_res.json()
const name = params['name']
return {
props: { videos, tags, name }, // will be passed to the page component as props
}
}
Lets's move your HTTP request from getServerSideProps to client side (your components)
// Functional component
useEffect(() => {
fetch(...)
}, [])
// Class-based component
componentDidMount() {
fetch(...)
}
If you still want to stick with getServerSideProps, maybe you have to upgrade/switch your host, or implement a proxy/wrapper server for handling your HTTP request and return response as fast as it can

how to download a file in dart using Stream and Completer

I need to download a huge file in dart and also show progress while downloading it.
So I implemented this function:
import 'dart:io';
Future<void> download(String url, String path) async {
HttpClient client = HttpClient();
final request = await client.getUrl(Uri.parse(url));
request.close().then((HttpClientResponse response) async {
final file = File(path).openWrite(mode: FileMode.writeOnlyAppend);
await response.listen((data) {
file.add(data);
// show progress bar
...
}, onError: (e) {
throw e;
}, onDone: () {
file.close().then((e) {
print("Done");
return;
});
});
});
}
This works great with for downloading 1 file at a time.
But I want to able to call it in a loop and control the number of downloads that can occur at once.
Which brings me to this question on how to implement this using a Stream and Completer.

Flutter: HTTP get request body is empty

I'm playing a bit with Flutter and try to perform a http get request. Though I'm always getting an empty body in the response.
For example with the following code :
import 'package:http/http.dart' as http;
[...]
http.Client client = new http.Client();
client
.get("https://www.googleapis.com/books/v1/volumes?q=$text")
.then((http.Response response) {
print(response.statusCode);
print(response.body);
setState(() {
_isLoading = false;
});
});
I get the following result :
200
{
Do you have any ideas ?
Thanks by advance !
EDIT
It appears that the problem only happens on iOS devices. It works as expected on Android.
Can you try this below code. The code is untested.
import 'dart:io';
import 'dart:convert';
main() async {
try {
var client = new HttpClient();
String text = "example";
var uri = Uri.parse("https://www.googleapis.com/books/v1/volumes?q=$text");
var request = await client.getUrl(uri);
var response = await request.close();
var responseBody = await response.transform(UTF8.decoder).join();
print(responseBody);
} catch (exception) {
print(exception);
}
}
Probably you forget the headers , for example :
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'X-Requested-With': 'XMLHttpRequest',
},
Try debugPrint() instead of print(). It will print all body to console
https://flutter.io/debugging/#print-and-debugprint-with-flutter-logs

How to set a timeout in Angular 2 when using HTTP + toPromise()?

I found this answer where the solution proposed is to use Observables to set http requests' timeout.
However my code is structured to use mostly promises (I use observables where I want to have data automatically updated - that's not the case of the API calls).
Here is my code (inspired by Angular 2 tutorial):
makePostRequest(requestUrl:string, requestBody: any, requestOptions?: RequestOptions): Promise<any> {
requestOptions = requestOptions || new RequestOptions({ headers: this._defaultHeaders });
return this._http.post(requestUrl, JSON.stringify(requestBody), requestOptions)
.toPromise()
.then(this.extractData)
.catch(this.handleError)
}
How to set a timeout and throw an error (if the timeout expires) that I then catch in .catch() or - alternatively - replicate the exact precise behavior with Observables (including converting the result to a Promise and not monitoring for monitoring for API update(*))?
(*) NOTE: I'm not sure whether Observables keep calling the APIs to check for new data, but that's not the point of my question, I just want to make sure this behavior does not occur.
I would expect this to do what you want (not tried):
makePostRequest(requestUrl:string, requestBody: any, requestOptions?: RequestOptions): Promise<any> {
requestOptions = requestOptions || new RequestOptions({ headers: this._defaultHeaders });
return this._http.post(requestUrl, JSON.stringify(requestBody), requestOptions)
.timeout(3000, new Error('timeout exceeded'))
.toPromise()
.then(this.extractData)
.catch(this.handleError)
}
From Angular2 timeout in http post request
The solution (right chain + imports) I found:
// ! must import these
...
import 'rxjs/add/observable/throw';
import 'rxjs/add/operator/map';
import 'rxjs/add/operator/toPromise';
return this._http.get(requestUrl, requestOptions)
.timeout(5000, new Error( 'HTTP (GET) timeout for path: ' + requestUrl))
.map(this.extractData)
.toPromise()
.catch(this.handleError);
I approached this a bit differently. I had logic that relied on a promise being returned—and doing .timeout caused it to immediately fail regardless of the timeout duration.
My solution was to create a new promise instead of using toPromise:
const timeoutInMs = 3000;
const request = this._http
.post(/* ... */)
.timeout(timeoutInMs);
return new Promise((resolve, reject) => {
request
.take(1)
.subscribe(
data => resolve(data),
error => reject(error),
);
});
If you use this a lot, you could refactor it into a function (not yet tested):
const toPromiseWithTimeout = <T>(obs: Observable<T>, ms): Promise<T> =>
new Promise<T>((resolve, reject) => {
obs
.timeout(ms)
.take(1)
.subscribe(
data => resolve(data),
error => reject(error),
);
});
And to use it:
const timeoutInMs = 3000;
const request = this._http
.post<ResponseType>(/* ... */);
return toPromiseWithTimeout(request, timeoutInMs);

Resources