Deno - How to fetch data from distant API or URL? - deno

I'm wondering how I can get data from other servers and API with deno ? Everything in the documentation teach me about making http servers and read files from local source. But I can't find anything useful about reading something on the network.
How can read JSON data from the Stripe API ? Or if I want to read a HTML file with text inside ?
Thank you for your time!

I am just giving you an example of the GET request for fetching repositories of Github.
You can change the URL and Request Configuration as per your need.
In the code given below, I am calling another API of Github. By using the fetch() method you can do that.
fetch() method first takes the URL as the first parameter and the next parameter is RequestInit which takes the request method type, headers, body, etc and at the end returning JSON response of that API call.
const githubResponse = async (): Promise<any> => {
const response = await fetch("https://api.github.com/search/repositories?q=android", {
method: "GET",
headers: {
"Content-Type": "application/json",
},
});
return response.json(); // For JSON Response
// return response.text(); // For HTML or Text Response
}
console.log(await githubResponse());
I have written the above code in a ts file named Testing.ts . So, you can run the above code by the command given below:
deno run --allow-net Testing.ts
Next, I am giving you a sample POST request code:
const githubResponse = async (): Promise<any> => {
const body: URLSearchParams = new URLSearchParams({
q: "AvijitKarmakar",
});
const response = await fetch("https://api.github.com/search/repositories", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: body
});
return response.json();
// return response.text(); // For HTML or Text Response
}
console.log(await githubResponse());
You can see that I have created a body object and passed it in the RequestInit through the body parameter and also changed the request method type to POST.

You'll need to do a HTTP Request, for that in Deno you use fetch, the same Web API the browsers use.
To read JSON response:
const res = await fetch('https://api.stripe.com');
const data = await res.json();
If you want HTML:
const res = await fetch('https://example.com');
const html = await res.text();
// Now you can use some HTML parsing lib
fetch requires the --allow-net flag.

Deno strives to be as close to the existent browser API as possible.
That means, you can use fetch. Example:
// fetch-kitten.ts
fetch("https://placekitten.com/200/300").then(async (d) =>
Deno.writeFile("kitten.jpg", new Uint8Array(await d.arrayBuffer()))
);
CLI:
deno run --allow-net --allow-write fetch-kitten.ts
Reference

Related

Trying to implement shopify webhooks but getting 'InternalServerError: stream is not readable'

I'm building an app for shopify and need to add the GDPR webhooks. My back end is handled using next.js and I'm writing a webhook handler to verify them. The docs havent been very helpful because they dont show how to do it with node. This is my verification function.
export function verifiedShopifyWebhookHandler(
next: (req, res, body) => Promise
): NextApiHandler {
return async (req, res) => {
const hmacHeader = req.headers['x-shopify-hmac-sha256'];
const rawBody = await getRawBody(req);
const digest = crypto.createHmac('sha256', process.env.SHOPIFY_API_SECRET).update(rawBody).digest('base64');
if (digest === hmacHeader) {
return next(req, res, rawBody);
}
const webhookId = req.headers['x-shopify-webhook-id'];
return res.status(401).end();
};
}
But I get this Error: error - InternalServerError: stream is not readable
I think it has to do with now Next.js parses the incoming requests before they are sent to my api. Any ideas?
I discovered the answer. Next.js was pre parsing the body in the context which made it so that I couldn't use the raw body parser to parse it. By setting this:
export const config = {
api: {
bodyParser: false
}
};
above the api function in the api file it prevented next from parsing it and causing the issue. I found the answer because people had the same issue integrating swipe and using the bodyParser.

Sending a file as binary stream with Axios POST request

I am trying to send a zip file to my backend which expects to get a binary stream. My code does work, but I'm afraid that I'm doing it wrong and that it might not be consistent.
function httpPostFile(inputPath: string, userName: string): Promise<AxiosResponse> {
const input = fs.createReadStream(inputPath);
const headers = {
'Content-Type': 'application/octet-stream'
};
const params = ...
const url = '/someUrl';
return axios.post(url, input, {headers: headers, params: params});
}
// then handle the promise
When I play with the numbers of the highWaterMark param of fs.createReadStream I get a 400 error more often for some values. Why? Do I have to take care of sending the data in chunks myself? Also, what happens in axios.post - how does it deal with the highWaterMark I set?

Firebase function throwing timeout error when using http event

Function code below
prepay.post('/' , (req, res) => {
req.on("data", function (chunk) {
strdat += chunk;
console.log(strdat);
}).on("end", function()
{
var data = JSON.parse(strdat);
var cryp = crypto.createHash('sha512');
var text = \\ some data;
cryp.update(text);
var hash = cryp.digest('hex');
res.setHeader("Content-Type", "text/json");
res.setHeader("Access-Control-Allow-Origin", "*");
res.end(JSON.stringify(hash));
});
req.on('error', function(err){
console.log(err.message)
});
});
exports.prepay = functions.https.onRequest(prepay);
=================================
this is tried on emulator
in the logs getting ! functions: Your function timed out after ~60s. To configure this timeout, see https://firebase.google.com/docs/functions/manage-functions#set_timeout_and_memory_allocation.
\AppData\Roaming\npm\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:660 throw new Error("Function timed out.");
works fine when ran locally with nodejs using node server.js
not sure if req.on supported by firebase will be helpful if I get some reference on req.on in firebase functions
Your event-based sample of code won't work due to the preprocessing of the request that is done by the Firebase Functions SDK. Simply put, all of the 'data' and 'end' events have occurred prior to your code being executed.
Inside of functions.https.onRequest, the request is consumed and parsed according to it's content type automatically as documented here. If your request body is one of the below recognised types, it will be parsed and available as request.body. If you wish to work with the raw buffer of data, it is exposed as a Buffer as request.rawBody.
Content Type Request Body Behavior
application/json '{"name":"John"}' request.body.name equals 'John'
application/octet-stream 'my text' request.body equals '6d792074657874' (the raw bytes of the request; see the Node.js Buffer documentation)
text/plain 'my text' request.body equals 'my text'
application/x-www-form-urlencoded 'name=John' request.body.name equals 'John'
This preprocessing allows you to get to the actual function of your code faster.
prepay.post('/' , (req, res) => {
const data = req.body;
const cryp = crypto.createHash('sha512');
const text = \\ some data;
cryp.update(text);
const hash = cryp.digest('hex');
res.setHeader("Access-Control-Allow-Origin", "*");
res.json(hash);
});
exports.prepay = functions.https.onRequest(prepay);

create a synchronous http.get()

Im trying to handle a login via promises and http.get but i fail so hard I get following error :
Object doesn't support property or method 'toPromise'
My code is :
return this.http.get('http://localhost:5000/login/', {
headers: authHeader
}).map((response) => {
return response.json()
}).toPromise(null);
ive got it from :
https://github.com/johnpapa/angular2-there-and-back-again/blob/master/src/app/core/character.service.ts
UPDATE :
JohnPapa updated his project my friends
https://github.com/johnpapa/angular2-there-and-back-again/blob/master/app/core/character.service.ts
I wonder if you actually use promise since the HTTP support of Angular relies on Observables.
To get the response, you simply need to return the observable for your call:
getSomething() {
return this.http.get('http://localhost:5000/login/', {
headers: authHeader
}).map((response) => {
return response.json()
})
}
When calling the method, you can then register callbacks using the subscribe method:
getSomething().subscribe(
data => handleData(data),
err => reject(err));
If you really want to use promises (with the toPromise method), you should import this:
import 'rxjs/Rx';
See this issue for more details: https://github.com/angular/angular/issues/5632#issuecomment-167026172.
Otherwise, FYI calls aren't synchronous regarding HTTP in browsers...
Hope it helps you,
Thierry
If you want, you can use a TypeScript wrapper for sync-request library.
This TypeScript strongly-typed, fluent wrapper library is ts-sync-request.
ts-sync-request on npm
With this library, you can make sync http calls like below:
Your TypeScript classes:
class Request
{
Email: string;
}
class Response
{
isValid: boolean;
}
Install package in project:
npm i ts-sync-request
Then
import { SyncRequestClient } from 'ts-sync-request/dist'
GET:
let email = "jdoe#xyz.com";
let url = "http://localhost:59039/api/Movies/validateEmail/" + email;
var response = new SyncRequestClient()
.addHeader("Authorization", "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1NDc2OTg1MzgsIm5iZiI6MTU0NzY5NDIxOCwiaHR0cDovL3NjaGVtYXMueG1sc29hcC5vcmcvd3MvMjAwNS8wNS9pZGVudGl0eS9jbGFpbXMvbmFtZSI6InN0cmluZyIsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcm9sZSI6InN0cmluZyIsIkRPQiI6IjEvMTcvMjAxOSIsImlzcyI6InlvdXIgYXBwIiwiYXVkIjoidGhlIGNsaWVudCBvZiB5b3VyIGFwcCJ9.qxFdcdAVKG2Idcsk_tftnkkyB2vsaQx5py1KSMy3fT4")
.get<Response>(url);
POST:
let url = "http://localhost:59039/api/Movies/validateEmailPost";
let request = new Request();
request.Email = "jdoe#xyz.com";
var response = new SyncRequestClient()
.addHeader("Authorization", "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1NDc2OTg1MzgsIm5iZiI6MTU0NzY5NDIxOCwiaHR0cDovL3NjaGVtYXMueG1sc29hcC5vcmcvd3MvMjAwNS8wNS9pZGVudGl0eS9jbGFpbXMvbmFtZSI6InN0cmluZyIsImh0dHA6Ly9zY2hlbWFzLm1pY3Jvc29mdC5jb20vd3MvMjAwOC8wNi9pZGVudGl0eS9jbGFpbXMvcm9sZSI6InN0cmluZyIsIkRPQiI6IjEvMTcvMjAxOSIsImlzcyI6InlvdXIgYXBwIiwiYXVkIjoidGhlIGNsaWVudCBvZiB5b3VyIGFwcCJ9.qxFdcdAVKG2Idcsk_tftnkkyB2vsaQx5py1KSMy3fT4")
.post<Request, Response>(url, request);
Hope this helps.

Webhook for Mailgun POST?

I am trying to store email messages as JSON (as parsed by Mailgun) in a Mongo.Collection through a Mailgun webhook. I set up an iron-router server-side route to handle the request, but this.request.body is empty. I am using Mailgun's "Send A Sample POST" to send the request, and the POST looks fine using e.g. http://requestb.in/. I was hoping that request.body would have the data, as mentioned in How do I access HTTP POST data from meteor?. What am I doing wrong?
Router.map(function () {
this.route('insertMessage', {
where: 'server',
path: '/api/insert/message',
action: function() {
var req = this.request;
var res = this.response;
console.log(req.body);
...
I'm not sure that is the right syntax. Have you tried using Router.route ?
Router.route('insertMessage',
function () {
// NodeJS request object
var request = this.request;
// NodeJS response object
var response = this.response;
console.log("========= request: =============");
console.log(request);
// EDIT: also check out this.params object
console.log("========= this.params: =============");
console.log(this.params);
// EDIT 2: close the response. oops.
return response.end();
},
{
where: 'server',
path: '/api/insert/message'
}
);
I think the issue is that Mailgun sends a multipart POST request, e.g. it sends "fields" as well as "files" (i.e. attachments) and iron-router does not set up a body parser for multipart requests. This issue is discussed here and here on iron-router's Github Issues. I found this comment particularly helpful, and now I can parse Mailgun's sample POST properly.
To get this working, in a new Meteor project, I did
$ meteor add iron:router
$ meteor add meteorhacks:npm
In a root-level packages.json I have
{
"busboy": "0.2.9"
}
which, using the meteorhacks:npm package, makes the "busboy" npm package available for use on the server via Meteor.npmRequire.
Finally, in a server/rest-api.js I have
Router.route('/restful', {where: 'server'})
.post(function () {
var msg = this.request.body;
console.log(msg);
console.log(_.keys(msg));
this.response.end('post request\n');
});
var Busboy = Meteor.npmRequire("Busboy");
Router.onBeforeAction(function (req, res, next) {
if (req.method === "POST") {
var body = {}; // Store body fields and then pass them to request.
var busboy = new Busboy({ headers: req.headers });
busboy.on("field", function(fieldname, value) {
body[fieldname] = value;
});
busboy.on("finish", function () {
// Done parsing form
req.body = body;
next();
});
req.pipe(busboy);
}
});
In this way I can ignore files (i.e., I don't have a busboy.on("file" part) and have a this.request.body available in my routes that has all the POST fields as JSON.

Resources