fetching mp3 file from MeteorJS and trying to convert it into a Blob so that I can play it - meteor

am playing around with downloading and serving mp3 files in Meteor.
I am trying to download an MP3 file (https://www.sample-videos.com/audio/mp3/crowd-cheering.mp3) on my MeteorJS Server side (to circumvent CORS issues) and then pass it back to the client to play it in a AUDIO tag.
In Meteor you use the Meteor.call function to call a server method. There is not much to configure, it's just a method call and a callback.
When I run the method I receive this:
content:
"ID3���#K `�)�<H� e0�)������1������J}��e����2L����������fȹ\�CO��ȹ'�����}$A�Lݓ����3D/����fijw��+�LF�$?��`R�l�YA:A��#�0��pq����4�.W"�P���2.Iƭ5��_I�d7d����L��p0��0A��cA�xc��ٲR�BL8䝠4���T��..etc..", data:null,
headers: {
accept-ranges:"bytes",
connection:"close",
content-length:"443926",
content-type:"audio/mpeg",
date:"Mon, 20 Aug 2018 13:36:11 GMT",
last-modified:"Fri, 17 Jun 2016 18:16:53 GMT",
server:"Apache",
statusCode:200
which is the working Mp3 file (the content-length is exactly the same as the file I write to disk on the MeteorJS Server side, and it is playable).
However, my following code doesn't let me convert the response into a BLOB:
```
MeteorObservable.call( 'episode.download', episode.url.url ).subscribe( ( result: any )=> {
console.log( 'response', result);
let URL = window.URL;
let blob = new Blob([ result.content ], {type: 'audio/mpeg'} );
console.log('blob', blob);
let audioUrl = URL.createObjectURL(blob);
let audioElement:any = document.getElementsByTagName('audio')[0];
audioElement.setAttribute("src", audioUrl);
audioElement.play();
})
When I run the code, the Blob has the wrong size and is not playable
Blob(769806) {size: 769806, type: "audio/mpeg"}
size:769806
type:"audio/mpeg"
__proto__:Blob
Uncaught (in promise) DOMException: Failed to load because no supported source was found.
On the backend I just run a return HTTP.get( url ); in the method which is using import { HTTP } from 'meteor/http'.
I have been trying to use btoa or atob but that doesn't work and as far as I know it is already a base64 encoded file, right?
I am not sure why the Blob constructor creates a larger file then the source returned from the backend. And I am not sure why it is not playing.
Can anyone point me to the right direction?

Finally found a solution that uses request instead of Meteor's HTTP:
First you need to install request and request-promise-native in order to make it easy to return your result to clients.
$ meteor npm install --save request request-promise-native
Now you just return the promise of the request in a Meteor method:
server/request.js
import { Meteor } from 'meteor/meteor'
import request from 'request-promise-native'
Meteor.methods({
getAudio (url) {
return request.get({url, encoding: null})
}
})
Notice the encoding: null flag, which causes the result to be binary. I found this in a comment of an answer related to downloading binary data via node. This causes not to use string but binary representation of the data (I don't know how but maybe it is a fallback that uses Node Buffer).
Now it gets interesting. On your client you wont receive a complex result anymore but either an Error or a Uint8Array which makes sense because Meteor uses EJSON to send data over the wires with DDP and the representation of binary data is a Uint8Array as described in the documentation.
Because you can just pass in a Uint8Array into a Blob you can now easily create the blob like so:
const blob = new Blob([utf8Array], {type: 'audio/mpeg'})
Summarizing all this into a small template if could look like this:
client/fetch.html
<template name="fetch">
<button id="fetchbutton">Fetch Mp3</button>
{{#if source}}
<audio id="player" src={{source}} preload="none" content="audio/mpeg" controls></audio>
{{/if}}
</template>
client/fetch.js
import { Template } from 'meteor/templating'
import { ReactiveVar } from 'meteor/reactive-var'
import './fetch.html'
Template.fetch.onCreated(function helloOnCreated () {
// counter starts at 0
this.source = new ReactiveVar(null)
})
Template.fetch.helpers({
source () {
return Template.instance().source.get()
},
})
Template.fetch.events({
'click #fetchbutton' (event, instance) {
Meteor.call('getAudio', 'https://www.sample-videos.com/audio/mp3/crowd-cheering.mp3', (err, uint8Array) => {
const blob = new Blob([uint8Array], {type: 'audio/mpeg'})
instance.source.set(window.URL.createObjectURL(blob))
})
},
})

Alternative solution is adding a REST endpoint *using Express) to your Meteor backend.
Instead of HTTP we use request and request-progress to send the data chunked in case of large files.
On the frontend I catch the chunks using https://angular.io/guide/http#listening-to-progress-events to show a loader and deal with the response.
I could listen to the download via
this.http.get( 'the URL to a mp3', { responseType: 'arraybuffer'} ).subscribe( ( res:any ) => {
var blob = new Blob( [res], { type: 'audio/mpeg' });
var url= window.URL.createObjectURL(blob);
window.open(url);
} );
The above example doesn't show progress by the way, you need to implement the progress-events as explained in the angular article. Happy to update the example to my final code when finished.
The Express setup on the Meteor Server:
/*
Source:http://www.mhurwi.com/meteor-with-express/
## api.class.ts
*/
import { WebApp } from 'meteor/webapp';
const express = require('express');
const trackRoute = express.Router();
const request = require('request');
const progress = require('request-progress');
export function api() {
const app = express();
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
app.use('/episodes', trackRoute);
trackRoute.get('/:url', (req, res) => {
res.set('content-type', 'audio/mp3');
res.set('accept-ranges', 'bytes');
// The options argument is optional so you can omit it
progress(request(req.params.url ), {
// throttle: 2000, // Throttle the progress event to 2000ms, defaults to 1000ms
// delay: 1000, // Only start to emit after 1000ms delay, defaults to 0ms
// lengthHeader: 'x-transfer-length' // Length header to use, defaults to content-length
})
.on('progress', function (state) {
// The state is an object that looks like this:
// {
// percent: 0.5, // Overall percent (between 0 to 1)
// speed: 554732, // The download speed in bytes/sec
// size: {
// total: 90044871, // The total payload size in bytes
// transferred: 27610959 // The transferred payload size in bytes
// },
// time: {
// elapsed: 36.235, // The total elapsed seconds since the start (3 decimals)
// remaining: 81.403 // The remaining seconds to finish (3 decimals)
// }
// }
console.log('progress', state);
})
.on('error', function (err) {
// Do something with err
})
.on('end', function () {
console.log('DONE');
// Do something after request finishes
})
.pipe(res);
});
WebApp.connectHandlers.use(app);
}
and then add this to your meteor startup:
import { Meteor } from 'meteor/meteor';
import { api } from './imports/lib/api.class';
Meteor.startup( () => {
api();
});

Related

NextJS Actioncable Proxy

So I'm trying to do two things at the same time and it's not going too well.
I have a NextJS app and a Rails API server this app connects to. For authentication I'm using a JWT token stored in an http-only encrypted cookie that the Rails API sets and the front end should not be touching. Naturally that creates a necessity for the frontend to send all the api requests though the NextJs server which proxies them to the real API.
To do that I have set up a next-http-proxy-middleware in my /pages/api/[...path] in the following way:
export const config = { api: { bodyParser: false, externalResolver: true } }
export default function handler(
req: NextApiRequest,
res: NextApiResponse
) {
httpProxyMiddleware(req, res, {
target: process.env.BACKEND_URL,
pathRewrite: [{ patternStr: "^/?api", replaceStr: "" }],
})
}
Which works great and life would be just great, but turns out I need to do the same thing with ActionCable subscriptions. Not to worry, found some handy tutorials, packed #rails/actioncable into my package list and off we go.
import {useCurrentUser} from "../../../data";
import {useEffect, useState} from "react";
const UserSocket = () => {
const { user } = useCurrentUser()
const [roomSocket, setRoomSocket] = useState<any>(null)
const loadConsumer = async () => {
// #ts-ignore
const { createConsumer } = await import("#rails/actioncable")
const newCable = createConsumer('/api/wsp')
console.log('Cable loaded')
setRoomSocket(newCable.subscriptions.create({
channel: 'RoomsChannel'
},{
connected: () => { console.log('Room Connected') },
received: (data: any) => { console.log(data) },
}))
return newCable
}
useEffect(() => {
if (typeof window !== 'undefined' && user?.id) {
console.log('Cable loading')
loadConsumer().then(() => {
console.log('Cable connected')
})
}
return () => { roomSocket?.disconnect() }
}, [typeof window, user?.id])
return <></>
}
export default UserSocket
Now when I go to load the page with that component, I get the log output all the way to Cable connected however I don't see the Room Connected part.
I tried looking at the requests made and for some reason I see 2 requests made to wsp. First is directed at the Rails backend (which means the proxy worked) but it lacks the Cookie headers and thus gets disconnected like this:
{
"type": "disconnect",
"reason": "unauthorized",
"reconnect": false
}
The second request is just shown as ws://localhost:5000/api/wsp (which is my NextJS dev server) with provisional headers and it just hangs up in pending. So neither actually connect properly to the websocket. But if I just replace the /api/wsp parameter with the actual hardcoded API address (ws://localhost:3000/wsp) it all works at once (that however would not work in production since those will be different domains).
Can anyone help me here? I might be missing something dead obvious but can't figure it out.

How to create file from /api in NextJS?

I am currently trying to create a temp file from /api/sendEmail.js with fs.mkdirSync
fs.mkdirSync(path.join(__dirname, "../../public"));
but on Vercel (where my app is running) all folders are read-only and I can't create any temp files.
Error:
ERROR
Error: EROFS: read-only file system, mkdir '/var/task/.next/server/public'
As I can see there are some questions about this but no clear answer, have any of you guys managed to do this?
Vercel allows creation of files in /tmp directory. However, there are limitations with this. https://github.com/vercel/vercel/discussions/5320
An example of /api function that writes and reads files is:
import fs from 'fs';
export default async function handler(req, res) {
const {
method,
body,
} = req
try {
switch (method) {
case 'GET': {
// read
// This line opens the file as a readable stream
const readStream = fs.createReadStream('/tmp/text.txt')
// This will wait until we know the readable stream is actually valid before piping
readStream.on('open', function () {
// This just pipes the read stream to the response object (which goes to the client)
readStream.pipe(res)
})
// This catches any errors that happen while creating the readable stream (usually invalid names)
readStream.on('error', function (err) {
res.end(err)
})
return
}
case 'POST':
// write
fs.writeFileSync('./test.txt', JSON.stringify(body))
break
default:
res.setHeader('Allow', ['GET', 'POST'])
res.status(405).end(`Method ${method} Not Allowed`)
}
// send result
return res.status(200).json({ message: 'Success' })
} catch (error) {
return res.status(500).json(error)
}
}
}
Also see: https://vercel.com/support/articles/how-can-i-use-files-in-serverless-functions

Meteor - Create a webhook

I'd like to implement a webhook with Meteor
I'm using the kadira/flowrouter Meteor plugin but no way to get POST data. Neither queryParams or params return the body of the message that I want to get.
FlowRouter.route('/my_weebhook', {
action: function(params, queryParams) {
console.log(queryParams);
console.log(params);
}
});
I do not know how to use kadira/flowrouter but you can use Meteor base package WebApp
to achieve what you need. Below is a sample code. You need to code or import something like this in your server.js
import { Meteor } from 'meteor/meteor';
import { WebApp } from 'meteor/webapp';
import url from "url";
WebApp.connectHandlers.use('/my_webhook', (req, res) => {
//Below two lines show how you can parse the query string from url if you need to
const urlParts = url.parse(req.url, true);
const someVar = urlParts.query.someVar;
//Below code shows how to get the request body if you need
let body = "";
req.on('data', Meteor.bindEnvironment(function (data) {
body += data;
}));
req.on('end', Meteor.bindEnvironment(function () {
//You can access complete request body data in the variable body here.
console.log("body : ", body);
//write code to save body in db or do whatever you want to do with body.
}));
//Below you send response from your webhook listener
res.writeHead(200);
res.end("OK");
});

Meteor method gets data as undefined

So I have this method in my component
uploadCallback (file) {
// TODO: Integrate dropbox with its SDK
// TODO: Pass the link to the editor
return new Promise(
(resolve, reject) => {
console.log('uploadCallback promise')
console.log('file', file)
const dataObject = {
file,
resolve,
reject
}
console.log('dataObject', dataObject)
Meteor.call('uploadToDropbox', dataObject, function (error, result) {
console.log('uploadToDropbox callback')
if (error) {
console.log('error', error)
}
if (result) {
console.log('result', result)
}
})
}
)
}
In my dataObject I am getting everything as needed. Here is what the console logs
uploadCallback promise
file File {name: "nodejs-2560x1440.png", lastModified: 1485410804857, lastModifiedDate: Thu Jan 26 2017 10:06:44 GMT+0400 (+04), webkitRelativePath: "", size: 1699460…}
dataObject Object {file: File}
uploadToDropbox callback
So everything seems to be ok here.
And here is my server code
import { Meteor } from 'meteor/meteor'
import Dropbox from 'dropbox'
console.log('dropbox settings', Meteor.settings.dropbox)
const dbx = new Dropbox({accessToken: Meteor.settings.dropbox.accessToken})
Meteor.methods({
'uploadToDropbox': function (dataObject) {
console.log('dataObject', dataObject)
const { file } = dataObject
console.log('file', file)
const { resolve, reject } = dataObject
console.log('resolve', resolve)
console.log('reject', reject)
dbx.filesUpload({path: '/' + file.name, contents: file})
.then(function (response) {
console.log(response)
resolve({ data: { link: 'http://dummy_image_src.com' } })
})
.catch(function (error) {
console.error(error)
reject('some error')
})
return false
}
})
The problem is here. dataObject is being passed almost empty
This is what the server logs
I20170217-11:44:36.141(4)? dataObject { file: {} }
I20170217-11:44:36.143(4)? file {}
I20170217-11:44:36.143(4)? resolve undefined
I20170217-11:44:36.144(4)? reject undefined
W20170217-11:44:36.371(4)? (STDERR) [TypeError: first argument must be a string or Buffer]
So why is this happening?
i suspect that File you're trying to pass to the method is a file handle. if true, then that's not going to work: even if the server did get that info, it has no access to your local filesystem to grab those bytes.
your solution is going to take 1 of 2 forms:
client uploads to dropbox
client reads bytes from file system into memory
client uploads bytes to dropbox
client receives some dropbox metadata about the uploaded file (e.g. location)
client calls server with that metadata info
server saves that info to db
server uploads to dropbox
client reads bytes from file system into memory
client formats that data into something that can be handled by JSON
client calls server with that JSON object
server uploads bytes to dropbox
server receives some dropbox metadata about the uploaded file (e.g. location)
server saves that info to db
which to do? it depends on which dropbox package/solution you're using and how you want to structure your app.
You are returning a promises not data, you have to wait for result and then return data.

Webhook for Mailgun POST?

I am trying to store email messages as JSON (as parsed by Mailgun) in a Mongo.Collection through a Mailgun webhook. I set up an iron-router server-side route to handle the request, but this.request.body is empty. I am using Mailgun's "Send A Sample POST" to send the request, and the POST looks fine using e.g. http://requestb.in/. I was hoping that request.body would have the data, as mentioned in How do I access HTTP POST data from meteor?. What am I doing wrong?
Router.map(function () {
this.route('insertMessage', {
where: 'server',
path: '/api/insert/message',
action: function() {
var req = this.request;
var res = this.response;
console.log(req.body);
...
I'm not sure that is the right syntax. Have you tried using Router.route ?
Router.route('insertMessage',
function () {
// NodeJS request object
var request = this.request;
// NodeJS response object
var response = this.response;
console.log("========= request: =============");
console.log(request);
// EDIT: also check out this.params object
console.log("========= this.params: =============");
console.log(this.params);
// EDIT 2: close the response. oops.
return response.end();
},
{
where: 'server',
path: '/api/insert/message'
}
);
I think the issue is that Mailgun sends a multipart POST request, e.g. it sends "fields" as well as "files" (i.e. attachments) and iron-router does not set up a body parser for multipart requests. This issue is discussed here and here on iron-router's Github Issues. I found this comment particularly helpful, and now I can parse Mailgun's sample POST properly.
To get this working, in a new Meteor project, I did
$ meteor add iron:router
$ meteor add meteorhacks:npm
In a root-level packages.json I have
{
"busboy": "0.2.9"
}
which, using the meteorhacks:npm package, makes the "busboy" npm package available for use on the server via Meteor.npmRequire.
Finally, in a server/rest-api.js I have
Router.route('/restful', {where: 'server'})
.post(function () {
var msg = this.request.body;
console.log(msg);
console.log(_.keys(msg));
this.response.end('post request\n');
});
var Busboy = Meteor.npmRequire("Busboy");
Router.onBeforeAction(function (req, res, next) {
if (req.method === "POST") {
var body = {}; // Store body fields and then pass them to request.
var busboy = new Busboy({ headers: req.headers });
busboy.on("field", function(fieldname, value) {
body[fieldname] = value;
});
busboy.on("finish", function () {
// Done parsing form
req.body = body;
next();
});
req.pipe(busboy);
}
});
In this way I can ignore files (i.e., I don't have a busboy.on("file" part) and have a this.request.body available in my routes that has all the POST fields as JSON.

Resources