I'm trying to pipe request output into clean-css to minify and then provide it as a response. The example app in question is built on restify.
Sample Objective Code :
var url = "http://www.example.com/css/style.css";
request.get(url).pipe(minify()).pipe(res);
How can I define the minify() function to accept stream and return one. I've tried dozens of packages, none worked.
Help would be much appreciated!
You can create a Transform stream. It gets the CSS code from the stream created by request.get, does the minification magic and returns the minified CSS.
Here's a working example:
const restify = require('restify');
const request = require('request');
const { Transform } = require('stream');
const CleanCSS = require('clean-css');
const server = restify.createServer();
const cleanCss = new CleanCSS();
const url = 'http://www.example.com/css/style.css';
function createMinifyTransform() {
return new Transform({
transform(chunk, encoding, callback) {
const output = cleanCss.minify(chunk.toString());
this.push(output.styles);
callback();
}
});
}
function respond(req, res) {
request
.get(url)
.pipe(createMinifyTransform())
.pipe(res);
}
server.get('/', respond);
server.listen(8080, () => console.log('%s listening at %s', server.name, server.url));
Related
My Svelte components import readable stores like this:
import { classes, locations, schedule } from 'stores.ts'
In stores.ts, I want to build the URL for fetch dynamically using page.host from $app/stores.
// Note: this is not a Svelte component; it's stores.ts
import { readable } from 'svelte/store'
import { getStores } from '$app/stores'
const { page } = getStores()
let FQDN
page.subscribe(({ host }) => {
FQDN = host
})
const getArray = async (url) => {
const response: Response = await fetch(url)
if (!response.ok) throw new Error(`Bad response trying to retrieve from ${url}.`)
return await response.json()
}
const getReadableStore = (url: string) => readable([], set => {
getArray(`http://${FQDN}${url}`)
.then(set)
.catch(err => console.error('Failed API call:', err))
return () => {}
})
export const classes = getReadableStore('/api/class/public.json')
export const locations = getReadableStore('/api/location/public.json')
export const schedule = getReadableStore('/api/schedule/public.json')
The sixth line throws this error...
Error: Function called outside component initialization
at get_current_component (/Users/nates/dev/shy-svelte/node_modules/svelte/internal/index.js:652:15)
at Proxy.getContext (/Users/nates/dev/shy-svelte/node_modules/svelte/internal/index.js:685:12)
at Module.getStores (/.svelte-kit/dev/runtime/app/stores.js:17:26)
at eval (/src/stores.ts:6:38)
at instantiateModule (/Users/nates/dev/shy-svelte/node_modules/#sveltejs/kit/node_modules/vite/dist/node/chunks/dep-e9a16784.js:68197:166)
Two questions...
What is the correct way to get page values from $app/stores outside of the context of a component? Is this possible? Answer from below: No, this is not possible outside the context of a component.
If I'm accessing a SvelteKit site, let's say http://localhost:3000/something or https://example.com and a Svelte component loads a readable store from stores.ts, is there a way in stores.ts to determine whether the original page request that loaded the component (which loaded from stores.ts) was http or https? Answer from below: No, this is not possible in stores.ts - only from a component.
UPDATE: Based on the feedback, I'm going to set a value in my .env called VITE_WEB_URL=http://localhost:3000 and change it for the production system. This cuts down on the number of lines of code and may be a better practice (comments welcome)...
// revised stores.ts
import { readable } from 'svelte/store'
const { VITE_WEB_URL } = import.meta.env
const getArray = async (url) => {
const response: Response = await fetch(url)
if (!response.ok) throw new Error(`Bad response trying to retrieve from ${url}.`)
return await response.json()
}
const getReadableStore = (url: string) => readable([], set => {
getArray(`${VITE_WEB_URL}${url}`)
.then(set)
.catch(err => console.error('Failed API call:', err))
return () => {}
})
export const classes = getReadableStore('/api/class/public.json')
export const locations = getReadableStore('/api/location/public.json')
export const schedule = getReadableStore('/api/schedule/public.json')
Extract from https://kit.svelte.dev/docs#modules-$app-stores
Because of that, the stores are not free-floating objects: they must be accessed during component initialisation, like anything else that would be accessed with getContext.
Therefore, since the readable store is bound to the context of a svelte component, I suggest you subscribe either way ($ or .subscribe) inside the component of the SvelteKit website and then send the protocol value (http or https) as parameter when it updates so that stores.ts stores it in a variable.
However, it looks like SvelteKit does not provide the protocol value, so parse the client side window.location.href in the page subscription and then send it.
Referencing a svelte store can be done everywhere.
Using the $: shorthand syntax, however, only works within a component.
$: BASE = `http://${$page.host}`
SvelteKit appears to delegate this to fetch indeed
I am trying to make a call within a firebase function to a locally managed server. I am not super familiar with node as a development environment so I am not sure what is the issue.
const functions = require('firebase-functions');
const https = require('http');
exports.testPost = functions.https.onRequest((req, res) => {
var options = {
host: 'localdevserver.edu',
port: 80,
path: '/my/endpoint'
};
let data = '';
http.get(options, function(resp){
resp.on('data', function(chunk){
//do something with chunk
data += chunk;
resp.on('end', console.log("dones"));
});
}).on("error", function(e){
console.log("Got error: " + e.message);
});
});
When I look in the Firebase Functions Log, it says either timeout or no reject defined.
With HTTP type functions, you need to send a response to the client in order to terminate the function. Otherwise it will time out.
res.send("OK");
Please read the documentation for more details.
You can use SYNC-REQUEST
npm install sync-request
var request = require('sync-request');
var res = request('GET', 'http://google.com');
console.log(res.body.toString('utf-8'));
the function would be something like this:
exports.testPost = functions.https.onRequest((req, res) => {
var request = require('sync-request');
var res = request('GET', 'http://google.com');
var res = res.body.toString('utf-8');
resp.on(res, console.log("dones"));
});
am playing around with downloading and serving mp3 files in Meteor.
I am trying to download an MP3 file (https://www.sample-videos.com/audio/mp3/crowd-cheering.mp3) on my MeteorJS Server side (to circumvent CORS issues) and then pass it back to the client to play it in a AUDIO tag.
In Meteor you use the Meteor.call function to call a server method. There is not much to configure, it's just a method call and a callback.
When I run the method I receive this:
content:
"ID3���#K `� )�<H� e0�)������1������J}��e����2L����������fȹ\�CO��ȹ'�����}$A�Lݓ����3D/����fijw��+�LF�$?��`R�l�YA:A��# �0��pq����4�.W"�P���2.Iƭ5��_I�d7d����L� �p0��0A��cA�xc��ٲR�BL8䝠4���T��..etc..", data:null,
headers: {
accept-ranges:"bytes",
connection:"close",
content-length:"443926",
content-type:"audio/mpeg",
date:"Mon, 20 Aug 2018 13:36:11 GMT",
last-modified:"Fri, 17 Jun 2016 18:16:53 GMT",
server:"Apache",
statusCode:200
which is the working Mp3 file (the content-length is exactly the same as the file I write to disk on the MeteorJS Server side, and it is playable).
However, my following code doesn't let me convert the response into a BLOB:
```
MeteorObservable.call( 'episode.download', episode.url.url ).subscribe( ( result: any )=> {
console.log( 'response', result);
let URL = window.URL;
let blob = new Blob([ result.content ], {type: 'audio/mpeg'} );
console.log('blob', blob);
let audioUrl = URL.createObjectURL(blob);
let audioElement:any = document.getElementsByTagName('audio')[0];
audioElement.setAttribute("src", audioUrl);
audioElement.play();
})
When I run the code, the Blob has the wrong size and is not playable
Blob(769806) {size: 769806, type: "audio/mpeg"}
size:769806
type:"audio/mpeg"
__proto__:Blob
Uncaught (in promise) DOMException: Failed to load because no supported source was found.
On the backend I just run a return HTTP.get( url ); in the method which is using import { HTTP } from 'meteor/http'.
I have been trying to use btoa or atob but that doesn't work and as far as I know it is already a base64 encoded file, right?
I am not sure why the Blob constructor creates a larger file then the source returned from the backend. And I am not sure why it is not playing.
Can anyone point me to the right direction?
Finally found a solution that uses request instead of Meteor's HTTP:
First you need to install request and request-promise-native in order to make it easy to return your result to clients.
$ meteor npm install --save request request-promise-native
Now you just return the promise of the request in a Meteor method:
server/request.js
import { Meteor } from 'meteor/meteor'
import request from 'request-promise-native'
Meteor.methods({
getAudio (url) {
return request.get({url, encoding: null})
}
})
Notice the encoding: null flag, which causes the result to be binary. I found this in a comment of an answer related to downloading binary data via node. This causes not to use string but binary representation of the data (I don't know how but maybe it is a fallback that uses Node Buffer).
Now it gets interesting. On your client you wont receive a complex result anymore but either an Error or a Uint8Array which makes sense because Meteor uses EJSON to send data over the wires with DDP and the representation of binary data is a Uint8Array as described in the documentation.
Because you can just pass in a Uint8Array into a Blob you can now easily create the blob like so:
const blob = new Blob([utf8Array], {type: 'audio/mpeg'})
Summarizing all this into a small template if could look like this:
client/fetch.html
<template name="fetch">
<button id="fetchbutton">Fetch Mp3</button>
{{#if source}}
<audio id="player" src={{source}} preload="none" content="audio/mpeg" controls></audio>
{{/if}}
</template>
client/fetch.js
import { Template } from 'meteor/templating'
import { ReactiveVar } from 'meteor/reactive-var'
import './fetch.html'
Template.fetch.onCreated(function helloOnCreated () {
// counter starts at 0
this.source = new ReactiveVar(null)
})
Template.fetch.helpers({
source () {
return Template.instance().source.get()
},
})
Template.fetch.events({
'click #fetchbutton' (event, instance) {
Meteor.call('getAudio', 'https://www.sample-videos.com/audio/mp3/crowd-cheering.mp3', (err, uint8Array) => {
const blob = new Blob([uint8Array], {type: 'audio/mpeg'})
instance.source.set(window.URL.createObjectURL(blob))
})
},
})
Alternative solution is adding a REST endpoint *using Express) to your Meteor backend.
Instead of HTTP we use request and request-progress to send the data chunked in case of large files.
On the frontend I catch the chunks using https://angular.io/guide/http#listening-to-progress-events to show a loader and deal with the response.
I could listen to the download via
this.http.get( 'the URL to a mp3', { responseType: 'arraybuffer'} ).subscribe( ( res:any ) => {
var blob = new Blob( [res], { type: 'audio/mpeg' });
var url= window.URL.createObjectURL(blob);
window.open(url);
} );
The above example doesn't show progress by the way, you need to implement the progress-events as explained in the angular article. Happy to update the example to my final code when finished.
The Express setup on the Meteor Server:
/*
Source:http://www.mhurwi.com/meteor-with-express/
## api.class.ts
*/
import { WebApp } from 'meteor/webapp';
const express = require('express');
const trackRoute = express.Router();
const request = require('request');
const progress = require('request-progress');
export function api() {
const app = express();
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
app.use('/episodes', trackRoute);
trackRoute.get('/:url', (req, res) => {
res.set('content-type', 'audio/mp3');
res.set('accept-ranges', 'bytes');
// The options argument is optional so you can omit it
progress(request(req.params.url ), {
// throttle: 2000, // Throttle the progress event to 2000ms, defaults to 1000ms
// delay: 1000, // Only start to emit after 1000ms delay, defaults to 0ms
// lengthHeader: 'x-transfer-length' // Length header to use, defaults to content-length
})
.on('progress', function (state) {
// The state is an object that looks like this:
// {
// percent: 0.5, // Overall percent (between 0 to 1)
// speed: 554732, // The download speed in bytes/sec
// size: {
// total: 90044871, // The total payload size in bytes
// transferred: 27610959 // The transferred payload size in bytes
// },
// time: {
// elapsed: 36.235, // The total elapsed seconds since the start (3 decimals)
// remaining: 81.403 // The remaining seconds to finish (3 decimals)
// }
// }
console.log('progress', state);
})
.on('error', function (err) {
// Do something with err
})
.on('end', function () {
console.log('DONE');
// Do something after request finishes
})
.pipe(res);
});
WebApp.connectHandlers.use(app);
}
and then add this to your meteor startup:
import { Meteor } from 'meteor/meteor';
import { api } from './imports/lib/api.class';
Meteor.startup( () => {
api();
});
I'd like to implement a webhook with Meteor
I'm using the kadira/flowrouter Meteor plugin but no way to get POST data. Neither queryParams or params return the body of the message that I want to get.
FlowRouter.route('/my_weebhook', {
action: function(params, queryParams) {
console.log(queryParams);
console.log(params);
}
});
I do not know how to use kadira/flowrouter but you can use Meteor base package WebApp
to achieve what you need. Below is a sample code. You need to code or import something like this in your server.js
import { Meteor } from 'meteor/meteor';
import { WebApp } from 'meteor/webapp';
import url from "url";
WebApp.connectHandlers.use('/my_webhook', (req, res) => {
//Below two lines show how you can parse the query string from url if you need to
const urlParts = url.parse(req.url, true);
const someVar = urlParts.query.someVar;
//Below code shows how to get the request body if you need
let body = "";
req.on('data', Meteor.bindEnvironment(function (data) {
body += data;
}));
req.on('end', Meteor.bindEnvironment(function () {
//You can access complete request body data in the variable body here.
console.log("body : ", body);
//write code to save body in db or do whatever you want to do with body.
}));
//Below you send response from your webhook listener
res.writeHead(200);
res.end("OK");
});
I was wondering how I might achieve the following using Ironrouter in meteorjs:
app.route("/api/tts").get(function(req,res){
res.type('audio/mpeg');
var text = req.query.q;
var request = require('request');
var url = "https://translate.google.pl/translate_tts?ie=UTF-8&q=" + text + "&tl=en&total=1&idx=0&client=t&prev=input";
request.get(url).pipe(res);
});
If you have iron:router installed already, then you already can. All you need to do is install request using meteorhacks:npm.
Then you simply write:
Router.route("/api/tts", function () {
// NodeJS request object
var req = this.request;
// NodeJS response object
var res = this.response;
res.type('audio/mpeg');
var text = req.query.q;
var request = Meteor.npmRequire('request');
var url = "https://translate.google.pl/translate_tts?ie=UTF-8&q=" + text + "&tl=en&total=1&idx=0&client=t&prev=input";
request.get(url).pipe(res);
}, { where: 'server' });
Let me know if that works.
You can't use IronRouter, Meteor routing is done on the client
(the answer from #rclai won't work because the request is still being sent from the client..)
This solution using the WebApp module shipped with Meteor to define server routes is exactly what you need.
e.g. something like this:
import { WebApp } from 'meteor/webapp';
WebApp.connectHandlers.use('/api/tts', (req, res, next) => {
var text = res.query.q;
var url = "https://translate.google.pl/translate_tts?ie=UTF-8&q=" + text + "&tl=en&total=1&idx=0&client=t&prev=input";
HTTP.call("GET", url, {}, function(err, response){
if(err){
res.writeHead(500);
res.end('Failed...');
}
else {
res.end(response.content);
}
});
});