How to list files in folder - meteor

How can I list all files inside a folder with Meteor.I have FS collection and cfs:filesystem installed on my app. I didn't find it in the doc.

Another way of doing this is by adding the shelljs npm module.
To add npm modules see: https://github.com/meteorhacks/npm
Then you just need to do something like:
var shell = Meteor.npmRequire('shelljs');
var list = shell.ls('/yourfolder');
Shelljs docs:
https://github.com/arturadib/shelljs

The short answer is that FS.Collection creates a Mongo collection that you can treat like any other, i.e., you can list entries using find().
The long answer...
Using cfs:filesystem, you can create a mongo database that mirrors a given folder on the server, like so:
// in lib/files.js
files = new FS.Collection("my_files", {
stores: [new FS.Store.FileSystem("my_files", {"~/test"})] // creates a ~/test folder at the home directory of your server and will put files there on insert
});
You can then access this collection on the client to upload files to the server to the ~test/ directory:
files.insert(new File(['Test file contents'], 'my_test_file'));
And then you can list the files on the server like so:
files.find(); // returns [ { createdByTransform: true,
_id: 't6NoXZZdx6hmJDEQh',
original:
{ name: 'my_test_file',
updatedAt: (Date)
size: (N),
type: '' },
uploadedAt: (Date),
copies: { my_files: [Object] },
collectionName: 'my_files'
}
The copies object appears to contain the actual names of the files created, e.g.,
files.findOne().copies
{
"my_files" : {
"name" : "testy1",
"type" : "",
"size" : 6,
"key" : "my_files-t6NoXZZdx6hmJDEQh-my_test_file", // This is the name of the file on the server at ~/test/
"updatedAt" : ISODate("2015-03-29T16:53:33Z"),
"createdAt" : ISODate("2015-03-29T16:53:33Z")
}
}
The problem with this approach is that it only tracks the changes made through the Collection; if you add something manually to the ~/test directory, it won't get mirrored into the Collection. For instance, if on the server I run something like...
mkfile 1k ~/test/my_files-aaaaaaaaaa-manually-created
Then I look for it in the collection, it won't be there:
files.findOne({"original.name": {$regex: ".*manually.*"}}) // returns undefined
If you just want a straightforward list of files on the server, you might consider just running an ls. From https://gentlenode.com/journal/meteor-14-execute-a-unix-command/33 you can execute any arbitrary UNIX command using Node's child_process.exec(). You can access the app root directory with process.env.PWD (from this question). So in the end if you wanted to list all the files in your public directory, for instance, you might do something like this:
exec = Npm.require('child_process').exec;
console.log("This is the root dir:");
console.log(process.env.PWD); // running from localhost returns: /Users/me/meteor_apps/test
child = exec('ls -la ' + process.env.PWD + '/public', function(error, stdout, stderr) {
// Fill in this callback with whatever you actually want to do with the information
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
if(error !== null) {
console.log('exec error: ' + error);
}
});
This will have to run on the server, so if you want the information on the client, you'll have to put it in a method. This is also pretty insecure, depending on how you structure it, so you'd want to think about how to stop people from listing all the files and folders on your server, or worse -- running arbitrary execs.
Which method you choose probably depends on what you're really trying to accomplish.

Related

Download a file from a URL to a user accessible location

I am building an app using Nativescript/Angular 2
I want to be able to download a file from a URL and save it to the device in a location the average user would have no problems finding it. I believe downloads would be the best place for this on both iOS and Android. Please correct me if I am wrong.
The file can be any file type, not just an image. So mainly spreadsheet, word document, pdf, png, jpg, etc.
I have searched online and through the documentation. The documentation describes a method called getFile which gets a file and saves it to your device.
I have implemented this in my code as follows:
download (id) {
console.log('Download Started');
getFile("https://raw.githubusercontent.com/NativeScript/NativeScript/master/apps/tests/logo.png").then(function (r) {
console.log(r.path);
}, function (e) {
//// Argument (e) is Error!
});
}
The problem with this is that it saves it to a non-user accessible location such as:
/data/user/0/com.myapp.example/files/logo.png
Update:
I have also tried specifying the path directly with:
fs.knownFolders.documents();
However, this method gets the documents folder for the current application that is NOT accessible by the user or external applications
After some unsuccessful attempts, I finally found how to save file to user "Downloads" folder (something like sdcard/Download). You can use android.os.Environment method to get this folder.
Add this in your component:
import { getFile } from 'tns-core-modules/http';
import * as fileSystem from "tns-core-modules/file-system";
import { isAndroid } from "tns-core-modules/platform";
import { alert } from "tns-core-modules/ui/dialogs";
declare var android;
<...>
public download (url, fileName) {
if (isAndroid) {
const permissions = require("nativescript-permissions");
permissions.requestPermission(android.Manifest.permission.WRITE_EXTERNAL_STORAGE, "I need these permissions because I'm cool")
.then(() => {
let downloadedFilePath = fileSystem.path.join(android.os.Environment.getExternalStoragePublicDirectory(android.os.Environment.DIRECTORY_DOWNLOADS).getAbsolutePath(), fileName);
getFile(url, downloadedFilePath).then(resultFile => {
alert({
title: 'Saved!',
okButtonText: 'OK',
message: `File saved here:\n${resultFile.path}`
});
}, error => {
alert({
title: 'Error',
okButtonText: 'OK',
message: `${error}`
});
});
});
}
}
What else you should know:
1) There is no any kind of download indicator, standard system download bar also not appears, and I don't know how to solve this.
2) For iOS you may try to use
const filePath = fileSystem.path.join(fileSystem.knownFolders.ios.downloads().path, fileName);
getFile(url, filePath).then((resultFile) => {}, (error) => {});
I think, it's the shame that NS docs don't talk straight, that you can't save files in user accessible location only with NS functions. I figured it out only when I read comments in file /node_modules/tns-core-modules/file-system/file-system.d.ts
Hope this helps you.
To get it working on iPhone, you can do the following (TypeScript):
import { knownFolders, path } from "tns-core-modules/file-system";
let destination = path.join(knownFolders.documents(), "file_name.txt");
// logic to save your file here ...
// the important thing is that you have to save your file in knownFolders.documents()
Then in Info.plist, you have to add the following permissions:
<key>LSSupportsOpeningDocumentsInPlace</key>
<true/>
<key>UIFileSharingEnabled</key>
<true/>
Now if you go to your iPhone's Files app > On My iPhone > Your App's Name, you should see the file there.
Basically, the Documents folder is a private folder inside your application's directory that only you can see. However, when you enable the two permissions above, it allows file sharing so that your user can access the folder and its contents.
The same documentation says that you can specify the file location like this:
download (id) {
console.log('Download Started');
var folder = fs.knownFolders.documents();
var file = fs.path.join(folder.path, "logo.png");
var url = "https://raw.githubusercontent.com/NativeScript/NativeScript/master/apps/tests/logo.png"
getFile(url, file).then(function (r) {
console.log(r.path);
}, function (e) {
//// Argument (e) is Error!
});
}
disclaimer: never tried it myself, just read the docs ...
You can specify a filesystem path directly, like this:
var folder = fs.Folder.fromPath('/sdcard/Download');
Note that /sdcard/Download will only work on Android; you can replace it with whatever (publicly accessible) folder you want to save your data to.
There doesn't yet seem to be a cross-platform way to choose a folder path, so you'll have to work out something manually. See this GitHub thread for more.
I realize that this is an older thread, but perhaps this can help someone:
If you use currentApp(), instead of documents(), you can access the folder you need. For example:
var directories = fs.knownFolders.currentApp();
var folder = directories.getFolder('./nameofaccessiblefolder');
I know this thread is 3 years ago but in case you have the same issue, I hope this solution will save time for you.
I solved the same issue by adding android:requestLegacyExternalStorage="true" inside the AndroidManifest.xml file
follow the thread here

How do I upload a image to my server with meteor?

I have a logo somewhere on my application page.
The application admin should be able to adjust the logo within the webapplication by simply upload a new one. What would be the best practice to achieve this?
How would I handle the upload on the server. It should replace the old logo with the new one. The name and location should stay the same.
Here is my approach:
I use the package UploadFS:
jalik:ufs
jalik:ufs-local
autopublish //it is still on, so the code below works without publish/subscribe I know that I will have to change that.
My code:
Upload
*.js Server & Client
//Almost Standard initialization - works so far
Logo = new Mongo.Collection('logo');
LogoStore = new UploadFS.store.Local({
collection: Logo,
name: 'logo',
path: '/uploads/logo',
mode: '0744', // directory permissions
writeMode: '0744', // file permissions
filter: new UploadFS.Filter({
minSize: 1,
maxSize: 1024 * 1000, // 1MB,
contentTypes: ['image/*'],
extensions: ['png']
})
});
*.html
//Standard initialization - works so far
<template name="upload">
<button type="button" name="upload">Select files</button>
</template>
*.js Client
//Almost Standard initialization - works so far
Template.upload.events({
'click button[name=upload]': function (ev) {
var self = this;
UploadFS.selectFiles(function (file) {
// Prepare the file to insert in database, note that we don't provide an URL,
// it will be set automatically by the uploader when file transfer is complete.
var logo = {
name: 'logo.png', //all uploaded images will have the same name
size: file.size,
type: file.type,
};
// Create a new Uploader for this file
var uploader = new UploadFS.Uploader({
// This is where the uploader will save the file
store: LogoStore,
// Optimize speed transfer by increasing/decreasing chunk size automatically
adaptive: true,
// Define the upload capacity (if upload speed is 1MB/s, then it will try to maintain upload at 80%, so 800KB/s)
// (used only if adaptive = true)
capacity: 0.8, // 80%
// The size of each chunk sent to the server
chunkSize: 8 * 1024, // 8k
// The max chunk size (used only if adaptive = true)
maxChunkSize: 128 * 1024, // 128k
// This tells how many tries to do if an error occurs during upload
maxTries: 5,
// The File/Blob object containing the data
data: file,
// The document to save in the collection
file: logo,
// The error callback
onError: function (err) {
console.error(err);
},
onAbort: function (file) {
console.log(file.name + ' upload has been aborted');
},
onComplete: function (file) {
console.log(file.name + ' has been uploaded');
},
onCreate: function (file) {
console.log(file.name + ' has been created with ID ' + file._id);
},
onProgress: function (file, progress) {
console.log(file.name + ' ' + (progress*100) + '% uploaded');
},
onStart: function (file) {
console.log(file.name + ' started');
},
onStop: function (file) {
console.log(file.name + ' stopped');
}
});
// Starts the upload
uploader.start();
// Stops the upload
uploader.stop();
// Abort the upload
uploader.abort();
});
}
});
Show uploaded Logo
*.html
<template name="whatever">
<img src="{{logoUrl}}" alt="Logo" >
</template>
*.js client only
Template.whatever.helpers({
logoUrl: function(){
return Logo.findOne().url;
}
})
So If I understand it right, what the code does is uploading the img to somewhere on the server. Also it stores some info about that image in a Mongo.Collection - Logo.
But I do not exactly know where those images are stored, in which folder. They are not stored in my default Project - Folder.
The url of an example img is: http://localhost:3000/ufs/logo/B4Fv5etkr7xQbvs5v/logo.png. That random string in the middle is the _id of that img. So I can not use a hardcoded url for that images to access them, because as soon as a new img is uploaded, that url will change completely.
Q1: So the first question is: Can I upload to the myProject/public/img folder directly? So that the url of the img would be something like:
http://localhost:3000/img/logo.png
Then I would need just to replace the old logo on the upload.
For now I have to deal with the generic url. So as next, I select the url of the now present image on the server from the Logo - collection and pass that url to my template to the place where the logo has to be placed. The problem with that is, that the url is loaded after everything else was loaded, so for several seconds I get an tag without an url in it. So that place shows the alt text only until the url is loaded. That is very ugly...
Q2: The question is, how could I get the url, before the tag is loaded. So that the logo appears with/before everything else, as if the url would be hardcoded in advance.
Q3: Is it possible to replace the old logo with the new uploaded one, on the upload? How?
Q4: If I delete the entry for the img from the Logo - Collection, is the image actually deleted from the server? Or do I have to delete it manually/in another way?
You can send a base64 encode image on server then using fs your can overwrite the file.
Like:
Client
readAsDataURL has base64 encoded data in the format of
data:image/jpeg;base64,/9j/4AAQSkZJRgABA...
So you need to get rid of the mime type and encoding information at the front.
contents = contents.split(',')[1];
Now you can send this base64 encoded data to server.
Server
Since you're receiving base64 encoded data, you can convert it buffer and write to file:
fs.writeFile(filepath, Buffer(argument,'base64'), err => {
//
})
In case the file name same as another file then it will completely overwrite your file.
Answer for question 1:
default image will be stored hide folder in your project .meteor/local/build/programs/server/ufs/uploads/
You can change destination by "path" as below code
new UploadFS.store.Local({
collection: Csvs.collection,
name: 'csv',
path: '../../../../../uploads/csv', //here change destination folder stored file
filter: new UploadFS.Filter({
maxSize: 1024 * 3000, // 3MB,
contentTypes: ['text/csv']
})
});
When doing the "Client" step in Pankaj Javav's answer, you may want to use the base64-image-upload package I whipped up, as I was having the same problem. It simplifies the process of uploading any base64 string to a server, and you do not need to get rid of the MIME type this way.

How to change ip to current url from subject in reset password email meteor

I have used the following code to set my reset email subject:
Accounts.emailTemplates.resetPassword.subject = function(user, url) {
var ul = Meteor.absoluteUrl();
var myArray = ul.split("//");
var array = myArray[1].split('/');
return "How to reset your password on "+array[0];
};
I want it to contain the current browser's url, but it's not happening.
This is what the subject looks like
How to reset your password on 139.59.9.214
but the desired outcome is:
How to reset your password on someName.com
where someName.com is my URL.
I would recommend handling this a bit differently. Your host name is tied to your environment, and depending on what your production environment looks like, deriving your hostname from the server might not always be the easiest thing to do (especially if you're behind proxies, load balancers, etc.). You could instead look into leveraging Meteor's Meteor.settings functionality, and create a settings file for each environment with a matching hostname setting. For example:
1) Create a settings_local.json file with the following contents:
{
"private": {
"hostname": "localhost:3000"
}
}
2) Create a settings.json file with the following contents:
{
"private": {
"hostname": "somename.com"
}
}
3) Adjust your code to look like:
Accounts.emailTemplates.resetPassword.subject = function (user, url) {
const hostname = Meteor.settings.private.hostname;
return `How to reset your password on ${hostname}`;
};
4) When working locally, start meteor like:
meteor --settings=settings_local.json
5) When deploying to production, make sure the contents or your settings.json file are taken into consideration. How you do this depends on how you're deploying to your prod environment. If using mup for example, it will automatically look for a settings.json to use in production. MDG's Galaxy will do the same.

How to save a file -in a folder- using FileSystem and CollectionFS ? (yeah, really.)

I think I'm missing something. I have read a lot of posts/examples and I can't save images on my system (I work locally).
What is my goal ?
I'm trying to save a file submitted by the user in a folder (server-side). Does it sound easy ? Maybe.
What's the issue ?
Short answer : I can't figure out how to save the file in my folder.
Do you want more information ?
The story of a file upload
I have read that to use the path parameter like new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" }) , I have to declare my collection server-side. But when I call Avatars.insert() (Avatars is the name of my collection), it seems like it doesn't exists. This makes sense because this collection exists only on the server.
So I've tried to declare the collection both server-side and client-side (I've read some examples about that) and that works ! The file is correctly added to MongoDB, but my folder is still empty (I'm not sure but I think this is because Avatars.insert() is called client-side so the collection used is the client-side one, the one which cannot take path parameter).
But no problem ! I've created 2 Meteor methods (one client-side and one server-side) called "updateAvatarFile". With this "trick", I'm able to do Meteor.call("updateAvatarFile", field.files[0]), which calls both server-side and client-side methods. So I can do some UI stuff in the client-side one and upload the file in the other. But I can't pass the file as a parameter.
field.files[0] contains the file client-side but server-side it's an empty object. My question is : How can I upload a file ?
I can't do it client-side (because I can't use path parameter) but I can pass the file to the server. I'm sure that I'm missing something but I can't figure what.
Here is how I go :
// /client/views/templates/settings.js
Template.settings.events({
'submit #updateAvatar': function (e, template) {
e.preventDefault();
const field = document.getElementsByName('avatar')[0];
Meteor.call('updateAvatarFile', field.files[0]);
}
});
// /client/lib/clientMethods.js
Meteor.methods({
'updateAvatarFile': function (file) {
// blabla
}
});
// /server/lib/serverMethods.js
Meteor.methods({
'updateAvatarFile': function (file) {
Avatars.insert(file, function (err, fileObj) {
if (err) {
console.log(err);
} else {
console.log(fileObj);
}
});
}
});
// /server/collections/serverAvatarCollection.js
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original", { path: "/public/images/user/avatar" }),
new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" })
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
//throw new Meteor.Error(403, message);
}
});
// /client/collections/clientAvatarCollection.js
// (this one is actually in a comment block)
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original"),
new FS.Store.FileSystem("thumb")
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
alert(message);
}
});
I've also tried to insert the file with the client-side method but I've got the same result (the file is added to MongoDB but not saved into a folder).
Using different path values didn't work either.
EDIT : Or maybe I'm trying to use the wrong package ? To my mind, transform a picture to chunks and save them into MongoDB sounds really weird and bad. Do you have any adivces ?
EDIT 2 :
answer to Michel Floyd (sorry about that, the character limit is annoying).
First, thanks for your answer !
1. At the moment, I'm just trying Meteor so I have both autopublish and insecure installed. Not publishing/subscribing to my collection cannot cause an issue, is that right ?
2. Before your answer I've tried to set up a collection available for both server and client by putting my avatarCollection.js in /collections. I was thinking that path which doesn't contains server or client are automatically available for the two sides. So what is the difference between /collections and /lib ? (I know that all files in a "lib" folder are loaded first). Is it a bad practice to put collections in /collections ? Maybe should I create a /lib/collections folder ?
3. (the last point, sorry for the long comment) I've tried what you advised above but it doesn't seems to work (or I am doing something wrong, again ><). When I use Avatars.insert(), CollectionFS don't save the file on my local storage. I've also checked the root of my HDD in case CollectionFS interpreted / to be the root of my machine but it doesn't. In the other hand, CollectionFS have created 4 collections in MongoDB (cfs._tempstore.chunks, cfs.avatars.filerecord, cfs_gridfs._tempstore.chunks and cfs_gridfs._tempstore.files) - the gridfs is weird. I have GridFS installed but I use FileSystem -. Those tables are not empty. That's why I think CollectionFS split my file into chunks and save them in MongoDB.
You're generally on the right track. CollectionFS uses storage adapters to deal with actual file storage. You can put files on S3, gridFS, or your local file system as you're trying to do. Putting the file contents in Mongo directly is usually avoided.
Firstly, define your collection:
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original", { path: "/public/images/user/avatar" }),
new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" })
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
//throw new Meteor.Error(403, message);
}
});
in /lib! This will make it available to both the server and the client.
Secondly, make sure you publish your avatars collection from the server and subscribe to it from the client. I don't see any publish/subscribe code in your question. You need it.
Thirdly, if you just do:
Avatars.insert(...);
on the client with a file then CollectionFS then CollectionFS will take care of storing it for you. The thing is, it won't be instantly available. It can take a little while for the actual upload and storage to happen. You can look at fileObj.isUploaded for example to see if the file is ready.

Using mongofiles with GridFS in a meteor app

I am starting to use GridFS within a Meteor app. I have set up the file collection "assetFiles" with a GridFS storage adapter like this :
AssetCollection = new Mongo.Collection( "assets" );
AssetFileStore = new FS.Store.GridFS( "assetFiles" );
AssetFilesCollection = new FS.Collection( "assetFiles", {
stores: [AssetFileStore]
});
AssetFilesCollection.allow({
insert: function(){
return true;
},
update: function(){
return true;
},
remove: function(){
return true;
},
download: function(){
return true;
}
});
I have inserted some files in it and using meteor mongo client checked they actually exist in db.
Now I would like to extract a file from this db to my file system using mongofiles utility.
using the meteor mongodb database, here is the list of collections :
meteor:PRIMARY> show collections
assets
cfs._tempstore.chunks
cfs.assetFiles.filerecord
cfs_gridfs._tempstore.chunks
cfs_gridfs._tempstore.files
cfs_gridfs.assetFiles.chunks
cfs_gridfs.assetFiles.files
meteor_accounts_loginServiceConfiguration
system.indexes
users
And I don't understand how with the mongofiles utility I could target my assetFiles GridFS file collection to get a particular file or even a list of files.
Here is my attempt:
./mongofiles -h 127.0.0.1:3001 -d meteor list
2015-05-11T17:34:40.701+0200 connected to: 127.0.0.1:3001
It just returns nothing while successfully connecting to db.
My db is on my own FS. I wanted to specify a collection name, but this parameter does not exists anymore apparently.
Thank you for your help!
you need to change the prefix to attach to the right collection.
$ mongofiles --help
....
--prefix= GridFS prefix to use (default is 'fs')`
eg.
mongofiles --port 3001 -d meteor --prefix 'cfs_gridfs.assetFiles' list
hope this helps!
accept please thanks!

Resources