Am trying to limit my uploads to amazon s3 to 10 pictures, i came across this link
https://github.com/themeteorchef/uploading-files-to-amazon-s3/blob/master/code/server/slingshot.js
for some reason it doesn't work with me
this is my code
Slingshot.createDirective( "uploadToAmazonS3Cg2", Slingshot.S3Storage, {
bucket: "bucket-name",
region: 'ap-southeast-1',
acl: "public-read",
authorize: function () {
return true;
},
key: function ( file ) {
var user = Meteor.users.findOne( this.userId );
return user.emails[0].address + "/screenshots" + "/" + file.name;
}
});
here is my uploader in the html file
Application ScreenShots:
{{> uploader config="2"}}
and this is how am calling my slingshot method based on the link i attached above
var uploader
if (config === '1') {
uploader = new Slingshot.Upload( "uploadToAmazonS3Cg1" );
}
if (config === '2') {
uploader = new Slingshot.Upload( "uploadToAmazonS3Cg2" );
} else {
uploader = new Slingshot.Upload( "uploadToAmazonS3Cg3" );
}
i returned true, from my understanding, it should allow me to upload as many as i want but i only can upload one file.
am i missing something here? are there any alternatives to set the limitation?
Taking a look at the source-code for slingshot, you need to upload the files one at a time. To limit the number of files uploaded in total by a user, you'd need to use the authorize function and store how many files they've uploaded in Mongo. I'm not sure if you mean 10 files total or 10 at a time based on your question, but if it was 10 files at a time you'd simply validate that using jQuery validation on your file input element.
Related
Project:
I am working on an E-commerce application and it has more than 1,600 products and 156 categories.
Problem:
Initially, on the first product page, 30 products will be fetched (due to the page limitation), but on the left sidebar, I need filters that will be decided on the basis of tags of all 1,600 products. So that's why I need all the products in the first fetch and then I will extract common tags by looping over all the products and immediately show them on the sidebar.
What do I want?
I am not sure but I think it would be the best solution if I generate a JSON file containing all the products and store it somewhere, where I can fetch just hitting the URL using REST API in Next JS (either in getServerSideProps or getStaticProps).
Caveat:
I tried by storing JSON file in ./public directory in next js application, it worked in localhost but not in vercel.
Here is the code I wrote for storing JSON file in ./public directory:
fs.writeFileSync("./public/products.json", JSON.stringify(products, null, 2)); //all 1,600 products
One solution it to fetch it directly from front-end (if the file is not too big) otherwise, for reading the file in getServerSideProps you will need a custom webpack configuration.
//next.config.js
const path = require("path")
const CopyPlugin = require("copy-webpack-plugin")
module.exports = {
target: "serverless",
future: {
webpack5: true,
},
webpack: function (config, { dev, isServer }) {
// Fixes npm packages that depend on `fs` module
if (!isServer) {
config.resolve.fallback.fs = false
}
// copy files you're interested in
if (!dev) {
config.plugins.push(
new CopyPlugin({
patterns: [{ from: "content", to: "content" }],
})
)
}
return config
},
}
Then you can create a utility function to get the file:
export async function getStaticFile(file) {
let basePath = process.cwd()
if (process.env.NODE_ENV === "production") {
basePath = path.join(process.cwd(), ".next/server/chunks")
}
const filePath = path.join(basePath, `file`)
const fileContent = await fs.readFile(filePath, "utf8")
return fileContent
}
There is an open issue regarding this:
Next.js API routes (and pages) should support reading files
Let's say I have a JSON object containing users' profile including firstname, lastname,..., and profileUrl from a web API. I've already saved the text data using Sqlite, and now I want to save those images in native storage and put the file path into the sqlite database replacing the profileUrl.
You can use cordova-plugin-file-transfer for that. There is even an Ionic Native wrapper available: https://ionicframework.com/docs/native/transfer/
Example:
download() {
const url = 'http://www.example.com/file.pdf';
fileTransfer.download(url, this.file.dataDirectory + 'file.pdf').then((entry) => {
console.log('download complete: ' + entry.toURL());
}, (error) => {
// handle error
});
}
I am building an app using Nativescript/Angular 2
I want to be able to download a file from a URL and save it to the device in a location the average user would have no problems finding it. I believe downloads would be the best place for this on both iOS and Android. Please correct me if I am wrong.
The file can be any file type, not just an image. So mainly spreadsheet, word document, pdf, png, jpg, etc.
I have searched online and through the documentation. The documentation describes a method called getFile which gets a file and saves it to your device.
I have implemented this in my code as follows:
download (id) {
console.log('Download Started');
getFile("https://raw.githubusercontent.com/NativeScript/NativeScript/master/apps/tests/logo.png").then(function (r) {
console.log(r.path);
}, function (e) {
//// Argument (e) is Error!
});
}
The problem with this is that it saves it to a non-user accessible location such as:
/data/user/0/com.myapp.example/files/logo.png
Update:
I have also tried specifying the path directly with:
fs.knownFolders.documents();
However, this method gets the documents folder for the current application that is NOT accessible by the user or external applications
After some unsuccessful attempts, I finally found how to save file to user "Downloads" folder (something like sdcard/Download). You can use android.os.Environment method to get this folder.
Add this in your component:
import { getFile } from 'tns-core-modules/http';
import * as fileSystem from "tns-core-modules/file-system";
import { isAndroid } from "tns-core-modules/platform";
import { alert } from "tns-core-modules/ui/dialogs";
declare var android;
<...>
public download (url, fileName) {
if (isAndroid) {
const permissions = require("nativescript-permissions");
permissions.requestPermission(android.Manifest.permission.WRITE_EXTERNAL_STORAGE, "I need these permissions because I'm cool")
.then(() => {
let downloadedFilePath = fileSystem.path.join(android.os.Environment.getExternalStoragePublicDirectory(android.os.Environment.DIRECTORY_DOWNLOADS).getAbsolutePath(), fileName);
getFile(url, downloadedFilePath).then(resultFile => {
alert({
title: 'Saved!',
okButtonText: 'OK',
message: `File saved here:\n${resultFile.path}`
});
}, error => {
alert({
title: 'Error',
okButtonText: 'OK',
message: `${error}`
});
});
});
}
}
What else you should know:
1) There is no any kind of download indicator, standard system download bar also not appears, and I don't know how to solve this.
2) For iOS you may try to use
const filePath = fileSystem.path.join(fileSystem.knownFolders.ios.downloads().path, fileName);
getFile(url, filePath).then((resultFile) => {}, (error) => {});
I think, it's the shame that NS docs don't talk straight, that you can't save files in user accessible location only with NS functions. I figured it out only when I read comments in file /node_modules/tns-core-modules/file-system/file-system.d.ts
Hope this helps you.
To get it working on iPhone, you can do the following (TypeScript):
import { knownFolders, path } from "tns-core-modules/file-system";
let destination = path.join(knownFolders.documents(), "file_name.txt");
// logic to save your file here ...
// the important thing is that you have to save your file in knownFolders.documents()
Then in Info.plist, you have to add the following permissions:
<key>LSSupportsOpeningDocumentsInPlace</key>
<true/>
<key>UIFileSharingEnabled</key>
<true/>
Now if you go to your iPhone's Files app > On My iPhone > Your App's Name, you should see the file there.
Basically, the Documents folder is a private folder inside your application's directory that only you can see. However, when you enable the two permissions above, it allows file sharing so that your user can access the folder and its contents.
The same documentation says that you can specify the file location like this:
download (id) {
console.log('Download Started');
var folder = fs.knownFolders.documents();
var file = fs.path.join(folder.path, "logo.png");
var url = "https://raw.githubusercontent.com/NativeScript/NativeScript/master/apps/tests/logo.png"
getFile(url, file).then(function (r) {
console.log(r.path);
}, function (e) {
//// Argument (e) is Error!
});
}
disclaimer: never tried it myself, just read the docs ...
You can specify a filesystem path directly, like this:
var folder = fs.Folder.fromPath('/sdcard/Download');
Note that /sdcard/Download will only work on Android; you can replace it with whatever (publicly accessible) folder you want to save your data to.
There doesn't yet seem to be a cross-platform way to choose a folder path, so you'll have to work out something manually. See this GitHub thread for more.
I realize that this is an older thread, but perhaps this can help someone:
If you use currentApp(), instead of documents(), you can access the folder you need. For example:
var directories = fs.knownFolders.currentApp();
var folder = directories.getFolder('./nameofaccessiblefolder');
I know this thread is 3 years ago but in case you have the same issue, I hope this solution will save time for you.
I solved the same issue by adding android:requestLegacyExternalStorage="true" inside the AndroidManifest.xml file
follow the thread here
I have a logo somewhere on my application page.
The application admin should be able to adjust the logo within the webapplication by simply upload a new one. What would be the best practice to achieve this?
How would I handle the upload on the server. It should replace the old logo with the new one. The name and location should stay the same.
Here is my approach:
I use the package UploadFS:
jalik:ufs
jalik:ufs-local
autopublish //it is still on, so the code below works without publish/subscribe I know that I will have to change that.
My code:
Upload
*.js Server & Client
//Almost Standard initialization - works so far
Logo = new Mongo.Collection('logo');
LogoStore = new UploadFS.store.Local({
collection: Logo,
name: 'logo',
path: '/uploads/logo',
mode: '0744', // directory permissions
writeMode: '0744', // file permissions
filter: new UploadFS.Filter({
minSize: 1,
maxSize: 1024 * 1000, // 1MB,
contentTypes: ['image/*'],
extensions: ['png']
})
});
*.html
//Standard initialization - works so far
<template name="upload">
<button type="button" name="upload">Select files</button>
</template>
*.js Client
//Almost Standard initialization - works so far
Template.upload.events({
'click button[name=upload]': function (ev) {
var self = this;
UploadFS.selectFiles(function (file) {
// Prepare the file to insert in database, note that we don't provide an URL,
// it will be set automatically by the uploader when file transfer is complete.
var logo = {
name: 'logo.png', //all uploaded images will have the same name
size: file.size,
type: file.type,
};
// Create a new Uploader for this file
var uploader = new UploadFS.Uploader({
// This is where the uploader will save the file
store: LogoStore,
// Optimize speed transfer by increasing/decreasing chunk size automatically
adaptive: true,
// Define the upload capacity (if upload speed is 1MB/s, then it will try to maintain upload at 80%, so 800KB/s)
// (used only if adaptive = true)
capacity: 0.8, // 80%
// The size of each chunk sent to the server
chunkSize: 8 * 1024, // 8k
// The max chunk size (used only if adaptive = true)
maxChunkSize: 128 * 1024, // 128k
// This tells how many tries to do if an error occurs during upload
maxTries: 5,
// The File/Blob object containing the data
data: file,
// The document to save in the collection
file: logo,
// The error callback
onError: function (err) {
console.error(err);
},
onAbort: function (file) {
console.log(file.name + ' upload has been aborted');
},
onComplete: function (file) {
console.log(file.name + ' has been uploaded');
},
onCreate: function (file) {
console.log(file.name + ' has been created with ID ' + file._id);
},
onProgress: function (file, progress) {
console.log(file.name + ' ' + (progress*100) + '% uploaded');
},
onStart: function (file) {
console.log(file.name + ' started');
},
onStop: function (file) {
console.log(file.name + ' stopped');
}
});
// Starts the upload
uploader.start();
// Stops the upload
uploader.stop();
// Abort the upload
uploader.abort();
});
}
});
Show uploaded Logo
*.html
<template name="whatever">
<img src="{{logoUrl}}" alt="Logo" >
</template>
*.js client only
Template.whatever.helpers({
logoUrl: function(){
return Logo.findOne().url;
}
})
So If I understand it right, what the code does is uploading the img to somewhere on the server. Also it stores some info about that image in a Mongo.Collection - Logo.
But I do not exactly know where those images are stored, in which folder. They are not stored in my default Project - Folder.
The url of an example img is: http://localhost:3000/ufs/logo/B4Fv5etkr7xQbvs5v/logo.png. That random string in the middle is the _id of that img. So I can not use a hardcoded url for that images to access them, because as soon as a new img is uploaded, that url will change completely.
Q1: So the first question is: Can I upload to the myProject/public/img folder directly? So that the url of the img would be something like:
http://localhost:3000/img/logo.png
Then I would need just to replace the old logo on the upload.
For now I have to deal with the generic url. So as next, I select the url of the now present image on the server from the Logo - collection and pass that url to my template to the place where the logo has to be placed. The problem with that is, that the url is loaded after everything else was loaded, so for several seconds I get an tag without an url in it. So that place shows the alt text only until the url is loaded. That is very ugly...
Q2: The question is, how could I get the url, before the tag is loaded. So that the logo appears with/before everything else, as if the url would be hardcoded in advance.
Q3: Is it possible to replace the old logo with the new uploaded one, on the upload? How?
Q4: If I delete the entry for the img from the Logo - Collection, is the image actually deleted from the server? Or do I have to delete it manually/in another way?
You can send a base64 encode image on server then using fs your can overwrite the file.
Like:
Client
readAsDataURL has base64 encoded data in the format of
data:image/jpeg;base64,/9j/4AAQSkZJRgABA...
So you need to get rid of the mime type and encoding information at the front.
contents = contents.split(',')[1];
Now you can send this base64 encoded data to server.
Server
Since you're receiving base64 encoded data, you can convert it buffer and write to file:
fs.writeFile(filepath, Buffer(argument,'base64'), err => {
//
})
In case the file name same as another file then it will completely overwrite your file.
Answer for question 1:
default image will be stored hide folder in your project .meteor/local/build/programs/server/ufs/uploads/
You can change destination by "path" as below code
new UploadFS.store.Local({
collection: Csvs.collection,
name: 'csv',
path: '../../../../../uploads/csv', //here change destination folder stored file
filter: new UploadFS.Filter({
maxSize: 1024 * 3000, // 3MB,
contentTypes: ['text/csv']
})
});
When doing the "Client" step in Pankaj Javav's answer, you may want to use the base64-image-upload package I whipped up, as I was having the same problem. It simplifies the process of uploading any base64 string to a server, and you do not need to get rid of the MIME type this way.
I think I'm missing something. I have read a lot of posts/examples and I can't save images on my system (I work locally).
What is my goal ?
I'm trying to save a file submitted by the user in a folder (server-side). Does it sound easy ? Maybe.
What's the issue ?
Short answer : I can't figure out how to save the file in my folder.
Do you want more information ?
The story of a file upload
I have read that to use the path parameter like new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" }) , I have to declare my collection server-side. But when I call Avatars.insert() (Avatars is the name of my collection), it seems like it doesn't exists. This makes sense because this collection exists only on the server.
So I've tried to declare the collection both server-side and client-side (I've read some examples about that) and that works ! The file is correctly added to MongoDB, but my folder is still empty (I'm not sure but I think this is because Avatars.insert() is called client-side so the collection used is the client-side one, the one which cannot take path parameter).
But no problem ! I've created 2 Meteor methods (one client-side and one server-side) called "updateAvatarFile". With this "trick", I'm able to do Meteor.call("updateAvatarFile", field.files[0]), which calls both server-side and client-side methods. So I can do some UI stuff in the client-side one and upload the file in the other. But I can't pass the file as a parameter.
field.files[0] contains the file client-side but server-side it's an empty object. My question is : How can I upload a file ?
I can't do it client-side (because I can't use path parameter) but I can pass the file to the server. I'm sure that I'm missing something but I can't figure what.
Here is how I go :
// /client/views/templates/settings.js
Template.settings.events({
'submit #updateAvatar': function (e, template) {
e.preventDefault();
const field = document.getElementsByName('avatar')[0];
Meteor.call('updateAvatarFile', field.files[0]);
}
});
// /client/lib/clientMethods.js
Meteor.methods({
'updateAvatarFile': function (file) {
// blabla
}
});
// /server/lib/serverMethods.js
Meteor.methods({
'updateAvatarFile': function (file) {
Avatars.insert(file, function (err, fileObj) {
if (err) {
console.log(err);
} else {
console.log(fileObj);
}
});
}
});
// /server/collections/serverAvatarCollection.js
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original", { path: "/public/images/user/avatar" }),
new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" })
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
//throw new Meteor.Error(403, message);
}
});
// /client/collections/clientAvatarCollection.js
// (this one is actually in a comment block)
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original"),
new FS.Store.FileSystem("thumb")
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
alert(message);
}
});
I've also tried to insert the file with the client-side method but I've got the same result (the file is added to MongoDB but not saved into a folder).
Using different path values didn't work either.
EDIT : Or maybe I'm trying to use the wrong package ? To my mind, transform a picture to chunks and save them into MongoDB sounds really weird and bad. Do you have any adivces ?
EDIT 2 :
answer to Michel Floyd (sorry about that, the character limit is annoying).
First, thanks for your answer !
1. At the moment, I'm just trying Meteor so I have both autopublish and insecure installed. Not publishing/subscribing to my collection cannot cause an issue, is that right ?
2. Before your answer I've tried to set up a collection available for both server and client by putting my avatarCollection.js in /collections. I was thinking that path which doesn't contains server or client are automatically available for the two sides. So what is the difference between /collections and /lib ? (I know that all files in a "lib" folder are loaded first). Is it a bad practice to put collections in /collections ? Maybe should I create a /lib/collections folder ?
3. (the last point, sorry for the long comment) I've tried what you advised above but it doesn't seems to work (or I am doing something wrong, again ><). When I use Avatars.insert(), CollectionFS don't save the file on my local storage. I've also checked the root of my HDD in case CollectionFS interpreted / to be the root of my machine but it doesn't. In the other hand, CollectionFS have created 4 collections in MongoDB (cfs._tempstore.chunks, cfs.avatars.filerecord, cfs_gridfs._tempstore.chunks and cfs_gridfs._tempstore.files) - the gridfs is weird. I have GridFS installed but I use FileSystem -. Those tables are not empty. That's why I think CollectionFS split my file into chunks and save them in MongoDB.
You're generally on the right track. CollectionFS uses storage adapters to deal with actual file storage. You can put files on S3, gridFS, or your local file system as you're trying to do. Putting the file contents in Mongo directly is usually avoided.
Firstly, define your collection:
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original", { path: "/public/images/user/avatar" }),
new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" })
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
//throw new Meteor.Error(403, message);
}
});
in /lib! This will make it available to both the server and the client.
Secondly, make sure you publish your avatars collection from the server and subscribe to it from the client. I don't see any publish/subscribe code in your question. You need it.
Thirdly, if you just do:
Avatars.insert(...);
on the client with a file then CollectionFS then CollectionFS will take care of storing it for you. The thing is, it won't be instantly available. It can take a little while for the actual upload and storage to happen. You can look at fileObj.isUploaded for example to see if the file is ready.