Download a file from a URL to a user accessible location - http

I am building an app using Nativescript/Angular 2
I want to be able to download a file from a URL and save it to the device in a location the average user would have no problems finding it. I believe downloads would be the best place for this on both iOS and Android. Please correct me if I am wrong.
The file can be any file type, not just an image. So mainly spreadsheet, word document, pdf, png, jpg, etc.
I have searched online and through the documentation. The documentation describes a method called getFile which gets a file and saves it to your device.
I have implemented this in my code as follows:
download (id) {
console.log('Download Started');
getFile("https://raw.githubusercontent.com/NativeScript/NativeScript/master/apps/tests/logo.png").then(function (r) {
console.log(r.path);
}, function (e) {
//// Argument (e) is Error!
});
}
The problem with this is that it saves it to a non-user accessible location such as:
/data/user/0/com.myapp.example/files/logo.png
Update:
I have also tried specifying the path directly with:
fs.knownFolders.documents();
However, this method gets the documents folder for the current application that is NOT accessible by the user or external applications

After some unsuccessful attempts, I finally found how to save file to user "Downloads" folder (something like sdcard/Download). You can use android.os.Environment method to get this folder.
Add this in your component:
import { getFile } from 'tns-core-modules/http';
import * as fileSystem from "tns-core-modules/file-system";
import { isAndroid } from "tns-core-modules/platform";
import { alert } from "tns-core-modules/ui/dialogs";
declare var android;
<...>
public download (url, fileName) {
if (isAndroid) {
const permissions = require("nativescript-permissions");
permissions.requestPermission(android.Manifest.permission.WRITE_EXTERNAL_STORAGE, "I need these permissions because I'm cool")
.then(() => {
let downloadedFilePath = fileSystem.path.join(android.os.Environment.getExternalStoragePublicDirectory(android.os.Environment.DIRECTORY_DOWNLOADS).getAbsolutePath(), fileName);
getFile(url, downloadedFilePath).then(resultFile => {
alert({
title: 'Saved!',
okButtonText: 'OK',
message: `File saved here:\n${resultFile.path}`
});
}, error => {
alert({
title: 'Error',
okButtonText: 'OK',
message: `${error}`
});
});
});
}
}
What else you should know:
1) There is no any kind of download indicator, standard system download bar also not appears, and I don't know how to solve this.
2) For iOS you may try to use
const filePath = fileSystem.path.join(fileSystem.knownFolders.ios.downloads().path, fileName);
getFile(url, filePath).then((resultFile) => {}, (error) => {});
I think, it's the shame that NS docs don't talk straight, that you can't save files in user accessible location only with NS functions. I figured it out only when I read comments in file /node_modules/tns-core-modules/file-system/file-system.d.ts
Hope this helps you.

To get it working on iPhone, you can do the following (TypeScript):
import { knownFolders, path } from "tns-core-modules/file-system";
let destination = path.join(knownFolders.documents(), "file_name.txt");
// logic to save your file here ...
// the important thing is that you have to save your file in knownFolders.documents()
Then in Info.plist, you have to add the following permissions:
<key>LSSupportsOpeningDocumentsInPlace</key>
<true/>
<key>UIFileSharingEnabled</key>
<true/>
Now if you go to your iPhone's Files app > On My iPhone > Your App's Name, you should see the file there.
Basically, the Documents folder is a private folder inside your application's directory that only you can see. However, when you enable the two permissions above, it allows file sharing so that your user can access the folder and its contents.

The same documentation says that you can specify the file location like this:
download (id) {
console.log('Download Started');
var folder = fs.knownFolders.documents();
var file = fs.path.join(folder.path, "logo.png");
var url = "https://raw.githubusercontent.com/NativeScript/NativeScript/master/apps/tests/logo.png"
getFile(url, file).then(function (r) {
console.log(r.path);
}, function (e) {
//// Argument (e) is Error!
});
}
disclaimer: never tried it myself, just read the docs ...

You can specify a filesystem path directly, like this:
var folder = fs.Folder.fromPath('/sdcard/Download');
Note that /sdcard/Download will only work on Android; you can replace it with whatever (publicly accessible) folder you want to save your data to.
There doesn't yet seem to be a cross-platform way to choose a folder path, so you'll have to work out something manually. See this GitHub thread for more.

I realize that this is an older thread, but perhaps this can help someone:
If you use currentApp(), instead of documents(), you can access the folder you need. For example:
var directories = fs.knownFolders.currentApp();
var folder = directories.getFolder('./nameofaccessiblefolder');

I know this thread is 3 years ago but in case you have the same issue, I hope this solution will save time for you.
I solved the same issue by adding android:requestLegacyExternalStorage="true" inside the AndroidManifest.xml file
follow the thread here

Related

Replace Google Fonts with self hosted fonts

I'm currently porting parts of a legacy codebase which has more than 100 themes that each come with their own css files. Those files are full with hardcoded links to Google Fonts which need to be replaced due to GDPR.
Is there some kind of automated tool available which scans through these files, replaces the link to Google Fonts and downloads all the assets? I've found a couple of semi-automated tools online but they all require copy & paste and manual download of the files. That's okay for 2-3 fonts but not for hundreds of them. Any tips for that?
I have put some efforts to create this NodeJS script.
This script searches for all css files and extracts the woff font url. Then, it replaces it with absolute path of the downloaded file against the url it found, also downloads the file in the relevant directory which can be clearly identified in the snippet as specified with fontDownloadDirectoryPath variable.
This script can be modified and improved further but as of now, provides the required functionality at its base level.
I hope this can serve as a starting point atleast to solve the stated problem or can be used completely as a solution changes few variables, given that my assumptions of few required things to get to this solution are correct.
Please feel free to modify, accordingly like the regex pattern to match something else, adding few other font types in the pattern, adding few more code to make it more robust and generalised, etc for other possibilities.
const path = require('path');
const fs = require('fs');
const https = require("https");
// update assets/css with your css path
const directoryPath = path.join(__dirname, 'assets/css');
let fontDownloadDirectoryPath = path.join(__dirname, 'assets/fonts')
let fontDownloadDirectoryFileFullPath = path.join(__dirname, 'assets/fonts/fontsDownloadUrlList.json')
fs.readdir(directoryPath, function (err, files) {
//handling error
if (err) {
return console.log('Unable to scan directory: ' + err);
}
//listing all files using forEach
files.forEach(function (file) {
// Do whatever you want to do with the file
let file_full_path = directoryPath + "/" + file
fs.readFile(file_full_path, 'utf8', (err, content) => {
if (err) {
console.error(err);
return;
}
// console.log(content);// show the content of readed file
let found = content.match(/url\(['"]([^"']+(woff2|eot|woff|ttf)["'])+\)/gi)
console.log(file_full_path, found);
let updated_content = content
if (found) {
if (fs.existsSync(fontDownloadDirectoryFileFullPath)) {
// enter the code to execute after the folder is there.
console.log('file exists')
fs.readFile(fontDownloadDirectoryFileFullPath, 'utf8', (err, read_content) => {
let read_content_json = JSON.parse(read_content)
read_content_json.push(...found)
fs.writeFile(fontDownloadDirectoryFileFullPath, JSON.stringify(read_content_json), function () { })
})
} else {
fs.writeFile(fontDownloadDirectoryFileFullPath, JSON.stringify(found), function () { })
}
console.log(found)
found.forEach((item) => {
let fontFileUrl = item.split("'")[1]
let fontFileName = fontFileUrl.split("/")[fontFileUrl.split("/").length - 1]
console.log(fontFileUrl, fontFileName)
https.get(fontFileUrl, response => {
var body = '';
var i = 0;
response.on('data', function (chunk) {
i++;
body += chunk;
console.log('BODY Part: ' + i);
});
response.on('end', function () {
console.log(body);
fs.writeFileSync(fontDownloadDirectoryPath + "/" + fontFileName, body, { encoding: 'utf8', flag: 'w' }, (err) => { console.log(err) })
console.log('Finished');
});
});
updated_content = updated_content.replace(item, "url('" + fontDownloadDirectoryPath + "/" + fontFileName + "')")
})
} else {
updated_content = content;
}
fs.writeFileSync(file_full_path, updated_content, { encoding: 'utf8', flag: 'w' })
});
});
});
I used below css file in root/assets/css directory with styles.css name for testing the above script:
#font-face {
font-family: 'BR Firma';
src: url('https://fonts.gstatic.com/s/opensans/v29/memSYaGs126MiZpBA-UvWbX2vVnXBbObj2OVZyOOSr4dVJWUgsiH0B4taVQUwaEQbjB_mQ.woff') format('woff');
font-weight: bold;
font-style: normal;
font-display: swap;
}
without having more background on the project, directory structure, and so on, I will outline how the task could be done as of now:
Scan all the directories or URLs of the project (if you run it locally or remotely) if the fonts are not being imported from one main CSS file (could happen).
Get all the Google Fonts URLs
Download all the assets (i.e. fonts from the links, maybe some pics also, etc.)
So, although you can totally do this locally with directories, here I will explain a way to do it with the browser for brevity - and possibly convenience - with Python. I am assuming you have access to the project's URLs, ofc.
You can follow this approach to scrape the URLs you want. Pass it a list from the sitemap to go through all the URLs in sequence. Then you can filter the list you get to account only for Google Fonts, simply add the in operator (as in here) to get true or false, respectively.
substring = 'https://fonts.googleapis.com'
if substring in element:
list.append(element)
else:
#do nothing, don't add it to list
Now you should have all the URLs you are interested in. If the project has several HTML pages with different fonts, those are the ones you need to scan - or maybe just all of them to be sure. Note: it is usually useful to store the list in a file to further add this code to the previous script. More info.
with open('urls.txt', 'w') as f:
f.write(element)
To download the assets, you can use this approach. Since you have all the URLs, you should be able to.
And that's pretty much it! If you add more insight into the project structure we could complete the scripts more precisely. Also, you can quickly use Jupyter Notebook to run the scripts as you tune them. In the meantime, the open details to clarify would be:
Scan websites or files?
Only HTML files or all the projects?
What to download? The font assets only?
Python script works fine for this task?
Any IDE can do, just "search and replace in files", with the appropriate patterns.
For example: PHPStorm: Find and replace text using regular expressions. Finding all the occurrences alone is already worth something and an IDE might help with "porting parts of a legacy codebase".

Edge browser Native File System removeEntry use

I'm working with the newest version of Edge (Canary release 86.0.615.0) and I can get the new Native File System API showOpenFilePicker to let me access files but I can't find a reference to the directoryHandle functions including the removeEntry function if the user elects to remove the file. Am I missing a special flag? I have an Origin-Tracker code and I also have the experimental flag set for the Native File System API.
If you have a directory handle, you can delete files or folders as in the example below:
// Delete a file.
await directoryHandle.removeEntry('Abandoned Projects.txt');
// Recursively delete a folder.
await directoryHandle.removeEntry('Old Stuff', { recursive: true });
You can obtain a directory handle from the picker:
const directoryHandle = await window.showDirectoryPicker();
To iterate over the entries of a directory, you can use the code snippet below:
for await (const entry of directoryHandle.values()) {
console.log(entry.kind, entry.name);
}

How to get local file path of image file downloaded from Firebase Storage in React Native

In my React Native app, i need to
Upload an image to Firebase Storage
Then Download it and edit it (crop/rotate)
Upload the edited image the Firebase Storage again
I was able to upload the image, get its URL using the getDownloadURL() method, and display it using Image component as it takes the url input right away.
Now I need the local file path in order to edit and reupload the edited image to Firebase Storage.
This question explains a getFile() method, but I cant find it firebase docs.
Another question give some direction but its in JAVA.
Some Blogs mention using libraries such as react-native-fs, react-native-fetch-blob etc, but it appears they are trying to save the file in storage/gallery. That is unnecessary for my requirement (i dont want to save images in gallery). I probably just need the path to the cached image where it gets downloaded.
Can someone please explain how to get local file path of downloaded image file in React Native lingo ?
EDIT:
Not using Expo. And I want it to work with Android and iOS both.
npm install rn-fetch-blob
Follow these steps:
a) follow the installation instructions.
a2) if you want to manually install the package without using rnpm, go to their wiki.
b) Finally, that's how I made it possible to download files within my app:
const { config, fs } = RNFetchBlob
let PictureDir = fs.dirs.PictureDir // this is the pictures directory. You can check the available directories in the wiki.
let options = {
fileCache: true,
addAndroidDownloads : {
useDownloadManager : true, // setting it to true will use the device's native download manager and will be shown in the notification bar.
notification : false,
path: PictureDir + "/me_"+Math.floor(date.getTime() + date.getSeconds() / 2), // this is the path where your downloaded file will live in
description : 'Downloading image.'
}
}
config(options).fetch('GET', "http://www.example.com/example.pdf").then((res) => {
// do some magic here
})
OR
const { uri: localUri } = await FileSystem.downloadAsync(remoteUri, FileSystem.documentDirectory + 'name.ext');
install react-native-fs
yarn add react-native-fs
and can get temporary path like this
var RNFS = require('react-native-fs');
const imageUrl = `http://www.example.com/abc.png`;
const imagePath = `${Platform.OS==="android"?"/storage/emulated/0/Download":RNFS.TemporaryDirectoryPath}/${((Math.random() * 1000) | 0)}.jpeg`;
RNFS.downloadFile({
fromUrl: imageUrl,
toFile: imagePath
}).promise
.then((result) => {
console.log(imagePath); //here you get temporary path
})
.catch((e) => {
console.log(e,"error");
})

How to save a file -in a folder- using FileSystem and CollectionFS ? (yeah, really.)

I think I'm missing something. I have read a lot of posts/examples and I can't save images on my system (I work locally).
What is my goal ?
I'm trying to save a file submitted by the user in a folder (server-side). Does it sound easy ? Maybe.
What's the issue ?
Short answer : I can't figure out how to save the file in my folder.
Do you want more information ?
The story of a file upload
I have read that to use the path parameter like new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" }) , I have to declare my collection server-side. But when I call Avatars.insert() (Avatars is the name of my collection), it seems like it doesn't exists. This makes sense because this collection exists only on the server.
So I've tried to declare the collection both server-side and client-side (I've read some examples about that) and that works ! The file is correctly added to MongoDB, but my folder is still empty (I'm not sure but I think this is because Avatars.insert() is called client-side so the collection used is the client-side one, the one which cannot take path parameter).
But no problem ! I've created 2 Meteor methods (one client-side and one server-side) called "updateAvatarFile". With this "trick", I'm able to do Meteor.call("updateAvatarFile", field.files[0]), which calls both server-side and client-side methods. So I can do some UI stuff in the client-side one and upload the file in the other. But I can't pass the file as a parameter.
field.files[0] contains the file client-side but server-side it's an empty object. My question is : How can I upload a file ?
I can't do it client-side (because I can't use path parameter) but I can pass the file to the server. I'm sure that I'm missing something but I can't figure what.
Here is how I go :
// /client/views/templates/settings.js
Template.settings.events({
'submit #updateAvatar': function (e, template) {
e.preventDefault();
const field = document.getElementsByName('avatar')[0];
Meteor.call('updateAvatarFile', field.files[0]);
}
});
// /client/lib/clientMethods.js
Meteor.methods({
'updateAvatarFile': function (file) {
// blabla
}
});
// /server/lib/serverMethods.js
Meteor.methods({
'updateAvatarFile': function (file) {
Avatars.insert(file, function (err, fileObj) {
if (err) {
console.log(err);
} else {
console.log(fileObj);
}
});
}
});
// /server/collections/serverAvatarCollection.js
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original", { path: "/public/images/user/avatar" }),
new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" })
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
//throw new Meteor.Error(403, message);
}
});
// /client/collections/clientAvatarCollection.js
// (this one is actually in a comment block)
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original"),
new FS.Store.FileSystem("thumb")
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
alert(message);
}
});
I've also tried to insert the file with the client-side method but I've got the same result (the file is added to MongoDB but not saved into a folder).
Using different path values didn't work either.
EDIT : Or maybe I'm trying to use the wrong package ? To my mind, transform a picture to chunks and save them into MongoDB sounds really weird and bad. Do you have any adivces ?
EDIT 2 :
answer to Michel Floyd (sorry about that, the character limit is annoying).
First, thanks for your answer !
1. At the moment, I'm just trying Meteor so I have both autopublish and insecure installed. Not publishing/subscribing to my collection cannot cause an issue, is that right ?
2. Before your answer I've tried to set up a collection available for both server and client by putting my avatarCollection.js in /collections. I was thinking that path which doesn't contains server or client are automatically available for the two sides. So what is the difference between /collections and /lib ? (I know that all files in a "lib" folder are loaded first). Is it a bad practice to put collections in /collections ? Maybe should I create a /lib/collections folder ?
3. (the last point, sorry for the long comment) I've tried what you advised above but it doesn't seems to work (or I am doing something wrong, again ><). When I use Avatars.insert(), CollectionFS don't save the file on my local storage. I've also checked the root of my HDD in case CollectionFS interpreted / to be the root of my machine but it doesn't. In the other hand, CollectionFS have created 4 collections in MongoDB (cfs._tempstore.chunks, cfs.avatars.filerecord, cfs_gridfs._tempstore.chunks and cfs_gridfs._tempstore.files) - the gridfs is weird. I have GridFS installed but I use FileSystem -. Those tables are not empty. That's why I think CollectionFS split my file into chunks and save them in MongoDB.
You're generally on the right track. CollectionFS uses storage adapters to deal with actual file storage. You can put files on S3, gridFS, or your local file system as you're trying to do. Putting the file contents in Mongo directly is usually avoided.
Firstly, define your collection:
Avatars = new FS.Collection("avatars", {
stores: [
new FS.Store.FileSystem("original", { path: "/public/images/user/avatar" }),
new FS.Store.FileSystem("thumb", { path: "/public/images/user/avatar" })
],
filter: {
maxSize: 1000000, //1Mo
allow: { contentTypes: ['image/*'] }
},
onInvalid: function (message) {
//throw new Meteor.Error(403, message);
}
});
in /lib! This will make it available to both the server and the client.
Secondly, make sure you publish your avatars collection from the server and subscribe to it from the client. I don't see any publish/subscribe code in your question. You need it.
Thirdly, if you just do:
Avatars.insert(...);
on the client with a file then CollectionFS then CollectionFS will take care of storing it for you. The thing is, it won't be instantly available. It can take a little while for the actual upload and storage to happen. You can look at fileObj.isUploaded for example to see if the file is ready.

How to list files in folder

How can I list all files inside a folder with Meteor.I have FS collection and cfs:filesystem installed on my app. I didn't find it in the doc.
Another way of doing this is by adding the shelljs npm module.
To add npm modules see: https://github.com/meteorhacks/npm
Then you just need to do something like:
var shell = Meteor.npmRequire('shelljs');
var list = shell.ls('/yourfolder');
Shelljs docs:
https://github.com/arturadib/shelljs
The short answer is that FS.Collection creates a Mongo collection that you can treat like any other, i.e., you can list entries using find().
The long answer...
Using cfs:filesystem, you can create a mongo database that mirrors a given folder on the server, like so:
// in lib/files.js
files = new FS.Collection("my_files", {
stores: [new FS.Store.FileSystem("my_files", {"~/test"})] // creates a ~/test folder at the home directory of your server and will put files there on insert
});
You can then access this collection on the client to upload files to the server to the ~test/ directory:
files.insert(new File(['Test file contents'], 'my_test_file'));
And then you can list the files on the server like so:
files.find(); // returns [ { createdByTransform: true,
_id: 't6NoXZZdx6hmJDEQh',
original:
{ name: 'my_test_file',
updatedAt: (Date)
size: (N),
type: '' },
uploadedAt: (Date),
copies: { my_files: [Object] },
collectionName: 'my_files'
}
The copies object appears to contain the actual names of the files created, e.g.,
files.findOne().copies
{
"my_files" : {
"name" : "testy1",
"type" : "",
"size" : 6,
"key" : "my_files-t6NoXZZdx6hmJDEQh-my_test_file", // This is the name of the file on the server at ~/test/
"updatedAt" : ISODate("2015-03-29T16:53:33Z"),
"createdAt" : ISODate("2015-03-29T16:53:33Z")
}
}
The problem with this approach is that it only tracks the changes made through the Collection; if you add something manually to the ~/test directory, it won't get mirrored into the Collection. For instance, if on the server I run something like...
mkfile 1k ~/test/my_files-aaaaaaaaaa-manually-created
Then I look for it in the collection, it won't be there:
files.findOne({"original.name": {$regex: ".*manually.*"}}) // returns undefined
If you just want a straightforward list of files on the server, you might consider just running an ls. From https://gentlenode.com/journal/meteor-14-execute-a-unix-command/33 you can execute any arbitrary UNIX command using Node's child_process.exec(). You can access the app root directory with process.env.PWD (from this question). So in the end if you wanted to list all the files in your public directory, for instance, you might do something like this:
exec = Npm.require('child_process').exec;
console.log("This is the root dir:");
console.log(process.env.PWD); // running from localhost returns: /Users/me/meteor_apps/test
child = exec('ls -la ' + process.env.PWD + '/public', function(error, stdout, stderr) {
// Fill in this callback with whatever you actually want to do with the information
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
if(error !== null) {
console.log('exec error: ' + error);
}
});
This will have to run on the server, so if you want the information on the client, you'll have to put it in a method. This is also pretty insecure, depending on how you structure it, so you'd want to think about how to stop people from listing all the files and folders on your server, or worse -- running arbitrary execs.
Which method you choose probably depends on what you're really trying to accomplish.

Resources