I can't find a satisfying answer to my question. Given an image url, I want to download it (without saving it to the disk) and to immediately upload it to an AWS Bucket. Here is my code :
self.downloadImage = function(url){
let response = HTTP.get(url, {
encoding:null // for binary
})
if (!response.headers['content-type'].split('/')[0] == 'image'){
throw new Error("not an image")
}
return {
data : response.content,
contentType : response.headers['content-type']
}
}
self.uploadImage = function(websiteName, imgUrl, callback){
// we retrieve the image
let image = downloadImage(imgUrl)
let imageName = self.extractImageName(imgUrl)
let filename = 'img/' +websiteName + "/" + imageName
let newUrl = `https://s3.${Meteor.settings.AWS.REGION}.amazonaws.com/${Meteor.settings.S3.BUCKET_NAME}/${filename}`
// makes the async function sync like
let putObjectSync = Meteor.wrapAsync(s3Client.putObject, s3Client)
try{
let res = putObjectSync({
Body: image.data,
ContentType : image.contentType,
ContentEncoding: 'base64',
Key: filename,
ACL:'public-read',
Bucket: Meteor.settings.S3.BUCKET_NAME
})
return newUrl
} catch(e) {
return null
}
}
Everything works fine, except that the image seems corrupted. So far I tried :
to use aldeed:http, in order to set the encoding to null when downloading, which seems a good strategy for images
not to use it and to pass the text content of the response directly as the upload body
to add base64 encoding in aws
Still corrupted. I feel very close to the solution, as the image as the correct type and file size, but still won't print in the browser or on my computer. Any idea about how to correctly encode/retrieve the data ?
Okay I found the answer by myself :
aldeed:meteor allow to add a responseType parameter to the get request. We simply need to set this option to buffer, so that we get the data as a buffer. Then we simply give this buffer, with no transformation, as the Body of the upload function.
Related
I am trying to get the depth data associated with an image in the PhotoLibrary.
I can get the image, and the URL, but I can't seem to get the aux data associated with it. The call to CGImageSourceCreateWithURL returns a source, but the call to CGImageSourceCopyAuxiliaryDataInfoAtIndex returns nil for both kCGImageAuxiliaryDataTypeDisparity and kCGImageAuxiliaryDataTypeDepth.
Is there something I am missing here?
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {
let image = info[UIImagePickerControllerOriginalImage]
let url = info[UIImagePickerControllerImageURL]
print("url=",url)
guard let source = CGImageSourceCreateWithURL(url as! CFURL, nil) else {
return
}
guard let auxDataInfo = CGImageSourceCopyAuxiliaryDataInfoAtIndex(source, 0, kCGImageAuxiliaryDataTypeDisparity) as? [AnyHashable : Any] else {
return
}
}
I struggled with this for a whole day! I finally figured it out, though, after watching the first half of the WWDC Video titled "Editing Images With Depth."
My problem was using a URL for the image that was not from the PHAsset.
Here is the link:
LINK TO WWDC VIDEO
If you don't feel like watching it, check out this function that I wrote that does pretty much exactly what is done in the video.
You have to provide the function the [info] that is returned from the DID_FINISH_PICKING_IMAGE_WITH_INFO function from the UIImagePickerDelegate.
Before using this function - note that is actually doesn't work! It is great to look at though, because it shows the steps clearly. But due to asynchronous behavior, the function will always return nil before it has a chance to set the local depth variable to the AVDepthData.
My solution was to break the function apart and use Grand Central Dispatch to create a Dispatch Group, enter it, retrieve the imageURL from the PHAsset, and then leave the Dispatch Group. Upon leaving the Dispatch Group, the DispatchGroup.NOTIFIED function then proceeded with the rest of the process.
I hope this helps!!!
func returndepthdata(usingimageinfo: [UIImagePickerController.InfoKey : Any]) -> AVDepthData? {
var depthdata: AVDepthData! = nil
if let photoasset = usingimageinfo[.phAsset] as? PHAsset {
let input = photoasset.requestContentEditingInput(with: nil, completionHandler: { (input, info) in
if let imageurl = input?.fullSizeImageURL {
if let source = CGImageSourceCreateWithURL(imageurl as CFURL, nil) {
if let imageproperties = CGImageSourceCopyProperties(source, nil) {
if let disparityinfo = CGImageSourceCopyAuxiliaryDataInfoAtIndex(source, 0, kCGImageAuxiliaryDataTypeDisparity) {
if let truedepthdata = try? AVDepthData(fromDictionaryRepresentation: disparityinfo as! [AnyHashable : Any]) {
depthdata = truedepthdata
}
}
}
}
}
})
}
return depthdata
}
The image URL supplied by UIImagePickerController does not include any of the metadata associated with depth. To get this information, you must access the PHAsset using the PhotoBook API.
First, import the API:
import Photos
Before you display your image picker, request user access to the photo book. You'll need to add an info dictionary key for Photo Library Usage for this to work:
switch PHPhotoLibrary.authorizationStatus() {
case .notDetermined:
PHPhotoLibrary.requestAuthorization { (status) in
if status == .authorized {
DispatchQueue.main.async {
// Display image picker here
}
}
}
case .authorized: // Display image picker here
case .denied, .restricted: // Display appropriate error here
}
Now, in your image picker delegate you can do this:
if let asset = info[.phAsset] as? PHAsset {
PHImageManager.default().requestImageData(for: asset, options: nil) { (imageData, dataType, orientation, info) in
let url = info?["PHImageFileURLKey"] as? URL
// Pass this URL to your existing code.
}
}
Note that the file may contain depth or disparity information. You can convert between these easily enough, but you may need to check which one you have using CGImageSourceCopyProperties(). Also look out for the new supplementary depth data, kCGImageAuxiliaryDataTypePortraitEffectsMatte, which gives a much higher resolution mask for just the subject person in portrait images and is great for doing greenscreen-style effects.
After generated downloadURL in firebase using:
{
...
let url = uploadTask.snapshot.downloadURL;
...
}
the url content is something similar:
https://firebasestorage.googleapis.com/v0/b/sd2ds-.23.appspot.com/o/Usr%2FEls%2 ...
No worries in using in the browser or HTML tags, however, when I try to use the url generated in photoURL an error is generated:
The photoURL field must be a valid URL.
I already tried use decodeURI(url) but nothing changed.
Firebase generates the url with %2 in / (slash) place and seems be the problem.
Any glue?
Thanks
Use decodeURIComponent.
Suggestion of function with error handling:
const validUrl = require('valid-url');
const fillPhotoUrl = function(p) { return (p && validUrl.isUri(p)) ? decodeURIComponent(p) : null; };
Then use it:
{
...
let url = fillPhotoUrl(uploadTask.snapshot.downloadURL);
...
}
If you want to know the difference between decodeURI and decodeURIComponent, see this question.
Not sure the best way to ask this question, but it relates to firebase file storage
I am using a image stored on firebase but using a uri so I need the file to not be a download url but rather a url where it is still hosted on firebase.. hence something like url.jpg (something like http://i.imgur.com/rebvLRB.jpg ) rather than something that just downloads the image to your disk
Does firebase offer something like that.. I couldn't find option that will allow me to look at the image hosted on firebase.. rather than download it..
Hopefully that makes sense..
The short answer to your question is: the browser performs actions primarily based on two headers: Content-Type and Content-Disposition. If the file is an image (Content-Type: image/*) and has a Content-Disposition: attachment, the browser will download it (likely what we're doing by default). Set the Content-Type to inline instead and it will display in the browser.
You can get a download URL via ref.getDownloadURL(). There are many different options to get the reference:
// Create a reference with an initial file path and name
var storage = firebase.storage();
var pathReference = storage.ref('images/stars.jpg');
// Create a reference from a Google Cloud Storage URI
var gsReference = storage.refFromURL('gs://bucket/images/stars.jpg')
// Create a reference from an HTTPS URL
var httpsReference = storage.refFromURL('https://firebasestorage.googleapis.com/b/bucket/o/images%20stars.jpg');
Note that in the URL, characters are URL escaped, e.g. / becomes %20
And here's an example of calling getDownloadURL()
storageRef.child('images/stars.jpg').getDownloadURL().then(function(url) {
// `url` is the download URL for 'images/stars.jpg'
// This can be downloaded directly:
var xhr = new XMLHttpRequest();
xhr.responseType = 'blob';
xhr.onload = function(event) {
var blob = xhr.response;
};
xhr.open('GET', url);
xhr.send();
// Or inserted into an <img> element:
var img = document.getElementById('myimg');
img.src = url;
}).catch(function(error) {
// Handle any errors
});
See more here: https://firebase.google.com/docs/storage/web/download-files
Pretty new user to Firebase here. I am trying to zip a .wav file from Firebase right now and I am currently stuck on using jszip in order to zip the files from the download URLs I pull from Firebase. The URLs look like this:
https://firebasestorage.googleapis.com/v0/b/instasample-d8eea.appspot.com/o/kick%2F1.wav?alt=media&token=91612541-83e5-4e82-ae1b-b56bc421e36b
Every time I click the download button on my site, this function runs. I successfully get the download URL, and put it into the var urls.
But I get an error that I am not allowed access to this file and an invalid state error. This puzzles me because I am able to manually go to the link and right click -> save target as just fine.
Thank you so much for any suggestions, go easy on me I am new to jszip and Firebase. This is my code:
function download(downloadType) {
//alert(downloadType)
var storageRef = firebase.storage().ref(downloadType + "/" + "1.wav"); //file name
//todo: get 16 random file links, use for loop for thing below
storageRef.getDownloadURL().then(function(url) {
var downloadLink = url; //download link
//////////////////////// do download stuff here ////////////////////////
var zip = new JSZip();
var count = 0;
var zipFilename = "zipFilename.zip";
var urls = [
downloadLink
];
urls.forEach(function(url){
var filename = "filename";
// loading a file and add it in a zip file
JSZipUtils.getBinaryContent(url, function (err, data) {
if(err) {
throw err; // or handle the error
}
zip.file(filename, data, {binary:true});
count++;
if (count == urls.length) {
var zipFile = zip.generate({type: "blob"});
saveAs(zipFile, zipFilename);
}
});
});
thanks to #MikeMcDonald i fixed both of these errors with this post: Firebase Storage and Access-Control-Allow-Origin
I am now getting:
Uncaught Error: This method has been removed in JSZip 3.0, please
check the upgrade guide.
Looks like I can finally move on. Thanks!
So i am using the sql.js library i.e. the port of sqlite in javascript which can be found here https://github.com/kripken/sql.js.
This is my code to open and read the database that comes from a flat file store locally.
First the file a local file is selected via this HTML
<input type="file" id="input" onchange="handleFiles(this.files)">
The js code behind the scenes is as follows,
function handleFiles(files) {
var file = files[0];
var reader = new FileReader();
reader.readAsBinaryString(file);
openDbOnFileLoad(reader);
function openDbOnFileLoad(reader){
setTimeout(function () {
if(reader.readyState == reader.DONE) {
//console.log(reader.result);
db = SQL.open(bin2Array(reader.result));
execute("SELECT * FROM table");
} else {
//console.log("Waiting for loading...");
openDbOnFileLoad(reader);
}
}, 500);
}
}
function execute(commands) {
commands = commands.replace(/\n/g, '; ');
try {
var data = db.exec(commands);
console.log(data);
} catch(e) {
console.log(e);
}
}
function bin2Array(bin) {
'use strict';
var i, size = bin.length, ary = [];
for (i = 0; i < size; i++) {
ary.push(bin.charCodeAt(i) & 0xFF);
}
return ary;
}
Now this works and i can access all the columns and values in the database, however there is one column which is of type blob and that just shows up as empty. Any ideas of how i can access the contents of this blob?
The correct answer!
So what I was trying to ask in this question is simply how to read the contents of a column of type blob using sql.js. The correct answer is to specify the column names in the question and for the column that contains data of type blob, get its contents using the hex function i.e. select column1,hex(column2) from table. It was by no means a question about the most efficient way of doing this. I have also written a blog post about this.
Here is a slightly modified copy of the function responsible for initializing my sqlite database:
sqlite.prototype._initQueryDb = function(file, callback) {
self = this;
var reader = new FileReader();
// Fires when the file blob is done loading to memory.
reader.onload = function(event) {
var arrayBuffer = event.target.result,
eightBitArray = new Uint8Array(arrayBuffer),
database = SQL.open(eightBitArray);
self._queryDb = database;
// Trigger the callback to the calling function
callback();
}
// Start reading the file blob.
reader.readAsArrayBuffer(file);
}
In this case, file is a local sqlite database handle that I get from an HTML input element. I specify a function to call when a change event happens to that input and get the blob from the resulting event.target.files[0] object.
For the sake of brevity on my part I left some things out but I can throw together a smaller and more simplified example if you are still struggling.
The answer is: with kripken's sql.js, that you mentioned above you can't. At least as of today (may 2014). The original author doesn't maintain sql.js anymore.
However, I'm the author of a fork of sql.js, that is available here: https://github.com/lovasoa/sql.js .
This fork brings several improvements, including support for prepared statements, in which, contrarily to the original version, values are handled in their natural javascript type, and not only as strings.
With this version, you can handle BLOBs (both for reading and writing), they appear as Uint8Arrays (that you can for instance convert to object URL to display contents to your users).
Here is an example of how to read blob data from a database:
var db = new SQL.Database(eightBitArray); // eightBitArray can be an Uint8Array
var stmt = db.prepare("SELECT blob_column FROM your_table");
while (stmt.step()) { // Executed once for every row of result
var my_blob = stmt.get()[0]; // Get the first column of result
//my_blob is now an Uint8Array, do whatever you want with it
}
db.close(); // Free the memory used by the database
You can see the full documentation here: http://lovasoa.github.io/sql.js/documentation/