Web Scrape Images and Load Them into Google Cloud Storage - web-scraping

I'm trying to web scrape all of the images from a web page, load the jpg files into Google Cloud Storage, and then run each image through the Vision API.
Any suggestions on how to do this in a scalable and automated way? Sounds like we could use a cloud function. Any recommendations and sample code to get started would be appreciated.
Thank you!

Related

High Google Firebase Storage Cost for saving and downloading .mp4 files?

I'm saving videos to Firebase Storage and creating a URL for it which I saving into Cloud Firestore. So when ever someone watch a video I call the URL and Firebase downloads it. That increased my project costs enormously.
The question that I have is, is there a way to save the mp4 files cheaper?
And maybe also make it faster to download?
The best solution will be if I can like I used before. So in the new solution, also use the download URL which I saved in cloud Firestore. So that I don't need change too much. My Project is using Flutter in client side.
I created a mobile app, where user can upload videos for example like Instagram or TikTok.
I'm willing to pay if you freelancer and can help.
You can use Youtube or Vimeo to store your videos for free (Vimeo is limited to 500mb).
Then you can use a package from pub.dev to display these videos.
YouTube package (does not support to download the file, only playback)
Vimeo package (max 500mb, then you have to pay)
So both have their advantages and disadvantages.
It is possible to upload videos to YouTube using Google API. There is an example done using flutter which can be ideal for your use-case. Have a look at this video on youtube.

Is it weird idea to use youtube instead of firebase storage service for my app's video cloud storage?

I'm now using Spark Plan of Firebase which is totally free. And recently getting my app involved with videos, I found that bandwidth is getting too high near my free plan's limitation, which means I need to pay for larger plan when i release my app to market. Then this idea popped up. I know that paying for cloud storage is normal and good idea for the service, but using youtube and embedding that youtube link into app ui doesn't sound impossible. Then it would be free..! Additionally, i found weird official presentation by Google: https://www.youtube.com/watch?v=N2OG1w6bGFo&feature=youtu.be&t=2060
How do you think of using youtube as free cloud storage for our app? Is it just weird and semi-wrong idea?
It's not very efficient storage and bandwidth wise, and it’s possible there might be a number of issues with the video quality messing up the data. It’s also possible that Youtube will flag you for unusual upload patterns.

How to run on demand scripts on server-side that could create a new file via Google Cloud Endpoints?

I'd like to ask if it's possible to run scripts (bash, etc) that could read and create files, from Google Cloud Endpoints. Which Google products are appropriate? How to configure this? (Java)
(There are many google products, but the development documentations seem to be difficult to find. Even a google search could return too many different unrelated results.)
Thank you.

Is it possible to embed Firebase Analytics data from my apps into a webpage?

I've searching a lot about someway to embed Firebase Analytics from my apps data into the backend UI from my website.
Is it possible to automate this process through some specific API methods to show those metrics from my Firebase Apps (like the number of installations or even the audience data)?
Thanks in advance!
Update: There now is an API to retrieve the analytics report data. See my answer here for details: Is there any api for dashboard analytics data?
The other two options are:
Use the "Export to CSV" option for the analytics in the Firebase console. This is a manual operation though, there is no public API.
Link your Firebase project to BigQuery, to automatically export the data and then use for example Data Studio to generate the charts you want.

Managing Azure Face API images

I'm playing with the Azure Face API and enjoying it very much.
I was wondering - where do the images I upload via the API (for example - in order to create a Person Group) stored? Can I view them or download them?
Thanks!
Image data is stored by the Azure by referencing them with a persistedId.
According to the discussion, there is no way to view or download the image:
https://cognitive.uservoice.com/forums/356325-apis/suggestions/14708886-possibility-to-retrieve-the-image-for-a-faceid

Resources