I tried googling but I can only find information about Google drive scanning files with file size less than 25MB. Does firebase storage do the same? I'm planning to build an app and want to ensure that users don't upload infected files. I'm checking if I have to implement virus scanning myself.
Cloud Storage isn't concerned with, and will not change the contents of, the data you put in it. You do, however, have an obligation to honor the Google Cloud terms of service.
If you want to scan the files for whatever reason, you have to do that yourself.
Related
Since the node v10 changes to Cloud Functions, one thing that we've come to accept is a default storage bundle within our GCP of which accrues a cost of ~$0.03 a month. It is believed to be Docker cache files that store the Cloud Functions, Node Modules, and Hosting Revisions.
But on an almost empty project, how can you have 535MB in storage when the source is only 83MB total?
no hosting
2 cloud functions
What are the contents of the Files? is it overhead for the revision history? is there any reasonable way to minimize it? I get asked these questions many times and I do not have an answer that I feel comfortable with.
It's the entire container image, so I imagine it contains lot of things that you don't deploy yourself - like the operating system the code runs on, and the runtime of the language you use (Node.js if you're using Firebase to deploy).
I have deployed my site with Firebase using my customed domain.
I checked my db usage and it seems fine (couple of megas) but then I go the hosting section and then the usage tab and see almost 1GB downloads, do you know what do the downloads in the hostage section mean?
It means that almost 1GB of data was downloaded from Firebase Hosting in your project.
If this is higher than you expected, you might want to check for the size of your files. Do you have any images? How big are those? How much data will each client read from Firebase Hosting? How many clients do you have? If you have a few large files that everyone loads, it adds up quickly.
I was thinking of making an app which plays a specific set of small audio files stored away in a firebase database, or any other database on the cloud.
The number of files may exceed hundreds, so wrapping all the files in the app might not be the best of ideas.
I'd really like to know how to go about this problem.
Downloading only the required file from firebase when it's clicked and playing as its being downloaded all the while caching it for future playback seems plausible, but I'm not quite sure how to implement it.
I'd appreciate a few pointers towards this, thanks
Yes. You are thinking in the right direction.
You might use these things.
App UI - Ionic2 will provide you a very nice and easy to implement platform to create a UI. Please refer this documentation for the basics and details about using Ionic2.
Local Storage - You can use this to store downloaded files. In any app where data is relatively large, using local storage is the smart choice. This helps you reducing the size of the initial app to be installed and download the content as and when needed.
A well defined database - Now, whether to use non-structured (No-SQL) or structured (SQL) is the first choice you have to make.
If it's just content - audio files download and play with no complex cross querying the database, then you can choose to use non-structured (No-SQL) database like Firebase database.
But, if you have good requirements for structuring data, query it with constraints like "Give me all the audio files list where a particular user has played it in last 10 days" or "Give me all the users who has played/downloaded a particular audio file more than 10 times" and so on, then you better use structured (SQL) database like PostgreSQL.
RxJS - Now, this might not be very important to do, but, if you use this from the start, it's a good choice. Advantages are, e.g. you might not have to wait for all the file to get downloaded before playing it. Use Observables and Promises for such a mechanism.
Could help with the specifics when needed. Hope this helps. :)
I maintain a web application (ASP.NET/IIS7/SQL2K8/Win2K8) that needs to access documents, actually hundreds of thousands of documents, and growing. Currently, they are all on a Windows 2K8 Server fileshare, being accessed by UNC path (SMB). The files are in a single flat directory and I'm trying to plan how to best improve this solution. I don't want to use the SQL Filestream attribute as it would be significant effort to migrate it all into that, and would really lock in to SQL Server. I also need to find a way to replicate the data for disaster recovery, so perhaps a solution can help with that too.
Options could be:
Segment files into multiple directories?
Application would add metadata for which directory it's on (or segment by other means)
Segment files into separate servers? (virtualize)
Backup becomes more complicated.
Application would add metadata for which server it's on
NAS Storage
SAN Storage
Put a service (WCF) in front of the files and have the app talk to the service
bonus of being reusable across many applications
Assuming I'm going to store on filesystem and not in database (I've read those disccusions here), which would be a more scalable solution?
You've got a couple issues:
- managing a large volume of (static?) files
- preparing for backups and disaster recovery of said files
I'll throw this out there, even though I'm not a fan of the answer, but you might poke around with the free SharePoint 2010 Foundation that's included with server 2k8. If you're having issues with finding the documents you need (either by search, taxonomy via tagging or other metadata) as well as document expiration and you don't want to buy a full blown document management system, this might be a solution. Of course it introduces new problems...
If your only desire is to have these files available to spit out on the web, then the file store like you're using now really is the simplest solution. For DR/redundancy purposes, I'd look at a) running them on a raid/SAN of some sort and b) auto-syncing them with the cloud (either azure or amazon). For b) you can get apps that make the cloud appear as a mapped drive and then use an rsync type software to keep the cloud up to date.
If you want to build something new and cool, you might think about moving the entire file archive into the cloud and just write a table in a db to manage the file name, old location, new cloud location and a redirector code that can provide the access tokens to requestors.
3 different approaches... your choice.
I am working on an application that creates video files and stores them in a folder in the C:\ drive. I speculate that there will be a large number of these files in the future and we would run out of disk space at some point of time (on our VPS). When the time comes that we have to upgrade, we either plan to use one of the Cloud providers to store files or our existing provider can add another disk (say D:\ drive).
Either way, I would want to design the app now in a way that in future, moving to different locations would not be an issue and would be transparent to the end user.
The code that creates these files supports 2 ways:
myObj.SetOutputToDisk(<path to store>); or
myObj.SetOutputToMemoryStream(ms);
If we go with the Cloud architecture, I assume we might have the following combination:
Cloud Files + Existing VPS or
Cloud Files + Cloud Windows Server
Given the unknowns at this time, how would I go about designing this?
Serve the files up from a subdomain. Say: media.yourdomain.com.
That way, you can trivially repoint DNS records to the new storage provider at some point in the future.
Also, I'd recommend storing the media files on another physical disk to the OS disk. So have a D:\ drive and store the media there.
You might want to look at the Managed Extensibility Framework as a way of adding extensions to your app for new storage methods without the need to rebuild the whole thing.
You need some way to record the storage location and method used, I'd expect some kind of database store that you could migrate to the cloud later if required.
Your question is very vague, you haven't put much work in yourself and as such you are unlikely to get the level of detail you are hoping for in the answers. At least try to implement the system and then ask specific questions around issues that you are having problems with.