As the question in the title says.
My database file could be pretty large so I don't want to make copies of it unnecessarily and I certainly don't want to build it in situ.
If the db file is a flutter asset, is there a way for sqlite to access it directly?
I've seen suggestions that I should copy the raw data of the asset into a file and then access it but that is a waste of storage. Or can I then delete the asset?
Is there a simple way of deploying the database as something other than an asset, ie as a raw file?
iOS (and, I think, Android?) will require you to copy the file into the app's working directory first. This is standard practice, and it's part of working within a protected file system. If somehow it's a deal killer to have copies in both your app bundle (consider this the pristine "master") and app documents folder (the "working copy"), I suppose you could also download it from a server on initial app launch, but... time is money.
It's truly not that big a deal, though. To do so, ensure the file in included in the app bundle via your pubspec.yaml file:
flutter:
assets:
- assets/stored_data.db
Then, before opening the database, copy it from the app bundle to your documents directory:
Edit: The following apparently fails to copy large files; see iKK's comment below if you experience such issues, as it looks like he's found a native solution. For smaller files, however, this should work fine.
// Create a new file within your document directory (Probably want to check whether it already exists first...)
Directory documentsDirectory = await getApplicationDocumentsDirectory();
String path = join(documentsDirectory.path, "working_data.db");
ByteData data = await rootBundle.load(join("assets", "stored_data.db"));
List<int> bytes = data.buffer.asUint8List(data.offsetInBytes, data.lengthInBytes);
await new File(path).writeAsBytes(bytes);
Now that you have the database within your app's document directory, you should be able to open it from the path. If you're using the tekartik/sqflite package (which is where I originally got these instructions), for instance, you can simply:
Database db = await openDatabase(path); // Opens SQL database, working_data.db
There's no reason (or ability) to delete the pristine copy from the app bundle (which would alter the original binary -- a huge nono). Eventually, the OS itself may end up offloading such dreck to a cloud server or other memory-management service, but that's for the platform to decide.
Related
still new to flutter and firebase. I understand how to store and retrieve images and display it on the app.
But how do I go about synchronizing files from the app's local asset folder to an asset folder stored in firebase storage? My intention is to check the cloud folder if a new image like an icon is recently uploaded, and download it to the app's local folder. If a file is removed in the cloud storage, it should also remove it from the local assets folder, mirroring it.
I need a way to compare local AssetManifest.json to the one on firebase storage. I Just need a little direction/algorithm to start with. Thanks for the help!
There is nothing specific built into Cloud Storage's Firebase SDK for this, so you'll have to build it in your own application code.
Using only the Cloud Storage for Firebase API
If you're just using Cloud Storage, you'll have to:
List all files that you're interested in from Storage.
Loop over the files.
Get the metadata for each file and check then the files was last updated
Compare that to when you wrote the local file.
Download the file if it is new or modified.
This approach will work, but it requires quite some calls to the Storage API, because there's no specific API to give you files that were modified since a specific date/time.
Storing metadata in a cloud database
You could also consider storing the metadata in a cloud database, like Firebase's Realtime Database or Cloud Firestore, and then use the query capabilities of that database to retrieve only files that were modified since your device last synchronized.
The recipe then becomes:
Determine when we last synchronized, which is a value you'll want to store in local storage/shared preferences.
Execute a query to the database to determine which files were added/modified since then.
Loop over the query results and...
Download each file that was modified.
In here, only step 2 and 4 make calls to the external APIs, so it is likely to be faster and cheaper to execute (but more work for you to write initially).
I have a local database file being put in my assets and resources folder. The database file gets copied as a local database after I create all the tables. If I release a new version with a new updated database file in the assets & resources file will the database overwrite the previous one? If so what happens with the local database I have already created in the user's app?
Do I have to delete all the tables upon new version start up then recreate them with the new data? Or can they be merged?
I am coming to an end of coding my first app and I wanted to take this into consideration. Trying to future proof as much as possible.
I have a SQLite .db file that I want to access through sqflite on Flutter. Where in the Flutter project am I supposed to put it so that I can access it both on Android and iOS? How do I make sure that it's shipped with the apk? All examples that I found assume that the db needs to be created from scratch at the first launch.
You can put the db file in your assets folder and declare it in your pubspec.yaml. On startup you can write it out to disk and then use that path with your connection string to connect the db.
You can read from assets using
var dbContent =
await rootBundle.load('assets/database/mydb.db');
Then write it out to your file system and go from there.
I've found that this problem is related to:
https://stackoverflow.com/a/51387985/3902715
Credits to R. C. Howell
I have been creating a video uploader using meteor and have been using CollectionFS to store the files. Unfortunately under heavy load it takes extended amounts of time to load the videos to display, all of the video files are around 50MB. In order to fix the issue of strain on the collection I want to save all files currently in the CollectionFS collection to the drive so I can place them on a CDN.
I do not know how to save the files to the hard drive, so any enlightenment on the subject would be helpful. The strain on the server is forcing meteor to run out of memory a little too often.
This is in fact a mongodb question rather than meteor. Taken from the official documentation, the following example shows you how to do this for one file. Of course, you can use find instead of findone and iterate over the result to write each individual file.
// returns GridFS bucket named "contracts"
GridFS myContracts = new GridFS(myDatabase, "contracts");
// retrieve GridFS object "smithco"
GridFSDBFile file = myContracts.findOne("smithco");
// saves the GridFS file to the file system
file.writeTo(new File("/tmp/smithco.pdf"));
There is also an official mongofiles utility that you can use for the same purpose:
mongofiles -d records get smithco.pdf
I want to import csv file(already uploaded in blob storage) in Azure.
For example I have uploaded test.csv on blob storage, now I just want to import that test.csv file in .net(azure) and after importing I will insert that data into azure database. I am using C# .net. Please suggest how can I achieve this. I want to follow below steps:-
Creating a cvs file with all rows.
Upload it as blob.
Parse it with a Worker role and insert it in the sql azure db.
Thanks.
A bit more clarification around your question would be helpful. Are you trying to upload a file to Azure blob storage? Download it from there for your app to consume? What language(s) are you using?
There are plenty of examples of loading files into and pulling them from Azure blob storage using .NET at least a handful for doing it with Java or PHP.
I you can clarify what you're trying to do, I'd be happy to point you at the appropriate ones. :)
-- answer based on comment update --
The steps for retrieving the blob are fairly straight forward:
1) create your Azure storage client using your azure storage credentials
add a using clause:
using Microsoft.WindowsAzure.StorageClient;
get a client for accessing blob storage:
CloudBlobClient tmpClient = new CloudBlobClient("<nameofyourconfigsetting>");
get a referrence to the blob you want to download:
CloudBlob myBlob = tmpClient.GetBlobReference("container/myblob.csv");
2) read the blob & save to a file
myBlob.DownloadToFile("<path>/myblob.csv");
The save location can be the %temp% location or if its a large file you may want to allocate some local storage space and put it there. The other thing you want to keep in mind is that if you are doing this in a role instance, you'll need to make sure you have measures in place to prevent two instances from concurrently trying to process the same file. If the file is small enough, you can probably even keep it as a memory stream and process it that way. If this is the case, you can use the DownloadToStream property of the CloudBlob object.
For additionally reading, I'd recommend checking out the MSDN library for the details on the StorageClient and CloudBlob contains. Additionally, the Windows Azure Platform Training Kit has some good labs to help you get a better understanding of how Azure Storage works.