Is possible with a for loop to look up the items in a folder and create corresponding objects in Processing?
Related
I'm writing my own R package to carry out some specific analyses for which I make a bunch of API calls to get some data from some websites. I have multiple keys for each API and I want to cycle them for two reasons:
Ensure I don't go over my daily limit
Depending on who is using the package, different keys may be used
All my keys are stored in a .csv file api_details.csv. This file is read by a function that gets the latest usage statistics and returns the key with the most calls available. I can add the .csv file to the package/data folder and it is available when the package is loaded but this presents two problems:
The .csv file is not read properly and all columns names are pasted together to create a single variable name and all values pasted together to create a single observation per row.
As I continue working, I would like to add more keys (and perhaps more details about the keys) to the api_details.csv but I'm not sure about how to do that.
I can save the details as an .RData file but I'm not sure about how it would be updated or read outside of R (by other people). Using .csv means that anyone using the package can easily add/remove some keys.
What's the best method to address 1, 2 above?
Is it possible to manage data stored amongst several files?
Let's say I have several files data1.realm, data2.realm, data3.realm, etc. containing objects with the same model. Is it possible to get a unique RLMRealm instance that will access the datas of all these files?
If not, what is the best way to handle this situation? Migration?
It's definitely possible to manage data stored amongst separate Realms, but it wouldn't be automatic. You would need to manage access to this data in your own app's logic.
RLMRealm instances themselves represent a single file on disk and cannot be dynamically created to reference combinations of other Realms. Once an RLMObject has been added to a parent RLMRealm, it cannot be moved/backed to another RLMRealm representing a different file.
It most likely depends on your specific use-cases, but the simplest solution would be to simply query for your objects in separate RLMRealm instances for each file, and placing the resulting RLMResults objects from each one in an NSArray.
While data can't be directly shared between Realms, you could use globally unique primary keys (For example NSUUID) to indicate relationships between objects in different Realms.
If you need, it's also possible to create Realmless copies of RLMObjects if you do end up wanting to move objects between Realms:
Dog *savedDog = [[Dog allObjects] firstObject];
Dog *copiedDog = [[Dog alloc] initWithValue:savedDog];
In my Windows 8/RT app I use SQLite DataBase (sqlite-net) witch store in Isolated Storage. In DataBase I have a lot of data, including files(images, pdf's and other) links. I get those links from web server. When I got link, I want to download file and store it locally.
My question is: what is the best way to store big number of files (100+)? One important think: I need to organize quickly find the desired file.
I have three ideas:
Create another DataBase only for files (I can't modify existing)
Create folder in IS and store here directly.
Create list of files and store it in IS.
Which would be better/faster? Or somebody have another great solution?
100 files isn't such a big number as you can easily store up to 100k files (or folders) in a single (NTFS) directory.
If you receive the files from a webserver then the question is whether the source makes sure there are no duplicate filenames. If this can't be assured, I'd recommend having a database table mapping from original filename and metadata to its hash (SHA256 or similar) and store the file with a filename corresponding to its hash.
Then, when using the file, you can pass pass it to the user using the original filename using the StorageFile API.
Going beyond 100k files, you could create a subfolder structure from the first two letters of the hash.
Either way, storing the file metadata in a database and the files in a directory has been the most useful approach for us in the past.
100 files with average size of 1MB is only 100MB.
Most people say that storing binary files in database is wrong and suggest storing files separately and only keep file names in database, but I think it is fine provided you know what you are doing and why.
Big advantage of storing files in database is that you keep files together with their properties logically in one place. Also, you can simply copy one file and this would backup everything.
Database also affords you transaction support. You may have some problems reading and writing BLOBs into database, but it is not very difficult.
I'm using Hadoop and working with a map task that creates files that I want to keep, currently I am passing these files through the collector to the reduce task. The reduce task then passes these files on to its collector, this allows me to retain the files.
My question is how do I reliably and efficiently keep the files created by map?
I know I can turn off the automatic deletion of map's output, but that is frowned upon are they any better approaches?
You could split it up into two jobs.
First create a map only job outputting the sequence files you want.
Then, taking your existing job (doing really nothing in the map anymore but you could do some crunching depending on your implementation & use cases) and reducing as you do now inputting the previous map only job through as your input to the second job.
You can wrap this all up in one jar running the 2 jars as such passing the output path as an argument to the second jobs input path.
I am sure there is no easy way to do this but I have ~400 excel files containing data. Each excel file contains the same headers. Is there an easy way of bulk inserting all 400 files at one time or in some form of batch process instead of doing in manually?
Thanks.
You could put your Excel files into a folder, then write a program to read their contents into a data table (either all at once or in batches) and then write the contents to DB using SqlBulkCopy class.