I want to create a folder structure in S3 with Terraform so that it defaults to all environments. The idea is to have structures similar to these:
locals {
folder = [
{
provider = "provider-1",
process = "batch-process",
final = ["people","cart"]
},
{
provider = "provider-1",
process = "online-process",
final = ["log","order"]
}]}
Bucket-1
provider-1
batch-process
people
cart
online-process
log
order
I managed to create a list with all the S3 directories I would like to have, but I know this is not the most efficient way.
I tried to follow some examples found here but they returned an error. What is the correct way to do this iteration without having to write the entire directory?
Related
How to create a S3 bucket policy for the multiple existing manually created (not through terraform) s3 buckets using terraform
For Example : I have A,B,C buckets created manually and now I wanted to add a s3 buket policy for all 3 S3 buckets , How can I achieve this through Terraform? Can we use some loop sort of thing here Please advise
Do you want the same policy applied to each bucket? That doesn't really work because the policies need to include the bucket name in the resource section. It's a weird limitation of S3 bucket policies that I've never quite understood, you can't use a wildcard (*) for the bucket name.
Anyways, you could do something like this, where you dynamically set the bucket name in the policy.
I'm just typing this from memory so please excuse any syntax errors.
locals {
// The names of the buckets you want to apply policies to
buckets = [
"mybucket",
"anotherbucket",
]
}
// Create a unique policy for each bucket, using
// the bucket name in the policy.
data "aws_iam_policy_document" "bucket_policies" {
for_each = toset(local.buckets)
statement {
actions = [
"s3:PutObject"
]
resources = [
"arn:aws:s3:::${each.key}/*"
]
principals {
type = "AWS"
identifiers = [
var.my_user_arn
]
}
}
}
// Apply each policy to its respective bucket
resource "aws_s3_bucket_policy" "policies" {
for_each = toset(local.buckets)
bucket = each.key
policy = data.aws_iam_policy_document.bucket_policies[each.key].json
}
I have a collection called access using the client's id a the document id and a list of ids called sites in an array.
I want to access the list of ideas in sites, using a model and store them in a list of string. the list will be different lengths depending on the sites each client has so I just want a list of the ids back, I can not get this to work. Where I am at, is I can store the data and if I print it I can see the ids however when I try to use them below it is not being able to see them.
If I manually insert a string list it all works, so I am guessing it is how I am working with the array.
Any help with this would be great! I am still a novice, Thank you
This is the model I am using
class Access {
List<String> sites = List<String>();
Access({this.sites});
Access.fromJson(Map <String,dynamic>parsedJson,)
: sites = (parsedJson["sites"] as List)?.map((e) => e as String)?.toList();
}
And this is where I am using it to only get the data for the ids within sites
QuerySnapshot document = await companySiteCollection.where(FieldPath.documentId,whereIn:**access.currentAccess.sites** ).getDocuments();
document.documents.forEach((doc){
Sites sites = Sites.toJson(doc.data, doc.documentID);
_sitesList.add(sites);
print(" ${_sitesList.length.toString()} database print");
});
siteNotifier.siteList = _sitesList;
}
I am working on using the Firebase database in a Unity project. I read that you want to structure your database as flat as possible for preformance so for a leaderboard I am structuring it like this
users
uid
name
uid
name
leaderboard
uid
score
uid
score
I want a class that can get this data and trigger a callback when it is done. I run this in my constructor.
root.Child("leaderboard").OrderByChild("score").LimitToLast(_leaderboardCount).ValueChanged += onValueChanged;
To trigger the callback with the data I wrote this.
void onValueChanged(object sender, ValueChangedEventArgs args)
{
int index = 0;
List<string> names = new List<string>();
foreach (DataSnapshot snapshot in args.Snapshot.Children)
{
root.Child("players").Child(snapshot.Key).GetValueAsync().ContinueWith(task =>
{
if (task.Result.Exists)
{
names.Add(task.Result.Child("name").Value.ToString());
index++;
if (index == args.Snapshot.ChildrenCount)
{
_callback(names);
}
}
});
}
}
I am wondering if there is a better way to do this that I am missing? I'm worried if the tasks finish out of order my leaderboard will be jumbled. Can I get the names and scores in one snapshot?
Thanks!
You can only load:
a single complete node
a subset of all child nodes matching a certain condition under a single node
It seems that what you are trying to do is get a subset of the child nodes from multiple roots, which isn't possible. It actually looks like you're trying to do a join, which on Firebase is something you have to do with client-side code.
Note that loading the subsequent nodes is often a lot more efficient than developers expect, since Firebase can usually pipeline the requests (everything goes over a single web socket connection). For more on this, see http://stackoverflow.com/questions/35931526/speed-up-fetching-posts-for-my-social-network-app-by-using-query-instead-of-obse/35932786#35932786.
I have the following models defined in sequelize:
Library, MediaParentDir, MediaSubDir and MediaSubDirEpisodes
The first three hold information about directories on the system and the last one holds information about files in a particular directory on the system.
The associations are as follows:
Library.MediaParentDirs = Library.hasMany(models.MediaParentDir, {onDelete: 'CASCADE'});
Library.MediaSubDirs = Library.hasMany(models.MediaSubDir, {onDelete: 'CASCADE'});
MediaParentDir.MediaSubDirs = MediaParentDir.hasMany(models.MediaSubDir, {onDelete: 'CASCADE'});
MediaSubDir.Episodes = MediaSubDir.hasMany(models.Episode, {onDelete: 'CASCADE'});
And this is how I populate the database on first run:
db.Library.find({
where: lib
}).then((existingLib) => {
let includes = [{
model: db.MediaParentDir,
include: [{
model: db.MediaSubDir,
include: [db.Episode]
}]
},
{
model: db.MediaSubDir,
include: [db.Episode]
}
];
let mediaParentDirs = removeIgnored(library.getMediaParentDirsFrom(lib))
.map((parentDir) => {
parentDir.MediaSubDirs = removeIgnored(library.getMediaSubDirsFrom(parentDir));
parentDir.MediaSubDirs.map((subDir) => {
subDir.Episodes = removeIgnored(library.getMediaSubDirEpisodesFrom(subDir));
return subDir;
});
return parentDir;
});
let mediaSubDirs = removeIgnored(library.getMediaSubDirsFrom(lib))
.map((subDir) => {
subDir.Episodes = removeIgnored(library.getMediaSubDirEpisodesFrom(subDir));
return subDir;
});
let updatedLib = db.Library.build({
name: lib.name,
path: lib.path,
type: lib.type,
// Add all media parent dirs and child sub dirs under media parent dirs
MediaParentDirs: mediaParentDirs,
// Add all media sub dirs directly under library
MediaSubDirs: mediaSubDirs,
}, {
include: includes
});
if (!existingLib)
return updatedLib.save();
// Record already exists. Update library data.
});
In the code above, I'm reading the library directory and gathering all the information about MediaParentDirs and other models mentioned previously. Finally, I build a Library instance with all the nested associations defined.
Now, if a library already exists, I need to update the data associated to it and its models. I already tried a few things:
Library.upsert() but this doesn't update the associations.
Library.update() same as above.
embed.update() from https://github.com/Wsiegenthaler/sequelize-embed but this requires me to supply object IDs explicitly
Is there any other way I could update the associated model instances?
Any help would be appreciated. Thanks.
Sequelize automatically adds setter methods to models that can be used to update associated data in the database.
Following is the code that I use to list the methods. Add this in index.js after associations are defined and restart node server.
for (let assoc of Object.keys(db[modelName].associations)) {
for (let accessor of Object.keys(db[modelName].associations[assoc].accessors)) {
console.log(db[modelName].name + '.' + db[modelName].associations[assoc].accessors[accessor]+'()');
}
I have an app based on cosmicmind graph database published on the AppStore.
It saves data locally in the shared sandbox, in order to share the database with the today extension.
In the new update I'm trying to add CloudKit feature and it works great, but I have two problems.
I'd like that users who already have the app could sync data that are stored locally in the iCloud database, but I don't really know how to do it.
The second problem is the today extension. Since the cloud data are stored in a second SQLite file, the extension that was working perfectly before, now returns an empty array.
Any help will be really appreciated!
Thank you
EDIT
Thank you for you fast answer.
This is the code I used locally in my datamanager:
func startDataManager() {
let search = Search<Entity>(graph: databaseManager2).for(types: "ElencoItem")
elencoItems = search.sync(completion: nil)
}
and this is the code I use now with icloud:
func startDataManager() {
databaseManager = Graph(cloud: "MyContainer", completion: { (done, error) in
if done {
debugPrint("ok")
let search = Search<Entity>(graph: self.databaseManager).for(types: "ElencoItem")
self.elencoItems = search.sync()
} else {
guard let error = error else { return }
debugPrint("error iCloud: \(error.localizedDescription)")
}
})
I also added all delegate methods and it works perfectly.
I solved the issue with today extension.
The problem was that the extension didn't see the array saved for cloud version. I saved like this:
let nuovo = Entity(type: "ElencoItem", graph: DataManager.shared.databaseManager)
nuovo["value"] = value
Now I save cloud data to a new array like:
let search = Search<Entity>(graph: DataManager.shared.databaseManager).for(types: "ElencoItem")
for item in search.sync() { {
let nuovo = Entity(type: "ElencoItem")
nuovo["value"] = item["value"]
DataManager.shared.elenco.append(nuovo)
DataManager.shared.databaseManager2.sync()
}
and the extension works great.
I'm trying now to migrate data saved locally to cloud but it doesn't work.
Data were saved locally like this:
let nuovo = Entity(type: "ElencoItem")
nuovo["value"] = value
I do the same as above to move data to cloud save like:
let search = Search<Entity>(graph: self.databaseManager10).for(types: "ElencoItem")
for item in search.sync() {
let nuovo = Entity(type: "ElencoItem", graph: DataManager.shared.databaseManager)
nuovo["value"] = item["value"]
DataManager.shared.elencoItems.append(nuovo)
DataManager.shared.databaseManager.sync()
print(DataManager.shared.elencoItems)
}
When I print the array after the for loop I can see all data, but after closing the app I can't find data anymore.
Any solution?
Thanks
So Graph has a sync mechanism with iCloud. If you use that, you would only need to write a migration function to move your data over, and then update your app to tap into the iCloud API. For the Today extension... not sure what that is.
Problem solved.
I moved the migration function from the datamanager to the first viewcontroller loaded, removed the print statement and it works perfectly!