This is how I delete thousands of entities in datastore: First, get 1st entity. If 1st entity exist, fetch 500 entities to delete. Second, defers deletealltarget again and again until 1st entity does not exist.
def deletealltarget(twaccount_db_key):
target_db = model.Target.query().filter(ndb.GenericProperty('twaccount_key') == twaccount_db_key).get()
if target_db:
target_dbs = model.Target.query().filter(ndb.GenericProperty('twaccount_key') == twaccount_db_key).fetch(500,keys_only=True)
ndb.delete_multi(target_dbs)
deferred.defer(deletealltarget,twaccount_db_key)
Is there any better way?
You could use delete_multi_async asynchronously, instead of using defer, but besides that, you are doing good with this way. For example, you are using other advices already told, like using keys_only.
Google recommends using this Dataflow template for bulk deletion., but I don't know if it fits your scenario.
Related
What's the easiest way to delete an entire partition in cosmos Db assuming that I'm using spirngboot with SQL API?
I have a class marked with the #Repository that extends the CosmosRepository and I want to delete every item from a particular partition.
I know that with CosmosClientBuilder I could do something like:
cosmosDbClient.getDatabase(dataBaseName)
.getContainer(container)
.deleteAllItemsByPartitionKey(
PartitionKey("partitionKey0001"),
CosmosItemRequestOptions())
Is it possible to access the container from the repository?
I don't want to use stored procedures for something that should be easy to do.
Thank you
Microsoft is releasing this feature since year now.
Its yet in preview state and you will need to opt to this before using
I've been using redux-crud, and really like it. I'm still very new to Redux itself, so the following question may seem a bit noob.
I'm at the point where I want to fetch just a single record freshly from the server when I enter the edit form for it. I can't rely on the record that might have been previously fetched into the state, to be the most accurate representation of it for editing purposes.
Based on my current understanding, it doesn't seem that redux-form is suited to fetching singles, rather it seems to suggest that I pull the record for edit out of the collection of records already in the state (previously fetched with the out of the box crud actions and reducers).
I have a type of record called Providers.
Am I right to say that I'm going to have to create a separate set of fetch actions and reducers that are suited to singular fetching?
So where Redux Crud would give me PROVIDERS_FETCH_SUCCESS, I would then need to go on and implement PROVIDER_FETCH_SUCCESS in a singular fashion? Or is there a simpler way out of the box with Redux Crud that I'm not seeing clearly?
Thanks!
redux-crud will manage updating items for you when you need to fetch individual entities. actionCreatorsFor and reducersFor both take optional secondary arguments to specify what
reduxCrud.actionCreatorsFor('providers', {key: '_id'});
reduxCrud.Map.reducersFor('providers', {key: '_id'});
You can then mark the action as "replace" to replace existing items that have matching key fields;
fetchSuccess(providers, {replace: true});
Or
fetchSuccess([provider], {replace: true});
I've a project with a huge amount of data. I need to make query based on user role/group/permission. This means that a query like
$fooRepository = $this->getDoctrine()
->getManager()
->getRepository(Foo::class)
->findAll();
should return different records if done by a ROLE_SUPER_ADMIN or by ROLE_USER. Also, I need to filter record based on relations and so on.
I've different solution in mind:
inject user role inside the repository's query
create a role based repository
create a query for each role
Inject user role in repository's query
In this case each repository should be responsible to provide right data. This is a solution similar to this. In that solution record are filtered by tenant.
Create role based repository
In this case I'll need to create different repositories and instantiate them differently. But I don think this can be easy in doctrine? while I am writing, ... I am thinking this is an exaggerated solution.
Create queries for each roles
At the moment I think this is the more natural way to do queries. I just imagine that a repository should contain
+ findAllStuffForGuestRole()
+ findAllStuffForAdminRole()
Each time I need to add a query, I MUST create different queries.
I think Doctrine filters are the cleanest and the simplest way to solve your problem. See the documentation
This is actually very straight forward with Symfony/Doctrine. Here's a great page in the official docs that explains it better than I could: http://symfony.com/doc/current/doctrine/repository.html
Iterations of this question have been asked in the past, but this presents unique challenges as it combines some of the issues in one larger problem.
I have an entity(User) that is used as the user class in my application, then I have another entity (UserExtra), in a one-to-one relationship with the user entity, UserExtra's id is the same as User. The foreign key is the same as the primary key.
When the user object is loaded (say by $this->getUser() or by {{ app.user }}, the UserExtra data is also loaded through a join. The whole point of having two entities is so I don't have to load all the data at once.
I even tried defining a custom UserLoaderInterface/UserProviderInterface Repository for User, making sure that refreshUser and loadUserByUsername would only load the User data (I'd like for the UserExtra data to sit in a proxy unless I explicitly need it) but when Doctrine goes to Hydrate the object, it issues an extra query to load the UserExtra data, thereby skipping the Proxy status.
Is there a way out of this?
there are many solution for your issue:
1) Change the owning side and inverse side http://developer.happyr.com/choose-owning-side-in-onetoone-relation - I don't think that's right from a DB design perspective every time.
2) In functions like find, findAll, etc, the inverse side in OneToOne is joined automatically (it's always like fetch EAGER). But in DQL, it's not working like fetch EAGER and that costs the additional queries. Possible solution is every time to join with the inverse entity
3) If an alternative result format (i.e. getArrayResult()) is sufficient for some use-cases, that could also avoid this problem.
4) Change inverse side to be OneToMany - just looks wrong, maybe could be a temporary workaround.
5) Force partial objects. No additional queries but also no lazy-loading: $query->setHint (Query::HINT_FORCE_PARTIAL_LOAD, true) - seams to me the only possible solution, but not without a price:
Partial Objects are a little bit risky, because your entity behavior is not normal. For example if you not specify in ->select() all associations that you will user you can have an error because your object will not be full, all not specifically selected associations will be null
6) Not mapping the inverse bi-directional OneToOne association and either use an explicit service or a more active record approach - https://github.com/doctrine/doctrine2/pull/970#issuecomment-38383961 - And it looks like Doctrine closed the issue
this question may help you : one to one relation load
Just dipping my toes into Linq2sql project after years of rolling my own SQL Server DB access routines.
Before I spend too much time figuring out how to make linq2sql behave like my custom code used to, I want to check to make sure that it isn't already "built" in behavior that I can just use by setting up the relationships right in the designer...
Very simple example:
I have two tables: Person and Notes, with a 1 to many relationship (1 Person, many notes), linked by Person.ID->Note.PersonID.
I have a stored procedure (all data access is done via SP's and I plan on continuing that) which makes the Link2SQL a bit more work for me.
sp_PersonGet(#ID int) which returns the person record and sp_PersonNotesGet(#PersonID) which returns a set of related notes for this person.
So far so good, I have an object:
Dim myPerson As Person = db.PersonGet(pnID).Single
and I can access my fields: myPerson.Name, myPerson.Phone etc.
and I can also do a
Dim myNotes As Notes = db.PersonNotesGet(pnID)
to get a set of notes and I can iterate thru this list like:
For Each N As Note In myNotes
( do something)
Next
This all works fine...BUT....What I would prefer is that if I call:
myPerson = db.PersonGet(pnID)
that I also end up with a myPerson.Notes collection that I can iterate thru.
For Each N As Note In myPerson.Notes
( do something)
Next
Basically, Linq2SQl would need to call 2 stored procedures each time a Person record is created...
Is this doable "out of the box", or is this something I need to code around for myself?
This is normally what we would call child collections and they can be eager loaded or lazy loaded. Read these:
http://davidhayden.com/blog/dave/archive/2009/01/08/QuickExamplesLINQToSQLPerformanceTuningPerformanceProfilingORMapper.aspx
http://www.thinqlinq.com/default/Fetching-child-records-using-Stored-Procedures-with-LINQ-to-SQL.aspx
It uses partial classes. You can add your own "Notes" property to your Person class and initialize it in it's GETter function. This would be better than populating the notes every time you load a person record.
I believe that you can do this more or less out of the box, although I haven't tried it -- I don't use stored procedures with LINQ. What you would need to do is change the Insert/Delete/Update methods from using the runtime to use your stored procedures. Then you'd create an Association between your two entity tables which would create an EntitySet of Notes on the Person class and a EntityRef of Person on the Notes class. You can set this up to load automatically or using lazy loading.
The only tricky bit, as far as I can see, is the change from using the runtime generated methods to using your stored procedures. I believe that you have to add them into the data context as methods (by dropping it on your table from the server explorer in the designer) before it is available to use instead.