I'm using doctrine to my API REST that returns entities with large relations and large info. I'm trying to improve my doctrine queries performance and I have a big question. Is better to write relations with dbal queries or with XToX doctrine relations?
Thanks
My advice use OneToMany, ManyToOne, ManyToMany etc. relations. Cause when you use that relations, everything will be faster on ORM level.
Related
I can't find how to deal properly with complex databases using ORM on Android. I tried to find an Open Source project to see how it works but can find one that suits what i'm looking for...
I learned about relational databases some years ago and worked on SQL Server and Oracle databases, huge ones. The first things i learned when designing a database is to avoid having several times the same data. The second things i learned is never do in code what you can do with SQL. So I'm facing several problems with Android and ORM since it looks like you abolutely have to use an ORM in Android to be a good developer...
Let's take an example and say we have 100 buildings with 50 people in each of these building, all buildings has a different address. I want to get all people with their building address. I can't put this in one table else the same strings will exists many times in the database. Since on each adress there are 50 people if I use only one table I will have the same address string 50 times for each building, so I create another table with only buildings and make a relationship between these two tables. This is a trivial case but i saw many times Android app storing the same data many times in one or two tables.... what the point to use relational database if you replicate data ?
Again this is a simplistic example but when you have 20 or 30 tables with complex relationships the query in ORM styles can quickly become unreadable compare to SQL. Therefore not all SQL join types are generally supported by ORM. Then you use SQL raw query but what's the point to use an ORM since you can't use the object mapping since you're not returning a table you can map to a class but the result of a query... or maybe there is something I didn't understand. What the point to use an ORM if you don't use the relationnal object mapping advantage or make the queries difficultly maintainable ?
I saw a lot of code too where the ORM is used to get data in several tables and then the filtering and joining part is made using code... what's the point to use a relational database if you have to do this in code ? actually doing this some years ago what seen as the worse thing to do... but now I saw it so many times on Android...
So another solution is to create a View in the db and map my object to this view. I can use the power of SQL and the power of relationnal object mapping of the ORM. But several ORM doesn't support Views, like GreenDao who is one of the most used ORM today as far as I know...
All the example i can find here and there are not dealing with complex databases or has this kind of bad practices. Or at least it was condidered as bad practices for years... does it changed ?
So what's the best way to deal with "complex" databases on Android ?
I had to manage relationships between documents over a NoSQL engine (Couchbase), and I figured out this way to solve my problem. Here is my solution and the steps that let me realize it:
https://forums.couchbase.com/t/document-relationships-using-arrays-and-views-passing-though-graph-theory/3281
My questions are:
What do you think about this solution?
Have you ever used something like this? How is it working?
Are there any better ideas? Critical points of this solution should be helpful
Thank you.
Interesting post Matteo. After reading it I realized that you can possibly improve on few aspects:
Consider 1-1 node relationships. In your post you focus on N-N node
relationships (sure one can argue that 1-1 is a subset of
N-N)...however I think there is a potential of having a different (optimized) implememgtaion for 1-1 relationships. for 1-1 I use node key
value as a field in my json doc (e.g. user: {name:string, dob:date,
addressID:string})
Node key design to address relationships: You can encode in the key
value relationship information, e.g. key: "user#11", "user#11#address#123", "address#123#user#11", etc.
Data integrity aspects: Take into consideration missing complex
transactions. i.e. you can't mutate several documents in one
transaction. The design should compensate for that.
I have used similar solution in my model design for Couchbase in the past. Its now in production for several years already and its performing just fine (load is about 250 tps)...I was trying to avoid as much as possible creating complex node relations and ended up having very few 1-1 and 1-N types.
I tested out this solutions and works well. I like the flexibility of the 'always possible' N-N relationships, because you can simply add the relationship document when you need it without changing the application logic. There is a drawback: you need to implement your own application logic constraints to avoid relationships abuse.
I noticed that using arrays there isn't a great advantage compared to JSON objects and sometimes it may be useful to have other relationships data, for example the weight (or cost) of the relationship. So I suggest you to use a relationship document that as it's own type:
{
"type": "relationship",
"documents": ["key1", "key2"],
"all-the-data-you-need": { ... }
}
Looking at the performance there isn't so much difference using objects over arrays.
Hope this helps someone! ;)
I just want to know what are advantages/drawback of a bidirectional relation in Doctrine with Symfony ?
All my relations are bidirectional, but I'm not sure if this will not cause a problem...
Thanks.
As long as you don't mark relations as EAGER I think you're good. to go.
However, there is a small overhead since PHP has to, at least, create Proxy instances. Beware of this if you plan on serializing objects. Some serialization mechanism are programmed to resolve (load) proxy if they hit one. That would mean an extra round-trip to database server.
Bottom line: As you develop your model, ask yourself "Do I really need this?". It's super easy to add it later if you find yourself in that situation. Also, when it comes to OneToMany and ManyToOne, pay special attention to owning/inverse side concepts as it could introduce a number of WTFs/minute :)
Hope this helps you a bit...
I'm using symfony 2.4 and doctrine, I have struggled coming up with the right mapping for relations in my entities classes.
Question: Do I really have to do it? Or can I just use joins in my repositories classes? I feel more comfortable with the latter, are there any drawbacks if just use joins?
Answer: if you don't wont to do it why do you use doctrine? It's really powerful tool and in my opinion, one of the most important part of this library.
If you need help with defining relations please don't hesitate to put a short code here. We can find a solution. In the beginning it's always hard to understand, but after creating a very first well mapped entities it becomes to be pretty clear how and why to do that.
It's of course possible to avoid defining relations, but after-effects will be painful.
I have looked at NHibernate and EntitySpaces and they both seem to work differently.
In EntitySpaces, you define the database tables and table relationships and the classes are generated for you.
In NHibernate, you define the classes and the table relationships are generated for you. This is what I am looking for.
Are there any other ASP.NET ORMs that generate tables from classes like NHibernate?
Any recommendations?
DataObjects.Net also uses "Code first" (Model first) approach.
See http://wiki.dataobjects.net/index.php?title=Features
Linq to SQL can create the database table structures and relationships from the classes, with the dataContext.CreateDatabase() method.
Mindscape LightSpeed offers this ability - part of complete scheme round-tripping.
Hope this helps
http://www.mindscape.co.nz/blog/index.php/2008/06/17/schema-round-tripping-in-the-lightspeed-designer/
I prefer an approach that I have full control to generate what I need as well. In the case of ORMs I get classes that match my tables. I believe that using my own domain of objects that derives from my business and not the underlying data store is the right way to go. My class hierarchies that represent my business data should be 100% independent from the data store.
LightSpeed has a really good Visual Studio designer that supports both generating .NET entity classes from the database and updating the database from your .NET entities.
This is something that NHibernate does.
And on the subject (that Draemon) started. My personal view is that unless performance is your absolute 1st priority and all other things must suffer to make that happen (e.g. when writing software for a manufacturing fab), you will be better off working on the domain model first.
My reasoning: you spend a lot more time coding against the domain than you do against the database itself -- especially when using an orm. So spend that time wisely.
I had fairly good success working with Genome ORM. It does many jobs for you. You can first design your domain model and then generate the DB scripts out of that. Beside this Genome generates DTOs for you. It is pretty good at that and saves a lot of time of developers.
http://www.genom-e.com