Related
There are many questions about seeding many-to-many relationships in Entity Framework. However, most of them are extremely old, and many-to-many behavior has changed significantly in EFCore5. The official docs recommend overriding OnModelCreating to implement ModelBuilder.Entity<>.HasData().
However, with the new many-to-many behavior (without explicit mappings), I can find no clear path to seed the intermediate tables. To use the example of this tutorial, the BookCategories class is now implicit. Therefore, there is no path to explicitly declare the intermediate table values while seeding.
I've also tried simply assigning the arrays, e.g.,:
public class Book
{
public int BookId { get; set; }
public string Title { get; set; }
public ICollection<Category> Categories { get; set; }
}
public class Category
{
public int CategoryId { get; set; }
public string CategoryName { get; set; }
public ICollection<Book> Books { get; set; }
}
And then at seed time:
Book book = new Book() { BookId = 1, Title = "Brave New World" }
Category category = new Category() { CategoryId = 1, CategoryName = "Dystopian" }
category.Books = new List<Book>() { book };
book.Categories = new List<Category>() { category };
modelBuilder.Entity<Book>().HasData(book);
modelBuilder.Entity<Category>().HasData(category);
... but there are no entries created for BookCategories in the resulting migration. This was somewhat expected, as this article suggests that one must explicitly seed the intermediate table. What I want is something like this:
modelBuilder.Entity<BookCategory>().HasData(
new BookCategory() { BookId = 1, CategoryId = 1 }
);
However, again, since there is no concrete class to describe BookCategories in EFCore5, the only way I can think of to seed the table is to manually edit the migration with additional MigrationBuilder.InsertData commands, which rather defeats the purpose of seeding data via application code.
However, again, since there is no concrete class to describe BookCategories in EFCore5
Actually, as explained in the What's new link, EF Core 5 allows you to have explicit join entity
public class BookCategory
{
public int BookId { get; set; }
public EBook Book { get; set; }
public int CategoryId { get; set; }
public Category Category { get; set; }
}
and configure the many-to-many relationship to use it
modelBuilder.Entity<Book>()
.HasMany(left => left.Categories)
.WithMany(right => right.Books)
.UsingEntity<BookCategory>(
right => right.HasOne(e => e.Category).WithMany(),
left => left.HasOne(e => e.Book).WithMany().HasForeignKey(e => e.BookId),
join => join.ToTable("BookCategories")
);
This way you can use all normal entity operations (query, change tracking, data model seeding etc.) with it
modelBuilder.Entity<BookCategory>().HasData(
new BookCategory() { BookId = 1, CategoryId = 1 }
);
still having the new many-to-many skip navigations mapping.
This is probably the simplest as well as the type-safe approach.
In case you thing it's too much, using the conventional join entity is also possible, but you need to know the shared dictionary entity type name, as well as the two shadow property names. Which as you will see by convention might not be what you expect.
So, by convention the join entity (and table) name is
{LeftEntityName}{RightEntityName}
and the shadow property (and column) names are
{LeftEntityNavigationPropertyName}{RightEntityKeyName}
{RightEntityNavigationPropertyName}{LeftEntityKeyName}
The first question would be - which is the left/right entity? The answer is (not documented yet) - by convention the left entity is the one which name is less in alphabetical order. So with your example Book is left, Category is right, so the join entity and table name would be BookCategory.
It can be changed adding explicit
modelBuilder.Entity<Category>()
.HasMany(left => left.Books)
.WithMany(right => right.Categories);
and now it would be CategoryBook.
In both cases the shadow property (and column) names would be
CategoriesCategoryId
BooksBookId
So neither the table name, nor the property/column names are what you'd normally do.
And apart from the database table/column names, the entity and property names are important because you'd need them for entity operations, including the data seeding in question.
With that being said, even if you don't create explicit join entity, it's better to configure fluently the one created automatically by EF Core convention:
modelBuilder.Entity<Book>()
.HasMany(left => left.Categories)
.WithMany(right => right.Books)
.UsingEntity("BookCategory", typeof(Dictionary<string, object>),
right => right.HasOne(typeof(Category)).WithMany().HasForeignKey("CategoryId"),
left => left.HasOne(typeof(Book)).WithMany().HasForeignKey("BookId"),
join => join.ToTable("BookCategories")
);
Now you can use the entity name to access the EntityTypeBuilder
modelBuilder.Entity("BookCategories")
and you can seed it similar to normal entities with shadow FK properties with anonymous type
modelBuilder.Entity("BookCategory").HasData(
new { BookId = 1, CategoryId = 1 }
);
or for this specific property bag type entity, also with Dictionary<string, object> instances
modelBuilder.Entity("BookCategory").HasData(
new Dictionary<string, object> { ["BookId"] = 1, ["CategoryId"] = 1 }
);
Update:
People seem to misinterpret the aforementioned "extra" steps and find them redundant and "too much", not needed.
I never said they are mandatory. If you know the conventional join entity and property names, go ahead directly to the last step and use anonymous type or Dictionary<string, object>.
I already explained the drawbacks of taking that route - loosing the C# type safety and using "magic" strings out of your control. You have to be smart enough to know the exact EF Core naming conventions and to realize that if you rename class Book to EBook the new join entity/table name will change from "BookCategory" to "CategoryEBook" as well as the order of the PK properties/columns, associated indexes etc.
Regarding the concrete problem with data seeding. If you really want to generalize it (OP attempt in their own answer), at least make it correctly by using the EF Core metadata system rather than reflection and assumptions. For instance, the following will extract these names from the EF Core metadata:
public static void HasJoinData<TFirst, TSecond>(
this ModelBuilder modelBuilder,
params (TFirst First, TSecond Second)[] data)
where TFirst : class where TSecond : class
=> modelBuilder.HasJoinData(data.AsEnumerable());
public static void HasJoinData<TFirst, TSecond>(
this ModelBuilder modelBuilder,
IEnumerable<(TFirst First, TSecond Second)> data)
where TFirst : class where TSecond : class
{
var firstEntityType = modelBuilder.Model.FindEntityType(typeof(TFirst));
var secondEntityType = modelBuilder.Model.FindEntityType(typeof(TSecond));
var firstToSecond = firstEntityType.GetSkipNavigations()
.Single(n => n.TargetEntityType == secondEntityType);
var joinEntityType = firstToSecond.JoinEntityType;
var firstProperty = firstToSecond.ForeignKey.Properties.Single();
var secondProperty = firstToSecond.Inverse.ForeignKey.Properties.Single();
var firstValueGetter = firstToSecond.ForeignKey.PrincipalKey.Properties.Single().GetGetter();
var secondValueGetter = firstToSecond.Inverse.ForeignKey.PrincipalKey.Properties.Single().GetGetter();
var seedData = data.Select(e => (object)new Dictionary<string, object>
{
[firstProperty.Name] = firstValueGetter.GetClrValue(e.First),
[secondProperty.Name] = secondValueGetter.GetClrValue(e.Second),
});
modelBuilder.Entity(joinEntityType.Name).HasData(seedData);
}
Also here you don't need to know which type is "left" and which is "right", neither requires special base class or interface. Just pass sequence of entity pairs and it will properly seed the conventional join entity, e.g. with OP example, both
modelBuilder.HasJoinData((book, category));
and
modelBuilder.HasJoinData((category, book));
would do.
Update (EF Core 5.0.2)
It's working well using the name of the associative table:
builder.Entity("ContractDeclarationType").HasData(
new { ContractsId = 1L, DeclarationTypesId = 1L },
new { ContractsId = 1L, DeclarationTypesId = 2L },
new { ContractsId = 1L, DeclarationTypesId = 3L });
I ended up whipping up a generic solution to this problem based upon the answer from Ivan (thanks!). I'm now able to seed all my M2M tables with this syntax:
// Add book1 and book2 to category1:
modelBuilder.HasM2MData(new [] { book1, book2 }, new [] { category1 });
This may not be fully robust, but it should work with conventional M2M mappings.
It makes some assumptions:
T1 & T2 Inherit from some ModelBase that provides an Id property.
T1 & T2 Have exactly one ICollection<OtherType> property.
You know the correct order (which model is T1 and which is T2) — this can be discovered by running the migration for the tables first and inspecting the migration.
You're running EFCore5 RC2 or later (see this issue).
public static void HasM2MData<T1, T2>
(this ModelBuilder mb, T1[] t1s, T2[] t2s)
where T1 : ModelBase where T2 : ModelBase
{
string table = $"{typeof(T1).Name}{typeof(T2).Name}";
PropertyInfo t1Prop = GetM2MProperty<T1, T2>();
PropertyInfo t2Prop = GetM2MProperty<T2, T1>();
string t1Key = $"{t1Prop.Name}Id";
string t2Key = $"{t2Prop.Name}Id";
foreach (T1 t1 in t1s) {
foreach (T2 t2 in t2s) {
mb.Entity(table).HasData(new Dictionary<string, object>() { [t2Key] = t1.Id, [t1Key] = t2.Id });
}
}
}
// Get a property on T1 which is assignable to type ICollection<T2>, representing the m2m relationship
private static PropertyInfo GetM2MProperty<T1, T2>() {
Type assignableType = typeof(ICollection<T2>);
List<PropertyInfo> props = typeof(T1).GetProperties()
.Where(pi => pi.PropertyType.IsAssignableTo(assignableType))
.ToList();
if (props.Count() != 1) {
throw new SystemException(
$"Expected {typeof(T1)} to have exactly one column of type {assignableType}; got: {props.Count()}");
}
return props.First();
}
In the migration, we see something like:
migrationBuilder.InsertData(
table: "BookCategory",
columns: new[] { "BooksId", "CategoriesId" },
values: new object[,]
{
{ "book1", "category1" },
{ "book2", "category1" }
});
Let's say I have a map (call it myClass.mapElem<Object, Object>) like so:
Key Val
A X
B Y
C X
I want to write HQL that can query the Values such that I can get back all instances of myClass where mapElem's value is 'X' (where 'X' is a fully populated object-- I just don't want to go through each element and say x.e1 = mapElem.e1 and x.e2=... etc). I know I can do this for the keys by using where ? in index(myClass.mapElem), I just need the corresponding statement for querying the values!
Thanks in advance...
ETA: Not sure if the syntax makes a difference, but the way I am actually querying this is like so:
select myClass.something from myClass mc join myClass.mapElem me where...
You should use elements(). I tried simulating your example with the following class
#Entity
#Table(name="Dummy")
public class TestClass {
private Integer id;
private Map<String, String> myMap;
#Id
#Column(name="DummyId")
#GeneratedValue(generator="native")
#GenericGenerator(name="native", strategy = "native")
public Integer getId() {
return id;
}
#ElementCollection
public Map<String, String> getMyMap() {
return myMap;
}
public void setId(Integer id) {
this.id = id;
}
public void setMyMap(Map<String, String> myMap) {
this.myMap = myMap;
}
}
And persisted a few instances, which constain maps of a similar structure to what you have in your example. (using < String, String > since the values in your example are strings)
The following query gives me all instances of TestClass, where the map contains a specific value
SELECT DISTINCT myClass
FROM TestClass myClass JOIN myClass.myMap myMap
WHERE ? in elements(myMap)
In my particular case, I ended up having to use an SQL query. Not ideal, but it worked.
The Goal is to have a list of options (that a user can chose through radio buttons) in one place(for eg: a yaml config file). No other place should have this list hard-coded
I've done something similar to create select elements, and I think enums worked just fine. Doing radio buttons should be very similar. I've set it up so that the labels can be defined in the messages file. I'm going to try to excerpt the relevant portions from my larger auto-form-generation code (using FastTags) the best I can. It's a bit heavy for this one case but it makes sense in the larger system.
I use the tag like #{form.selector 'order.status' /}, which looks find the variable named order in the template, sees that status is declared as public Status status, and then goes to find all the values of the Status enum and generate options for them in the select element.
First, I use a FieldContext object which just contains a bunch of info that's used by the other code to determine what to generate along with some utility methods:
public class FieldContext {
public final Map<?,?> args;
public final ExecutableTemplate template;
public final int fromLine;
public Class clazz = null;
public Field field = null;
public Object object = null;
public Object value = null;
private Map<String,String> attrs = new HashMap<String,String>();
private Map<String,Boolean> printed = new HashMap<String,Boolean>();
private List<Option> options;
...
Then I have this in another helper class (its info gets added to the FieldContext):
public List<Option> determineOptions(FieldContext context) {
List<Option> options = new ArrayList<Option>();
if (context.field.getType().isEnum()) {
for (Object option : context.field.getType().getEnumConstants()) {
options.add(new Option(option.toString(), Message.get(option.toString())));
}
}
return options;
}
then the tag declaration is
public static void _selector(Map<?,?> args, Closure body, PrintWriter out, ExecutableTemplate template, int fromLine) {
String field_name = args.get("arg").toString();
TagContext.current().data.put("name", field_name);
SelectHelper helper = HelperFactory.getHelper(SelectHelper.class);
try {
FieldContext context = new FieldContext(field_name, args, template, fromLine);
helper.autoconfigure(context);
TagContext.current().data.put("selected", helper.determineValue(context));
out.print("<div class=\"formutil-field formutil-selector\">");
out.print("<label for=\"" + context.getAttr("id") + "\">");
out.print(helper.findOrCreateLabel(context));
out.print("</label>");
out.print("<select");
context.printAttribute(out, "id", "name");
out.print(">");
if (context.hasOptions()) {
for (Option option : context.getOptions()) {
out.print("<option value=\"" + option.value + "\">" + option.label + "</option>");
}
}
out.print("</select>");
context.printErrorIfPresent(out);
context.printValidationHints(out);
out.println("</div>");
}
...
}
Class Carro {
String name
String marca
String matricula
}
Class CarroMovel{
String pro1
String prop2
String prop3
Carro carro
static hasMany = [ carros: Carro]
}
def save2 = {
def carroInstance = new Carro()
def carroMovelInstance = new CarroMovel()
carroInstance.name = params.name
carroInstance.marca = params.marca
carroInstance.matricula = params.matricula
carroMovelInstance.prop1 = params.carroMovel.prop1
carroMovelInstance.prop2 = params.carroMovel.prop2
carroMovelInstance.prop3 = params.carroMovel.prop3
carroInstance.save()
carroMovelInstance.carro = carroInstance
carroMovelInstance.save()
}
The CarroInstance is saving, but the carroMovelInstance isn't. I cannot figure it out. Any help would be apreciated.
Grooveek is correct in that you haven't ever invoked carroMovelInstance.save().
However, it might be simpler for you to simply take advantage of Grails' databinding, instead of unnecessarily creating the associations and manually binding the parameters.
// Update your Carro domain.
def Carro {
String name
String marca
String matricula
// will cause persistence operations to cascade from CarroMovel to Carro
static belongsTo = CarroMovel
}
// Update your save2 action.
// By passing 'params' to the CarroMovel constructor, Grails will bind request
// parameters to domain properties of the same name; it even works with associations!
def save2 = {
def carroMovelInstance = new CarroMovel(params)
if(carroMovelInstance.validate) {
carroMovelInstance.save()
}
}
Read up on Grails Data Binding, particularly the parts about associations. Additionally, read "Understanding Cascading Updates and Deletes" to understand how a call to save() on a parent domain object will (or will not) cascade to an associated domain object.
you never ask for the carroMovelInstance to save... The carro instance has nor reference to carroMovel instance so there is no cascading of saving
We are currently building a Map manually based on the two fields that are returned by a named JPA query because JPA 2.1 only provides a getResultList() method:
#NamedQuery{name="myQuery",query="select c.name, c.number from Client c"}
HashMap<Long,String> myMap = new HashMap<Long,String>();
for(Client c: em.createNamedQuery("myQuery").getResultList() ){
myMap.put(c.getNumber, c.getName);
}
But, I feel like a custom mapper or similar would be more performant since this list could easily be 30,000+ results.
Any ideas to build a Map without iterating manually.
(I am using OpenJPA, not hibernate)
Returning a Map result using JPA Query getResultStream
Since the JPA 2.2 version, you can use the getResultStream Query method to transform the List<Tuple> result into a Map<Integer, Integer>:
Map<Integer, Integer> postCountByYearMap = entityManager.createQuery("""
select
YEAR(p.createdOn) as year,
count(p) as postCount
from
Post p
group by
YEAR(p.createdOn)
""", Tuple.class)
.getResultStream()
.collect(
Collectors.toMap(
tuple -> ((Number) tuple.get("year")).intValue(),
tuple -> ((Number) tuple.get("postCount")).intValue()
)
);
Returning a Map result using JPA Query getResultList and Java stream
If you're using JPA 2.1 or older versions but your application is running on Java 8 or a newer version, then you can use getResultList and transform the List<Tuple> to a Java 8 stream:
Map<Integer, Integer> postCountByYearMap = entityManager.createQuery("""
select
YEAR(p.createdOn) as year,
count(p) as postCount
from
Post p
group by
YEAR(p.createdOn)
""", Tuple.class)
.getResultList()
.stream()
.collect(
Collectors.toMap(
tuple -> ((Number) tuple.get("year")).intValue(),
tuple -> ((Number) tuple.get("postCount")).intValue()
)
);
Returning a Map result using a Hibernate-specific ResultTransformer
Another option is to use the MapResultTransformer class provided by the Hibernate Types open-source project:
Map<Number, Number> postCountByYearMap = (Map<Number, Number>) entityManager.createQuery("""
select
YEAR(p.createdOn) as year,
count(p) as postCount
from
Post p
group by
YEAR(p.createdOn)
""")
.unwrap(org.hibernate.query.Query.class)
.setResultTransformer(
new MapResultTransformer<Number, Number>()
)
.getSingleResult();
The MapResultTransformer is suitable for projects still running on Java 6 or using older Hibernate versions.
Avoid returning large result sets
The OP said:
But, I feel like a custom mapper or similar would be more performant
since this list could easily be 30,000+ results.
This is a terrible idea. You never need to select 30k records. How would that fit in the UI? Or, why would you operate on such a large batch of records?
You should use query pagination as this will help you reduce the transaction response time and provide better concurrency.
There is no standard way to get JPA to return a map.
see related question: JPA 2.0 native query results as map
Iterating manually should be fine. The time to iterate a list/map in memory is going to be small relative to the time to execute/return the query results. I wouldn't try to futz with the JPA internals or customization unless there was conclusive evidence that manual iteration was not workable.
Also, if you have other places where you turn query result Lists into Maps, you probably want to refactor that into a utility method with a parameter to indicate the map key property.
You can retrieve a list of java.util.Map.Entry instead.
Therefore the collection in your entity should be modeled as a Map:
#OneToMany
#MapKeyEnumerated(EnumType.STRING)
public Map<PhoneType, PhoneNumber> phones;
In the example PhoneType is a simple enum, PhoneNumber is an entity. In your query use the ENTRY keyword that was introduced in JPA 2.0 for map operations:
public List<Entry> getPersonPhones(){
return em.createQuery("SELECT ENTRY(pn) FROM Person p JOIN p.phones pn",java.util.Map.Entry.class).getResultList();
}
You are now ready to retrieve the entries and start working with it:
List<java.util.Map.Entry> phoneEntries = personDao.getPersonPhoneNumbers();
for (java.util.Map.Entry<PhoneType, PhoneNumber> entry: phoneEntries){
//entry.key(), entry.value()
}
If you still need the entries in a map but don't want to iterate through your list of entries manually, have a look on this post Convert Set<Map.Entry<K, V>> to HashMap<K, V> which works with Java 8.
This works fine.
Repository code :
#Repository
public interface BookRepository extends CrudRepository<Book,Id> {
#Query("SELECT b.name, b.author from Book b")
List<Object[]> findBooks();
}
service.java
List<Object[]> list = bookRepository.findBooks();
for (Object[] ob : list){
String key = (String)ob[0];
String value = (String)ob[1];
}
link https://codereview.stackexchange.com/questions/1409/jpa-query-to-return-a-map
Map<String,Object> map = null;
try {
EntityManager entityManager = getEntityManager();
Query query = entityManager.createNativeQuery(sql);
query.setHint(QueryHints.RESULT_TYPE, ResultType.Map);
map = (Map<String,Object>) query.getSingleResult();
}catch (Exception e){ }
List<Map<String,Object>> list = null;
try {
EntityManager entityManager = getEntityManager();
Query query = entityManager.createNativeQuery(sql);
query.setHint(QueryHints.RESULT_TYPE, ResultType.Map);
list = query.getResultList();
}catch (Exception e){ }
JPA v2.2
Though I am late here, but if someone reaches here for solution, here is my custom working solution for multiple selected columns with multiple rows:
Query query = this.entityManager.createNativeQuery("SELECT abc, xyz, pqr,...FROM...", Tuple.class);
.
.
.
List<Tuple> lst = query.getResultList();
List<Map<String, Object>> result = convertTuplesToMap(lst);
Implementation of convertTuplesToMap():
public static List<Map<String, Object>> convertTuplesToMap(List<Tuple> tuples) {
List<Map<String, Object>> result = new ArrayList<>();
for (Tuple single : tuples) {
Map<String, Object> tempMap = new HashMap<>();
for (TupleElement<?> key : single.getElements()) {
tempMap.put(key.getAlias(), single.get(key));
}
result.add(tempMap);
}
return result;
}
in case java 8
there built in entry "CustomEntryClass"
since return is stream, then caller function (repoistory layer) must have #Transactional(readonly=true|false) annotation, otherwithe exception will be thrown
make sure you will use full qualified name of class CustomEntryClass...
#Query("select new CustomEntryClass(config.propertyName, config.propertyValue) " +
"from ClientConfigBO config where config.clientCode =:clientCode ")
Stream<CustomEntryClass<String, String>> getByClientCodeMap(#Param("clientCode") String clientCode);
With custom result class and a bit of Guava, this is my approach which works quite well:
public static class SlugPair {
String canonicalSlug;
String slug;
public SlugPair(String canonicalSlug, String slug) {
super();
this.canonicalSlug = canonicalSlug;
this.slug = slug;
}
}
...
final TypedQuery<SlugPair> query = em.createQuery(
"SELECT NEW com.quikdo.core.impl.JpaPlaceRepository$SlugPair(e.canonicalSlug, e.slug) FROM "
+ entityClass.getName() + " e WHERE e.canonicalSlug IN :canonicalSlugs",
SlugPair.class);
query.setParameter("canonicalSlugs", canonicalSlugs);
final Map<String, SlugPair> existingSlugs =
FluentIterable.from(query.getResultList()).uniqueIndex(
new Function<SlugPair, String>() {
#Override #Nullable
public String apply(#Nullable SlugPair input) {
return input.canonicalSlug;
}
});
using java 8 (+) you can get results as a list of array object (each column will from select will have same index on results array) by hibernate entity manger, and then from results list into stream, map results into entry (key, value), then collect them into map of same type.
final String sql = "SELECT ID, MODE FROM MODES";
List<Object[]> result = entityManager.createNativeQuery(sql).getResultList();
return result.stream()
.map(o -> new AbstractMap.SimpleEntry<>(Long.valueOf(o[0].toString()), String.valueOf(o[1])))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
The easiest and simplest way worked for me is:
String[] columns = {"id","name","salary","phone","address", "dob"};
String query = "SELECT id,name,salary,phone,address,dob from users ";
List<Object[]> queryResp = em.createNativeQuery(query).getResultList();
List<Map<String,String>> dataList = new ArrayList<>();
for(Object[] obj : queryResp) {
Map<String,String> row = new HashMap<>(columns.length);
for(int i=0; i<columns.length; i++) {
if(obj[i]!=null)
row.put(columns[i], obj[i].toString());
else
row.put(columns[i], "");
}
dataList.add(row);
}
Please refer, JPA 2.0 native query results as map
In your case in Postgres, it would be something like,
List<String> list = em.createNativeQuery("select cast(json_object_agg(c.number, c.name) as text) from schema.client c")
.getResultList();
//handle exception here, this is just sample
Map map = new ObjectMapper().readValue(list.get(0), Map.class);
Kindly note, I am just sharing my workaround with Postgres.
How about this ?
#NamedNativeQueries({
#NamedNativeQuery(
name="myQuery",
query="select c.name, c.number from Client c",
resultClass=RegularClient.class
)
})
and
public static List<RegularClient> runMyQuery() {
return entityManager().createNamedQuery("myQuery").getResultList();
}