I am looking to retrieve the items from my Cosmos Db collection and I have a list of Ids which I want to retrieve. all code are running in an Azure Function and below is the code where I get all Ids in List filteredResult where I get list of all Ids. I am looking the best way to complete this code to retrieve all items from my Cosmos Db collection considering 30-40 Ids at a time.
public static void Run([ServiceBusTrigger("testSB", "SubscriberName", Connection = "AzureServiceBusString")] string mySbMsg,
[CosmosDB(
databaseName: "DBName",
collectionName: "CollectionName",
ConnectionStringSetting = "CosmosDBConnection")] DocumentClient client,
ILogger log)
{
try {
log.LogInformation($"C# ServiceBus topic trigger function processed message: {mySbMsg}");
var jsonSerializerSettings = new JsonSerializerSettings();
jsonSerializerSettings.MissingMemberHandling = MissingMemberHandling.Ignore;
List<MyItem> lists = JsonConvert.DeserializeObject<List<MyItem>>(mySbMsg, jsonSerializerSettings);
List<string> filteredResult = (from s in lists
where s.DocType == "TEST"
select s.Id).ToList();
}
This issue has recently been covered under Microsoft Q&A forum. Posting the answer here again for broader reach.
This code can be used after you created the 'filteredResult' (just replace the 'input' list with 'filteredResult')
List<string> input = new List<string>();
input.Add("1");
input.Add("2");
input.Add("3");
var option = new FeedOptions { EnableCrossPartitionQuery = true };
IQueryable<Family> queryable = client.CreateDocumentQuery<Family>(UriFactory.CreateDocumentCollectionUri("families", "items").ToString(), "SELECT * FROM books where books.id IN " + "('" + string.Join( "','", input) + "')",option);
List<Family> posts = queryable.ToList();
Console.WriteLine("Read count = {0}", posts.Count);
Also notice I created a model class for document properties as below:
public class Family
{
public int id;
public string city;
}
I want populate a TokenBox from a database using the property tokenBoxSettings.Properties.DataSource
TokenBoxView.cshtml
groupSettings.Items.Add(
formLayoutSettings.Items.Add(i =>
{
i.FieldName = "email";
i.Caption = "Email";
i.NestedExtensionType = FormLayoutNestedExtensionItemType.TokenBox;
TokenBoxSettings tokenBoxSettings = (TokenBoxSettings) i.NestedExtensionSettings;
tokenBoxSettings.Width = 350;
//data binding
tokenBoxSettings.Properties.DataSource = mainController.GetMails();
tokenBoxSettings.Properties.TextField = "email_empresarial";
tokenBoxSettings.Properties.ValueField = "email_empresarial";
tokenBoxSettings.Properties.IncrementalFilteringMode = IncrementalFilteringMode.Contains;
tokenBoxSettings.Properties.ValueSeparator = ';';
})
);
TokenBoxController.cs
//mainController
//I created a dictionary based on the result of select
public Dictionary<string, string> GetMails()
{
var email = db.usuario.ToList().Select(e => new { e.email_empresarial });
var emails = new Dictionary<string, string>();
foreach (var mail in email)
{
correos.Add(mail.ToString(), mail.ToString());
}
return emails;
}
But it shows me the "object explicitly", I only need the value, for example kenneth or manuel
tokenBox list
What am I doing wrong? or with what other approach I can do?
You are specifying same email_empresarial field name for both tokenBoxSettings.Properties.TextField and tokenBoxSettings.Properties.ValueField.
Since you are binding your TokenBox to Dictionary, try changing settings for TextField and ValueField to reference Dictionary Key and Value, like this:
tokenBoxSettings.Properties.TextField = "Value";
tokenBoxSettings.Properties.ValueField = "Key";
Also, in your GetMail() method you have declared the var emails but in the loop you are adding items to the undeclared correos variable. Are you sure you don't have a bug here?
Another note, in the Dictionary returned by GetMails() you populate both dictionary keys and values with the same value of mail.ToString(). Are you sure you really need to use Dictionary to bind your TokenBox? If keys and values are equal you may try going with plain List<string>.
I have my dynamo db table as follows:
HashKey(xyz) ,RangeKey(timestamp)
Now for each hash key i have set of range key.
Now i want to query based on set of hashkey and i want only most recent value correspoding to that to be fetched .I dont want to do in memory sorting and then picking most recent version.
Can i do this in any way?
Use case is that i will do a bulkget and pass set of hashkey (say 100) , so i want to get one record for each hashkey
You (currently) can't set constraints on a batch get. See http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API_BatchGetItems.html
However, for single hash keys, you can set the direction using ScanIndexForward. See http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API_Query.html for information.
Sample java code:
new QueryRequest().withTableName("table-name")
.withScanIndexForward(false)
.withLimit(1)
.withHashKeyValue(new AttributeValue().withS("hash-key"));
It will not be very efficient though, as you will need to make this call 100 times.
Use ScanIndexForward(true for ascending and false for descending) and can also limit the result using setLimit value of Query Expression.
Please find below the code where used QueryPage for finding the single record.
public EventLogEntitySave fetchLatestEvents(String id) {
EventLogEntitySave entity = new EventLogEntitySave();
entity.setId(id);
DynamoDBQueryExpression<EventLogEntitySave> queryExpression = new DynamoDBQueryExpression<EventLogEntitySave>().withHashKeyValues(entity);
queryExpression.setScanIndexForward(false);
queryExpression.withLimit(1);
queryExpression.setLimit(1);
List<EventLogEntitySave> result = dynamoDBMapper.queryPage(EventLogEntitySave.class, queryExpression).getResults();
System.out.println("size of records = "+result.size() );
result.get(0);
}
#DynamoDBTable(tableName = "PROD_EA_Test")
public class EventLogEntitySave {
#DynamoDBHashKey
private String id;
private String reconciliationProcessId;
private String vin;
private String source;
}
public class DynamoDBConfig {
#Bean
public AmazonDynamoDB amazonDynamoDB() {
String accesskey = "";
String secretkey = "";
//
// creating dynamo client
BasicAWSCredentials credentials = new BasicAWSCredentials(accesskey, secretkey);
AmazonDynamoDB dynamo = new AmazonDynamoDBClient(credentials);
dynamo.setRegion(Region.getRegion(Regions.US_WEST_2));
return dynamo;
}
We are currently building a Map manually based on the two fields that are returned by a named JPA query because JPA 2.1 only provides a getResultList() method:
#NamedQuery{name="myQuery",query="select c.name, c.number from Client c"}
HashMap<Long,String> myMap = new HashMap<Long,String>();
for(Client c: em.createNamedQuery("myQuery").getResultList() ){
myMap.put(c.getNumber, c.getName);
}
But, I feel like a custom mapper or similar would be more performant since this list could easily be 30,000+ results.
Any ideas to build a Map without iterating manually.
(I am using OpenJPA, not hibernate)
Returning a Map result using JPA Query getResultStream
Since the JPA 2.2 version, you can use the getResultStream Query method to transform the List<Tuple> result into a Map<Integer, Integer>:
Map<Integer, Integer> postCountByYearMap = entityManager.createQuery("""
select
YEAR(p.createdOn) as year,
count(p) as postCount
from
Post p
group by
YEAR(p.createdOn)
""", Tuple.class)
.getResultStream()
.collect(
Collectors.toMap(
tuple -> ((Number) tuple.get("year")).intValue(),
tuple -> ((Number) tuple.get("postCount")).intValue()
)
);
Returning a Map result using JPA Query getResultList and Java stream
If you're using JPA 2.1 or older versions but your application is running on Java 8 or a newer version, then you can use getResultList and transform the List<Tuple> to a Java 8 stream:
Map<Integer, Integer> postCountByYearMap = entityManager.createQuery("""
select
YEAR(p.createdOn) as year,
count(p) as postCount
from
Post p
group by
YEAR(p.createdOn)
""", Tuple.class)
.getResultList()
.stream()
.collect(
Collectors.toMap(
tuple -> ((Number) tuple.get("year")).intValue(),
tuple -> ((Number) tuple.get("postCount")).intValue()
)
);
Returning a Map result using a Hibernate-specific ResultTransformer
Another option is to use the MapResultTransformer class provided by the Hibernate Types open-source project:
Map<Number, Number> postCountByYearMap = (Map<Number, Number>) entityManager.createQuery("""
select
YEAR(p.createdOn) as year,
count(p) as postCount
from
Post p
group by
YEAR(p.createdOn)
""")
.unwrap(org.hibernate.query.Query.class)
.setResultTransformer(
new MapResultTransformer<Number, Number>()
)
.getSingleResult();
The MapResultTransformer is suitable for projects still running on Java 6 or using older Hibernate versions.
Avoid returning large result sets
The OP said:
But, I feel like a custom mapper or similar would be more performant
since this list could easily be 30,000+ results.
This is a terrible idea. You never need to select 30k records. How would that fit in the UI? Or, why would you operate on such a large batch of records?
You should use query pagination as this will help you reduce the transaction response time and provide better concurrency.
There is no standard way to get JPA to return a map.
see related question: JPA 2.0 native query results as map
Iterating manually should be fine. The time to iterate a list/map in memory is going to be small relative to the time to execute/return the query results. I wouldn't try to futz with the JPA internals or customization unless there was conclusive evidence that manual iteration was not workable.
Also, if you have other places where you turn query result Lists into Maps, you probably want to refactor that into a utility method with a parameter to indicate the map key property.
You can retrieve a list of java.util.Map.Entry instead.
Therefore the collection in your entity should be modeled as a Map:
#OneToMany
#MapKeyEnumerated(EnumType.STRING)
public Map<PhoneType, PhoneNumber> phones;
In the example PhoneType is a simple enum, PhoneNumber is an entity. In your query use the ENTRY keyword that was introduced in JPA 2.0 for map operations:
public List<Entry> getPersonPhones(){
return em.createQuery("SELECT ENTRY(pn) FROM Person p JOIN p.phones pn",java.util.Map.Entry.class).getResultList();
}
You are now ready to retrieve the entries and start working with it:
List<java.util.Map.Entry> phoneEntries = personDao.getPersonPhoneNumbers();
for (java.util.Map.Entry<PhoneType, PhoneNumber> entry: phoneEntries){
//entry.key(), entry.value()
}
If you still need the entries in a map but don't want to iterate through your list of entries manually, have a look on this post Convert Set<Map.Entry<K, V>> to HashMap<K, V> which works with Java 8.
This works fine.
Repository code :
#Repository
public interface BookRepository extends CrudRepository<Book,Id> {
#Query("SELECT b.name, b.author from Book b")
List<Object[]> findBooks();
}
service.java
List<Object[]> list = bookRepository.findBooks();
for (Object[] ob : list){
String key = (String)ob[0];
String value = (String)ob[1];
}
link https://codereview.stackexchange.com/questions/1409/jpa-query-to-return-a-map
Map<String,Object> map = null;
try {
EntityManager entityManager = getEntityManager();
Query query = entityManager.createNativeQuery(sql);
query.setHint(QueryHints.RESULT_TYPE, ResultType.Map);
map = (Map<String,Object>) query.getSingleResult();
}catch (Exception e){ }
List<Map<String,Object>> list = null;
try {
EntityManager entityManager = getEntityManager();
Query query = entityManager.createNativeQuery(sql);
query.setHint(QueryHints.RESULT_TYPE, ResultType.Map);
list = query.getResultList();
}catch (Exception e){ }
JPA v2.2
Though I am late here, but if someone reaches here for solution, here is my custom working solution for multiple selected columns with multiple rows:
Query query = this.entityManager.createNativeQuery("SELECT abc, xyz, pqr,...FROM...", Tuple.class);
.
.
.
List<Tuple> lst = query.getResultList();
List<Map<String, Object>> result = convertTuplesToMap(lst);
Implementation of convertTuplesToMap():
public static List<Map<String, Object>> convertTuplesToMap(List<Tuple> tuples) {
List<Map<String, Object>> result = new ArrayList<>();
for (Tuple single : tuples) {
Map<String, Object> tempMap = new HashMap<>();
for (TupleElement<?> key : single.getElements()) {
tempMap.put(key.getAlias(), single.get(key));
}
result.add(tempMap);
}
return result;
}
in case java 8
there built in entry "CustomEntryClass"
since return is stream, then caller function (repoistory layer) must have #Transactional(readonly=true|false) annotation, otherwithe exception will be thrown
make sure you will use full qualified name of class CustomEntryClass...
#Query("select new CustomEntryClass(config.propertyName, config.propertyValue) " +
"from ClientConfigBO config where config.clientCode =:clientCode ")
Stream<CustomEntryClass<String, String>> getByClientCodeMap(#Param("clientCode") String clientCode);
With custom result class and a bit of Guava, this is my approach which works quite well:
public static class SlugPair {
String canonicalSlug;
String slug;
public SlugPair(String canonicalSlug, String slug) {
super();
this.canonicalSlug = canonicalSlug;
this.slug = slug;
}
}
...
final TypedQuery<SlugPair> query = em.createQuery(
"SELECT NEW com.quikdo.core.impl.JpaPlaceRepository$SlugPair(e.canonicalSlug, e.slug) FROM "
+ entityClass.getName() + " e WHERE e.canonicalSlug IN :canonicalSlugs",
SlugPair.class);
query.setParameter("canonicalSlugs", canonicalSlugs);
final Map<String, SlugPair> existingSlugs =
FluentIterable.from(query.getResultList()).uniqueIndex(
new Function<SlugPair, String>() {
#Override #Nullable
public String apply(#Nullable SlugPair input) {
return input.canonicalSlug;
}
});
using java 8 (+) you can get results as a list of array object (each column will from select will have same index on results array) by hibernate entity manger, and then from results list into stream, map results into entry (key, value), then collect them into map of same type.
final String sql = "SELECT ID, MODE FROM MODES";
List<Object[]> result = entityManager.createNativeQuery(sql).getResultList();
return result.stream()
.map(o -> new AbstractMap.SimpleEntry<>(Long.valueOf(o[0].toString()), String.valueOf(o[1])))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
The easiest and simplest way worked for me is:
String[] columns = {"id","name","salary","phone","address", "dob"};
String query = "SELECT id,name,salary,phone,address,dob from users ";
List<Object[]> queryResp = em.createNativeQuery(query).getResultList();
List<Map<String,String>> dataList = new ArrayList<>();
for(Object[] obj : queryResp) {
Map<String,String> row = new HashMap<>(columns.length);
for(int i=0; i<columns.length; i++) {
if(obj[i]!=null)
row.put(columns[i], obj[i].toString());
else
row.put(columns[i], "");
}
dataList.add(row);
}
Please refer, JPA 2.0 native query results as map
In your case in Postgres, it would be something like,
List<String> list = em.createNativeQuery("select cast(json_object_agg(c.number, c.name) as text) from schema.client c")
.getResultList();
//handle exception here, this is just sample
Map map = new ObjectMapper().readValue(list.get(0), Map.class);
Kindly note, I am just sharing my workaround with Postgres.
How about this ?
#NamedNativeQueries({
#NamedNativeQuery(
name="myQuery",
query="select c.name, c.number from Client c",
resultClass=RegularClient.class
)
})
and
public static List<RegularClient> runMyQuery() {
return entityManager().createNamedQuery("myQuery").getResultList();
}
Not really sure how to do this but i can cache the dictionary like this:
Cache.Insert("CacheName", Dictionary)
need some direction. the dictionary is two string values taken from a database. The user will input a string and i need to compare it against the values in the cached dictionary.
In general you need to access the object from the cache, cast it, and the use the ContainsKey property. Here is an example:
First add the dictionary to the Cache:
IDictionary<string, string> testDict = new Dictionary<string, string>();
testDict.Add("Test", "test");
Cache.Insert("dict", testDict);
Then, when you need to do so, access the cached object and use it ContainsKey property to determine whether it contains the searched key or not.
var dict = Cache["dict"] as IDictionary<string, string>;
if (dict != null)
{
string testValue = "test";
if(dict.ContainsKey(testValue))
{
/* some logic here */
}
}
You can access the value the following way:
if (dict != null)
{
string testValue = "test";
if(dict.ContainsKey(testValue))
{
/* some logic here */
string value = dict[testValue];
}
}
You can get the dictionary out of the cache by writing
var dict = (Dictionary<X, Y>) cache["CacheName"];