DocumentDb Repository to Query Child Documents Generically - azure-cosmosdb

In DocumentDb, is it possible to search for child documents that meet a certain criteria without having to involve the parent class in the query?
BACKGROUND
I'm (trying to start by) using the DocumentDbRepository.cs that is generated for you automatically in the Azure Portal when you create a new Azure Cosmos DB account. However, it's obvious that this was meant merely as a starting point and will require some additional work for individual scenarios.
In a C# Console app (.NET Core) I have a simple parent-child relationship between Company and Employees:
public class Customer
{
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
[JsonProperty(PropertyName = "name")]
public string Name { get; set; }
[JsonProperty(PropertyName = "location")]
public string Location { get; set; }
[JsonProperty(PropertyName = "employees")]
public List<Employee> Employees { get; set; }
public Customer()
{
Employees = new List<Employee>();
}
}
public class Employee
{
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
[JsonProperty(PropertyName = "firstName")]
public string FirstName { get; set; }
[JsonProperty(PropertyName = "lastName")]
public string LastName { get; set; }
[JsonProperty(PropertyName = "sales")]
public double Sales { get; set; }
}
In the Document Explorer, I can see that I have one instance of this class structure like so:
{
"id": "7",
"name": "ACME Corp",
"location": "New York",
"employees": [
{
"id": "c4202793-da55-4324-88c9-b9c9fe8f4b6c",
"firstName": "John",
"lastName": "Smith",
"sales": 123
}
]
}
If I wanted to get all Companies that meet a certain criteria, it would be a fairly easy operation using the generated DocumentDbRepository.cs methods:
DocumentDBRepository<Customer>.Initialize();
var customers = DocumentDBRepository<Customer>.GetItemsAsync(p => p.Location.Equals("New York")).Result;
... for reference, the generated GetItemsAsync() from Microsoft method looks like this:
public static async Task<IEnumerable<T>> GetItemsAsync(Expression<Func<T, bool>> predicate)
{
IDocumentQuery<T> query = client.CreateDocumentQuery<T>(
UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId),
new FeedOptions { MaxItemCount = -1 })
.Where(predicate)
.AsDocumentQuery();
List<T> results = new List<T>();
while (query.HasMoreResults)
{
results.AddRange(await query.ExecuteNextAsync<T>());
}
return results;
}
PROBLEM
HOWEVER, if I want to retrieve ONLY EMPLOYEES regardless of the Company they belong to, I'm not sure how to write a method in the repository class that will accomplish this.
First, I think I'll need some sort of type property so I can differentiate what a Customer is versus an Employee (versus other domain class types I may want to also add in the same collection).
Second, I would probably query that using that type property for all queries and not use the DocumentDbRepository.cs methods which seem to only work with root data. In other words, the DocumentDbRepository.cs methods seem to only be concerned with non-hierarchical entities.
But this is where things break down ... given the generic nature of this sample repository class, I can't quite connect the dots in my mind required to query sub-documents / children.
I am merely asking for a nudge in the right direction here. Thank you.

I want to retrieve ONLY EMPLOYEES regardless of the Company
If I understanding correctly, you want to query employees according employees' property from Customer. If it is that case, we could do that with SQL as following, and we just need to change the where <filter_condition> as we want.
SELECT c as employees
FROM c IN Customer.employees
WHERE c.firstName = 'John'
I test with your mentioned document from Azure portal, it works correctly on my side.
The following is the c# demo code:
var endpointUrl = "https://yourdocumentdbname.documents.azure.com:443/";
var authorizationKey = "xxxxx";
var databaseId = "database name";
var collectionId = "collecton name";
var client = new DocumentClient(new Uri(endpointUrl), authorizationKey);
var sql = "SELECT c as employee FROM c IN Customer.employees WHERE c.firstName = 'xx'";
var collection = client.CreateDocumentCollectionIfNotExistsAsync(
UriFactory.CreateDatabaseUri(databaseId), new DocumentCollection
{
Id = collectionId
}).Result.Resource;
var query = client.CreateDocumentQuery(collection.SelfLink, sql).AsDocumentQuery();
while (query.HasMoreResults)
{
var documents = query.ExecuteNextAsync().Result;
//do something
}

Related

dapper left join & mapping query

There is one-to-many relation between Brand and Campaign entities,
With given Id I need to select Campaign and related Brand entity along with it.
TO do so:
public async Task<Campaign> GetAsync(int id)
{
using var dbConnection = _context.CreateConnection();
string query = #"SELECT c.[Id], c.[BrandId], c.[StartDate],
c.[EndDate],b.[Id] FROM [dbo].[Campaign] c
left join [dbo].[Brand] b on c.[BrandId] = b.[Id]
WHERE c.[Id] = #Id";
var campaign = await dbConnection.QueryAsync<Campaign, Brand, Campaign>(query, (campaign, brand) =>
{
campaign.Brand = brand;
return campaign;
}, splitOn: "BrandId", param: new { Id = id });
return campaign.FirstOrDefault();
}
code above not throws exception but child brand entity is not correct.(its dummy record, and BrandId is 0 whereas its valu is 5 in database)
whats missing here?
entity def:
public class Campaign : SqlEntityBase
{
public int BrandId { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
public Brand Brand { get; set; }
}
public class Brand : SqlEntityBase
{
public string Name { get; set; }
public List<Campaign> Campaign { get; set; } = new List<Campaign>();
}
Your SQL query doesn't make sense. You're splitting on BrandId in dapper but you never select anything to do with the brand in the query. With your current code, dapper is parsing the SQL column Id into your Campaign POCO (which it cant do because there isn't a property called Id nor have you anything mapped to it in the campaign class).
And then it see's BrandId and then parses everything after that into the Brand POCO, but your remaining columns that you're selecting are the start and end dates for the campaign.
In summary: You need to include the Brand data into the SQL query. You're joining onto the table, but only selecting the campaign data.

RavenDB array search returns random results

I'm trying to perform a search on top of a dictionary using the Search method from RavenDB 4. Strangely, if the search term is the word in or it I get random results back. I'm absolutely sure that none of the records contains those words. It also happens when executing the equivalent lucene query on the studio. It works as expected when I enter a valid search term like the employee's name, number, etc.
I've managed to create this simple scenario based on the real one.
Here's the index:
public class Search : AbstractIndexCreationTask<Employee, Page>
{
public Search()
{
Map = employees => from employee in employees
select new
{
Id = employee.Id,
Details = employee.Details
};
Reduce = results => from result in results
group result by new
{
result.Id,
result.Details
}
into g
select new
{
g.Key.Id,
g.Key.Details
};
Index("Details", FieldIndexing.Search);
}
}
Employee class:
public class Employee
{
public string Id { get; set; }
public Dictionary<string, object> Details { get; set; }
}
Adding employees:
details = new Dictionary<string, object>();
details.Add("EmployeeNo", 25);
details.Add("FirstNames", "Yuri");
details.Add("Surname", "Cardoso");
details.Add("PositionCode", "XYZ");
details.Add("PositionTitle", "Developer");
employee = new Employee
{
Details = details
};
session.Store(employee);
session.SaveChanges();
Search method:
var searchTerm = "in";
var result = session
.Query<Page, Search>()
.Search(i => i.Details, $"EmployeeNo:({searchTerm})")
.Search(i => i.Details, $"FirstNames:({searchTerm})", options: SearchOptions.Or)
.Search(i => i.Details, $"Surname:({searchTerm})", options: SearchOptions.Or)
.Search(i => i.Details, $"PositionCode:({searchTerm})", options: SearchOptions.Or)
.Search(i => i.Details, $"PositionTitle:({searchTerm})", options: SearchOptions.Or)
.ToList();
Lucene query outputed:
from index 'Search' where search(Details, "EmployeeNo:(it)")
or search(Details, "FirstNames:(it)")
or search(Details, "Surname:(it)")
or search(Details, "PositionCode:(it)")
or search(Details, "PositionTitle:(it)")
Any idea why random results are returned when those specific words are enterered?
The issue is stop words. Certain terms are so common, that they are meaningless for searching using full text search.
is, it, they, are, etc.
They are erased by the query analyzer.
See the discussion here: https://ravendb.net/docs/article-page/4.2/Csharp/indexes/using-analyzers
You can use a whitespace analyzer, instead of the Standard Analyzer, since the former doesn't eliminate stop words.
After getting help from the RavenDB group guys, we've managed to find a solution for my scenario.
Employee:
public class Employee
{
public string Id { get; set; }
public string DepartmentId { get; set; }
public Dictionary<string, object> Details { get; set; }
}
Department:
public class Department
{
public string Id { get; set; }
public string Name { get; set; }
}
Page:
public class Page
{
public string Id { get; set; }
public string Department { get; set; }
public Dictionary<string, object> Details { get; set; }
}
Index (with dynamic fields):
public class Search : AbstractIndexCreationTask<Employee, Page>
{
public Search()
{
Map = employees => from employee in employees
let dept = LoadDocument<Department>(employee.DepartmentId)
select new
{
employee.Id,
Department = dept.Name,
_ = employee.Details.Select(x => CreateField(x.Key, x.Value))
};
Store(x => x.Department, FieldStorage.Yes);
Index(Constants.Documents.Indexing.Fields.AllFields, FieldIndexing.Search);
}
}
Query:
using (var session = DocumentStoreHolder.Store.OpenAsyncSession())
{
var searchTearm = "*yu* *dev*";
var result = await session
.Advanced
.AsyncDocumentQuery<Page, Search>()
.Search("Department", searchTearm)
.Search("EmployeeNo", searchTearm)
.Search("FirstNames", searchTearm)
.Search("Surname", searchTearm)
.Search("PositionCode", searchTearm)
.Search("PositionTitle", searchTearm)
.SelectFields<Page>()
.ToListAsync();
}
Everything seems to be working fine this way, no more random results.
Big thanks to Ayende and Egor.

Modeling mongodb subobjects in ASP.NET MVC application

I am running into issues after adding a sub-object to my mongo documents. The query no longer returns results, even though I've added an object to my model to store the new sub-object.
I believe the issue is in adding the class for the sub-object to the object model. I can't seem to find any references anywhere online, so perhaps I'm searching for the wrong thing?
Mongo elements look as so:
{
_id: [id],
Name: "Paul",
Phone1: {
Name: "Work",
Number: "15551234567"
},
Phone2: {
Name: "Work",
Number: "15551234567"
}
}
In C# my model looks as so:
public class PersonModel {
[BsonId]
public ObjectId _Id { get; set; }
public string Name { get; set; }
public Phone Phone1 { get; set; }
public Phone Phone2 { get; set; }
}
public class Phone {
public string Name { get; set; }
public string Number { get; set; }
}
My query looks as so:
public async Task<List<PersonModel>> GetPerson(string name)
{
var people = new List<PersonModel>();
var allDocuments = await PersonCollection.FindAsync(
ds => ds.Name == name);
await allDocuments.ForEachAsync(doc => people.Add(doc));
return people;
}
Any references to a working example would be appreciated.
Thank you for looking.
The above implementation is correct. After many hours of trouble shooting it turned out I didn't have the datapoint in my database that I was querying against. Unbelievable.
If anyone else is struggling, I also found this guide that confirmed I was dealing with the subobject correctly: https://www.codementor.io/pmbanugo/working-with-mongodb-in-net-1-basics-g4frivcvz

PartitionKey property in Cosmos DB needs to be a string always?

I am serializing and storing the below class object into a Cosmos DB partitioned collection with partition key path "/targetId" set on the collection.
public class DataItem
{
[JsonProperty(PropertyName = "id")]
public string Id { get; set; }
[JsonProperty(PropertyName = "name")]
public string Name { get; set; }
[JsonProperty(PropertyName = "city")]
public string City { get; set; }
/// This is my partition key
[JsonProperty(PropertyName = "targetId")]
public long TargetId { get; set; }
}
In the above class, when I have the property TagretId as long, I get error the below error:
Requests originating from scripts cannot reference partition keys other
than the one for which client request was submitted
However, when I change the TargetId property to type "string", it works fine.
Is there any type restrcition on the partition key property while using it with Cosmos DB?
Updated code
List<DataItem> items = GetDataItems(); // This comes from UI as JSON, actually
var groupedItems = items.GroupBy(x => x.TargetId);
foreach (var groupItem in groupedItems)
{
// I even tried like below converting to dynamic JSON array. Still got the same exception
//string argsJson = JsonConvert.SerializeObject(groupItem.ToArray());
//var args = new dynamic[] { JsonConvert.DeserializeObject<dynamic[]>(argsJson) };
RequestOptions requestOptions = new RequestOptions { PartitionKey = new PartitionKey(groupItem.Key.ToString()) };
var result = await client.ExecuteStoredProcedureAsync<int>(
UriFactory.CreateStoredProcedureUri(DatabaseId, CollectionId, sprocId),
requestOptions,
groupItem.ToList());
}
Cosmos DB allows partition key to be number. But it looks EF Core needs to make sure partition key field can be converted to string.
I found this way can solve the problem.
builder.Property(p => p.MyProperty).ToJsonProperty("myProperty")
.HasConversion();

Entity Framework : adding record with related data

I have a very simple Situation with 2 tables
public class Movie
{
[Key]
public Guid ID { get; set; }
public string Name { get; set; }
public byte[] Hash { get; set; }
public int GenreID{ get; set; }
[ForeignKey("GenreID")]
public virtual Genre genre{ get; set; }
}
and
public class Genre
{
public int ID { get; set; }
public string Name { get; set; }
}
Now, in an import sequence I want to create new movies and link the Genre with the existing entries in the Genre table or create new Genre entries if they don't exist.
Movie m = new Movie();
m.ID = Guid.NewGuid();
IndexerContext db = new IndexerContext();
var genre = db.Genre.Where(g => g.Name== genreValue).FirstOrDefault();
if(genre!= null)
{
m.GenreID= genre.GenreID;
}
else
{
genre= new Genre();
genre.Name = genreValue;
db.Genres.Add(genre);
var genreCreated= db.Genre.Where(g => g.Name== genreValue).FirstOrDefault();
m.GenreID= genreCreated.GenreID;
}
Now the problem is, it doesn't work. The last line fails because genreCreated is null.
Plus I think I must doing it wrong - it can't be that difficult in Entity Framework.
can anyone help me?
db.Genres.Add(genre);
This does not send insert statement to database - this instructs entity framework that new record should be inserted when saving changes. Genre will be saved (and created id available) after you call db.SaveChanges(); As for now, you do not have save call, so genreCreated is null.
In your situation - fix is simple, you do not need to select genreCreated from db. Just setting m.Genre to new value should do the job
Movie m = new Movie();
m.ID = Guid.NewGuid();
IndexerContext db = new IndexerContext();
var genre = db.Genre.Where(g => g.Name== genreValue).FirstOrDefault();
if(genre! = null)
{
m.GenreID = genre.GenreID;
}
else
{
genre = new Genre();
genre.Name = genreValue;
m.Genre = genre;
}
db.SaveChanges(); //m.GenreID will automatically be set to newly inserted genre
After the add statement you need to save it:
Try
genre= new Genre();
genre.Name = genreValue;
db.Genres.Add(genre);
db.SaveChanges();

Resources