How to paginate results when filtering on left join table column - mikro-orm

Let's define for example these entities:
#Entity()
export class Author {
#OneToMany(() => Book, book => book.author)
books = new Collection<Book>(this);
}
#Entity()
export class Book {
#Property()
name: string;
#ManyToOne()
author: Author;
}
const authors = await authorRepository.find({ books: {name: 'Test'} }, {
limit: 10
});
As you can see i want to select all authors that have books with name 'Test', but this will generate the following query:
select `e0`.* from `author` as `e0`
left join `book` as `e1` on `e0`.`id` = `e1`.`author_id`
where `e1`.`name` = 'Test' limit 10
The problem is when i have more than 2 authors and each of them has more than 10 books with name 'Test', this query will return only the first author because of the limit clause.
I am not sure if this is a bug in the ORM or it's a expected behavior.
One way to fix that is to select all rows without the limit clause and do the pagination in memory like they are doing in hibernate, but i am not sure how much memory will be used with very large tables and this might block the event loop in NodeJS while processing them.

You could fallback to query builder here to apply group by clause:
const qb = await authorRepository.createQueryBuilder('a');
qb.select('a.*').where({ books: { name: 'Test' } }).limit(10).groupBy('a.id');
const authors = await qb.getResult();
Will think about how to support this use case directly via EM/repository API.

Related

Firebase index required but no link provided

Tip: instead of creating indexes here, run queries in your code – if you're missing any indexes, you'll see errors with links to create them.
Learn more
I'm getting an error no matching index found without the usual direct link to create the index required.
My database is structured as follows:
answers (collection)
- $docId (document)
- $refId (collection)
- $docId (document)
- created_at (timestamp)
- type (string)
My query is:
firebase.firestore()
.collection('answers')
.doc('abcdHajM930Gabcd')
.collection('abcdetIo25xYqAabcd')
.orderBy("created_at", "desc")
.limit(5)
.where("type", "==", "3")
I've tried all combinations of indexes I could think of (screenshot below), What am I doing wrong?
collection
field
type
answers
created_at: DESC
type: ARRAY
answers
created_at: DESC
type: ARRAY
answers
created_at: DESC
type: ASC
answers
created_at: DESC
type: ASC
answers
type: ASC
created_at: DESC
answers
type: ASC
created_at: DESC
Update: The issue seems to be fixed and Javascript client SDK now returns the index creation link.
I'm not totally sure if that's intended by the Firebase JS SDK or no, but if you try running your query in a Cloud Function (just use the emulator) then it should throw an error containing the link to create the index.
export const getIndexLink = functions.https.onRequest(async (request, response) => {
// functions.logger.info("Hello logs!", {structuredData: true});
try {
const snap = await admin.firestore().collection('answers')
.doc('abcdHajM930Gabcd')
.collection('abcdetIo25xYqAabcd')
.where("type", "==", "3")
.orderBy("created_at", "desc")
.limit(5).get()
console.log(snap.size)
response.send("Hello from Firebase!" + snap.size);
} catch (error) {
console.log(error)
response.send(error.message)
}
});
I tried copying your query and created an index using the URL and the index turns out to be:
Your fields that your are using are not a part of documents of "answers" collection. Now if you change the sub-collection name, it'll require another index. What does your Firestore structure look like? I'd recommend restructuring it a bit and get a same sub-collection name and create an index with collectionGroup scope.

Sequelize Create a new instance and add association

I've looked at the documentation and read through some issues related to this on the sequelize github repo but so far haven't found any solutions to this, IMHO, simple operation.
I have two models: Users and Cards. They are associated with a one-to-one relationship.
Orders.create({
date: req.body.date,
...
card: req.body.card //FK to a card ID
})
req.body.card is the id of the card. When I get the created object back though card is null.
I also tried creating the order first and then adding an association after but haven't had any luck with that either. I found some stuff about a set function that would set the association but I only saw examples of it with hasMany relationships.
I'm not trying to do anything fancy - just want to set the card id in the card field on an Order.
There are a few issues with your model definition:
Orders.hasOne(models.Cards, { foreignKey: 'id', as: 'card', });
In your definition, the foreign key will be placed on the cards table.
Naming the foreignKey as id will collide with the models own id field.
Try the following:
const Card = sequelize.define('card', {
name: { type: Sequelize.STRING }
});
const Order = sequelize.define('order', {
name: { type: Sequelize.STRING }
});
Order.hasOne(Card, { foreignKey: 'order_id' }); // if foreignKey is not included, will default to orderId
const order = await Order.create({
name: 'first order'
})
const card = await Card.create({
name: 'first card',
order_id: order.id
})

How do I merge slices of state for my presentational component properly (ngrx/store)?

I'm still trying to learn ngrx - the store is set-up and everything seems to work fine. The database I'm using is SQL so basically I'm "clongin" the tables and load them into the store but have problems joining the state in the end when selecting an employee.
The employee entity looks something like this:
export class Employee = {
id: number;
firstName: string;
lastName: string;
degreeId?: number;
degree: Degree;
}
export class Degree = {
id: number;
description: string;
}
Now in my component, I'd like to get the specific employee to display like this:
{
id: 1,
firstName: George,
lastName: Costanza,
degreeId: 2,
degree: {
id: 2,
description: 'College'
}
}
What I tried is to create a selector that merges these two entities:
export const getEmployeeWithAllData = createSelector(
getSelectedEmployee,
getRelationalData,
(employee, data) => {
const employeesDegree = degree[employee.degreeId]
employee.degree = employeesDegree
return employee;
}
);
This does seem to work if I don't use ngrx-store-freeze - so since I don't know if I am creating the selector correctly or if ngrx-store-freeze has a bug I'm asking this question.
Am I really mutating state when I do this?
If yes, how can I select a specific employee from my store with all the relational data that he has?
What I'm doing doesn't really feel right. In my actual application the employee has about 8 fields of relational data which I need to join...
Edit: I forgot to include the error ngrx-store-freeze throws:
ERROR TypeError: Cannot assign to read only property 'degree' of object '[object Object]'
at eval (employee.selector.ts:85)
at eval (store.es5.js:602)
at memoized (store.es5.js:539)
at defaultStateFn (store.es5.js:573)
at eval (store.es5.js:605)
at MapSubscriber.memoized [as project] (store.es5.js:539)
at MapSubscriber._next (map.js:79)
at MapSubscriber.Subscriber.next (Subscriber.js:95)
at MapSubscriber._next (map.js:85)
at MapSubscriber.Subscriber.next (Subscriber.js:95)
yes, you are trying to mutate the state. You can prevent this simply like bellow
export const getEmployeeWithAllData = createSelector(
getSelectedEmployee,
getRelationalData,
(employee, data) => {
const emp = JSON.parse(JSON.stringify(employee)
emp.degree = degree[employee.degreeId]
return emp;
}
)

How to combine data model types with document ids?

I'm working with Firestore and Typescript.
For the data models I have types definitions. For example User could be this:
interface User {
name: string;
age: number;
}
The users are stored in the database in the users collection under a unique name/id.
In Firebase when you query a collection, the ids of the documents are available on the document reference, and do not come with the data. In a common use-case for front-end, you want to retrieve an array of records with their ids, because you probably want to interact with them and need to identify each.
So I made a query similar to the code below, where the id is merged into the resulting array:
async function getUsers(): Promise<any[]> {
const query = db.collection("users")
const snapshot = await query.get();
const results = snapshot.docs.map(doc => {
return { ...doc.data(), id: doc.id };
});
}
Now the problem is, that I have a User type, but it can't be used here because it does not contain an id field.
A naïve solution could be to create a new type:
interface UserWithId extends User {
id: string
}
And write the function like:
async function getUsers(): Promise<UserWithId[]> {}
But this doesn't feel right to me, because you would have to potentially do this for many types.
A better solution I think would be to create a generic type:
type DatabaseRecord<T> = {
id: string,
data: T
}
Thus keeping data and ids separate in the returning results:
const results = snapshot.docs.map(doc => {
return { data: doc.data(), id: doc.id };
});
... and use the function signature:
async function getUsers(): Promise<DatabaseRecord<User>[]> {}
I would favour the second over the first solution, because creating new types for each case feels silly. But I am still not sure if that is the best approach.
This seems like such a common scenario but I didn't manage to find any documentation on this. I have seen developers simply write the id in the model data, essentially duplicating the document name in its data, but that to me seems like a big mistake.
I can imagine that if you don't use Typescript (of Flow) that you just don't care about the resulting structure and simply merge the id with the data, but this is one of the reasons I really love using type annotation in JS. It forces you think more about your data and you end up writing cleaner code.

Joining a table through another join table using withRelated

I have 3 tables:
Order: id
Item: id
OrderItems: order_id, item_id
I want to be able to fetch a particular Order and get all the Items related to that Order, without the clutter introduced by doing Order.where(...).fetch({withRelated: ['orderItem.item']).
Ideally I'd like to do Order.where(...).fetch({withRelated: ['items']) and the items relationship knows to go through the OrderItems table to get the information.
I have tried the relationship
items() {
return this.hasMany('Item').through('OrderItem')
}
but that doesn't seem to work as I'd expected.
Is this possible using Bookshelf's API without writing a manual join?
The relationship you tried:
items() {
return this.hasMany('Item').through('OrderItem')
}
Means to Bookshelf:
Item(id, OrderItem_id) -> OrderItem(id, Order_id) -> Order(id)
Or a 1:n:m relationship.
But your database says:
Item(id) <- OrderItem(Item_id, Order_id) -> Order(id)
This kind of relationship on Bookshelf maps better as belongsToMany():
bookshelf.plugin('registry') // to ease the cyclic mapping
const Item = bookshelf.Model.extend({
tableName: 'item',
orders: function() {
this.belongsToMany('Order', 'OrderItem')
},
})
bookshelf.model('Item', Item)
const Order = bookshelf.Model.extend({
tableName: 'order',
items: function() {
//return this.hasMany('Item').through('OrderItem')
return this.belongsToMany('Item', 'OrderItem')
},
})
bookshelf.model('Order', Order)
Note that until you start placing useful data on the intermediate table you don't need to create a model for it.
With that a Order.where(...).fetch({withRelated: 'items'}).then(...) query should work as expected.

Resources