I am having trouble with a query.
Fiddle: https://www.db-fiddle.com/f/JXQHw1VzF7vAowNLFrxv5/1
This is not going to work.
So my question is: What has to be done to get a result when I wanna use both conditions.
(attr_key = 0 AND attr_value & 201326592 = 201326592)
AND
(attr_key = 30 AND attr_value & 8 = 8)
Thanks in advance!
Best regards
One way to check for the presence of some number of key value pairs in the items_attributes table would be to use conditional aggregation:
SELECT i.id
FROM items i
LEFT JOIN items_attributes ia
ON i.id = ia.owner
GROUP BY
i.id
HAVING
SUM(CASE WHEN ia.key = 0 AND ia.value = 201326592 THEN 1 ELSE 0 END) > 0 AND
SUM(CASE WHEN ia.key = 30 AND ia.value = 8 THEN 1 ELSE 0 END) > 0
The trick in the above query is that we scan each cluster of key/value pairs for each item, and then check whether the pairs you expect are present.
Note: My query just returns id values from items matching all key value pairs. If you want to bring in other columns from either of the two tables, you may simply add on more joins to what I wrote above.
Trying to optimize a query, which has multiple counts for objects in subordinate table (used aliases in SQLAlchemy). In Witch Academia terms, something like this:
SELECT
exam.id AS exam_id,
exam.name AS exam_name,
count(tried_witch.id) AS tried,
count(passed_witch.id) AS passed,
count(failed_witch.id) AS failed
FROM exam
LEFT OUTER JOIN witch AS tried_witch
ON tried_witch.exam_id = exam.id AND
tried_witch.is_failed = 0 AND
tried_witch.status != "passed"
LEFT OUTER JOIN witch AS passed_witch
ON passed_witch.exam_id = exam.id AND
passed_witch.is_failed = 0 AND
passed_witch.status = "passed"
LEFT OUTER JOIN witch AS failed_witch
ON failed_witch.exam_id = exam.id AND
failed_witch.is_failed = 1
GROUP BY exam.id, exam.name
ORDER BY tried ASC
LIMIT 20
Number of witches can be large (hundreds of thousands), number of exams is lower (hundreds), so the above query is quite slow. In a lot of similar questions I've found answers, which propose the above, but I feel like a totally different approach is needed here. I am stuck at coming up with alternative. NB, there is a need to order by calculated counts. It is also important to have zeros as counts, of course, where due. (do not pay attention to a somewhat funny model: witches can easily clone themselves to go to multiple exams, thus per exam identity)
With one EXISTS subquery, which is not reflected in the above and does not influence the ouotcome, the situation is:
# Query_time: 1.135747 Lock_time: 0.000209 Rows_sent: 20 Rows_examined: 98174
# Rows_affected: 0
# Full_scan: Yes Full_join: No Tmp_table: Yes Tmp_table_on_disk: Yes
# Filesort: Yes Filesort_on_disk: No Merge_passes: 0 Priority_queue: No
Updated query, which is still quite slow:
SELECT
exam.id AS exam_id,
exam.name AS exam_name,
count(CASE WHEN (witch.status != "passed" AND witch.is_failed = 0)
THEN witch.id
ELSE NULL END) AS tried,
count(CASE WHEN (witch.status = "passed" AND witch.is_failed = 0)
THEN witch.id
ELSE NULL END) AS passed,
count(CASE WHEN (witch.is_failed = 1)
THEN witch.id
ELSE NULL END) AS failed
FROM exam
LEFT OUTER JOIN witch ON witch.exam_id = exam.id
GROUP BY exam.id, exam.name
ORDER BY tried ASC
LIMIT 20
Indexing is the key to get performance of the query.
I do not know MariaDB at all, so not sure what the possibilities are. But if it is anything like Microsoft SQL Server, then here is what I would try:
Create ONE composite index covering ALL the required columns: witch_id, status and is_failed. If the query uses that index, that should be it. Here the order of the included columns might be very important. Then profile the query in order to understand if the index is used. See Optimization and Indexes documentation page.
Consider Generated (Virtual and Persistent) Columns.
It looks like all the information for classification of the witch into tried, passed or failed bucket is contained in the row for witch. Therefore, you can basically create those virtual columns on the database table directly and use PERSISTENT option. This option allows creating index on it. Then you can create an index specifically for this query containing witch_id and three virtual columns: tried, passed and failed. Make sure you query uses it, and that should be pretty good. The query will then look very simple:
SELECT exam.id,
exam.name,
sum(witch.tried) AS tried,
sum(witch.passed) AS passed,
sum(witch.failed) AS failed
FROM exam
INNER JOIN witch ON exam.id = witch.exam_id
GROUP BY exam.id,
exam.name
ORDER BY sum(witch.tried)
LIMIT 20
Although query simple comparisons and AND/OR clauses, you are basically offloading the calculation of the 3 statuses to the database during INSERT/UPDATE. Then during SELECT you query should be much faster.
Your example does not specify any result filtering (WHERE clause), but if you have one, it might also have an impact on the way one optimises indices for query performance.
Original answer: Below is the originally proposed change to the query.
Here i assume that indexing part of the optimisation has been already done.
Could you try with SUM instead of COUNT?
SELECT exam.id,
exam.name,
sum(CASE
WHEN (witch.is_failed = 0
AND witch.status != 'passed') THEN 1
ELSE 0
END) AS tried,
sum(CASE
WHEN (witch.is_failed = 0
AND witch.status = 'passed') THEN 1
ELSE 0
END) AS passed,
sum(CASE
WHEN (witch.is_failed = 1) THEN 1
ELSE 0
END) AS failed
FROM exam
INNER JOIN witch ON exam.id = witch.exam_id
GROUP BY exam.id,
exam.name
ORDER BY sum(CASE
WHEN (witch.is_failed = 0
AND witch.status != 'passed') THEN 1
ELSE 0
END)
LIMIT 20
The rest:
Given you have specified sqlalchemy in your answer, here is the sqlalchemy code, which i used to model and generate the query:
# model
class Exam(Base):
id = Column(Integer, primary_key=True)
name = Column(String)
class Witch(Base):
id = Column(Integer, primary_key=True)
exam_id = Column(Integer, ForeignKey('exam.id'))
is_failed = Column(Integer)
status = Column(String)
exam = relationship(Exam, backref='witches')
# computed fields
#hybrid_property
def tried(self):
return self.is_failed == 0 and self.status != 'passed'
#hybrid_property
def passed(self):
return self.is_failed == 0 and self.status == 'passed'
#hybrid_property
def failed(self):
return self.is_failed == 1
# computed fields: expression
#tried.expression
def _tried_expression(cls):
return case([(and_(
cls.is_failed == 0,
cls.status != 'passed',
), 1)], else_=0)
#passed.expression
def _passed_expression(cls):
return case([(and_(
cls.status == 'passed',
cls.is_failed == 0,
), 1)], else_=0)
#failed.expression
def _failed_expression(cls):
return case([(cls.is_failed == 1, 1)], else_=0)
and:
# query
q = (
session.query(
Exam.id, Exam.name,
func.sum(Witch.tried).label("tried"),
func.sum(Witch.passed).label("passed"),
func.sum(Witch.failed).label("failed"),
)
.join(Witch)
.group_by(Exam.id, Exam.name)
.order_by(func.sum(Witch.tried))
.limit(20)
)
I have a query that Counts 2 columns from 2 separate tables using subqueries, which works. Now I have to implement into this query the ability to filter out these results based on the Date of a Call Record. I will post the query in which I am working with:
SELECT (m.FirstName || " " || m.LastName) AS Members,
(
SELECT count(CallToLineOfficers.MemberID)
FROM CallToLineOfficers
WHERE CallToLineOfficers.MemberID = m.MemberID
)
+ (
SELECT count(CallToMembers.MemberID)
FROM CallToMembers
WHERE CallToMembers.MemberID = m.MemberID
) AS Tally
FROM Members AS m, Call, CallToMembers, CallToLineOfficers
Join Call on CallToMembers.CallID = Call.CallID
and CallToLineOfficers.CallID = Call.CallI
WHERE m.FirstName <> 'None'
-- and Call.Date between '2017-03-21' and '2017-03-22'
GROUP BY m.MemberID
ORDER BY m.LastName ASC;
Ok, so table Call stores the Date and its PK is CallID. Both CallToLineOfficers and CallToMembers are Bridge Tables that also contain only CallID and MemberID. With the current query, where the Date is commented out, that Date range should only return all names, but a count of 1 should appear under 1 person's name.
I have tried joining Call.CallID with both Bridge Tables' CallIDs without any luck, though I think this is the right way to do it. Could someone help point me in the right direction? I am lost. (I tried explaining this the best I could, so if you need more info, let me know.)
UPDATED: Here is a screenshot of what I am getting:
Based on the provided date in the sample, the new results, with the Date, should be:
Bob Clark - 1
Rob Catalano - 1
Matt Butler - 1
Danielle Davidson - 1
Jerry Chuska - 1
Tom Cramer - 1
Everyone else should be 0.
At the moment, the subqueries filter only on the member ID. So for any member ID in the outer query, they return the full count.
To reduce the count, you have to filter in the subqueries:
SELECT (FirstName || " " || LastName) AS Members,
(
SELECT count(*)
FROM CallToLineOfficers
JOIN Call USING (CallID)
WHERE MemberID = m.MemberID
AND Date BETWEEN '2017-03-21' AND '2017-03-22'
)
+ (
SELECT count(*)
FROM CallToMembers
JOIN Call USING (CallID)
WHERE MemberID = m.MemberID
AND Date BETWEEN '2017-03-21' AND '2017-03-22'
) AS Tally
FROM Members AS m
WHERE FirstName <> 'None'
ORDER BY LastName ASC;
I have two tables: TblAppointmentTypes and TblEmployeeInfo. There is many to many relation between these two tables with a joined table that is TblEmployeeServices. There are two records in TblAppointmentTypes and four records in TblEmployeeInfo. All the two records of TblAppointmentTypes is assigned to all of the records in TblEmployeeInfo, that there is eight records in TblEmployeeServices. I want to retrive all the services that are assigned to all the four Employees, it should return 2 that is assigned to all the four employees, but my query return 8 records, four duplicate services.
I am using Telerik Open Access ORM. Here is the code:
public static List<TblAppointmentType> GetAllAppointmentType(List<int> employeeIDs)
{
var list = new List<TblAppointmentType>();
if (employeeIDs != null && employeeIDs.Count > 0)
{
var dc = new Entities();
list = (from a in dc.TblAppointmentTypes.Distinct()
join e in dc.TblEmployeeServices on a.ID equals e.AppointmentTypeID
where a.IsDeleted == false && employeeIDs.Contains<int>(e.EmployeeID)
select a).ToList();
}
return list;
}
As I figured it out, You want to get all AppointmentTypes which are related to all Employees.
It means that AppTypes that are related to 3 Employees (in your case) should be excluded.
So, the most efficient code that I have tried is like this :
list = (from a in dc.TblAppointmentTypes
join e in dc.TblEmployeeServices on a.ID equals e.AppointmentTypeID
where a.IsDeleted == false
group e.EmployeeID by a into g
where g.count() == employeeIDs.Count()
select g.Key).ToList();
It is not very clear to me why I am not able to get the number of rows returned using linq-to-sql
I had this query to use as validation:
var obj1 = (from c in context.sistema_DocType_Index
where c.indexId == id
select c).First();
if(obj1 != null) {}
I was getting a null exception using First() method if there was no row returned. Ok, so I decided to use Count().
var obj1 = (from c in context.sistema_DocType_Index
where c.indexId == id
select c).Count();
if(obj1 > 0) {}
I have 3 rows returning from the database but the Count() gives me 0.
Why is that?
you can use Any();method. which is the best use for this case. Any()
if( YourDataCollection.Any(SomeCOndtion==SOmeValue))
{
// do some logic
}