I am writing a SQL query which gives me a slow performance. Because of that it gives me 504 gateway timeout problem. Please help me to remake this query so that my output results faster. I will put the query below.
select
r.c1,
parent_item.c2,
parent_item.c3,
parent_item.c4,
parent_item.c5,
parent_item.c6,
parent_item.c7,
pt.c8,
child_item.c9,
t.c10,
child_item.c11,
from
table1 child_item,
table2 t,
table3 r,
table1 parent_item,
table4 pt
where
r.col1 = child_item.id and
t.id=child_item.typeid and
parent_item.id = r.parent_itemid and
pt.id = parent_item.typeid and parent_item.id=800 and
parent_item.id = (select
itemid
from
table5
where
itemid=parent_item.id
((10!= 1) ? and (holder_itemid in (10,100) and level > 0): "")) and
child_item.id = (select
itemid
from
table5
where
itemid=child_item.id
((10 != 1) ? and (holder_itemid in (10,100) and level > 0) : ""))
order by
r.parent_itemid,
r.relation_typeid,
r.ordinal
It's likely the two sub-queries, but we don't have enough information about your schema.
You should run your query though EXPLAIN and see what it says.
JOINs might help, but again, we can't tell for sure.
It’s very difficult to accurately point out performance problems if we don’t know your database schema. (The database schema means your table definitions, indexes etc.)
Also, what is this bit supposed to do?
((10!= 1) ? and (holder_itemid in (10,100) and level > 0): ""))
AFAICS, this is not a valid SQL query, and will result in a syntax error.
Related
I am wondering if there is a way to delay Delete transactions in an SQR program? I am having an issue where I am deleting data from a table that has a trigger on it that inserts rows into another (Audit) table and the only key on the table that can be altered is a date/time stamp, but because these transactions are occuring at the exact same time (down to the hundreth of a second) it is causing duplicate insert errors on the sql trigger. I was hoping if there is a way to delay each row to be deleted in SQR that would be a good work-around to this issue. Thanks!
BEGIN-PROCEDURE DELETE-PSROLEUSER-ROLES
SHOW 'BEGINNING DELETE-PSROLEUSER-ROLES'
BEGIN-SQL
DELETE PSRO
FROM PSROLEUSER PSRO
INNER JOIN PS_GH_AD_X_WALK B ON B.OPRID = PSRO.ROLEUSER
INNER JOIN HRDEV01_FOR_BUDGET_MODULE.HRDEV92B.dbo.PS_JOB C ON C.EMPLID = B.GH_AD_EMPLID AND B.GH_AD_EMPLID <> ''
WHERE C.EFFDT =
(SELECT MAX(A_ED.EFFDT) FROM HRDEV01_FOR_BUDGET_MODULE.HRDEV92B.dbo.PS_JOB A_ED
WHERE C.EMPLID = A_ED.EMPLID
AND C.EMPL_RCD = A_ED.EMPL_RCD
AND A_ED.EFFDT <= SUBSTRING(CONVERT(CHAR,GETDATE(),121), 1, 10))
AND C.ACTION = 'TER'
END-SQL
END-PROCEDURE DELETE-PSROLEUSER-ROLES
Please move all of the inner joins to the where clause for the delete. You probably don't intend to delete rows from these inner joined tables, some of which may return multiple rows for an emplid.
Apparently I'm missing something with how LINQ to entities works. Hopefully one of you all can educate me.
Please try the below locally and let me know if you are seeing the same results. Something is really strange here...
Lets look at a very simple LINQ expression using navigation properties.
This was generated in LinqPad in a C# statement.
var result = (from ge in group_execution
where ge.automation_sequences.project.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
OR, we can use joins...which will lead to the same bad results, but lets continue...
var result = (from ge in group_execution
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
These very simple LINQ expressions generate the following SQL:
SELECT
[Filter1].[id1] AS [id],
[Filter1].[parent_group_exec_id] AS [parent_group_exec_id],
[Extent5].[client_id] AS [client_id]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[automation_sequence_id] AS [automation_sequence_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id]
FROM [dbo].[group_execution] AS [Extent1]
INNER JOIN [dbo].[automation_sequences] AS [Extent2] ON [Extent1].[automation_sequence_id] = [Extent2].[id]
INNER JOIN [dbo].[project] AS [Extent3] ON [Extent2].[project_id] = [Extent3].[id]
WHERE ([Extent1].[parent_group_exec_id] IS NOT NULL) AND (1 = [Extent3].[client_id]) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[automation_sequences] AS [Extent4] ON [Filter1].[automation_sequence_id] = [Extent4].[id]
LEFT OUTER JOIN [dbo].[project] AS [Extent5] ON [Extent4].[project_id] = [Extent5].[id]
This baffles me. For the life of me I can't understand why LINQ is doing this. It's horrible, just look at the execution plan:
Now lets manually clean this up in SSMS and view the correct SQL and execution plan:
Much better, but how do we get LINQ to act this way?
Is anyone else seeing this? Has anyone else ever saw this and corrected it and if so how?
Thanks for looking into this.
UPDATE, attempting Chris Schaller fix:
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x=>x.client_id == 1 && x.parent_group_exec_id != null);
result.Dump();
Just so you all know I'm monitoring the SQL through SQL Server Profiler. If anyone knows of any issues doing it this way let me know.
UPDATE, a fix for JOINS, but not nav properties, and a cause, but why?
Here's your solution:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
Null checks shouldn't cause the framework to mess up like this. Why should I have to write it this way? This just seems like a defect to me in the framework. It will make my dynamic expressions a little bit more difficult to write, but maybe I can find a way.
Navigation Properties still mess up...so I'm still really sad. Picture below:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
where ge.automation_sequences.project.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
move your where clause to after you have defined the structure of the select statement
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x => x.client_id == 1 && x.parent_group_exec_id != null)
result.Dump();
Remember that Linq-to-entities flattens the results of queries to execute as SQL and then hydrates the object graph from those results.
When your query uses navigation properties or joins the query parser has to allow for zero results from those sub queries (Extents) to make sure that all columns that are required in the output and any interim processing are represented. By explicitly specifying a filter on a table for != null early in the query the parser knows that there is no further possibility that the field and any relationships linked by that field will be null, until then the parser prepares the query as if the joins will return null results
It is worth checking, but i wonder if UseDatabaseNullSemantics has anything to do with this?
Try:
dbContext.Configuration.UseDatabaseNullSemantics = false
In Linq, we can specify Where clauses as often as we like, improve the resulting SQL we should filter early in the query and granularly.
The parser engine is optimized to follow and implement your query sequentially, and generate good SQL at the end of it. Don't try to write linq-to-entities the same way that you structure your SQL, I know it's counter intuitive because the syntax is similar
A good technique is to assume that before each clause all the records from the previous statements have been loaded into memory, and that the next operation will affect all of those records. So you want to reduce the records before each additional operation by specifying a filter before moving on to the next clause
In general, if you have a filter condition based on the root table, apply this to the query before define all other joins and filters and even selects, you will get much cleaner sql.
I want to create a new calculated member in OLAP Cube to count the number of distinct clients, I'm trying to write this expression, but i don't know how to make it in MDX:
Distinct count ([DIM.Clients].[CuNumber], where Sum([Measures].[QQT - FACT Ventes] >=1)
Any help please !
Thanks!
Hi thank you for replies,
I spent couple of days trying to make the query work, but without big progress.
First, I ran an SQL query on my datawarehouse to know what result should I get on my OLAP Cube,
this is my SQL query:
use [Warehouse]
select count(*) as count_row
From
(Select F.FaCunumberX
from [dbo].[Dim_FaClients] F
inner join [dbo].[FACT_Ventes] V on F.[SK_FAClients] = V.SK_FaClients
inner join [dbo].[Dim_Date] D on D.SK_Date = V.SK_Date
where
D.Year = '2014'
Group by F.FaCunumberX
having SUM(V.QQT) >= 1) test
the result I got is : 26026
On my OLAP Cube I tried several queries, but I didn't get the same result
this is some of the expressions that I tried :
WITH SET MySet AS
(Filter({[DIM FA Clients].[FaCuNumberX].[FaCuNumberX]}*{([Dim Date].[Year].&[2014],[Measures].[QQT - Fact Ventes])},[Measures].[QQT - Fact Ventes]>1 or [Measures].[QQT - Fact Ventes]=1)
MEMBER MEASURES.SETDISTINCTCOUNT AS
DISTINCTCOUNT(MySet)
SELECT {MEASURES.SETDISTINCTCOUNT} ON 0
FROM [CubeAll]
the result I got with this one is : 31575
I tried also this expression :
DistinctCount(Filter([DIM.Clients].[CuNumber].[CuNumber].Members,
[Measures].[QQT - FACT Ventes] >= 1
)
)
the same result : 31575
sincerely, I don't see what I'm missing on my expressions.
Thank's for your help !
This would be something like
DistinctCount(Filter([DIM.Clients].[CuNumber].[CuNumber].Members,
[Measures].[QQT - FACT Ventes] >= 1
)
)
See the documentation of Filter and DistinctCount for details.
I am writing a report for the desktop support team in the company where I work. The report needs to produce a set of new starters within a specified time frame passed in from an ASP.NET application. Currently there is a one to many relationship between our Worker table and Contract table. We hire a lot of contractors and they sometimes come back after a number of months but are still treated like new starters as new machines need to be configured along with desk space.
A new contract is added for every pay review, job title change and new starter. We need to filter out all but the new starter. The newest contract that is added for job changes and pay reviews is always one day after the end date of the previous contract naturally. As I am only still a fresher in the grand scheme of things I am struggling with a set of functions I am trying to use to achieve my goal.
WHERE
(dbo.[Contract].StartDate BETWEEN #StartDateF AND #EndDateF) AND DATEDIFF(day, SELECT MAX(StartDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID, SELECT MAX(EndDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID)> 1
I basically want to find out in the instance an employee has more than one contract, regardless of leaving and coming back or pay review, if the current active contract is one day different to the previous contract. This should by my thinking give me all new starters only.
Trouble is I am still trying to get my head around when to use aggregate functions not in a select and when to apply the HAVING clause.
Any help would be appreciated to help me understand why my lack of understanding is causing this query/logic to fail.
Thanks
EDIT
Ok I am still bashing away at this solution and this is syntactically incorrect. In an attempt to remove some of the ambiguity here is the query, with an update;
Declare #StartDateF varchar(10)
Set #StartDateF = '2012-08-03'
Declare #EndDateF varchar(10)
Set #EndDateF = '2012-09-04'
SELECT w1.Worker_ID, w1.Title, w1.FirstName, w1.Surname,w1.Gender, w1.DateofBirth,
dbo.[Contract].StartDate, (select w2.surname + ',' + w2.firstname from worker w2 WITH (NOLOCK) where w2.worker_ID = w1.manager)as Manager, dbo.Grade.GradeDescription AS JobTitle, dbo.Grade.Discipline,
CASE WHEN dbo.[Contract].ContractType_ID = 1 OR dbo.[Contract].ContractType_ID = 2 OR dbo.[Contract].ContractType_ID = 5 OR dbo.[Contract].ContractType_ID = 6
THEN 'Staff' ELSE 'Contractor' END AS ContractType
FROM dbo.Worker w1 WITH (NOLOCK) inner join
dbo.[Contract] WITH (NOLOCK) ON dbo.[Contract].Worker_ID = w1.Worker_ID inner join
dbo.Grade WITH (NOLOCK) ON dbo.Grade.Grade_ID = dbo.[Contract].Grade_ID
WHERE
(dbo.[Contract].StartDate BETWEEN #StartDateF AND #EndDateF AND EndDate IS NULL)
group by
w1.Worker_ID, w1.Title, w1.FirstName, w1.Surname,w1.Gender, w1.DateofBirth,
dbo.[Contract].StartDate, manager, dbo.Grade.Discipline,dbo.Grade.GradeDescription, dbo.[Contract].ContractType_ID
Having DATEDIFF(day, SELECT MAX(StartDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID, SELECT MAX(EndDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID)
I have added the group by and the having clause but now I am getting the following errors
Msg 156, Level 15, State 1, Line 24
Incorrect syntax near the keyword 'SELECT'.
Msg 102, Level 15, State 1, Line 24
Incorrect syntax near ','.
Msg 102, Level 15, State 1, Line 24
Incorrect syntax near ')'.
These all relate the the functions in the having clause no doubt you can see. But I cannot understand what is wrong with this query and this is mainly the question. I need to understand the SQL functions enough so that I can implement th correct solution.
I have followed up the DATEDIFF() function here http://msdn.microsoft.com/en-us/library/ms189794.aspx
I can see that using functions within this function is acceptable according to the MS documentation.
EDIT
Commenting out the Having clause gives me the result set I expect. It is showing people with changes to contracts(pay rise) but this is information that no one should be seeing, these are now the only records that need filtering out
EDIT
I have made some improvements and overcome the error messages now, but I am still getting people where pay rises have occured. Here is the amended query from the group by
group by
w1.Worker_ID, w1.Title, w1.FirstName, w1.Surname,w1.Gender, w1.DateofBirth,
dbo.[Contract].StartDate, manager, dbo.Grade.Discipline,dbo.Grade.GradeDescription, dbo.[Contract].ContractType_ID, w1.Worker_ID
Having
(((dbo.[Contract].StartDate BETWEEN #StartDateF AND #EndDateF)
AND COUNT(dbo.[Contract].Worker_ID) = 1)
OR
((dbo.[Contract].StartDate BETWEEN #StartDateF AND #EndDateF)
AND DATEDIFF(day, (SELECT MAX(EndDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID), (SELECT MAX(StartDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID))>1))
To get workers with more than one contract, you would use:
select c.workerID
from Contract c
group by c.workerID
having count(distinct contractID) > 1
It sounds, though, like you only want to count everything but the new start ones. You can do this with something like:
select w.workerID
from Contract c
where c.ContractType = 'New'
group by w.workerID
having count(distinct contractID) > 1
Because you didn't provide the details of what the tables look like, what sample input data looks like, and the results you want to achieve, this is about the best that can be done.
WHERE ( (dbo.[Contract].StartDate BETWEEN #StartDateF AND #EndDateF)AND dbo.[Contract].Worker_ID
IN (select worker_id from dbo.[Contract]
group by worker_id
having count(worker_id) = 1))
OR
((dbo.[Contract].StartDate BETWEEN #StartDateF AND #EndDateF)
AND DATEDIFF(day, (SELECT MAX(EndDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID), (SELECT MAX(StartDate)FROM dbo.[Contract] WHERE dbo.[Contract].Worker_ID = w1.Worker_ID))>1
AND dbo.[Contract].Worker_ID = w1.Worker_ID )
Now works for me :)
As far as I know, doctrine dql does not allow subqueries inside a join.
I have a table and a trasnlation table. The relation is one to many. One record has many translations.
In order to get the right translation row from the translatiosn table I did subselects in the select clause:
$query = $this->getDoctrine()->getEntityManager()
->createQuery('
SELECT w.id, w.pastid, w.name, w.jsonParameters as params, m.id as milestone_id, m.name as milestone_name,
m.slug as milestone_slug, m.startdate as milestone_start, m.enddate as milestone_end,
w.expand as expand, w.backgroundcolor as background, w.colorschema as colorschema, w.headline as headline, w.subheadline as subheadline, w.text as text, w.expandheight as expandheight,
w.url as url, w.created as created, w.updated as updated, wt.name as wtname, ws.weight as width, ws.height as height
, (SELECT t.headline FROM AdminBundle:widgetTranslation t WHERE t.widget = w.id and t.locale = :published) AS headline_trans
, (SELECT t2.subheadline FROM AdminBundle:widgetTranslation t2 WHERE t2.widget = w.id and t2.locale = :published) AS subheadline_trans
, (SELECT t3.text FROM AdminBundle:widgetTranslation t3 WHERE t3.widget = w.id and t3.locale = :published) AS text_trans
FROM AdminBundle:Widget w
JOIN w.milestone m
JOIN w.widgetType wt
JOIN w.widgetShape ws
WHERE w.published = 1
ORDER BY m.order, w.order
')->setParameter('published', $currentLocale);
$result = $query->getArrayResult();
This query does the job but Iam worried about performance, is there a better query to do this?
First, I need to clarify a little thing.
There is something really wrong in your question.
You are using an ORM, right? But you're talking about tables.
An ORM doesn't care about how your tables are organized or linked together. It only matters on the relationship between the objects themselves.
However, if you are worried about performance, the best thing to do, as you would do it with any SQL query, is to EXPLAIN it.
Since you're using Symfony 2.0, you may use the profiler (accessible in dev env) to see the EXPLAIN results of your query.
Make sure that is using index, and the number of rows to lookup is not too high.
Depending on your result you may need to use another strategy to fetch your translation or re-write your query in native SQL to allow more flexibility.