Slow performance of query [duplicate] - sqlite

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Slow Performance of Sql Query
Hi,
I have asked the performance of query and i tried to simplyfy it.but still it not works.I am adding my query below.Please can you simplify it more effectively
select
r.parent_itemid f_id,
parent_item.name f_name,
parent_item.typeid f_typeid,
parent_item.ownerid f_ownerid,
parent_item.created f_created,
parent_item.modifiedby f_modifiedby,
parent_item.modified f_modified,
pt.name f_tname,
child_item.id i_id,
t.name i_tname,
child_item.typeid i_typeid,
child_item.name i_name,
child_item.ownerid i_ownerid,
child_item.created i_created,
child_item.modifiedby i_modifiedby,
child_item.modified i_modified,
r.ordinal i_ordinal
from
item child_item,
type t,
relation r,
item parent_item,
type pt
where
r.child_itemid = child_item.id and
t.id=child_item.typeid and
parent_item.id = r.parent_itemid and
pt.id = parent_item.typeid
and parent_item.id in (
select
itemid
from
permission
where
itemid=parent_item.id and
(holder_itemid in (10,100) and level > 0) )
order by
r.parent_itemid,
r.relation_typeid,
r.ordinal
Thanks you
regards
jennie

You don't need the correlated subquery on the permissions. Any other problems need to be fixed by checking the indexes indexes on the join fields (like item.child_itemid ) & filter fields (like permission.holder_itemid) will help the performance of your query
select
r.parent_itemid f_id,
parent_item.name f_name,
parent_item.typeid f_typeid,
parent_item.ownerid f_ownerid,
parent_item.created f_created,
parent_item.modifiedby f_modifiedby,
parent_item.modified f_modified,
pt.name f_tname,
child_item.id i_id,
t.name i_tname,
child_item.typeid i_typeid,
child_item.name i_name,
child_item.ownerid i_ownerid,
child_item.created i_created,
child_item.modifiedby i_modifiedby,
child_item.modified i_modified,
r.ordinal i_ordinal
from
item child_item,
type t,
relation r,
item parent_item,
type pt,
permission p
where
r.child_itemid = child_item.id
and t.id=child_item.typeid
and parent_item.id = r.parent_itemid
and pt.id = parent_item.typeid
and parent_item.id = p.itemid
and p.holder_itemid in (10, 100)
and p.level > 0
order by
r.parent_itemid,
r.relation_typeid,
r.ordinal

Try removing the subquery, something like:
select
r.parent_itemid f_id,
parent_item.name f_name,
parent_item.typeid f_typeid,
parent_item.ownerid f_ownerid,
parent_item.created f_created,
parent_item.modifiedby f_modifiedby,
parent_item.modified f_modified,
pt.name f_tname,
child_item.id i_id,
t.name i_tname,
child_item.typeid i_typeid,
child_item.name i_name,
child_item.ownerid i_ownerid,
child_item.created i_created,
child_item.modifiedby i_modifiedby,
child_item.modified i_modified,
r.ordinal i_ordinal
from
item child_item,
type t,
relation r,
item parent_item,
type pt,
permission perm /* <<< added this line <<< */
where
r.child_itemid = child_item.id and
t.id=child_item.typeid and
parent_item.id = r.parent_itemid and
pt.id = parent_item.typeid
and parent_item.id = perm.itemid /* <<< modified this line <<< */
and perm.itemid = parent_item.id /* <<< copied these 2 lines from the subquery <<< */
and (perm.holder_itemid in (10,100) and perm.level > 0) ) /* <<< */
order by
r.parent_itemid,
r.relation_typeid,
r.ordinal
Give it a try, and see if it works and makes any difference.

Related

Why does not work SELECT query in SQLite3?

Insert:
INSERT INTO MATCHES ("url", "league", "home_team", "away_team", "current_time", "current_score", "penalty_not_score_team", "penalty_not_score_time", "penalty_not_score_name")
VALUES ('https://www.myscore.com.ua/match/8nsN4E4k/#match-summary', 'АРГЕНТИНА: Кубок Аргентины - 1/16', 'Химнасия Мендоса', 'Ривер Плейт', 'Послес.п.', '1 - 2', 'home', '4''', 'Andrada B.')
Result:
Result: query completed successfully
At line 1:
INSERT INTO MATCHES ("url", "league", "home_team", "away_team", "current_time", "current_score", "penalty_not_score_team", "penalty_not_score_time", "penalty_not_score_name")
VALUES ('https://www.myscore.com.ua/match/8nsN4E4k/#match-s...', 'АРГЕНТИНА: Кубок Аргентины - 1/16', 'Химнасия Мендоса', 'Ривер Плейт', 'Послес.п.', '1 - 2', 'home', '4''', 'Andrada B.')
Select:
SELECT *
FROM MATCHES
WHERE (url="https://www.myscore.com.ua/match/8nsN4E4k/#match-summary" AND league="АРГЕНТИНА: Кубок Аргентины - 1/16" AND home_team="Химнасия Мендоса" AND away_team="Ривер Плейт" AND current_time="Послес.п." AND current_score="1 - 2" AND penalty_not_score_team="home" AND penalty_not_score_time="4'" AND penalty_not_score_name="Andrada B.")
Result:
Result: 0 rows returned in 1ms
At line 1:
SELECT *
FROM MATCHES
WHERE (url="https://www.myscore.com.ua/match/8nsN4E4k/#match-s..." AND league="АРГЕНТИНА: Кубок Аргентины - 1/16" AND home_team="Химнасия Мендоса" AND away_team="Ривер Плейт" AND current_time="Послес.п." AND current_score="1 - 2" AND penalty_not_score_team="home" AND penalty_not_score_time="4'" AND penalty_not_score_name="Andrada B.")
The record in the database is 100%, I watch it with DB Browser for SQLite3
I tried different syntaxes - to no avail.
Short queries work, but I need to sample all items.
You have to enquote current_time otherwise it is treated as function call SELECT current_time:
AND current_time="Послес.п."
=>
AND "current_time"='Послес.п.'
db<>fiddle demo
CREATE TABLE t(current_time CHAR(10));
INSERT INTO t(current_time) VALUES('aaa');
-- 1 row
SELECT * FROM t;
-- 0 rows returned
SELECT *
FROM t
WHERE current_time = 'aaa';
-- 1 row returned
SELECT *
FROM t
WHERE "current_time" = 'aaa';

Application Engine Peoplecode bind variables

I have the below PeopleCode step in an Application Engine program that reads a CSV file using a File Layout and then inserts the data into a table, and I am just trying to get a better understanding of how the the line of code (&SQL1 = CreateSQL("%Insert(:1)");) in the below script gets generated. It looks like the CreateSQL is using a bind variable (:1) inside the Insert statement, but I am struggling as where to find where this variable is defined in the program.
Function EditRecord(&REC As Record) Returns boolean;
Local integer &E;
&REC.ExecuteEdits(%Edit_Required + %Edit_DateRange + %Edit_YesNo + %Edit_OneZero);
If &REC.IsEditError Then
For &E = 1 To &REC.FieldCount
&MYFIELD = &REC.GetField(&E);
If &MYFIELD.EditError Then
&MSGNUM = &MYFIELD.MessageNumber;
&MSGSET = &MYFIELD.MessageSetNumber;
&LOGFILE.WriteLine("****Record:" | &REC.Name | ", Field:" | &MYFIELD.Name);
&LOGFILE.WriteLine("****" | MsgGet(&MSGSET, &MSGNUM, ""));
End-If;
End-For;
Return False;
Else
Return True;
End-If;
End-Function;
Function ImportSegment(&RS2 As Rowset, &RSParent As Rowset)
Local Rowset &RS1, &RSP;
Local string &RecordName;
Local Record &REC2, &RECP;
Local SQL &SQL1;
Local integer &I, &L;
&SQL1 = CreateSQL("%Insert(:1)");
rem &SQL1 = CreateSQL("%Insert(:1) Order by COUNT_ORDER");
&RecordName = "RECORD." | &RS2.DBRecordName;
&REC2 = CreateRecord(#(&RecordName));
&RECP = &RSParent(1).GetRecord(#(&RecordName));
For &I = 1 To &RS2.ActiveRowCount
&RS2(&I).GetRecord(1).CopyFieldsTo(&REC2);
If (EditRecord(&REC2)) Then
&SQL1.Execute(&REC2);
&RS2(&I).GetRecord(1).CopyFieldsTo(&RECP);
For &L = 1 To &RS2.GetRow(&I).ChildCount
&RS1 = &RS2.GetRow(&I).GetRowset(&L);
If (&RS1 <> Null) Then
&RSP = &RSParent.GetRow(1).GetRowset(&L);
ImportSegment(&RS1, &RSP);
End-If;
End-For;
If &RSParent.ActiveRowCount > 0 Then
&RSParent.DeleteRow(1);
End-If;
Else
&LOGFILE.WriteRowset(&RS);
&LOGFILE.WriteLine("****Correct error in this record and delete all error messages");
&LOGFILE.WriteRecord(&REC2);
For &L = 1 To &RS2.GetRow(&I).ChildCount
&RS1 = &RS2.GetRow(&I).GetRowset(&L);
If (&RS1 <> Null) Then
&LOGFILE.WriteRowset(&RS1);
End-If;
End-For;
End-If;
End-For;
End-Function;
rem *****************************************************************;
rem * PeopleCode to Import Data *;
rem *****************************************************************;
Local File &FILE1, &FILE3;
Local Record &REC1;
Local SQL &SQL1;
Local Rowset &RS1, &RS2;
Local integer &M;
&FILE1 = GetFile("\\nt115\apps\interface_prod\interface_in\Item_Loader\ItemPriceFile.csv", "r", "a", %FilePath_Absolute);
&LOGFILE = GetFile("\\nt115\apps\interface_prod\interface_in\Item_Loader\ItemPriceFile.txt", "r", "a", %FilePath_Absolute);
&FILE1.SetFileLayout(FileLayout.GH_ITM_PR_UPDT);
&LOGFILE.SetFileLayout(FileLayout.GH_ITM_PR_UPDT);
&RS1 = &FILE1.CreateRowset();
&RS = CreateRowset(Record.GH_ITM_PR_UPDT);
REM &SQL1 = CreateSQL("%Insert(:1)");
&SQL1 = CreateSQL("%Insert(:1)");
/*Skip Header Row: The following line of code reads the first line in the file layout (the header)
and does nothing. Then the pointer goes to the next line in the file and starts using the
file.readrowset*/
&some_boolean = &FILE1.ReadLine(&string);
&RS1 = &FILE1.ReadRowset();
While &RS1 <> Null
ImportSegment(&RS1, &RS);
&RS1 = &FILE1.ReadRowset();
End-While;
&FILE1.Close();
&LOGFILE.Close();
The :1 is coming from the line further down &SQL1.Execute(&REC2);
&REC2 gets assigned a record object, so the line &SQL1.Execute(&REC2); evaluates to %Insert(your_record_object)
Here is a simple example that's doing basically the same thing
Here is a description of %Insert
Answer because too long to comment:
The table name is most likely (PS_)GH_ITM_PR_UPDT. The general consensus is to name the FileLayout the same as the record it is based on.
If not, it is defined in FileLayout.GH_ITM_PR_UPDT. Open the FileLayout, right click the segment and under 'Selected Node Properties' you will find the 'File Record Name'.
In your code this record is carried over into &RS1.
&FILE1.SetFileLayout(FileLayout.GH_ITM_PR_UPDT);
&RS1 = &FILE1.CreateRowset();
The rowset is a collection of rows. A row consists of records and a record is a row of data from a database table. (Peoplesoft Object Data Types are fun...)
This rowset is filled with data in the following statement:
&RS1 = &FILE1.ReadRowset();
This uses your file as input and outputs a rowset collection, mapping the data to records based on how you defined your FileLayout.
The result is fed into the ImportSegment function:
ImportSegment(&RS1, &RS);
Function ImportSegment(&RS2 As Rowset, &RSParent As Rowset)
&RS2 in the function is a reference to &RS1 in the rest of your code.
The table name is also hidden here:
&RecordName = "RECORD." | &RS2.DBRecordName;
So if you can't/don't want to check the FileLayout, you could output &RS2.DBRecordName with a messagebox and your answer will be Message Log of your Process Monitor.
Finally a record object is created for this database table and it is filled with a row from the rowset. This record is inserted into the database table:
&REC2 = CreateRecord(#(&RecordName));
&RS2(&I).GetRecord(1).CopyFieldsTo(&REC2);
&SQL1 = CreateSQL("%Insert(:1)");
&SQL1.Execute(&REC2);
TLDR:
Table name can be found in the FileLayout or output in the ImportSegment Function as &RS2.DBRecordName

Doctrine Query Builder Length

how to reproduce in query builder this
AND LENGTH( CONCAT_WS('', lsc.door_code, lsc.alarm_code, lsc.service_code, lsc.master_log) ) > 0"
I try like this
$orX = $qb->expr()->orX();
$orX
->add($qb->expr()->length('lsc.doorCode > 0'))
->add($qb->expr()->length('lsc.alarmCode > 0'))
->add($qb->expr()->length('lsc.serviceCode > 0'))
->add($qb->expr()->length('lsc.masterLog > 0'));
$qb->andWhere($orX);
have error:
[Syntax Error] line 0, col 549: Error: Expected Doctrine\ORM\Query\Lexer::T_CLOSE_PARENTHESIS, got '>'
and my query
SELECT
CONCAT(
IFNULL(location.name, ''),' (',IFNULL(location.streetAddress, ''),' ',IFNULL(location.staircase, ''),' ',IFNULL(location.flatNumber, ''),' ',IFNULL(location.postal, ''),' ',IFNULL(location.postOffice, ''),')') as address,
lsc.doorCode,
lsc.serviceCode,
lsc.alarmCode,
lsc.masterLog
FROM AppBundle:LocationServiceCompany lsc INNER JOIN lsc.location location WHERE lsc.serviceCompany = :sc AND (LENGTH(lsc.doorCode > 0) OR LENGTH(lsc.alarmCode > 0) OR LENGTH(lsc.serviceCode > 0) OR LENGTH(lsc.masterLog > 0))
how to use length ?
I came here because I had similar problem with using length in query builder.
I tried ($qb->expr()->length('lsc.doorCode > 0') as suggested above but it did not work. The SQL output of this is "LENGTH(lsc.doorCode > 0)" which throws an error.
My solution to this problem is:
$qb->expr()->gt($qb->expr()->length('lsc.doorCode'), 0)
which generates "LENGTH(lsc.doorCode) > 0" that is correct SQL syntax.
You have a typo here:
$orX = $qb->expr()->orX();
$orX
->add($qb->expr()->length('lsc.doorCode > 0')); <--
->add($qb->expr()->length('lsc.alarmCode > 0'))
->add($qb->expr()->length('lsc.serviceCode > 0'))
->add($qb->expr()->length('lsc.masterLog > 0'));
$qb->andWhere($orX);
remove the ; and get rid of the syntax error.
And for the concat length, you can try something like this:
$query->andWhere("LENGTH( CONCAT_WS('', lsc.door_code, lsc.alarm_code, lsc.service_code, lsc.master_log) ) > 0")
After the update
You can run native SQL queries with Doctrine like mentioned here:
http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/native-sql.html#the-nativequery-class
For reference:
<?php
use Doctrine\ORM\Query\ResultSetMapping;
$rsm = new ResultSetMapping();
// build rsm here
$query = $entityManager->createNativeQuery('SELECT id, name, discr FROM users WHERE name = ?', $rsm);
$query->setParameter(1, 'romanb');
$users = $query->getResult();
If your updated query works, this would be a way to make it work.

Bosun how to add series with different tags?

I'm trying to add 4 series using bosun expressions. They are from 1,2,3,4 weeks ago. I shifted them using shift() to have current time. But I can't add them since they have the shift=1w etc tags. How can I add these series together?
Thank you
edit: here's the query for 2 weeks
$period = d("1w")
$duration = d("30m")
$week1end = tod(1 * $period )
$week1start = tod(1 * $period + $duration )
$week2end = tod(2 * $period )
$week2start = tod(2 * $period + $duration )
$q1 = q("avg:1m-avg:os.cpu{host=myhost}", $week1start, $week1end)
$q2 = q("avg:1m-avg:os.cpu{host=myhost}", $week2start, $week2end)
$shiftedq1 = shift($q1, "1w")
$shiftedq2 = shift($q2, "2w")
$shiftedq1+ $shiftedq2
edit: here's what Bosun said
The problem is similar to: How do I add the series present in the output of an over query:
over("avg:1m-avg:os.cpu{host=myhost}", "30m", "1w", 2)
There is a new function called addtags that is pending documentation (see https://raw.githubusercontent.com/bosun-monitor/bosun/master/docs/expressions.md for draft) which seems to work when combined with rename. Changing the last line to:
$shiftedq1+addtags(rename($shiftedq2,"shift=shiftq2"),"shift=1w")
should generate a single result group like { host=hostname, shift=1w, shiftq2=2w }. If you add additional queries for q3 and q4 you probably need to rename the shift tag for those to unique values like shiftq3 and shiftq4.
If you were using a numbersets instead of seriessets, then the Transpose function would let you "Drop" the unwanted tags. This is useful when generating alerts, since crit and warn need a single number value not a series set:
$average_per_q = avg(merge($shiftedq1,$shiftedq2))
$sum_over_all = sum(t($average_per_q,"host"))
Result: { host=hostname } 7.008055555555557
Side note you probably want to use a counter for os.cpu instead of a gauge. Example: $q1 = q("avg:1m-avg:rate{counter,,1}:os.cpu{. Without that rate section you are using the raw counter values instead of the gauge value.

Finding people within a distance of x miles with graticule gem and Rails 3.1

I am using Graticule to find people within a distance of x miles from an event. I need to send mail to everyone who is within x miles of the event.
Here is my code :-
user.rb
def self.weekly_update
#users = User.all
#users.each do |u|
#events = Event.all_with_distance([u.geo_lat, u.geo_lng]).where("start > ?", Time.zone.now).where("distance < 15")
UsersMailer.weekly_mail(u.email, #events).deliver
end
end
def self.all_with_distance(origin)
distance_sql = sql_for_distance(origin)
select("#{table_name}.*, #{distance_sql} AS distance").select("`locations`.`geo_lat`, `locations`.`geo_lng`, `locations`.`name` as location_name").joins(:location)
end
geo_search.rb
module GeoSearch
def sql_for_distance(origin)
Graticule::Distance::Spherical.to_sql(
:latitude => origin[0],
:longitude => origin[1],
:latitude_column => "`locations`.`geo_lat`",
:longitude_column => "`locations`.`geo_lng`",
:units => :kilometers
)
end
end
end
This is the error which I am getting :-
/usr/local/ruby/lib/ruby/gems/1.9.1/gems/railties-3.1.0/lib/rails/commands/runner.rb:49:in `eval': Mysql2::Error: Unknown column 'distance' in 'where clause': SELECT events.*, (ACOS( SIN(RADIANS(26.8465108)) * SIN(RADIANS(`locations`.`geo_lat`)) + COS(RADIANS(26.8465108)) * COS(RADIANS(`locations`.`geo_lat`)) * COS(RADIANS(`locations`.`geo_lng`) - RADIANS(80.9466832)) ) * 6378.135) AS distance, `locations`.`geo_lat`, `locations`.`geo_lng`, `locations`.`name` as location_name FROM `events` INNER JOIN `locations` ON `locations`.`id` = `events`.`location_id` WHERE (start > '2011-12-10 10:38:20') AND (distance < 15) (ActionView::Template::Error)
If I remove .where("distance < 15") to .order("distance") everything works fine.
In both standard SQL and the dialect supported by MySQL, column aliases can't be used in a WHERE clause; you must use the column. Here's what it would look like for your code, placing the call to where in all_with_distance:
select("#{table_name}.*, #{distance_sql} AS distance")
.select("`locations`.`geo_lat`, `locations`.`geo_lng`, `locations`.`name` as location_name")
.joins(:location)
.where("#{distance_sql} < 50")

Resources