I am creating a app with ionic 3. In the app I added a search filter which filter users according to their role. I have 5 tables from which I have to pull the data.In term to do so, I did these steps..
created each table index with the required data.
CREATE INDEX IF NOT EXISTS AccountIndexTable ON Accounts(id,roleId,firstName,lastName,email,accountId);
CREATE INDEX IF NOT EXISTS AddressIndexTable ON Addresses(id,addressTypeId,street,city,state,country,pincode,accountId)
CREATE INDEX IF NOT EXISTS CompanyIndexTable ON Companies(id,name,accountId)
CREATE INDEX IF NOT EXISTS CommunicationIndexTable ON Communications(id,phone,accountId)
CREATE INDEX IF NOT EXISTS OptionalInfoIndexTable ON CustomerOptionalInfos(id,customerType,accountId)
and then i used join to get data. Before that I have questions
1: Do I need to use the indexed table in my join?
the Join query:
SELECT Accounts.accountId AS accountID,
Accounts.roleId AS roleID,
Accounts.email AS email,
Accounts.firstName || " " || Accounts.lastName AS name,
Addresses.street AS street,
Addresses.city AS city,
Addresses.state AS state,
Addresses.country AS country,
Addresses.pincode AS pincode,
Communications.phone AS phone,
Companies.name AS compName,
CustomerOptionalInfos.customerType AS cType
FROM Accounts
LEFT JOIN Addresses ON Addresses.accountId=Accounts.accountId
AND Addresses.addressTypeId=1
LEFT JOIN Communications ON Communications.accountId=Accounts.accountId
LEFT JOIN Companies ON Companies.accountId=Accounts.accountId
LEFT JOIN CustomerOptionalInfos ON CustomerOptionalInfos.accountId=Accounts.accountId
WHERE 1=1
AND Accounts.isDelete!='true'
AND Accounts.firstName!= ''
AND Accounts.roleId = 5
ORDER BY Accounts.firstName ASC
LIMIT ?,?
It still takes +2 seconds to execute. Please suggest me the better way of doing this. I have a large number of data.
Related
I have two tables: Player(name, email) and TeamPlayer(PlayerName, Team, Active). Tables Player and TeamPlayer are connected via Player.name = TeamPlayer.PlayerName.
I want to get all the elements of TeamPlayer, but also the elements in Player not in TeamPlayer. The attribute Active is important also. I execute the next command, but it did not work:
SELECT DISTINCT Player.Name, TeamPlayer.Team, TeamPlayer.Active FROM Player LEFT JOIN TeamPlayer ON TeamPlayer.PlayerName=Player.Name ORDER BY TeamPlayer.Active;
Is it possible?
You can use UNION ALL for the table TeamPlayer and the rows from Player that don't exist in TeamPlayer:
SELECT PlayerName AS Name, Team, null AS email, Active
FROM TeamPlayer
UNION ALL
SELECT t.Name, null, t.email, 0
FROM Player t
WHERE NOT EXISTS (
SELECT 1 FROM TeamPlayer
WHERE PlayerName = t.Name
)
ORDER BY Active
Since the 2 tables don't have the same columns, the non existing columns for each case will be NULL or 0 for the column Active.
You can change it as you wish.
I've got a database that I'm tying into our access control system. I'm trying to create a report showing the user's first entry and last exit from the building. I've put a query together which hows me the right single result for one user, but I want to automate this to run through a list of users.
The query for grabbing the first entry for one user (as it is currently) is as follows:
SELECT TOP 1 AccessLog.ID, AccessLog.Date, First(AccessLog.Time) AS FirstOfTime, AccessLog.User, AccessLog.Details, AccessLog.Event, AccessLog.Department, AccessLog.Where
FROM AccessLog
GROUP BY AccessLog.ID, AccessLog.Date, AccessLog.User, AccessLog.Details, AccessLog.Event, AccessLog.Department, AccessLog.Where
HAVING (((AccessLog.Date)=#DATE#) AND ((AccessLog.User)="USERNAME") AND ((AccessLog.Where)="Front Door (In)"))
ORDER BY First(AccessLog.Time);
I have a separate table of all the usernames, would it be possible to do a run-as for this?
Basically I'm a complete Access n00b, in the past I'd have done this with PHP and a recurring function but this should be fairly easy to achieve within Access, shouldn't it? Any help much appreciated.
You can get a list of the arrival times for all users by date with the following query. Save the query definition under the name "ArriveTimes":
SELECT AccessLog.Date, AccessLog.User, Min(AccessLog.Time) AS ArriveTime
FROM AccessLog
WHERE (((AccessLog.Where)="Front Door (In)"))
GROUP BY AccessLog.Date, AccessLog.User;
Similarly, you can get the departure times with a similar query saved as "DepartTimes":
SELECT AccessLog.Date, AccessLog.User, Max(AccessLog.Time) AS DepartTime
FROM AccessLog
WHERE (((AccessLog.Where)="Front Door (Out)"))
GROUP BY AccessLog.Date, AccessLog.User;
Then you can join the [ArriveTimes] and the [DepartTimes] queries together (JOIN on [Date] and [User]) and pull the other details by joining on two instances of the [AccessLog] table (JOIN on [Date], [User], and [Time]):
SELECT ArriveTimes.Date, ArriveTimes.User,
ArriveTimes.ArriveTime, AccessLog_A.Details AS ArriveDetails,
DepartTimes.DepartTime, AccessLog_D.Details AS DepartDetails
FROM
(AccessLog AS AccessLog_A INNER JOIN
(ArriveTimes INNER JOIN DepartTimes
ON (ArriveTimes.User = DepartTimes.User)
AND (ArriveTimes.Date = DepartTimes.Date)
)
ON (ArriveTimes.ArriveTime = AccessLog_A.Time)
AND (AccessLog_A.User = ArriveTimes.User)
AND (AccessLog_A.Date = ArriveTimes.Date)
)
INNER JOIN AccessLog AS AccessLog_D
ON (DepartTimes.DepartTime = AccessLog_D.Time)
AND (DepartTimes.User = AccessLog_D.User)
AND (DepartTimes.Date = AccessLog_D.Date);
Table: "user"
- Userid - Name -
(every userid is unique)
Table: "money spend"
- Userid - money spend -
(a userid may have several entries with different "money spend")
Now, I need the total sum of the money spend by a user.
To conclude, I need the following view:
- name - sum (money spend) -
Wich statement may give me this result?
You can use an aggregate function and group by:
select u.name, sum(ms.money)
from user u, money_spend ms
where u.userid = ms.userid
group by u.userid
Note that this here assumes that every user has at least 1 row in the money_spend table: http://www.sqlite.org/lang_aggfunc.html
Due to the way that aggregate functions work, you could set up the money_spend table with a 0 value for each user so you don't run into any issues :)
Because you might have users without any entry in table money_spend you need an outer join:
select n.name, sum(ms.money)
from user n
left outer join money_spend ms on (n.userid = ms.userid)
group by n.name
Edit: To be sure this all works I just tried this
create table user(userid, name);
insert into user values (1, 'user1');
insert into user values (2, 'user2');
insert into user values (3, 'user3');
create table moneyspend(userid, amount);
insert into moneyspend values (1,10);
insert into moneyspend values (1,20);
insert into moneyspend values (2,100);
select * from user;
select * from moneyspend;
select u.name, sum(m.amount)
from user u
left outer join moneyspend m on (u.userid = m.userid)
group by u.name;
drop table user;
drop table moneyspend;
The console output is the following (testSQLite is my test DB, testsql.sql is the above)
hols-Mac:div4 hol$ sqlite3 -init testsql.sql testSQLite
-- Loading resources from testsql.sql
1|user1
2|user2
3|user3
1|10
1|20
2|100
user1|30
user2|100
user3|
SQLite version 3.6.12
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite>
I have a 800MB MS Access database that I migrated to SQLite. The structure of the database is as follows (the SQLite database, after migration, is around 330MB):
The table Occurrence has 1,600,000 records. The table looks like:
CREATE TABLE Occurrence
(
SimulationID INTEGER, SimRunID INTEGER, OccurrenceID INTEGER,
OccurrenceTypeID INTEGER, Period INTEGER, HasSucceeded BOOL,
PRIMARY KEY (SimulationID, SimRunID, OccurrenceID)
)
It has the following indexes:
CREATE INDEX "Occurrence_HasSucceeded_idx" ON "Occurrence" ("HasSucceeded" ASC)
CREATE INDEX "Occurrence_OccurrenceID_idx" ON "Occurrence" ("OccurrenceID" ASC)
CREATE INDEX "Occurrence_SimRunID_idx" ON "Occurrence" ("SimRunID" ASC)
CREATE INDEX "Occurrence_SimulationID_idx" ON "Occurrence" ("SimulationID" ASC)
The table OccurrenceParticipant has 3,400,000 records. The table looks like:
CREATE TABLE OccurrenceParticipant
(
SimulationID INTEGER, SimRunID INTEGER, OccurrenceID INTEGER,
RoleTypeID INTEGER, ParticipantID INTEGER
)
It has the following indexes:
CREATE INDEX "OccurrenceParticipant_OccurrenceID_idx" ON "OccurrenceParticipant" ("OccurrenceID" ASC)
CREATE INDEX "OccurrenceParticipant_ParticipantID_idx" ON "OccurrenceParticipant" ("ParticipantID" ASC)
CREATE INDEX "OccurrenceParticipant_RoleType_idx" ON "OccurrenceParticipant" ("RoleTypeID" ASC)
CREATE INDEX "OccurrenceParticipant_SimRunID_idx" ON "OccurrenceParticipant" ("SimRunID" ASC)
CREATE INDEX "OccurrenceParticipant_SimulationID_idx" ON "OccurrenceParticipant" ("SimulationID" ASC)
The table InitialParticipant has 130 records. The structure of the table is
CREATE TABLE InitialParticipant
(
ParticipantID INTEGER PRIMARY KEY, ParticipantTypeID INTEGER,
ParticipantGroupID INTEGER
)
The table has the following indexes:
CREATE INDEX "initialpart_participantTypeID_idx" ON "InitialParticipant" ("ParticipantGroupID" ASC)
CREATE INDEX "initialpart_ParticipantID_idx" ON "InitialParticipant" ("ParticipantID" ASC)
The table ParticipantGroup has 22 records. It looks like
CREATE TABLE ParticipantGroup (
ParticipantGroupID INTEGER, ParticipantGroupTypeID INTEGER,
Description varchar (50), PRIMARY KEY( ParticipantGroupID )
)
The table has the following index:
CREATE INDEX "ParticipantGroup_ParticipantGroupID_idx" ON "ParticipantGroup" ("ParticipantGroupID" ASC)
The table tmpSimArgs has 18 records. It has the following structure:
CREATE TABLE tmpSimArgs (SimulationID varchar, SimRunID int(10))
And the following indexes:
CREATE INDEX tmpSimArgs_SimRunID_idx ON tmpSimArgs(SimRunID ASC)
CREATE INDEX tmpSimArgs_SimulationID_idx ON tmpSimArgs(SimulationID ASC)
The table ‘tmpPartArgs’ has 80 records. It has the below structure:
CREATE TABLE tmpPartArgs(participantID INT)
And the below index:
CREATE INDEX tmpPartArgs_participantID_idx ON tmpPartArgs(participantID ASC)
I have a query that involves multiple INNER JOINs and the problem I am facing is the Access version of the query takes about a second whereas the SQLite version of the same query takes 10 seconds (about 10 times slow!) It is impossible for me to migrate back to Access and SQLite is my only option.
I am new to writing database queries hence these queries might look stupid, so please advise on anything you see faulty or kid-dish.
The query in Access is (the entire query takes 1 second to execute):
SELECT ParticipantGroup.Description, Occurrence.SimulationID, Occurrence.SimRunID, Occurrence.Period, Count(OccurrenceParticipant.ParticipantID) AS CountOfParticipantID FROM
(
ParticipantGroup INNER JOIN InitialParticipant ON ParticipantGroup.ParticipantGroupID = InitialParticipant.ParticipantGroupID
) INNER JOIN
(
tmpPartArgs INNER JOIN
(
(
tmpSimArgs INNER JOIN Occurrence ON (tmpSimArgs.SimRunID = Occurrence.SimRunID) AND (tmpSimArgs.SimulationID = Occurrence.SimulationID)
) INNER JOIN OccurrenceParticipant ON (Occurrence.OccurrenceID = OccurrenceParticipant.OccurrenceID) AND (Occurrence.SimRunID = OccurrenceParticipant.SimRunID) AND (Occurrence.SimulationID = OccurrenceParticipant.SimulationID)
) ON tmpPartArgs.participantID = OccurrenceParticipant.ParticipantID
) ON InitialParticipant.ParticipantID = OccurrenceParticipant.ParticipantID WHERE (((OccurrenceParticipant.RoleTypeID)=52 Or (OccurrenceParticipant.RoleTypeID)=49)) AND Occurrence.HasSucceeded = True GROUP BY ParticipantGroup.Description, Occurrence.SimulationID, Occurrence.SimRunID, Occurrence.Period;
The SQLite query is as follows (this query takes around 10 seconds):
SELECT ij1.Description, ij2.occSimulationID, ij2.occSimRunID, ij2.Period, Count(ij2.occpParticipantID) AS CountOfParticipantID FROM
(
SELECT ip.ParticipantGroupID AS ipParticipantGroupID, ip.ParticipantID AS ipParticipantID, ip.ParticipantTypeID, pg.ParticipantGroupID AS pgParticipantGroupID, pg.ParticipantGroupTypeID, pg.Description FROM ParticipantGroup as pg INNER JOIN InitialParticipant AS ip ON pg.ParticipantGroupID = ip.ParticipantGroupID
) AS ij1 INNER JOIN
(
SELECT tpa.participantID AS tpaParticipantID, ij3.* FROM tmpPartArgs AS tpa INNER JOIN
(
SELECT ij4.*, occp.SimulationID as occpSimulationID, occp.SimRunID AS occpSimRunID, occp.OccurrenceID AS occpOccurrenceID, occp.ParticipantID AS occpParticipantID, occp.RoleTypeID FROM
(
SELECT tsa.SimulationID AS tsaSimulationID, tsa.SimRunID AS tsaSimRunID, occ.SimulationID AS occSimulationID, occ.SimRunID AS occSimRunID, occ.OccurrenceID AS occOccurrenceID, occ.OccurrenceTypeID, occ.Period, occ.HasSucceeded FROM tmpSimArgs AS tsa INNER JOIN Occurrence AS occ ON (tsa.SimRunID = occ.SimRunID) AND (tsa.SimulationID = occ.SimulationID)
) AS ij4 INNER JOIN OccurrenceParticipant AS occp ON (occOccurrenceID = occpOccurrenceID) AND (occSimRunID = occpSimRunID) AND (occSimulationID = occpSimulationID)
) AS ij3 ON tpa.participantID = ij3.occpParticipantID
) AS ij2 ON ij1.ipParticipantID = ij2.occpParticipantID WHERE (((ij2.RoleTypeID)=52 Or (ij2.RoleTypeID)=49)) AND ij2.HasSucceeded = 1 GROUP BY ij1.Description, ij2.occSimulationID, ij2.occSimRunID, ij2.Period;
I don’t know what I am doing wrong here. I have all the indexes but I thinking I am missing declaring some key index that will do the trick for me. The interesting thing is before migration my ‘research’ on SQLite showed that SQLite is faster, smaller and better in all aspects than Access. But I cant seem to get SQLite work faster than Access in terms of querying. I reiterate that I am new to SQLite and obviously do not have much idea as well as experience so if any learned soul could help me out with this, it will be much appreciated.
I have reformatting your code (using my home-brew sql formatter) to hopefully make it easier for others to read..
Reformatted Query:
SELECT
ij1.Description,
ij2.occSimulationID,
ij2.occSimRunID,
ij2.Period,
Count(ij2.occpParticipantID) AS CountOfParticipantID
FROM (
SELECT
ip.ParticipantGroupID AS ipParticipantGroupID,
ip.ParticipantID AS ipParticipantID,
ip.ParticipantTypeID,
pg.ParticipantGroupID AS pgParticipantGroupID,
pg.ParticipantGroupTypeID,
pg.Description
FROM ParticipantGroup AS pg
INNER JOIN InitialParticipant AS ip
ON pg.ParticipantGroupID = ip.ParticipantGroupID
) AS ij1
INNER JOIN (
SELECT
tpa.participantID AS tpaParticipantID,
ij3.*
FROM tmpPartArgs AS tpa
INNER JOIN (
SELECT
ij4.*,
occp.SimulationID AS occpSimulationID,
occp.SimRunID AS occpSimRunID,
occp.OccurrenceID AS occpOccurrenceID,
occp.ParticipantID AS occpParticipantID,
occp.RoleTypeID
FROM (
SELECT
tsa.SimulationID AS tsaSimulationID,
tsa.SimRunID AS tsaSimRunID,
occ.SimulationID AS occSimulationID,
occ.SimRunID AS occSimRunID,
occ.OccurrenceID AS occOccurrenceID,
occ.OccurrenceTypeID,
occ.Period,
occ.HasSucceeded
FROM tmpSimArgs AS tsa
INNER JOIN Occurrence AS occ
ON (tsa.SimRunID = occ.SimRunID)
AND (tsa.SimulationID = occ.SimulationID)
) AS ij4
INNER JOIN OccurrenceParticipant AS occp
ON (occOccurrenceID = occpOccurrenceID)
AND (occSimRunID = occpSimRunID)
AND (occSimulationID = occpSimulationID)
) AS ij3
ON tpa.participantID = ij3.occpParticipantID
) AS ij2
ON ij1.ipParticipantID = ij2.occpParticipantID
WHERE (
(
(ij2.RoleTypeID) = 52
OR
(ij2.RoleTypeID) = 49
)
)
AND ij2.HasSucceeded = 1
GROUP BY
ij1.Description,
ij2.occSimulationID,
ij2.occSimRunID,
ij2.Period;
As per JohnFx (above), I was confused by the derived views. I think there is actually no need for it, especially since they are all inner joins. So, below I have attempted to reduce the complexity. Please review and test for performance. I have had to do a cross join with tmpSimArgs since it is only joined to Occurence - I assume this is desired behaviour.
SELECT
pg.Description,
occ.SimulationID,
occ.SimRunID,
occ.Period,
COUNT(occp.ParticipantID) AS CountOfParticipantID
FROM ParticipantGroup AS pg
INNER JOIN InitialParticipant AS ip
ON pg.ParticipantGroupID = ip.ParticipantGroupID
CROSS JOIN tmpSimArgs AS tsa
INNER JOIN Occurrence AS occ
ON tsa.SimRunID = occ.SimRunID
AND tsa.SimulationID = occ.SimulationID
INNER JOIN OccurrenceParticipant AS occp
ON occ.OccurrenceID = occp.OccurrenceID
AND occ.SimRunID = occp.SimRunID
AND occ.SimulationID = occp.SimulationID
INNER JOIN tmpPartArgs AS tpa
ON tpa.participantID = occp.ParticipantID
WHERE occ.HasSucceeded = 1
AND (occp.RoleTypeID = 52 OR occp.RoleTypeID = 49 )
GROUP BY
pg.Description,
occ.SimulationID,
occ.SimRunID,
occ.Period;
I have presented a smaller scaled down version of my query. Hope this is more clear and legible than my earlier one.
SELECT5 * FROM
(
SELECT4 FROM ParticipantGroup as pg INNER JOIN InitialParticipant AS ip ON pg.ParticipantGroupID = ip.ParticipantGroupID
) AS ij1 INNER JOIN
(
SELECT3 * FROM tmpPartArgs AS tpa INNER JOIN
(
SELECT2 * FROM
(
SELECT1 * FROM tmpSimArgs AS tsa INNER JOIN Occurrence AS occ ON (tsa.SimRunID = occ.SimRunID) AND (tsa.SimulationID = occ.SimulationID)
) AS ij4 INNER JOIN OccurrenceParticipant AS occp ON (occOccurrenceID = occpOccurrenceID) AND (occSimRunID = occpSimRunID) AND (occSimulationID = occpSimulationID)
) AS ij3 ON tpa.participantID = ij3.occpParticipantID
) AS ij2 ON ij1.ipParticipantID = ij2.occpParticipantID WHERE (((ij2.RoleTypeID)=52 Or (ij2.RoleTypeID)=49)) AND ij2.HasSucceeded = 1
The application that I am working on is a Simulation application and in order to understand the context of the above query I thought it necessary to give a brief explanation of the application. Let us assume there is a planet with some initial resources and living agents. The planet is allowed to exist for 1000 years and the actions performed by the agents are monitored and stored in the database. After 1000 years the planet is destroyed and again re-created with the same set of initial resources and living agents as the first time. This (the creation and destruction) is repeated 18 times and all the actions of the agents performed during those 1000 years are stored in the database. Thus our entire experiment consists of 18 re-creations which is termed as the ‘Simulation’. Each of the 18 times the planet is recreated is termed as a run and each of the 1000 years of a run is called a period. So a ‘Simulation’ consists of 18 runs and each run consists of 1000 periods. At the start of each run, we assign the ‘Simulation’ an initial set of knowledge items and dynamic agents that interact with each other and the items. A knowledge item is stored by an agent inside a knowledge store. The knowledge store is also considered to be a participating entity in our Simulation. But this concept (regarding knowledge stores) is not important. I have tried to be detailed about every SELECT statement and the tables involved.
SELECT1: I think this query could be replaced by just the table ‘Occurrence’, since it does nothing much. The table Occurrence stores the different actions taken by the agents, in each period of every simulation run of a particular ‘Simulation’. Normally each ‘Simulation’ consists of 18 runs. And each run consists of a 1000 periods. An agent is allowed to take an action in every period of every run in the ‘Simulation’. But the Occurrence table does not store any details about the agents that perform the actions. The Occurrence table might store data related to multiple ‘Simulations’.
SELECT2: This query simply returns the details of actions performed in every period of every run of a ‘Simulation’ along with the details of all participants of the ‘Simulation’ like their respective ParticipantIDs. The OccurrenceParticipant table stores records for every participating entity of the Simulation and that includes agents, knowledge stores, knowledge items, etc.
SELECT3: This query returns only those records from the pseudo table ij3 that are due to agents and knowledge items. All records in ij3 concerning knowledge items will be filtered out.
SELECT4: This query attaches the ‘Description’ field to every record of ‘InitialParticipant’. Please note that the column ‘Description’ is an Output column of the entire query. The table InitialParticipant contains a record for every agent and every knowledge item that is initially assigned to the ‘Simulation’
SELECT5: This final query returns all records from the pseudo table ij2 for which the RoleType of the participating entity (which may either be an agent or a knowledge item) is 49 or 52.
I would suggest moving the ij2.RoleTypeID filtering from the outermost query to ij3, use IN instead of OR, and move the HasSucceeded query to ij4.
I am trying to do an SQL query on two tables to retrieve multiple columns with a certain column (PostID) to be distinct (and it is not the primary key of the that table).
In addition, I need the selected distinct rows to be the latest (one of the columns retrieved is the entry date).
Detailed description:
I am building a forum like application, using 3 tables to store data.
I use table1 to store user details, table2 to store the meta data for posts, table3 to store the post details, updates, and replies (postID is unique in table2 pointing towards an original post, while in table3, it is used to show the original post and updates and replies).
Table columns:
table1 (UserID, FullName, mobile, etc.)
table2 (postID, UserID, EntryDate, Deleted columns)
table3 (postdetailsId, PostID, UserID, Entrydate, etc.)
I am trying to retrieve all the posts for 1 user in a gridview, my SQL query uses the USERID to retrieve all his posts from the table. However, it is retrieving the original post and all its updates, and I only want to retrieve the latest update of each post.
How can it be done fully in SQL (I know I can do it in C# with the returned results)?
My query:
SELECT T1.FullName, T3.PostID, T3.EntryDate, T3.Title
FROM Table1 as T1, Table3 as T3
WHERE T3.UserID = T1.UserID
AND T3.UserID = #UserID
You could use GROUP BY PostID along with MAX(EntryDate)
SELECT *
FROM (
SELECT posts.*, ROW_NUMBER() OVER (PARTITION BY post_updates.UserID, post_updates.PostID ORDER BY post_updates.EntryDate DESC) AS rn
FROM table1 users
JOIN table3 post_updates
ON post_updates.userID = users.UserID
WHERE users.UserID = #UserID
) q
WHERE rn = 1