I have an SQLite database containing event log records with a timestamp and the event details. Some of these events are alarms generated by the system. What I need to do can be best explained in the following pseudo code:
for each event where type="alarm" get preceeding events between event.timestamp - 1 hour and event.timestamp
What I do now is query the database once to find the timestamp of each alarm and then loop over the result to get all the time periods for each alarm. This creates a list of transactions that I want to use for association rule mining but I feel that leaving SQL to then come back to it in a loop is inefficient. I've searched for answers to this question but I think I can't find the right keywords to search for.
I've edited the question as it wasn't as clear as I myself thought
Perhaps a self-join?
SELECT {column list}
from events e1, events e2
where e1.type="alarm"
and e2.time between (date(e1.time),'-1 hour') and e1.time
Related
I just started figuring out DynamoDB.
I have a simple table has date attribute(ex. 20160101) as HASH and created_at attribute(ex. 20160101185332) as RANGE.
I'd like to get latest N items from the table.
First, SCAN command does not have ScanIndexForward option. I think it's not possible with SCAN.
Next, QUERY command. It seems to be work if I repeat QUERY command several times to get enough number of items(cuz, I don't know how many items have same key value). - for example, I can query using today first and repeat for the day before if the result does not give enough items.
How can I do the job more efficiently? Or, can I query without KEY value?
as you described your table, you cant do it more efficiently, and you cant query dynamodb without KEY(hash) value
look at the answer here:
dynamodb get earliest inserted distinct values from a table
There are several date/time fields in this view and I can never wrap my head around which column to order by (and which secondary column) in order to retrieve a list of SQL statements in the order in which they were executed on the server.
StartTime - The timestamp associated with when the query was submitted to Teradata for parsing.
FirstStepTime - The timestamp associated with when the first step of the query was executed.
FirstRespTime - The timestamp associated with when the first row was returned to the client.
The gap in time between the StartTime and FirstStep time include parsing time and any workload throttle delay that was enforced by Teradata's Dynamic Workload Manager. For the sake for keeping things simple here I will defer to an excellent article written by Teradata's Carrie Ballinger on dealing with delay time here.
I've got a table like so:
CREATE TABLE IF NOT EXISTS grades(_id, timestamp, extra);
I want to create an index on "timestamp", so I'm doing:
CREATE INDEX idx_timestamp ON grades(timestamp);
I want to select 20 records at a time based off the timestamp then:
SELECT * FROM grades WHERE timestamp > 123 ORDER BY timestamp ASC LIMIT 20;
So, is there a more efficient way I can define the column "timestamp"? I'm just guessing that specifying it as an indexed column is all we can do, and specifying "ASC" for sort order is a no-op - or can I tell sqlite to store records presorted by timestamp in the first place?
I'm basically trying to implement a paging system, selecting a chronologically ordered page of 20 items at a time.
Thanks
This is fine. It will use the index to order, so it will increase the speed. although, depending what you are doing you will may want to cache some records.
SELECT * FROM grades WHERE timestamp > 123 ORDER BY timestamp ASC LIMIT 200;
We can now get 200 at a time, but on we now will let javascript or the server handle the paging. Basically you would keep track of where the paging is and then only when needed hit the database again. Also, by using the more limited WHERE clause for the timestamp, it's actually quite fast and efficient. Better than using the LIMIT N,M.
If you do the second option, you will be able to cache that query as well, depending how often it gets hit. So, if multiple people keeping querying that some thing, the database will cache it and it will come back really fast since it's already there.
I've got a query where I'm trying to get the hours in duration (eg 6.5 hours) between two different times.
In my database, time and date are held in different fields so I can efficiently query on just a startDate, or endDate as I never query specifically on time.
My query looks like this
SELECT COUNT(*), IFNULL(SUM(TIMEDIFF(endTime,startTime)),0) FROM events WHERE user=18
Sometimes an event will go overnight, so the difference between times needs to take into account the differences between the dates as well.
I've been trying
SELECT COUNT(*), IFNULL(SUM(TIMEDIFF(CONCAT(endDate,' ',endTime),CONCAT(startDate,' ',startTime))),0) FROM events WHERE user=18
Unfortunately I only get errors when I do this, and I can't seem to combine the two fields into a single timestamp.
Pretty sure your problem is that your concatenated values are being sent to TIMEDIFF() as strings rather than DATETIMEs. Try calling the DATETIME function on them:
SELECT COUNT(*), IFNULL(SUM(TIMEDIFF(DATETIME(CONCAT(endDate,' ',endTime)),DATETIME(CONCAT(startDate,' ',startTime)))),0) FROM events WHERE user=18
I don't have a MySQL DB in front of my to test that, but I think that or some similar form of it is what you are looking for. There's an example of it in the MySQL docs involving MICROSECOND:
http://dev.mysql.com/doc/refman/5.0/en/datetime.html
Edit: Hmm... looks like TIMEDIFF is supposed to work with strings. Worth trying anyway.
TIMEDIFF(endDate,startDate) + TIMEDIFF(endTime,startTime)
I am trying to correct the sort order of my ASP.NET drop down list.
The problem I have is that I need to select a distinct Serial number and have these numbers organised by DateTime Desc.
However I cannot ORDER BY DateTime if using DISTINCT without selecting the DateTime field in my query.
However if I select DateTime this selects every data value associated with a single Serial number and results in duplications.
The purpose of my page is to display data for ALL Serials, or data associated to one serial. When a new cycle begins (because it is a new production run) the Serial reverts to 1. So I cannot simply organise by serial number either.
When I use the following SQL statement the list box is in the order I require but after a period of time (usually a few hours) the order changes and appears to have no organised structure.
alt text http://img7.imageshack.us/i/captureky.jpg/
I'm fairly new to ASP.NET / SQL, does anyone know of a solution to my problem.
If you have multiple date times for each serial number, then which do you want to use for ordering? If the most recent, try this:
SELECT SerialNumber,
MAX(DateTimeField)
FROM Table
GROUP BY SerialNumber
ORDER BY 2 DESC
I don´t know if everybody agrees with that, but when I see a DISTINCT in a query the first thought that goes trough my mind is "This is wrong". Generally, DISTINCT is not necessary and it´s used when the person writing the query doesnt know very well what he is doing and this might be the case since you said you are new with Sql.
Without complete knowledge of your model is difficult to assist you a hundred percente, but I would say that you should use a GROUP BY clause instead of DISTINCT, then you can order it correctly.