I have a table where id is primary key and squence.
In transaction I call sequence
select NEXTVAL(next_cons_id) as next_cons_id
then create a new record where field id = next_cons_id got from code bellow.
DDL sequence
CREATE SEQUENCE IF NOT EXISTS next_cons_id START WITH 1;
In most cases it work well.
Having done a small study I found some patterns.
Id, ... gap ..., nextId
606401,605557,
604556,605401,
603555,604401,
602594,603401,
601594,602401
Ids with dates:
601594 2019-10-10 00:04:54
602401 2019-10-10 01:36:33
602594 2019-10-10 21:46:43
603401 2019-10-11 01:19:49
603555 2019-10-11 21:41:26
604401 2019-10-12 00:30:09
604556 2019-10-13 00:15:32
605401 2019-10-13 03:51:42
605557 2019-10-13 22:42:22
606401 2019-10-14 02:14:28
And after I did create console script that make insert by one record per second in test table:
237343 2019-10-14 00:31:03 2019-10-14 00:31:03
237344 2019-10-14 00:31:03 2019-10-14 00:31:03
....gap...
238000 2019-10-14 00:31:04 2019-10-14 00:31:04
It table use only test script.
But it time we can see some problem:
There were no network problems at this point
In monitoring of zabbix rollback of transactions not found.
Database has one replication server.
Server version: 10.3.13
Do you have any ideas to fix this?
UPD:
I remove the sequence and recreate this table with
id primary key autouncrement but this case has been repeated.
Related
Example Query -
traces
| project message, timestamp
This outputs something like this -
message | timestamp
------------------------------
A 2022-07-09 00:00:00
B 2022-07-11 01:00:00
A 2022-07-11 02:00:00
I want a query that gives this as output -
message | timestamp
------------------------------
A 2022-07-09 00:00:00
B 2022-07-11 01:00:00
you could use the min() aggregation function if you only have 2 fields of interest (message, timestamp), or the arg_min() aggregation function otherwise.
e.g.
traces | summarize min(timestamp) by message
traces | summarize arg_min(timestamp, *) by message
I am using Cassandra and I have a column of data that represents epoch time milliseconds stored as a bigint type:
request_time bigint
I would like to convert this to a date time object in order to work with certain days. How can I achieve that conversion?
I have tried:
select todate(request_time)
# InvalidRequest: Error from server: code=2200 [Invalid query] message="Input type Int64 is not supported in ToDate."
and also:
select dateof(mintimeuuid(request_time))
# InvalidRequest: Error from server: code=2200 [Invalid query] message="Unable to coerce Int64 into type CassandraTimestamp"
My Cassandra version is:
[cqlsh 5.0.1 | Cassandra 3.11.0 | CQL spec 3.4.4 | Native protocol v4]
Use function -
todate()
Example below -
select request_time from testkspc.test;
request_time
---------------
123344345
123344687
123456329
223344687
1613808000000
323344687
123456789
123344329
423344687
523344687
select todate(request_time) from testkspc.test ;
system.todate(request_time)
-----------------------------
1970-01-02
1970-01-02
1970-01-02
1970-01-03
2021-02-20
1970-01-04
1970-01-02
1970-01-02
1970-01-05
1970-01-07
Further you can use totimestamp function.
Directly using this function on bigint type gives error -
select totimestamp(request_time) from testkspc.test ;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid call to function totimestamp, none of its type signatures match (known type signatures: system.totimestamp : (date) -> timestamp, system.totimestamp : (timeuuid) -> timestamp)"
We can wrap todate inside totimestamp.
select totimestamp(todate(request_time)) from testkspc.test ;
system.totimestamp(system.todate(request_time))
-------------------------------------------------
1970-01-02 00:00:00.000000+0000
1970-01-02 00:00:00.000000+0000
1970-01-02 00:00:00.000000+0000
1970-01-03 00:00:00.000000+0000
2021-02-20 00:00:00.000000+0000
1970-01-04 00:00:00.000000+0000
1970-01-02 00:00:00.000000+0000
1970-01-02 00:00:00.000000+0000
1970-01-05 00:00:00.000000+0000
1970-01-07 00:00:00.000000+0000
You had the right idea by using the built-in CQL function toDate().
I've deployed a Cassandra 3.11.0 cluster (the same as the version you're running) and can confirm that it works. Here's an example output:
cqlsh:stackoverflow> SELECT request_time, todate(request_time) FROM testbigint ;
request_time | system.todate(request_time)
---------------+-----------------------------
1654671885000 | 2022-06-08
1654 | 1970-01-01
In your case, I believe the underlying issue is that your table contains invalid data in the request_time column. You need to check the data in your table and try again.
As a side note, Apache Cassandra 3.11.0 was released 5 years ago and you shouldn't even bother using it for testing. At the time of writing, the current supported version is C* 3.11.13. Cheers!
I am examining prescription patterns within a large EHR dataset. The data is structured so that we are given several key bits of information, such as patient_num, encounter_num, ordering_date, medication, age_event (age at event) etc. Example below:
Patient_num enc_num ordering_date medication age_event
1111 888888 07NOV2008 Wellbutrin 48
1111 876578 11MAY2011 Bupropion 50
2222 999999 08DEC2009 Amitriptyline 32
2222 999999 08DEC2009 Escitalopram 32
3333 656463 12APR2007 Imipramine 44
3333 643211 21DEC2008 Zoloft 45
3333 543213 02FEB2009 Fluoxetine 45
Currently I have the dataset sorted by patient_id then by ordering_date so that I can see what each individual was prescribed during their encounters in a longitudinal fashion. For now, I am most concerned with the prescription(s) that were made during their first visit. I wrote some code to count the number of prescriptions and had originally restricted later analyses to RX = 1, but as we can see, that doesn't work for people with multiple scripts on the same encounter (Patient 2222).
data pt_meds_;
set pt_meds;
by patient_num;
if first.patient_num then RX = 1;
else RX + 1;
run;
Patient_num enc_num ordering_date medication age_event RX
1111 888888 07NOV2008 Wellbutrin 48 1
1111 876578 11MAY2011 Bupropion 50 2
2222 999999 08DEC2009 Amitriptyline 32 1
2222 999999 08DEC2009 Escitalopram 32 2
3333 656463 12APR2007 Imipramine 44 1
3333 643211 21DEC2008 Zoloft 45 2
3333 543213 02FEB2009 Fluoxetine 45 3
I think it would be more appropriate to recode the encounter numbers into a new variable so that they reflect a style similar to the RX variable. Where each encounter is listed 1-n, and the number will repeat if multiple scripts are made in the same encounter. Such as below:
Patient_num enc_num ordering_date medication age_event RX Enc_
1111 888888 07NOV2008 Wellbutrin 48 1 1
1111 876578 11MAY2011 Bupropion 50 2 2
2222 999999 08DEC2009 Amitriptyline 32 1 1
2222 999999 08DEC2009 Escitalopram 32 2 1
3333 656463 12APR2007 Imipramine 44 1 1
3333 643211 21DEC2008 Zoloft 45 2 2
3333 543213 02FEB2009 Fluoxetine 45 3 3
From what I have seen, this could be possible with a variant of the above code using 2 BY groups (patient_num & enc_num), but I can't seem to get it. I think the first. / last. codes require sorting, but if I am to sort by enc_num, they won't be in chronological order because the encounter numbers are generated by the system and depend on all other encounters going in at that time.
I tried to do the following code (using ordering_date instead because its already sorted properly) but everything under Enc_ is printed as a 1. I'm sure my logic is all wrong. Any thoughts?
data pt_meds_test;
set pt_meds_;
by patient_num ordering_date;
if first.patient_num;
if first.ordering_date then enc_ = 1;
else enc_ + 1;
run;
First
.First/.Last flags doesn't require sorting if data is properly ordered or you use NOTSORTED in your BY statement. If your variable in BY statement is not properly ordered then BY statment will throw error and stop executing when encounter deviations. Like this:
data class;
set sashelp.class;
by age;
first = first.age;
last = last.age;
run;
ERROR: BY variables are not properly sorted on data set SASHELP.CLASS.
Name=Alfred Sex=M Age=14 Height=69 Weight=112.5 FIRST.Age=1 LAST.Age=1 first=. last=. _ERROR_=1 _N_=1
NOTE: The SAS System stopped processing this step because of errors.
NOTE: There were 2 observations read from the data set SASHELP.CLASS.
Try this code to see how exacly .first/.last flags works:
data pt_meds_test;
set pt_meds_;
by patient_num ordering_date;
fp = first.patient_num;
lp = last.patient_num;
fo = first.ordering_date;
lo = last.ordering_date;
run;
Second
Those condidions works differently than you think:
if expression;
If expression is true then continue with next instructions after if.
Otherwise return to begining of data step (no implicit output). This also implies your observation is not retained in the output.
In most cases if without then is equivalent to where. However
whereworks faster but it is limited to variables that comes from data set you are reading
if can be used with any type of expression including calculated fields
More info:: IF
Statement, Subsetting
Third
I think lag() function can be your answear.
data pt_meds_test;
set pt_meds_;
by patient_num;
retain enc_;
prev_patient_num = lag(patient_num);
prev_ordering_date = lag(ordering_date);
if first.patient_num then enc_ = 1;
else if patient_num = prev_patient_num and ordering_date ne prev_ordering_date then enc_ + 1;
end;
run;
With lag() function you can look what was the value of vairalbe on the previos observation and compare it with current one later.
But be carefull. lag() doesn't look for variable value from previous observation. It takes vale of variable and stores it in a FIFO queue with size of 1. On next call it retrives stored value from queue and put new value there.
More info: LAG Function
I'm not sure if this hurts the rest of your analysis, but what about just
proc freq data=pt_meds noprint;
tables patient_num ordering_date / out=pt_meds_freq;
run;
data pt_meds_freq2;
set pt_meds_freq;
by patient_num ordering_date;
if first.patient_num;
run;
I need to get the data in a column of a table Cassandra Database. I am using RCassandra for this. After getting the data I need to do some text mining on it. Please suggest me how do connect to cassandra, and get the data into my R Script using RCassandra
My RScript :
library(RCassandra)
connect.handle <- RC.connect(host="127.0.0.1", port=9160)
RC.cluster.name(connect.handle)
RC.use(connect.handle, 'mykeyspace')
sourcetable <- RC.read.table(connect.handle, "sourcetable")
print(ncol(sourcetable))
print(nrow(sourcetable))
print(sourcetable)
This will print the output as:
> print(ncol(sourcetable))
[1] 1
> print(nrow(sourcetable))
[1] 18
> print(sourcetable)
144 BBC News
158 IBN Live
123 Reuters
131 IBN Live
But my cassandra table contains four columns, but here its showing only 1 column. I need to get each column values separated. So how do I get the individual column values(Eg.each feedurl) What changes should I make in my R script?
My cassandra table, named sourcetable
I have used Cassandra and R with the correct Cran Jar files, but RCassandra is easier. RCassandra is a direct interface to Cassandra without the use of Java. To connect to Cassandra you will use RC.connect to return a connection handle like this.
RC.connect(host = <xxx>, port = <xxx>)
RC.login(conn, username = "bar", password = "foo")
You can then use a RC.get command to retrieve data or RC.ReadTable command to read table data.
BUT, First you should read THIS
I am confused as well. Table demo.emp has 4 row and 4 columns ( empid, deptid, first_name and last_name). Neither RC.get nor RC.read.table gets the all the data.
cqlsh:demo> select * from emp;
empid | deptid | first_name | last_name
-------+--------+------------+-----------
1 | 1 | John | Doe
1 | 2 | Mia | Lewis
2 | 1 | Jean | Doe
2 | 2 | Manny | Lewis
> RC.get.range.slices(c, "emp", limit=10)
[[1]]
key value ts
1 1.474796e+15
2 John 1.474796e+15
3 Doe 1.474796e+15
4 1.474796e+15
5 Mia 1.474796e+15
[[2]]
key value ts
1 1.474796e+15
2 Jean 1.474796e+15
3 Doe 1.474796e+15
4 1.474796e+15
5 Manny 1.474796e+15
I am running neo-4j 1.8.2 on a remote unix box. I am using this jar (https://github.com/jexp/batch-import/downloads).
nodes.csv is same as given in example:
name age works_on
Michael 37 neo4j
Selina 14
Rana 6
Selma 4
rels.csv is like this:
start end type since counter:int
1 2 FATHER_OF 1998-07-10 1
1 3 FATHER_OF 2007-09-15 2
1 4 FATHER_OF 2008-05-03 3
3 4 SISTER_OF 2008-05-03 5
2 3 SISTER_OF 2007-09-15 7
But i am getting this exception :
Using Existing Configuration File
Total import time: 0 seconds
Exception in thread "main" java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
at org.neo4j.batchimport.Importer$Data.split(Importer.java:156)
at org.neo4j.batchimport.Importer$Data.update(Importer.java:167)
at org.neo4j.batchimport.Importer.importNodes(Importer.java:226)
at org.neo4j.batchimport.Importer.main(Importer.java:83)
I am new to neo4j, was checking if this importer can save some coding effort.
It would be great if someone can point to the probable mistake.
Thanks for help!
--Edit:--
My nodes.csv
name dob city state s_id balance desc mgr_primary mgr_secondary mgr_tertiary mgr_name mgr_status
John Von 8/11/1928 Denver CO 1114-010 7.5 RA 0023-0990 0100-0110 Doozman Keith Active
my rels.csv
start end type since status f_type f_num
2 1 address_of
1 3 has_account 5 Active
4 3 f_of Primary 0111-0230
Hi I had some issues in the past with the batch import script.
The formating of your file must be very rigorous, which means :
no extra spaces where not expected, like the ones I see in the first line of your rels.csv before "start"
no multiple spaces in place of the tab. If your files are exactly like what you've copied here, you have 4 spaces instead of on tab, and this is not going to work, as the script uses a tokenizer looking for tabs !!!
I had this issue because I always convert tabs to 4 spaces, and once I understood that, I stopped doing it for my csv !