I have an Oracle GoldenGate setup with Oracle database as my source and a non-Oracle target. My source database is in Timezone A. Is there a way to configure my Extract/Replicat so that all data in my target is in UTC or some customizable Timezone B?
To make it clearer, suppose i have a database in EST where I have a date column with value 2018-11-26 17:01:49 I want the data to be converted to UTC and the replicated value would be 2018-11-26 22:01:49. Is there a clean(or any) way to achieve this?
This conversion only works on a REPLICAT. It does a full calculation of times and goes from a source date column named dt to a target column named dt5 that is to be converted to the date + 5 hours (you can adjust it to whatever time zone you must convert the data). It assumes your source table is t4 and target table is t4_copy (for SCHEMA scratch)
MAP scratch.t4, TARGET scratch.t4_copy,
COLMAP ( USEDEFAULTS,
dt5 = #DATE ('YYYY-MM-DD HH:MI:SS', 'JTS',
#COMPUTE (#DATE ('JTS', 'YYYY-MM-DD HH:MI:SS', dt) + 18000000000 ) )
);
JTS (Juilan timestamp) produce numbers you can use in numeric expressions. The unit is microseconds (that's why the +18000000000)
Related
I tried to update a lot of rows in date format ('DD-MON-YY'). S
So, in cases such 22-FEB-99, I got 22th February 1999, instead of 22th February 2099.
Is there a way to specify the year in such cases?
I tried to update a lot of rows in date format ('DD-MON-YY').
If you have a DATE column then that is a binary data type consisting of 7 bytes (representing century, year-of-century, month, day, hour, minute and second); it ALWAYS has those components and it is NEVER stored in any (human-readable) format.
What you probably mean is that your client application (SQL Developer) is displaying the dates in the format DD-MON-RR and it is not showing you the century or time components.
Is there a way to specify the year in such cases?
Rather than letting the client application apply a default format for displaying the date, you can apply an explicit format using TO_CHAR:
SELECT TO_CHAR(your_date_column, 'FMDDth Month YYYY', 'NLS_DATE_LANGUAGE=English')
AS formatted_date
FROM your_table;
Which, for the sample data:
CREATE TABLE your_table (your_date_column) AS
SELECT SYSDATE FROM DUAL UNION ALL
SELECT DATE '2022-01-01' FROM DUAL UNION ALL
SELECT DATE '1970-01-01' FROM DUAL;
Outputs:
FORMATTED_DATE
2nd November 2022
1st January 2022
1st January 1970
in cases such 22-FEB-99, I got 22th February 1999, instead of 22th February 2099.
If you get the value 22th February 1999 then that is because the date is stored as 1999 and not 2099. Note: that a DATE data type always has a century component so the query will display what is stored in the column.
You have probably inserted (or updated) the date as a string:
INSERT INTO your_table (your_date_column)
VALUES('22-FEB-99');
'22-02-99' is not a date data type, it is a string literal. Oracle has tried to be helpful and implicitly convert the date to a string using the NLS_DATE_FORMAT session parameter effectively converting the query to:
INSERT INTO your_table (your_date_column)
VALUES(
TO_DATE(
'22-FEB-99',
(SELECT value FROM NLS_SESSION_PARAMETERS WHERE parameter = 'NLS_DATE_FORMAT')
)
);
However, your NLS_DATE_FORMAT session parameter is set to DD-MON-RR and the RR format model will convert 99 to 1999 and not 2099.
What you need to do is NEVER use implicit conversion from strings to dates; instead:
Use a date literal:
INSERT INTO your_table(your_date_column) VALUES (DATE '2099-02-22');
Use a timestamp literal (which also allows you to specify the time):
INSERT INTO your_table(your_date_column) VALUES (TIMESTAMP '2099-02-22 00:00:00');
Explicitly convert the string using TO_CHAR and the correct format model:
INSERT INTO your_table(your_date_column)
VALUES (TO_DATE('22-FEB-99', 'DD-MON-YY', 'NLS_DATE_LANGUAGE=English'));
fiddle
Fixing invalid data
If you have the data as 1999 and want 2099 then you will need to fix it:
UPDATE your_table
SET your_date_column = your_date_column + INTERVAL '100' YEAR(3)
WHERE your_date_column = DATE '1999-02-22';
or:
UPDATE your_table
SET your_date_column = ADD_MONTHS(your_date_column, 1200)
WHERE your_date_column = DATE '1999-02-22';
or:
UPDATE your_table
SET your_date_column = DATE '2099-02-22'
WHERE your_date_column = DATE '1999-02-22';
2-digits year is evil (does Y2K ring a bell?). You should really use 4 digits for years. Meanwhile, see whether YY vs. RR format models do any difference in your case.
(you don't have to alter the session; my database uses different date format and language so I'm setting it now)
SQL> alter session set nls_date_format = 'dd.mm.yyyy';
Session altered.
SQL> alter session set nls_date_language = 'english';
Session altered.
Sample data and how yy/rr affect the result:
SQL> with test (col) as
2 (select '22-FEB-99' from dual)
3 select to_date(col, 'dd-mon-yy') val1,
4 to_date(col, 'dd-mon-rr') val2
5 from test;
VAL1 VAL2
---------- ----------
22.02.2099 22.02.1999
SQL>
I am working locally with an sqllite DB. I have imported some records from teradata where there was a date field in the format of 'YYYY-MM-DD'. When i imported the records the date switched from a date to a number. I know this is a feature of sqllite and that one can access it via date(sqllite_date) when selecting it in a where clause.
My problem is that the dates now appear to be a bit odd. For example the year appears to be negative.
Is there anyway to recover this to the correct format?
Below is an example of converting a number in the database into a date
SELECT date(18386)
# -4662-03-28
SELECT datetime('now')
# 2021-02-11 10:41:52
SELECT date(sqllite_date) FROM mydb
# Returns -4662-03-28
# Should return 2020-05-04
I am very new to this area so apologies if this is a basic question. Thank you very much for your time
In SQLite you can store dates as TEXT, REAL or INTEGER.
It seems that you stored the dates in a column with INTEGER or REAL affinity.
In this case, if you use the function date(), it considers a value like 18386 as a Julian day, meaning the number of days since noon in Greenwich on November 24, 4714 B.C.
This is why date(18386) returns 4662-03-28B.C.
But I suspect that the date values that you have are the number of days since '1970-01-01'.
In this case, 18386 days after '1970-01-01' is '2020-05-04'.
So you can get the dates in the format YYYY-MM-DD if you add the value of your column as days to '1970-01-01':
SELECT date('1970-01-01', datecolumn || ' day') FROM tablename
Or by transforming your date values to seconds and treat them as UNIX time (the number of seconds since '1970-01-01 00:00:00 UTC'):
SELECT date(datecolumn * 24 * 3600, 'unixepoch') FROM tablename
Replace datecolumn with the name of your column.
I am trying to count the number of times each weekday occurs in a dataframe. My dataframe looks like this when printed on screen (assume the columns are labeled "x")
I am currently using
sqldf("SELECT x, strftime('%w', x) FROM maxkwpeakdates")
also, if I run the code
sqldf("SELECT strftime('%w', date('now'))")
I get the right weekday.
Knowing that strftime is from sqlite and that their datetime functions are 0 based, those numbers are off. Looking at the 1st line the result should be a 3 because April 6, 2011 occurred on a Wednesday. The 10th line should also result in a 3 because January 4th 2012 also occurred on a Wednesday.
My data is actually in a dataframe stored within a shiny app which is why I am not using something like RMySQL and am instead using sqldf. I want to be able to run SQL queries on my dataframe so that I can keep the data inside the app and so that I can calculate things like average number of specific weekdays in a time period. Here is an example of my dataframe which is called maxkwpeakdates:
I did another test with my data and compared the results of using
sqldf("SELECT x, strftime('%w', x) FROM maxkwpeakdates")
and
weekdays(as.Date(maxkwpeakdates$x))
Then I stored these results in a dataframe so I could compare the results.
Notice that using as.Date reported the correct weekdays but using strftime was almost always off. Is this a problem with R talking to sqlite through strftime? Is there a way to fix this so I get the correct result with strftime?
1) SQLite SQLite does not have any date/time type so all R does is send the internal representation, seconds since the Epoch, to SQLite and then it is up to the SQL statement to interpret it somehow. SQLite does have datetime functions but they don't assume R's internal representation and conversion is required like this:
dd <- data.frame(now = Sys.time()) # test data
dd
## now
## 1 2017-03-29 07:39:30
format(dd$now, "%A") # check
## [1] "Wednesday"
sqldf("select strftime('%w', now, 'unixepoch', 'localtime') dayno from dd")
## dayno
## 1 3
2) H2 An easier alternative is to use the H2 database back end to sqldf. If the RH2 package is loaded sqldf will default to it. H2 does have true date and datetime types making manpulation of such objects much easier.
library(RH2)
library(sqldf)
sqldf("select day_of_week(now)-1 dayno, dayname(now) dayname from dd")
## dayno dayname
## 1 3 Wednesday
I have a SAS code that I need to convert into R.
My SAS code is something like this -
proc sql;
create table data as
select a.*,b.qty from Sales as a inner join Units as b
on a.id=b.id and put(a.date,yymmn6.)=put(c.date,yymmn6.)
quit;
I know that put(a.date,yymmn6.) converts the date into a SAS date value. But what does a.date become after this function? If date=01jan2012, put(a.date,yymmn6.) makes it as some SAS value that represents 201201 or 20120101? i.e. the SAS value created will stand for the whole date or just the year and mon of the date?
Currently, I am writing the R code for this as -
data <- sqldf("select a.*,b.qty from Sales as a inner join Units as b
on a.id=b.id and a.date=c.date")
Should I be doing it as -
Sales$date <- as.yearmon(Sales$date)
Units$date <- as.yearmon(Units$date)
data <- sqldf("select a.*,b.qty from Sales as a inner join Units as b
on a.id=b.id and a.date=c.date")
I don't have access to SAS and hence, I cannot try this out on a sample data. Any help would be great. Thanks!
put(a.date,yymmn6.) converts a numeric date value to a character value stored as yyyymm (e.g. 201201). Therefore the join condition is matching all dates where the month and year are the same, but not necessarily the day.
I'm not sure of the best way of achieving this in R, but you seem to have some ideas on this.
Hope this helps.
When you use put(a.date,yymmn6.) the output of that function is a character. Put takes a numeric input and format and outputs the formatted numeric value as character. input function does the opposite.
data mydata;
sas_numeric_date = "01jan2012"d;
sas_yyyymm_char_date = put(sas_numeric_date, yymmn6.);
sas_yyyymm_numeric_date = input(sas_yyyymm_char_date, yymmn6.);
output;
sas_numeric_date = "29Feb2012"d;
sas_yyyymm_char_date = put(sas_numeric_date, yymmn6.);
sas_yyyymm_numeric_date = input(sas_yyyymm_char_date, yymmn6.);
output;
format sas_numeric_date sas_yyyymm_numeric_date date9.;
run;
sas_numeric_date sas_yyyymm_char_date sas_yyyymm_numeric_date
01Jan2012 201201 01Jan2012
29Feb2012 201202 01Feb2012
So, when you apply the yymmn6. as informat on sas_yyyymm_char_date - which itself is in yyyymm format, the resulting value is numeric and day part in the date defaults to the first day of the month as shown above.
I'm trying to re-partition some table using week number counting from some day:
my_fact table contains a field called time_stamp of type TIMESTAMPTZ
Unfortunately, re-partition doesn't work, and I'm getting the error:
MyDB=> ALTER TABLE my_fact PARTITION BY MOD(TIMESTAMPDIFF('day', time_stamp::TIMESTAMP, TIMESTAMP '2013-09-23'), 156) REORGANIZE;
NOTICE 4954: The new partitioning scheme will produce 12 partitions
ROLLBACK 2552: Cannot use meta function or non-deterministic function in PARTITION BY expression
Should the cast of time_stamp to TIMESTAMP strip any time zone related info from this field thus making it deterministic?
Thanks!
Take a look at the date_part() function, you can use the TIMESTAMPTZ as its source column:
Example :
**The number of the week of the calendar year that the day is in.**
SELECT EXTRACT(WEEK FROM TIMESTAMP '2001-02-16 20:38:40');
Result: 7
SELECT EXTRACT(WEEK FROM DATE '2001-02-16');
Result: 7
Since I got no answer, I'm writing here what I've ended up with:
ALTER TABLE my_fact PARTITION BY
MOD(
TIMESTAMPDIFF(
'day',
'2013-09-23'::timestamptz AT TIME ZONE 'UTC',
time_stamp AT TIME ZONE 'UTC'),
156)
REORGANIZE;
This solution works.