We have tables in oracle having date field that we are pulling via Nifi with Avro logical types true.
The select * or column name is bringing that record with logical type as timestamp-millis. It may be because oracle stores date with hours, minutes, second. Is there any way to get it in date format in avro logical type?
Related
In Sqlite I want to extract the date and time portions of a DateTime field separately in a view and return them also as a datetime, not strings. I've tried Cast, Date(), datetime(), but they all return strings.
I've read the SQLite documentation and understand how there is not an actual Date data type. Yet a Table field defined as DateTime is able to be parsed as a Date by an Excel query, but calculations on that field are not. I'm trying to do all data prep in the database view.
My data has the following field taken directly from the table definition:
LastModifiedDate datetime
I want the date (without time) to have the same DateTime data type as LastModifiedDate, not Text, because I use this view in many spreadsheets. I can apply Excel Date functions and formatting to LastModifiedDate field directly as returned from the ODBC query to Excel, and want to do the same to the Date-only part. I don't want to have to put a string-to-date conversion in every spreadsheet when I know it can get the date natively from Sqlite in LastModifiedDate.
SELECT LastModifiedDate,
date(LastModifiedDate) as Datepart,
cast(LastModifiedDate as numeric) as Date2
FROM Transactions
LastModifiedDate Datepart Date2
2019-07-28 18:22:38.9165394 2019-07-28 2019
LastModifiedDate in the above query is interpreted in Excel as a date to which date formats and date functions can be applied with no further processing required. Datepart above is returned as Text to Excel, and I can't apply date functions and formats without further pre-processing in Excel. I would like Datepart to be interpreted a date in Excel just as LastModifiedDate is.
I'm looking at the ch-werner.de sqliteodbc-0.9998. It will return an ODBC TIMESTAMP type only if the column decltype starts with timestamp or datetime. It returns ODBC TIME only for decltypes starting with time and ODBC DATE only for decltypes starting with date.
sqlite3 provides this decltype only for result table columns that are direct database column references. So if your SELECT statement has some expression that is more than a plain column reference, the decltype is lost. sqlite3 works like this at least up to version 3.39.0. It is documented.
The CAST expression converts the value of given expression to a storage classes by the determined affinity of the given declared type, but does not assign decltype to the result.
If you want to see the decltypes for query columns, you can use the sqlite3 cli and give it command .stats 2. Then it'll output the column declared types for each statement it executes.
If the decltype is found, the sqliteodbc-0.9998 will always parse string values into ODBC types. If DSN Option JDConv is enabled, it'll also parse floating point julianday values (whether provided as float or a string of a float) into ODBC types and when writing it'll write floating point into database.
If you can afford to change the schema, you can add a generated virtual column. This is cheap in storage, because data is not affected, but it costs when you query the column. This column can calculate other column into the values and decltypes you need for ODBC.
ALTER TABLE data ADD COLUMN
Datepart date AS (date(LastModifiedDate))
Then to get the Datepart, you simply query the column.
SELECT Datepart FROM data
I have a sqlite db (it is the spiceworks db) and I am mapping the tables to grails domain classes. There is a table in particular that is in a datetime format (yyyy-MM-dd HH:mm:ss). An example of such a date in the db for anyone who wants to verify would be: 2015-06-26 15:32:39
I created the domain class and mapped my properties to the columns. Let grails generate the views so that they are default. When I try to get to the index page I get:
URI
/spiceworks/weeklyReportItem
Class
java.text.ParseException
Message
Unparseable date: "2015-06-26 15:32:39" does not match (\p{Nd}++)\Q-\E(\p{Nd}++)\Q-\E(\p{Nd}++)\Q \E(\p{Nd}++)\Q:\E(\p{Nd}++)\Q:\E(\p{Nd}++)\Q.\E(\p{Nd}++)
I've used MSSQL datetimes in the past and have never ran into this issue. In the database, the data type for the problematic column is datetime. Anyone know what's going on?
EDIT: I only have Read permissions on the db and the spiceworks source code isn't open source.
If you look at that regular expression, you'll see it's looking for a ISO8601 timestamp formatted like YYYY-MM-DD HH:mm:SS.SSS - in other words, it needs a decimal seconds, and you're just providing whole seconds.
An easy fix would be to update every existing value in the relevant column by appending '.000' to it, and update the insertion routines to do the same.
If you refer to the sqlite date time functions, there's a %f format specifier for strftime() that can be used to produce fractional seconds that might be helpful if you're building the timestamp directly in an insert query.
In the database, the data type for the problematic column is datetime
Sqlite3 doesn't have a datetime type. The timestamps you're storing are strings. More information.
I am working on a SQLite Database which contains a column which stores value in format yyyy-MM-dd HH:mm:ss. Now I need to create a filter to select rows with filter as this datetime column.
Query:
Select * from tbl_locations where datetime >= '2013-09-11 00:00:00' and datetime <='2013-09-13 00:00:00'
Above query is returning null set despite containing values in this slot(which I verified using select statement without filter.)
Any suggestion how can i get the required data set?
Perhaps this excerpt from the SQLite documentation will help you:
1.2 Date and Time Datatype
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
The date and time functions that you can use in your SQL to build your query are documented at http://www.sqlite.org/lang_datefunc.html
Ok, I tested this out in MySQL but hopefully it will work. I had a table that used timestamps, and changed the column to be of type text. Then I tried the following SQL query and got the same results that I normally would (besides trailing decimals)
SELECT timestamp(stock_quote_timestamp)
FROM stock.stock_quote
WHERE stock_quote_timestamp < timestamp('2013-10-07 11:05:30')##high_date
AND stock_quote_timestamp > timestamp('2013-10-03 14:09:03');##low_date;
So basically, just convert your text statements to timestamps so that they compare correctly. Oh, and you'll also need to state what else you're SELECTing, or you could do a compound select statement: SELECT *, timestamp(stock_quote_timestamp)...
I cannot seem to figure out why datetime does not work for me on some data I imported from CSV. I have a column, TIMESTAMP, which is of type datetime.
Select TIMESTAMP from GPS limit 1 <-This gives me a time, "6/29/2009 00:00:00"
Select datetime(TIMESTAMP) from GPS limit 1 <- This gives me a pink field in SQLite manager, which seems empty.
Select datetime('now') from GPS limit 1 <- This gives me the current date and time. ("2012-12-19 20:45:17") It is formatted differently than my other data - is there a datatype issue?
What is going on? Did my "Timestamp" data not actually get converted into a DATETIME object? Why is it stored as text? Is there a way to fix this?
SQLite does not have a native date/time type; dates are stored either as numbers or as strings.
To be understood by SQLite's built-in date functions, date strings must have a format like yyyy-mm-dd hh:mm:ss.
I use sqlite3 C/C++ API to retrieve rows from a table using SELECT query. I don't see any sqlite3_column_timestamp() to retrieve a timestamp column value after sqlite3_step().. How to get timestamp values ?
SQLite does not have a special timestamp data type.
When you want to use any of SQLite's date and time functions, you have to store timestamps in one of the formats supported by them, i.e., a string like YYYY-MM-DD HH:MM:SS or HH:MM:SS, a julian date number, or a Unix timestamp number.
You can declare a table column type as DATETIME, but SQLite will just ignore that type; SQLite always allows to put values of any type in any column. Such a declaration would be useful only as documentation.
The column/value accessors will only have types corresponding to the data types they support directly (NULL, INTEGER, REAL, TEXT, BLOB).
You would use the TEXT access to get/set the column value of dates.
There are some helper functions within SQL that they provide that let you to handle them in your queries.
I am not familiar with SQLite Manager, but I would assume that it is only reporting the data type that the table was declared with.
When parsing CREATE statements, sqlite understands the intention of many well supported datatypes and automatically maps them to what is appropriate for its internal storage structure. VARCHAR would be mapped to TEXT, for instance. I assume the column was declared DATETIME and sqlite just internally mapped it to TEXT.