Can you tell my the date in the SQLite database is taking as
12/3/1899 instead of 12/3/2009.
I am inserting correct date while saving in this format. I can't understand what is the reason.What conversion has to be done in Insert Statement. Can any one help me out.
CREATE TABLE [PIs] ( [PIGUID] GUID PRIMARY KEY NOT NULL,
[CompanyGUID] GUID NOT NULL,
[No] varCHAR(50) NOT NULL,
[Dt] TIMESTAMP NOT NULL,
[SupplierLgrGUID] GUID NOT NULL,
[SupplierLgrAddressGUID] GUID NOT NULL,
[SupplierBillNo] varCHAR(50) NULL,
[SupplierBillDt] TIMESTAMP NULL,
[CrDays] INTEGER NULL,
[DueDt] TIMESTAMP NULL,
[Narration] varCHAR(300) NULL,
[CreatedDt] TIMESTAMP NOT NULL,
[LastEditedDt] TIMESTAMP NOT NULL,
)
My Insert Statement is as follow:
INSERT INTO PIs(
PIGUID,CompanyGUID,No,Dt,SupplierLgrGUID,SupplierLgrAddressGUID,
SupplierBillNo,SupplierBillDt,CrDays,DueDt,Narration,CreatedDt,
LastEditedDt)
VALUES(
'806aeec2-762a-432e-800f-0354df3b7852' ,
'375888f5-e1a5-4c75-9154-62ffc83dca97', 'PI/0809/004' ,
datetime('8/19/2009 12:44:25 PM'),
'ff376218-c2d9-4e02-86e6-e90c8d5efc43',
'7dad4725-2e37-4596-88f4-7b088f0d91c4', '0021',
datetime('8/19/2009 12:44:25 PM'), 12,
datetime('8/31/2009 12:44:25 PM'), 'narration',
datetime('now'),datetime('now')
)
I have installed SQLite 3.3.8 and my operating system is Windows Vista.
Any help would be appreciated..
Regards
Asif
The sqlite format for date is the ISO standard ie YYYY-MM-DD and not the one you have used
See Sqllite date format
Try
SELECT date('now');
to see
This link has some information you should read on using TIMESTAMP and DATETIME for SQLite.
Here's an excerpt:
TIMESTAMP, DATETIME
A string type of unlimited length used to store date/time combinations. The required format is 'YYYY-MM-DD HH:MM:SS', anything not following this pattern is ignored.
I had trouble with this one also... If your using VB.NET here is how to insert the TimeStamp, I mean DateTime into SQLlite
DateTime.UtcNow.ToString("o")
The important part is the .ToString("o") because this formats the date to ISO 8601
Here is the link that explains everything about the format - http://msdn.microsoft.com/en-us/library/az4se3k1.aspx#Roundtrip
It looks like this when it goes into the database.
2010-12-16T02:09:55.7870590Z
But After you insert the data it looks like this.
12/16/2010 2:09:55 AM
Hope this helps someone else out there.
Related
I constantly retrieve JSON data from some API and put that data into a MariaDB table.
The JSON ships with a timestamp which I'd like to place an index on, because this attribute is used for querying the table.
The JSON looks something like this (stripped):
{
"time": "2021-12-26T14:00:00.007294Z",
"some_measure": "0.10031"
}
I create a table:
CREATE TABLE some_table (
my_json JSON NOT NULL,
time TIMESTAMP AS (JSON_VALUE(my_json , '$.time')),
some_measure DOUBLE AS (JSON_VALUE(my_json , '$.some_measure'))
)
ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
my_json holds the entire JSON snippet, time and some_measure are virtual columns properly extracting the corresponding JSON values on the fly.
Now, trying to add an index on the TIMESTAMP attribute:
CREATE INDEX some_index ON some_table (time);
This fails:
SQL Error [1292] [22007]: (conn=454) Incorrect datetime value:
'2021-12-26T14:00:00.007294Z' for column `some_db`.`some_table`.`time` at row 1
How can I add an index on that timestamp?
The issue here is that converting a string (the JSON timestamp) to a TIMESTAMP is non-deterministic because it involves server side settings (sql_mode) and timezone settings.
Indexing virtual columns which are non-deterministic is not supported.
You would want to use a VARCHAR data type instead and index that:
CREATE TABLE some_table (
my_json JSON NOT NULL,
time VARCHAR(100) AS (JSON_VALUE(my_json , '$.time')),
some_measure DOUBLE AS (JSON_VALUE(my_json , '$.some_measure'))
)
ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
You should be able to create your index:
CREATE INDEX some_index ON some_table (`time`);
You can still query time because MariaDB automatically converts DATETIMEs if used against a VARCHAR:
SELECT
*
FROM some_table
WHERE time > '2008-12-31 23:59:59' + INTERVAL 1 SECOND;
The query will use the index:
I finally came up with a solution that works for me.
Changes are:
use STR_TO_DATE() to create a valid DATETIME from the JSON timestamp
make the generated (virtual) column PERSISTENT
use data type DATETIME instead of TIMESTAMP
So the new code looks like this:
CREATE TABLE some_table (
my_json JSON NOT NULL,
time DATETIME AS (STR_TO_DATE((JSON_VALUE(my_json , '$.time')), '%Y-%m-%d%#%T%.%#%#')) PERSISTENT,
some_measure DOUBLE AS (JSON_VALUE(my_json , '$.some_measure'))
)
ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
CREATE INDEX some_index ON some_table (`time`);
I am facing an issue with the fastload where my Timestamp fields are getting rejected to error table.
Below is the value of timestamp(6) field in my flat file.
23-06-2016 11:51:21.000000 23-06-2016 11:51:21.000000
Below is my code:
SET RECORD VARTEXT "ยก";
DEFINE
TRANSACTION_SOURCE_TYPE_ID (VARCHAR(54))
,TRANSACTION_SOURCE_TYPE_CODE (VARCHAR(20))
,TRANSACTION_SOURCE_TYPE_DESC (VARCHAR(110))
,EFFECTIVE_START_DATE (VARCHAR(54))
,EFFECTIVE_END_DATE (VARCHAR(54))
,COUNTRY_CODE (VARCHAR(13))
,SOURCE_SYSTEM_ID (VARCHAR(54))
,DW_LOAD_TIMESTAMP (VARCHAR(76))
,DW_UPD_LOAD_TIMESTAMP (VARCHAR(76))
,FORCE_SKEW_KEY (VARCHAR(51))
FILE=?INPUT_FILE;
SHOW;
INSERT INTO ?DWSBKPDB.TRANSACTION_TMP
(
TRANSACTION_SOURCE_TYPE_ID
,TRANSACTION_SOURCE_TYPE_CODE
,TRANSACTION_SOURCE_TYPE_DESC
,EFFECTIVE_START_DATE
,EFFECTIVE_END_DATE
,COUNTRY_CODE
,SOURCE_SYSTEM_ID
,DW_LOAD_TIMESTAMP
,DW_UPD_LOAD_TIMESTAMP
,FORCE_SKEW_KEY
)
VALUES
(
:TRANSACTION_SOURCE_TYPE_ID
,:TRANSACTION_SOURCE_TYPE_CODE
,:TRANSACTION_SOURCE_TYPE_DESC
,:EFFECTIVE_START_DATE (DATE, FORMAT 'YYYY-MM-DD')
,:EFFECTIVE_END_DATE (DATE, FORMAT 'YYYY-MM-DD')
,:COUNTRY_CODE
,:SOURCE_SYSTEM_ID
,:DW_LOAD_TIMESTAMP (TIMESTAMP, FORMAT 'YYYY-MM-DDBHH:MI:SS.S(6)')
,:DW_UPD_LOAD_TIMESTAMP (TIMESTAMP, FORMAT 'YYYY-MM-DDBHH:MI:SS.S(6)')
,:FORCE_SKEW_KEY
);
DW_LOAD_TIMESTAMP is creating problem here.
Any idea as to why this is happening.
Regards,
Amit
Your data obviously doesn't match the FORMAT.
:DW_LOAD_TIMESTAMP(TIMESTAMP, FORMAT 'DD-MM-YYYYBHH:MI:SS.S(6)')
Let's say I need to CAST(birth_date AS DATE FORMAT 'MM/DD/YYYY')
If the birth_date field contains nulls or invalid characters it throws and untranslatable character error
Of course I can use the regex, otranslate but all that overcomplicates the sql
Is there any way to suppress all these errors ? CAST if you can, otherwise make it null?
The burden of checking whether the data fits into the data type you wish to store it must reside somewhere. You could use CASE {regular expression matching} THEN CAST() ELSE NULL END which may be the cleanest way to address the data quality validation in your SQL.
Otherwise, pre-process your data file to replace bad data with a token you can replace with NULL in your SQL. You can consider doing this in PowerShell, UNIX shell scripting, or perhaps a third-party tool (e.g. address cleansing/formatting, etc.).
Since there is no built in way to say CAST(<field> as <datatype) IGNORING ERRORS AS <alias> you could use a TPT script instead.
In TPT APPLY you can have an INSERT statement route errors into two different Error tables.
Something like the following would get you close. This is something that you would run after your dirty date table is loaded to get them into a clean date table.
DEFINE JOB DATA_insert_Example
(
DEFINE OPERATOR data_insert_Example
TYPE UPDATE
SCHEMA *
ATTRIBUTES
(
VARCHAR UserName,
VARCHAR UserPassword,
VARCHAR LogTable,
VARCHAR TargetTable,
INTEGER BufferSize,
INTEGER ErrorLimit = 5,
INTEGER MaxSessions = 4,
INTEGER MinSessions = 1,
INTEGER TenacityHours,
INTEGER TenacitySleep,
VARCHAR AccountID,
VARCHAR AmpCheck,
VARCHAR DeleteTask,
VARCHAR ErrorTable1 = '<yourdatabase>.<yourcleantable>'||'_ET',
VARCHAR ErrorTable2 = '<yourdatabase>.<yourcleantable>'||'_UV',
VARCHAR NotifyExit,
VARCHAR NotifyExitIsDLL,
VARCHAR NotifyLevel,
VARCHAR NotifyMethod,
VARCHAR NotifyString,
VARCHAR PauseAcq,
VARCHAR PrivateLogName,
VARCHAR TdpId,
VARCHAR TraceLevel,
VARCHAR WorkingDatabase = <yourdatabase>,
VARCHAR WorkTable = '<yourdatabase>.<yourcleantable>'||'_Work'
);
DEFINE SCHEMA data_insert_schema
(
field1 VARCHAR(20),
field2 VARCHAR(20),
field3 VARCHAR(20),
field4 VARCHAR(20)
);
DEFINE OPERATOR data_insert_export
TYPE EXPORT
SCHEMA W_0_s_DATA_esuh
ATTRIBUTES
(
VARCHAR UserName,
VARCHAR UserPassword,
STEP UPS
(
APPLY
(
'INSERT INTO <yourdatabase>.<yourcleantable>
field1,
field2,
field3,
field4
VALUES (
:field1,
:field2,
:field3,
:field4,
)';
)
TO OPERATOR
(
data_insert_Example[1]
ATTRIBUTES
(
UserName = <yourusername>,
UserPassword = <yourpass>,
LogTable = <yourdatabase>.<yourcleantable> ||'_LOG',
TargetTable = <yourdatabase>.<yourcleantable> ,
TdpId = <yourserverip/address>
)
)
SELECT * FROM OPERATOR
(
data_insert_export[1]
ATTRIBUTES
(
UserName = <yourusername>,
UserPassword = <yourpassword>,
SelectStmt = 'SELECT field1,field2,field3,field4 FROM <yourdatabase>.<yourtable> ;',
TdpId = '<yourserverip/address>'
)
);
);
)
Obviously, though, this is quite a bit more overkill than a simple REGEX. RegEx feels way overwhelming when you first start using it, but I think it's a completely reasonable usecase for checking dates stored as string literals before trying to convert them into their proper data type.
Overall, it sounds like you have garbage for data, so I totally get the frustration. Unfortunately for garbage data there is no magic bullet. You'll need some decent ETL between the garbage and your clean output.
Here I am Giving The Queries What I am using:
1.INSERT into sample values(convert(varchar,'19-11-2014 10:10:41',103))
2.INSERT into sample values (convert(varchar,'2014-11-19 10:10:41',103))
3.INSERT into sample values(convert(varchar,'11-19-2014 10:10:41',103))
The database Format is:yyyy/mm/dd HH:mm:ss:mmm
In Above Queries First one Throwing Error and Remaining Two Queries Working Fine.so How to insert Any Datetime Format into Sql No change in Query.
Please Reply As Early As Possible,Thank You.
Since you said using ASP.Net then why are you using plain SQL queries?
Use parameters instead.
Lets say you have a date in string format 19/11/2014 and you need to insert into db so the correct way will be first convert the string into a date like
DateTime date= DateTime.ParseExact("19/11/2014 00:00:00","dd/MM/yyyy HH:mm:ss", System.Globalization.CultureInfo.InvariantCulture);
now use parametrised query to insert see details-->http://www.aspsnippets.com/Articles/ASPNet-SqlDataSource-pass-value-to-SelectParameter-using-QueryString-Parameter-Example.aspx
Using parameters will serve you two main purpose
You will be safe from SQL Injection attack
the problem that you are currently facing with format of DateTime will go away since the serialization will be done by ASP.Net.
this is very general , but complicated situation to get the input date which in proper format or not. Which ultimately gives an error at run-time , even in the production server we get the error and we rectify the issue hour and hour.
Ultimately sqlserver introduce the new function called "ISDATE" or TRY_PARSE-only sql2012 or above, which return the 1 if input data is date format.
declare #dTable table ( datecolumn datetime)
INSERT into #dTable values (case isdate('19-11-2014 10:10:41') when 1 then '19-11-2014 10:10:41' else null end )
INSERT into #dTable values ('2014-11-19 10:10:41')
INSERT into #dTable values ('11-19-2014 10:10:41')
select * from #dTable
If still you want , this is not right way, then you can create a function of dateformat, in which you can give string datevalue in any format and function give the data date or null return.
check this links.
http://www.codeproject.com/Articles/576178/cast-convert-format-try-parse-date-and-time-sql#4
create function convertStringIntoDate
(
#stringValue varchar(50)
)
RETURNS datetime
AS
BEGIN
DECLARE #datereturn datetime
set #datereturn = TRY_PARSE( #stringValue)
RETURN #datereturn
END
or you can check this logic too.
declare #dt varchar(50) = '19-11-2014 10:10:41'
declare #dTable table ( datecolumn datetime)
INSERT into #dTable values (
case
when isdate(CONVERT( varchar(50), #dt)) = 1 then CONVERT( varchar(50), #dt) --'19-11-2014 10:10:41'
when isdate(CONVERT( varchar(50), #dt, 103) ) = 1 then CONVERT( datetime, #dt , 103 ) --'19-11-2014 10:10:41'
when isdate(CONVERT( varchar(50), #dt, 102) ) = 1 then CONVERT( datetime, #dt , 102 ) --'19-11-2014 10:10:41'
--when --give other format as above given and if not set in any dateformat , then simply return null
else
null
end )
select * from #dTable
Try this.
INSERT into sample values (convert(datetime,'19-11-2014 10:10:41',103))
INSERT into sample values (convert(datetime,'2014-11-19 10:10:41',102))
INSERT into sample values (convert(datetime,'11-19-2014 10:10:41',102))
I have an SQLite database, version 3 and I am using C# to create an application that uses this database.
I want to use a timestamp field in a table for concurrency, but I notice that when I insert a new record, this field is not set, and is null.
For example, in MS SQL Server if I use a timestamp field it is updated by the database and I don't have to set it by myself. Is this possible in SQLite?
Just declare a default value for a field:
CREATE TABLE MyTable(
ID INTEGER PRIMARY KEY,
Name TEXT,
Other STUFF,
Timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
);
However, if your INSERT command explicitly sets this field to NULL, it will be set to NULL.
You can create TIMESTAMP field in table on the SQLite, see this:
CREATE TABLE my_table (
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
name VARCHAR(64),
sqltime TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
);
INSERT INTO my_table(name, sqltime) VALUES('test1', '2010-05-28T15:36:56.200');
INSERT INTO my_table(name, sqltime) VALUES('test2', '2010-08-28T13:40:02.200');
INSERT INTO my_table(name) VALUES('test3');
This is the result:
SELECT * FROM my_table;
Reading datefunc a working example of automatic datetime completion would be:
sqlite> CREATE TABLE 'test' (
...> 'id' INTEGER PRIMARY KEY,
...> 'dt1' DATETIME NOT NULL DEFAULT (datetime(CURRENT_TIMESTAMP, 'localtime')),
...> 'dt2' DATETIME NOT NULL DEFAULT (strftime('%Y-%m-%d %H:%M:%S', 'now', 'localtime')),
...> 'dt3' DATETIME NOT NULL DEFAULT (strftime('%Y-%m-%d %H:%M:%f', 'now', 'localtime'))
...> );
Let's insert some rows in a way that initiates automatic datetime completion:
sqlite> INSERT INTO 'test' ('id') VALUES (null);
sqlite> INSERT INTO 'test' ('id') VALUES (null);
The stored data clearly shows that the first two are the same but not the third function:
sqlite> SELECT * FROM 'test';
1|2017-09-26 09:10:08|2017-09-26 09:10:08|2017-09-26 09:10:08.053
2|2017-09-26 09:10:56|2017-09-26 09:10:56|2017-09-26 09:10:56.894
Pay attention that SQLite functions are surrounded in parenthesis!
How difficult was this to show it in one example?
Have fun!
you can use triggers. works very well
CREATE TABLE MyTable(
ID INTEGER PRIMARY KEY,
Name TEXT,
Other STUFF,
Timestamp DATETIME);
CREATE TRIGGER insert_Timestamp_Trigger
AFTER INSERT ON MyTable
BEGIN
UPDATE MyTable SET Timestamp =STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW') WHERE id = NEW.id;
END;
CREATE TRIGGER update_Timestamp_Trigger
AFTER UPDATE On MyTable
BEGIN
UPDATE MyTable SET Timestamp = STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW') WHERE id = NEW.id;
END;
To complement answers above...
If you are using EF, adorn the property with Data Annotation [Timestamp], then
go to the overrided OnModelCreating, inside your context class, and add this Fluent API code:
modelBuilder.Entity<YourEntity>()
.Property(b => b.Timestamp)
.ValueGeneratedOnAddOrUpdate()
.IsConcurrencyToken()
.ForSqliteHasDefaultValueSql("CURRENT_TIMESTAMP");
It will make a default value to every data that will be insert into this table.
you can use the custom datetime by using...
create table noteTable3
(created_at DATETIME DEFAULT (STRFTIME('%d-%m-%Y %H:%M', 'NOW','localtime')),
title text not null, myNotes text not null);
use 'NOW','localtime' to get the current system date else it will show some past or other time in your Database after insertion time in your db.
Thanks You...
If you use the SQLite DB-Browser you can change the default value in this way:
Choose database structure
select the table
modify table
in your column put under 'default value' the value: =(datetime('now','localtime'))
I recommend to make an update of your database before, because a wrong format in the value can lead to problems in the SQLLite Browser.