How to use DateTime with SQLite via the SQLProvider type-provider?
SQLite doesn't really have a date and time datatype (see Data types) and stores dates as text. I can pass a date string and query it, and get back a string. So this works, where Date1 in table3 was stored as a string:
query {
for r in table3 do
select (r.Date1)
} |> Seq.toList
val it : string list =
["2016/06/09 0:00:00"; "2016/06/05 0:00:00"; "2016/06/04 0:00:00";
"2016/06/12 0:00:00"; "2016/06/10 0:00:00"; "2016/06/06 0:00:00";
It is also possible to store Date1 as a DateTime, and in another table I have it as such. That is even though SQLite doesn't understand DateTime, I can create a column with the DateTime data type, and I can store a DateTime value in it. I can extract this value in C# (or LinqPad) for example. But when I try to access it via the type provider (the same type provider that let me store the DateTime value), it gives the following error:
System.FormatException: String was not recognized as a valid DateTime.
> at System.DateTimeParse.ParseExactMultiple(String s, String[] formats, DateTimeFormatInfo dtfi, DateTimeStyles style)
at System.Data.SQLite.SQLiteConvert.ToDateTime(String dateText, SQLiteDateFormats format, DateTimeKind kind, String formatString)
at System.Data.SQLite.SQLite3.GetDateTime(SQLiteStatement stmt, Int32 index)
at System.Data.SQLite.SQLite3.GetValue(SQLiteStatement stmt, SQLiteConnectionFlags flags, Int32 index, SQLiteType typ)
at System.Data.SQLite.SQLiteDataReader.GetValue(Int32 i)
at <StartupCode$FSharp-Data-SqlProvider>.$SqlRuntime.DataContext.FSharp-Data-Sql-Common-ISqlDataContext-ReadEntities#153.GenerateNext(IEnumerable`1& next)
at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at Microsoft.FSharp.Collections.SeqModule.ToArray[T](IEnumerable`1 source)
at FSharp.Data.Sql.Runtime.QueryImplementation.executeQuery(ISqlDataContext dc, ISqlProvider provider, SqlExp sqlExp, List`1 ti)
at FSharp.Data.Sql.Runtime.QueryImplementation.SqlQueryable`1.System-Collections-Generic-IEnumerable`1-GetEnumerator()
at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
at <StartupCode$FSI_0075>.$FSI_0075.main#()
The difference between the working (type provider) query in table3 and the one with the error in table2 is the data type of the column:
LinqPad correctly sees it as a DateTime in Table2 and this query works there:
var dt2 = new System.DateTime(2016,8,30,0,0,0,DateTimeKind.Local);
Table2
.Where(r => r.Date1 == dt2).Dump();
DateTime type is working as intended with the following versions with SQLProvider 1.0.31 and SQLite 1.0.102:
#if INTERACTIVE
#I #"..\packages\SQLProvider.1.0.31\lib"
#r "FSharp.Data.SqlProvider.dll"
#I #"..\packages\System.Data.SQLite.Core.1.0.102.0\lib\net46"
#r "System.Data.SQLite.dll"
#I #"..\packages\System.Data.SQLite.Linq.1.0.102.0\lib\net46"
#r "System.Data.SQLite.Linq.dll"
#endif
open System
open FSharp.Data.Sql
//open System.Data.SQLite
//open System.Data.SQLite.Linq
[<Literal>]
let connectionString = "Data Source="+ #"C:\tmp\databaseFile.db3"
[<Literal>]
let resolutionPath = __SOURCE_DIRECTORY__ + #"..\..\packages\System.Data.SQLite.Core.1.0.102.0\lib\net46"
type sql = SqlDataProvider<
Common.DatabaseProviderTypes.SQLITE,
connectionString,
ResolutionPath = resolutionPath,
CaseSensitivityChange = Common.CaseSensitivityChange.ORIGINAL>
let ctx = sql.GetDataContext()
let table2 = ctx.Main.Table2 //DateTime
let table3 = ctx.Main.Table3 //Text
Related
Found the issue:
SqlKata compiler was transforming the column names into string literals, so that was returned when a matching column was not located.
Updating the queries to use brackets instead of quotes resolved the issue.
Created github issue here regarding the issue: https://github.com/sqlkata/querybuilder/issues/655
Initial post contents retained below.
I was doing some unit testing against a Sqlite database, ensuring that my methods for creation and reading all work fine (They do). But One of the tests failed, and I am absolutely confused as to why.
The Sqlite db consists of a single table, defined below:
TableName: Students
Columns: ID (Primary Key), FirstName (string), LastName (string)
The following query works properly, returning the 'FirstName' value within the db:
"SELECT \"FirstName\" FROM \"Students\" WHERE \"ID\" = #p0"
The following query I would expect would cause an exception, since the column name does not exist:
"SELECT \"UnknownCol\" FROM \"Students\" WHERE \"ID\" = #p0"
Instead, I receive the value 'UnknownCol' as a string result.
For reference, I’m using the same method (which processes a DbCommand object) to perform the same thing at against an Excel file via OledbCommand. That function produces an exception (not a helpful one, but atleast it error our). So I know the underlying method works.
Why would sqlite return the name of a column that doesn't exist in that query?
Additional Info Edit:
Using an OledbConnection to read from an Excel sheet using the same method results in the following exception when I request an invalid column within the query (which while it doesn't tell you its a bad query due to invalid column name, atleast it errors out):
Exception Message: No value given for one or more required parameters.
Full code chain:
//db object has a method that returns a SqliteConnection, and has a 'Compiler' property that returns the SqlKata.Compiler object for SqlLite
var qry = new SqlKata.Query("Students").Select("UnknownCol").Where("ID",1);
return GetValue(db.GetConnection(), qry, db.Compiler);
//Results in the following sql:
"SELECT \"UnknownCol\" FROM \"Students\" WHERE \"ID\" = 1"
---
public static object GetValue(DbConnection connection, Query query, SqlKata.Compilers.Compiler compiler)
{
using (var cmd = connection.CreateCommand(query, compiler))
{
connection.Open();
try
{
return cmd.ExecuteScalar();
}
finally
{
connection.Close();
}
}
}
public static DbCommand CreateCommand(this DbConnection connection, SqlKata.Query query, SqlKata.Compilers.Compiler compiler)
{
if (connection is null) throw new ArgumentNullException(nameof(connection));
if (compiler is null) throw new ArgumentNullException(nameof(compiler));
var result = compiler.Compile(query ?? throw new ArgumentNullException(nameof(query)));
var cmd = connection.CreateCommand();
cmd.CommandText = result.Sql;
foreach (var p in result.NamedBindings)
{
_ = cmd.AddParameter(p.Key, p.Value);
}
return cmd;
}
public static DbParameter AddParameter(this DbCommand command, string name, object value)
{
var par = command.CreateParameter();
par.ParameterName = name;
par.Value = value;
command.Parameters.Add(par);
return par;
}
It's legal to select a string litteral in SQL. This is a valid SQL query which returns the mentioned string:
SELECT 'UnknownCol';
It will return a single row containing this string litteral.
The following query is similar
SELECT 'UnknownCol' FROM students;
For each row in your table, it will return a row with this string litteral.
Here is an example on a test table with a few rows in a test database:
sqlite> select 'a string litteral' from test;
a string litteral
a string litteral
a string litteral
a string litteral
a string litteral
sqlite> select count(1) from test;
5
sqlite>
If you want to query a specific column name instead of a string litteral you have to remove the '' characters around the column name.
Then this is the result with an undefined column:
sqlite> select unknowncol from test;
Parse error: no such column: unknowncol
select unknowncol from test;
^--- error here
sqlite>
or for a defined column:
sqlite> select id from test;
1
2
3
4
6
sqlite>
So I'm trying to check if some of my values match, for that I'm using an SQLite object and casting it as date and passing it as a parameter which is also a Date, I just want to know if this is the right way to do it?
"AND R1.TIMING = ?
AND R1.VARIETY = ?
AND R1.JOBACRES = ?
AND CAST (R1.PLANTINGDATE AS DATE) = ?
new object[] { item.TIMING, item.VARIETY, item.JOBACRES, item.PLANTINGDATE.Date }).ToList();
This is written in C#, since I want to execute this query in C# with SQLite, any inputs would be helpful
I would like to get two columns like SoccerStatus, SoccerDate from the SoccerAvailability in SQLite database into a list where SoccerDate="Today" using LINQ query ? I have tried the below query, but its throws exception, I need CurrentDate alone, no time is needed in my query.
List<SoccerAvailability> myList = (from x in conn.Table<SoccerAvailability>().Where(x => x.CurrentDate == DateTime.Now.ToString("ddMMyyyy")) select x).ToList();
Unhandled Exception:
SQLite.SQLiteException: no such function: tostring
convert the current date to a string before you execute your query
var currentDate = DateTime.Now.ToString("ddMMyyyy");
List<SoccerAvailability> myList = (from x in conn.Table<SoccerAvailability>().Where(x => x.CurrentDate == currentDate) select x).ToList();
I'm trying to translate this SQL statement to LINQ:
SELECT sessionid, userid, CAST(sessiondate AS DATETIME) + CAST(sessiontime AS DATETIME) AS sessiondatetime FROM sometable
where sessiondate is of type DATE and sessiontime is of type TIME.
I've tried the following:
var query = from session in table
select new
{
session.Id,
session.UserId,
DateTime = session.Date + session.Time
};
where table is the return value of a GetTable<Session>() call on a DataContext instance and the Session class maps sessionid to Id, userid to UserId, sessiondate to Date (DateTime), and sessiontime to Time (TimeSpan).
The LINQ gets translated to this rather lengthy SQL statement:
SELECT [t0].[sessionid] AS [Id], [t0].[userid] AS [UserId], CONVERT(DateTime,DATEADD(HOUR, DATEPART(HOUR, [t0].[sessiontime]), CONVERT(DateTime,DATEADD(MINUTE, DATEPART(MINUTE, [t0].[sessiontime]), CONVERT(DateTime,DATEADD(SECOND, DATEPART(SECOND, [t0].[sessiontime]), CONVERT(DateTime,DATEADD(MILLISECOND, DATEPART(MILLISECOND, [t0].[sessiontime]), [t0].[sessiondate])))))))) AS [DateTime] FROM [sometable] AS [t0]
Unfortunately, when attempting to execute that statement, it tells me that "The datepart millisecond is not supported by date function dateadd for data type date." I'm guessing it's unhappy about the DATEADD call with milliseconds. Is there a way to fix this?
Edit: note that both session.Date and session.Time are nullable.
That unfortunate data structure makes the code unfortunately ugly:
var query = from session in table
select new
{
session.id,
session.userid,
combinedDate = new DateTime(
session.Date.Year, session.Date.Month, session.Date.Day,
session.Time.Hour, session.Time.Minute, session.Time.Second,
session.Time.Millisecond)
};
I want to create a table in SQLite in which one of the field is for date, in which date and time of current instance should save. Which data type should I use?
I'm planning to use 'timestamp'. How to insert current timestamp value to the field? Also how to write content values for this date field?
SQLite supports the standard SQL variables CURRENT_DATE, CURRENT_TIME, and CURRENT_TIMESTAMP:
INSERT INTO Date (LastModifiedTime) VALUES(CURRENT_TIMESTAMP)
The default data type for dates/times in SQLite is TEXT.
ContentValues do not allow to use generic SQL expressions, only fixed values, so you have to read the current time in Java:
cv.put("LastModifiedTime",
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date()));
INSERT INTO Date (LastModifiedTime) VALUES(DateTime('now'))
Use this site for further reference.
To get the current local(system) time, add the 'localtime' option:
select datetime('now', 'localtime');
I'm using timestamps a lot in my app. For me the best way to keep the timestamp is to convert it in milliseconds. After that it is easy to convert it to any locale.
If you need the current time use System.currentTimeMillis().
Content values are easy to use, you just and field and value, like:
ContentValues ins_reminder = new ContentValues();
ins_reminder.put("REMIND_TIMESTAMP", System.currentTimeMillis());
Since SQLite 3.38.0, there is a unixepoch() function that returns UNIX timestamp in integer. Does the same thing as strftime('%s').
References:
release log draft
check-in
In my case i wanted to have a timestamp with fractions of a second.
The keyword CURRENT_TIMESTAMP has only a precision of YYYY-MM-DD HH:MM:SS (see docs DEFAULT clause).
The function strftime() can return fractions of a second
Example to use strftime() in an INSERT
INSERT INTO YourTable (TimeStamp)
VALUES (strftime('%Y-%m-%d %H:%M:%S:%s'))
Comparison of CURRENT_TIMESTAMP and strftime()
SELECT 'CURRENT_TIMESTAMP' as Timestamp_Command,
CURRENT_TIMESTAMP as TimeStamp_Precision,
'only seconds' as Timestamp_Comment
UNION ALL
SELECT 'strftime(%Y-%m-%d %H:%M:%S:%s)' as Timestamp_Command,
(strftime('%Y-%m-%d %H:%M:%S:%s')) as TimeStamp_Precision,
'with fraction of a second' as Timestamp_Comment
Example to use it in c#
The following is based on bulk insert in sqlite with ado.net
public static void InsertBulk(SqliteConnection connection)
{
connection.Open();
using (var transaction = connection.BeginTransaction())
{
var command = connection.CreateCommand();
command.CommandText =
#"INSERT INTO BulkInsertTable (CreatedOn, TimeStamp)
VALUES ($createdOn, strftime('%Y-%m-%d %H:%M:%S:%s'))";
var parameter3 = command.CreateParameter();
parameter3.ParameterName = "$createdOn";
command.Parameters.Add(parameter3);
// Insert a lot of data
// calling System.DateTime.Now outside the loop is faster
var universalTime = System.DateTime.Now.ToUniversalTime();
for (var i = 0; i < 15_000; i++)
{
parameter3.Value = System.DateTime.Now.ToUniversalTime();
// faster
// parameter3.Value = universalTime;
command.ExecuteNonQuery();
}
transaction.Commit();
}
connection.Close();
}