How to automatically set timestamp in room SQLite database? - sqlite

I am trying to have SQLite create automatic timestamps with CURRENT_TIMESTAMP.
I took the liberty of using Google's code:
// roomVersion = '2.2.2'
#Entity
public class Playlist {
#PrimaryKey(autoGenerate = true)
long playlistId;
String name;
#Nullable
String description;
#ColumnInfo(defaultValue = "normal")
String category;
#ColumnInfo(defaultValue = "CURRENT_TIMESTAMP")
String createdTime;
#ColumnInfo(defaultValue = "CURRENT_TIMESTAMP")
String lastModifiedTime;
}
#Dao
interface PlaylistDao {
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insert(playlist: Playlist): Long
}
This translates into an SQLite-Statement:
CREATE TABLE `Playlist` (
`playlistId` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
`name` TEXT,
`description` TEXT,
`category` TEXT DEFAULT 'normal',
`createdTime` TEXT DEFAULT CURRENT_TIMESTAMP,
`lastModifiedTime` TEXT DEFAULT CURRENT_TIMESTAMP
)
I did make one insert:
mDb.playListDao().insert(Playlist().apply { name = "Test 1" })
But the timestamps are always Null.
With the DB Browser for SQLite I added another entry, here I get timestamps.
How do I insert without a Null-Timestamp in room?
(Info: createdTime is also always the same as lastModifiedTime. I think this has to be done with triggers in SQLite, but that is a different problem not to be discussed here).

You don't need to use another class, you can use #Query as an alternative to the convenience #Insert.
as per :-
There are 4 type of statements supported in Query methods: SELECT, INSERT, UPDATE, and DELETE.
Query
e.g.
#Query("INSERT INTO test_table001 (name) VALUES(:name) ")
void insert(String name);
You are also not limited to CURRENT_TIMESTAMP as the only means of getting the current timestamp you can use embedded datetime functions (as is shown below), which can store the value more efficiently and also be more flexible e.g. you could adjust the current time using modifiers such as '+7 days'.
If you consider the following :-
#Entity(tableName = "test_table001")
public class TestTable001 {
#PrimaryKey
Long id;
#ColumnInfo(defaultValue = "CURRENT_TIMESTAMP")
String dt1;
#ColumnInfo(defaultValue = "(datetime('now'))")
String dt2;
#ColumnInfo(defaultValue = "(strftime('%s','now'))")
String dt3;
String name;
}
Note that the inefficient autogenerate = true has not been used BUT as will be shown you can still have an SQLite assigned id (note that you must use the type Long/Integer as opposed to long or int)
Also note the alternative ways of getting the current date time (the latter being more efficient as the value will ultimately be stored as an Integer (max 8 bytes) rather than a more byte hungry String).
With a Dao as :-
#Dao
public interface TestTable001Dao {
#Insert()
long insert(TestTable001 testTable001);
#Query("INSERT INTO test_table001 (name) VALUES(:name) ")
long insert(String name);
#Query("SELECT * FROM test_table001")
List<TestTable001> getAllTestTable001();
}
And the following to test/demonstrate :-
public class MainActivity extends AppCompatActivity {
AppDatabase mRoomDB;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
mRoomDB = Room.databaseBuilder(this,AppDatabase.class,"testdb")
.allowMainThreadQueries()
.build();
TestTable001 tt01 = new TestTable001();
tt01.setName("tt01");
mRoomDB.useTestTable001().insert(tt01);
mRoomDB.useTestTable001().insert("tt02");
logAllTestTable001();
}
private void logAllTestTable001() {
for (TestTable001 tt: mRoomDB.useTestTable001().getAllTestTable001()) {
Log.d(
"TTINFO",
"ID = " + tt.getId() +
" Name = " + tt.getName() +
" Date1 = " + tt.getDt1() +
" Date2 = " + tt.getDt2() +
" Date3 = " + tt.getDt3());
}
}
}
The result is :-
2019-12-14 03:18:32.569 D/TTINFO: ID = 1 Name = tt01 Date1 = null Date2 = null Date3 = null
2019-12-14 03:18:32.569 D/TTINFO: ID = 2 Name = tt02 Date1 = 2019-12-13 16:18:32 Date2 = 2019-12-13 16:18:32 Date3 = 1576253912

Found it. Did not read the manual.
You have to create a 2nd class without the auto-set fields to insert.
public class NameAndDescription {
String name;
String description
}
I think, this is not a good idea.
If you have an autoincrement field in the DB it will get an automatically updated value when you pass 0.
Likewise the default value of the timestamp should be used when passing null or "".

I found the best solution was creating an abstract Dao that implemented the insert and update methods. I didn't get the default value to work (perhaps I was doing something wrong). Take a look at my answer here: How to implement created_at and updated_at column using Room Persistence ORM tools in android

Related

room/SQLite background scheduler/trigger that runs automatically

I've got a sample table:
I have a Sample table which has a "creationDate" as an atribute. What I want is a way to increment(update) the "numOfTimesUpdated" attribute each 24h since the creationdate. so lets say "creationdate" is 01.01.2021 12:12 AM => numOfTimesUpdated=0, 02.01.2021 12:12 AM => numOfTimesUpdated=1, 03.01.2021 12:12 AM => numOfTimesUpdated=3.
How can I implement something like this in the best way?
Does SQLite has some kind of background scheduler/trigger where a UPDATE Query gets automatically called? Or Is my only chance the client side(application) using smth. like an ApplicationManager?
How can I implement something like this in the best way?
You don't appear to even need a numberOfTimesUpdated column as the number of days since the creationDate can be calculated when required.
If the date/time were stored in a supported format (e.g. YYYY-MM-DD HH:MM) it is very simple.
For example consider this :-
DROP TABLE IF EXISTS table1;
CREATE TABLE IF NOT EXISTS table1 (id INTEGER PRIMARY KEY, name TEXT, creationdate TEXT);
INSERT INTO table1 VALUES
(null,'myname','2021-01-02'),(null,'anothername','2021-03-03'),(null,'andanother','2021-06-06')
;
SELECT *,strftime('%s','now')/(60 * 60 * 24) - strftime('%s',creationdate)/(60 * 60 * 24) AS numOfTimesUpdated FROM table1;
DROP TABLE IF EXISTS table1;
It :-
Drops the table if it exists
Creates the table
Inserts 3 rows with creation dates (21st Jan 2021, 3rd March 2021 and 6th June 2021)
Extracts all of the rows PLUS a calculated column with the number of days since the creation date.
Cleans up the test environment by deleting the table.
The results as run on 13th June 2021 are :-
Does SQLite has some kind of background scheduler/trigger where a UPDATE Query gets automatically called?
not time based.
Or Is my only chance the client side(application) using smth. like an ApplicationManager?
Yes, but again you don't appear to need this.
Working Room Example
The following is a working room example that implements the SQLite example above: -
The Table1 Entity :-
#Entity(tableName = "table1")
public class Table1 {
#PrimaryKey
Long id;
String name;
String creationDate;
public Table1(){}
#Ignore
public Table1(String name, String creationDate) {
this.name = name;
this.creationDate = creationDate;
}
}
Note as in theory id's can be long long instead of int has been used. As long MUST have a value Long has been used to allow autogenerated id's (without the inefficient AUTOGENERATE).
A POJO Table1WithNumberOfUpdates to get the Table1 with the additional calculated column:-
public class Table1WithNumberOfUpdates {
#Embedded
Table1 table1;
int numOfTimesUpdated;
}
A Dao AllDao to allow inserts and extracting a List of Table1WithNumberOfUpdates objects :-
#Dao
interface AllDao {
#Insert
long insert(Table1 table1);
#Query("SELECT *, strftime('%s','now')/(60 * 60 * 24) - strftime('%s',creationdate)/(60 * 60 * 24) AS numOfTimesUpdated FROM table1")
List<Table1WithNumberOfUpdates> getTable1WithNumberOfUpdatesList();
}
A standard #Database that returns an instance of the Database :-
#Database(entities = {Table1.class},exportSchema = false,version = 1)
abstract class TheDatabase extends RoomDatabase {
abstract AllDao getAllDao();
private static volatile TheDatabase instance;
public static TheDatabase getInstance(Context context) {
if (instance == null) {
instance = Room.databaseBuilder(
context,
TheDatabase.class,
"state.db"
)
.allowMainThreadQueries()
.build();
}
return instance;
}
}
And finally some code in an Activity to add the three rows and extract the result outputting it to the log :-
public class MainActivity extends AppCompatActivity {
TheDatabase db;
AllDao dao;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
//Instantiate Database and get dao
db = TheDatabase.getInstance(this);
dao = db.getAllDao();
dao.insert(new Table1("myname","2021-01-02"));
dao.insert(new Table1("anothername","2021-03-03"));
dao.insert(new Table1("andanothername","2021-06-06"));
for (Table1WithNumberOfUpdates t: dao.getTable1WithNumberOfUpdatesList()) {
Log.d("TABLE1INFO","Name is " + t.table1.name + " Created = " + t.table1.creationDate + " Updates Since Creation = " + t.numOfTimesUpdated);
}
}
}
Result :-
2021-06-14 10:17:44.498 D/TABLE1INFO: Name is myname Created = 2021-01-02 Updates Since Creation = 163
2021-06-14 10:17:44.499 D/TABLE1INFO: Name is anothername Created = 2021-03-03 Updates Since Creation = 103
2021-06-14 10:17:44.499 D/TABLE1INFO: Name is andanothername Created = 2021-06-06 Updates Since Creation = 8
Days are 1 extra due to device using local time zone as opposed to the SQLite Tool used above (Navicat for SQLite).

Correctly convert DateTime property with Dapper on SQLite

I'm using Dapper to insert and get objects to/from SQLite: one object have a property of type DateTime (and DateTimeOffset) that I have to persist and retrieve with milliseconds precision. I can't find a way to correctly retrieve the value because Dapper fail with:
System.FormatException : String was not recognized as a valid DateTime.
in System.DateTimeParse.ParseExactMultiple(String s, String[] formats, DateTimeFormatInfo dtfi, DateTimeStyles style)
in System.DateTime.ParseExact(String s, String[] formats, IFormatProvider provider, DateTimeStyles style)
in System.Data.SQLite.SQLiteConvert.ToDateTime(String dateText, SQLiteDateFormats format, DateTimeKind kind, String formatString)
in System.Data.SQLite.SQLite3.GetDateTime(SQLiteStatement stmt, Int32 index)
in System.Data.SQLite.SQLite3.GetValue(SQLiteStatement stmt, SQLiteConnectionFlags flags, Int32 index, SQLiteType typ)
in System.Data.SQLite.SQLiteDataReader.GetValue(Int32 i)
in System.Data.SQLite.SQLiteDataReader.GetValues(Object[] values)
in Dapper.SqlMapper.<>c__DisplayClass5d.<GetDapperRowDeserializer>b__5c(IDataReader r) in SqlMapper.cs: line 2587
in Dapper.SqlMapper.<QueryImpl>d__11`1.MoveNext() in SqlMapper.cs: line 1572
in System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
in System.Linq.Enumerable.ToList(IEnumerable`1 source)
in Dapper.SqlMapper.Query(IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Boolean buffered, Nullable`1 commandTimeout, Nullable`1 commandType) in SqlMapper.cs: line 1443
in Dapper.SqlMapper.Query(IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Boolean buffered, Nullable`1 commandTimeout, Nullable`1 commandType) in SqlMapper.cs: line 1382
What do I have to try? Column is of type DATETIME.
Do I have to create a custom TypeHandler and convert DateTime to and from a SQLite string in format "o"?
Dapper version 1.38
I know it's old, but I have found the solution.
After a lot of digging and analyzing Dapper code I came up with this (notice that this is 2019 year):
First you will have to create date time handler:
public class DateTimeHandler : SqlMapper.TypeHandler<DateTimeOffset>
{
private readonly TimeZoneInfo databaseTimeZone = TimeZoneInfo.Local;
public static readonly DateTimeHandler Default = new DateTimeHandler();
public DateTimeHandler()
{
}
public override DateTimeOffset Parse(object value)
{
DateTime storedDateTime;
if (value == null)
storedDateTime = DateTime.MinValue;
else
storedDateTime = (DateTime)value;
if (storedDateTime.ToUniversalTime() <= DateTimeOffset.MinValue.UtcDateTime)
return DateTimeOffset.MinValue;
else
return new DateTimeOffset(storedDateTime, databaseTimeZone.BaseUtcOffset);
}
public override void SetValue(IDbDataParameter parameter, DateTimeOffset value)
{
DateTime paramVal = value.ToOffset(this.databaseTimeZone.BaseUtcOffset).DateTime;
parameter.Value = paramVal;
}
}
Now, notice that Dapper translates .Net's type DateTimeOffset to dbType - DateTimeOffset. You need to remove this mapping and add your own like this:
SqlMapper.RemoveTypeMap(typeof(DateTimeOffset));
SqlMapper.AddTypeHandler(DateTimeHandler.Default);
That's all. Now everytime Dapper will see DateTimeOffset property in your model, it will run your DateTimeHandler to manage this.
I have found that custom TypeHandler for base types can't be used because of default typeMap that is choosen before looking for TypeHandler.
I have opened an issue dapper-dot-net but in the mean time I have solved replacing via reflection the default typeMap with a new one like the previous minus the four key DateTime, DateTime?, DateTimeOffset, DateTimeOffset?
I've made a slight modification to Adam Jachocki's solution as it didn't work for me. I am storing a date as TEXT in Sqlite and Dapper was giving me a string instead of a DateTime as the object value to parse. Apparently, Sqlite stores datetime values using three different data types: INTEGER (unix epoch), TEXT (ISO 8601 YYYY-MM-DD HH:MM:SS.SSS), and REAL ("number of days since noon in Greenwich on November 24, 4741 B.C."). That last one is really out there, so it isn't supported in the code below.
See the sqlite docs and this page for more info.
Below is my implementation of the DateTimeOffset TypeHandler. The rest of Adam's solution remains the same.
internal class DateTimeOffsetHandler : SqlMapper.TypeHandler<DateTimeOffset>
{
private static readonly TimeZoneInfo databaseTimeZone = TimeZoneInfo.Local;
private static readonly DateTime unixOrigin = new DateTime(1970, 1, 1, 0, 0, 0, 0);
public static DateTimeOffsetHandler Default { get; } = new DateTimeOffsetHandler();
public DateTimeOffsetHandler() {}
public override DateTimeOffset Parse(object value)
{
if (!TryGetDateTime(value, out DateTime storedDateValue))
{
throw new InvalidOperationException($"Unable to parse value {value} as DateTimeOffset");
}
if (storedDateValue.ToUniversalTime() <= DateTimeOffset.MinValue.UtcDateTime)
{
return DateTimeOffset.MinValue;
}
else
{
return new DateTimeOffset(storedDateValue, databaseTimeZone.BaseUtcOffset);
}
}
public override void SetValue(IDbDataParameter parameter, DateTimeOffset value)
{
DateTime paramVal = value.ToOffset(databaseTimeZone.BaseUtcOffset).DateTime;
parameter.Value = paramVal;
}
private bool TryGetDateTime(object value, out DateTime dateTimeValue)
{
dateTimeValue = default;
if (value is DateTime d)
{
dateTimeValue = d;
return true;
}
if (value is string v)
{
dateTimeValue = DateTime.Parse(v);
return true;
}
if (long.TryParse(value?.ToString() ?? string.Empty, out long l))
{
dateTimeValue = unixOrigin.AddSeconds(l);
return true;
}
if (float.TryParse(value?.ToString() ?? string.Empty, out float f))
{
throw new InvalidOperationException("Unsupported Sqlite datetime type, REAL.");
}
return false;
}
}

Dapper and Downward Integer Conversion

I am checking out v1.25 of Dapper with Sqlite via System.Data.Sqlite. If I run this query:
var rowCount = dbc.Query<int>("SELECT COUNT(*) AS RowCount FROM Data").Single();
I get the following error: System.InvalidCastException: Specified cast is not valid
This is because Sqlite returns the above value as an Int64, which I can verify with the following code. This will throw "Int64":
var row = dbc.Query("SELECT COUNT(*) AS RowCount FROM Data").Single();
Type t = row.RowCount.GetType();
throw new System.Exception(t.FullName);
Now, the following code will actually handle the downward conversion from Int64 to Int32:
public class QuerySummary
{
public int RecordCount { get; set; }
}
var qs = dbc.Query<QuerySummary>("SELECT COUNT(*) AS RecordCount FROM Data").Single();
rowCount = qs.RecordCount;
throw new System.Exception(rowCount.ToString());
When I throw this exception, it gives me the actual row count, indicating that Dapper handled the conversion for me.
My question is, why is it that dbc.Query<int> does not handle the downward conversion in a similar way to dbc.Query<QuerySummary>? Is this intended behavior?
No, that is not intentional. I've committed and pushed changes to github which make the following pass (it fails on 1.25); it should appear on NuGet at some point soon too:
// http://stackoverflow.com/q/23696254/23354
public void DownwardIntegerConversion()
{
const string sql = "select cast(42 as bigint) as Value";
int i = connection.Query<HasInt32>(sql).Single().Value;
Assert.IsEqualTo(42, i);
i = connection.Query<int>(sql).Single();
Assert.IsEqualTo(42, i);
}

dynamo db query

I have my dynamo db table as follows:
HashKey(xyz) ,RangeKey(timestamp)
Now for each hash key i have set of range key.
Now i want to query based on set of hashkey and i want only most recent value correspoding to that to be fetched .I dont want to do in memory sorting and then picking most recent version.
Can i do this in any way?
Use case is that i will do a bulkget and pass set of hashkey (say 100) , so i want to get one record for each hashkey
You (currently) can't set constraints on a batch get. See http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API_BatchGetItems.html
However, for single hash keys, you can set the direction using ScanIndexForward. See http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API_Query.html for information.
Sample java code:
new QueryRequest().withTableName("table-name")
.withScanIndexForward(false)
.withLimit(1)
.withHashKeyValue(new AttributeValue().withS("hash-key"));
It will not be very efficient though, as you will need to make this call 100 times.
Use ScanIndexForward(true for ascending and false for descending) and can also limit the result using setLimit value of Query Expression.
Please find below the code where used QueryPage for finding the single record.
public EventLogEntitySave fetchLatestEvents(String id) {
EventLogEntitySave entity = new EventLogEntitySave();
entity.setId(id);
DynamoDBQueryExpression<EventLogEntitySave> queryExpression = new DynamoDBQueryExpression<EventLogEntitySave>().withHashKeyValues(entity);
queryExpression.setScanIndexForward(false);
queryExpression.withLimit(1);
queryExpression.setLimit(1);
List<EventLogEntitySave> result = dynamoDBMapper.queryPage(EventLogEntitySave.class, queryExpression).getResults();
System.out.println("size of records = "+result.size() );
result.get(0);
}
#DynamoDBTable(tableName = "PROD_EA_Test")
public class EventLogEntitySave {
#DynamoDBHashKey
private String id;
private String reconciliationProcessId;
private String vin;
private String source;
}
public class DynamoDBConfig {
#Bean
public AmazonDynamoDB amazonDynamoDB() {
String accesskey = "";
String secretkey = "";
//
// creating dynamo client
BasicAWSCredentials credentials = new BasicAWSCredentials(accesskey, secretkey);
AmazonDynamoDB dynamo = new AmazonDynamoDBClient(credentials);
dynamo.setRegion(Region.getRegion(Regions.US_WEST_2));
return dynamo;
}

Linq2EF: Return DateTime with a TimeZone Offset

Our database has all times stored as UTC, and we know the user's current timezone, so want to return it relative to that. So we want to incorporate the offset in a LINQ projection as so:
var users = from u in this.Context.Users
select new UserWithCorrectedDate
{
Id = u.Id,
FirstName = u.FirstName,
LastName = u.LastName,
RegistrationDate = u.RegistrationDate.Value.AddHours(-5)
};
Of course, Linq2EF cannot convert "AddHours" into a canonical function. Is there another way to do this?
UPDATE:
Another thought, if the timezone offset was stored in the database as another column, would there be a way to have the DB perform the calculation (date + offset)?
Linq to Entities supports SqlFunctions. If you know the offset number of hours you could do something like this:
var users = from u in this.Context.Users
select new UserWithCorrectedDate
{
Id = u.Id,
FirstName = u.FirstName,
LastName = u.LastName,
RegistrationDate = SqlFunctions.DateAdd("hour", -5, u.RegistrationDate.Value)
};
If you want to be more precise by using the Timezone function, you could do something like this (this is a bit of a workaround):
public static DateTime UtcToNewYorkTime(DateTime utcDateTime)
{
TimeZoneInfo tzi = TimeZoneInfo.FindSystemTimeZoneById("Eastern Standard Time");
DateTime converted = TimeZoneInfo.ConvertTimeFromUtc(utcDateTime, tzi);
return converted;
}
And then in your POCO UserWithCorrectedDate object, have a get only property for the Timezoned date ie.
public class UserWithCorrectDate
{
public DateTime UTCDate {get;set;}
public DateTime NYDate
{
get
{
return Utilities.UtcToNewYorkTime(this.UTCDate);
}
}
}
The quick and dirty way to do this is to convert to a list, and just linq to object to get it done:
from u in this.Context.Users.ToList()
select new { ... }

Resources