SQLite + Mybatis-guice throw ArrayIndexOutOfBoundsException - sqlite

I'm trying to implement a Guice module that to lets Guacamole use SQLite as a backend. The Guacamole project has a generic JDBC base module. This lets you implement modules for specific datastores with less code. Most of the lines of code end up being in mapper XML files. The project provides PostgreSQL and MySQL implementations.
I based this SQLite module off of the MySQL module. For the mapper XML files, SQLite and MySQL are similar enough that I didn't have to make any changes. However, when I try to use the SQLite module, I get this error:
### Error querying database. Cause: java.lang.ArrayIndexOutOfBoundsException: 2
### The error may exist in org/apache/guacamole/auth/jdbc/connectiongroup/ConnectionGroupMapper.xml
### The error may involve defaultParameterMap
### The error occurred while setting parameters
### SQL: SELECT guacamole_connection_group.connection_group_id, connection_group_name, parent_id, type, max_connections, max_connections_per_user, enable_session_affinity FROM guacamole_connection_group JOIN guacamole_connection_group_permission ON guacamole_connection_group_permission.connection_group_id = guacamole_connection_group.connection_group_id WHERE guacamole_connection_group.connection_group_id IN ( ? ) AND user_id = ? AND permission = 'READ'; SELECT parent_id, guacamole_connection_group.connection_group_id FROM guacamole_connection_group JOIN guacamole_connection_group_permission ON guacamole_connection_group_permission.connection_group_id = guacamole_connection_group.connection_group_id WHERE parent_id IN ( ? ) AND user_id = ? AND permission = 'READ'; SELECT parent_id, guacamole_connection.connection_id FROM guacamole_connection JOIN guacamole_connection_permission ON guacamole_connection_permission.connection_id = guacamole_connection.connection_id WHERE parent_id IN ( ? ) AND user_id = ? AND permission = 'READ';
### Cause: java.lang.ArrayIndexOutOfBoundsException: 2
It looks like the problem is that two parameters are passed to the query, but each is repeated three times. When MyBatis generates the PreparedStatement, it acts as if there are six parameters that needed to be passed in.
Here's the query it has a problem with:
<!-- Select multiple connection groups by identifier only if readable -->
<select id="selectReadable" resultMap="ConnectionGroupResultMap"
resultSets="connectionGroups,childConnectionGroups,childConnections">
SELECT
guacamole_connection_group.connection_group_id,
connection_group_name,
parent_id,
type,
max_connections,
max_connections_per_user,
enable_session_affinity
FROM guacamole_connection_group
JOIN guacamole_connection_group_permission ON guacamole_connection_group_permission.connection_group_id = guacamole_connection_group.connection_group_id
WHERE guacamole_connection_group.connection_group_id IN
<foreach collection="identifiers" item="identifier"
open="(" separator="," close=")">
#{identifier,jdbcType=VARCHAR}
</foreach>
AND user_id = #{user.objectID,jdbcType=INTEGER}
AND permission = 'READ';
SELECT parent_id, guacamole_connection_group.connection_group_id
FROM guacamole_connection_group
JOIN guacamole_connection_group_permission ON guacamole_connection_group_permission.connection_group_id = guacamole_connection_group.connection_group_id
WHERE parent_id IN
<foreach collection="identifiers" item="identifier"
open="(" separator="," close=")">
#{identifier,jdbcType=VARCHAR}
</foreach>
AND user_id = #{user.objectID,jdbcType=INTEGER}
AND permission = 'READ';
SELECT parent_id, guacamole_connection.connection_id
FROM guacamole_connection
JOIN guacamole_connection_permission ON guacamole_connection_permission.connection_id = guacamole_connection.connection_id
WHERE parent_id IN
<foreach collection="identifiers" item="identifier"
open="(" separator="," close=")">
#{identifier,jdbcType=VARCHAR}
</foreach>
AND user_id = #{user.objectID,jdbcType=INTEGER}
AND permission = 'READ';
</select>
If I manually populate the parameters, I can execute this against the SQLite database. Also, the MySQL version works fine.
What the heck is going on? What can I do to debug this? Is it a MyBatis problem or something with the JDBC connector?
If it helps, you can see the code for the module here.
Here's the method for the parameter the mapper related to this query. The full mapper classes for ConnectionGroup are here and here. The full mapper XML for my SQLite module is here.
Collection<ModelType> selectReadable(#Param("user") UserModel user,
#Param("identifiers") Collection<String> identifiers);
This is what the ConnectionGroupResultMap looks like:
<resultMap id="ConnectionGroupResultMap" type="org.apache.guacamole.auth.jdbc.connectiongroup.ConnectionGroupModel" >
<!-- Connection group properties -->
<id column="connection_group_id" property="objectID" jdbcType="INTEGER"/>
<result column="connection_group_name" property="name" jdbcType="VARCHAR"/>
<result column="parent_id" property="parentIdentifier" jdbcType="INTEGER"/>
<result column="type" property="type" jdbcType="VARCHAR"
javaType="org.apache.guacamole.net.auth.ConnectionGroup$Type"/>
<result column="max_connections" property="maxConnections" jdbcType="INTEGER"/>
<result column="max_connections_per_user" property="maxConnectionsPerUser" jdbcType="INTEGER"/>
<result column="enable_session_affinity" property="sessionAffinityEnabled" jdbcType="BOOLEAN"/>
<!-- Child connection groups -->
<collection property="connectionGroupIdentifiers" resultSet="childConnectionGroups" ofType="java.lang.String"
column="connection_group_id" foreignColumn="parent_id">
<result column="connection_group_id"/>
</collection>
<!-- Child connections -->
<collection property="connectionIdentifiers" resultSet="childConnections" ofType="java.lang.String"
column="connection_group_id" foreignColumn="parent_id">
<result column="connection_id"/>
</collection>
</resultMap>

David,
I know this is a little old, but I'm also trying to implement a SQLite JDBC module, and ran into exactly the same problem. I've managed to track down the source of the issue, and have filed an issue on the JDBC SQLite github page:
https://github.com/xerial/sqlite-jdbc/issues/277
Basically, the SQLite JDBC driver computes one of its array sizes based on the parameter count for a prepared statement. However, in the cases you mentioned, there are multiple SELECT statements which take the same parameters, so the array needs to be x * y (x = parameter count, y = number of select statements) rather than just the number of parameters. When it tries to prepare the statement, it hits the first position beyond the parameter count and generates this exception.
The other JDBC modules - MySQL, PostgreSQL, and SQL Server, as well as Oracle and H2 that I'm messing with - seem to handle this situation correctly (well, Oracle is a little...special...), but the SQLite driver does not.
I was able to work around the issue in a really, really kludgy way, by creating two different result maps, one for the generic select and one for the read that checks for READ permissions, and then break out each of the select queries into its own SELECT block and call those from the collection inside the result map. It's nowhere near as elegant as the existing code, but it works.

Related

How to get all the elements of an association?

I have created a CDS views as follows:
define view YGAC_I_REQUEST_ROLE
with parameters
pm_req_id : grfn_guid,
#Consumption.defaultValue: 'ROL'
pm_item_type : grac_prov_item_type,
#Consumption.defaultValue: 'AP'
pm_approval : grac_approval_status
as select from YGAC_I_REQ_PROVISION_ITEM as provitem
association [1..1] to YGAC_I_ROLE as _Role on _Role.RoleId = provitem.ProvisionItemId
association [1..*] to YGAC_I_ROLE_RS as _Relation on _Relation.RoleId1 = provitem.ProvisionItemId
{
key ReqId,
key ReqIdItem,
Connector,
ProvisionItemId,
ActionType,
ValidFrom,
ValidTo,
_Role.RoleId,
_Role.RoleName,
_Role.RoleType,
_Role,
_Relation
}
where
ReqId = $parameters.pm_req_id
and ProvisionItemType = $parameters.pm_item_type
and ApprovalStatus = $parameters.pm_approval
Then I have consumed in ABAP:
SELECT
FROM ygac_i_request_role( pm_req_id = #lv_test,
pm_item_type = #lv_item_type,
pm_approval = #lv_approval
)
FIELDS reqid,
connector,
provisionitemid
INTO TABLE #DATA(lt_result).
How to get the list of _Relation according to selection above.
This is generally not possible like in ABAP SQL queries:
SELECT m~*, kt~*
FROM mara AS m
JOIN makt AS kt
...
This contradicts the whole idea of CDS associations, because they were created to join on-demand and to reduce redundant calls to database. Fetching all fields negates the whole idea of "lazy join".
However, there is another syntax in FROM clause which is enabled by path expressions that allows querying underlining associations both fully and by separate elements. Here is how
SELECT *
FROM ygac_i_request_role( pm_req_id = #lv_test )
\_Role AS role
INTO TABLE #DATA(lt_result).
This fetches all the fields of _Role association into internal table.
Note: remember, it is not possible to fetch all the published associations of current view simultaneously, only one path per query.
Possible workaround is to use JOIN
SELECT *
FROM ygac_i_request_role AS main
JOIN ygac_i_request_role
\_Role AS role
ON main~ProvisionItemId = role~RoleId
JOIN ygac_i_request_role
\_Relation AS relation
ON main~ProvisionItemId = relation~RoleId1
INTO TABLE #DATA(lt_table).
This creates deep-structured type with dedicated structure for every join association like this:
If you are not comfortable with such structure for you task, lt_table should be declared statically to put up all the fields in a flat way
TYPES BEGIN OF ty_table.
INCLUDE TYPE ygac_i_request_role.
INCLUDE TYPE ygac_i_role.
INCLUDE TYPE ygac_i_role_rs.
TYPES END OF ty_table.

Creating a new table in sqlite database [duplicate]

I'm having some strange feeling abour sqlite3 parameters that I would like to expose to you.
This is my query and the fail message :
#query
'SELECT id FROM ? WHERE key = ? AND (userid = '0' OR userid = ?) ORDER BY userid DESC LIMIT 1;'
#error message, fails when calling sqlite3_prepare()
error: 'near "?": syntax error'
In my code it looks like:
// Query is a helper class, at creation it does an sqlite3_preprare()
Query q("SELECT id FROM ? WHERE key = ? AND (userid = 0 OR userid = ?) ORDER BY userid DESC LIMIT 1;");
// bind arguments
q.bindString(1, _db_name.c_str() ); // class member, the table name
q.bindString(2, key.c_str()); // function argument (std::string)
q.bindInt (3, currentID); // function argument (int)
q.execute();
I have the feeling that I can't use sqlite parameters for the table name, but I can't find the confirmation in the Sqlite3 C API.
Do you know what's wrong with my query?
Do I have to pre-process my SQL statement to include the table name before preparing the query?
Ooookay, should have looked more thoroughly on SO.
Answers:
- SQLite Parameters - Not allowing tablename as parameter
- Variable table name in sqlite
They are meant for Python, but I guess the same applies for C++.
tl;dr:
You can't pass the table name as a parameter.
If anyone have a link in the SQLite documentation where I have the confirmation of this, I'll gladly accept the answer.
I know this is super old already but since your query is just a string you can always append the table name like this in C++:
std::string queryString = "SELECT id FROM " + std::string(_db_name);
or in objective-C:
[#"SELECT id FROM " stringByAppendingString:_db_name];

ORA-00904: "Extent1"."IsModified": invalid identifier,

Generated CodeFirst classes from working EDMX. When I try to connect database I get this error. it is looking for some columns which are not in the table. also these columns not in POCO and Fluent Mapping classes.
works fine with EDMX, have issue only with Codefirst
Why below columns are added in the query? is there any properties, method which can disable these columns getting included?
so because of this I get below error.
An error occurred while executing the command definition. See the inner exception for details.
Inner Exception:ORA-00904: "Extent1"."IsModified": invalid identifier
public TestContext() :
base(GetDefaultConnection(), true)
{
Database.SetInitializer<TestContext>(null);
this.Database.Log = Console.Write;
Configure();
}
SELECT
"Extent1".USERID,
"Extent1".FIRST_NAME,
"Extent1".LAST_NAME,
"Extent1".EMAIL_ADDRESS,
"Extent1".PHONE_NO,
"Extent1".FAX_NO,
"Extent1".ID,
"Extent1"."DisableEvent",
"Extent1"."RowNumberValue",
"Extent1"."IsNew",
"Extent1"."IsModified"
FROM USERS "Extent1"
WHERE (((UPPER("Extent1".USERID)) = :p__linq__0) OR ((UPPER("Extent1".USERID) IS NULL) AND (:p__linq__0 IS NULL))) AND ROWNUM <= 1
-- p__linq__0: 'SYSADM' (Type = String, Size = 6)
-- Executing at 3/11/2015 14:25:40 +05:30
below columns not in actual table.
"Extent1"."DisableEvent",
"Extent1"."RowNumberValue",
"Extent1"."IsNew",
"Extent1"."IsModified"
FROM USERS "Extent1"
Based on what I see, I can say that those rows are for UOW(Unit Of Work) implementation.
Basically those fields are used for Paging - RowNumberValue, IsNew is used to mark an entity which was added and IsModified is for an entity pending update.
If this answer is still correct then ef6 is not supported by ODP https://stackoverflow.com/a/23307234/79379

Using MicroLite on SQLite

I set up a SQLite db with the same schema as my existing SQL server db and noted the following...
SQLite field names (and presumably everything else) are case sensitive.
MicroLite's SqlBuilder appears to insert the prefix 'dbo.' before the table name, which SQLite doesn't like...
This query works...
query = new SqlQuery("SELECT [ClubID], [Name] FROM [Clubs] WHERE [ClubID] = #p0", 3);
clubs = session.Fetch<MicroLiteClub>(query);
This one doesn't...
query = SqlBuilder.Select("*")
.From(typeof(MicroLiteClub))
.Where("ClubID = #p0", 3)
.OrWhere("ClubID = #p1", 22)
.OrderByDescending("Name")
.ToSqlQuery();
clubs = session.Fetch<MicroLiteClub>(query);
MicroLite logged: "no such table: dbo.Clubs"
This is happening because SQLite doesn't support table schemas like MS SQL Server does.
In the hand crafted query, you are not specifying a schame for the table FROM [Clubs] however in your mapping attribute you will have specified dbo as the schema like this:
[Table(schema: "dbo", name: "Clubs")]
The SqlBuilder doesn't know what SQL Dialect is in use so if a schame is present on the table mapping, it will be used. This means that it would generate FROM [dbo].[Clubs]. To rectify this, simply remove the schema value on the TableAttribute as is optional from MicroLite 2.1 onwards.
On a side note, MicroLite 2.1 introduced support for In in the SqlBuilder fluent API so you could change:
.Where("ClubID = #p0", 3)
.OrWhere("ClubID = #p1", 22)
to
.Where("ClubID").In(3, 22)

How to handle a missing feature of SQLite : disable triggers?

How to handle a missing feature of SQLite: disable triggers?
I don't have it stored the name of triggers for a specific table.
For example how can I drop all triggers?
What would you do?
So here it is 2015 and there still is no 'disable triggers' in SQLite. For a mobile Application this can be problematic--especially if it's a corporate App requiring offline functionality and local data.
An initial data load can be slowed to crawl by trigger execution even when you don't wrap each insert in an individual transaction.
I solved this issue using SQLite SQL fairly simply. I have a settings table that doesn't participate in the init load. It holds 'list' of key/value pairs. I have one key called 'fireTrigger' with a bit value of 0 or 1. Every trigger I have has an expression that selects value and if it equals 1 it fires the trigger, otherwise it doesn't.
This expression is in addition to any expressions evaluated on the data relating to the trigger. e.g.:
AND 1 = (SELECT val FROM MTSSettings WHERE key = 'fireTrigger')
In simple clean effect this allows me to disable/enable the trigger with a simple UPDATE to the settings table
I wrote a very simple extension function to set a boolean value to true or false.
And a function to retrieve this value (GetAllTriggersOn()).
With this function I can define all my triggers like:
CREATE TRIGGER tr_table1_update AFTER UPDATE ON TABLE1 WHEN GetAllTriggersOn()
BEGIN
-- ...
END
SQLite stores schema (meta) information in the built-in sqlite_master table.
To get a list of available triggers use the below query:
SELECT name FROM sqlite_master
WHERE type = 'trigger' -- AND tbl_name = 'a_table_name'
Set a flag in your database and use it in the triggers WHEN condition.
Say you want to create a trigger on the "clients" table after an insert. You have created a table "trigger_settings" with a TINYINT "triggers_on" field - this is your flag. Then you can set the field to 0 if you want to turn off the filters and to 1 when you want to turn them back on.
Then you create your filter with a WHEN condition that checks the "triggers_on" field.
For example:
CREATE TRIGGER IF NOT EXISTS log_client_data_after_insert
AFTER INSERT
ON [clients]
WHEN (SELECT triggers_on FROM trigger_settings)=1
BEGIN
your_statement
END;
Maybe you can make a stored procedures for droping and creating them. Is that good for you ?
Expanding on Nick Dandoulakis's answer, you could drop all relevant triggers and then reinstate them before the transaction's completion:
BEGIN;
SELECT name, sql FROM sqlite_master WHERE type = 'trigger' AND tbl_name = 'mytable';
-- store all results
-- for each name: DROP TRIGGER $name;
-- do normal work
-- for each sql: execute the SQL verbatim
COMMIT;
Expanding other answers this is how i'm doing it. Take into account that this is disabling all triggers for all tables in the database except some of then used by spatialite
SQLITE_FILE=/tmp/my.sqlite
# Define output sql files as variables
CREATE_TRIGGER_SQL=/tmp/create_triggers.sql
DROP_TRIGGER_SQL=/tmp/drop_triggers.sql
## Dump CREATE TRIGGER statements to a file ##
# To wrap statements in a transaction
echo -e "BEGIN;\n\n" > "${CREATE_TRIGGER_SQL}"
# `SELECT sql` does not output semicolons, so we must concatenate them
sqlite3 -bail "${SQLITE_FILE}" "SELECT sql || ';' FROM sqlite_master WHERE type = 'trigger' AND (name NOT LIKE 'gid_%' AND name NOT LIKE 'ggi_%' AND name NOT LIKE 'ggu_%' AND name NOT LIKE 'gii_%' AND name NOT LIKE 'giu_%' AND name NOT LIKE 'vwgcau_%' AND name NOT LIKE 'vtgcau_%' AND name NOT LIKE 'gcau_%' AND name NOT LIKE 'geometry_columns_%' AND name NOT LIKE 'gcfi_%' AND name NOT LIKE 'gctm_%' AND name NOT LIKE 'vtgcfi_%' AND name NOT LIKE 'vwgcfi_%' AND name NOT LIKE 'vtgcs_%' AND name NOT LIKE 'vwgc_%' AND name NOT LIKE 'vtgc_%' AND name NOT LIKE 'gcs_%');" >> "${CREATE_TRIGGER_SQL}"
echo -e "\n\nCOMMIT;" >> "${CREATE_TRIGGER_SQL}"
## Dump DROP TRIGGER statements to a file ##
echo -e "BEGIN;\n\n" > "${DROP_TRIGGER_SQL}"
sqlite3 -bail "${SQLITE_FILE}" "SELECT 'DROP TRIGGER ' || name || ';' FROM sqlite_master WHERE type = 'trigger' AND (name NOT LIKE 'gid_%' AND name NOT LIKE 'ggi_%' AND name NOT LIKE 'ggu_%' AND name NOT LIKE 'gii_%' AND name NOT LIKE 'giu_%' AND name NOT LIKE 'vwgcau_%' AND name NOT LIKE 'vtgcau_%' AND name NOT LIKE 'gcau_%' AND name NOT LIKE 'geometry_columns_%' AND name NOT LIKE 'gcfi_%' AND name NOT LIKE 'gctm_%' AND name NOT LIKE 'vtgcfi_%' AND name NOT LIKE 'vwgcfi_%' AND name NOT LIKE 'vtgcs_%' AND name NOT LIKE 'vwgc_%' AND name NOT LIKE 'vtgc_%' AND name NOT LIKE 'gcs_%');" >> "${DROP_TRIGGER_SQL}"
echo -e "\n\nCOMMIT;" >> "${DROP_TRIGGER_SQL}"
## Execute like ##
sqlite3 -bail /"${SQLITE_FILE}" < "${DROP_TRIGGER_SQL}"
# do things
sqlite3 -bail /"${SQLITE_FILE}" < "${CREATE_TRIGGER_SQL}"

Resources