I set up a SQLite db with the same schema as my existing SQL server db and noted the following...
SQLite field names (and presumably everything else) are case sensitive.
MicroLite's SqlBuilder appears to insert the prefix 'dbo.' before the table name, which SQLite doesn't like...
This query works...
query = new SqlQuery("SELECT [ClubID], [Name] FROM [Clubs] WHERE [ClubID] = #p0", 3);
clubs = session.Fetch<MicroLiteClub>(query);
This one doesn't...
query = SqlBuilder.Select("*")
.From(typeof(MicroLiteClub))
.Where("ClubID = #p0", 3)
.OrWhere("ClubID = #p1", 22)
.OrderByDescending("Name")
.ToSqlQuery();
clubs = session.Fetch<MicroLiteClub>(query);
MicroLite logged: "no such table: dbo.Clubs"
This is happening because SQLite doesn't support table schemas like MS SQL Server does.
In the hand crafted query, you are not specifying a schame for the table FROM [Clubs] however in your mapping attribute you will have specified dbo as the schema like this:
[Table(schema: "dbo", name: "Clubs")]
The SqlBuilder doesn't know what SQL Dialect is in use so if a schame is present on the table mapping, it will be used. This means that it would generate FROM [dbo].[Clubs]. To rectify this, simply remove the schema value on the TableAttribute as is optional from MicroLite 2.1 onwards.
On a side note, MicroLite 2.1 introduced support for In in the SqlBuilder fluent API so you could change:
.Where("ClubID = #p0", 3)
.OrWhere("ClubID = #p1", 22)
to
.Where("ClubID").In(3, 22)
Related
In cosmosDB, I need to insert a large amount of data in a new container.
create_table_sql = f"""
CREATE TABLE IF NOT EXISTS cosmosCatalog.`{cosmosDatabaseName}`.{cosmosContainerName}
USING cosmos.oltp
OPTIONS(spark.cosmos.database = '{cosmosDatabaseName}')
TBLPROPERTIES(partitionKeyPath = '/id', manualThroughput = '10000', indexingPolicy = 'AllProperties', defaultTtlInSeconds = '-1');
"""
spark.sql(create_table_sql)
# Read data with spark
data = (
spark.read.format("csv")
.options(header="True", inferSchema="True", delimiter=";")
.load(spark_file_path)
)
cfg = {
"spark.cosmos.accountEndpoint": "https://XXXXXXXXXX.documents.azure.com:443/",
"spark.cosmos.accountKey": "XXXXXXXXXXXXXXXXXXXXXX",
"spark.cosmos.database": cosmosDatabaseName,
"spark.cosmos.container": cosmosContainerName,
}
data.write.format("cosmos.oltp").options(**cfg).mode("APPEND").save()
Then after this insert I would like to change the Manual Throughput of this container.
alter_table = f"""
ALTER TABLE cosmosCatalog.`{cosmosDatabaseName}`.{cosmosContainerName}
SET TBLPROPERTIES( manualThroughput = '400');
"""
spark.sql(alter_table)
Py4JJavaError: An error occurred while calling o342.sql.
: java.lang.UnsupportedOperationException
I find no documentation online on how to change TBLPROPERTIES for a cosmosdb table in sparkSQL. I know I can edit it on the Azure Portal and also with azure cli, but I would like to keep it in sparkSQL.
This is not supported with the spark connector for NOSQL API , you might need to track the issue here. So you might need to do it through CLI command or portal or SDK (Java)
FYI : Cosmos NOSQL API container is not similar to Table in SQL, so alter commands will not work.
I have created a CDS views as follows:
define view YGAC_I_REQUEST_ROLE
with parameters
pm_req_id : grfn_guid,
#Consumption.defaultValue: 'ROL'
pm_item_type : grac_prov_item_type,
#Consumption.defaultValue: 'AP'
pm_approval : grac_approval_status
as select from YGAC_I_REQ_PROVISION_ITEM as provitem
association [1..1] to YGAC_I_ROLE as _Role on _Role.RoleId = provitem.ProvisionItemId
association [1..*] to YGAC_I_ROLE_RS as _Relation on _Relation.RoleId1 = provitem.ProvisionItemId
{
key ReqId,
key ReqIdItem,
Connector,
ProvisionItemId,
ActionType,
ValidFrom,
ValidTo,
_Role.RoleId,
_Role.RoleName,
_Role.RoleType,
_Role,
_Relation
}
where
ReqId = $parameters.pm_req_id
and ProvisionItemType = $parameters.pm_item_type
and ApprovalStatus = $parameters.pm_approval
Then I have consumed in ABAP:
SELECT
FROM ygac_i_request_role( pm_req_id = #lv_test,
pm_item_type = #lv_item_type,
pm_approval = #lv_approval
)
FIELDS reqid,
connector,
provisionitemid
INTO TABLE #DATA(lt_result).
How to get the list of _Relation according to selection above.
This is generally not possible like in ABAP SQL queries:
SELECT m~*, kt~*
FROM mara AS m
JOIN makt AS kt
...
This contradicts the whole idea of CDS associations, because they were created to join on-demand and to reduce redundant calls to database. Fetching all fields negates the whole idea of "lazy join".
However, there is another syntax in FROM clause which is enabled by path expressions that allows querying underlining associations both fully and by separate elements. Here is how
SELECT *
FROM ygac_i_request_role( pm_req_id = #lv_test )
\_Role AS role
INTO TABLE #DATA(lt_result).
This fetches all the fields of _Role association into internal table.
Note: remember, it is not possible to fetch all the published associations of current view simultaneously, only one path per query.
Possible workaround is to use JOIN
SELECT *
FROM ygac_i_request_role AS main
JOIN ygac_i_request_role
\_Role AS role
ON main~ProvisionItemId = role~RoleId
JOIN ygac_i_request_role
\_Relation AS relation
ON main~ProvisionItemId = relation~RoleId1
INTO TABLE #DATA(lt_table).
This creates deep-structured type with dedicated structure for every join association like this:
If you are not comfortable with such structure for you task, lt_table should be declared statically to put up all the fields in a flat way
TYPES BEGIN OF ty_table.
INCLUDE TYPE ygac_i_request_role.
INCLUDE TYPE ygac_i_role.
INCLUDE TYPE ygac_i_role_rs.
TYPES END OF ty_table.
I'm having some strange feeling abour sqlite3 parameters that I would like to expose to you.
This is my query and the fail message :
#query
'SELECT id FROM ? WHERE key = ? AND (userid = '0' OR userid = ?) ORDER BY userid DESC LIMIT 1;'
#error message, fails when calling sqlite3_prepare()
error: 'near "?": syntax error'
In my code it looks like:
// Query is a helper class, at creation it does an sqlite3_preprare()
Query q("SELECT id FROM ? WHERE key = ? AND (userid = 0 OR userid = ?) ORDER BY userid DESC LIMIT 1;");
// bind arguments
q.bindString(1, _db_name.c_str() ); // class member, the table name
q.bindString(2, key.c_str()); // function argument (std::string)
q.bindInt (3, currentID); // function argument (int)
q.execute();
I have the feeling that I can't use sqlite parameters for the table name, but I can't find the confirmation in the Sqlite3 C API.
Do you know what's wrong with my query?
Do I have to pre-process my SQL statement to include the table name before preparing the query?
Ooookay, should have looked more thoroughly on SO.
Answers:
- SQLite Parameters - Not allowing tablename as parameter
- Variable table name in sqlite
They are meant for Python, but I guess the same applies for C++.
tl;dr:
You can't pass the table name as a parameter.
If anyone have a link in the SQLite documentation where I have the confirmation of this, I'll gladly accept the answer.
I know this is super old already but since your query is just a string you can always append the table name like this in C++:
std::string queryString = "SELECT id FROM " + std::string(_db_name);
or in objective-C:
[#"SELECT id FROM " stringByAppendingString:_db_name];
In SQLite I can run the following query to get a list of columns in a table:
PRAGMA table_info(myTable)
This gives me the columns but no information about what the primary keys may be. Additionally, I can run the following two queries for finding indexes and foreign keys:
PRAGMA index_list(myTable)
PRAGMA foreign_key_list(myTable)
But I cannot seem to figure out how to view the primary keys. Does anyone know how I can go about doing this?
Note: I also know that I can do:
select * from sqlite_master where type = 'table' and name ='myTable';
And it will give the the create table statement which shows the primary keys. But I am looking for a way to do this without parsing the create statement.
The table_info DOES give you a column named pk (last one) indicating if it is a primary key (if so the index of it in the key) or not (zero).
To clarify, from the documentation:
The "pk" column in the result set is zero for columns that are not
part of the primary key, and is the index of the column in the primary
key for columns that are part of the primary key.
Hopefully this helps someone:
After some research and pain the command that worked for me to find the primary key column name was:
SELECT l.name FROM pragma_table_info("Table_Name") as l WHERE l.pk = 1;
For the ones trying to retrieve a pk name in android, and while using the ROOM library.
#Oogway101's answer was throwing an error: "no such column [your_table_name] ... etc.. etc...
my way of query submition was:
String pkSearch = "SELECT l.name FROM pragma_table_info(" + tableName + ") as l WHERE l.pk = 1;";
database.query(new SimpleSQLiteQuery(pkSearch)
I tried using the (") quotations and still error.
String pkSearch = "SELECT l.name FROM pragma_table_info(\"" + tableName + "\") as l WHERE l.pk = 1;";
So my solution was this:
String pragmaInfo = "PRAGMA table_info(" + tableName + ");";
Cursor c = database.query(new SimpleSQLiteQuery(pragmaInfo));
String id = null;
c.moveToFirst();
do {
if (c.getInt(5) == 1) {
id = c.getString(1);
}
} while (c.moveToNext() && id == null);
Log.println(Log.ASSERT, TAG, "AbstractDao: pk is: " + id);
The explanation is that:
A) PRAGMA table_info returns a cursor with various indices, the response is atleast of length 6... didnt check more...
B) index 1 has the column name.
C) index 5 has the "pk" value, either 0 if it is not a primary key, or 1 if its a pk.
You can define more than one pk so this will not bring an accurate result if your table has more than one (IMHO more than one is bad design and balloons the complexity of the database beyond human comprehension).
So how will this fit into the #Dao? (you may ask...)
When making the Dao "abstract" you have access to a default constructor which has the database in it:
from the docummentation:
An abstract #Dao class can optionally have a constructor that takes a Database as its only parameter.
this is the constructor that will grant you access to the query.
There is a catch though...
You may use the Dao during a database creation with the .addCallback() method:
instance = Room.databaseBuilder(context.getApplicationContext(),
AppDatabase2.class, "database")
.addCallback(
//You may use the Daos here.
)
.build();
If you run a query in the constructor of the Dao, the database will enter a feedback loop of infinite instantiation.
This means that the query MUST be used LAZILY (just at the moment the user needs something), and because the value will never change, it can be stored. and never re-queried.
I need to get column names and their tables in a SQLite database. What I need is a resultset with 2 columns: table_name | column_name.
In MySQL, I'm able to get this information with a SQL query on database INFORMATION_SCHEMA. However the SQLite offers table sqlite_master:
sqlite> create table students (id INTEGER, name TEXT);
sqlite> select * from sqlite_master;
table|students|students|2|CREATE TABLE students (id INTEGER, name TEXT)
which results a DDL construction query (CREATE TABLE) which is not helpful for me and I need to parse this to get relevant information.
I need to get list of tables and join them with columns or just get columns along with table name column. So PRAGMA table_info(TABLENAME) is not working for me since I don't have table name. I want to get all column metadata in the database.
Is there a better way to get that information as a result set by querying database?
You've basically named the solution in your question.
To get a list of tables (and views), query sqlite_master as in
SELECT name, sql FROM sqlite_master
WHERE type='table'
ORDER BY name;
(see the SQLite FAQ)
To get information about the columns in a specific table, use PRAGMA table_info(table-name); as explained in the SQLite PRAGMA documentation.
I don't know of any way to get tablename|columnname returned as the result of a single query. I don't believe SQLite supports this. Your best bet is probably to use the two methods together to return the information you're looking for - first get the list of tables using sqlite_master, then loop through them to get their columns using PRAGMA table_info().
Recent versions of SQLite allow you to select against PRAGMA results now, which makes this easy:
SELECT
m.name as table_name,
p.name as column_name
FROM
sqlite_master AS m
JOIN
pragma_table_info(m.name) AS p
ORDER BY
m.name,
p.cid
where p.cid holds the column order of the CREATE TABLE statement, zero-indexed.
David Garoutte answered this here, but this SQL should execute faster, and columns are ordered by the schema, not alphabetically.
Note that table_info also contains
type (the datatype, like integer or text),
notnull (1 if the column has a NOT NULL constraint)
dflt_value (NULL if no default value)
pk (1 if the column is the table's primary key, else 0)
RTFM: https://www.sqlite.org/pragma.html#pragma_table_info
There are ".tables" and ".schema [table_name]" commands which give kind of a separated version to the result you get from "select * from sqlite_master;"
There is also "pragma table_info([table_name]);" command to get a better result for parsing instead of a construction query:
sqlite> .tables
students
sqlite> .schema students
create table students(id INTEGER, name TEXT);
sqlite> pragma table_info(students);
0|id|INTEGER|0||0
1|name|TEXT|0||0
Hope, it helps to some extent...
Another useful trick is to first get all the table names from sqlite_master.
Then for each one, fire off a query "select * from t where 1 = 0". If you analyze the structure of the resulting query - depends on what language/api you're calling it from - you get a rich structure describing the columns.
In python
c = ...db.cursor()
c.execute("select * from t where 1=0");
c.fetchall();
print c.description;
Juraj
PS. I'm in the habit of using 'where 1=0' because the record limiting syntax seems to vary from db to db. Furthermore, a good database will optimize out this always-false clause.
The same effect, in SQLite, is achieved with 'limit 0'.
FYI, if you're using .Net you can use the DbConnection.GetSchema method to retrieve information that usually is in INFORMATION_SCHEMA. If you have an abstraction layer you can have the same code for all types of databases (NOTE that MySQL seems to swich the 1st 2 arguments of the restrictions array).
Try this sqlite table schema parser, I implemented the sqlite table parser for parsing the table definitions in PHP.
It returns the full definitions (unique, primary key, type, precision, not null, references, table constraints... etc)
https://github.com/maghead/sqlite-parser
The syntax follows sqlite create table statement syntax: http://www.sqlite.org/lang_createtable.html
This is an old question but because of the number of times it has been viewed we are adding to the question for the simple reason most of the answers tell you how to find the TABLE names in the SQLite Database
WHAT DO YOU DO WHEN THE TABLE NAME IS NOT IN THE DATABASE ?
This is happening to our app because we are creating TABLES programmatically
So the code below will deal with the issue when the TABLE is NOT in or created by the Database Enjoy
public void toPageTwo(View view){
if(etQuizTable.getText().toString().equals("")){
Toast.makeText(getApplicationContext(), "Enter Table Name\n\n"
+" OR"+"\n\nMake Table First", Toast.LENGTH_LONG
).show();
etQuizTable.requestFocus();
return;
}
NEW_TABLE = etQuizTable.getText().toString().trim();
db = dbHelper.getWritableDatabase();
ArrayList<String> arrTblNames = new ArrayList<>();
Cursor c = db.rawQuery("SELECT name FROM sqlite_master WHERE
type='table'", null);
if (c.moveToFirst()) {
while ( !c.isAfterLast() ) {
arrTblNames.add( c.getString( c.getColumnIndex("name")) );
c.moveToNext();
}
}
c.close();
db.close();
boolean matchFound = false;
for(int i=0;i<arrTblNames.size();i++) {
if(arrTblNames.get(i).equals(NEW_TABLE)) {
Intent intent = new Intent(ManageTables.this, TableCreate.class
);
startActivity( intent );
matchFound = true;
}
}
if (!matchFound) {
Toast.makeText(getApplicationContext(), "No Such Table\n\n"
+" OR"+"\n\nMake Table First", Toast.LENGTH_LONG
).show();
etQuizTable.requestFocus();
}
}