I am trying to declare a table using SQLAlchemy. I'd like to include a BIGINT auto incrementing primary key in the table. This does not seem to work with sqlite as the DB backend. On the other hand, having INTEGER auto incrementing primary key works just fine.
I read that sqlite has ROWID that is a signed bigint. But is there a way to have a BIGINT auto increment field? This way I can swap backends without worrying about db specific issues (assuming MySQL and Postgres support bigint auto incrementing fields).
Thanks.
For others who get here via Google and just need a solution I have written the following code:
# SQLAlchemy does not map BigInt to Int by default on the sqlite dialect.
# It should, but it doesnt.
from sqlalchemy import BigInteger
from sqlalchemy.dialects import postgresql, mysql, sqlite
BigIntegerType = BigInteger()
BigIntegerType = BigIntegerType.with_variant(postgresql.BIGINT(), 'postgresql')
BigIntegerType = BigIntegerType.with_variant(mysql.BIGINT(), 'mysql')
BigIntegerType = BigIntegerType.with_variant(sqlite.INTEGER(), 'sqlite')
This will allow you to use BIGINT on a database, and INT for when you run unit tests.
Sqlite doesn't allow BIGINT used as an primary key with autoincrement.
But, due to dynamic nature of sqlite column types, you can make a backend-specific column type and use INTEGER type in case of sqlite backend, see SQLAlchemy: How to conditionally choose type for column by depending on its backend.
Hope that helps.
Related
I have to create a table with autoincrement column. I am creating an web app with sqlite as a background and sql alchemy is the orm layer.Python and Flask as front end. I am creating some department list and department id should be auto incremented.When I try to add department through UI I dont provide department id.Because department id is the primary key and should be auto incremented.I have added the department name and department jobs through UI without any error.But when I try to list the departments list I am getting error.
AttributeError: 'NoneType' object has no attribute 'department_id'
What I tried is,My sqlite
create table Departments(department_id primarykey integer autoincrement,department_name char,department_jobs char);
When I try creating this schema I am getting an error called 'syntax error near autoincrement'
I tried by using capital letter,auto_increment,auto increment.
Nothing is working
My sql alchemy looks like this
class Departments(db.Model):
"Adding the department"
department_id = db.Column(db.Integer,primary_key=True)
department_name = db.Column(db.String(50),nullable=False)
department_jobs = db.Column(db.String(40),nullable=False)
What I am expecting here is how do I do the auto incrementin sqllite and sqlalchemy so that I can use it in both frontend and backend.
You have coded PRIMARYKEY instead of PRIMARY KEY and you should also code INTEGER PRIMARY KEY as the column type should appear first.
There is no need, from your explanation, to code AUTOINCREMENT.
Not using AUTOINCREMENT will be more efficient and will as far as you are concerned do the same thing. i.e. if the value for the department_id is not supplied, then SQLite will automatically generate a value which will be 1 for the first row that is inserted and then typically 1 greater for the next row and so on (SQLite does not guarantee montonically increasing numbers).
SQLite Autoincrement which includes
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.
I'd sugggest just using :-
create table Departments(department_id INTEGER PRIMARY KEY,department_name char,department_jobs char);
I have a Xamarin.Forms app that uses a SQLite database locally on the device. Here's some sample data structure:
Table x: id, name
Table y: id, name
Table x_y: id, x_id, y_id
Since SQLite doesn't support altering columns, one of the schema updates we sent down in a patch did the following:
Rename table x to x_temp
Create new/updated table x
Insert all data from table x_temp into table x
Drop table if exists x
That seems to work just fine. However, when I'm attempting to run an insert statement on table x_y, I am getting a SQLite exception: "no such table: main.x_temp".
When I look at the SQLite query string while debugging there is no mention of table x_temp whatsoever. So, if I delete the entire database and re-create everything the insert works just fine.
I'm from a MSSQL background, am I not understanding something about SQLite in general? Is the foreign key constraint from table x_y trying to reference x_temp because I renamed the original table (I may have just answered my own question)? If that's the case, surely there is a way around this without having to cascade and re-create every table?
Any input would be appreciated. Thanks!
I believe that your issue may be related to the SQlite version in conjunction with whether or not Foreign Key Support has been turned on.
That is the likliehood is that :-
Is the foreign key constraint from table x_y trying to reference
x_temp because I renamed the original table (I may have just answered
my own question)?
Would be the issue, as you likely have Foreign Key Support turned on as per :-
Prior to version 3.26.0 (2018-12-01), FOREIGN KEY references to a table that is renamed were only edited if the PRAGMA foreign_keys=ON, or in other words if foreign key constraints were begin enforced.
With PRAGMA foreign_keys=OFF, FOREIGN KEY constraints would not be changed when the table that the foreign key referred to (the "parent table") was renamed.
Beginning with version 3.26.0, FOREIGN KEY constraints are always converted when a table is renamed, unless the PRAGMA legacy_alter_table=ON setting is engaged. The following table summaries the difference:
SQL As Understood By SQLite - ALTER TABLE
If that's the case, surely there is a way around this without having
to cascade and re-create every table?
Yes, as the latest version of SQlite on Android is 3.19.0 (I believe), then you can turn Foreign Key support off using the foreign_keys pragma when renaming the table.
Note Foreign Keys cannot be turned off within a transaction.
See SQL As Understood By SQLite - ALTER TABLE and PRAGMA foreign_keys = boolean;
I have a DB in a Rails app I'm developing that had a product_type field defined as varchar(255) (in SQLite Manager) that ended up being a foreign key. I changed the name to product_type_id and the type to integer using a migration in Rails to use the existing id field in the other product_type file. When it was complete, SQLite Manager in Firefox shows the type as integer(255) rather than just integer.
What is an integer(255)? An answer to another question here said the (255) was a display width. Is that correct? There is no data in the database at present. Should I delete and recreate the field to get an integer, otherwise the data types on the two fields won't match.
The integer(255) came about because a string field [ varchar(255) ] was converted to an integer and the DB maintained the field size so it might possibly hold the old data when converted during the change. Apparently, most DBs don't support integers that large. Some will ignore the size specification but others will error. Regardless, the result was not what I'd intended.
I found that the way to remove the limit was to perform a migration in Rails and set the limit to nil.
class RemoveIntegerLimits < ActiveRecord::Migration
def up
change_column :inv_x_refs, :company_id, :integer, limit: nil
end
end
I was psyched about the possibility of using SQLite as a database solution during development so that I could focus on writing the code first and dynamically generating the db at runtime using NHibernate's ShemaExport functionality. However, I'm running into a few issues, not the least of which is that it seems that SQLite requires me to use Int64 for my primary keys (vs, say, Int32 or Guid). Is there any way around this?
Note: I should specify that this is in the context of an app using NHibernate. It is not strictly speaking the case that one can't create a table in SQLite with an INT datatype, but the behavior when you save and retrieve the data seems to indicate that it's being stored and/or retrieved as Int64.
SQLite will let you use any field in your table as a PRIMARY KEY. Doing so will implicitly create a UNIQUE index on the field. This is then the field that you, as a developer, can consider to be the primary unique identifier for the field. It can be any supported SQLite data type (below).
SQLite will always create an implicit internal numeric identifier for every table. It will have several aliases including RowID, OID, and _ROWID_. If you create your primary key as INTEGER PRIMARY KEY then it will use the same field as your primary key and SQLite's internal numeric identifier.
SQLite doesn't have a concept of Int32 or Int64 or Guid data types. It only has four data types: INT, REAL, TEXT, and BLOB. When you run DDL against SQLite if you use anything other than these four identifiers, SQLite will use a set of rules to determine which type to use. Basically, Int32 and Int64 are treated as aliases of INT and end up doing the exact same thing.
Even once you've created the tables with the data types you mentioned for each field, all you set is the type affinity for that field. SQLite does not enforce data types. Any data can be put into any field regardless of the declared type. SQLite will use the type affinity to convert data if possible, so if you insert '123' as a text string into an INT field, it will store it as the number 123.
The only exception to the type affinity is INTEGER PRIMARY KEY FIELDS. Those must be integers.
Integers in SQLite are always stored with a variable length field. So depending on the size of the integer, you may actually get an Int32 back for some rows an Int64 for others, all within the same field. This depends on the wrapper you're using, in this case NHibernate (I guess with System.Data.SQLite).
It does not require you to use Int64, however, it is possible that it only allows that when you specify a numeric primary key. Because sqlite doesn't really have referential integrity checking (though there has been recent discussion of this and perhaps dr hipp has even implemented, i haven't checked lately), all primary key means is "Make this column unique and create an index on it". there isn't much special about it. You can certainly use varchar or text for a primary key. for example, this works:
create table t_test (
theID varchar(36) primary key,
nm varchar(50)
)
in the above you could use theID to store a guid in text form.
More info can be found here: http://www.sqlite.org/lang_createtable.html#rowid
#weenet ... per your comments, the following code works just fine.
i think you need to post your code if you're still having troubles.
create table t_test2 (
theID int32 primary key,
nm varchar(50)
);
insert into t_test2 (theID, nm) values (1, 'don');
insert into t_test2 (theID, nm) values (2, 'weenet');
select * from t_test2;
additionally, this code works fine (varchar as a primary key):
create table t_test (
theID varchar(36) primary key,
nm varchar(50)
)
insert into t_test (theID, nm) values ('abcdefg', 'don');
insert into t_test (theID, nm) values ('hijklmnop', 'weenet');
select * from t_test
SQLite3 gives you a default primary key called rowid for each table if you don't specify a primary key. However, it looks like there are some disadvantages to relying on this:
The VACUUM command may change the ROWIDs of entries in tables that do not have an explicit INTEGER PRIMARY KEY.
http://www.sqlite.org/lang_vacuum.html
I want to alter an existing SQLite3 database to use explicit primary keys rather than implicit rowid's so I have the ability to run vacuum when necessary. Can I do this without rebuilding the whole database?
You don't need to rebuild the whole database. However since SQLite doesn't support ALTER TABLE statements you need to:
create a temporary table with the correct schema
copy all data from the original table to the temp table
delete the original table
rename the temp table
I suggest you use a app such as SQLiteman to do this for you.