when trying to migrate the following files,
drop table test2;
CREATE TABLE test2
(id INTEGER PRIMARY KEY AUTOINCREMENT,
name VARCHAR(30),
message VARCHAR(200),
timestamp TIMESTAMP(7));
I got the following result:
2019-12-07 10:54:33,129 [main] INFO migratus.core - Starting migrations
2019-12-07 10:54:33,253 [main] DEBUG migratus.migrations - Looking for migrations in #object[java.io.File 0x4f880f4a /home/jonas/Dropbox/prog/web/clojure/test2/resources/migrations]
2019-12-07 10:54:33,258 [main] INFO migratus.core - Ending migrations
It seems that it doesn't find the connection and therefore no database is created. Could that be the case?
found the solution, set the DATABASE_URL to the corresponding database:
export DATABASE_URL="jdbc:sqlite:./test2.db"
where test2 is the name of the app and sqlite is the name of the database
Related
What is the right way to backup and restore a MariaDB database that has sequence generation enabled (i.e. NOT autoincrement)? (This includes migrating to a new server.)
Is it possible to instruct the sequence generator to pick up indexing table data at a specific ID value? How?
Steps I take to create my issue
I wish to transfer an application to a new server:
Backup data on source server:
mysqldump --skip-opt --no-create-db --no-create-info --hex-blob [database-name] [...list of tables...] > data-backup.sql
On target server, create new empty database (same name)
Build/run JHipster Spring application on target server: java -jar myapp.jar (Running this application recreates/configures a new instance of the database on the target server.)
Restore data:
mysql [database-name] < data-backup.sql
All the above steps produce no errors (so far).
Problem
When I follow these steps, the database is restored (apparently perfectly). I can log in to the application and access all information. BUT when I attempt to create new entities (i.e. save something to the database), I get an ID 'Duplicate entry' error in the server logs:
2022-03-24 12:54:43.775 ERROR 11277 --- [ XNIO-1 task-1] o.h.e.jdbc.batch.internal.BatchingBatch : HHH000315: Exception executing batch [java.sql.BatchUpdateException: (conn=33) Duplicate entry '1001' for key 'PRIMARY'], SQL: insert into product (name, id) values (?, ?)
2022-03-24 12:54:43.776 WARN 11277 --- [ XNIO-1 task-1] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 1062, SQLState: 23000
2022-03-24 12:54:43.776 ERROR 11277 --- [ XNIO-1 task-1] o.h.engine.jdbc.spi.SqlExceptionHelper : (conn=33) Duplicate entry '1001' for key 'PRIMARY'
2022-03-24 12:54:43.779 ERROR 11277 --- [ XNIO-1 task-1] o.z.problem.spring.common.AdviceTraits : Internal Server Error
org.springframework.dao.DataIntegrityViolationException: could not execute batch; SQL [insert into product (name, id) values (?, ?)]; constraint [PRIMARY]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute batch
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:276)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:233)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:566)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at com.mycompany.app.web.rest.ProductResource$$EnhancerBySpringCGLIB$$84c14d6d.createProduct(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
...
Clearly my backup/restore process is not accounting properly for the sequence generator, which generates ID values that conflict with the existing data.
What I am doing wrong? What is the right process of backing up/restoring such a database?
Environment: JHipster 7.7.0 (Angular, monolithic), MariaDB 10.4, OpenJDK 16.0.2_7, OS Windows 10 Pro and openSUSE 15.2, Firefox 98.0.2 and Chrome 99.0.4844.84.
PS: I previously reported this issue here, aimed at the JHipster community, but got limited response. I think I need a MySQL/MariaDB expert opinion on this.
(Apologies in advance: I'm not a database expert. The technique I outline above has served me well for years, but previously I was dealing with AUTO_INCREMENT. This sequence generator has me baffled.)
Ok! I have solutions.
[For the sake if these notes, let's call the database: mydata. Also, in JHipster, the MariaDB sequence generator is called: sequence_generator]
Let's consider two situations:
(1) Simple migration
If you are merely migrating the application to a new server, the process is straight forward:
Step 1: On the original server backup and secure your database: mysqldump -u root -p mydata > mydata.sql
Step 2: Transfer the SQL file to the new server, along with the JHipster JAR file
Step 3: On the new server, create an empty database with the same name, and restore the data: mysql -u root -p mydata < mydata.sql
Step 4: Now launch your JHipster application, and everything should work
(2) Model modification
The assumption is that you have modified your model in some way (e.g. added properties to one or more entities). This solution is fiddly, but it works (for me).
Step 1: Backup your database, and secure it (in case something goes wrong): mysqldump -u root -p mydata > mydata.sql
Step 2: Backup and secure the original JHipster JAR that works with the original database
Step 3: Duplicate your database (schema and data) in a new table: mydata_bk
Step 4: Drop your original database, and create a new empty database
Step 5: Launch your new JHipster JAR, and give it time to create the new database schema, then stop the application
Step 6: Use a tool (DataGrip, sqlYog, etc) to compare the old (mydata_bk) and new schema (mydata), and modify the old schema to match the new schema
Step 7: Restore/copy all data from mydata_bk to mydata, EXCEPT for the tables DATABASECHANGELOG, DATABASECHANGELOGLOCK and the special sequence_generator table
Step 8: Open the mydata.sql SQL file, and at the top, after initial comments, one of the first instructions will read:
--
-- Sequence structure for `sequence_generator`
--
DROP SEQUENCE IF EXISTS `sequence_generator`;
CREATE SEQUENCE `sequence_generator` start with 2000 minvalue 1 maxvalue 9223372036854775806 increment by 50 cache 1000 nocycle ENGINE=InnoDB;
SELECT SETVAL(`sequence_generator`, 201050, 0);
The specific numbers may vary, but the broad details will be similar. In a MariaDB SQL console type/execute each of those SQL statements: DROP SEQUENCE ...;, CREATE SEQUENCE ...;, and SELECT SETVAL(...);
Step 9: Launch your JHipster application.
Hope this helps others that run into similar issues. Let me know if you have a better approach!
I use PHPUnit to test by PDO wrapper. One of my tests is an integration test that proves that the abstraction layer produces the right syntax, or binds PDOStatement in the right way, etc. For this, I use an in-memory SQLite database, and declare ext-pdo_sqlite as a dev-dependency of my project.
When Github Actions runs a build, dependencies all get installed, hinting that pdo_sqlite is present on the system. This is further confirmed by the fact that the connection succeeds. However, when I run an insert query to set up my database, I get the following error:
PDOException: SQLSTATE[HY000]: General error: 1 no such column: TRUE
The query looks like this:
INSERT INTO `persons` (`id`, `name`, `dob`, `weight`, `is_dev`, `teeth`) VALUES (NULL, 'Anton', '1987-11-03', 71.3, TRUE, 31)
Table creation is done with PDO like this:
$pdo->exec("CREATE TABLE `$tableName` (`id` int(6) PRIMARY KEY, `name` varchar(255), `dob` datetime, `weight` float(5), `is_dev` bool, `teeth` int(2))");
This only manifests on CI, and local tests pass fine - even on the same PHP version as CI.
Any ideas? Am I just missing something in the syntax? Could it be that GH Actions has only a dummy implementation of SQLite or something?
Note: This is an open source project, and you can see everything, including the failing build, in dhii/pdo-query#1.
We created a new database by coping from an older flyway managed database. (flyway version table also copied over)
Running flyway against this new database produces the following flyway error
Apr 24 19:00:19 ip-xx flyway: Unable to insert row for version '184' in Schema History table "public"."flyway_schema_history"
Apr 24 19:00:19 ip--xx flyway: -----------------------------------------------------------------------------------------------
Apr 24 19:00:19 ip--xx flyway: SQL State : 23502
Apr 24 19:00:19 ip--xx flyway: Error Code : 0
Apr 24 19:00:19 ip--xx flyway: Message : ERROR: null value in column "installed_on" violates not-null constraint
In my case, the error occurred while upgrading from flyway version 5.x to 6.4.3.
The definition of installed_on column of the flyway_schema_history table (at least in PostgreSQL) has changed from:
installed_on timestamp NOT NULL;
to:
installed_on timestamp NOT NULL DEFAULT now ()
Adding DEFAULT now() on the existing table has fixed the problem.
Which version of Flyway are you using, and has it changed when you moved database? Which database are you using?
If you could create an issue at github.com/flyway/flyway/issues with reproduction steps then the team will investigate for you.
I have aspnet zero project and trying to use it with the Postgresql. I followed the instructions and when i tried to Update-Database it gaves me error:
Failed executing DbCommand (18ms) [Parameters=[], CommandType='Text',
CommandTimeout='30']
CREATE SEQUENCE "AppSubscriptionPayments_Id_seq" START WITH 1 INCREMENT BY 1
NO MINVALUE NO MAXVALUE NO CYCLE;
ALTER TABLE "AppSubscriptionPayments" ALTER COLUMN "Id" TYPE int8;
ALTER TABLE "AppSubscriptionPayments" ALTER COLUMN "Id" SET NOT NULL;
ALTER TABLE "AppSubscriptionPayments" ALTER COLUMN "Id" SET DEFAULT
(nextval('"AppSubscriptionPayments_Id_seq"'));
ALTER SEQUENCE "AppSubscriptionPayments_Id_seq" OWNED BY
"AppSubscriptionPayments"."Id"
42P01: "AppSubscriptionPayments" object is not exists
I'm Using Postgresql 10, Visual Studio 2017 and ASPNET Zero.
Delete all existing migrations and Add-Migration from scratch.
Be sure there's not an existing database (that specified in the connection string) while you add-migrations. If you have an already created database before, migrations cannot be built correctly.
https://aspnetboilerplate.com/Pages/Documents/EF-Core-PostgreSql-Integration
One of our MariaDB users decided to copy a database via a shell script that pipes the output of a mysqldump to another table.
If you restore the mysqldump to a database that hasn't been created before, it works just fine. However, if you then drop that database, re-create it, and then try to run the script again, it throws the following error:
ERROR 1005 (HY000) at line 25: Can't create table
core_dev.addresses (errno: 121 "Duplicate key on write or update")
mysqldump: Got errno 32 on write
SHOW ENGINE INNODB STATUS yields the following insight:
2017-05-03 09:34:19 7f9929fb4b00 Error in foreign key constraint
creation for table core_dev.addresses. A foreign key constraint of
name core_dev.addresses_ibfk_2 already exists. (Note that
internally InnoDB adds 'databasename' in front of the user-defined
constraint name.) Note that InnoDB's FOREIGN KEY system tables store
constraint names as case-insensitive, with the MySQL standard
latin1_swedish_ci collation. If you create tables or databases whose
names differ only in the character case, then collisions in constraint
names can occur. Workaround: name your constraints explicitly with
unique names.
However, the core_dev database is entirely empty:
MariaDB [(none)]> SHOW TABLES IN core_dev;
Empty set (0.01 sec)
What could the problem be? I'm not able to replicate this problem on any of our other MariaDB 10.1 servers. The only difference is that this particular one is running on CentOS 6, while the others are on CentOS 7.