So what I understood after upgrading my flyway version because of some requirements is that flyway-core-2.2 introduced some validation for Flyway placeholders.
Now, the convention of placeholder syntax is ${name} uniform across most libraries. In our migration scripts, we are inserting a string in a mysql table column called stretchySql and that string holds some placeholder elements of our own which is meant to be interpreted at runtime by the application layer.
UPDATE `stretchy_parameter` SET `parameter_sql`='select r.id as report_id from stretchy_report r where r.report_category = \'${reportCategory}\'
I don't want flyway to interpret something embedded in a string as its own placeholder and throw an error. So basically, is there some way to switch off flyway placeholder validation(since we don't use it) without reverting back to an older version?
As of Flyway 3.2.1 (not sure when it was introduced) there is a new boolean setting called flyway.placeholderReplacement, which defaults to true, and you can use that to effectively disable the feature. On the API it can be accessed via the setPlaceholderReplacement(value) and isPlaceholderReplacement() methods of a Flyway object.
I know 2.2 probably didn't have that option and the accepted answer was probably the best way to solve it, but I decided this alternative could be useful to users of newer versions bumping into this page.
While you can't disable it, you can set the placeholder prefix or suffix to something that will never match. This will effectively achieve the same thing.
http://flywaydb.org/documentation/api/javadoc/org/flywaydb/core/Flyway.html#setPlaceholderPrefix(java.lang.String)
Related
I am having issues while using the filename default placeholder, in repeatable migrations. I get the error as ERROR: Failed to populate value for default placeholder: ${flyway:filename}
However, I am able to use the same placeholder in non-repeatable and other callbacks like beforeMigrate, afterMigrate etc. Also, able to use all other default placeholders like timestamp, user in both repeatable and non-repeatable migration scripts.
We face the problem only with 'filename' placeholder in repeatable migration scripts.
Is there any limitation(I believe it might not be) on using the 'filename' placeholder in repeatable migrations?
Did any one had similar issue?
or does it require any other configurations?
We are using flyway version 8.5.11 and command-line utility.
I'm adding liquibase to an existing project where tables and data already exist. I would like to know how I might limit the scope of changelogSync[SQL] to a subset of available changes.
Background
I've run liquibase generateChangeLog to capture the current state and placed this into say src/main/resources/db/changelog/changes/V2021.04.13.00.00.00__init01.yaml.
I've also added another changeset to cover some new requirements in a new file. Let's call it src/main/resources/db/changelog/changes/V2021.04.13.00.00.00__new-feature.yaml.
I've added a main changelog file src/main/resources/db/changelog/db.changelog-master.yaml with the following contents:
databaseChangeLog:
- includeAll:
path: changes
relativeToChangelogFile: true
I now want to ensure that when I run liquibase changelogSync[SQL] against a particular version of the db that the scope is limited to the first changelog init01, thereby allowing from that point on a liquibase update or updateToTag et al, to continue with changes following init01.
I'm surprised to see that the changelogSync[SQL] commands don't seem to offer some way (that I can see from the docs for how to do this.
Besides printing the SQL and manually changing it, is there something I've missed? Any suggested approaches welcome. Thanks!
What about changelogSyncToTagSQL ?
Wouldn't it cover your needs?
Or maybe you could try changelogSyncSQL with additional parameters "label" or and "context" ?
changelogSyncToTagSQL
context
labels
As it stands, the only solution I've found is to generate the SQL and then manually edit its contents to filter the change sets which don't correspond to your current schema.
Then you can apply the sql to the db.
Despite setting flyway.placeholderReplacement=false I keep seeing error about no value provided for placeholder expression in sql by Flyway
ERROR: Unexpected error
org.flywaydb.core.api.FlywayException: No value provided for placeholder expressions: & conditions. Check your configuration!
at org.flywaydb.core.internal.database.oracle.pro.SQLPlusPlaceholderReplacer.replacePlaceholders(SQLPlusPlaceholderReplacer.java:132)
at org.flywaydb.core.internal.util.line.PlaceholderReplacingLine.getLine(PlaceholderReplacingLine.java:36)
at org.flywaydb.core.internal.database.ExecutableSqlScript.extractStatements(ExecutableSqlScript.java:156)
at org.flywaydb.core.internal.database.ExecutableSqlScript.(ExecutableSqlScript.java:133)
at org.flywaydb.core.internal.database.oracle.OracleSqlScript.(OracleSqlScript.java:61)
at org.flywaydb.core.internal.database.oracle.OracleDatabase.doCreateSqlScript(OracleDatabase.java:126)
at org.flywaydb.core.internal.database.Database.createSqlScript(Database.java:163)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.getSqlScript(SqlMigrationExecutor.java:96)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeInTransaction(SqlMigrationExecutor.java:109)
at org.flywaydb.core.internal.command.DbMigrate.isExecuteGroupInTransaction(DbMigrate.java:312)
at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:275)
at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244)
at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:53)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:163)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:160)
at org.flywaydb.core.internal.database.Connection$1.call(Connection.java:145)
at org.flywaydb.core.internal.util.jdbc.TransactionTemplate.execute(TransactionTemplate.java:74)
at org.flywaydb.core.internal.database.Connection.lock(Connection.java:141)
at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:150)
at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:160)
at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:138)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:947)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:910)
at org.flywaydb.core.Flyway.execute(Flyway.java:1238)
at org.flywaydb.core.Flyway.migrate(Flyway.java:910)
at org.flywaydb.commandline.Main.executeOperation(Main.java:161)
at org.flywaydb.commandline.Main.main(Main.java:108)
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Using spring boot, in application.yml.
Add the below placeholderReplacement: false
flyway:
baseline-on-migrate: false
sql-migration-prefix: V
table: migration
placeholderReplacement: false
This was happening, because i had in my .SQL migration file, HTML code with ${name}.
So it was trying to replace that! and i want it to be as is, to be inserted in the database.
Summary: in my case i want it always disabled, as i have no use of it
References: Possible Usages In Different Configuration
flyway.placeholderReplacement=false is only for Flyway placeholders, not SQL*Plus placeholders.
To disable SQL*Plus-specific placeholders, you must include SET DEFINE OFF in your script.
I think, best way to resolve this error - it's override default prefix and suffix
flyway.placeholderPrefix=$${
flyway.placeholderSuffix=}
because disabling this functionality may be unacceptably for some reasons: using flyway variables, for instance.
If your use case is similar to mine (you cannot disable placeholders and you cannot change the prefix), an alternative way is to break the placeholder value. Let's says you SQL is:
UPDATE table SET value = '${variable}';
You can avoid FlyWay picking up the placeholder by using something like:
UPDATE table SET value = '$' + '{variable}';
you can encode the whole string with UTF-8, then decode it when you need to use it
In PhpStorm, I can change global, project or directories settings with existing SQL Dialects, but is there a way to configure SQL Dialects as Symfony/DQL in PhpStorm or a way to detect that App:Panel is a valid entity, not a table? (App:Panel table name is te_panel)
I read this answer which explains that we have to add a Java plugin, because it's currently not possible to add a new SQL Dialect on PhpStorm.
As example, this is an error that PhpStorm is throwing:
The : between App and Panel is not understood. It cannot understand the table name provided (because I provide the name of the Symfony entity).
DQL is not supported.
https://youtrack.jetbrains.com/issue/WI-9948 -- watch this ticket (star/vote/comment) to get notified on any progress.
You may try and treat App:Panel as placeholder (similar to how it was described in that linked question). But I have no ideas if it will help (have not really worked with Symfony/DQL so cannot test it myself).
What I may suggest though -- threat the whole query as plain text. Yes, no syntax highlighting and stuff but will not show errors either.
How? One way by placing special comment just before the string, e.g.
->query(/** #lang text */'SELECT ...');
Or by disabling Language Injection rule for SQL altogether.
Alternatively try what has been suggested in this comment -- custom SQL detection syntax(?): https://gist.github.com/willemnviljoen/d20ad8ad0cc365a7e80744328246610f
we upgraded from HV 4.x to HV 5.4.2 and now when we have interface like following
#NotNull
List<AccountInfo> getMultiClientAccountBalances(#NotNull ClientContext clientContext, #NotNull Optional<AccountFilter> accountFilter);
I'm getting error:
javax.validation.UnexpectedTypeException: HV000186: The constraint of type 'javax.validation.constraints.NotNull' defined on 'getMultiClientAccountBalances.arg1' has multiple matching constraint validators which is due to an additional value handler of type 'org.hibernate.validator.internal.engine.valuehandling.OptionalValueUnwrapper'. It is unclear which value needs validating. Clarify configuration via #UnwrapValidatedValue.
I know it can be fixed by adding #UnwrapValidatedValue to the field, but this must be added to every method what is a lot of work for me. Is there any simpler solution (besides upgrade to HV6.x)
Unfortunately I don't see how we could change this behavior in 5.4 without breaking other use cases.
And there is no easy way to disable the optional value handler as it's added unconditionally.
So I would say you have two solutions:
a search and replace of all #Constraint Optional to add the UnwrapValidatedValue option
or move to HV 6, where we totally reworked this feature and where, I think, it should work as you expect it. I know you didn't want this answer but it is what it is...
The issue with 1. is that we removed this annotation from HV (it was experimental) in favor of a standard feature included in Bean Validation so you will have to remove it when moving to 6.
I don't know your exact environment but HV 6 is highly compatible with the previous versions so it might work very well. Just be careful about the dependencies as we changed the groupId of the artifact from org.hibernate to org.hibernate.validator. Also be aware that you need to update the validation-api from 1.1 to 2.0.
6 is already very stable and if you have any issues with it, we will fix them right away.