Despite setting flyway.placeholderReplacement=false I keep seeing error about no value provided for placeholder expression in sql by Flyway
ERROR: Unexpected error
org.flywaydb.core.api.FlywayException: No value provided for placeholder expressions: & conditions. Check your configuration!
at org.flywaydb.core.internal.database.oracle.pro.SQLPlusPlaceholderReplacer.replacePlaceholders(SQLPlusPlaceholderReplacer.java:132)
at org.flywaydb.core.internal.util.line.PlaceholderReplacingLine.getLine(PlaceholderReplacingLine.java:36)
at org.flywaydb.core.internal.database.ExecutableSqlScript.extractStatements(ExecutableSqlScript.java:156)
at org.flywaydb.core.internal.database.ExecutableSqlScript.(ExecutableSqlScript.java:133)
at org.flywaydb.core.internal.database.oracle.OracleSqlScript.(OracleSqlScript.java:61)
at org.flywaydb.core.internal.database.oracle.OracleDatabase.doCreateSqlScript(OracleDatabase.java:126)
at org.flywaydb.core.internal.database.Database.createSqlScript(Database.java:163)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.getSqlScript(SqlMigrationExecutor.java:96)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeInTransaction(SqlMigrationExecutor.java:109)
at org.flywaydb.core.internal.command.DbMigrate.isExecuteGroupInTransaction(DbMigrate.java:312)
at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:275)
at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244)
at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:53)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:163)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:160)
at org.flywaydb.core.internal.database.Connection$1.call(Connection.java:145)
at org.flywaydb.core.internal.util.jdbc.TransactionTemplate.execute(TransactionTemplate.java:74)
at org.flywaydb.core.internal.database.Connection.lock(Connection.java:141)
at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:150)
at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:160)
at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:138)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:947)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:910)
at org.flywaydb.core.Flyway.execute(Flyway.java:1238)
at org.flywaydb.core.Flyway.migrate(Flyway.java:910)
at org.flywaydb.commandline.Main.executeOperation(Main.java:161)
at org.flywaydb.commandline.Main.main(Main.java:108)
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Using spring boot, in application.yml.
Add the below placeholderReplacement: false
flyway:
baseline-on-migrate: false
sql-migration-prefix: V
table: migration
placeholderReplacement: false
This was happening, because i had in my .SQL migration file, HTML code with ${name}.
So it was trying to replace that! and i want it to be as is, to be inserted in the database.
Summary: in my case i want it always disabled, as i have no use of it
References: Possible Usages In Different Configuration
flyway.placeholderReplacement=false is only for Flyway placeholders, not SQL*Plus placeholders.
To disable SQL*Plus-specific placeholders, you must include SET DEFINE OFF in your script.
I think, best way to resolve this error - it's override default prefix and suffix
flyway.placeholderPrefix=$${
flyway.placeholderSuffix=}
because disabling this functionality may be unacceptably for some reasons: using flyway variables, for instance.
If your use case is similar to mine (you cannot disable placeholders and you cannot change the prefix), an alternative way is to break the placeholder value. Let's says you SQL is:
UPDATE table SET value = '${variable}';
You can avoid FlyWay picking up the placeholder by using something like:
UPDATE table SET value = '$' + '{variable}';
you can encode the whole string with UTF-8, then decode it when you need to use it
Related
I am having issues while using the filename default placeholder, in repeatable migrations. I get the error as ERROR: Failed to populate value for default placeholder: ${flyway:filename}
However, I am able to use the same placeholder in non-repeatable and other callbacks like beforeMigrate, afterMigrate etc. Also, able to use all other default placeholders like timestamp, user in both repeatable and non-repeatable migration scripts.
We face the problem only with 'filename' placeholder in repeatable migration scripts.
Is there any limitation(I believe it might not be) on using the 'filename' placeholder in repeatable migrations?
Did any one had similar issue?
or does it require any other configurations?
We are using flyway version 8.5.11 and command-line utility.
I'm adding liquibase to an existing project where tables and data already exist. I would like to know how I might limit the scope of changelogSync[SQL] to a subset of available changes.
Background
I've run liquibase generateChangeLog to capture the current state and placed this into say src/main/resources/db/changelog/changes/V2021.04.13.00.00.00__init01.yaml.
I've also added another changeset to cover some new requirements in a new file. Let's call it src/main/resources/db/changelog/changes/V2021.04.13.00.00.00__new-feature.yaml.
I've added a main changelog file src/main/resources/db/changelog/db.changelog-master.yaml with the following contents:
databaseChangeLog:
- includeAll:
path: changes
relativeToChangelogFile: true
I now want to ensure that when I run liquibase changelogSync[SQL] against a particular version of the db that the scope is limited to the first changelog init01, thereby allowing from that point on a liquibase update or updateToTag et al, to continue with changes following init01.
I'm surprised to see that the changelogSync[SQL] commands don't seem to offer some way (that I can see from the docs for how to do this.
Besides printing the SQL and manually changing it, is there something I've missed? Any suggested approaches welcome. Thanks!
What about changelogSyncToTagSQL ?
Wouldn't it cover your needs?
Or maybe you could try changelogSyncSQL with additional parameters "label" or and "context" ?
changelogSyncToTagSQL
context
labels
As it stands, the only solution I've found is to generate the SQL and then manually edit its contents to filter the change sets which don't correspond to your current schema.
Then you can apply the sql to the db.
I have recently been trying to upgrade flyway from v4 to v6.5.3. During the process, I faced an issue related to placeholders.
The migration fails with the following error.
ERROR: Unable to parse statement in
D:\Softwares\flyway-commandline-6.5.5-windows-x64\flyway-6.5.5\sql\V3__mysql-7.0.sql
at line 101 col 1. See
https://flywaydb.org/documentation/knownparserlimitations for more
information: No value provided for placeholder: ${'), NOW(), NOW())}.
Check your configuration! Caused by: No value provided for
placeholder: ${'), NOW(), NOW())}. Check your configuration!
SQL,
insert into `configuration`(key, value, created_date , updated_date) values('LOG_LOCATION', REPLACE('${MY_LOG_LOCATION}','#{','${'), NOW(), NOW());
To resolve the above failure, I replaced "${" with "$\{", but this is something I didn't look out for.
On debugging the flyway code, I saw it fails in parsing (during validation, method org.flywaydb.core.internal.parser.Parser.readToken()) of the SQL.
Why is it considering the 3rd argument of REPLACE function as a placeholder?
Note: The above SQL works in Flyway v4
If you read the documentation here, you can see that flyway uses a particular syntax for it's placeholders, ${somestring}. In your code, you have a ${, but it's not defining a placeholder. While Flyway is a pretty sophisticated tool, the mechanism here is simply using string matching. If you want to, you can modify your Flyway instance to use different escape characters for the placeholders. This could be handy if you have to have strings in your code that match that ${ syntax. You can read about that here. Scroll down to where you define the PlaceHolderPrefix and PlaceHolderSuffix. Change those to a different combination of values that are not used in your code and you should be fine.
In PhpStorm, I can change global, project or directories settings with existing SQL Dialects, but is there a way to configure SQL Dialects as Symfony/DQL in PhpStorm or a way to detect that App:Panel is a valid entity, not a table? (App:Panel table name is te_panel)
I read this answer which explains that we have to add a Java plugin, because it's currently not possible to add a new SQL Dialect on PhpStorm.
As example, this is an error that PhpStorm is throwing:
The : between App and Panel is not understood. It cannot understand the table name provided (because I provide the name of the Symfony entity).
DQL is not supported.
https://youtrack.jetbrains.com/issue/WI-9948 -- watch this ticket (star/vote/comment) to get notified on any progress.
You may try and treat App:Panel as placeholder (similar to how it was described in that linked question). But I have no ideas if it will help (have not really worked with Symfony/DQL so cannot test it myself).
What I may suggest though -- threat the whole query as plain text. Yes, no syntax highlighting and stuff but will not show errors either.
How? One way by placing special comment just before the string, e.g.
->query(/** #lang text */'SELECT ...');
Or by disabling Language Injection rule for SQL altogether.
Alternatively try what has been suggested in this comment -- custom SQL detection syntax(?): https://gist.github.com/willemnviljoen/d20ad8ad0cc365a7e80744328246610f
So what I understood after upgrading my flyway version because of some requirements is that flyway-core-2.2 introduced some validation for Flyway placeholders.
Now, the convention of placeholder syntax is ${name} uniform across most libraries. In our migration scripts, we are inserting a string in a mysql table column called stretchySql and that string holds some placeholder elements of our own which is meant to be interpreted at runtime by the application layer.
UPDATE `stretchy_parameter` SET `parameter_sql`='select r.id as report_id from stretchy_report r where r.report_category = \'${reportCategory}\'
I don't want flyway to interpret something embedded in a string as its own placeholder and throw an error. So basically, is there some way to switch off flyway placeholder validation(since we don't use it) without reverting back to an older version?
As of Flyway 3.2.1 (not sure when it was introduced) there is a new boolean setting called flyway.placeholderReplacement, which defaults to true, and you can use that to effectively disable the feature. On the API it can be accessed via the setPlaceholderReplacement(value) and isPlaceholderReplacement() methods of a Flyway object.
I know 2.2 probably didn't have that option and the accepted answer was probably the best way to solve it, but I decided this alternative could be useful to users of newer versions bumping into this page.
While you can't disable it, you can set the placeholder prefix or suffix to something that will never match. This will effectively achieve the same thing.
http://flywaydb.org/documentation/api/javadoc/org/flywaydb/core/Flyway.html#setPlaceholderPrefix(java.lang.String)