I have recently been trying to upgrade flyway from v4 to v6.5.3. During the process, I faced an issue related to placeholders.
The migration fails with the following error.
ERROR: Unable to parse statement in
D:\Softwares\flyway-commandline-6.5.5-windows-x64\flyway-6.5.5\sql\V3__mysql-7.0.sql
at line 101 col 1. See
https://flywaydb.org/documentation/knownparserlimitations for more
information: No value provided for placeholder: ${'), NOW(), NOW())}.
Check your configuration! Caused by: No value provided for
placeholder: ${'), NOW(), NOW())}. Check your configuration!
SQL,
insert into `configuration`(key, value, created_date , updated_date) values('LOG_LOCATION', REPLACE('${MY_LOG_LOCATION}','#{','${'), NOW(), NOW());
To resolve the above failure, I replaced "${" with "$\{", but this is something I didn't look out for.
On debugging the flyway code, I saw it fails in parsing (during validation, method org.flywaydb.core.internal.parser.Parser.readToken()) of the SQL.
Why is it considering the 3rd argument of REPLACE function as a placeholder?
Note: The above SQL works in Flyway v4
If you read the documentation here, you can see that flyway uses a particular syntax for it's placeholders, ${somestring}. In your code, you have a ${, but it's not defining a placeholder. While Flyway is a pretty sophisticated tool, the mechanism here is simply using string matching. If you want to, you can modify your Flyway instance to use different escape characters for the placeholders. This could be handy if you have to have strings in your code that match that ${ syntax. You can read about that here. Scroll down to where you define the PlaceHolderPrefix and PlaceHolderSuffix. Change those to a different combination of values that are not used in your code and you should be fine.
Related
I am having issues while using the filename default placeholder, in repeatable migrations. I get the error as ERROR: Failed to populate value for default placeholder: ${flyway:filename}
However, I am able to use the same placeholder in non-repeatable and other callbacks like beforeMigrate, afterMigrate etc. Also, able to use all other default placeholders like timestamp, user in both repeatable and non-repeatable migration scripts.
We face the problem only with 'filename' placeholder in repeatable migration scripts.
Is there any limitation(I believe it might not be) on using the 'filename' placeholder in repeatable migrations?
Did any one had similar issue?
or does it require any other configurations?
We are using flyway version 8.5.11 and command-line utility.
I had the following code, which was running well with EF Core 2.1:
.FirstOrDefault(a => (a.Name.Equals(b, StringComparison.InvariantCultureIgnoreCase).
(Ok, running well means I got the right results even if it was being evaluated in the client side and I didn't know it).
I updagred to EF Core 3.0 and I didn't get any error, but this code was not giving the expected results.
I saw here a solution. I tried a.Name.ToLower() == b.ToLower() but then I got the the error:
Error CA1304 The behavior of 'string.ToLower()' could vary based on the current user's locale settings. Replace this call in 'MyFunction(string, string)' with a call to 'string.ToLower(CultureInfo)'
If I use a ToLower(CultureInfo.InvariantCulture) I get the message:
Error CA1308 In method 'MyFunction', replace the call to 'ToLower' with 'ToUpperInvariant'.
If I use ToUpperInvariant(), then I get the error (I'm already aware of the LINQ breaking changes in EF Core 3.0):
The LINQ expression (... all the expression...) could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync().
So, I am the starting point.
Is there a way to both comply with CA1304 and run the query in the DB and not in the client side?
The solution, as PanagiotisKanavos commented, was to simply use a.Name == b. Easy and it works!
Despite setting flyway.placeholderReplacement=false I keep seeing error about no value provided for placeholder expression in sql by Flyway
ERROR: Unexpected error
org.flywaydb.core.api.FlywayException: No value provided for placeholder expressions: & conditions. Check your configuration!
at org.flywaydb.core.internal.database.oracle.pro.SQLPlusPlaceholderReplacer.replacePlaceholders(SQLPlusPlaceholderReplacer.java:132)
at org.flywaydb.core.internal.util.line.PlaceholderReplacingLine.getLine(PlaceholderReplacingLine.java:36)
at org.flywaydb.core.internal.database.ExecutableSqlScript.extractStatements(ExecutableSqlScript.java:156)
at org.flywaydb.core.internal.database.ExecutableSqlScript.(ExecutableSqlScript.java:133)
at org.flywaydb.core.internal.database.oracle.OracleSqlScript.(OracleSqlScript.java:61)
at org.flywaydb.core.internal.database.oracle.OracleDatabase.doCreateSqlScript(OracleDatabase.java:126)
at org.flywaydb.core.internal.database.Database.createSqlScript(Database.java:163)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.getSqlScript(SqlMigrationExecutor.java:96)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeInTransaction(SqlMigrationExecutor.java:109)
at org.flywaydb.core.internal.command.DbMigrate.isExecuteGroupInTransaction(DbMigrate.java:312)
at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:275)
at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244)
at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:53)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:163)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:160)
at org.flywaydb.core.internal.database.Connection$1.call(Connection.java:145)
at org.flywaydb.core.internal.util.jdbc.TransactionTemplate.execute(TransactionTemplate.java:74)
at org.flywaydb.core.internal.database.Connection.lock(Connection.java:141)
at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:150)
at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:160)
at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:138)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:947)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:910)
at org.flywaydb.core.Flyway.execute(Flyway.java:1238)
at org.flywaydb.core.Flyway.migrate(Flyway.java:910)
at org.flywaydb.commandline.Main.executeOperation(Main.java:161)
at org.flywaydb.commandline.Main.main(Main.java:108)
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Using spring boot, in application.yml.
Add the below placeholderReplacement: false
flyway:
baseline-on-migrate: false
sql-migration-prefix: V
table: migration
placeholderReplacement: false
This was happening, because i had in my .SQL migration file, HTML code with ${name}.
So it was trying to replace that! and i want it to be as is, to be inserted in the database.
Summary: in my case i want it always disabled, as i have no use of it
References: Possible Usages In Different Configuration
flyway.placeholderReplacement=false is only for Flyway placeholders, not SQL*Plus placeholders.
To disable SQL*Plus-specific placeholders, you must include SET DEFINE OFF in your script.
I think, best way to resolve this error - it's override default prefix and suffix
flyway.placeholderPrefix=$${
flyway.placeholderSuffix=}
because disabling this functionality may be unacceptably for some reasons: using flyway variables, for instance.
If your use case is similar to mine (you cannot disable placeholders and you cannot change the prefix), an alternative way is to break the placeholder value. Let's says you SQL is:
UPDATE table SET value = '${variable}';
You can avoid FlyWay picking up the placeholder by using something like:
UPDATE table SET value = '$' + '{variable}';
you can encode the whole string with UTF-8, then decode it when you need to use it
I changed Change in persistence.xml
I also changed column definition (columnDefinition="XDB.XMLType") for xml fields
I checked OpenJpa(http://openjpa.208410.n2.nabble.com/Oracle-XMLType-fetch-problems-td6208344.html) site and IBM (http://www.ibm.com/support/knowledgecenter/SS7J6S_7.5.0/com.ibm.wsadapters.jca.jdbc.doc/env/doc/rjdb_problemsolutions.html)
My env is OpenJpa 2.0 and WAS 7
its throwing exception
org.apache.openjpa.persistence.PersistenceException: ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at "SYS.XMLTYPE", line 169
Please suggest without changing OpenJpa2.0 as its part of IBM WebSphere Application Server V7.0 how can i handle sys.XMLTYPE data, i am migrating my application from db2 to Oracle in same environment.
Writing XML data can be tricky some times! Getting the correct drivers and things defined properly can have its challenges. I can not say exactly what you need to do given the lack of info on your domain model and such, but let me give some general things to look for. First, there is an XML test in the OpenJPA test framework if you want to make reference to it. It can be seen publicly here:
https://apache.googlesource.com/openjpa/+/refs/heads/trunk/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/jdbc/oracle/
Or, another test using an "XMLValueHandler" (likely this is beyond the scope of what you are looking for):
https://apache.googlesource.com/openjpa/+/refs/heads/trunk/openjpa-persistence-jdbc/src/test/java/org/apache/openjpa/persistence/xmlmapping/query/
Second, (stating the obvious) I assume you have a column in Oracle defined as "XMLTYPE". Also, I see you are using schema SYS. I'm sure you are aware but this is a system/admin schema......just for sanity sake you might want to first get things running using a non-system/admin schema just so we don't get hung up with any issues with your OpenJPA client not having the correct permissions.
Next, you need the following definition:
#Lob #Basic
#Column(name = "ANXMLCOLUMN", columnDefinition="XMLCOLUMN XMLType")
private String anXMLString;
The #Lob I think will be necessary if you are using data greater than 4000 chars (this was mentioned in one of the comments). To start I'd use a very small set of data (a couple characters), once that works, then experiment with > 4k.
Next, make sure to use the correct JDBC driver. The last time I experimented with an XMLType I used the Oracle JDBC 11.2.0.2 driver.
Finally, you might need to use the property "openjpa.jdbc.DBDictionary" with value "oracle(supportsSetClob=true,maxEmbeddedClobSize=-1)". Again, experiment with this AND look at the OpenJPA documentation on these properties to determine if they are necessary in your scenario. I think the supportsSetClob=true will only be necessary for older version (pre-2.2.x) of OpenJPA. You might also need to use property "openjpa.jdbc.SchemaFactory" with value "native". I would suggest you first try without either or these two properties. If that doesn't help, then experiment with these two properties. I know this is vague, but I don't know what your DDL or domain model looks like so I have to keep in vague.
Thanks,
Heath Thomann
So what I understood after upgrading my flyway version because of some requirements is that flyway-core-2.2 introduced some validation for Flyway placeholders.
Now, the convention of placeholder syntax is ${name} uniform across most libraries. In our migration scripts, we are inserting a string in a mysql table column called stretchySql and that string holds some placeholder elements of our own which is meant to be interpreted at runtime by the application layer.
UPDATE `stretchy_parameter` SET `parameter_sql`='select r.id as report_id from stretchy_report r where r.report_category = \'${reportCategory}\'
I don't want flyway to interpret something embedded in a string as its own placeholder and throw an error. So basically, is there some way to switch off flyway placeholder validation(since we don't use it) without reverting back to an older version?
As of Flyway 3.2.1 (not sure when it was introduced) there is a new boolean setting called flyway.placeholderReplacement, which defaults to true, and you can use that to effectively disable the feature. On the API it can be accessed via the setPlaceholderReplacement(value) and isPlaceholderReplacement() methods of a Flyway object.
I know 2.2 probably didn't have that option and the accepted answer was probably the best way to solve it, but I decided this alternative could be useful to users of newer versions bumping into this page.
While you can't disable it, you can set the placeholder prefix or suffix to something that will never match. This will effectively achieve the same thing.
http://flywaydb.org/documentation/api/javadoc/org/flywaydb/core/Flyway.html#setPlaceholderPrefix(java.lang.String)