Hi I have tried to script my tpt to enable it to insert emoji's into a table. However it is not inserting it as expected, it is coming out as random characters 👌 👠ðŸ‘
Even though i have used the unicodepassthrough ='on' and it is enabled as i can it in the logs. We are using TD version 16.20 and the column is set to Unicode also.
any help with this would be much appreciated?
Related
Despite setting flyway.placeholderReplacement=false I keep seeing error about no value provided for placeholder expression in sql by Flyway
ERROR: Unexpected error
org.flywaydb.core.api.FlywayException: No value provided for placeholder expressions: & conditions. Check your configuration!
at org.flywaydb.core.internal.database.oracle.pro.SQLPlusPlaceholderReplacer.replacePlaceholders(SQLPlusPlaceholderReplacer.java:132)
at org.flywaydb.core.internal.util.line.PlaceholderReplacingLine.getLine(PlaceholderReplacingLine.java:36)
at org.flywaydb.core.internal.database.ExecutableSqlScript.extractStatements(ExecutableSqlScript.java:156)
at org.flywaydb.core.internal.database.ExecutableSqlScript.(ExecutableSqlScript.java:133)
at org.flywaydb.core.internal.database.oracle.OracleSqlScript.(OracleSqlScript.java:61)
at org.flywaydb.core.internal.database.oracle.OracleDatabase.doCreateSqlScript(OracleDatabase.java:126)
at org.flywaydb.core.internal.database.Database.createSqlScript(Database.java:163)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.getSqlScript(SqlMigrationExecutor.java:96)
at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.executeInTransaction(SqlMigrationExecutor.java:109)
at org.flywaydb.core.internal.command.DbMigrate.isExecuteGroupInTransaction(DbMigrate.java:312)
at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:275)
at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244)
at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:53)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:163)
at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:160)
at org.flywaydb.core.internal.database.Connection$1.call(Connection.java:145)
at org.flywaydb.core.internal.util.jdbc.TransactionTemplate.execute(TransactionTemplate.java:74)
at org.flywaydb.core.internal.database.Connection.lock(Connection.java:141)
at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:150)
at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:160)
at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:138)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:947)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:910)
at org.flywaydb.core.Flyway.execute(Flyway.java:1238)
at org.flywaydb.core.Flyway.migrate(Flyway.java:910)
at org.flywaydb.commandline.Main.executeOperation(Main.java:161)
at org.flywaydb.commandline.Main.main(Main.java:108)
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Using spring boot, in application.yml.
Add the below placeholderReplacement: false
flyway:
baseline-on-migrate: false
sql-migration-prefix: V
table: migration
placeholderReplacement: false
This was happening, because i had in my .SQL migration file, HTML code with ${name}.
So it was trying to replace that! and i want it to be as is, to be inserted in the database.
Summary: in my case i want it always disabled, as i have no use of it
References: Possible Usages In Different Configuration
flyway.placeholderReplacement=false is only for Flyway placeholders, not SQL*Plus placeholders.
To disable SQL*Plus-specific placeholders, you must include SET DEFINE OFF in your script.
I think, best way to resolve this error - it's override default prefix and suffix
flyway.placeholderPrefix=$${
flyway.placeholderSuffix=}
because disabling this functionality may be unacceptably for some reasons: using flyway variables, for instance.
If your use case is similar to mine (you cannot disable placeholders and you cannot change the prefix), an alternative way is to break the placeholder value. Let's says you SQL is:
UPDATE table SET value = '${variable}';
You can avoid FlyWay picking up the placeholder by using something like:
UPDATE table SET value = '$' + '{variable}';
you can encode the whole string with UTF-8, then decode it when you need to use it
I'm running a u-sql script similar to this:
#output =
SELECT Tag,
Json
FROM #table;
OUTPUT #output
TO #"/Dir/{Tag}/filename.txt"
USING Outputters.Text(quoting : false);
The problem is that #output is empty and the execution crashes. I already checked that if I don't use {tag} in the output path the script works well (it writes an empty file but that's the expectable).
Is there a way to avoid the crash and simply don't output anything?
Thank you
The form you are using is not yet publicly supported. Output to Files (U-SQL) documents the only supported version right now.
That said, depending on the runtime that you are using, and the flags that you have set in the script, you might be running the private preview feature of outputting to a set of files. In that case, I would expect that it would work properly.
Are you able to share a job link?
Straight forward question ..
The documentation for Oracle 10 states:
Oracle 10g sql*loader documentation
(Note, I linked to 10g since it was most convenient, I'll take an answer for Oracle 10 and/or Oracle 11, either way is fine - I suspect it'll be the same answer though - so I added both tags).
ERRORS (errors to allow)
Default: To see the default value for this parameter, invoke SQLLoader without any parameters, as described in Invoking SQLLoader.
ERRORS specifies the maximum number of insert errors to allow. If the number of errors exceeds the value specified for ERRORS, then SQL*Loader terminates the load. To permit no errors at all, set ERRORS=0. To specify that all errors be allowed, use a very high number.
(Emphasis mine).
So, since Oracle handles up to NUMBER(38) .. I tried:
ERRORS=999999999999999999999999999999999999
(36 digits)
and promptly got this error:
SQL*Loader-100: Syntax error on command-line
Trying a much smaller number:
ERRORS=999999
works fine.
So what's the maximum value you can use here ?
I don't find it in the documentation, so not sure if I'm looking in the wrong place, or it's just not in there :)
And yeah, I need a LARGE number, I'm loading a multi-million row file, so I'd like to use the largest possible to avoid any future issues.
IMHO sqlldr is not supporting number(39). I think all number parameter in sql loader are integer data type. And common limits for integer is 2147483647.
sqlldr xxxx control=ctl.ctl errors=2147483648 -> exception
sqlldr xxxx control=ctl.ctl errors=2147483647 -> works fine
I solved the problem setting
errors=-1
worked fine on Oracle 11g
I am working on Oracle 11g NACHA eText rtf template and i am facing problems while creating this template as per Bank needs. I need help while using substring extended function in column Originating DFI Identification.
My function is like this: substr(BankAccountNumber,1,8)
But when, I test the template using template viewer i got following erorr:
[112616_130010213][][PROCEDURE] Log Level is changed to PROCEDURE
[112616_130011163][][ERROR] expression: is invalid
Time: 4.706 sec.
FO Formatting failed.
End of Process.
Anyone here can help me with this please ?
The syntax looks correct, try making it uppercase. Check to see if that tag contains any data, and that the datatype is Alpha in the etext template.
I am trying to use filters to select specific tables to replicate.
I tried running this with the installer
./tools/tungsten-installer --master-slave -a \
...
--svc-extractor-filters=replicate \
--property=replicator.filter.replicate.do=test,*.foo"
and got this exception in trepctl status after the master had not installed properly:
Plugin class name property is missing or null: key=replicator.filter.replicate
which file is this properties file? How do I find it? Moreover, in specifying the settings for the filter, how do I know what exactly to put?
I discovered that I am supposed to Modify the configuration template file prior to configuration according to Issue 219 but what changes am I supposed to make in tungsten-replicator-2.0.5-diff that will later on be patched to the extraction?
Issue 254 suggests that If you want to apply a filter out of the box, you can use these options with tungsten-installer:
-a --property=replicator.filter.Replicate.ignoreFilter=schema_x.tablex,schema_x,tabley,schema_y,tablez
--svc-thl-filter=Replicate
However when I try using this for --property=replicator.filter.replicate.do,
but the problem is still the same:
pendingExceptionMessage: Plugin class name property is missing or null: key=replicator.filter.replicate
Your assistance will be greatly appreciated.
Rumbi
Update:
Hi
I had a look at this file: /root/tungsten/tungsten-replicator/samples/
conf/filters/default/tableignore.tpl .Acoording to this sample, a
static-SERVICE_NAME.properties file is supposed to have something like
this configured, please confirm if this is the correct syntax:
replicator.filter.tabledo=com.continuent.tungsten.replicator.filter.JavaScr iptFilter
replicator.filter.tabledo.script=${replicator.home.dir}/samples/
scripts/javascript-advanced/tabledo.js
replicator.filter.tabledo.tables=foo(database).bar(table)
replicator.stage.thl-to-dbms.filters=tabledo
However, I did not find tabledo.js (or something similar) in the
directory where tableignore.js exists. Could I please have the
location of this file. If there is an alternative way of specifiying
--property=replicator.filter.replicate.do=test without the use of
this .js file, your suggestions are most welcome.
Download the latest version of tungsten replicator. The missing tpl file was added about a month ago. After installation, the filtered tables should be added to static-service.properties under the section FILTERS.
Locate your replicator configuration file in static-YOUR_SERVICE_NAME.properties, e.g.
/opt/continuent/tungsten/tungsten-replicator/conf/static-mysql2vertica.properties
Make sure the individual dbms properties are set, in particular the setting replicator.applier.dbms:
# Batch applier basic configuration information.
replicator.applier.dbms=com.continuent.tungsten.replicator.applier.batch.SimpleBatchApplier
replicator.applier.dbms.url=jdbc:mysql:thin://${replicator.global.db.host}:${replicator.global.db.port}/tungsten_${service.name}?createDB=true
replicator.applier.dbms.driver=org.drizzle.jdbc.DrizzleDriver
replicator.applier.dbms.user=${replicator.global.db.user}
replicator.applier.dbms.password=${replicator.global.db.password}
replicator.applier.dbms.startupScript=${replicator.home.dir}/samples/scripts/batch/mysql-connect.sql
# Timezone and character set.
replicator.applier.dbms.timezone=GMT+0:00
replicator.applier.dbms.charset=UTF-8
# Parameters for loading and merging via stage tables.
replicator.applier.dbms.stageTablePrefix=stage_xxx_
replicator.applier.dbms.stageDirectory=/tmp/staging
replicator.applier.dbms.stageLoadScript=${replicator.home.dir}/samples/scripts/batch/mysql-load.sql
replicator.applier.dbms.stageMergeScript=${replicator.home.dir}/samples/scripts/batch/mysql-merge.sql
replicator.applier.dbms.cleanUpFiles=false
Depending on the database you are replicating to you may have to omit/modify some of the lines.
For more information see:
https://code.google.com/p/tungsten-replicator/wiki/Replicator_Batch_Loading
I don't know if this problem is still open or not.
I am using this version 2.0.6-xxx and installing the service using the parameters works for me.
I would like to point it out, that as the parameter says "--svc-extractor-filters" defines an extractor filter. Meaning that the parameters will guide the extraction of data in the master server.
If you intend to use it on the slave service, you should use the "--svc-applier-filters".
The parameters
--svc-extractor-filters=replicate \
--property=replicator.filter.replicate.do=test,*.foo"
supposed to create the following in the properties file:
This is the filter set up.
replicator.filter.replicate=com.continuent.tungsten.replicator.filter.ReplicateFilter
replicator.filter.replicate.ignore=
replicator.filter.replicate.do=test,*.foo
And you should also be able to find the
replicator.stage.binlog-to-q.filters=replicate
parameter set.
If you intend to use this filter in the slave, please find the line with:
replicator.stage.q-to-dbms.filters=mysqlsessions,pkey,bidiSlave
and change it as
replicator.stage.q-to-dbms.filters=mysqlsessions,pkey,bidiSlave,replicate
Hope this brief description did help to you!