WE have flyway integrated with Redshift and we are using this as a simple java main program to run all our schema migrations. We also use the info command to print the current version of the database, However this command successfully runs or at least appears to run but does not print the version number.
We have version 4.2 of the flyway jar. What is that we may be missing? Thanks
To manually recreate what the info command line option does in java code you can copy what its implementation does (from the source):
MigrationInfoDumper.dumpToAsciiTable(flyway.info().all())
An example from the docs is shown below:
+-------------+------------------------+---------------------+---------+
| Version | Description | Installed on | State |
+-------------+------------------------+---------------------+---------+
| 1 | Initial structure | | Pending |
| 1.1 | Populate table | | Pending |
| 1.3 | And his brother | | Pending |
+-------------+------------------------+---------------------+---------+
Related
I just installed this plugin to monitor SQL errors:
https://mariadb.com/kb/en/sql-error-log-plugin/
but I can't for the life of me find anything on how to modify the default system variables for that error log plugin, and where.
Anyone?!
You can't modify error plugin log variables via SQL statements. They are either read-only and/or they need to be specified at startup (see source code).
So you have to pass these values either to mariadbd/mysqld or you have to specify these in your configuration file.
Example in /etc/my.cnf:
[server]
sql-error-log-filename=foo.log
sql-error-log-rotate=ON
restart server and check the values
MariaDB [test]> show variables like 'sql_error%';
+--------------------------+---------+
| Variable_name | Value |
+--------------------------+---------+
| sql_error_log_filename | foo.log |
| sql_error_log_rate | 1 |
| sql_error_log_rotate | ON |
| sql_error_log_rotations | 9 |
| sql_error_log_size_limit | 1000000 |
+--------------------------+---------+
I am working with flyway Db migration, and I have download flyway zip folder and placed into my local computer.
I have two files in the sql folder, i.e V1__Create_person_table.sql and V2__Add_people.sql.
Flyway info
Flyway Community Edition 5.2.4 by Boxfuse
Database: jdbc:h2:file:./foobardb (H2 1.4)
Schema version: << Empty Schema >>
+-----------+---------+---------------------+------+--------------+---------+
| Category | Version | Description | Type | Installed On | State |
+-----------+---------+---------------------+------+--------------+---------+
| Versioned | 1 | Create person table | SQL | | Pending |
| Versioned | 2 | Add people | SQL | | Pending |
+-----------+---------+---------------------+------+--------------+---------+
Flyway migrate
Flyway Community Edition 5.2.4 by Boxfuse
Database: jdbc:h2:file:./foobardb (H2 1.4)
Successfully validated 2 migrations (execution time 00:00.020s)
Creating Schema History table: "PUBLIC"."flyway_schema_history"
Current version of schema "PUBLIC": << Empty Schema >>
Migrating schema "PUBLIC" to version 1 - Create person table
Migrating schema "PUBLIC" to version 2 - Add people
Successfully applied 2 migrations to schema "PUBLIC" (execution time 00:00.092s)
Flyway info
Flyway Community Edition 5.2.4 by Boxfuse
Database: jdbc:h2:file:./foobardb (H2 1.4)
Schema version: 2
+-----------+---------+---------------------+------+---------------------+---------+
| Category | Version | Description | Type | Installed On | State |
+-----------+---------+---------------------+------+---------------------+---------+
| Versioned | 1 | Create person table | SQL | 2019-08-19 12:12:40 | Success |
| Versioned | 2 | Add people | SQL | 2019-08-19 12:12:40 | Success |
+-----------+---------+---------------------+------+---------------------+---------+
Now, here the question is: if I want to update or edit somehing in above two sql files, how can I do that, should I edit existing file version 1, version 2 and
save the file and run all the above command again?
You should not edit your existed scripts. I have to add a new one e.g. V3__Update_person_table.sql and correctly update it.
P.S.
In the big project, we have tens of scripts that iteratively modify the empty database to achieve a current status.
After that, usually, when moving to the next release version, we merge all existed scripts into one or two (when we do not need to keep history anymore).
Notes
This is correct. We do not change existed scripts (this is part of CI/CD). All changes should be added additionally with new scripts. Flyway accepts a directory with all scripts.
I have installed Osquery utility on my machine. When I fire an SQL command, it gives output to STDOUT. Is there any way to redirect that output to a file?
$ sudo osqueryi
I0314 10:57:51.644351 3958 database.cpp:563] Checking database version for migration
I0314 10:57:51.644912 3958 database.cpp:587] Performing migration: 0 -> 1
I0314 10:57:51.645279 3958 database.cpp:619] Migration 0 -> 1 successfully completed!
I0314 10:57:51.645627 3958 database.cpp:587] Performing migration: 1 -> 2
I0314 10:57:51.646088 3958 database.cpp:619] Migration 1 -> 2 successfully completed!
Using a virtual database. Need help, type '.help'
osquery>
osquery>
osquery> SELECT * from memory_info;
+--------------+-------------+----------+----------+-------------+-----------+----------+------------+-----------+
| memory_total | memory_free | buffers | cached | swap_cached | active | inactive | swap_total | swap_free |
+--------------+-------------+----------+----------+-------------+-----------+----------+------------+-----------+
| 513617920 | 270921728 | 15110144 | 99860480 | 0 | 145080320 | 59494400 | 0 | 0 |
+--------------+-------------+----------+----------+-------------+-----------+----------+------------+-----------+
osquery>
I want this output in a file. I checked Osquery official documentation. But it hasn't been helpful to solve this particular problem. https://osquery.readthedocs.io/en/stable/introduction/sql/#sql-as-understood-by-osquery
You can use the redirection facilities of your shell:
$ osqueryi --json 'select * from osquery_info' > res.json
$ cat res.json
[
{"build_distro":"10.12","build_platform":"darwin","config_hash":"e7c68185a7252c23585d53d04ecefb77b3ebf99c","config_valid":"1","extensions":"inactive","instance_id":"38201952-9a75-41dc-b2f8-188c2119cda1","pid":"26255","start_time":"1552676034","uuid":"4740D59F-699E-5B29-960B-979AAF9BBEEB","version":"3.3.0","watcher":"-1"}
]
Note that in this example we use JSON output. There are other options available: --csv, --line, --list.
As seph explained in https://stackoverflow.com/a/55164199/491710, it is a common use-case to schedule queries in osqueryd and push the results into a logging pipeline.
osqueryi is generally for interactive use. When saving to files, or having osquery part of a data pipeline, people usually configure scheduled queries with osqueryd.
https://osquery.readthedocs.io/en/stable/deployment/configuration/ has some pretty simple examples of a configuration.
You could also specify the query on the command line, and then do whatever you're doing in the shell.
I am trying to impelment flywaydb in our process. In our env each client has their own instance of the DB.
I have a bash that loops through the clients to run migrate. So the command looks like
flyway -url=jdbc:jtds:sqlserver://localhost:1434/main_client_$ID migrate
This all works when all the clients start from the baseline. But as we add new customers their DB will reflect the newest code. Now we have older clients started with V1(and all the migration scripts to V2) and new clients with the latest DB V2.
I thought i could do something like :::
flyway baseline -url=jdbc:jtds:sqlserver://localhost:1434/main_client_3
--baselineVersion=2 --baselineDescription="Base 2 version"
but when i do it this way then call info i see something like :
+---------+-----------------------+---------------------+---------+
| Version | Description | Installed on | State |
+---------+-----------------------+---------------------+---------+
| 1 | << Flyway Baseline >> | 2015-06-08 22:07:54 | Success |
| 1.1 | update | | Pending |
| 1.2.0 | update | | Pending |
| 1.2.1 | update | | Pending |
+---------+-----------------------+---------------------+---------+
if i look in the DB i see the version value of schema_version set to 1.
if via the DB i force schema_version column value to 1.2.0 i see
+---------+-----------------------+---------------------+---------+
| Version | Description | Installed on | State |
+---------+-----------------------+---------------------+---------+
| 1 | Base version initial | | <Baseln |
| 1.1 | update | | <Baseln |
| 1.2.0 | << Flyway Baseline >> | 2015-06-08 22:07:54 | Success |
| 1.2.1 | update | | Pending |
+---------+-----------------------+---------------------+---------+
This is what i want.
But i can not figure out how to set the value via the baseline command
Thanks for any help
All parameters should be passed in with - not --
I used mysqldump to export my database and then I imported it into MySQL on my other server. I can now see all my tables if I do "show tables" but I can't actually select from or describe any of them.
ERROR 1146 (42S02): Table 'mydatabase.user' doesn't exist
All of my tables are InnoDB. I saw one issue people had where they were using old_passwords, so I explicitly set that to 0 in my.cnf and I made sure all of the passwords in the mysql table were 41 hexadecimal digits as they should be for the new passwords.
The reason "show tables;" works is because mysqld will scan the database directory for .frm files only. As long as they exist, it sees a table definition.
If you imported the data into MySQL and this error message happens, the first thing I would immediately do is run this command: (BTW This is MySQL 5.1.45, but works in MySQL 5.x anyway)
mysql> show engines;
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine | Support | Comment | Transactions | XA | Savepoints |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| InnoDB | YES | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
| MyISAM | DEFAULT | Default engine as of MySQL 3.23 with great performance | NO | NO | NO |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
8 rows in set (0.00 sec)
If the server you imported the data into says InnoDB is disabled, then you have a big problem. Here is what you should do:
1) Drop all the Data from the New Import DB Server
2) Cleanup InnoDB Setup
3) run SHOW ENGINES; and make sure InnoDB is fully operational !!!
4) Reload the mysqldump into the new import server
Give it a Try !!!
I had this problem when I changed from a windows server to a Linux server.
Tables are files, and windows files are case insesitive, but linux files are case sensitive.
In my aplication, in the sql queries, some times I used uppercase tablenames and other times lowercase, so, sometimes I obtained the same result as you.
I my case it was SQLCA.DBParm parameter.
I used SQLCA.DBParm = "Databse = "sle_database.text"" but it must be
SQLCA.DBParm = "Database='" +sle_database.text+ "'"
Explain : you are going to combine three strings :
a) Database=' - "Database='"
b) (name of the database) - +sle_database.text+
c) ' - "'"