I'm using MariaDB to import some files in XML.
Here is a snippet of the code I'm using:
CREATE TABLE invoices (
InvoiceNumber VARCHAR(20),
InvoiceStatus CHAR (1),
InvoiceDate CHAR (10),
Period CHAR (2)
)
;
DROP TABLE if EXISTS temptbl;
create table temp02 (
InvoiceNumber VARCHAR(20) xpath='InvoiceNo',
InvoiceStatus CHAR(1) xpath='DocumentStatus/InvoiceStatus',
InvoiceDate CHAR (10) xpath='InvoiceDate',
Period CHAR (2) xpath='Period'
)
engine=CONNECT table_type=XML file_name='..\\importmaridb\\month01.xml'
tabname='AuditFile' option_list='rownode=SourceDocuments/SalesInvoices/Invoice';
INSERT INTO invoices
SELECT * FROM temptbl;
I then repeat 12x the code of importing to table "temptbl" changing only the file name to reflect the other months.
I would like to have a loop to iterate the files each time.
I believe part of the solution would be to create a table with the file names and an auto increment column, where I would loop through the numbers.
I've attempted to define a variable and substituting in the code, like file_name=#path. But MariaDB gives me an error a syntax error.
| VARIABLE_NAME | VARIABLE_VALUE | VARIABLE_TYPE | CHARACTER_SET_NAME |
+---------------+----------------+---------------+--------------------+
| path | ..\importmaridb| VARCHAR | utf8mb4 |
| | \month01.xml | | |
Can someone give me some pointers, and even if this is possible?
Related
I'm coding in python (PySide2), but the overall concept concerns the Qt Framework.
Here are two tables of mine:
Table "recordings":
| Column | Type |
| -------- | -------------------|
| id | int (primary key) |
| param1 | int |
| param2 | int |
| ... | int |
| paramN | int |
Table "analyzed_recs":
| Column | Type |
| -------- | -------------------|
| id | int (primary key) |
| rec_id | int (foreign key) | <-- Points to recordings.id
| paramN | int |
I need in my program to display param1 and param2 from the former. In Qt I used a QSqlRelationalTable to fulfill this objective:
def _init_db_models(self):
self.analyzed_recs_sql_model = QSqlTableModel(self, self.db)
self.analyzed_recs_sql_model.setTable("analyzed_recs")
rec_id = self.analyzed_recs_sql_model.fieldIndex('rec_id')
self.analyzed_recs_sql_model.setRelation(rec_id, QSqlRelation("recordings", "id", "param1"))
self.analyzed_recs_sql_model.setRelation(rec_id, QSqlRelation("recordings", "id", "param2"))
self.analyzed_recs_sql_model.select()
self.analyzed_data_table.setModel(self.analyzed_recs_sql_model)
This code works fine in displaying the desired fields.
However, when it comes to update a record in analyzed_recs:
record = self.analyzed_recs_sql_model.record()
record.remove(record.indexOf("id"))
record.setValue("rec_id", self.current_rec_id)
record.setValue("param1", self.param1)
record.setValue("param2", param2)
self.analyzed_recs_sql_model.insertRecord(-1, record)
self.analyzed_recs_sql_model.submitAll()
The column rec_id is not set (NULL) into the table (the other params are correctly inserted into the table).
On the contrary, if I avoid using QSqlRelationalTableModel and take QSqlTableModel instead, the insertion is performed correctly (as I expected), but I lose the INNER JOIN display feature.
I was thinking as a work around to create two distinct models, a QSqlRelationalTableModel only for displaying and a QSqlTableModel only for editing the data. However I don't like the extra workload of syncing the two.
I'm sure there is a Qt feature to achieve this, but unfortunately I'm missing it.
Any suggestion?
I've had the same problem using PYQT.
The record object returned by calling record() method has no fields named 'rec_id' because the QSqlRelationalTableModel changes it with the referenced field name 'param1'. We can verify the field names using:
fieldcount = record.count()
for i in range(fieldcount):
logging.info("field %s %s", i, record.fieldName(i))
so we need to add the field before assigning it:
record = self.analyzed_recs_sql_model.record()
record.remove(record.indexOf("id"))
record.append(QSqlField("rec_id"))
record.setValue("rec_id", self.current_rec_id)
I'm trying to export/import a BD from one system to another but the import fails with the following error:
ERROR 1062 (23000) at line 8232: Duplicate entry '0-3-30168717-com_liferay_product_navigation_product_menu_web_...' for key 'IX_C7057FF7'
That table is defined as such:
CREATE TABLE `PortletPreferences` (
`portletPreferencesId` bigint(20) NOT NULL,
`ownerId` bigint(20) DEFAULT NULL,
`ownerType` int(11) DEFAULT NULL,
`plid` bigint(20) DEFAULT NULL,
`portletId` varchar(200) DEFAULT NULL,
`preferences` longtext DEFAULT NULL,
`mvccVersion` bigint(20) NOT NULL DEFAULT 0,
`companyId` bigint(20) DEFAULT NULL,
PRIMARY KEY (`portletPreferencesId`),
UNIQUE KEY `IX_C7057FF7` (`ownerId`,`ownerType`,`plid`,`portletId`),
In the mysql dump file, I see these two entries:
(31453178,0,3,30168717,'com_liferay_product_navigation_product_menu_web_portlet_ProductMenuPortlet','<portlet-preferences />',0,10132)
(31524539,0,3,30168717,'com_liferay_product_navigation_product_menu_web_portlet_ProductMenuPortlet','<portlet-preferences />',0,10132)
So, yep, there are two entries with the same unique key. How is that possible?!?
Knowing this, I ran the following select statement against the source DB:
select portletPreferencesId, ownerId, ownerType, plid, portletId from PortletPreferences where ownerId = 0 AND ownerType = 3 AND plid = 30168717 AND portletId like 'com_liferay_product_navigation_product_menu_web%';
And it outputs just ONE LINE!
+----------------------+---------+-----------+----------+----------------------------------------------------------------------------+
| portletPreferencesId | ownerId | ownerType | plid | portletId |
+----------------------+---------+-----------+----------+----------------------------------------------------------------------------+
| 31524539 | 0 | 3 | 30168717 | com_liferay_product_navigation_product_menu_web_portlet_ProductMenuPortlet |
+----------------------+---------+-----------+----------+----------------------------------------------------------------------------+
By the portletPreferencesId field, it outputs the second entry in the dump file. So I did one more select for the other row as such:
select portletPreferencesId, ownerId, ownerType, plid, portletId from PortletPreferences where portletPreferencesId = 31453178;
And I get:
+----------------------+---------+-----------+----------+----------------------------------------------------------------------------+
| portletPreferencesId | ownerId | ownerType | plid | portletId |
+----------------------+---------+-----------+----------+----------------------------------------------------------------------------+
| 31453178 | 0 | 3 | 30168717 | com_liferay_product_navigation_product_menu_web_portlet_ProductMenuPortlet |
+----------------------+---------+-----------+----------+----------------------------------------------------------------------------+
My question is, what's going on?!? Why is that entry not output by the first select statement and why is it there in the first place if those fields were supposed to be unique???
I have a bad feeling about the state of the source database :-( Oh, and that's just one. I have multiple duplicate keys like that in that table :-(
Thanks
There are occasional bugs like MDEV-15250 that can cause duplicate entries to occur.
Sometimes you may not see these as parts of the optimizer expect only a single row because of the unique constraint so won't search beyond it. Maybe a query across a range including the entries would be more likely to show it (which is what mysqldump would have done).
If you don't think its related to ALTER TABLE occurring at the same time as insertions (like MDEV-15250), and have a good hunch as the set of operations on the table that may have escaped the unique key enforcement, can you please create a bug report.
Query1
cluster(x).database('$systemdb').Operations
| where Operation == "DatabaseCreate" and Database contains "oci-"| where State =='Completed'
and StartedOn between (datetime(2020-04-07) .. 3d)
| distinct Database , StartedOn
| order by StartedOn desc
Output of my query1 is list of databases , now I have to pass each db value into query2 to get buildnumber
Query2:
set query_take_max_records=5000;
let view=datatable(Property:string,Value:dynamic)[];
let viewFile=datatable(FileName:string)[];
alias database db = cluster(x).database('y');
let latestInfoFile = toscalar((
union isfuzzy=true viewFile,database('db').['TextFileLogs']
| where FileName contains "AzureStackStampInformation"
| distinct FileName
| order by FileName
| take 1));
union isfuzzy=true view,(
database('db').['TextFileLogs']
| where FileName == latestInfoFile
| distinct LineNumber,FileLineContent
| order by LineNumber asc
| summarize StampInfo=(toobject(strcat_array(makelist(FileLineContent,100000), "\r\n")))
| mvexpand bagexpansion=array StampInfo
| project Property=tostring(StampInfo[0]), Value=StampInfo[1]
)|where Property contains "StampVersion" | project BuildNumber = Value;
database() function: is a special scoping function, and it does not support non-constant arguments due to security consideration.
As a result - you cannot use sub-query to fetch list of databases and then operate on this list as input for database() function.
This behavior is described at:
https://learn.microsoft.com/en-us/azure/kusto/query/databasefunction?pivots=azuredataexplorer
Syntax
database(stringConstant)
Arguments
stringConstant: Name of the database that is referenced. Database identified can be either DatabaseName or PrettyName. Argument has to be constant prior of query execution, i.e. cannot come from sub-query evaluation.
I have a table like this:
CREATE TABLE test (
height int(10) CHECK(height>5)
);
When I try to remove check constraint by:
ALTER TABLE test DROP CONSTRAINT height;
I got this error message:
ERROR 1091 (42000): Can't DROP CONSTRAINT `height`; check that it exists
Here is the SHOW CREATE TABLE test; command output:
+-------+-------------------------------------------------------------------------------------------------------------------+
| Table | Create Table
|
+-------+-------------------------------------------------------------------------------------------------------------------+
| test | CREATE TABLE `test` (
`height` int(10) DEFAULT NULL CHECK (`height` > 5)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+-------+-------------------------------------------------------------------------------------------------------------------+
And here is the SELECT * from information_schema.table_constraints where TABLE_NAME = 'test'; output:
+--------------------+-------------------+-----------------+------------------+------------+-----------------+
| CONSTRAINT_CATALOG | CONSTRAINT_SCHEMA | CONSTRAINT_NAME | TABLE_SCHEMA | TABLE_NAME | CONSTRAINT_TYPE |
+--------------------+-------------------+-----------------+------------------+------------+-----------------+
| def | test_db | height | test_db | test | CHECK |
+--------------------+-------------------+-----------------+------------------+------------+-----------------+
CREATE TABLE :: Constraint
Expressions
...
MariaDB 10.2.1 introduced two ways to define a constraint:
CHECK(expression) given as part of a column definition.
CONSTRAINT [constraint_name] CHECK (expression)
...
If you define the constraint using the first form (column constraint), you can remove it using MODIFY COLUMN:
ALTER TABLE `test`
MODIFY COLUMN `height` INT(10);
If you use the second form (table constraint), you can remove it using DROP CONSTRAINT:
ALTER TABLE `test`
DROP CONSTRAINT `height`;
See dbfiddle.
I have a database called av2web, which contains 130 MyISAM tables and 20 innodb tables. I wanna take mysqldump of these 20 innodb tables, and export it to another database as MyISAM tables.
Can you tell me a quicker way to achieve this?
Thanks
Pedro Alvarez Espinoza.
If this was an one-off operation I'd do:
use DB;
show table status name where engine='innodb';
and do a rectangular copy/paste from the Name column:
+-----------+--------+---------+------------+-
| Name | Engine | Version | Row_format |
+-----------+--------+---------+------------+-
| countries | InnoDB | 10 | Compact |
| foo3 | InnoDB | 10 | Compact |
| foo5 | InnoDB | 10 | Compact |
| lol | InnoDB | 10 | Compact |
| people | InnoDB | 10 | Compact |
+-----------+--------+---------+------------+-
to a text editor and convert it to a command
mysqldump -u USER DB countries foo3 foo5 lol people > DUMP.sql
and then import after replacing all instances of ENGINE=InnoDB with ENGINE=MyISAM in DUMP.sql
If you want to avoid the rectangular copy/paste magic you can do something like:
use information_schema;
select group_concat(table_name separator ' ') from tables
where table_schema='DB' and engine='innodb';
which will return countries foo3 foo5 lol people
I know this is an old question. I just want to share this script that genrate the mysqldump command and also shows how to restore it
This following portion of the script will generate a command to create a mysql backup/dump
SET SESSION group_concat_max_len = 100000000; -- this is very important when you have lots of table to make sure all the tables get included
SET #userName = 'root'; -- the username that you will login with to generate the dump
SET #databaseName = 'my_database_name'; -- the database name to look up the tables from
SET #extraOptions = '--compact --compress'; -- any additional mydqldump options https://dev.mysql.com/doc/refman/5.6/en/mysqldump.html
SET #engineName = 'innodb'; -- the engine name to filter down the table by
SET #filename = '"D:/MySQL Backups/my_database_name.sql"'; -- the full path of where to generate the backup too
-- This query will generate the mysqldump command to generate the backup
SELECT
CASE WHEN tableNames IS NULL
THEN 'No tables found. Make sure you set the variables correctly.'
ELSE CONCAT_WS(' ','mysqldump -p -u', #userName, #databaseName, tableNames, #extraOptions, '>', #filename)
END AS command
FROM (
SELECT GROUP_CONCAT(table_name SEPARATOR ' ') AS tableNames
FROM INFORMATION_SCHEMA.TABLES
WHERE table_schema= #databaseName AND ENGINE= #engineName
) AS s;
This following portion of the script will generate a command to restore mysql backup/dump into a specific database on the same or a different server
SET #restoreIntoDatabasename = #databaseName; -- the name of the new database you wish to restore into
SET #restoreFromFile = #filename; -- the full path of the filename you want to restore from
-- This query will generate the command to use to restore the generated backup into mysql
SELECT CONCAT_WS(' ', 'mysql -p -u root', #restoreIntoDatabasename, '<', #restoreFromFile);