ERROR MY-011542 Repl Plugin group_replication reported: 'Table repl_test does not have any PRIMARY KEY - innodb

I installed an InnoDB Cluster recently and trying to create a table without any primary key or equivalent to test the cluster index concept where "InnoDB internally generates a hidden clustered index named GEN_CLUST_INDEX on a synthetic column containing row ID values if the table has no PRIMARY KEY or suitable UNIQUE index".
I created table as below:
create table repl_test (Name varchar(10));
Checked for the creation of GEN_CLUST_INDEX:
select * from mysql.innodb_index_stats where database_name='test' and table_name = 'repl_test';
+---------------+------------+-----------------+---------------------+--------------+------------+-------------+-----------------------------------+
| database_name | table_name | index_name | last_update | stat_name | stat_value | sample_size | stat_description |
+---------------+------------+-----------------+---------------------+--------------+------------+-------------+-----------------------------------+
| test | repl_test | GEN_CLUST_INDEX | 2019-02-22 06:29:26 | n_diff_pfx01 | 0 | 1 | DB_ROW_ID |
| test | repl_test | GEN_CLUST_INDEX | 2019-02-22 06:29:26 | n_leaf_pages | 1 | NULL | Number of leaf pages in the index |
| test | repl_test | GEN_CLUST_INDEX | 2019-02-22 06:29:26 | size | 1 | NULL | Number of pages in the index |
+---------------+------------+-----------------+---------------------+--------------+------------+-------------+-----------------------------------+
3 rows in set (0.00 sec)
But, when I try to insert row, I get the below error:
insert into repl_test values ('John');
ERROR 3098 (HY000): The table does not comply with the requirements by an external plugin.
2019-02-22T14:32:53.177700Z 594022 [ERROR] [MY-011542] [Repl] Plugin group_replication reported: 'Table repl_test does not have any PRIMARY KEY. This is not compatible with Group Replication.'
Below is my conf file:
[client]
port = 3306
socket = /tmp/mysql.sock
[mysqld_safe]
socket = /tmp/mysql.sock
[mysqld]
socket = /tmp/mysql.sock
port = 3306
basedir = /mysql/product/8.0/TEST
datadir = /mysql/data/TEST/innodb_data
log-error = /mysql/admin/TEST/innodb_logs/mysql.log
log_bin = /mysql/binlog/TEST/innodb_logs/mysql-bin
server-id=1
max_connections = 500
open_files_limit = 65535
expire_logs_days = 15
innodb_flush_log_at_trx_commit=1
sync_binlog=1
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
binlog_checksum = NONE
enforce_gtid_consistency = ON
gtid_mode = ON
relay-log=<<hostname>>-relay-bin
My MySQL version: mysql Ver 8.0.11 for linux-glibc2.12 on x86_64 (MySQL Community Server - GPL)

Auto-generation of the PK should work fine for non-clustered setups. However, InnoDB Cluster (Group Replication) and Galera need a user PK (or UNIQUE that can be promoted).
If you like, file a documentation bug with bugs.mysql.com complaining that this restriction is not clear. Be sure to point to the page that needs fixing.

Related

How to fix mariadb replication: Could not execute Delete_rows_v1 event on table db.tableName; Index for table './db/tableName.MYI'

We have 2 mariadb servers in replication (master-slave). Those 2 servers were turn off unexpectedly. When mariadb servers get online myisam tables were checked on db1 and db2:
| 85 | db | ip:55336 | db | Query | 4398 | Checking table | tableName | 0.000 |
I have changed a slave to read master binary log from new file and new position (I think there was no lag on replication) but when I start slave on db2 I got replication error:
Could not execute Delete_rows_v1 event on table db.tableName; Index for table './db/tableName.MYI' is corrupt; try to repair it, Error_code: 126; handler error HA_ERR_WRONG_IN_RECORD; the event's master log binlog-file-01, end_log_pos 8980
Can you help me how can I fix it?
we deleted all rows in db1 for this table. Should I remove all rows for this table on db2? and then skip all steps in replication which are connected with that? there was lot of this rows.
Additionally:
On DB2:
Exec_Master_Log_Pos: 810
When I read event logs for this file on db1:
MariaDB [(none)]> SHOW BINLOG EVENTS IN 'binlog-file-01' from 810 limit 5;
+---------------------------+------+----------------+-----------+-------------+------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+---------------------------+------+----------------+-----------+-------------+------------------------------------------------+
| binlog-file-01 | 810 | Gtid | 1 | 852 | BEGIN GTID 0-1-5630806796 |
| binlog-file-01 | 852 | Annotate_rows | 1 | 908 | DELETE FROM tableName |
| binlog-file-01 | 908 | Table_map | 1 | 993 | table_id: 107 (db.tableName) |
| binlog-file-01 | 993 | Delete_rows_v1 | 1 | 8980 | table_id: 107 |
| binlog-file-01 | 8980 | Delete_rows_v1 | 1 | 17006 | table_id: 107 |
+---------------------------+------+----------------+-----------+-------------+------------------------------------------------+
5 rows in set (0.02 sec)
Can I just skip this on replication?
./db/tableName.MYI indicates a corrupt index. If check table db.tableName didn't resolve this you can try:
drop and recreate the indexes on this table on the replica
truncate table tableName (as the binary log appears to be a delete from tableName without a where clause.
If the replication still fails on this binary log entry you can skip it with:
SET GLOBAL sql_slave_skip_counter=1
reference: ref.
Recommend changing your MyISAM tables to Aria or InnoDB to be crash safe in the case of power failure. Also set sync_binlog=1 for persistent safety.

Query all primary roads from PostGis Database filled with OSM Data within a boundary

I imported OpenStreetMap data through osm2pgsql into PgSQL (PostGIS)
and I would like to get an SF object from the data containing
all primary roads (geometry) within a an area (bbox) into R.
I got lost since I would like to have also relations and nodes
and im not sure if only a query on planet_osm_roads will be sufficient and how the imported data structure is different to osm xml data im normaly working with.
I understand it is probably a bit broader question but
I would appreciate a start so to say to understand the query language better.
This is my approach but sadly i get an error
conn <- RPostgreSQL::dbConnect("PostgreSQL", host = MYHOST,
dbname = "osm_data", user = "postgres", password = MYPASSWORD)
pgPostGIS(conn)
a<-pgGetGeom(conn, c("public", "planet_osm_roads"), geom = "way", gid = "osm_id",
other.cols = FALSE, clauses = "WHERE highway = 'primary' && ST_MakeEnvelope(11.2364353533134, 47.8050651144447, 11.8882527806375, 48.2423300001326)")
a<-st_as_sf(a)
This is an error i get:
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: syntax error at or near "ST_MakeEnvelope"
LINE 2: ...lic"."planet_osm_roads" WHERE "way" IS NOT NULL ST_MakeEnv...
^
)
Error in pgGetGeom(conn, c("public", "planet_osm_roads"), geom = "way", :
No geometries found.
In addition: Warning message:
In postgresqlQuickSQL(conn, statement, ...) :
Could not create execute: SELECT DISTINCT a.geo AS type
FROM (SELECT ST_GeometryType("way") as geo FROM "public"."planet_osm_roads" WHERE "way" IS NOT NULL ST_MakeEnvelope(11.2364353533134, 47.8050651144447, 11.8882527806375, 48.2423300001326)) a;
This is the db:
osm_data=# \d
List of relations
Schema | Name | Type | Owner
----------+--------------------+----------+----------
public | geography_columns | view | postgres
public | geometry_columns | view | postgres
public | planet_osm_line | table | postgres
public | planet_osm_nodes | table | postgres
public | planet_osm_point | table | postgres
public | planet_osm_polygon | table | postgres
public | planet_osm_rels | table | postgres
public | planet_osm_roads | table | postgres
public | planet_osm_ways | table | postgres
public | spatial_ref_sys | table | postgres
topology | layer | table | postgres
topology | topology | table | postgres
topology | topology_id_seq | sequence | postgres
schema_name table_name geom_column geometry_type type
1 public planet_osm_line way LINESTRING GEOMETRY
2 public planet_osm_point way POINT GEOMETRY
3 public planet_osm_polygon way GEOMETRY GEOMETRY
4 public planet_osm_roads way LINESTRING GEOMETRY
Table "public.planet_osm_roads"
Column | Type | Collation | Nullable | Default
--------------------+---------------------------+-----------+----------+---------
osm_id | bigint | | |
access | text | | |
addr:housename | text | | |
addr:housenumber | text | | |
addr:interpolation | text | | |
admin_level | text | | |
aerialway | text | | |
aeroway | text | | |
amenity | text | | |
area | text | | |
barrier | text | | |
bicycle | text | | |
brand | text | | |
bridge | text | | |
boundary | text | | |
building | text | | |
construction | text | | |
Your query looks just fine. Check the following example:
WITH planet_osm_roads (highway,way) AS (
VALUES
('primary','SRID=3857;POINT(1283861.57 6113504.88)'::geometry), --inside your bbox
('secondary','SRID=3857;POINT(1286919.06 6067184.04)'::geometry) --somewhere else ..
)
SELECT highway,ST_AsText(way)
FROM planet_osm_roads
WHERE
highway IN ('primary','secondary','tertiary') AND
planet_osm_roads.way && ST_Transform(
ST_MakeEnvelope(11.2364353533134,47.8050651144447,11.8882527806375,48.2423300001326, 4326),3857
);
highway | st_astext
---------+------------------------------
primary | POINT(1283861.57 6113504.88)
This image illustrates the BBOX and the points used in the example above
Check the documentation for more information on the bbox intersection operator &&.
However, there are a few things to consider.
In case you're creating the BBOX yourself in order to have an area for ST_Contains, consider simply using ST_DWithin. It will basically do the same, but you only have to provide a reference point and the distance.
Depending on the distribution of highway types in the table planet_osm_roads and considering that your queries will always look for either primary,secondary or tertiary highways, creating a partial index could significantly improve performance. As the documentation says:
A partial index is an index built over a subset of a table; the subset
is defined by a conditional expression (called the predicate of the
partial index). The index contains entries only for those table rows
that satisfy the predicate. Partial indexes are a specialized feature,
but there are several situations in which they are useful.
So try something like this:
CREATE INDEX idx_planet_osm_roads_way ON planet_osm_roads USING gist(way)
WHERE highway IN ('primary','secondary','tertiary');
And also highway needs to be indexed. So try this ..
CREATE INDEX idx_planet_osm_roads_highway ON planet_osm_roads (highway);
.. or even another partial index, in case you can't delete the other data but you don't need it for anything:
CREATE INDEX idx_planet_osm_roads_highway ON planet_osm_roads (highway)
WHERE highway IN ('primary','secondary','tertiary');
You can always identify bottlenecks and check if the query planer is using your index with EXPLAIN.
Further reading
Getting all Buildings in range of 5 miles from specified coordinates
Buffers (Circle) in PostGIS
How can I set data from on table to another according spatial relation geometries in these tables
I figured it out.
This is how you would get an SF object out of PostGIS Database filled with OSM Data within 11.2364353533134,47.8050651144447,11.8882527806375,48.2423300001326 BBOX:
library(sf)
library(RPostgreSQL)
library(tictoc)
pw <- MYPASSWORD
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, dbname = "osm_data",
host = "localhost", port = 5432,
user = "postgres", password = pw)
tic()
sf_data = st_read(con, query = "SELECT osm_id,name,highway,way
FROM planet_osm_roads
WHERE highway = 'primary' OR highway = 'secondary' OR highway = 'tertiary'AND ST_Contains(
ST_Transform(
ST_MakeEnvelope(11.2364353533134,47.8050651144447,11.8882527806375,48.2423300001326,4326)
,3857)
,planet_osm_roads.way);")
toc()
RPostgreSQL::dbDisconnect(con)
I have to verify if BBOX values are actually getting considered .. im not sure.

Mariadb Galera Cluster Multi-Master Slow execution of queries

I have a problem with Galera multi-master. When I run query(SELECT) in node1 is executed for 0.025 sec , but on node2 same query is executed for 5.172 sec and in node3 result is 4.347 sec.
Node1 config :
[mysqld]
log_error=/var/log/mariadb.log
[galera]
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.2.47,192.168.2.48,192.168.2.49,172.29.44.11"
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
wsrep_cluster_name="CCCluster"
bind-address=0.0.0.0
wsrep_node_address="192.168.2.47"
wsrep_node_name=”node1"
wsrep_sst_method=rsync
Node2 config :
[mysqld]
log_error=/var/log/mariadb.log
[galera]
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.2.47,192.168.2.48,192.168.2.49,172.29.44.11"
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
wsrep_cluster_name="CCCluster"
bind-address=0.0.0.0
wsrep_node_address="192.168.2.48"
wsrep_node_name="node2"
wsrep_sst_method=rsync
Node3 config:
[mysqld]
log_error=/var/log/mariadb.log
[galera]
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.2.47,192.168.2.48,192.168.2.49,172.29.44.11"
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
wsrep_cluster_name="CCCluster"
bind-address=0.0.0.0
wsrep_node_address="192.168.2.49"
wsrep_node_name="node1"
wsrep_sst_method=rsync
The query I'am execute on nodes is:
select `users`.*, `upd`.`first_name`, `upd`.`middle_name`, `upd`.`last_name`,
CONCAT_WS(upd.first_name, upd.middle_name, upd.last_name) AS full_name
from `users`
inner join `users_personal_data` as `upd` ON `upd`.`user_id` = `users`.`id`
order by `id` desc
limit 20 offset 0;
Where does this problem come from?
Result from EXPLAIN SELECT is :
+------+-------------+-------+------+-------------------------------------+------+---------+------+------+-------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+-------------------------------------+------+---------+------+------+-------------------------------------------------+
| 1 | SIMPLE | users | ALL | PRIMARY | NULL | NULL | NULL | 1 | Using temporary; Using filesort |
| 1 | SIMPLE | upd | ALL | users_personal_data_user_id_foreign | NULL | NULL | NULL | 1 | Using where; Using join buffer (flat, BNL join) |
+------+-------------+-------+------+-------------------------------------+------+---------+------+------+-------------------------------------------------+

Activiti and candidate groups

In APS 1.8.1, I have defined a process where each task has a candidate group.
When I login in with a user that belongs to a candidate group, I cannot see the process instance.
I have found out that when I try to access the process instances, APS executes the following query in the database:
select distinct RES.* , DEF.KEY_ as PROC_DEF_KEY_, DEF.NAME_ as PROC_DEF_NAME_, DEF.VERSION_ as PROC_DEF_VERSION_, DEF.DEPLOYMENT_ID_ as DEPLOYMENT_ID_
from ACT_HI_PROCINST RES
left outer join ACT_RE_PROCDEF DEF on RES.PROC_DEF_ID_ = DEF.ID_
left join ACT_HI_IDENTITYLINK I_OR0 on I_OR0.PROC_INST_ID_ = RES.ID_
WHERE RES.TENANT_ID_ = 'tenant_1'
and
( (
exists(select LINK.USER_ID_ from ACT_HI_IDENTITYLINK LINK where USER_ID_ = '1003' and LINK.PROC_INST_ID_ = RES.ID_)
)
or (
I_OR0.TYPE_ = 'participant'
and
I_OR0.GROUP_ID_ IN ('1','2','2023','2013','2024','2009','2025','2026','2027','2028','2029','2007','2018','2020','2017','2015','2012','2003','2021','2019','2004','2002','2005','2030','2031','2032','2011','2006','2008','2014','2010','2016','2022','2033','2034','2035','2036','2037','1003')
) )
order by RES.START_TIME_ desc
LIMIT 50 OFFSET 0
This query does not return any record for two reasons:
In my ACT_HI_IDENTITYLINK no tasks have both the group_id_ and the proc_inst_id_ set.
The type of the record is "candidate" but the query is looking for "participant"
select * fro m ACT_HI_IDENTITYLINK;
-[ RECORD 1 ]-+----------
id_ | 260228
group_id_ |
type_ | starter
user_id_ | 1002
task_id_ |
proc_inst_id_ | 260226
-[ RECORD 2 ]-+----------
id_ | 260294
group_id_ | 2006
type_ | candidate
user_id_ |
task_id_ | 260293
proc_inst_id_ |
-[ RECORD 3 ]-+----------
id_ | 260300
group_id_ | 2009
type_ | candidate
user_id_ |
task_id_ | 260299
proc_inst_id_ |
-[ RECORD 4 ]-+----------
id_ | 262503
group_id_ |
type_ | starter
user_id_ | 1002
task_id_ |
proc_inst_id_ | 262501
-[ RECORD 5 ]-+----------
id_ | 262569
group_id_ | 2016
type_ | candidate
user_id_ |
task_id_ | 262568
proc_inst_id_ |
-[ RECORD 6 ]-+----------
id_ | 262575
group_id_ | 2027
type_ | candidate
user_id_ |
task_id_ | 262574
proc_inst_id_ |
Why the query is looking only for "participant" and why the records that have type_ = 'candidate' do not have any proc_inst_id_ set ?
UPDATE:
The problem with the constraint "participant" has a simple workaround: it would be enough to add the same candidate group as a participant.
See also Feature allowing "Participant" configuration in BPM Modeler
Unfortunately this is not enough to solve the second problem. The record is still not returned because the column proc_inst_id_ is not set.
I tried to update the column manually on the "participant" record and I have verified that doing so the page is accessible and works well.
Does anyone know why the column is not set ?
A possible solution (or workaround until ACTIVITI-696 is fixed) is to add each group added as candidate to a task as a participant of the process instance.
There is a REST API that does it:
POST /enterprise/process-instances/{processInstanceId}/identitylinks
What this API does should be done by a task listener that will automatically add the candidate groups of the created task as participant of the process instance.
To add the new identity link, use in the listener the following lines:
ActivitiEntityEvent aee = (ActivitiEntityEvent)activitiEvent;
TaskEntity taskEntity = (TaskEntity)aee.getEntity();
List<IdentityLinkEntity> identities = taskEntity.getIdentityLinks();
if (identities != null) {
for (IdentityLinkEntity identityLinkEntity : identities) {
String groupId = identityLinkEntity.getGroupId();
runtimeService.addGroupIdentityLink(activitiEvent.getProcessInstanceId(), groupId, "participant");
};
}
first try to check that your workflow is really started by access to "workflow I have started". You should see your task in "active task" if not, that means there is some errors in your definitions. If everything is ok, check your group name and don’t forget to add "GROUP_"myGRPName.
If you want to see the workflow instances it’s simpler with web script and services.

How is availability zone list order determined by the nova api in openstack?

I want to change the default option for availability zone in my openstack setup in horizon. However, I am having trouble finding out what determines the order of the availability zones as returned by the nova api. For example, running openstack availability zone list I get:
+--------------+-------------+
| Zone Name | Zone Status |
+--------------+-------------+
| zone2 | available |
| zone1 | available |
| internal | available |
| zone3 | available |
+--------------+-------------+
which is the same order as in horizon's dropdown box. However, querying the database directly, I get:
mysql> select * from aggregate_metadata;
+---------------------+------------+------------+----+--------------+-------------------+--------------+---------+
| created_at | updated_at | deleted_at | id | aggregate_id | key | value | deleted |
+---------------------+------------+------------+----+--------------+-------------------+--------------+---------+
| 2015-06-12 08:43:07 | NULL | NULL | 1 | 1 | availability_zone | zone1 | 0 |
| 2015-06-12 08:43:08 | NULL | NULL | 2 | 2 | availability_zone | zone2 | 0 |
| 2015-10-26 05:30:15 | NULL | NULL | 3 | 3 | availability_zone | zone3 | 0 |
+---------------------+------------+------------+----+--------------+-------------------+--------------+---------+
3 rows in set (0.00 sec)
Obviously, the openstack api is doing some sorting before returning the result... however, I can't figure out how it is being sorted nor how I could control the sorting.
get_availability_zones is the function used by nova api to collect list of availability zones.
This function gets list of available services(which is sorted based on the id) ,adds availability zone name is added to those services.
Since service list is the first step it's id defines the order and not the zone name.
The sort order can be modified in different ways based on the requirement.
Sort the order at frontend (horizon)
Modify this line with
ng-options="zone.value as zone.label for zone in model.availabilityZones | orderBy:'value'"
Sort the order at backend (nova-api)
Add available_zones.sort()not_available_zones.sort() before return statements in get_availability_zones function

Resources