Queries return duplicates in JDO/Datanucleus/H2 - jdo

I'm adding 2 object to a database, a Person and a Student (subclass of Person). When I query on Person, it returns each instance e twice. When I query on Student, it return both instances, even though Person is not a sub-class of Student. The code is based on the jdo-test-template from datanucleus. I'm using Datanucleus 5.0.0m1.
tx.begin();
Person p = new Person(0, "Pete");
Student s = new Student(1, "Sarah");
pm.makePersistent(p);
pm.makePersistent(s);
Query<Person> qP = pm.newQuery(Person.class);
Collection<Person>cP = (Collection<Person>) qP.execute();
for (Person p2: cP) {
System.out.println("Person: " + p2.getName() + " " + p2.getId() + " " + System.identityHashCode(p2));
}
Query<Student> qS = pm.newQuery(Student.class);
Collection<Student>c = (Collection<Student>) qS.execute();
for (Student s2: c) {
System.out.println("Student: " + s2.getName() + " " + s2.getId() + " " + System.identityHashCode(s2));
}
tx.commit();
The Person class is unchanged from the example template:
#PersistenceCapable(detachable="true")
public class Person {
#PrimaryKey
Long id;
String name;
public Person(long id, String name) {
this.id = id;
this.name = name;
}
public String getName() {
return name;
}
public Long getId() {
return id;
}
}
The Student class:
#PersistenceCapable(detachable="true")
public class Student extends Person {
public Student(long id, String name) {
super(id, name);
}
}
I also added Student to the persistence.xml file:
<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
version="1.0">
<persistence-unit name="MyTest">
<!-- Add all of your model classes here -->
<class>mydomain.model.Person</class>
<class>mydomain.model.Student</class>
<exclude-unlisted-classes />
<properties>
<!-- Update these datastore details if different -->
<property name="javax.jdo.PersistenceManagerFactoryClass" value="org.datanucleus.api.jdo.JDOPersistenceManagerFactory"/>
<property name="javax.jdo.option.ConnectionURL" value="jdbc:h2:mem:nucleus"/>
<property name="javax.jdo.option.ConnectionDriverName" value="org.h2.Driver"/>
<property name="javax.jdo.option.ConnectionUserName" value="sa"/>
<property name="javax.jdo.option.ConnectionPassword" value=""/>
<property name="datanucleus.schema.autoCreateAll" value="true"/>
</properties>
</persistence-unit>
</persistence>
When running the program, I get the following output:
Person: Sarah 1 454305524
Person: Sarah 1 1536471117
Person: Pete 0 1961945640
Person: Pete 0 1898155970
Student: Pete 0 1898155970
Student: Sarah 1 1536471117
Looking at the System.identityHashCode(...), it is returning 4 distinct Java instances for the first query.
Am I doing anything wrong? Or is the output expected?
EDIT
I just confirmed that DataNucleus 4.1.8 behaves the same as 5.0.0m1
EDIT
From the logfile:
17:36:21,660 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compiling "SELECT FROM mydomain.model.Person"
17:36:21,668 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compile Time = 8 ms
17:36:21,668 (main) DEBUG [DataNucleus.Query] - QueryCompilation:
[symbols: this type=mydomain.model.Person]
17:36:21,669 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compiling "SELECT FROM mydomain.model.Person" for datastore
17:36:21,697 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compile Time for datastore = 28 ms
17:36:21,698 (main) DEBUG [DataNucleus.Query] - SELECT FROM mydomain.model.Person Query compiled to datastore query "SELECT 'mydomain.model.Person ' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0 UNION SELECT 'mydomain.model.Student' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0"
17:36:21,698 (main) DEBUG [DataNucleus.Persistence] - ExecutionContext.internalFlush() process started using ordered flush - 2 enlisted objects
17:36:21,698 (main) DEBUG [DataNucleus.Persistence] - ExecutionContext.internalFlush() process finished
17:36:21,698 (main) DEBUG [DataNucleus.Connection] - ManagedConnection found in the pool : "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#3c41ed1d [conn=org.datanucleus.store.rdbms.datasource.dbcp.PoolingDataSource$PoolGuardConnectionWrapper#22ff4249, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" for key="org.datanucleus.ExecutionContextImpl#40ef3420" in factory="ConnectionFactory:tx[org.datanucleus.store.rdbms.ConnectionFactoryImpl#5b12b668]"
17:36:21,698 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Executing "SELECT FROM mydomain.model.Person" ...
17:36:21,698 (main) DEBUG [DataNucleus.Datastore.Native] - BATCH [INSERT INTO PERSON ("NAME",ID) VALUES (<'Pete'>,<0>); INSERT INTO PERSON ("NAME",ID) VALUES (<'Sarah'>,<1>)]
17:36:21,699 (main) DEBUG [DataNucleus.Datastore] - Execution Time = 1 ms (number of rows = [1, 1]) on PreparedStatement "org.datanucleus.store.rdbms.ParamLoggingPreparedStatement#1573f9fc"
17:36:21,700 (main) DEBUG [DataNucleus.Datastore] - Closing PreparedStatement "org.datanucleus.store.rdbms.datasource.dbcp.DelegatingPreparedStatement#5939a379"
17:36:21,701 (main) DEBUG [DataNucleus.Datastore.Native] - SELECT 'mydomain.model.Person ' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0 UNION SELECT 'mydomain.model.Student' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0
17:36:21,702 (main) DEBUG [DataNucleus.Datastore.Retrieve] - Execution Time = 1 ms
17:36:21,705 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Execution Time = 7 ms
17:36:21,707 (main) DEBUG [DataNucleus.Cache] - Object with id "mydomain.model.Person:1" not found in Level 1 cache [cache size = 2]
17:36:21,707 (main) DEBUG [DataNucleus.Cache] - Object with id "mydomain.model.Person:1" not found in Level 2 cache
17:36:21,708 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Person#1f97cf0d" (id="mydomain.model.Person:1") added to Level 1 cache (loadedFlags="[YN]")
17:36:21,709 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Person#1f97cf0d" (id="1") added to Level 2 cache (fields="[0, 1]", version="")
17:36:21,711 (main) DEBUG [DataNucleus.Lifecycle] - Object "mydomain.model.Person#1f97cf0d" (id="mydomain.model.Person:1") has a lifecycle change : "HOLLOW"->"P_CLEAN"
17:36:21,711 (main) DEBUG [DataNucleus.Transaction] - Object "mydomain.model.Person#1f97cf0d" (id="1") enlisted in transactional cache
17:36:21,712 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Student#477b4cdf" (id="mydomain.model.Student:1") taken from Level 1 cache (loadedFlags="[YY]") [cache size = 3]
17:36:21,712 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Person#8dbdac1" (id="mydomain.model.Person:0") taken from Level 1 cache (loadedFlags="[YY]") [cache size = 3]
17:36:21,712 (main) DEBUG [DataNucleus.Cache] - Object with id "mydomain.model.Student:0" not found in Level 1 cache [cache size = 3]
17:36:21,713 (main) DEBUG [DataNucleus.Cache] - Object with id "mydomain.model.Student:0" not found in Level 2 cache
17:36:21,713 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Student#49dc7102" (id="mydomain.model.Student:0") added to Level 1 cache (loadedFlags="[YN]")
17:36:21,713 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Student#49dc7102" (id="0") added to Level 2 cache (fields="[0, 1]", version="")
17:36:21,713 (main) DEBUG [DataNucleus.Lifecycle] - Object "mydomain.model.Student#49dc7102" (id="mydomain.model.Student:0") has a lifecycle change : "HOLLOW"->"P_CLEAN"
17:36:21,713 (main) DEBUG [DataNucleus.Transaction] - Object "mydomain.model.Student#49dc7102" (id="0") enlisted in transactional cache
17:36:21,713 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compiling "SELECT FROM mydomain.model.Student"
17:36:21,713 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compile Time = 0 ms
17:36:21,713 (main) DEBUG [DataNucleus.Query] - QueryCompilation:
[symbols: this type=mydomain.model.Student]
17:36:21,713 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compiling "SELECT FROM mydomain.model.Student" for datastore
17:36:21,714 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Compile Time for datastore = 1 ms
17:36:21,714 (main) DEBUG [DataNucleus.Query] - SELECT FROM mydomain.model.Student Query compiled to datastore query "SELECT 'mydomain.model.Student' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0"
17:36:21,714 (main) DEBUG [DataNucleus.Connection] - ManagedConnection found in the pool : "org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl#3c41ed1d [conn=org.datanucleus.store.rdbms.datasource.dbcp.PoolingDataSource$PoolGuardConnectionWrapper#22ff4249, commitOnRelease=false, closeOnRelease=false, closeOnTxnEnd=true]" for key="org.datanucleus.ExecutionContextImpl#40ef3420" in factory="ConnectionFactory:tx[org.datanucleus.store.rdbms.ConnectionFactoryImpl#5b12b668]"
17:36:21,714 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Executing "SELECT FROM mydomain.model.Student" ...
17:36:21,714 (main) DEBUG [DataNucleus.Datastore] - Closing PreparedStatement "org.datanucleus.store.rdbms.datasource.dbcp.DelegatingPreparedStatement#6b8ca3c8"
17:36:21,715 (main) DEBUG [DataNucleus.Datastore.Native] - SELECT 'mydomain.model.Student' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0
17:36:21,715 (main) DEBUG [DataNucleus.Datastore.Retrieve] - Execution Time = 1 ms
17:36:21,715 (main) DEBUG [DataNucleus.Query] - JDOQL Query : Execution Time = 1 ms
17:36:21,715 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Student#49dc7102" (id="mydomain.model.Student:0") taken from Level 1 cache (loadedFlags="[YY]") [cache size = 4]
17:36:21,715 (main) DEBUG [DataNucleus.Cache] - Object "mydomain.model.Student#477b4cdf" (id="mydomain.model.Student:1") taken from Level 1 cache (loadedFlags="[YY]") [cache size = 4]
I think it is interesting that the Student query is executed on the PERSON table without further parameters. Not sure how they map inheritance in the database, but if all classes are mapped into a single table then I would at least expect one column with the type identifier:
17:36:21,715 (main) DEBUG [DataNucleus.Datastore.Native] - SELECT 'mydomain.model.Student' AS NUCLEUS_TYPE,A0.ID,A0."NAME" FROM PERSON A0

You haven't specified the inheritance strategy and how you want your persistence mechanism to distinguish between classes sharing a table.
Use #Inheritance and #Discriminator as per these docs. While it will default the inheritance strategy to be NEW_TABLE for the base class and SUPERCLASS_TABLE for the sub class, it will NOT default any discriminator because maybe you didn't want to have one and never have a need for separating what is stored in that table

Related

dcm4che 5.22.0 dcmqrscp IncompatibleSpecificCharaterSetException

I am trying to use the new dcm4che release 5.22.0 and more specifically the dcmqrscp tool:
https://github.com/dcm4che/dcm4che/blob/master/dcm4che-tool/dcm4che-tool-dcmqrscp/README.md
I am starting an instance:
dcmqrscp -b DCMROUTER:435 --dicomdir /media/cdrom/DICOMDIR
But i am getting the following exception during the association request (no matter the study i am trying to send):
11:12:29,896 INFO - Accept connection Socket[addr=/127.0.0.1,port=52127,localport=4006]
11:12:29,917 DEBUG - /127.0.0.1:4006<-/127.0.0.1:52127(1): enter state: Sta2 - Transport connection open
11:12:29,937 INFO - DCMROUTER<-MAYAM(1) >> A-ASSOCIATE-RQ
11:12:29,942 DEBUG - A-ASSOCIATE-RQ[
calledAET: DCMROUTER
callingAET: MAYAM
applicationContext: 1.2.840.10008.3.1.1.1 - DICOM Application Context Name
implClassUID: 1.2.40.0.13.1.1
implVersionName: dcm4che-2.0
maxPDULength: 16384
maxOpsInvoked/maxOpsPerformed: 0/0
PresentationContext[id: 1
as: 1.2.840.10008.5.1.4.1.1.4 - MR Image Storage
ts: 1.2.840.10008.1.2 - Implicit VR Little Endian
]
PresentationContext[id: 3
as: 1.2.840.10008.5.1.4.1.1.4 - MR Image Storage
ts: 1.2.840.10008.1.2.1 - Explicit VR Little Endian
]
]
11:12:29,942 DEBUG - DCMROUTER<-MAYAM(1): enter state: Sta3 - Awaiting local A-ASSOCIATE response primitive
11:12:29,942 INFO - DCMROUTER<-MAYAM(1) << A-ASSOCIATE-AC
11:12:29,942 DEBUG - A-ASSOCIATE-AC[
calledAET: DCMROUTER
callingAET: MAYAM
applicationContext: 1.2.840.10008.3.1.1.1 - DICOM Application Context Name
implClassUID: 1.2.40.0.13.1.3
implVersionName: dcm4che-5.22.0
maxPDULength: 16378
maxOpsInvoked/maxOpsPerformed: 0/0
PresentationContext[id: 1
result: 0 - acceptance
ts: 1.2.840.10008.1.2 - Implicit VR Little Endian
]
PresentationContext[id: 3
result: 0 - acceptance
ts: 1.2.840.10008.1.2.1 - Explicit VR Little Endian
]
]
11:12:29,943 DEBUG - DCMROUTER<-MAYAM(1): enter state: Sta6 - Association established and ready for data transfer
11:12:29,947 INFO - DCMROUTER<-MAYAM(1) >> 1:C-STORE-RQ[pcid=3, prior=0
cuid=1.2.840.10008.5.1.4.1.1.4 - MR Image Storage
iuid=2.25.236888348187743071893701493905889154404.2 - ?
tsuid=1.2.840.10008.1.2.1 - Explicit VR Little Endian]
11:12:29,968 DEBUG - DCMROUTER<-MAYAM(1) >> 1:C-STORE-RQ Command:
(0000,0002) UI [1.2.840.10008.5.1.4.1.1.4] AffectedSOPClassUID
(0000,0100) US [1] CommandField
(0000,0110) US [1] MessageID
(0000,0700) US [0] Priority
(0000,0800) US [0] CommandDataSetType
(0000,1000) UI [2.25.236888348187743071893701493905889154404.2] AffectedSOPIns
11:12:29,969 DEBUG - DCMROUTER<-MAYAM(1) >> 1:C-STORE-RQ Dataset receiving...
11:12:29,969 INFO - DCMROUTER<-MAYAM(1): M-WRITE \media\cdrom\2.25.236888348187743071893701493905889154404.2
11:12:29,988 INFO - DCMROUTER<-MAYAM(1): M-RENAME \media\cdrom\2.25.236888348187743071893701493905889154404.2
11:12:30,072 INFO - M-UPDATE \media\cdrom\DICOMDIR: add STUDY Record
11:12:30,074 INFO - DCMROUTER<-MAYAM(1): M-DELETE \media\cdrom\DICOM\67EEF683\3AA0D620\7E6BFF30
11:12:30,075 INFO - DCMROUTER<-MAYAM(1): processing 1:C-STORE-RQ[pcid=3, prior=0
cuid=1.2.840.10008.5.1.4.1.1.4 - MR Image Storage
iuid=2.25.236888348187743071893701493905889154404.2 - ?
tsuid=1.2.840.10008.1.2.1 - Explicit VR Little Endian] failed. Caused by:
org.dcm4che3.net.service.DicomServiceException: org.dcm4che3.data.IncompatibleSpecificCharaterSetException: Specific Character Sets [] and [ISO_IR 100] not compatible
at org.dcm4che3.tool.dcmqrscp.DcmQRSCP$CStoreSCPImpl.store(DcmQRSCP.java:176)
at org.dcm4che3.net.service.BasicCStoreSCP.onDimseRQ(BasicCStoreSCP.java:72)
at org.dcm4che3.net.service.DicomServiceRegistry.onDimseRQ(DicomServiceRegistry.java:86)
at org.dcm4che3.net.ApplicationEntity.onDimseRQ(ApplicationEntity.java:474)
at org.dcm4che3.net.Association.onDimseRQ(Association.java:746)
at org.dcm4che3.net.PDUDecoder.decodeDIMSE(PDUDecoder.java:467)
at org.dcm4che3.net.Association.handlePDataTF(Association.java:729)
at org.dcm4che3.net.State$4.onPDataTF(State.java:103)
at org.dcm4che3.net.Association.onPDataTF(Association.java:725)
at org.dcm4che3.net.PDUDecoder.nextPDU(PDUDecoder.java:177)
at org.dcm4che3.net.Association$2.run(Association.java:562)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.dcm4che3.data.IncompatibleSpecificCharaterSetException: Specific Character Sets [] and [ISO_IR 100] not compatible
at org.dcm4che3.data.Attributes.add(Attributes.java:2103)
at org.dcm4che3.data.Attributes.addSelected(Attributes.java:2060)
at org.dcm4che3.media.RecordFactory.createRecord(RecordFactory.java:255)
at org.dcm4che3.tool.dcmqrscp.DcmQRSCP.addDicomDirRecords(DcmQRSCP.java:1087)
at org.dcm4che3.tool.dcmqrscp.DcmQRSCP$CStoreSCPImpl.store(DcmQRSCP.java:167)
... 13 more
11:12:30,083 INFO - DCMROUTER<-MAYAM(1) << 1:C-STORE-RSP[pcid=3, status=110H, errorComment=org.dcm4che3.data.IncompatibleSpecificCharaterSetException: Spec
tsuid=1.2.840.10008.1.2.1 - Explicit VR Little Endian]
11:12:30,083 DEBUG - DCMROUTER<-MAYAM(1) << 1:C-STORE-RSP Command:
(0000,0100) US [32769] CommandField
(0000,0120) US [1] MessageIDBeingRespondedTo
(0000,0800) US [257] CommandDataSetType
(0000,0900) US [272] Status
(0000,0902) LO [org.dcm4che3.data.IncompatibleSpecificCharaterSetException: Sp
11:12:30,088 INFO - DCMROUTER<-MAYAM(1) >> A-RELEASE-RQ
11:12:30,088 DEBUG - DCMROUTER<-MAYAM(1): enter state: Sta8 - Awaiting local A-RELEASE response primitive
11:12:30,088 INFO - DCMROUTER<-MAYAM(1) << A-RELEASE-RP
11:12:30,089 DEBUG - DCMROUTER<-MAYAM(1): enter state: Sta13 - Awaiting Transport Connection Close Indication
11:12:30,093 DEBUG - DCMROUTER<-MAYAM(1): closing Socket[addr=/127.0.0.1,port=52127,localport=4006] in 50 ms
11:12:30,142 INFO - DCMROUTER<-MAYAM(1): close Socket[addr=/127.0.0.1,port=52127,localport=4006]
11:12:30,142 DEBUG - DCMROUTER<-MAYAM(1): enter state: Sta1 - Idle
Any ideas what has changed in 5.22.0 and might cause this error?
As commented by #thanili:
It was a bug, fixed hopefully now. Please refer to github source code for more details.
Looking at source code on github, it seems the exception is recently added.
package org.dcm4che3.data;
/**
* #author Gunter Zeilinger (gunterze#protonmail.com)
* #since Mar 2020
*/
public class IncompatibleSpecificCharaterSetException extends IllegalArgumentException {
public IncompatibleSpecificCharaterSetException(String s) {
super(s);
}
}
Apparently, you need to set the supported Specific Character Sets yourself.
public void setSpecificCharacterSet(String... codes) {
ensureModifiable();
decodeStringValuesUsingSpecificCharacterSet();
setString(Tag.SpecificCharacterSet, VR.CS, codes);
}
Without it, it throws exception as here and here.
On command line as mentioned in readme:
--fs-desc-cs <code> Character Set used in File-set
Descriptor File ("ISO_IR 100" =
ISO Latin 1)

[ERROR]: Unknown column 'DamageModifier' in 'field list'

I installed the DB with updates and I got this error:
Opening DatabasePool 'wotlk_world'. Asynchronous connections: 1, synchronous connections: 1. MySQL client library: 5.6.42 MySQL server ver: 5.6.42 MySQL client library: 5.6.42 MySQL server ver: 5.6.42 [ERROR]: In mysql_stmt_prepare() id: 4, sql: "SELECT entryorguid, source_type, id, link, event_type, event_phase_mask, event_chance, event_flags, event_param1, event_param2, event_param3, event_param4, event_param5, action_type, action_param1, action_param2, action_param3, action_param4, action_param5, action_param6, target_type, target_param1, target_param2, target_param3, target_param4, target_x, target_y, target_z, target_o FROM smart_scripts ORDER BY entryorguid, source_type, id, link" [ERROR]: Unknown column 'event_param5' in 'field list' [ERROR]: In mysql_stmt_prepare() id: 54, sql: "SELECT difficulty_entry_1, difficulty_entry_2, difficulty_entry_3, KillCredit1, KillCredit2, modelid1, modelid2, modelid3, modelid4, name, subname, IconName, gossip_menu_id, minlevel, maxlevel, exp, faction, npcflag, speed_walk, speed_run, scale, rank, mindmg, maxdmg, dmgschool, attackpower, DamageModifier, BaseAttackTime, RangeAttackTime, unit_class, unit_flags, unit_flags2, dynamicflags, family, trainer_type, trainer_spell, trainer_class, trainer_race, minrangedmg, maxrangedmg, rangedattackpower, type, type_flags, lootid, pickpocketloot, skinloot, resistance1, resistance2, resistance3, resistance4, resistance5, resistance6, spell1, spell2, spell3, spell4, spell5, spell6, spell7, spell8, PetSpellDataId, VehicleId, mingold, maxgold, AIName, MovementType, InhabitType, HoverHeight, HealthModifier, ManaModifier, ArmorModifier, RacialLeader, movementId, RegenHealth, mechanic_immune_mask, flags_extra, ScriptName FROM creature_template WHERE entry = ?" [ERROR]: Unknown column 'DamageModifier' in 'field list' DatabasePool wotlk_world NOT opened. There were errors opening the MySQL connections. Check your SQLDriverLogFile for specific errors. Cannot connect to world database
127.0.0.1;3306;root;ascent;wotlk_world
The problem is:
[ERROR]: Unknown column 'DamageModifier' in 'field list'
It looks like your world DB is not up to date. So you need to update it properly. To do that you can either use the DB assembler script (bin/acore-db-asm) or manually importing the missing sql files from data/sql/updates/db_world.
To make sure your DB is up to date, check the name of the last column of the table version_db_world of your world database. It should match with the most recent sql file name of the direcotry data/sql/updates/db_world.
I recommend reading:
How to make sure that the DB is up to date

MariaDB + MaxScale Replication Error : The slave I/O thread stops because a fatal error is encountered when it tried to SELECT #master_binlog_checksum

I am trying to setup Real-time Data Streaming to Kafka with MaxScale CDC with MariaDB veriosn 10.0.32. After configuring replication, I am getting the status:
"The slave I/O thread stops because a fatal error is encountered when it tried to SELECT #master_binlog_checksum".
Below are all of my configurations:
MariaDB - Configuration
server-id = 1
#report_host = master1
#auto_increment_increment = 2
#auto_increment_offset = 1
log_bin = /var/log/mysql/mariadb-bin
log_bin_index = /var/log/mysql/mariadb-bin.index
binlog_format = row
binlog_row_image = full
# not fab for performance, but safer
#sync_binlog = 1
expire_logs_days = 10
max_binlog_size = 100M
# slaves
#relay_log = /var/log/mysql/relay-bin
#relay_log_index = /var/log/mysql/relay-bin.index
#relay_log_info_file = /var/log/mysql/relay-bin.info
#log_slave_updates
#read_only
MaxScale Configuration
[server1]
type=server
address=192.168.56.102
port=3306
protocol=MariaDBBackend
[Replication]
type=service
router=binlogrouter
version_string=10.0.27-log
user=myuser
passwd=mypwd
server_id=3
#binlogdir=/var/lib/maxscale
#mariadb10-compatibility=1
router_options=binlogdir=/var/lib/maxscale,mariadb10-compatibility=1
#slave_sql_verify_checksum=1
[Replication Listener]
type=listener
service=Replication
protocol=MySQLClient
port=5308
Starting Replication
CHANGE MASTER TO MASTER_HOST='192.168.56.102', MASTER_PORT=5308, MASTER_USER='myuser', MASTER_PASSWORD='mypwd', MASTER_LOG_POS=328, MASTER_LOG_FILE='mariadb-bin.000018';
START SLAVE;
Replication Status
Master_Host: 192.168.56.102
Master_User: myuser
Master_Port: 5308
Connect_Retry: 60
Master_Log_File: mariadb-bin.000018
Read_Master_Log_Pos: 328
Relay_Log_File: mysqld-relay-bin.000002
Relay_Log_Pos: 4
Relay_Master_Log_File: mariadb-bin.000018
**Slave_IO_Running: No**
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 328
Relay_Log_Space: 248
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1593
Last_IO_Error: **The slave I/O thread stops because a fatal error is encountered when it tried to SELECT #master_binlog_checksum. Error:**
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 0
Master_SSL_Crl:
Master_SSL_Crlpath:
Using_Gtid: No
Gtid_IO_Pos:
The binlogrouter performs the following query to set the value of #master_binlog_checksum (real replication slaves perform the same query).
SET #master_binlog_checksum = ##global.binlog_checksum
Checking what the output of it is will probably explain why the replication won't start. Most likely the SET query failed which is why the latter SELECT #master_binlog_checksum query returns unexpected results.
In cases like these, it is recommended to open a bug report on the MariaDB Jira under the MaxScale project. This way the possibility of a bug is ruled out and if it turns out to be a configuration problem, the documentation can be updated to more clearly explain how to configure MaxScale.

DSS boxcarring skipping operation

I'm using DSS 3.5.0 with PostgreSQL, and a set of operations in a request box is not working in a very peculiar way. I've successfully used request boxes containing thousands of operations in this same project, including operations very similar to those that fail. One of these large request boxes failed, and after spending some time looking for the operations that caused the problem, we were able to reduce it to a set of five operations.
The problem
Looking at the PostgreSQL logs, the query issued by one of the operations is not executed because it never reaches the database.
I'll call the operations O1, O2, O3, O4 and O5 and their queries Q1, Q2, Q3, Q4 and Q5. Playing with the request and checking the resulting database log, we ended up with:
Request box contains O1-O2-O3-O4-O5: database executes Q1-Q2-Q3-Q5
Request box contains O1-O2-O4-O5: database executes Q1-Q2-Q4-Q5
Request box contains O1-O2-O3-O4: database executes Q1-Q2-Q3-Q4
Request box contains O1-O2-O3-O4-O4-O5: database executes Q1-Q2-Q3-Q5
So, it looks weird and it doesn't seem to follow a clearly discernible pattern.
All operations perform correctly if sent separately to the DSS, or in two different request boxes. The exact nature of the operations doesn't seem to be directly linked to the problem because the same operations are used countless times in other scenarios. The queries are not especially long or complex.
Operation 1: updates a record in table A
Operation 2: deletes a record from table B
Operation 3: inserts a record in table B
Operation 4: inserts a record in table A
Operation 5: inserts a record in table B (same as operation 3)
Errors and logs
The actual error message issued by PostgreSQL for operation 5 is
ERROR: null value in column "element_id" violates not-null constraint
This is expected because operation 4 (the one that disappears) inserts a value that is later used to resolve element_id for operation 5.
The PostgreSQL log reports:
LOG: execute <unnamed>: BEGIN
LOG: execute <unnamed>: UPDATE public.project_element SET element_uuid=$1,location_id=$2,from_revit=$3,name=$4,type=$5,model=NULLIF($6,0),parent_element=(SELECT PE.ELEMENT_ID FROM PROJECT_ELEMENT PE WHERE PE.PROJECT_ID = $7 AND (PE.ELEMENT_ID = $8 OR (PE.ELEMENT_UUID = $9 AND PE.ELEMENT_UUID IS NOT NULL))) ,left_border=$10,right_border=$11 WHERE element_id=$12
DETAIL: parameters: $1 = '(element-uuid)', $2 = '85', $3 = '1', $4 = '(some-text)', $5 = '3', $6 = '0', $7 = '22', $8 = NULL, $9 = '(parent-uuid)', $10 = NULL, $11 = NULL, $12 = '9983'
LOG: execute <unnamed>: DELETE FROM ELEMENT_PROPERTY WHERE ELEMENT_ID = (SELECT PE.ELEMENT_ID FROM PROJECT_ELEMENT PE WHERE PE.ELEMENT_ID = $1 AND PE.PROJECT_ID = $2) AND NAME = $3
DETAIL: parameters: $1 = '9983', $2 = '22', $3 = 'num_ports'
LOG: execute <unnamed>: INSERT INTO public.element_property(name,value,type,element_id) VALUES($1,$2,$3,(^M SELECT PE.ELEMENT_ID FROM PROJECT_ELEMENT PE WHERE PE.PROJECT_ID = $4 AND (PE.ELEMENT_ID = $5 OR (PE.ELEMENT_UUID = $6 AND PE.ELEMENT_UUID IS NOT NULL))))
DETAIL: parameters: $1 = 'num_ports', $2 = '48', $3 = '0', $4 = '22', $5 = NULL, $6 = '(element-uuid)'
LOG: execute <unnamed>: INSERT INTO public.element_property(name,value,type,element_id) VALUES($1,$2,$3,(SELECT PE.ELEMENT_ID FROM PROJECT_ELEMENT PE WHERE PE.PROJECT_ID = $4 AND (PE.ELEMENT_ID = $5 OR (PE.ELEMENT_UUID = $6 AND PE.ELEMENT_UUID IS NOT NULL))))
DETAIL: parameters: $1 = 'port_num', $2 = '6', $3 = '0', $4 = '22', $5 = NULL, $6 = '(other-uuid)'
ERROR: null value in column "element_id" violates not-null constraint
DETAIL: Failing row contains (port_num, 6, 0, null).
STATEMENT: INSERT INTO public.element_property(name,value,type,element_id) VALUES($1,$2,$3,(SELECT PE.ELEMENT_ID FROM PROJECT_ELEMENT PE WHERE PE.PROJECT_ID = $4 AND (PE.ELEMENT_ID = $5 OR (PE.ELEMENT_UUID = $6 AND PE.ELEMENT_UUID IS NOT NULL))))
LOG: execute S_2: BEGIN
LOG: execute S_1: ROLLBACK
DSS log starts with an exception, but I'm not sure if it's really related to this problem. The following log goes from the request box start to the first time it complains about the error message returned from PostgreSQL. DSS complains multiple times after that.
DEBUG - {org.apache.axis2.transport.http.AxisServlet}
java.lang.NullPointerException
at javax.servlet.GenericServlet.getServletContext(GenericServlet.java:123)
at org.apache.axis2.transport.http.AxisServlet.createMessageContext(AxisServlet.java:715)
at org.apache.axis2.transport.http.AxisServlet$RestRequestProcessor.<init>(AxisServlet.java:819)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:227)
at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.ui.filters.CSRFPreventionFilter.doFilter(CSRFPreventionFilter.java:88)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.ui.filters.CRLFPreventionFilter.doFilter(CRLFPreventionFilter.java:59)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1739)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1698)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:744)
DEBUG - Input contentType (application/json) {org.apache.axis2.builder.BuilderUtil}
DEBUG - CharSetEncoding defaulted (UTF-8) {org.apache.axis2.builder.BuilderUtil}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Checking for Service using target endpoint address : /services/iims {org.apache.axis2.dispatchers.RequestURIBasedServiceDispatcher}
DEBUG - org.apache.axis2.i18n.resource::handleGetObject(servicefound) {org.apache.axis2.i18n.ProjectResourceBundle}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Found AxisService : iims {org.apache.axis2.engine.AbstractDispatcher}
DEBUG - Attempt to check for Operation using HTTP Location failed {org.apache.axis2.dispatchers.HTTPLocationBasedDispatcher}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Attempted to check for Operation using target endpoint URI, but the operation fragment was missing {org.apache.axis2.dispatchers.RequestURIBasedOperationDispatcher}
DEBUG - getAction (null) from org.apache.axis2.client.Options#279e70a {org.apache.axis2.client.Options}
DEBUG - SoapAction is (null) {org.apache.axis2.context.MessageContext}
DEBUG - createSOAPEnvelope using Builder (class org.apache.axis2.json.JSONOMBuilder) selected from type (application/json) {org.apache.axis2.transport.TransportUtils}
DEBUG - getAction (null) from org.apache.axis2.client.Options#279e70a {org.apache.axis2.client.Options}
DEBUG - SoapAction is (null) {org.apache.axis2.context.MessageContext}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Checking for Operation using Action : null {org.apache.axis2.dispatchers.ActionBasedOperationDispatcher}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Attempted to check for Operation using target endpoint URI, but the operation fragment was missing {org.apache.axis2.dispatchers.RequestURIBasedOperationDispatcher}
DEBUG - Axis operation is null {org.apache.axis2.json.gson.JSONMessageHandler}
DEBUG - No headers present corresponding to http://www.w3.org/2005/08/addressing {org.apache.axis2.handlers.addressing.AddressingInHandler}
DEBUG - No headers present corresponding to http://schemas.xmlsoap.org/ws/2004/08/addressing {org.apache.axis2.handlers.addressing.AddressingInHandler}
DEBUG - getAction (null) from org.apache.axis2.client.Options#279e70a {org.apache.axis2.client.Options}
DEBUG - SoapAction is (null) {org.apache.axis2.context.MessageContext}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Checking for Operation using Action : null {org.apache.axis2.dispatchers.ActionBasedOperationDispatcher}
DEBUG - getAction (null) from org.apache.axis2.client.Options#279e70a {org.apache.axis2.client.Options}
DEBUG - SoapAction is (null) {org.apache.axis2.context.MessageContext}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Checking for Operation using Action : null {org.apache.axis2.dispatchers.ActionBasedOperationDispatcher}
DEBUG - Get operation for request_box {org.apache.axis2.description.AxisService}
DEBUG - Found axis operation: org.apache.axis2.description.InOutAxisOperation#682d0c2c {org.apache.axis2.description.AxisService}
DEBUG - org.apache.axis2.i18n.resource::handleGetObject(operationfound) {org.apache.axis2.i18n.ProjectResourceBundle}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] Found AxisOperation : request_box {org.apache.axis2.engine.AbstractDispatcher}
DEBUG - getAddressingRequirementParemeterValue: value: 'null' {org.apache.axis2.addressing.AddressingHelper}
DEBUG - [MessageContext: logID=f9462531f982d008b3e2aacd88bfd07f4a7e4905c354170e] isReplyRedirected: ReplyTo is null. Returning false {org.apache.axis2.addressing.AddressingHelper}
DEBUG - getAction (null) from org.apache.axis2.client.Options#112f42cb {org.apache.axis2.client.Options}
DEBUG - Old WSAAction is (null) {org.apache.axis2.context.MessageContext}
DEBUG - New WSAAction is (urn:request_boxResponse) {org.apache.axis2.context.MessageContext}
DEBUG - setAction Old action is (null) {org.apache.axis2.client.Options}
DEBUG - setAction New action is (urn:request_boxResponse) {org.apache.axis2.client.Options}
DEBUG - messageID is null. {org.apache.axis2.context.ConfigurationContext}
DEBUG - forceExpand: changing prefix from to {org.apache.axiom.om.impl.llom.OMSourcedElementImpl}
DEBUG - DXXATransactionManager.begin() {org.wso2.carbon.dataservices.core.description.xa.DSSXATransactionManager}
DEBUG - Creating data source connection {org.wso2.carbon.dataservices.core.description.config.SQLConfig}
ERROR - ERROR: null value in column "element_id" violates not-null constraint_ Detalhe: Failing row contains (port_num, 6, 0, null). (Sanitized) {org.wso2.carbon.dataservices.core.description.query.SQLQuery}
org.postgresql.util.PSQLException: ERROR: null value in column "element_id" violates not-null constraint
The implementation
This is the actual request box that fails (some field contents replaced to reduce noise):
{
"request_box":{
"update_project_element_operation":{
"name":"(some-text)",
"element_id":9983,
"element_uuid":"(element-uuid)",
"from_revit":1,
"project_id":22,
"parent_element_uuid":"(parent-uuid)",
"type":3,
"location_id":85,
"model":0
},
"delete_element_property_operation":{
"name":"num_ports",
"element_id":9983,
"project_id":22
},
"insert_element_property_operation":{
"project_id":22,
"element_uuid":"(element-uuid)",
"name":"num_ports",
"value":"48"
},
"insert_project_element_operation":{
"name":"(this operation disappears)",
"element_id":0,
"element_uuid":"(other-uuid)",
"from_revit":1,
"project_id":22,
"parent_element_uuid":"(element-uuid)",
"type":10,
"location_id":85,
"model":0
},
"insert_element_property_operation":{
"project_id":22,
"element_uuid":"(other-uuid)",
"name":"port_num",
"value":"6"
}
}
}
I can provide detailed table, query and operation definitions if necessary. All operations were used before, and each one of them work if issued separately or in two different request boxes. It seems to be a issue directly linked to DSS boxcarring.
Any ideas?
After a few weeks of investigation, including direct contact to the WSO2 support, we concluded that this unusual problem was caused by the JSON to XML conversion inside DSS. This may be related to the fact that the request box representation in JSON format can contain non-unique names (and according to RFC 7159 the behavior in this case is unpredictable and implementation-defined). It should be noted that we also used a request box with thousands of repetitions of the same name without any visible problem, so it isn't a straightforward consequence of all non-unique names not being correctly processed.
When we tried the same request box in XML, all operations were correctly executed. To avoid changing the application, we followed WSO2's advice and had the ESB converting the application-generated JSON to XML. Preliminar tests showed that in this case the XML was correctly generated, however we decided to slightly adjust the JSON generator to issue an array of operation objects instead of an object containing members with non-unique names, to avoid the undefined behavior and the possibility of new, unpredictable problems in JSON parsing.
WSO2 is aware of this problem and it may or may not be fixed by an upcoming release of DSS. Until then, the safer way to avoid request box suprises seems to be to use XML instead of JSON when sending transactions to DSS using request boxes.

Index state never change to ENABLED on Titan with Amazon DynamoDB backend

I'm trying to use composite index on DynamoDB and the index never switches from from INSTALLED to REGISTERED state.
Here is the code I used to create it
graph.tx().rollback(); //Never create new indexes while a transaction is active
TitanManagement mgmt=graph.openManagement();
PropertyKey propertyKey=getOrCreateIfNotExist(mgmt, "propertyKeyName");
String indexName = makePropertyKeyIndexName(propertyKey);
if (mgmt.getGraphIndex(indexName)==null) {
mgmt.buildIndex(indexName, Vertex.class).addKey(propertyKey).buildCompositeIndex();
mgmt.commit();
graph.tx().commit();
ManagementSystem.awaitGraphIndexStatus(graph, indexName).status(SchemaStatus.REGISTERED).call();
}else {
mgmt.rollback();
}
A sample of the log is:
...
...
612775 [main] INFO
com.thinkaurelius.titan.graphdb.database.management.GraphIndexStatusWatcher
- Some key(s) on index myIndex do not currently have status REGISTERED: type=INSTALLED 613275 [main] INFO
com.thinkaurelius.titan.graphdb.database.management.GraphIndexStatusWatcher
- Some key(s) on index typeIndex do not currently have status REGISTERED: type=INSTALLED 613275 [main] INFO
com.thinkaurelius.titan.graphdb.database.management.GraphIndexStatusWatcher
- Timed out (PT1M) while waiting for index typeIndex to converge on status REGISTERED
Waiting for a longer time does the trick. Example:
ManagementSystem.awaitGraphIndexStatus(graph, propertyKeyIndexName)
.status(SchemaStatus.ENABLED)
.timeout(10, ChronoUnit.MINUTES) // set timeout to 10 min
.call();

Resources