Mule ESB 3.3 How to start transactional flows concurrently - asynchronous

Question

I would like to process set of messages concurrently, but I can't manage to make them transacional unless I set the VM to request-response... in which case the processing is not concurrent.
Mule documentation states that "Mule transactions are configured on synchronous endpoints", but I don't quite understand this restriction.
It is clear that inside a flow that you want to be transactional one should not spawn asynch flows, but it is not clear (to me) why one can't start (from a non tx main flow) any number of asynchronous flows each being transactional.
In other words why does this work fine:
but if I change the VM's to "one-way" it fails with:
org.mule.transaction.IllegalTransactionStateException: Can only bind "javax.sql.DataSource/java.sql.Connection" type resources
Is there a way around this?
XML for the flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule>
<spring:beans>
<spring:bean id="dataSource" name="dataSource" class="org.enhydra.jdbc.standard.StandardDataSource" destroy-method="shutdown">
<spring:property name="driverName" value="org.h2.Driver" />
<spring:property name="url" value="jdbc:h2:tcp://localhost/~/mule" />
<spring:property name="user" value="sa" />
<spring:property name="password">
<spring:value></spring:value>
</spring:property>
</spring:bean>
<spring:bean id="transactionFactory" name="transactionFactory" class="org.mule.transport.jdbc.JdbcTransactionFactory" />
</spring:beans>
<jdbc:connector name="dbConnector" dataSource-ref="dataSource" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database" />
<flow name="triggerFlow" doc:name="triggerFlow">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8081" doc:name="HTTP" />
<set-payload value="#[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]]" doc:name="[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"/>
<collection-splitter doc:name="Collection Splitter"/>
<vm:outbound-endpoint exchange-pattern="request-response" path="flow" doc:name="flow" />
</flow>
<flow name="flow" doc:name="flow">
<vm:inbound-endpoint exchange-pattern="request-response" path="flow" doc:name="flow">
<custom-transaction factory-ref="transactionFactory" action="ALWAYS_BEGIN" timeout="10" />
</vm:inbound-endpoint>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
<jdbc:outbound-endpoint exchange-pattern="request-response" queryKey="insert" queryTimeout="-1" connector-ref="dbConnector" doc:name="Insert 1">
<jdbc:transaction action="ALWAYS_JOIN" />
<jdbc:query key="insert" value="insert into test values (#[payload], 'Test 1')" />
</jdbc:outbound-endpoint>
<jdbc:outbound-endpoint exchange-pattern="request-response" queryKey="insert2" queryTimeout="-1" connector-ref="dbConnector" doc:name="Insert 2">
<jdbc:transaction action="ALWAYS_JOIN" />
<jdbc:query key="insert2" value="insert into test values (#[payload + 10], 'Test 2')" />
</jdbc:outbound-endpoint>
</flow>
</mule>
Thanks in advance.

Answer 1

Mule documentation states that "Mule transactions are configured on synchronous endpoints", but I don't quite understand this restriction.
This restriction is due to the fact that, in Mule like in Spring, and more generally in Java, transactions are thread-bound. With asynchronous flows, multiple threads are involved, hence the transaction-thread association can't be maintained.
So no, you can't split/fork/parallelize/asynchronous process messages and also have transactions in Mule.
org.mule.transaction.IllegalTransactionStateException: Can only bind "javax.sql.DataSource/java.sql.Connection" type resources
But this is unrelated to the first question IMO: this is because you forcefully, via a custom-transaction, try to enrol a VM endpoint in a JDBC transaction. This can not work. Use an XA transaction if you want to enrol heterogeneous resources.
EDIT: From what you've said in the comments, you do not want to enrol the VM endpoint in the transaction, so just enrol the JDBC endpoints as in here:
<transactional action="ALWAYS_BEGIN">
<jdbc:outbound-endpoint exchange-pattern="request-response"
queryKey="insert" queryTimeout="-1" connector-ref="dbConnector"
doc:name="insert into test values (1, 'Test 1')">
<jdbc:transaction action="ALWAYS_JOIN" />
<jdbc:query key="insert"
value="insert into test values (1, 'Test 1')" />
</jdbc:outbound-endpoint>
<jdbc:outbound-endpoint exchange-pattern="request-response"
queryKey="insert2" queryTimeout="-1" connector-ref="dbConnector"
doc:name="insert into test values (2, 'Test 2')">
<jdbc:transaction action="ALWAYS_JOIN" />
<jdbc:query key="insert2"
value="insert into test values (2, 'Test 2')" />
</jdbc:outbound-endpoint>
</transactional>
This works fine with a one-way inbound endpoint.

Answer 2

I managed to make it work. I had to add an intermediate async flow wich invokes the sync/tx flow:
I think this is ugly and unnecesary and that it would be perfectly ok to invoke it as the original post, but for reasons than are beyond me Mule makes you jump through hoops for this.
Following is the flow's XML:
<?xml version="1.0" encoding="UTF-8"?>
<mule>
<spring:beans>
<spring:bean id="dataSource" name="dataSource" class="org.enhydra.jdbc.standard.StandardDataSource" destroy-method="shutdown">
<spring:property name="driverName" value="org.h2.Driver" />
<spring:property name="url" value="jdbc:h2:tcp://localhost/~/mule" />
<spring:property name="user" value="sa" />
<spring:property name="password">
<spring:value></spring:value>
</spring:property>
</spring:bean>
<spring:bean id="transactionFactory" name="transactionFactory" class="org.mule.transport.jdbc.JdbcTransactionFactory" />
</spring:beans>
<jdbc:connector name="dbConnector" dataSource-ref="dataSource" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database" />
<flow name="triggerFlow" doc:name="triggerFlow">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8081" doc:name="HTTP" />
<set-payload value="#[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]]" doc:name="[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"/>
<collection-splitter doc:name="Collection Splitter"/>
<vm:outbound-endpoint exchange-pattern="one-way" path="async" doc:name="async" />
</flow>
<flow name="simpletransactionFlow1" doc:name="simpletransactionFlow1">
<vm:inbound-endpoint exchange-pattern="one-way" path="async" doc:name="async"/>
<vm:outbound-endpoint exchange-pattern="request-response" path="flow" doc:name="flow"/>
</flow>
<flow name="flow" doc:name="flow">
<vm:inbound-endpoint exchange-pattern="request-response" path="flow" doc:name="flow">
<custom-transaction factory-ref="transactionFactory" action="ALWAYS_BEGIN" timeout="10" />
</vm:inbound-endpoint>
<logger message="#[groovy:Thread.currentThread().getName()], payload=#[payload]" level="INFO" doc:name="Logger"/>
<jdbc:outbound-endpoint exchange-pattern="request-response" queryKey="insert" queryTimeout="-1" connector-ref="dbConnector" doc:name="Insert 1">
<jdbc:transaction action="ALWAYS_JOIN" />
<jdbc:query key="insert" value="insert into test values (#[payload], 'Test 1')" />
</jdbc:outbound-endpoint>
<jdbc:outbound-endpoint exchange-pattern="request-response" queryKey="insert2" queryTimeout="-1" connector-ref="dbConnector" doc:name="Insert 2">
<jdbc:transaction action="ALWAYS_JOIN" />
<jdbc:query key="insert2" value="insert into test values (#[payload + 10], 'Test 2')" />
</jdbc:outbound-endpoint>
</flow>
</mule>

Answer 3

This is discussed in the Mule documentation. It is a little more cumbersome that it needs to be perhaps, but it's not too bad, and only actually requires one VM Transport, not two:
I changed it because I wanted to use TCP instead of HTTP, but it's basically the same example.
The first flow receives untransacted TCP, does something with it (just a Byte Array To String here, but possibly you'd have validation or something), pushes the input to the VM Connector, then echos the input back to the TCP stream. Returning after the VM Connector guarantees that the TCP source that it can let go of its message, if it's doing a little low-level guaranteed delivery.
The VM Connector is one-way, and has a persistent store connected to it so it can't lose the message. This shows an error in Mule Studio 3.5, but it works fine.
Then the middle flow takes over, and contains a transaction so the VM Connector won't let go of the message unless the business logic subflow completes successfully.
Finally the subflow runs; currently just Thread.sleep()ing for 5 seconds so I can see it's all working. Telnetting in has an instant echo back to the Telnet console, then five seconds later another echo.
Hope that helps! I've only spent a couple of hours with Mule altogether, but this seems to be correct.