BizTalk 2020 Sftp Receive Location Polling stopped working without any feedback - sftp

I'm using Sftp adapter in BizTalk 2020 for receiving and sending files.
In the past the corresponding Receive Location stopped working because of Connection Failures. A restart of the Receive Location was the solution here.
Therefore we enabled Logs within the Receive Location Configuration - now every time the polling is executed, some Logs were written into the configured file.
The Poll Interval is set to 20 mins.
Last week the Polling of the Receive Location stopped working again.
No logs were written into the configured log file after the last execution - which means that the polling wasn't executed anymore. The Eventviewer also didnt show any Logs.
The problem is that we don't have any BizTalk feedback (like disabling Receive Location etc.).
A restart of the Receive Location didn't help.
I needed to restart the Hostinstance.
Are there any common issues known for the constellation of the Usage of Sftp Adapter?
Is there a way to prevent getting this scenario again?

Related

Continuously running send pipeline instance

An instance of a BizTalk send pipeline has started to run continuously. On 09/12/2021 an attempt was made to send a file via SFTP, which retried several times but ultimately failed due to a network issue. The error from the event logs is:
The adapter failed to transmit message going to send port "Deliver Outgoing - SFTP" with URL "sftp://xxx.xxxxxx.co.nz:22/To_****/%SourceFileName%". It will be retransmitted after the retry interval specified for this Send Port. Details:"WinSCP.SessionRemoteException: Network error: Software caused connection abort.
For some reason BizTalk made another send attempt at 1:49pm on 10/12/2021 which succeeded as confirmed by the administrator of the SFTP site. Despite this, BizTalk continued making intermittent send attempts and the pipeline instance is still running. The same file has been sent 4 times to the SFTP server.
The pipeline instance in theory should have suspended at 9:47pm on 09/12/2021. I have been able to confirm definitively whether anybody resumed it, but it seems unlikely at this stage. In any case, after sending successfully the pipeline instance should have terminated and should not be re-executing intermittently.
Does anybody know what could account for this behaviour? This is occurring on BTS2020 with CU2 applied.
I've sent messages over SFTP where the WinSCP interpretation of the date-modified attribute doesn't work with a specific type of SFTP server.
With the WinSCP GUI a dialogue box appears and you can disregard this error, but this option isn't available with BizTalk's GUI. This error appears when a file with the same filename already exists on the server and is supposed to be overwritten.
My solution was to create a pipeline component that removed %SourceFileName% on the server. The pipeline component (just like WinSCP GUI) can disregard the modified-date.

How to recover with a retry from gremlin NoHostAvailableException

I am using Gremlin Java driver to connect to a local gremlin server.
Simple code flow
Creating client
Cluster cluster = Cluster.build().addContactPoint(<endp>).port(<port>).enableSsl(false).create()
Client client = cluster.connect();
Submit Script
client.submit("g.V().count()");
If when i submit the first time the Gremlin server is down, on subsequent retries after bringing back gremlin server, connection still fails to create.
Exception First attempt when Gremlin Server is down:
org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts are considered unavailable due to previous exceptions. Check the error log to find the actual reason
Exception After Gremlin server is brought back up:
tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts are considered unavailable due to previous exceptions
One thing to note is i do not create client on retry just do
Submit Script
client.submit("g.V().count()");
It is quite possible that Gremlin server may go down anytime, how to recover in such circumstances. Fundamentally is
NoHostAvailableExceptio
recoverable?
Normally, the Client should attempt to reconnect to a host that is previously marked unavailable. By default, it should be retrying the host at 1 second intervals as governed by this configuration: connectionPool.reconnectInterval. In your case, however I think you've uncovered a bug where the reconnect attempts are not started because the Client was never able to reach the host in the first place. As of 3.4.11, you can only remedy this by recreating the Client as you noted in your comments. I've created an issue to track this problem here: TINKERPOP-2569

BizTalk: Waiting for file written prior to the rest of the orchestration executing

What I am trying to do is ensure that my message is sent to it's destination prior to the rest of the orchestration executing.
My message is large (> 200M), in the orchestration I have the send shape to write the message to a physical location followed by a step to kick off another application to consume the file.
My problem is that this step runs prior to the file send port finishing its writing, and then the application fails to get the file.
I has followed Kent Weare's post: BizTalk Delivery Notification (Thanks Kent Weare), but was unsuccessful.
Could anyone give me some suggestions?

Different server & browser HTTP status codes

I have a small python web application running on nginx with unicorn. The web application refresh it's page automatically every 1 minute.
Every day I see that around the same hour, the browser reports a 504 Gateway Time-out error and the application stops refreshing obviously.
I checked it with both chrome and firefox on two different client machines and two different server machines and found out it happens almost everyday on the same time (different time for each web server).
The weird thing is that looking at the web server access log I identify these calls and they are reported with 200 OK status code.
Could it be the the browser reports a different error code than the server due to connection issues? Any ideas how should I keep investigating it?
We found out the indeed our server had a maintenance procedure which blocked the access to it. Although it finished the request after a while the browser "gave up" and returned a timeout error. Once the maintenance procedure was canceled - the issue was resolved.
Yes - the server is able to serve the page ok so returns 200, but the client cannot finish the connection.
It could be a part of your infrastructure (firewall?) is choosing to update or something, although the odds of this happening at the exact same time of your request is slim unless it's a long running request or gateway outage.

delays in receiving and processing BTS message

I have a strange problem , I receive messages using http receive dll on IIS virtual directory and it does not start processing on orchestration until 2 or more minutes after it is received i'm using custom pipeline and BizTalk 2009, so anyone can help me please.
i found out that what happened is that BizTalk host increase memory usage incrementally until it reach something like 1,3 GB then it raise memory over follow exception and i should restart all host instances to get back to work

Resources