Why do calls to Google APIs from Compute Engine fail intermittently with error "Network is unreachable" and an IPv6 subnet? - google-cloud-datastore

Some calls to Google APIs made from Google Compute Engine instances fail intermittently with the error message:
cURL error 7: Failed to connect to 2607:f8b0:400c:c0b::5f: Network is unreachable (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
I'm encountering this error intermittently when trying to access Datastore, but other people have encountered this error calling other Google APIs from Compute Engine instances. The error seems to occur when we are making multiple calls to the Datastore API in a short period of time (though the instances are otherwise not highly utilized). There are no other indications of problems for other services running on the same VMs at the time of these errors.
The call is being made from CentOS 7 VM instances running PHP code on Apache using Google's PHP SDK. The VMs do not have public IP addresses. Most of the time, we can access Datastore and other APIs from the same VMs without problems, so we know that the network and VM configuration are correct.
Specific sub-questions:
What causes these errors?
Why are we getting a "Network is unreachable" error, instead of a meaningful error message from Datastore?
Why is there an IPv6 subnet in the error message? We haven't enabled IPv6, and Google doesn't seem to support connecting to Google APIs and services from an external IPv6 address.

Related

Trouble connecting to gRPC server on AWS Fargate

I have a Python gRPC server running on AWS Fargate (configured very similar to this AWS guide here), and another AWS Fargate task (call it the "client") that attempts to make a connection to my gRPC server (also using Python gRPC). However, the client is unable to make a call to my server, with the following error:
<_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"#1619057124.216955000","description":"Failed to pick subchannel",
"file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5397,
"referenced_errors":[{"created":"#1619057124.216950000","description":"failed to connect to all addresses",
"file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc",
"file_line":398,"grpc_status":14}]}"
Based on my reading online, it seems like there are myriad situations in which this error is thrown, and I'm having trouble figuring out which one pertains to my case. Here is some additional information:
When running client and server locally, I am able to successfully connect by having the client connect to localhost:[PORT]
I have configured an application load balancer target group following the guide from AWS here that makes health check requests to the / route of my gRPC server, using the gRPC protocol, and expect gRPC response code 12 (UNIMPLEMENTED); these health check requests are coming back as expected, which I believe implies the load balancer is able to successfully communicate with the server (although I could be misunderstanding)
I configured a service discovery system (following this guide here) that should allow me to reach my gRPC server within my VPC via the name service-name.dev.co.local. I can confirm that the corresponding DNS record exists in Route 53, and when I SSH into my VPC, I am indeed able to ping service-name.dev.co.local successfully.
Anyone have any ideas? Would appreciate any and all advice, and I'm happy to answer any further questions.
Thank you for your help!
on your grpc server use 0.0.0.0:[port] and expose this port with TCP on your container.

Docusign Connect / webhook error: The underlying connection was closed: An unexpected error occurred on a send

Not sure if this is a Docusign or ngix question.I'm working on integrating an application with Docusign and I keep seeing this somewhat vague error below in the Docusign Connect logs. In our nginx logs I see that a POST to our application's /webhook endpoint was attempted but doesn't go through. I've specified TLS 1.2 and have tried increasing our nginx timeout but that doesn't seem to fix it.
One theory I have is that our server's certificate isn't chained to a Microsoft trusted CA but I would expect a different error if that was the case.
Any help or guidance would be greatly appreciated.
This most likely would require you to get the IT folks managing this server and networking involved. And yes, they may need to install a certificate, but other errors can be related to the firewall blocking certain requests, an anti-virus blocking requests and even DNS related error preventing the HTTP request from being sent and received by the server.
We highly recommend to use a public cloud for Connect and we have plenty of examples how this can work, while still having your code run on your own IT server.

Firebase deploy error. Server Error. Client network socket disconnected before secure TLS connection was established

I'm trying to deploy a firebase function from my terminal using this comment firebase deploy
But getting this error. Cant understand why its happening.
deploying functions
functions: ensuring required API cloudfunctions.googleapis.com is enabled...
functions: required API cloudfunctions.googleapis.com is enabled
functions: preparing functions directory for uploading...
functions: packaged functions (27.96 KB) for uploading
functions: Upload Error: Server Error. Client network socket disconnected before secure TLS connection was established
Error: Server Error. Client network socket disconnected before secure TLS connection was established
After spending lots of time and investigation I have found that this is a network issue. I am guessing you are trying it from Bangladesh Network.
For some reason Govt has blocked the Google Cloud Storage Upload API, so it is failing to upload.
You should try with VPN. Use Free VPN like Hotspot Shield and try to upload the cloud function. I have finally able to upload my function through VPN. Please try.
I had the same error for quite a while and still happened sometimes.
For a rescue, I use a VPN. It is a network issue. Could be happened because of the ISP or may be a country wise block of Google Cloud Storage AP
From firebase debug log I have found a bit more about this error. I am not sure what could have done for this.
Request causing trouble or error
REQUEST PUT https://storage.googleapis.com/gcf-upload-us-central1-1.zip?
[warn] ! functions: Upload Error: Server Error. Client network socket disconnected before secure TLS connection was established

AMQP handshake timeout error while deploying AWS Corda Enterprise Template

I am deploying AWS Corda Enterprise Template. The Quick start deployed the stack as per the defined CloudFormation template. I can see 2 AWS instances, up and running as Corda nodes, in Hot-Cold setup with a load balancer.
However the Log for Corda node has following ERROR related to AMQP communication.
[ERROR] 2018-10-18T05:47:55,743Z [Thread-3
(ActiveMQ-scheduled-threads)] core.server.lambda$channelActive$0 -
AMQ224088: Timeout (10 seconds) while handshaking has occurred. {}
What can be possible reason for this error? This error keeps on occurring after a certain time interval. So it looks like some connectivity issue to me.
Note: The load balancer shows the status of this AWS Corda instances as healty (In Service). So I believe the Corda node has booted up successfully.
The ERROR message isn't necessarily tied to AMQP. Perhaps you were confused by the "AMQ" in the error ID (AMQ224088)?
In any event, this error indicates that something on the network is connecting to the ActiveMQ Artemis broker, but it's not completing any protocol handshake. This is commonly seen with, for example, load balancers that do a health check by creating a socket connection without sending any real data just to see if the port is open on the target machine.

Configure HTTP Access to Analysis Services - Won't work with basic authentication

Followed the instructions in https://msdn.microsoft.com/en-us/library/gg492140.aspx
SSAS is running on a hosted server. No domain.
Able to connect with anonymous authentication, but when basic is enabled, neither SSMS or Excel will connect.
Error from SSMS is "The remote server returned an error (401). Unauthorized"
Credentials are set up on the remote machine as DOMAIN:443
Credentials have admin privileges on the server.
Excel says "connection cannot be made to remote machine or SSAS is not running"
HTTP or HTTPS doesn't matter.
When it connects with anonymous, no cubes are returned so it is apparently doing nothing with the userid/password. Just connecting.
When I connect via port 2383 using the same credentials, I have no problem in SSMS or Excel.
Help is GREATLY appreciated. Thanks.
This was partially solved by starting over and following the steps in this article:
https://technet.microsoft.com/en-us/library/cc917670.aspx
Honestly, I can't pin down the exact resolution, but I believe it has to do with using a different folder structure (other than inetpub/wwwroot...) on my D drive and therefore losing the standard security configuration.
I created an OLAP folder on C:/inetpub/wwwroot and it works... with the exception of SSL which still does not work.
HTTP connects fine, but not HTTPS. Still working on that.

Resources