.NET Core Console Application - SSL connect error - .net-core

Context:
I have an n-tier dynamic-ish application that currently builds its elastic indexes at app startup.
While this was ok to get things going, I'd prefer to have a small CLI application/util that I can later re-use in some scripts and or run ad-hoc if I need to do things like, creating a new instance, seeding mock data etc.
Problem:
When I use the exact same code that worked in my dotnetcore web app's startup, in my console app, I get the following issue:
Invalid NEST response built from a unsuccessful low level call on PUT: /jobs-ng
# Audit trail of this API call:
- [1] BadRequest: Node: https://localhost:9200/ Took: 00:00:01.3948230
# OriginalException: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: SSL connect error
at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at Elasticsearch.Net.HttpConnection.Request[TReturn](RequestData requestData) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Connection\HttpConnection-CoreFx.cs:line 78
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
Additional Info:
Currently my ElasticClient class only gets set like this:
var uri = new Uri("https://localhost:9200");
services.AddSingleton(new ElasticClient(new ConnectionSettings(uri)));
... thus I don't explicitly set a cert anywhere. (Hasn't been required thus far)
My best guess is that the native console app is missing something that the default web app projects provide, but as to what, I don't know?
I have set DisableDirectStreaming() as per the error log, but still no change.
I believe this might be a bug within .Net Core, but I'm hoping someone knows of a work around, or way around this issue for the time being (other that having this code be part of the we apps startup? (which is what my current working solution is))

There is currently a documented issue, within the dotnet-core project's CURL implementation/usage thereof/within CURL on OSX... you get the idea. It seems to have been fixed for 1.1, but as I'm on 2.0 (and I've attempted all of the suggested fixes on the various github threads, to no success), I'm unable to verify if that fix works.
Thus the answer:
There is a bug thats preventing this from working on OSX currently.
Workaround:
Since my console app / cli-tool will run ad hoc internal to our applications network / same machine instance, I don't, in theory need HTTPS for this utility, HTTP should be sufficient. (Once I made that change, the code worked fine across the board)

Related

Randomly occurring Unable to retrieve document from: '[PII is hidden]'

I came along with so many questions and answers but could not understand or find solution as my case is bit different.
I am using Azure with openIdconnect for Active directory authentication (ASP.Net framework 4.6.1)
it has been years it was working fine but a day before yesterday this error occurred for only about 1 hour. error was occurring when trying to do AD authentication.
"
Exception Message: IDX20803: Unable to obtain configuration from: '[PII is hidden]'.
Inner Exception: System.IO.IOException: IDX20804: Unable to retrieve document from: '[PII is hidden]'. ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
StackTrace: System.InvalidOperationException: IDX20803: Unable to obtain configuration from: '[PII is hidden]'. ---> System.IO.IOException: IDX20804: Unable to retrieve document from: '[PII is hidden]'. ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request)."
during this local authentication and remaining whole application was working fine. After 1 hour this issue was automatically resolved.
same happened yesterday. issue occurred for 1 hour and resolved automatically.
Other solution are suggesting to modify IdentityModelEventSource.ShowPII = true;
But I dont think so it requires code modification.
Please guide what can be reason that this happens for only short time.
Note: this is happening on only one server all applications on other servers are working fine
Please check if below are causes.
The error usually appears when application's configuration is not
able to retrieve the OIDC metadata properly. Please make sure
Instance,Domain,TenantId,ClientId are correct.
And Please make sure to use the latest version (or to 4.7.2) of your
dot-net framework as few tasks may get to require updated / latest
version of β€œ.NET” framework in order for them to work properly.
This error might occur if the application is running on TLS 1.1 or TLS 1.0,as they are depreciated. Use the protocol - TLS 1.2 for application.
In some cases packages maybe defaulting to TLS 1.1 even after that when loading that metadata and may take time to check for the correct one.
To resolve, try to add the following in Global.asax.cs which will allow the openid-configuration to be obtained as it is pointed to tls1.2 or above. Make sure to change the tls to 1.2 in portal also.
protected void Application_Start()
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Ssl3; // allow TLSV1.2 and SSL3 only
//other code
}
References:
azure ad enable-support-tls-environment | microsoft docs
Error: Unable to retrieve document from:
'https://login.microsoftonline.com/.well-known/openid-configuration'- Microsoft Q&A

Asp.net core api with GraphQL and CosmosDB giving SocketException (10013)

I've build a small graphQL api in asp.net core that gets some data from a Azure Cosmos database. Locally it runs fine with a CosmosEmulator.
If I connect the local running api directly to the CosmosDb running in Azure it works also fine.
When I deploy it through a devops pipeline, the graphQL api gives the following error:
GraphQL.ExecutionError: Error trying to resolve products.\r\n ---> Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: 00000000-0000-0000-0000-000000000000; Reason: (An attempt was made to access a socket in a way forbidden by its access permissions.);\r\n ---> System.Net.Sockets.SocketException (10013): An attempt was made to access a socket in a way forbidden by its access permissions.\r\n at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)\r\n --- End of inner exception stack trace ---\r\n at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)\r\n--- Cosmos Diagnostics ---{\"DiagnosticVersion\":\"2\",\"Summary\":{\"StartUtc\":\"2020-11-16T19:32:58.2689693Z\",\"RunningElapsedTimeInMs\":137.7423,\"UserAgent\":\"cosmos-netstandard-sdk/3.12.0|3.11.4|03|X86|Microsoft Windows 10.0.14393|.NET Core 3.1.8|\",\"TotalRequestCount\":0,\"FailedRequestCount\":0},\"Context\":[]}\r\n --- End of inner exception stack trace ---
It seems that something between api and database isn't working but I can't figure it out. Should I change settings in the webapp configuration in Azure?

IDX10108: The address specified is not valid as per HTTPS scheme

I hope someone can help me with solving my problem. I know similar questions have been posted before, and I looked thru them all. None of the solutions worked for me thus far.
I'm getting the following error when my client is trying to connect to IdentityServer4:
System.ArgumentException: IDX10108: The address specified is not valid as per HTTPS scheme. Please specify an https address for security reasons. If you want to test with http address, set the RequireHttps property on IDocumentRetriever to false.
Parameter name: http://subdomain.domain.com/.well-known/openid-configuration/jwks
at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.d__24.MoveNext()
Everything worked fine locally, when run from VS 2017. When I tried to deploy to staging environment and host website in IIS (7.5) I started getting this error.
My staging setup:
IdentityServer4 running on top of ASP.net core 2.0 as it's own instance.
My client running on top of ASP.net core 2.0 as it's own instance.
Any help would be greatly appreciated
Most likely you have found the solution by now, but I ran into same error today and saw this post. So adding what I found and maybe someone else would get benefit from it.
For me, the Authority URL (OpenIdConnectAuthenticationOptions.Authority) that I was forming from config values was not correct and causing this error, fixing that fixed my this error.

Insight messages not showing in Visual Studio or Azure

Im seeing AI messages in the debug window but not in the diagnostics window/ hub , Azure or the application insights VS screens,
Debug window
Application Insights Telemetry: {"name":"Microsoft.ApplicationInsights.e71aa06ae1d142b4ae04393d63a71d57.Exception","time":"2016-07-16T09:46:37.7435251Z","iKey":"xyz","tags": etc
Not using a config file
for (;;)
{
var telemetryClient = new TelemetryClient();
telemetryClient.InstrumentationKey = "abc";
telemetryClient.Context.Session.Id = Guid.NewGuid().ToString();
telemetryClient.Context.User.AccountId = "Ben";
telemetryClient.Context.Component.Version = "1.0";
telemetryClient.TrackEvent("Application Start");
telemetryClient.TrackException(new ApplicationException("test exc"));
telemetryClient.Flush();
Thread.Sleep(3000);
}
This is in several solutions One a Console the other .Core . Any ideas ?
Add diagnostics shows.
Value for property 'Failed to load an item from the storage. file: 20160531051527_dbd26772482e4dbab6791b63f0ded9d3.trn Exception: System.UriFormatException: Invalid URI: The format of the URI could not be determined.
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at Microsoft.ApplicationInsights.Channel.Storage.Peek()' of vshub.exe was not found. Populating it by default.: OnNext: {0}
Value for property 'Failed to load transmission from file. File full path: C:\Users\bklooste\AppData\Local\Microsoft\ApplicationInsights\vstelAIF-312cbd79-9dbb-4c48-a7da-3cc2a931cb70\20160712090841_9436f519411b4a10b4f551897382fcc6.trn, Exception: System.UriFormatException: Invalid URI: The format of the URI could not be determined.
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at Microsoft.ApplicationInsights.Channel.Storage.d__14.MoveNext()' of vshub.exe was not found. Populating it by default.: OnNext: {0}
Value for property 'Failed to load an item from the storage. file: 20160712090841_9436f519411b4a10b4f551897382fcc6.trn Exception: System.UriFormatException: Invalid URI: The format of the URI could not be determined.
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
Possibly not an answer but will help those who search for this stack trace and don't read the comments on the question.
The OPs question was resolved when they checked the Instrumentation Key.
The stack trace is generated by vshub.exe when it attempts to read the files at C:\Users\bklooste\AppData\Local\Microsoft\ApplicationInsights\vstelAIF-312cbd79β€Œβ€‹-9dbb-4c48-a7da-3cc2a931cb70 to send in the offline telemetry.
This offline telemetry is provided by the Application Insights Persisted HTTP package. Without knowing the actual contents of the file it seems likely there is some corruption in the file. The file should look something like the snippet below. If for some reason the file was truncated or the URI is actually malformed then this exception will be thrown.
You either need to correct the URI, or (preferrably) just delete the file.
https://vortex.data.microsoft.com/collect/v1
Content-Type:application/x-json-stream
Content-Encoding:gzip
H4sIAAAAAAAEAJV...

Google OpenId: No OpenID endpoint found (intermittent)

Usually using the Google OpenId works fine, thousands of times a day, then it will start intermittently going wrong and timing out for an hours or so (some requests will validate but not all). Repeated validation will eventually work.
Error messages are:
Event code: 200000
Event message: No OpenID endpoint found. : https://www.google.com/accounts/o8/id
Sequence contains no elements
Adding in log4net yields:
DotNetOpenAuth.Yadis:
Error while performing discovery on: "https://www.google.com/accounts/o8/id":
DotNetOpenAuth.Messaging.ProtocolException:
Error occurred while sending a direct message or getting the response.
---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at DotNetOpenAuth.Messaging.StandardWebRequestHandler.GetResponse
(HttpWebRequest request, DirectWebRequestOptions options)
in c:\...\Dot...Core\Messaging\StandardWebRequestHandler.cs:line 127
--- End of inner exception stack trace ---
at DotNetOpenAuth.Messaging.StandardWebRequestHandler.GetResponse
(HttpWebRequest request, DirectWebRequestOptions options)
in c:\...\Dot...Core\Messaging\StandardWebRequestHandler.cs:line 175
at DotNetOpenAuth.Messaging.UntrustedWebRequestHandler.GetResponse
(HttpWebRequest request, DirectWebRequestOptions options)
in c:\...\Dot...Core\Messaging\UntrustedWebRequestHandler.cs:line 250
at DotNetOpenAuth.Yadis.Yadis.Request
(IDirectWebRequestHandler requestHandler,
Uri uri, Boolean requireSsl, String[] acceptTypes)
in c:\...\Dot...OpenId\Yadis\Yadis.cs:line 172
at DotNetOpenAuth.Yadis.Yadis.Discover
(IDirectWebRequestHandler requestHandler, UriIdentifier uri, Boolean requireSsl)
in c:\...\DotNetOpenAuth.OpenId\Yadis\Yadis.cs:line 63
at DotNetOpenAuth.OpenId.UriDiscoveryService.Discover
(Identifier identifier, IDirectWebRequestHandler requestHandler,
Boolean& abortDiscoveryChain)
in c:\...\DotNet...OpenId\OpenId\UriDiscoveryService.cs:line 51
at DotNetOpenAuth.OpenId.IdentifierDiscoveryServices.Discover
(Identifier identifier)
in c:\...\Dot...OpenId\OpenId\IdentifierDiscoveryServices.cs:line 58
at DotNetOpenAuth.OpenId.RelyingParty.AuthenticationRequest.Create
(Identifier userSuppliedIdentifier, OpenIdRelyingParty relyingParty,
Realm realm, Uri returnToUrl, Boolean createNewAssociationsAsNeeded)
in ...OpenId.RelyingParty\OpenId\RelyingParty\AuthenticationRequest.cs:line 364
And
DotNetOpenAuth.Http WebException:
Timeout from https://www.google.com/accounts/o8/id, no response available.
Any ideas?
It sounds like you need to fix your network latency. It seems highly unlikely that Google would be the bottleneck here.
You may also want to increase the HTTP timeouts on your end to reduce the failure rate. The full set of options is available here. Specifically you're probably looking for:
<untrustedWebRequest
timeout="00:00:10"
readWriteTimeout="00:00:01.500" />
Check out the configurations link to see the context of where this goes.
We recently ran into this same issue. Having read several different scenarios and having gone through the trace steps I finally concluded, as I have seen elsewhere that this problem can be caused by a DNS server issue. In our case we had a production server that has been in use for over 18 months and just recently started getting the same issue as mentioned above, but it was very consistent on this one server. Another server on a another network and our development computers did not have any issues.
Long story short I changed the primary DNS for the production server to Google's public DNS, 8.8.8.8 and it instantly started working. I had manually flushed the DNS cache on the production server prior to this (without positive outcome) so it leads me to believe the DNS server (provided by our hosting center) had a bad cache entry that was ultimately causing the problem.
Hope this helps someone else who runs across this scenario.

Resources