I'm currently facing an issue with grpc-web, and a loadbalancer.
Trying to call our grpc webservices from our gateway API, results in the following error:
Status(StatusCode="Unknown", Detail="Bad gRPC response. HTTP status code: 411")
It appears that the either of the following headers are required, content-length or Transfer-Encoding.
I have a method for setting metadata in my client.
private async Task<Metadata> SetMetadata()
{
//More stuff here
headers.Add("Transfer-Encoding", "chunked");
return headers;
}
Here is how i create my client:
private async Task<Services.Protobuf.ServiceClient> CreateClient()
{
var httpMessageHandler = new HttpClientHandler();
_grpcChannel ??= GrpcChannel.ForAddress(
await _serviceAddressProvider.GetServiceAddress<ServiceClient>() ??
throw new InvalidOperationException(),
new GrpcChannelOptions()
{
HttpHandler = new GrpcWebHandler(httpMessageHandler)
});
return new(_grpcChannel);
}
And here is how i use the two
var serviceClient = await CreateClient();
var request = new Request
{
//Request stuff
};
var getListReply = await serviceClient.GetListReplyAsync(request, await SetMetadata());
Now. The issue is that I cannot set either Transfer-Encoding or Content-Lenght headers. They simply get stripped somewhere.
If fiddler is running they get added (by fiddler i assume), and the request actually works. But if fiddler is not running, the headers are not there, and i get the above error. (I honestly don't understand the part with fiddler, i'm only reporting what i'm seeing).
Does anyone have any idea why this happens? and if it's even possible to add the headers i'm trying to add with grpc-web?
I don't know much about grpc-web but grpc-gateway does strip HTTP headers if they don't have a grpcmetadata prefix when it forwards the HTTP request to the grpc server
You can take a look at this issue thread https://github.com/grpc-ecosystem/grpc-gateway/issues/1244
Related
There must be something wrong with my c# code. I am trying to download some Json from an Azure Blob. I can download the Json in Postman and from MS Edge however, using my code there are no apparent errors in the request but there is no content in the response. Presumably there is something wrong with my code.
async Task GetJson()
{
var request = new HttpRequestMessage
{
Method = new HttpMethod("GET"),
RequestUri = new Uri("https://xxx.blob.core.windows.net/trading/m5.json")
};
request.Headers.Add("Accept", "application/json");
request.SetBrowserRequestMode(BrowserRequestMode.NoCors);
var response = await http.SendAsync(request);
var json = await response.Content.ReadAsStringAsync();
}
This was asked on GitHub and apparently it is by design.
When you remove request.SetBrowserRequestMode(BrowserRequestMode.NoCors); line you will see the No 'Access-Control-Allow-Origin' header is present error.
Specifiying BrowserRequestMode.NoCors does not let you bypass the Browser security rules. It just simplifies the request headers.
I've scoured stackoverflow looking for ways to make synchronous API calls in Blazor WASM, and come up empty. The rest is a fairly length explanation of why I think I want to achieve this, but since Blazor WASM runs single-threaded, all of the ways I can find to achieve this are out of scope. If I've missed something or someone spots a better approach, I sincerely appreciate the effort to read through the rest of this...
I'm working on a Blazor WASM application that targets a GraphQL endpoint. Access to the GraphQL endpoint is granted by passing an appropriate Authorization JWT which has to be refreshed at least every 30 minutes from a login API. I'm using a 3rd party GraphQL library (strawberry-shake) which utilizes the singleton pattern to wrap an HttpClient that is used to make all of the calls to the GraphQL endpoint. I can configure the HttpClient using code like this:
builder.Services
.AddFxClient() // strawberry-shake client
.ConfigureHttpClient((sp, client) =>
{
client.BaseAddress =
new Uri(
"https://[application url]/graphql"); // GraphQL endpoint
var token = "[api token]"; // token retrieved from login API
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
});
The trick now is getting the API token from the login API at least every 30 minutes. To accomplish this, I created a service that tracks the age of the token and gets a new token from the login API when necessary. Pared down, the essential bits of the code to get a token look like this:
public async Task<string> GetAccessToken()
{
if ((_expirationDateTime ?? DateTime.Now).AddSeconds(-300) < DateTime.Now)
{
try
{
var jwt = new
{
token =
"[custom JWT for login API validation]"
};
var payload = JsonSerializer.Serialize(jwt);
var content = new StringContent(payload, Encoding.UTF8, "application/json");
var postResponse = await _httpClient.PostAsync("https://[login API url]/login", content);
var responseString = await postResponse.Content.ReadAsStringAsync();
_accessToken = JsonSerializer.Deserialize<AuthenticationResponse>(responseString).access_token;
_expirationDateTime = DateTime.Now.AddSeconds(1800);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
return _accessToken;
}
So, now I need to wire this up to the code which configures the HttpClient used by the GraphQL service. This is where I'm running into trouble. I started with code that looks like this:
// Add login service
builder.Services.AddSingleton<FxAuthClient>();
// Wire up GraphQL client
builder.Services
.AddFxClient()
.ConfigureHttpClient(async (sp, client) =>
{
client.BaseAddress =
new Uri(
"https://[application url]/graphql");
var token = await sp.GetRequiredService<FxAuthClient>().GetAccessToken();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
});
This "works" when the application is loaded [somewhat surprisingly, since notice I'm not "await"ing the GetAccessToken()]. But the behavior if I let the 30 minute timer run out is that the first attempt I make to access the GraphQL endpoint uses the expired token and not the new token. I can see that GetAccessToken() refreshes expired token properly, and is getting called every time I utilize the FxClient, but except for the first usage of FxClient, the GetAccessToken() code actually runs after the GraphQL request. So in essence, it always uses the previous token.
I can't seem to find anyway to ensure that GetAccessToken() happens first, since in Blazor WASM you are confined to a single thread, so all of the normal ways of enforcing synchronous behavior fails, and there isn't an asynchronous way to configure the FxClient's HttpClient.
Can anyone see a way to get this to work? I'm thinking I may need to resort to writing a wrapper around the strawberry FxClient, or perhaps an asynchronous extension method that wraps the ConfigureHttpClient() function, but so far I've tried to avoid this [mostly because I kept feeling like there must be an "easier" way to do this]. I'm wondering if anyone knows away to force synchronous behavior of the call to the login API in Blazor WASM, sees another approach that would work, or can offer any other suggestion?
Lastly, it occurs to me that it might be useful to see a little more detail of the ConfigureHttpClient method. It is autogenerated, so I can't really change it, but here it is:
public static IClientBuilder<T> ConfigureHttpClient<T>(
this IClientBuilder<T> clientBuilder,
Action<IServiceProvider, HttpClient> configureClient,
Action<IHttpClientBuilder>? configureClientBuilder = null)
where T : IStoreAccessor
{
if (clientBuilder == null)
{
throw new ArgumentNullException(nameof(clientBuilder));
}
if (configureClient == null)
{
throw new ArgumentNullException(nameof(configureClient));
}
IHttpClientBuilder builder = clientBuilder.Services
.AddHttpClient(clientBuilder.ClientName, (sp, client) =>
{
client.DefaultRequestHeaders.UserAgent.Add(
new ProductInfoHeaderValue(
new ProductHeaderValue(
_userAgentName,
_userAgentVersion)));
configureClient(sp, client);
});
configureClientBuilder?.Invoke(builder);
return clientBuilder;
}
I have written a HTTP client, where I am reading the data response from a REST web service. My confusion arises after reading multiple blogs on EntityUtils.consume() and EntiryUtils.toString(). I wanted to know the following:
If EntityUtils.toString(..) ONLY is sufficient as it also closes the stream after reading char bytes. Or I should also do EntityUtils.consume(..) as a good practice.
If both toString() and consume() operation can be used. If yes, then what should be there order.
If I EntityUtils.toString() closes the stream; then why the next call in EntityUtils.consume(..) operations which is entity.isStreaming() still returns true?
Could anyone guide me here to use these operations in a standard way. I am using HTTP version 4+.
I have to use these configurations in multithreaded(web-app) environment.
Thanks
I looked at the recommended example from the apache httpclient commons website.
In the example, they used EntityUtils.toString(..) without needing to use EntityUtils.consume(..) before or after.
They mention that calling httpclient.close() ensures all resources are closed.
source: https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientWithResponseHandler.java
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpGet httpget = new HttpGet("http://httpbin.org/");
System.out.println("Executing request " + httpget.getRequestLine());
// Create a custom response handler
ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
#Override
public String handleResponse(
final HttpResponse response) throws ClientProtocolException, IOException {
int status = response.getStatusLine().getStatusCode();
if (status >= 200 && status < 300) {
HttpEntity entity = response.getEntity();
return entity != null ? EntityUtils.toString(entity) : null;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
}
};
String responseBody = httpclient.execute(httpget, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
} finally {
httpclient.close();
}
This is what is quoted for the above example:
This example demonstrates how to process HTTP responses using a response handler. This is the recommended way of executing HTTP requests and processing HTTP responses. This approach enables the caller to concentrate on the process of digesting HTTP responses and to delegate the task of system resource deallocation to HttpClient. The use of an HTTP response handler guarantees that the underlying HTTP connection will be released back to the connection manager automatically in all cases.
I have a spring-boot project using Apache Camel.
I want to read a message from an activemq queue containing a file and send it to a web server.
I am trying to find the proper way to do this.
I believe I can make something like:
from("activemq:queue").bean(MyBean.class, "process")
And manually build a http request but I can't help thinking there is probably a better way to do it. Like:
from("activemq:queue").bean(MyBean.class, "process")
.setHeader(Exchange.HTTP_METHOD,constant("POST"))
.to("http://localhost:8080/test");
But I don't know how to manipulate the "exchange" to have a valid http Message.
MyBean receives an Exchange object containing a JmsMessage. I see that there is also a HTTPMessage but I don't think I should build that manually. (It requires HTTPRequest and Response objects I am not sure how to get.)
Can someone shed some light on this problem?
Update
I am going for the bean solution.
from("activemq:queue").bean(MyBean.class, "sendMultipart");
public void sendMultipart(Exchange exchange) {
ByteArrayInputStream in = new ByteArrayInputStream((byte[]) exchange.getIn().getBody());
InputStreamBody contentBody = new InputStreamBody(in, ContentType.create("application/octet-stream"), "filename");
HttpEntity entity = MultipartEntityBuilder
.create()
.addPart("file", contentBody)
.build();
HttpPost httpPost = new HttpPost("http://localhost:8080/upload/");
httpPost.setEntity(entity);
CloseableHttpClient httpClient = HttpClients.createDefault();
try {
CloseableHttpResponse httpResponse = httpClient.execute(httpPost);
System.out.println(httpResponse);
} catch (IOException e) {
e.printStackTrace();
}
}
Updated post
I found this http://hilton.org.uk/blog/camel-multipart-form-data. It allows you to leverage the camel http component.
"jms:queue/SomeQ" ==> {
process(toMultipart)
setHeader(Exchange.CONTENT_TYPE, "multipart/form-data")
process((e: Exchange) => e.getIn.setHeader(Exchange.HTTP_URI,"http://localhost:8111/foo"))
to ("http:DUMMY")
}
def toMultipart(exchange: Exchange): Unit = {
val data = exchange.in[java.io.File]
val entity = MultipartEntityBuilder.create()
entity.addBinaryBody("file", data)
entity.addTextBody("name", "sample-data")
// Set multipart entity as the outgoing message’s body…
exchange.in = entity.build
}
Side note: this would really be a nice use-case to try-out reactive streams.
Original post
I am still having some problems understanding your actual problem. Perhaps some code might help:
I am now assuming you are receiving bytes in some character encoding and want to sent it onward to a dynamically established http-endpoint.
Is the following something you are looking for (code is in camel's scala-dsl)
"jms:queue/SomeQ" ==> {
convertBodyTo(classOf[String],"UTF-32" )
process((e: Exchange) => e.in = e.in[String].toUpperCase + "!")
process((e: Exchange) => e.getIn.setHeader(Exchange.HTTP_URI,"http://localhost:8111/foo"))
to ("http:DUMMY")
}
It will be send as an HTTP POST as the body is not null.
I receive it all well on another endpoint i created to ensure the code above is correct:
"jetty:http://localhost:8111/foo" ==> {
log("received on http 8111 endpoint ${body}")
}
I want to modify the response body from the token endpoint response.
I've tried to intercept the /Token request with a MessageHandler but it doesn't work.
I'm able to add some additional informations to the response by overriding the OAuthAuthorizationServerProvider.TokenEndpointmethod, but I'm not able to create my own response body.
Is there a way to intercept the /Token request?
Edit
I found out how to remove the response body content from the token endpoint response, like this: HttpContext.Current.Response.SuppressContent = true;
It seems the right way to achieve my goal, but now when I use the context.AdditionalResponseParameters.Add() method to add my custom information, the SuppressContent block any alterations.
Now I have something like this:
// Removing the body from the token endpoint response
HttpContext.Current.Response.SuppressContent = true;
// Add custom informations
context.AdditionalResponseParameters.Add("a", "test");
To simply add new items to the JSON token response, you can use TokenEndpointResponse instead of the TokenEndpoint notification.
If you're looking for a way to completely replace the token response prepared by the OAuth2 authorization server by your own one, there's sadly no easy way to do that because OAuthAuthorizationServerHandler.InvokeTokenEndpointAsync doesn't check the OAuthTokenEndpointContext.IsRequestCompleted property after invoking the TokenEndpointResponse notification.
https://github.com/aspnet/AspNetKatana/blob/dev/src/Microsoft.Owin.Security.OAuth/OAuthAuthorizationServerHandler.cs
This is a known issue, but it was too late to include it in Katana 3 when I suggested to fix it.
You should give Owin.Security.OpenIdConnect.Server a try: it's an a fork of the OAuthAuthorizationServerMiddleware designed for Katana 3.0 and 4.0.
https://www.nuget.org/packages/Owin.Security.OpenIdConnect.Server/1.0.2
Of course, it includes the correct check to allow bypassing the default token request processing (this was even one of the first things I fixed when forking it).
You were almost there +Samoji #Samoji and really helped/inspired me to get the answer.
// Add custom informations
context.AdditionalResponseParameters.Add("a", "test");
// Overwrite the old content
var newToken = context.AccessToken;
context.AdditionalResponseParameters.Add("access_token", newToken);
I found it just replaced my old token with my new.
This question is similar to How to extend IdentityServer4 workflow to run custom code
So you can create custom middleware and register it before OAuth2 service in Startup:
public void Configuration(IAppBuilder app)
{
....
app.Use(ResponseBodyEditorMiddleware.EditResponse);
app.UseOAuthAuthorizationServer(...);
...
}
where custom middleware is:
public static async Task EditResponse(IOwinContext context, Func<Task> next)
{
// get the original body
var body = context.Response.Body;
// replace the original body with a memory stream
var buffer = new MemoryStream();
context.Response.Body = buffer;
// invoke the next middleware from the pipeline
await next.Invoke();
// get a body as string
var bodyString = Encoding.UTF8.GetString(buffer.GetBuffer());
// make some changes to the body
bodyString = $"The body has been replaced!{Environment.NewLine}Original body:{Environment.NewLine}{bodyString}";
// update the memory stream
var bytes = Encoding.UTF8.GetBytes(bodyString);
buffer.SetLength(0);
buffer.Write(bytes, 0, bytes.Length);
// replace the memory stream with updated body
buffer.Position = 0;
await buffer.CopyToAsync(body);
context.Response.Body = body;
}
The best way to intercept request and response is via MessageHandler if you want to avoid doing so after a request has reached the IControllerFactory handler in the pipeline - obviously in that case use a custom 'Attribute'
I have used MessageHandlers in the past to intercept request to api/token, create a new request and get the response, create a new response.
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
//create a new auth request
var authrequest = new HttpRequestMessage();
authrequest.RequestUri = new Uri(string.Format("{0}{1}", customBaseUriFromConfig, yourApiTokenPathFromConfig));
//copy headers from the request into the new authrequest
foreach(var header in request.Headers)
{
authrequest.Headers.Add(header.Key, header.Value);
}
//add authorization header for your SPA application's client and secret verification
//this to avoid adding client id and secret in your SPA
var authorizationHeader =
Convert.ToBase64String(Encoding.UTF8.GetBytes(string.Format("{0}:{1}", _clientIdFromConfig, _secretKeyFromConfig)));
//copy content from original request
authrequest.Content = request.Content;
//add the authorization header to the client for api token
var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(request.Headers.Authorization.Scheme, authorizationHeader);
var response = await client.PostAsync(authrequest.RequestUri, authrequest.Content, cancellationToken);
if(response.StatusCode == HttpStatusCode.OK)
{
response.Headers.Add("MyCustomHeader", "Value");
//modify other attributes on the response
}
return response;
}
This works for me perfectly. There is, however, the configuration for this handler required in the WebApiConfig.cs file (RouteConfig.cs if you're using ASP.NET MVC).
Can you elaborate on what it is that does not work for you on the handler?