I want to send to webdav server a 100 files in one request. I'm getting list of messages, then I creating binary body parts of them. This is my code which gets me 301 http error response code. When I send one file it's working but expected behaviour is to send parts of files. And I want it to be created one file by one on the server, is it possible?
CloseableHttpClient client = HttpClients.createDefault();
HttpPut httpPost = new HttpPut("http://localhost:8888/webdav"); // I also tried with / at the end
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
messages.forEach(m -> {
builder.addBinaryBody("file", new File(Paths.get(m.getPath()).toAbsolutePath().toString()),
ContentType.APPLICATION_OCTET_STREAM, m.getFileName() + ".encryptedByAes");
builder.addBinaryBody("file", new File(Paths.get(m.getAesPath()).toAbsolutePath().toString()),
ContentType.APPLICATION_OCTET_STREAM, m.getFileName() + ".aes");
});
HttpEntity multipart = builder.build();
httpPost.setEntity(multipart);
UsernamePasswordCredentials creds
= new UsernamePasswordCredentials("test", "test");
httpPost.addHeader(new BasicScheme().authenticate(creds, httpPost, null));
CloseableHttpResponse response = client.execute(httpPost);
if (response.getStatusLine().getStatusCode() != HTTP_OK) {
throw new WebDavException("Error while executing request to webdav server with messages");
}
client.close();
Related
I'm currently facing an issue with grpc-web, and a loadbalancer.
Trying to call our grpc webservices from our gateway API, results in the following error:
Status(StatusCode="Unknown", Detail="Bad gRPC response. HTTP status code: 411")
It appears that the either of the following headers are required, content-length or Transfer-Encoding.
I have a method for setting metadata in my client.
private async Task<Metadata> SetMetadata()
{
//More stuff here
headers.Add("Transfer-Encoding", "chunked");
return headers;
}
Here is how i create my client:
private async Task<Services.Protobuf.ServiceClient> CreateClient()
{
var httpMessageHandler = new HttpClientHandler();
_grpcChannel ??= GrpcChannel.ForAddress(
await _serviceAddressProvider.GetServiceAddress<ServiceClient>() ??
throw new InvalidOperationException(),
new GrpcChannelOptions()
{
HttpHandler = new GrpcWebHandler(httpMessageHandler)
});
return new(_grpcChannel);
}
And here is how i use the two
var serviceClient = await CreateClient();
var request = new Request
{
//Request stuff
};
var getListReply = await serviceClient.GetListReplyAsync(request, await SetMetadata());
Now. The issue is that I cannot set either Transfer-Encoding or Content-Lenght headers. They simply get stripped somewhere.
If fiddler is running they get added (by fiddler i assume), and the request actually works. But if fiddler is not running, the headers are not there, and i get the above error. (I honestly don't understand the part with fiddler, i'm only reporting what i'm seeing).
Does anyone have any idea why this happens? and if it's even possible to add the headers i'm trying to add with grpc-web?
I don't know much about grpc-web but grpc-gateway does strip HTTP headers if they don't have a grpcmetadata prefix when it forwards the HTTP request to the grpc server
You can take a look at this issue thread https://github.com/grpc-ecosystem/grpc-gateway/issues/1244
I have written a HTTP client, where I am reading the data response from a REST web service. My confusion arises after reading multiple blogs on EntityUtils.consume() and EntiryUtils.toString(). I wanted to know the following:
If EntityUtils.toString(..) ONLY is sufficient as it also closes the stream after reading char bytes. Or I should also do EntityUtils.consume(..) as a good practice.
If both toString() and consume() operation can be used. If yes, then what should be there order.
If I EntityUtils.toString() closes the stream; then why the next call in EntityUtils.consume(..) operations which is entity.isStreaming() still returns true?
Could anyone guide me here to use these operations in a standard way. I am using HTTP version 4+.
I have to use these configurations in multithreaded(web-app) environment.
Thanks
I looked at the recommended example from the apache httpclient commons website.
In the example, they used EntityUtils.toString(..) without needing to use EntityUtils.consume(..) before or after.
They mention that calling httpclient.close() ensures all resources are closed.
source: https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientWithResponseHandler.java
CloseableHttpClient httpclient = HttpClients.createDefault();
try {
HttpGet httpget = new HttpGet("http://httpbin.org/");
System.out.println("Executing request " + httpget.getRequestLine());
// Create a custom response handler
ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
#Override
public String handleResponse(
final HttpResponse response) throws ClientProtocolException, IOException {
int status = response.getStatusLine().getStatusCode();
if (status >= 200 && status < 300) {
HttpEntity entity = response.getEntity();
return entity != null ? EntityUtils.toString(entity) : null;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
}
};
String responseBody = httpclient.execute(httpget, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
} finally {
httpclient.close();
}
This is what is quoted for the above example:
This example demonstrates how to process HTTP responses using a response handler. This is the recommended way of executing HTTP requests and processing HTTP responses. This approach enables the caller to concentrate on the process of digesting HTTP responses and to delegate the task of system resource deallocation to HttpClient. The use of an HTTP response handler guarantees that the underlying HTTP connection will be released back to the connection manager automatically in all cases.
Exported Qlik Sense certificate using QMC (client.pfx, root.cer, server.pfx).
Imported certificates into IIS web server using MMC. Server and client certificates to store Personal->Certificates, root to store Trusted Root Certification Authorities.
Requested QRS API from ASP.NET controller using QlikClient certificate from store (code below). Tried various user IDs and directories, including INTERNAL/sa_repository, but in all cases got an error "An error occurred while sending the request. The client certificate credentials were not recognized".
Endpoint for test : https://server:4242/qrs/about
I've searched the web but I haven't managed to find what I'm doing wrong, what credentials I should provide.
On the other hand, as I converted exported certificates to separate .key/.crt files (using https://www.markbrilman.nl/2011/08/howto-convert-a-pfx-to-a-seperate-key-crt-file/) and used them in the Postman from web server, it worked without any problem, actually with any UserId in header (i guess it's ignored in that case).
ASP.NET controller:
public X509Certificate2 LoadQlikCertificate()
{
X509Certificate2 certificate = null;
try
{
// Open certification store (MMC)
X509Store store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);
// Get certiface based on the friendly name
certificate = store.Certificates.Cast<X509Certificate2>().FirstOrDefault(c => c.FriendlyName == "QlikClient");
// Logging for debugging purposes
if (certificate != null)
{
logger.Log(LogLevel.Warning, $"Certificate: {certificate.FriendlyName} {certificate.GetSerialNumberString()}");
}
else
{
logger.Log(LogLevel.Warning, $"Certificate: No certificate");
}
// Close certification store
store.Close();
// Return certificate
return certificate;
}
catch (Exception e)
{
...
}
}
/* Get Qlik API response
***********************/
[HttpGet("getqlikapi")]
public IActionResult GetQlikAPI()
{
// Get Qlik certificate
var certificate = this.LoadQlikCertificate();
try
{
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
// Set server name
string server = "server";
// HARDCODED USER AND DIRECTORY FOR TESTING
string userID = "sa_repository"; // tried also other user ids
string userDirectory = "INTERNAL";
// Set Xrfkey header to prevent cross-site request forgery
string xrfkey = "abcdefg123456789";
// Create URL to REST endpoint
string url = $"https://{server}:4242/qrs/about?xrfkey={xrfkey}";
// The JSON object containing the UserId and UserDirectory
string body = $"{{ 'UserId': '{userID}', 'UserDirectory': '{userDirectory}', 'Attributes': [] }}";
// Encode the json object and get the bytes
byte[] bodyBytes = Encoding.UTF8.GetBytes(body);
// Create the HTTP Request
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
// Add the method to authentication the user
request.ClientCertificates.Add(certificate);
// POST request will be used
request.Method = "POST";
// The request will accept responses in JSON format
request.Accept = "application/json";
// A header is added to validate that this request contains a valid cross-site scripting key (the same key as the one used in the url)
request.Headers.Add("X-Qlik-Xrfkey", xrfkey);
request.ContentType = "application/json";
request.ContentLength = bodyBytes.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(bodyBytes, 0, bodyBytes.Length);
requestStream.Close();
// Make the web request and get response
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
// Return string in response
//return new OkObjectResult(stream != null ? new StreamReader(stream).ReadToEnd() : string.Empty);
return new OkObjectResult("test");
}
catch (Exception e)
{
...
}
}
I ran into this issue on a system we are building.
The problem was that the user did not have rights to the certificate.
Open certificate manager (Start > Manage Computer Certificates)
Find the required certificate.
Right-click cert > All Tasks > Manage Private Keys > Add > [Select the appropriate user]
Note: Manage User Certificates does not have the Manage Private Keys option.
I have a spring-boot project using Apache Camel.
I want to read a message from an activemq queue containing a file and send it to a web server.
I am trying to find the proper way to do this.
I believe I can make something like:
from("activemq:queue").bean(MyBean.class, "process")
And manually build a http request but I can't help thinking there is probably a better way to do it. Like:
from("activemq:queue").bean(MyBean.class, "process")
.setHeader(Exchange.HTTP_METHOD,constant("POST"))
.to("http://localhost:8080/test");
But I don't know how to manipulate the "exchange" to have a valid http Message.
MyBean receives an Exchange object containing a JmsMessage. I see that there is also a HTTPMessage but I don't think I should build that manually. (It requires HTTPRequest and Response objects I am not sure how to get.)
Can someone shed some light on this problem?
Update
I am going for the bean solution.
from("activemq:queue").bean(MyBean.class, "sendMultipart");
public void sendMultipart(Exchange exchange) {
ByteArrayInputStream in = new ByteArrayInputStream((byte[]) exchange.getIn().getBody());
InputStreamBody contentBody = new InputStreamBody(in, ContentType.create("application/octet-stream"), "filename");
HttpEntity entity = MultipartEntityBuilder
.create()
.addPart("file", contentBody)
.build();
HttpPost httpPost = new HttpPost("http://localhost:8080/upload/");
httpPost.setEntity(entity);
CloseableHttpClient httpClient = HttpClients.createDefault();
try {
CloseableHttpResponse httpResponse = httpClient.execute(httpPost);
System.out.println(httpResponse);
} catch (IOException e) {
e.printStackTrace();
}
}
Updated post
I found this http://hilton.org.uk/blog/camel-multipart-form-data. It allows you to leverage the camel http component.
"jms:queue/SomeQ" ==> {
process(toMultipart)
setHeader(Exchange.CONTENT_TYPE, "multipart/form-data")
process((e: Exchange) => e.getIn.setHeader(Exchange.HTTP_URI,"http://localhost:8111/foo"))
to ("http:DUMMY")
}
def toMultipart(exchange: Exchange): Unit = {
val data = exchange.in[java.io.File]
val entity = MultipartEntityBuilder.create()
entity.addBinaryBody("file", data)
entity.addTextBody("name", "sample-data")
// Set multipart entity as the outgoing message’s body…
exchange.in = entity.build
}
Side note: this would really be a nice use-case to try-out reactive streams.
Original post
I am still having some problems understanding your actual problem. Perhaps some code might help:
I am now assuming you are receiving bytes in some character encoding and want to sent it onward to a dynamically established http-endpoint.
Is the following something you are looking for (code is in camel's scala-dsl)
"jms:queue/SomeQ" ==> {
convertBodyTo(classOf[String],"UTF-32" )
process((e: Exchange) => e.in = e.in[String].toUpperCase + "!")
process((e: Exchange) => e.getIn.setHeader(Exchange.HTTP_URI,"http://localhost:8111/foo"))
to ("http:DUMMY")
}
It will be send as an HTTP POST as the body is not null.
I receive it all well on another endpoint i created to ensure the code above is correct:
"jetty:http://localhost:8111/foo" ==> {
log("received on http 8111 endpoint ${body}")
}
I am using Apache abdera to post multipart request to IBM connection 4.0 API. I am getting nullpointer exception from Abdera API. Please let me know what's the root cause.
private void createEntryWithAttachment(){
try {
String activityId = "urn:lsid:ibm.com:oa:662d0dc7-0308-48ee-8291-d730c733d2d1";
String activityIdLocal = activityId.substring(activityId.lastIndexOf(":")+1, activityId.length());
String createEntryLocal = createEntry+activityIdLocal;
Abdera abdera = new Abdera();
AbderaClient client = new AbderaClient(abdera);
AbderaClient.registerTrustManager();
System.out.println("pd --->"+pd);
client.addCookie("poktam2cl.iespc.ibm.com", "PD-S-SESSION-ID", pd, "/", null, true);
RequestOptions requestOptions = client.getDefaultRequestOptions();
requestOptions.setUseChunked(true);
requestOptions.setHeader("Connection", "close");
requestOptions.setHeader("Content-Type", "multipart/related;type=\"application/atom+xml\"");
requestOptions.setContentType("multipart/related;type=\"application/atom+xml\"");
requestOptions.setSlug("Sample.txt");
Credentials credentials = new UsernamePasswordCredentials(username, password);
client.addCredentials(createEntryLocal, AuthScope.ANY_REALM,AuthScope.ANY_SCHEME, credentials);
Entry entry = abdera.getFactory().newEntry();
entry.setTitle("create entry with attachment title ");
entry.setContent("create entry with attachment content");
javax.xml.namespace.QName field = new QName("http://www.ibm.com/xmlns/prod/sn", "field", "snx");
org.apache.abdera.model.Element fieldElement = entry.addExtension(field);
fieldElement.setAttributeValue("type", "file");
fieldElement.setAttributeValue("name", "sampletextfile1");
fieldElement.setAttributeValue("position", "3000");
FileInputStream fis = new FileInputStream(filepath);
requestOptions.setHeader("Content-Length", "35");
entry.addCategory("http://www.ibm.com/xmlns/prod/sn/type","entry", "Entry");
ClientResponse response = client.post(createEntryLocal, entry, fis, "multipart/related;type=\"application/atom+xml\"", requestOptions );
System.out.println("Entry Created with attachment's resp: " + response.getStatus());
if(response.getStatus() == 201){
System.out.println("Entry Created with attachment successfully .....");
printIBMConnectionErrorMessage(response);
}else{
System.out.println("Entry with attachment creation failed");
printIBMConnectionErrorMessage(response);
//System.exit(0);
}
} catch (Exception e) {
e.printStackTrace();
}
}
Output
java.lang.NullPointerException
at org.apache.abdera.protocol.client.util.MultipartRelatedRequestEntity.writeInput(MultipartRelatedRequestEntity.java:74)
at org.apache.abdera.protocol.client.util.MultipartRelatedRequestEntity.writeRequest(MultipartRelatedRequestEntity.java:59)
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:499)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.apache.abdera.protocol.client.AbderaClient.execute(AbderaClient.java:688)
at org.apache.abdera.protocol.client.AbderaClient.post(AbderaClient.java:306)
at JavaAgentEntryWithAttachment.createEntryWithAttachment(JavaAgentEntryWithAttachment.java:157)
at JavaAgentEntryWithAttachment.main(JavaAgentEntryWithAttachment.java:66)
This exception is coming from abdera API, class called MultipartRelatedRequestEntity.java, Line no 74. I have placed line no 74 source code below. So its clear that contentSrc is null & Abdera API not allowing me to set this value. Please let me know what I am missing here.
String contentId = entry.getContentSrc().toString();
I did in two steps:
Send the file
Call to update the data
Each with the good mime type. You can not send the file with XML mime type. And put the length of the file.
It is possible to avoid the nullpointer and do it in one request. I had the same issue and created another issue and managed to find a solution. You can find it here.
It comes down to the following code example where you create a HttpClient Part which can contain a StringPart and a FilePart
final Entry entry = // ... Create your Entry
final RequestOptions options = this.client.getDefaultRequestOptions();
options.setHeader("Content-Type", "multipart/related;type=\"application/atom+xml\"");
StringPart entryPart = new StringPart("entry", entry.toString());
entryPart.setContentType("application/atom+xml");
FilePart filePart = new FilePart("file", new File(resource.getFile()));
RequestEntity request = new MultipartRequestEntity(new Part[] { entryPart, filePart}, this.client.getHttpClientParams());
ClientResponse response = client.post(this.url + this.activityId, request, options);
Hope this will help people in the future if they are using Abdera.