Send a PUT request with a file and custom headers - http

I am stuck at a problem.
I need to upload a file (a CSV file) to a server by using Delphi. The server have certain requirements that must be followed :
The request must be an HTTP PUT request.
Must contain the Client ID in the URL request, like this : example.com/upload/{Client ID} ({Client ID} would be replaced).
Must contain the future file name that will be received also in the URL after the client ID, like this : example.com/upload/{Client ID}/{Future File Name} ({Future File Name} would be replaced), which can be different from your local file name.
The API Key must be in the request header (header name: x-api-key). Example : lXViWTzFic9sM8qe9Ew7JME8xTdBAOMJHdIjK7XkjQ00OWr.
I could do this in CURL, like this example : (Client ID : d32krpq, and Future file name : 20181023_update.csv)
curl -v -X PUT https://example.com/upload/d32krpq/20181023_update.csv --upload-file 20181023_update.csv --header "x-api-key: lXViWTzFic9sM8qe9Ew7JME8xTdBAOMJHdIjK7XkjQ00OWr"
How to achieve this using Delphi components (using either TNetHttpClient or TRESTClient)?

In Delphi, sending multiple fields (files, custom headers, body fields, ...) in your PUT request is possible in both HTTP client frameworks:
In TNetHTTPClient through the method Put(), you can use the overloaded function with the local file path's string parameter. And in the function's optional parameter of TNetHeaders object, you specify your custom header parameters.
TNetHeaders is a dynamic array of the TNameValuePair record.
So, you have to create a dynamic array of TNameValuePair, then you create an instance of the TNameValuePair record with your API key and add it to the dynamic array.
With the exception that in TNetHTTPClient, you have to specify the URL segments parameter directly in your URL's request by yourself.
And finally, you call the Put() method with your file path's string and TNetHeaders objects, like this example:
procedure TForm1.PutFile;
var
LHTTP: TNetHTTPClient;
LHeaders: array of TNameValuePair;
begin
SetLength(LHeaders, 1);
LHeaders[0] := TNameValuePair.Create('x-api-key', 'lXViWTzFic9sM8qe9Ew7JME8xTdBAOMJHdIjK7XkjQ00OWr');
LHTTP := TNetHTTPClient.Create(nil);
try
LHTTP.Put('https://example.com/upload/d32krpq/20181023_update.csv', 'C:/MyFile.csv', nil, LHeaders);
finally
LHTTP.Free;
end;
end;
For the REST Client library, you can simply use the TRESTRequest.AddFile() function to add a file, and you can use the AddParameter() function to add any custom fields as a body, an URL segment or a header parameter.
Another advantage in TRESTClient is that you can use a base URL or a resource where you define URL segments, like "example.com/upload/{Client ID}/{Future File Name}".
Drop the components TRESTClient, TRESTRequest, and TRESTResponse on the form, set the TRESTClient.BaseURL to your upload server, and write your code. Like in this example:
procedure TForm1.PutFile;
begin
try
try
RESTRequest1.Method := TRESTRequestMethod.rmPUT;
RESTRequest1.AddFile('File1', 'C:\MyFile.csv', TRESTContentType.ctTEXT_CSV);
// Adding the parameters to be included in the URL
RESTRequest1.AddField('Client ID', 'd32krpq', pkURLSEGMENT);
RESTRequest1.AddField('Future File Name', '20181023_update.csv', pkURLSEGMENT);
// Adding a header parameter
RESTRequest1.AddField('x-api-key', 'lXViWTzFic9sM8qe9Ew7JME8xTdBAOMJHdIjK7XkjQ00OWr', pkHTTPHeader);
RESTRequest1.Execute;
except
ShowMessage('Uploading failed');
end;
finally
ShowMessage(RESTRequest1.Response.Content); // here you can see your server's response
end;
end;

Here is an alternative approach using Indy's TIdHTTP component:
FS := TIdReadFileExclusiveStream.Create('20181023_update.csv');
try
IdHTTP1.Request.CustomHeaders.Values['x-api-key'] := 'lXViWTzFic9sM8qe9Ew7JME8xTdBAOMJHdIjK7XkjQ00OWr';
IdHTTP1.Put('https://example.com/upload/d32krpq/20181023_update.csv', FS);
finally
FS.Free;
end;

Related

how can i send a post request via Business Central?

I would like to create a PostRequest in my Business Central Extension that authenticates me in my web service and returns me a token. I send my username and password in the body of the request to my web service and I also receive the token in JSON format in the body.I want to create the post request using HttpClient.
I use the following link as a template: https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/methods-auto/httpcontent/httpcontent-data-type
procedure sendPostRequest(uri: Text)
var
client: HttpClient;
content: HttpContent;
contentHeaders: HttpHeaders;
response: HttpResponseMessage;
request: HttpRequestMessage;
begin
content.GetHeaders(contentHeaders);
contentHeaders.Clear();
contentHeaders.Add('Content-Type', 'application/json');
request.Content:= content;
request.SetRequestUri(uri);
request.Method := 'POST';
end;
procedure SetURLsToDefault(var MessagingServiceSetup: Record "Messaging Service Setup WMR")
begin
MessagingServiceSetup."Service URL" := '202.212.127:8800';
end;
And I have a couple of questions:
1) the basic url is 202.212.127:8800 for my API gateway. To be able to authenticate myself I have to access 202.212.127:8800/authenticate. Is there a method in which you can create urls?
2) how do I get my username and password in the content?
3) and how do I get the token and can I save it in the field?
can someone tell me how to get the PostRequest up and running?
Common method to create different URLs is like this:
Create a setup table
Create fields like "Base Url", User, Pass etc.
I propose this pattern for your code:
SendRequest(Method; Url; Body)
Begin
...
Couple of functions (Your Api and Auth):
Authenticate()
begin
Method = 'post';
Url = SetupTable."Base Url" + '/authenticate';
Body = (Use AL Json stack and incorporate your user pass)
SendRequest(Method; Url; Body);
end;
Function1()
begin
Method = 'get';
Url = SetupTable."Base Url" + '/apiPath-Function1';
Body = '';
SendRequest(Method; Url; Body);
end
Function2()
begin
Method = 'post';
Url = SetupTable."Base Url" + '/apiPath-Function2';
Body = (Use AL Json stack and incorporate your body structure);
SendRequest(Method; Url; Body);
end;
To get your user pass into the content you need to check the documentation of the Api you're trying to call. It's usually described in details, it can be a simple header for basic authentication or a complex Jwt.
For receiving a token, again you need to check your Api documentation first, but essentially after making a Rest call (like: client.Send(RequestMessage, ResponseMessage); inside your SendRequest method), you get a response back and you can use AL Json stack to carve information out.
This is a fine article on how to proceed:
https://jackmallender.com/2019/03/04/interacting-with-rest-apis-using-json-from-within-business-central-part-1-an-introduction-to-the-httpclient-data-type/
Basically a string could work as an url. Depends on what you want. It is good practice to have a setup for your web service calls, so I am with Babak. You can set up a table in which you store the links, credentials - whatsoever.
and 4) I suggest Waldos Rest App for web service calls. you can download the source here: https://github.com/waldo1001/waldo.restapp
It encapsulated the calls, has helper functions for json handling as well. Using the "REST Helper" Codeunit. You can break down your call to:
local procedure DoCallWebservice(URI: Text; User: Text; Pass: Text; var Token: Text);
var
RESTHelper: Codeunit "REST Helper WLD";
begin
RRESTHelper.Initialize('GET', URI);
RESTHelper.SetContentType('application/json');
RESTHelper.AddBody('{"user":"USERNAME","pass":"PASSWORD"}');
if RESTHelper.Send() then
Token := RESTHelper.GetResponseContentAsText();
end;
Obviously, you need to parse the response (JSONHelper) to your needs. Look at the code of the codeunit, it's more or less self explanatory.

Is it possible to run http.ListenAndServe() AND ReadFromUDP() concurrently?

I am trying to write a simple web app that will listen for UDP packets.
But I can either only listen for UDP packets, or run the web app...
I am not familiar with GoLang, but here's the code I'm using to...
listen for UDP:
ServerConn, _ := net.ListenUDP("udp", &net.UDPAddr{IP:[]byte{#,#,#,#},Port:####,Zone:""})
defer ServerConn.Close()
buf := make([]byte, 1024)
for {
n, addr, _ := ServerConn.ReadFromUDP(buf)
fmt.Println("Received ", string(buf[0:n]), " from ", addr)
}
Server logic:
package main
We import 4 important libraries
1. “net/http” to access the core go http functionality
2. “fmt” for formatting our text
3. “html/template” a library that allows us to interact with our html file.
4. "time" - a library for working with date and time.
import (
"net/http"
"fmt"
"time"
"html/template"
)
//Create a struct that holds information to be displayed in our HTML file
type Welcome struct {
Name string
Time string
}
//Go application entrypoint
func main() {
//Instantiate a Welcome struct object and pass in some random information.
//We shall get the name of the user as a query parameter from the URL
welcome := Welcome{"Anonymous", time.Now().Format(time.Stamp)}
//We tell Go exactly where we can find our html file. We ask Go to parse the html file (Notice
// the relative path). We wrap it in a call to template.Must() which handles any errors and halts if there are fatal errors
templates := template.Must(template.ParseFiles("templates/welcome-template.html"))
//Our HTML comes with CSS that go needs to provide when we run the app. Here we tell go to create
// a handle that looks in the static directory, go then uses the "/static/" as a url that our
//html can refer to when looking for our css and other files.
http.Handle("/static/", //final url can be anything
http.StripPrefix("/static/",
http.FileServer(http.Dir("static")))) //Go looks in the relative "static" directory first using http.FileServer(), then matches it to a
//url of our choice as shown in http.Handle("/static/"). This url is what we need when referencing our css files
//once the server begins. Our html code would therefore be <link rel="stylesheet" href="/static/stylesheet/...">
//It is important to note the url in http.Handle can be whatever we like, so long as we are consistent.
//This method takes in the URL path "/" and a function that takes in a response writer, and a http request.
http.HandleFunc("/" , func(w http.ResponseWriter, r *http.Request) {
//Takes the name from the URL query e.g ?name=Martin, will set welcome.Name = Martin.
if name := r.FormValue("name"); name != "" {
welcome.Name = name;
}
//If errors show an internal server error message
//I also pass the welcome struct to the welcome-template.html file.
if err := templates.ExecuteTemplate(w, "welcome-template.html", welcome); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
})
//Start the web server, set the port to listen to 8080. Without a path it assumes localhost
//Print any errors from starting the webserver using fmt
fmt.Println("Listening");
fmt.Println(http.ListenAndServe(":8080", nil));
}
taken from(https://medium.com/google-cloud/building-a-go-web-app-from-scratch-to-deploying-on-google-cloud-part-1-building-a-simple-go-aee452a2e654)
I tried putting both of these extracts in 1 file, as well as running 2 files at the same time using
go run *.go
Any help would be appreciated!
You're going to need to start looking into goroutines - since you're asking to do two things concurrently. I suggest doing some reading into channels, goroutines, and concurrency in general :)

Arbitrary response content types in Feathers

I have a custom service that must return data in CSV format.
I can't use a standard Express route, because I need Feathers' hooks on this endpoint.
I couldn't find an example of a Feathers service returning non-HTML, non-JSON data, and have found no way to specify a response content type.
Using res.set('Content-Type', 'text/csv') before returning from the service method didn't work; the final Content-Type header was reset to application/json, even though the method's return value was a regular string.
How can I properly set arbitrary response content types in Feathers' custom service methods?
You can customize the response format like this:
const feathers = require('feathers');
const rest = require('feathers-rest');
const app = feathers();
function restFormatter(req, res) {
res.format({
'text/plain': function() {
res.end(`The Message is: "${res.data.text}"`);
}
});
}
app.configure(rest(restFormatter));
The complete documentation can be found here.
Using your own service specific middleware to send the response should also work.

Golang Modify HTTP Request Parameters Such As URL Path Before Routing

It's common practice in some cases to pass plain URIs as suffix of the path instead of a query parameter. Here is an example from Internet Archive's Wayback Machine.
https://web.archive.org/web/20150825082012/http://example.com/
In this example, user is requesting a copy of http://example.com/ as captured at 2015-08-25 08:20:12. If we were to implement similar service in Go, we probably would have a router as follows:
http.HandleFunc("/web/", returnArchivedCopy)
Then in the returnArchivedCopy handler function, we will split r.URL.Path (where r is the Request object) to extract the date-time and the target URL. However there is a problem in this style of URL scheme; Go's net/http package calls cleanPath function on the path portion to sanitize it. This sanitization process does various cleanup tasks such as eeliminating . and .. from the path and replace multiple slashes with a single one. This later operation makes sense when because in Unix systems // in the file path are same as /. However this causes an issue in the above described use case as http://example becomes http:/example and the server internally returns a redirect response to the client with the sanitized path.
I am wondering, what are my options in this case? Is there a way to ask HTTP not to sanitize the request path while still utilizing all the default behavior that is shipped with the default (or slightly modified) server, multiplexer, and handler? Or is there a way to modify the request parameters (path in this case) before it hits the multiplexer's routing patterns. If the later is possible, we might try to perform something like URL encoding to avoid the redirect and later decode the URL back in the handler function before extracting desired bits.
I have experimented with some custom handlers and multiplexers, but I am new to Go, hence I was not quite sure how to delegate the routing back to the default handlers after making changes in the request.
You can implement a wrapper mux, that falls back to the default one, here's a very simple example:
func main() {
http.HandleFunc("/blah", func(w http.ResponseWriter, req *http.Request) {
w.Write([]byte("w00t"))
})
http.ListenAndServe(":9090", http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
p := strings.SplitN(req.URL.RequestURI()[1:] /*trim the first slash*/, "/", 3)
if len(p) != 3 || p[0] != "web" {
http.DefaultServeMux.ServeHTTP(w, req)
return
}
t, err := time.Parse("20060102150405", p[1])
if err != nil {
http.Error(w, "invalid time", 400)
return
}
url := p[2]
fmt.Fprintf(w, "requested url %v # %v", url, t)
}))
}

How to safely handle raw (file) data in Java?

An image gets corrupted while being retrieved (through HTTP) and then sent (through HTTP) to a database. Image's raw data is handled in String form.
The service sends a GET for an image file, receives response with the raw image data (response's body) and the Content-Type. Then, a PUT request is sent with the aforementioned request's body and Content-Type header. (The PUT request is constructed by providing the body in String) This PUT request is sent to a RESTful database (CouchDB), creating an attachment (for those unfamiliar with CouchDB an attachment acts like a static file).
Now I have the original image, which my service GETs and PUTs to a database, and this 'copy' of the original image, that I can now GET from the database. If I then `curl --head -v "[copy's url]" it has the Content-Type of the original image, but Content-Length has changed, went from 200kb to about 400kb. If I GET the 'copy' image with a browser, it is not rendered, whereas, the original renders fine. It is corrupted.
What might be the cause? My guess is that while handling the raw data as a string, my framework guesses the encoding wrong and corrupts it. I have not been able to confirm or deny this. How could I handle this raw data/request body in a safe manner, or how could I properly handle the encoding (if that proves to be the problem)?
Details: Play2 Framework's HTTP client, Scala. Below a test to reproduce:
"able to copy an image" in {
def waitFor[T](future:Future[T]):T = { // to bypass futures
Await.result(future, Duration(10000, "millis"))
}
val originalImageUrl = "http://laughingsquid.com/wp-content/uploads/grumpy-cat.jpg"
val couchdbUrl = "http://admin:admin#localhost:5984/testdb"
val getOriginal:ws.Response = waitFor(WS.url(originalImageUrl).get)
getOriginal.status mustEqual 200
val rawImage:String = getOriginal.body
val originalContentType = getOriginal.header("Content-Type").get
// need an empty doc to have something to attach the attachment to
val emptyDocUrl = couchdbUrl + "/empty_doc"
val putEmptyDoc:ws.Response = waitFor(WS.url(emptyDocUrl).put("{}"))
putEmptyDoc.status mustEqual 201
//uploading an attachment will require the doc's revision
val emptyDocRev = (putEmptyDoc.json \ "rev").as[String]
// create actual attachment/static file
val attachmentUrl = emptyDocUrl + "/0"
val putAttachment:ws.Response = waitFor(WS.url(attachmentUrl)
.withHeaders(("If-Match", emptyDocRev), ("Content-Type", originalContentType))
.put(rawImage))
putAttachment.status mustEqual 201
// retrieve attachment
val getAttachment:ws.Response = waitFor(WS.url(attachmentUrl).get)
getAttachment.status mustEqual 200
val attachmentContentType = getAttachment.header("Content-Type").get
originalContentType mustEqual attachmentContentType
val originalAndCopyMatch = getOriginal.body == getAttachment.body
originalAndCopyMatch aka "original matches copy" must beTrue // << false
}
Fails at the last 'must':
[error] x able to copy an image
[error] original matches copy is false (ApplicationSpec.scala:112)
The conversion to String is definitely going to cause problems. You need to work with the bytes as Daniel mentioned.
Looking at the source it looks like ws.Response is just a wrapper. If you get to the underlying class then there are some methods that may help you. On the Java side, someone made a commit on GitHub to expose more ways of getting the response data other than a String.
I'm not familiar with scala but something like this may work:
getOriginal.getAHCResponse.getResponseBodyAsBytes
// instead of getOriginal.body
WS.scala
https://github.com/playframework/playframework/blob/master/framework/src/play/src/main/scala/play/api/libs/ws/WS.scala
WS.java
Here you can see that Response has some new methods, getBodyAsStream() and asByteArray.
https://github.com/playframework/playframework/blob/master/framework/src/play-java/src/main/java/play/libs/WS.java

Resources