Get Html response with Retrofit - retrofit

I'm new to Retrofit. I make a POST request to a website. Website returns response as HTML. So I will parse it. However Retrofit try to parse it as JSON. How can do it?
#FormUrlEncoded
#POST("/login.php?action=login")
void postCredentials(#Field("username") String username,
#Field("password") String password);
Should I use a callback?

Retrofit uses a converter to process responses from endpoints and requests as well. By default, Retrofit uses GsonConverter, which encoded JSON responses to Java objects using the gson library. You can override that to supply your own converter when constructing your Retrofit instance.
The interface you need to implement is available here (github.com). Here's a short tutorial as well, although for using Jackson library, many bits are still relevant: futurestud.io/blog
Also note that the converter works both ways, converting requests and responses. Since you want HTML parsing in one direction only, you may want to use GsonConverter in your custom converter, to convert outgoing Java objects to JSON, in the toBody method.

May be not the best solution but this how i managed to get the source of an html page with retrofit:
MainActivity.java
ApiInterface apiService = ApiClient.getClient(context).create(ApiInterface.class);
//Because synchrone in the main thread, i don't respect myself :p
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
//Execution of the call
Call<ResponseBody> call = apiService.url();
response = call.execute();
//Decode the response text/html (gzip encoded)
ByteArrayInputStream bais = new ByteArrayInputStream(((ResponseBody)response.body()).bytes());
GZIPInputStream gzis = new GZIPInputStream(bais);
InputStreamReader reader = new InputStreamReader(gzis);
BufferedReader in = new BufferedReader(reader);
String readed;
while ((readed = in.readLine()) != null) {
System.out.println(readed); //Log the result
}
ApiInterface.java
#GET("/")
Call<ResponseBody> url();
ApiClient.java
public static final String BASE_URL = "https://www.google.com";
private static Retrofit retrofit = null;
public static Retrofit getClient(Context context) {
if (retrofit==null) {
OkHttpClient okHttpClient = new OkHttpClient().newBuilder()
.build();
retrofit = new Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(ScalarsConverterFactory.create())
.client(okHttpClient)
.build();
}
return retrofit;
}

Related

Unsupported Media Types when POST to web api

Here is the client :
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost/MP.Business.Implementation.FaceAPI/");
client.DefaultRequestHeaders
.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/octet-stream"));
using (var request = new HttpRequestMessage(HttpMethod.Post, client.BaseAddress + "api/Recognition/Recognize"))
{
request.Content = new ByteArrayContent(pic);
request.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
await client.PostAsync(request.RequestUri, request.Content);
}
}
and the server :
[System.Web.Http.HttpPost]
public string Recognize(byte[] img)
{
//do someth with the byte []
}
I am getting error:
415 Unsupported Media Type
all the time - The request entity's media type 'application/octet-stream' is not supported for this resource. What can i do about it? I've found some answered threads here , but it didnt help.
While byte[] would be a great way to represent application/octet-stream data, this is not the case by default in Web API.
My workaround is in ASP.NET Core 1.1 - the details may be different in other variants.
In your controller method, remove the img parameter. Instead, refer to the Request.Body, which is a Stream. e.g. to save to a file:
using (var stream = new FileStream(someLocalPath, FileMode.Create))
{
Request.Body.CopyTo(stream);
}
The situation is similar for returning binary data from a GET controller method. If you make the return type byte[] then it is formatted with base64! This makes it significantly larger. Modern browsers are perfectly capable of handling raw binary data so this is no longer a sensible default.
Fortunately there is a Response.Body https://github.com/danielearwicker/ByteArrayFormatters:
Response.ContentType = "application/octet-stream";
Response.Body.Write(myArray, 0, myArray.Length);
Make the return type of your controller method void.
UPDATE
I've created a nuget package that enables direct use of byte[] in controller methods. See: https://github.com/danielearwicker/ByteArrayFormatters

Can I disable model binding and use the raw request body in an action in dotnet core?

I want to setup an endpoint for testing webhooks from third parties. Their documentation is uniformly poor and there is no way ahead of time to tell exactly what I will be getting. What I've done is setup an ApiController that will just take a request and add a row to a table with what they are sending. This lets me at least verify they are calling the webhook, and to see the data so I can program to it.
// ANY api/webook/*
[Route("{*path}")]
public ActionResult Any(string path)
{
string method = Request.Method;
string name = "path";
string apiUrl = Request.Path;
string apiQuery = Request.QueryString.ToString();
string apiHeaders = JsonConvert.SerializeObject(Request.Headers);
string apiBody = null;
using (StreamReader reader = new StreamReader(Request.Body))
{
apiBody = reader.ReadToEnd();
}
Add(method, name, apiUrl, apiQuery, apiHeaders, apiBody);
return new JsonResult(new { }, JsonSettings.Default);
}
This works great, except for this new webhook I am usign that posts as form data so some middleware is reading the body and it ends up null in my code. Is there any way to disable the model processing so I can get at the request body?
You could actually use model binding to your advantage and skip all that stream reading, using the FromBody attribute. Try this:
[Route("{*path}")]
[HttpPost]
public ActionResult Any(string path, [FromBody] string apiBody)

Static HttpClient thread safe on ASP.net HttpRequest

We are creating a wrapper for HttpClient. As we are going to follow performance optimization guidance from https://github.com/mspnp/performance-optimization. We want to avoid anti-pattern - Improper instantiation mentioned in that document. I referred this guidance to my team to use static HttpClient. The feedback I have got is on thread-safety. Each request has a header containing user claim. Since I have a static HttpClient, will it be thread-safe? If we have multiple requests hitting the code (for example GET) at the same time, will it be a race condition to set header? We have implementation as below.
public class HttpClientHelper{
private static readonly HttpClient _HttpClient;
static HttpClientHelper() {
HttpClient = new HttpClient();
HttpClient.Timeout = TimeSpan.FromMinutes(SOME_CONFIG_VALUE);
}
public async Task<HttpResponseMessage> CallHttpClientPostAsync(string requestUri, HttpContent requestBody)
{
AddHttpRequestHeader(httpClient);
var response = await httpClient.PostAsync(requestUri, requestBody); //Potential thread synchronization issue???
return response;
}
public HttpResponseMessage CallHttpClientGet(string requestUri)
{
AddHttpRequestHeader(httpClient);
var response = httpClient.GetAsync(requestUri).Result; //Potential thread synchronization issue???
return response;
}
private void AddHttpRequestHeader(HttpClient client)
{
string HeaderName = "CorrelationId";
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(Properties.Settings.Default.HttpClientAuthHeaderScheme, GetTokenFromClaims()); //Race condition???
if (client.DefaultRequestHeaders.Contains(HeaderName))
client.DefaultRequestHeaders.Remove(HeaderName);
client.DefaultRequestHeaders.Add(HeaderName, Trace.CorrelationManager.ActivityId.ToString());
}
}
Your team is correct, this is far from thread safe. Consider this scenario:
Thread A sets CorrelationId header to "foo".
Thread B sets CorrelationId header to "bar".
Thread A sends request, which contains thread B's CorrelationId.
A better approach would be for your CallXXX methods to create new HttpRequestMessage objects, and set the header on those, and use HttpClient.SendAsync to make the call.
Keep in mind also that re-using HttpClient instances is only beneficial if you're making multiple calls to the same host.

Returning binary file from controller in ASP.NET Web API

I'm working on a web service using ASP.NET MVC's new WebAPI that will serve up binary files, mostly .cab and .exe files.
The following controller method seems to work, meaning that it returns a file, but it's setting the content type to application/json:
public HttpResponseMessage<Stream> Post(string version, string environment, string filetype)
{
var path = #"C:\Temp\test.exe";
var stream = new FileStream(path, FileMode.Open);
return new HttpResponseMessage<Stream>(stream, new MediaTypeHeaderValue("application/octet-stream"));
}
Is there a better way to do this?
Try using a simple HttpResponseMessage with its Content property set to a StreamContent:
// using System.IO;
// using System.Net.Http;
// using System.Net.Http.Headers;
public HttpResponseMessage Post(string version, string environment,
string filetype)
{
var path = #"C:\Temp\test.exe";
HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.OK);
var stream = new FileStream(path, FileMode.Open, FileAccess.Read);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType =
new MediaTypeHeaderValue("application/octet-stream");
return result;
}
A few things to note about the stream used:
You must not call stream.Dispose(), since Web API still needs to be able to access it when it processes the controller method's result to send data back to the client. Therefore, do not use a using (var stream = …) block. Web API will dispose the stream for you.
Make sure that the stream has its current position set to 0 (i.e. the beginning of the stream's data). In the above example, this is a given since you've only just opened the file. However, in other scenarios (such as when you first write some binary data to a MemoryStream), make sure to stream.Seek(0, SeekOrigin.Begin); or set stream.Position = 0;
With file streams, explicitly specifying FileAccess.Read permission can help prevent access rights issues on web servers; IIS application pool accounts are often given only read / list / execute access rights to the wwwroot.
For Web API 2, you can implement IHttpActionResult. Here's mine:
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading;
using System.Threading.Tasks;
using System.Web;
using System.Web.Http;
class FileResult : IHttpActionResult
{
private readonly string _filePath;
private readonly string _contentType;
public FileResult(string filePath, string contentType = null)
{
if (filePath == null) throw new ArgumentNullException("filePath");
_filePath = filePath;
_contentType = contentType;
}
public Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StreamContent(File.OpenRead(_filePath))
};
var contentType = _contentType ?? MimeMapping.GetMimeMapping(Path.GetExtension(_filePath));
response.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
return Task.FromResult(response);
}
}
Then something like this in your controller:
[Route("Images/{*imagePath}")]
public IHttpActionResult GetImage(string imagePath)
{
var serverPath = Path.Combine(_rootPath, imagePath);
var fileInfo = new FileInfo(serverPath);
return !fileInfo.Exists
? (IHttpActionResult) NotFound()
: new FileResult(fileInfo.FullName);
}
And here's one way you can tell IIS to ignore requests with an extension so that the request will make it to the controller:
<!-- web.config -->
<system.webServer>
<modules runAllManagedModulesForAllRequests="true"/>
For those using .NET Core:
You can make use of the IActionResult interface in an API controller method, like so.
[HttpGet("GetReportData/{year}")]
public async Task<IActionResult> GetReportData(int year)
{
// Render Excel document in memory and return as Byte[]
Byte[] file = await this._reportDao.RenderReportAsExcel(year);
return File(file, "application/vnd.openxmlformats", "fileName.xlsx");
}
This example is simplified, but should get the point across. In .NET Core this process is so much simpler than in previous versions of .NET - i.e. no setting response type, content, headers, etc.
Also, of course the MIME type for the file and the extension will depend on individual needs.
Reference: SO Post Answer by #NKosi
While the suggested solution works fine, there is another way to return a byte array from the controller, with response stream properly formatted :
In the request, set header "Accept: application/octet-stream".
Server-side, add a media type formatter to support this mime type.
Unfortunately, WebApi does not include any formatter for "application/octet-stream". There is an implementation here on GitHub: BinaryMediaTypeFormatter (there are minor adaptations to make it work for webapi 2, method signatures changed).
You can add this formatter into your global config :
HttpConfiguration config;
// ...
config.Formatters.Add(new BinaryMediaTypeFormatter(false));
WebApi should now use BinaryMediaTypeFormatter if the request specifies the correct Accept header.
I prefer this solution because an action controller returning byte[] is more comfortable to test. Though, the other solution allows you more control if you want to return another content-type than "application/octet-stream" (for example "image/gif").
For anyone having the problem of the API being called more than once while downloading a fairly large file using the method in the accepted answer, please set response buffering to true
System.Web.HttpContext.Current.Response.Buffer = true;
This makes sure that the entire binary content is buffered on the server side before it is sent to the client. Otherwise you will see multiple request being sent to the controller and if you do not handle it properly, the file will become corrupt.
The overload that you're using sets the enumeration of serialization formatters. You need to specify the content type explicitly like:
httpResponseMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
You could try
httpResponseMessage.Content.Headers.Add("Content-Type", "application/octet-stream");
You can try the following code snippet
httpResponseMessage.Content.Headers.Add("Content-Type", "application/octet-stream");
Hope it will work for you.

spring-mvc with resteasy character encoding problem on jetty server

I am trying to implement restful protocol on jetty server. I have runnable server and i can access it from my rest client. My server side project is a maven project. I have a problem about the character encoding.When i check response, before send it from controller, there is no encoding problem. But after i return response to client, i see broken data. Response header is UTF-8. Also i have a listener for this problem and i am setting to request and response to UTF-8. I guess problem happens when i try to write my response data to response.
#GET
#Path("/")
#Produces({"application/xml;charset=UTF-8","application/json;charset=UTF-8"})
public String getPersons(#Context HttpServletRequest request, #Context HttpServletResponse response) {
List<Person> persons = personService.getPersons(testUserId, collectionOption, null);
if (persons == null) {
persons = new ArrayList<Person>();
}
String result = JsonUtil.listToJson(persons);
//result doesnt has any encoding problem at this line
response.setContentType("application/json");
response.setContentLength(result.length());
response.setCharacterEncoding("utf-8");
//i guess problem happen after this line
return result;
}
Is there any jetty configuration or resteasy configuration for it? Or is there any way to solve this problem? Thanks for your helps.
Which resteasy version are you using? There is a known issue (RESTEASY-467) with Strings in 2.0.1 an prior.
These are your options:
1) force the encoding returning byte[]
public byte[] getPersons
and then
return result.getBytes("UTF8");
2) return List (or create a PersonListing if you need it)
public List<Person> getPersons
and let resteasy handle the json transformation.
3) return a StreamingOutput
NOTE: with this option the "Content-Length" header will be unknown.
return new StreamingOutput()
{
public void write(OutputStream outputStream) throws IOException, WebApplicationException
{
PrintStream writer = new PrintStream(outputStream, true, "UTF-8");
writer.println(result);
}
};
4) upgrade to 2.2-beta-1 or newer version.

Resources