I am using spring RestTemplate to download a file. The file size is small.
I want to get base64 encoded String. but I see the base64 encoded string is truncated from what it is supposed to be.
Here is my code
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(
new ByteArrayHttpMessageConverter());
StreamResourceReader reader = new StreamResourceReader();
restTemplate.execute(uri, HttpMethod.POST, null,
new StreamResponseExtractor(reader));
return reader.getEncodedString();
StreamResourceReader.java
public class StreamResourceReader {
private String encodeString;
public void read(InputStream content) {
try {
encodeString = Base64.encodeBase64String(IOUtils.toByteArray(content));
} catch (IOException e) {
throw new IllegalStateException(e);
}
}
public ByteArrayOutputStream getOutputStream(){
return outputStream;
}
public String getEncodedString() {
return encodeString;
}
}
StreamResponseExtractor.java
public class StreamResponseExtractor implements ResponseExtractor<InputStream> {
private StreamResourceReader reader;
public StreamResponseExtractor(StreamResourceReader resourceReader) {
this.reader=resourceReader;
}
#Override
public InputStream extractData(ClientHttpResponse response) throws IOException {
reader.read(response.getBody());
return null;
}
}
EDIT
just found out that inputStream is truncated. I dont know why and what the fix is. any help here would be appreciated.
Thanks
To confirm if your input stream is indeed truncated you can try few things. What IOUtils.toByteArray(content) does is buffers internally the content of input stream and returns the buffer. You can compare the length of buffer array with the byte array the file actually represents. You can do latter with below code
String filePath = "/test.txt";
byte[] fileByteArray= Files.readAllBytes(Paths.get(filePath));
Also ClientHttpResponse ( client view of http response) too has the inputstream available which you can check for content.
InputStream getBody() throws IOException;
As a test for this scenario , I created spring boot Rest client using Rest Template (using the code you shared) and a service for file download again using Spring Boot. On comparing the base encoded String from download vs direct file access, both return same content (compared using String equals method).
UPDATE: Another thing worth trying is just use java.net.HttpURLConnection
in a simple program (for help see here) and try to download the content and check whether this works properly because behind all the Spring abstractions, in this case the underlying object used is HttpURLConnection only
SimpleClientHttpResponse extends AbstractClientHttpResponse {
public InputStream getBody() throws IOException {
InputStream errorStream = this.connection.getErrorStream();
this.responseStream = (errorStream != null ? errorStream : this.connection.getInputStream());
return this.responseStream;
}
...........
...........
}
If this also gives you the same issue, then it's time to look at the server side. May be the server is not sending the complete data.
I am currently uploading a file via the kendo fileuploader to an api controller using ASP.NET core RC-1. I am receiving a periodic error of "object reference not set to instance of object" when attempting to read the stream following opening the stream with IFormFile.OpenReadStream().
My controller is:
[HttpPost]
[Route("api/{domain}/[controller]")]
public async Task<IActionResult> Post([FromRoute]string domain, [FromForm]IFormFile file, [FromForm]WebDocument document)
{
if (ModelState.IsValid)
{
if (file.Length > 0)
{
var userName =
Request.HttpContext.User.Claims
.FirstOrDefault(c => c.Type == ClaimTypesEx.FullName)?
.Value;
var uploadedFileName =
ContentDispositionHeaderValue.Parse(file.ContentDisposition).FileName.Trim('"');
document.Domain = domain;
document.MimeType = file.ContentType;
document.SizeInBytes = file.Length;
document.ChangedBy = userName;
document.FileName = (string.IsNullOrEmpty(document.FileName)) ? uploadedFileName : document.FileName;
try
{
document = await CommandStack.For<WebDocument>()
.AddOrUpdateAsync(document, file.OpenReadStream()).ConfigureAwait(false);
}
catch (Exception e)
{
return new HttpStatusCodeResult(500);
}
return Ok(document);
}
}
return new BadRequestResult();
}
And the error is being thrown when I actually try to read the stream when it is going into blob storage:
public async Task<Uri> CreateOrUpdateBlobAsync(string containerName, string fileName, string mimeType,
Stream fileStream)
{
var container = Client.GetContainerReference(containerName);
var blob = container.GetBlockBlobReference(fileName);
//Error HERE
await blob.UploadFromStreamAsync(fileStream);
blob.Properties.ContentType = mimeType;
await blob.SetPropertiesAsync();
return blob.Uri;
}
What I am having trouble with is this is sporadic and there seems to be no defined pattern of which files are accepted and which ones generate the error. At first I thought it might be a size issue but that is not the case as I have several larger files uploaded successfully and then one small file will throw the error. Images seem to work fine and it is hit or miss on other file types with no rhyme or reason that I can figure out.
I have a file being uploaded using http post request using multipart/form-data to my class that is extending from ApiController.
In a dummy project, I am able to use:
HttpPostedFileBase hpf = Request.Files[file] as HttpPostedFileBase
to get the file inside my controller method where my Request is of type System.Web.HttpRequestWrapper.
But inside another production app where I have constraints of not adding any libraries/dlls, I don't see anything inside System.Web.HttpRequestWrapper.
My simple requirement is to get the posted file and convert it to a byte array to be able to store that into a database.
Any thoughts?
This code sample is from a ASP.NET Web API project I did sometime ago. It allowed uploading of an image file. I removed parts that were not relevant to your question.
public async Task<HttpResponseMessage> Post()
{
if (!Request.Content.IsMimeMultipartContent())
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
try
{
var provider = await Request.Content.ReadAsMultipartAsync(new MultipartMemoryStreamProvider());
var firstImage = provider.Contents.FirstOrDefault();
if (firstImage == null || firstImage.Headers.ContentDisposition.FileName == null)
return Request.CreateResponse(HttpStatusCode.BadRequest);
using (var ms = new MemoryStream())
{
await firstImage.CopyToAsync(ms);
var byteArray = ms.ToArray();
}
return Request.CreateResponse(HttpStatusCode.Created);
}
catch (Exception ex)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex);
}
}
In a web api method I am generating a file and then streaming it to the response like so
public async Task<HttpResponseMessage> GetFile() {
FileInfo file = generateFile();
var msg = Request.CreateResponse(HttpStatusCode.OK);
msg.Content = new StreamContent(file.OpenRead());
msg.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
msg.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") {FileName = file.Name};
return msg;
}
because this a generated file I want to delete it after the response has finished streaming but I can't seem to find a hook in the pipeline for this.
I suppose that I can put a reference to the file in a static and set up a custom MessageHandler that pulls values out of this same static variable and deletes. However, this seems like it can't possibly be right both because of the use of a static (when this should all be per-request) and because I'd have to register a separate route.
I've seen this question but it seems to not really have much of a useful response.
Nice scenario!...the problem with using message handlers is that response writing happens at the host layers and below message handlers layer, so they are not ideal...
Following is an example of how you could do it:
msg.Content = new CustomStreamContent(generatedFilePath);
public class CustomStreamContent : StreamContent
{
string filePath;
public CustomStreamContent(string filePath)
: this(File.OpenRead(filePath))
{
this.filePath = filePath;
}
private CustomStreamContent(Stream fileStream)
: base(content: fileStream)
{
}
protected override void Dispose(bool disposing)
{
//close the file stream
base.Dispose(disposing);
try
{
File.Delete(this.filePath);
}
catch (Exception ex)
{
//log this exception somewhere so that you know something bad happened
}
}
}
By the way, are you generating this file because you are converting some data into PDF. If yes, then I think you could use PushStreamContent for this purpose by directly writing the converted data into the response stream. This way you need not generate a file first and then worry about deleting it later.
We performed same action in WebAPI. I needed to delete file just after it downloaded form server.
We can create custom response message class. It takes file path as parameter and delete it once its transmitted.
public class FileResponseMessage : HttpResponseMessage
{
private readonly string _filePath;
public FileHttpResponseMessage(string filePath)
{
this._filePath= filePath;
}
protected override void Dispose(bool disposing)
{
base.Dispose(disposing);
Content.Dispose();
File.Delete(_filePath);
}
}
Use this class as below code and it will delete your file once it will be written on response stream.
var response = new FileResponseMessage(filePath);
response.StatusCode = HttpStatusCode.OK;
response.Content = new StreamContent(new FileStream(filePath, FileMode.Open, FileAccess.Read));
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = "MyReport.pdf"
};
return response;
For my ASP.NET MVC 4 project, I'm trying to implement a custom view engine to find an "Index.cshtml" view file if one exists within a folder. Additionally, I'm throwing a 404 for all view paths that are not found.
The 404 works when a view file doesn't exist. When a view file does exist, the view engine will then try looking for a .Mobile.cshtml file using the FileExists() function. There is no .mobile.cshtml file, so it throws an exception. Why does the view engine still look for a .mobile.cshtml file when it has found the non-mobile file already?
For example, when the view engine is able to find a view path at "~/Views/About/History/Index.cshtml", it will then try finding the file "~/Views/About/History/Index.Mobile.cshtml". Below is my full code for the custom view engine.
namespace System.Web.Mvc
{
// Extend where RazorViewEngine looks for view files.
// This looks for path/index.ext file if no path.ext file is found
// Ex: looks for "about/history/index.chstml" if "about/history.cshtml" is not found.
public class CustomViewEngine : RazorViewEngine
{
public BeckmanViewEngine()
{
AreaViewLocationFormats = new[]
{
"~/Areas/{2}/Views/{1}/{0}/Index.cshtml",
};
ViewLocationFormats = new[]
{
"~/Views/{1}/{0}/Index.cshtml",
};
}
// Return 404 Exception if viewpath file in existing path is not found
protected override bool FileExists(ControllerContext context, string path)
{
if (!base.FileExists(context, path))
{
throw new HttpException(404, "HTTP/1.1 404 Not Found");
}
return true;
}
}
}
I have found the answer after digging a bit in the MVC 4 source code.
The RazorViewEngine derives from BuildManagerViewEngine, and this one in turns derives from VirtualPathProviderViewEngine.
It is VirtualPathProviderViewEngine the one that implements the method FindView:
public virtual ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
{
if (controllerContext == null)
{
throw new ArgumentNullException("controllerContext");
}
if (String.IsNullOrEmpty(viewName))
{
throw new ArgumentException(MvcResources.Common_NullOrEmpty, "viewName");
}
string[] viewLocationsSearched;
string[] masterLocationsSearched;
string controllerName = controllerContext.RouteData.GetRequiredString("controller");
string viewPath = GetPath(controllerContext, ViewLocationFormats, AreaViewLocationFormats, "ViewLocationFormats", viewName, controllerName, CacheKeyPrefixView, useCache, out viewLocationsSearched);
string masterPath = GetPath(controllerContext, MasterLocationFormats, AreaMasterLocationFormats, "MasterLocationFormats", masterName, controllerName, CacheKeyPrefixMaster, useCache, out masterLocationsSearched);
if (String.IsNullOrEmpty(viewPath) || (String.IsNullOrEmpty(masterPath) && !String.IsNullOrEmpty(masterName)))
{
return new ViewEngineResult(viewLocationsSearched.Union(masterLocationsSearched));
}
return new ViewEngineResult(CreateView(controllerContext, viewPath, masterPath), this);
}
That GetPath method used there will do something like this when the view path has not been cached yet:
return nameRepresentsPath
? GetPathFromSpecificName(controllerContext, name, cacheKey, ref searchedLocations)
: GetPathFromGeneralName(controllerContext, viewLocations, name, controllerName, areaName, cacheKey, ref searchedLocations);
Getting there! The interesting method is GetPathFromGeneralName, which is the one trying to build the whole path for the view and checking if that path exists. The method is looping through each of the view locations that were registered in the View Engine, updating the view path with the display mode valid for current HttpContext and then checking if the resolved path exists. If so, the view has been found, is assigned to the result, cached and the result path returned.
private string GetPathFromGeneralName(ControllerContext controllerContext, List<ViewLocation> locations, string name, string controllerName, string areaName, string cacheKey, ref string[] searchedLocations)
{
string result = String.Empty;
searchedLocations = new string[locations.Count];
for (int i = 0; i < locations.Count; i++)
{
ViewLocation location = locations[i];
string virtualPath = location.Format(name, controllerName, areaName);
DisplayInfo virtualPathDisplayInfo = DisplayModeProvider.GetDisplayInfoForVirtualPath(virtualPath, controllerContext.HttpContext, path => FileExists(controllerContext, path), controllerContext.DisplayMode);
if (virtualPathDisplayInfo != null)
{
string resolvedVirtualPath = virtualPathDisplayInfo.FilePath;
searchedLocations = _emptyLocations;
result = resolvedVirtualPath;
ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, AppendDisplayModeToCacheKey(cacheKey, virtualPathDisplayInfo.DisplayMode.DisplayModeId), result);
if (controllerContext.DisplayMode == null)
{
controllerContext.DisplayMode = virtualPathDisplayInfo.DisplayMode;
}
// Populate the cache for all other display modes. We want to cache both file system hits and misses so that we can distinguish
// in future requests whether a file's status was evicted from the cache (null value) or if the file doesn't exist (empty string).
IEnumerable<IDisplayMode> allDisplayModes = DisplayModeProvider.Modes;
foreach (IDisplayMode displayMode in allDisplayModes)
{
if (displayMode.DisplayModeId != virtualPathDisplayInfo.DisplayMode.DisplayModeId)
{
DisplayInfo displayInfoToCache = displayMode.GetDisplayInfo(controllerContext.HttpContext, virtualPath, virtualPathExists: path => FileExists(controllerContext, path));
string cacheValue = String.Empty;
if (displayInfoToCache != null && displayInfoToCache.FilePath != null)
{
cacheValue = displayInfoToCache.FilePath;
}
ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, AppendDisplayModeToCacheKey(cacheKey, displayMode.DisplayModeId), cacheValue);
}
}
break;
}
searchedLocations[i] = virtualPath;
}
return result;
}
You may have noticed that I haven´t talked about a piece of code with the following comment (reformatted for clarity):
// Populate the cache for all other display modes.
// We want to cache both file system hits and misses so that we can distinguish
// in future requests whether a file's status was evicted from the cache
// (null value) or if the file doesn't exist (empty string).
That (and the piece of code below the comment :)) means that once MVC 4 has found the first valid path from the View Locations registered in the View Engine, it will also check if the view file for all of the additional display modes that were not tested exist, so that information can be included in the cache (although just for that view location and not all of the locations available in the view engine).
Notice also, how it is passing a lambda to each of the tested display modes for checking if the file for that mode exists:
DisplayInfo displayInfoToCache = displayMode.GetDisplayInfo(
controllerContext.HttpContext,
virtualPath,
virtualPathExists: path => FileExists(controllerContext, path));
So that explains why when you override FileExists it is also being called for the mobile view, even when it has already found the non-mobile view.
In any case, display modes can be removed the same way they can be added: by updating the DisplayModes collection when the application starts. For example, removing the Mobile display mode and leaving just the default and unspecific one (You cannot clear the collection or no view will ever be found):
...
using System.Web.WebPages;
...
protected void Application_Start()
{
DisplayModeProvider.Instance.Modes.Remove(
DisplayModeProvider.Instance.Modes
.Single(m => m.DisplayModeId == "Mobile"));
Quite a long answer but hopefully it makes sense!
Have you tried removing Mobile DisplayModeProvider. You can achieve this by running the following in Application_Start:
var mobileDisplayMode = DisplayModeProvider.Instance.Modes.FirstOrDefault(a => a.DisplayModeId == "Mobile");
if (mobileDisplayMode != null)
{
DisplayModeProvider.Instance.Modes.Remove(mobileDisplayMode);
}
THe problem that you are getting is an expected behavior because FindView method queries DisplayModeProvider.