In my simple Web API, I'm working on a controller which reads a JSON file and then has to return the content as a pure JSON string. However, once read, the latter is stained by many \r, \" and \n which, AFAIK, render its direct parsing impossible. I don't really know how to solve this, I tried with some quite dumb .Replace("\r", ""), to no avail, as I don't know how to work around "\"". I'm stuck and I have no idea what to do.
Can someone please help me? I'm sorry if this has been asked and solved somewhere else, but I could not find an already answered question to solve my issue.
[Produces("application/json")]
public async Task<string> JsonData()
{
using var reader = new StreamReader("Mocks\\data.json");
string fileContent = await reader.ReadToEndAsync();
fileContent = fileContent.Replace("\r", "").Replace("\n", "").Replace("\"", "\""); // Doesn't do the job
return fileContent;
}
Here's how it's given out:
{\n\r\"name\":\"lloyd\",\n\r\"mydata\":\"somedata\",\n\r\"moredata\"...}
I'd like all the formatting stuff to disappear:
{
"name":"lloyd",
"mydata":"somedata",
"moredata": ...
}
Just convert to object and return it：
[Produces("application/json")]
[HttpGet]
public async Task<Object> JsonData()
{
var reader = new StreamReader("data.json");
string fileContent = await reader.ReadToEndAsync();
var data = JsonConvert.DeserializeObject(fileContent);
return data;
}
Related
I am creating an app where user can upload their text file and find out about its most used word.
I have tried to follow this doc to get used to the idea of using AZURE STORAGE BLOBS - https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet
But I am super newbie and having a hard time figuring it out how to adapt those blobs methods for my POST method.
This my sudo - what I think I need in my controller and what needs to happen when POST method is triggered.
a.No need for DELETE or PUT, not replacing the data nor deleting in this app
b.Maybe need a GET method, but as soon as POST method is triggered, it should pass the text context to the FE component
POST method
connect with azure storage account
if it is a first time of POST, create a container to store the text file
a. how can I connect with the existing container if the new container has already been made? I found this, but this is for the old CloudBlobContainer. Not the new SDK 12 version.
.GetContainerReference($"{containerName}");
upload the text file to the container
get the chosen file's text content and return
And here is my controller.
public class HomeController : Controller
{
private IConfiguration _configuration;
public HomeController(IConfiguration Configuration)
{
_configuration = Configuration;
}
public IActionResult Index()
{
return View();
}
[HttpPost("UploadText")]
public async Task<IActionResult> Post(List<IFormFile> files)
{
if (files != null)
{
try
{
string connectionString = Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING");
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
string containerName = "textdata" + Guid.NewGuid().ToString();
BlobContainerClient containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);
//Q. How to write a if condition here so if the POST method has already triggered and container already created, just upload the data. Do not create a new container?
string fileName = //Q. how to get the chosen file name and replace with newly assignmed name?
string localFilePath = //Q. how to get the local file path so I can pass on to the FileStream?
BlobClient blobClient = containerClient.GetBlobClient(fileName);
using FileStream uploadFileStream = System.IO.File.OpenRead(localFilePath);
await blobClient.UploadAsync(uploadFileStream, true);
uploadFileStream.Close();
string data = System.IO.File.ReadAllText(localFilePath, Encoding.UTF8);
//Q. If I use fetch('Home').then... from FE component, will it receive this data? in which form will it receive? JSON?
return Content(data);
}
catch
{
//Q. how to use storageExeption for the error messages
}
finally
{
//Q. what is suitable to execute in finally? return the Content(data) here?
if (files != null)
{
//files.Close();
}
}
}
//Q. what to pass on inside of the Ok() in this scenario?
return Ok();
}
}
Q1. How can I check if the POST method has been already triggered, and created the Container? If so how can I get the container name and connect to it?
Q2. Should I give a new assigned name to the chosen file? How can I do so?
Q3. How can I get the chosen file's name so I can pass in order to process Q2?
Q4. How to get the local file path so I can pass on to the FileStream?
Q5. How to return the Content data and pass to the FE? by using fetch('Home').then... like this?
Q6. How can I use storageExeption for the error messages
Q7. What is suitable to execute in finally? return the Content(data) here?
Q8. What to pass on inside of the Ok() in this scenario?
Any help is welcomed! I know I asked a lot of Qs here. Thanks a lot!
Update: add a sample code, you can modify it as per your need.
[HttpPost]
public async Task<IActionResult> SaveFile(List<IFormFile> files)
{
if (files == null || files.Count == 0) return Content("file not selected");
string connectionString = "xxxxxxxx";
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
string containerName = "textdata" + Guid.NewGuid().ToString();;
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
containerClient.CreateIfNotExists();
foreach (var file in files)
{
//use this line of code to get file name
string fileName = Path.GetFileName(file.FileName);
BlobClient blobClient = containerClient.GetBlobClient(fileName);
//directly read file content
using (var stream = file.OpenReadStream())
{
await blobClient.UploadAsync(stream);
}
}
//other code
return View();
}
Original answer:
When using List<IFormFile>, you should use foreach code block to iterate each file in the list.
Q2. Should I give a new assigned name to the chosen file? How can I do
so?
If you want to keep the file original name, in the foreach statement like below:
foreach (var file in myfiles)
{
Path.GetFileName(file.FileName)
//other code
}
And if you want to assign a new file name when uploaded to blob storage, you should define the new name in this line of code: BlobClient blobClient = containerClient.GetBlobClient("the new file name").
Q3. How can I get the chosen file's name so I can pass in order to
process Q2?
refer to Q2.
Q4. How to get the local file path so I can pass on to the FileStream?
You can use code like this: string localFilePath = file.FileName; to get the path, and then combine with the file name. But there is a better way, you can directly use this line of code Stream uploadFileStream = file.OpenReadStream().
Q5. How to return the Content data and pass to the FE? by using
fetch('Home').then... like this?
Not clear what's it meaning. Can you provide more details?
Q6. How can I use storageExeption for the error messages
The storageExeption does not exist in the latest version, you should install the older one.
You can refer to this link for more details.
#Ivan's answer is what the documentation seems the recommend; however, I was having a strange issue where my stream was always prematurely closed before the upload had time to complete. To anyone else who might run into this problem, going the BinaryData route helped me. Here's what that looks like:
await using var ms = new MemoryStream();
await file.CopyToAsync(ms);
var data = new BinaryData(ms.ToArray());
await blobClient.UploadAsync(data);
This is actually a 2-part question related directly to .net core 3.0 and specifically with PipeWriter: 1) How should I read in the HttpResponse body? 2) How can I update the HttpResponse? I'm asking both questions because I feel like the solution will likely involve the same understanding and code.
Below is how I got this working in .net core 2.2 - note that this is using streams instead of PipeWriter and other "ugly" things associated with streams - eg. MemoryStream, Seek, StreamReader, etc.
public class MyMiddleware
{
private RequestDelegate Next { get; }
public MyMiddleware(RequestDelegate next) => Next = next;
public async Task Invoke(HttpContext context)
{
var httpResponse = context.Response;
var originalBody = httpResponse.Body;
var newBody = new MemoryStream();
httpResponse.Body = newBody;
try
{
await Next(context);
}
catch (Exception)
{
// In this scenario, I would log out the actual error and am returning this "nice" error
httpResponse.StatusCode = StatusCodes.Status500InternalServerError;
httpResponse.ContentType = "application/json"; // I'm setting this because I might have a serialized object instead of a plain string
httpResponse.Body = originalBody;
await httpResponse.WriteAsync("We're sorry, but something went wrong with your request.");
return;
}
// If everything worked
newBody.Seek(0, SeekOrigin.Begin);
var response = new StreamReader(newBody).ReadToEnd(); // This is the only way to read the existing response body
httpResponse.Body = originalBody;
await context.Response.WriteAsync(response);
}
}
How would this work using PipeWriter? Eg. it seems that working with pipes instead of the underlying stream is preferable, but I can not yet find any examples on how to use this to replace my above code?
Is there a scenario where I need to wait for the stream/pipe to finish writing before I can read it back out and/or replace it with a new string? I've never personally done this, but looking at examples of PipeReader seems to indicate to read things in chunks and check for IsComplete.
To Update HttpRepsonse is
private async Task WriteDataToResponseBodyAsync(PipeWriter writer, string jsonValue)
{
// use an oversized size guess
Memory<byte> workspace = writer.GetMemory();
// write the data to the workspace
int bytes = Encoding.ASCII.GetBytes(
jsonValue, workspace.Span);
// tell the pipe how much of the workspace
// we actually want to commit
writer.Advance(bytes);
// this is **not** the same as Stream.Flush!
await writer.FlushAsync();
}
Long story short, no matter what I try VeraCode continues to flag 8 lines of my code as flaws with CWE 918. This is old code so I'm not sure why it's suddenly being flagged.
Here's an example [offending] method with the flagged line in bold
public virtual async Task<HttpResponseMessage> Put(string controller = "", Dictionary<string, object> parameters = null, object body = null)
{
if (string.IsNullOrWhiteSpace(ApiBaseUrl)) return null;
HttpResponseMessage response = null;
using (var client = GetHttpClient())
{
client.BaseAddress = new Uri(ApiBaseUrl);
if (!string.IsNullOrEmpty(Token)) client.DefaultRequestHeaders.Add("Token-Key", Token);
if (!string.IsNullOrEmpty(DeviceId)) client.DefaultRequestHeaders.Add("DeviceId", DeviceId);
var url = GenerateUrl(controller, parameters);
var requestBody = GeneratedHttpContent(body);
if (requestBody == null) requestBody = new StringContent("");
**response = await client.PutAsync(url, requestBody);**
await LogError(response);
return response;
}
}
Here's my proposed fix that utilized an extension method to validate the URL
var url = GenerateUrl(controller, parameters);
var requestBody = GeneratedHttpContent(body);
if (requestBody == null) requestBody = new StringContent("");
**if (url.IsValidUrl())
{
response = await client.PutAsync(url, requestBody);
}
else
{
response = new HttpResponseMessage(HttpStatusCode.BadRequest);
}**
await LogError(response);
return response;
Here is the extension method with a VeraCode attribute
[RedirectUrlCleanser]
public static bool IsValidUrl(this string source)
{
return Uri.TryCreate(source, UriKind.RelativeOrAbsolute, out Uri uriResult) && Uri.IsWellFormedUriString(source, UriKind.RelativeOrAbsolute);
}
I can have VeraCode automatically mitigate based on the attribute, but our client will be performing their own scan and certainly won't have that setting enabled.
Any ideas on how I can resolve this would be appreciated.
The true source of the flaw is inside of your GenerateUrl method which is unfortunately not shown, but here is the general idea of what the Veracode is complaining about.
For CWE ID 918 it is hard to make Veracode recognize your fix unless you have static URL. You need to validate all your inputs that become parts of your request URL.
Below is what I found at the Veracode site:
https://community.veracode.com/s/question/0D52T00004i1UiSSAU/how-to-fix-cwe-918-veracode-flaw-on-webrequest-getresponce-method
The complete solution existed only for the case where you have single or some small number of possible input values (white list):
public WebResponse ProxyImage(string image_host, string image_path)
{
string validated_image_host = AllowedHosts.Host1;
if (image_host.Equals(AllowedHosts.Host2))
validated_image_host = AllowedHosts.Host2;
string validated_image = AllowedImages.Image1;
if (image_path.Equals(AllowedImages.Image2))
validated_image = AllowedImages.Image2;
string url = $"http://{validated_image_host}.example.com/{validated_image}";
return WebRequest.Create(url).GetResponse();
}
If the set of possible valid values is too large for that kind of validation then you need to fix the flaw by implementing dynamic validation of inputs using regular expressions. Unfortunately, Veracode is not smart enough to recognize that kind of fix, so "mitigation by design" is still required.
public WebResponse ProxyImage(string image_host, string image_path)
{
var image_host_regex = new System.Text.RegularExpressions.Regex("^[a-z]{1,10}$");
if (!image_host_regex.Match(image_host).Success)
throw new ArgumentException("Invalid image_host");
var image_path_regex = new System.Text.RegularExpressions.Regex("^/[a-z]{1,10}/[a-z]{1,255}.png$");
if (!image_path_regex.Match(image_path).Success)
throw new ArgumentException("Invalid image_host");
string url = $"http://{image_host}.example.com/{image_path}";
return WebRequest.Create(url).GetResponse();
}
Another way to fix this issue (which is kind of a hack) is to append your query string parameters in the baseAddress of the HttpClient, this way the veracode will not treat it like a flaw.
Here is how the solution would look like
public async Task<Data> GetData(string input)
{
try
{
var httpClient = new HttpClient();
//Appended the parameter in base address to
//to fix veracode flaw issue
httpClient.BaseAddress = new Uri($"https://someurl.com/somefunction/{input}");
//passing empty string in GetStringAsync to make sure
//veracode doesn't treat it like modifying url
var content = await httpClient.GetStringAsync("");
return JsonConvert.DeserializeObject<Data>(content);
}
}
The old version of this question got too long so by the end of numerous attemts to solve this issue I came up that all can be taken down to a simple question. Why does this produce a SystemObjectDisposed.
private async void PickPhotoButton_OnClicked(object sender, EventArgs e)
{
_globalStream = await DependencyService.Get<IPicturePicker>
().GetImageStreamAsync();
_globalArray = StreamToByteArray(_globalStream);
var gal = new GalleryResource()
{
Pic = _globalArray
};
MemoryObjects.CurrentGallery = gal;
var ctr = HelperMethods.GetInstance<GalleryController>();
await ctr.Post();
}
public byte[] StreamToByteArray(Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
The stream arrives from the native side, I turn it into a byte array and pass it into my repository. Everyting work with a dummy byte array so something is wrong with the stream object that possibly gets closed or disposed at point.
The exception is thrown in the repository at this point:
var response = await _client.PostAsync(endPoint, _repService.ConvertObjectToStringContent(obj));
ConvertObjectToStringContent(obj) - not this part of it. From here it actually returns with a value and the byte array is seen inside the debug ie. the byte array stay with a valid lenght all way through.
The only event that does take place when we do finish picking the photo from the library is the following:
void OnImagePickerFinishedPickingMedia(object sender,
UIImagePickerMediaPickedEventArgs args)
{
UIImage image = args.EditedImage ?? args.OriginalImage;
if (image != null)
{
// Convert UIImage to .NET Stream object
NSData data = image.AsJPEG(1);
Stream stream = data.AsStream();
// Set the Stream as the completion of the Task
taskCompletionSource.SetResult(stream);
}
else
{
taskCompletionSource.SetResult(null);
}
imagePicker.DismissModalViewController(true);
}
However it doesn't seem to dispose the stream and even if it did we already got a byte array from it.
Tried even doing this inside Native code
var client = new HttpClient();
var c = new MultipartFormDataContent();
c.Add(new StreamContent(image.AsJPEG(1).AsStream()));
var response = await client.PostAsync(Settings.EndPoint + "api/gallery/", c);
Same error.
I think your problem lies somewhere in this line _byteArray = ToByteArray(_array);
ToByteArray(stream) seems to return you the byte array maybe via conversion from a stream, and this stream might still have a reference to the bytearray. And it might have become disposed.
If it's inside this method, please post it, I wanna knowww
I'm not quite experienced enough to exactly tell what it is about, but maybe my suggestions will be hitting the right spot!
Btw your code looks real clean, I like it!
So, although this issue did come up in the first place with the CrossMedia plugin https://github.com/jamesmontemagno/MediaPlugin it did the same error.
However the error only comes up if you for instance pick a photo like this:
var _mediaFile = await CrossMedia.Current.PickPhotoAsync();
So, when I did this:
var _mediaFile = await CrossMedia.Current.PickPhotoAsync(new
Plugin.Media.Abstractions.PickMediaOptions
{
PhotoSize = Plugin.Media.Abstractions.PhotoSize.Small,
CompressionQuality = 90,
});
The error went away. No idea why.
I have an asp.net page with a WebMethod on it to pass JSON back to my javascript.
Bellow is the web method:
[WebMethod]
public static string getData(Dictionary<string, string> d) {
string response = "{ \"firstname\": \"John\", \"lastname\": \"Smith\" }";
return response;
}
When this is returned to the client it is formatted as follows:
{ \"d\": \"{ \"firstname\": \"John\", \"lastname\": \"Smith\" }\" }
The problem is the double quotes wrapping everything under 'd'. Is there something I've missed in the web method or some other means of returning the data without the quotes? I don't really want to be stripping it out on the client everytime. Also I've seen other articles where this doesn't happen.
Any help would be appreciated thanks.
I assume that you want to return the JSON representation of the object
{
firstname:"John",
lastname:"Smith"
}
but your method signature is returning a string. The ASP.Net framework serialisation is correctly serialising the string response. Put another way, if your function was
string response = "foo";
return response;
You would not be surprised if the output was
{"d":{"foo"}}
It just happens that response has double quotes that need to be escaped.
You obviously just want to get at the object. You have 2 options: -
1) use eval in your javascript to turn the string into an object e.g.
function onSuccessCallback(retval) {
var obj = eval(retval.d);
}`
2) or (and this is my prefered solution) have your method return an actual object and let the JSON serialisationof the framework do the heavy lifting for you
[WebMethod]
public static object getData(Dictionary<string, string> d) {
var response = new { firstname = "John", lastname="Smith" };
return response;
}
You will see that this generates the response that you probably originally expected (e.g.
{"d":{"firstname":"John", "lastname":"Smith"}}
Actually this entire issue exists because you're trying to out-think ASP.Net web services. You need to setup a class for your data to be returned and use that class (or List(of YourClass)) to queue up results and return them.
A great article explaining all this (a very common pitfall) is: http://encosia.com/asp-net-web-services-mistake-manual-json-serialization/
I had a similar issue with my code. I was trying to return an XmlDocument as JSON to a calling script but returning XmlDocument from the WebService returned an empty set of arrays (as XmlDocument is NOT serializable!).
If i set the ScriptService with the attribute ResponseFormat.JSON then my JSON was double-escaped.
The way to out-fox ASP.NET is to tell ASP.NET that you're returning XML and then it won't double-escape your JSON ;-)
[WebMethod(EnableSession = true)]
[ScriptMethod(ResponseFormat = ResponseFormat.Xml)]
public String MyJSONWebService(String sParam1, String sParam2, String sParam3)
{
... do stuff.....
XmlDocument oXMLDocument = new XmlDocument();
oXMLDocument.LoadXml(sXML);
sJSON = JsonConvert.SerializeXmlNode(oXMLDocument.SelectSingleNode("autnresponse"));
return sJSON;
}
I know it's a hack but .....