I have a foreach loop which cycles through a bunch of images, and uploads them using this code:
foreach(var image in fetchimages) {
string fileName = "https://www.mywebsite.co.uk" + image.ImageOriginalURL;
var uploadParams = new ImageUploadParams()
{
File = new FileDescription(fileName)
};
var uploadResult = cloudinary.Upload(uploadParams);
var mytest = new ImageUploadResult();
myurl = mytest.SecureUri;
db.Execute("UPDATE Property_Images SET NewURL = #0 WHERE ImageID = 145", myurl);
}
However, each time, the myurl variable is empty. I'm thinking i possibly have the ImageUploadResult() in the wrong place in the foreach loop?
You are creating an empty upload result and checking the value there. Please try uploadResult.SecureUri
Also, it's best to check uploadResult.Error to see if an error occurred in the upload.
Related
I am using the System.Net.WebClient.DownloadStringTaskAsync async method to upload a web page content and process it or just save it on my local folder. Everything is fine but when the web page contains some special characters like ™ or ®, they are not getting downloaded. Am I missing something here?
String contentToScrapeURL = "https://www.naylornetwork.com/aaho-advertorial/newsletter.asp?issueID=89542";
Boolean success = true;
using (System.Net.WebClient wc = new System.Net.WebClient())
{
String pageSourceCode = await wc.DownloadStringTaskAsync(contentToScrapeURL);
String path = #"C:\MyProjects\TestingThings\App_Data\" + "test.html";
File.WriteAllText(path, pageSourceCode);
}
Found it, or remembered it.
I did set the System.Net.WebClient.Encoding to Encoding.UTF8
So this below is the updated code
using (System.Net.WebClient wc = new System.Net.WebClient())
{
wc.Encoding = Encoding.UTF8;
String pageSourceCode = await wc.DownloadStringTaskAsync(contentToScrapeURL);
String path = #"C:\MyProjects\TestingThings\App_Data\" + "test.html";
File.WriteAllText(path, pageSourceCode);
}
I want to read a 50milion record from 15G txt file and write in to elastic search
if (file.Length > 0)
{
string wwroot = _he.WebRootPath;
string contentpath = _he.ContentRootPath;
string path = Path.Combine(wwroot, "file/" + foldername);
if (!Directory.Exists(path))
{
var rcheck = Directory.CreateDirectory(path);
}
var filename = file.FileName;
var filepath = Path.Combine(path, filename);
if (filepath.Any())
{
using (FileStream stream = new FileStream(Path.Combine(path, filename), FileMode.Create))
{
file.CopyTo(stream);
}
}
string[] lines = System.IO.File.ReadAllLines(filepath);
var Plist = new List<Person>();
int i = 0;
foreach (var line in lines)
{
var newperson = new Person();
string[] sub = line.Split(":");
newperson.PId = sub[1];
newperson.FirstName = sub[2];
newperson.LastName = sub[3];
newperson.Gender = sub[4];
Plist.Add(newperson);
}
return View();
I can read and upload file but when in want to add to list I get error and only read 16000 items and my application is shutdown.
You need to read the file using a buffer. With a proper reading logic based on a buffer, you'll be able to read a file of any size.
This line here:
System.IO.File.ReadAllLines(filepath);
Reads ALL the content of 15 GB file at once, and attempts to put it all into memory. I don't know how your code managed to get past that line without throwing an OutOfMemoryException (reading "only" 4.62 GB file ate 19.2 GB of my memory when debugging).
Instead, use a buffer of a single line:
using var streamReader = File.OpenText(bigFilePath);
var fileLine = string.Empty;
while ((fileLine = streamReader.ReadLine()) != null)
{
// Your string line reading logic.
}
You will most probably not be able to keep all the records in the memory (depending on memory available), also sending them one by one to Elasticsearch would be an opposite of efficiency... so, you'll need to find a middle ground between those limitations. I would suggest batching, that is, sending records in a fixed-size groups. The size is for you to pick, but note that it shouldn't be super large or minimal, otherwise the benefits of using batching will be smaller.
Full code:
static void Main()
{
string wwroot = _he.WebRootPath;
string contentpath = _he.ContentRootPath;
string path = Path.Combine(wwroot, "file/" + foldername);
var peopleListBatch = new List<Person>();
const int BatchSize = 1024;
using var streamReader = File.OpenText(path);
var fileLine = string.Empty;
while ((fileLine = streamReader.ReadLine()) != null)
{
var lineParts = fileLine.Split(":");
var newperson = new Person
{
PId = lineParts[1],
FirstName = lineParts[2],
LastName = lineParts[3],
Gender = lineParts[4],
};
peopleListBatch.Add(newperson);
// Add to Elastic, but only when batch is full.
if (peopleListBatch.Count == BatchSize)
{
AddPersonsToElasticSearch(peopleListBatch);
peopleListBatch.Clear();
}
}
// Add remaining people, if any.
if (peopleListBatch.Count > 0)
{
AddPersonsToElasticSearch(peopleListBatch);
peopleListBatch.Clear();
}
}
Inserting to Elasticsearch is another story, and I leave that task to you:
static void AddPersonsToElasticSearch(List<Person> people)
{
// TODO: Add your inserting logic here.
}
Here is what I'm trying to accomplish.
I am creating an asp.net MVC application. My restrictions are that I cannot programmatically save anything to the file structure of the server, so I can't save it as a physical file on the host, and then grab it for client download.
I am loading a PDF to a stream, extracting information from the PDF, dynamically building an excel file, and then offering the file for download to the client. My code is below.
// Loads the incoming PDF document to stream
PdfDocument doc = new PdfDocument();
using (var stream = model.BudgetPdfFile.OpenReadStream())
{
doc.LoadFromStream(stream);
}
var pageCount = doc.Pages.Count;
var date = DateTime.Now.ToShortDateString().Replace("/", "-");
// Extracts data from the PDF and separates it by NewLine
SimpleTextExtractionStrategy strategy = new SimpleTextExtractionStrategy();
StringBuilder allText = new StringBuilder();
for (var i = 0; i < pageCount; i++)
{
allText.Append(doc.Pages[i].ExtractText(strategy));
}
var fullDocText = allText.ToString();
List<string> linesList = new List<string>(fullDocText.Split(new[] { Environment.NewLine }, StringSplitOptions.None).ToList());
// generates a comparison list for output data manipulation from static data
var finalList = linesList.BuildFinalList(budgetItems);
// creates a new Spire.PDF.Workbook for the final output excel file
var result = new Workbook();
// checks for whether the submitted budget is for a case in litigation or not and builds the correct excel workbook
if (model.isTrial)
{
result = ExportExcelBudget.TrialBudgetSheet(model, finalList);
}
else
{
result = ExportExcelBudget.PreTrialBudgetSheet(model, finalList);
}
Absolutely everything up to the last section below works perfectly. However, I cannot figure out how to load the workbook into a new stream and then return the file for download.
// saves the final workbook to a stream and offers it for download to the client
Stream outStream = new MemoryStream();
var fileName = "Budget Report_" + model.ClaimNumber + "_" + date + ".xlsx";
var contentType = "application/vnd.ms-excel";
result.SaveToStream(outStream, Spire.Xls.FileFormat.Version2016);
return File(outStream, contentType, fileName);
I've searched and tried multiple different variations but when the application hits the return File(), it returns a null.
I've stepped through execution and the results seem to be there, but it's not passing anything. Any help on what is wrong here would be greatly appreciated.
Stream outStream = new MemoryStream();
var fileName = "Budget Report_" + model.ClaimNumber + "_" + date + ".xlsx";
var contentType = "application/vnd.ms-excel";
result.SaveToStream(outStream, Spire.Xls.FileFormat.Version2016);
**outStream.Position = 0;**
return File(outStream, contentType, fileName);
Had to reset the stream position to 0. Working perfectly now.
I have an ASP.NET MVC application that sends parameters to an R script, the R script then generates a file and puts it in a folder locally. The process works perfectly on my local machine through Visual Studio but goes in and out after I publish and use IIS. Below is how I am initializing R.NET and getting these values from my view via AJAX.
I also have placed both of these PATHs in my system environment variables.
If anyone knows why IIS only works right after I restart my OS and then stops working shortly after I would greatly appreciate it. Seems odd that I have no problems in Visual Studio that I face in IIS.
[HttpGet]
public JsonResult Index1(string modelTypes = null, string fileNames = null, string[] locations = null, string lowerLimits = null, string upperLimits = null, string areas = null, string productivities = null, string[] fobs = null)
{
#ViewBag.UserName = System.Environment.UserName;
string userName = #ViewBag.UserName;
//Declare field parameters
var strModelTypeValue = modelTypes;
string strFileName = fileNames;
var strLocationValue = locations;
var strLowerLimitValue = lowerLimits;
var strUpperLimitValue = upperLimits;
string strAreaValue = areas;
string strProductivityValue = productivities;
string strFobValue = fobs.ToString();
var libraryLocation = "C:/Users/" + userName + "/Documents/R/win-library/3.2";
string rPath = #"C:\Program Files\R\R-3.3.2\bin\x64";
string rHome = #"C:\Program Files\R\R-3.3.2";
//Initialize REngine
REngine.SetEnvironmentVariables(rPath, rHome);
REngine engine = REngine.GetInstance();
engine.Initialize();
if (fobs.Length > 1)
{
strFobValue = string.Join(" ", fobs);
}
else
{
strFobValue = "ALL";
}
//Declare R Script path
var rScriptPath = "C:/Users/" + userName + "/Documents/R/RWDir/Loop_Optimization.R";
//Check to see if there was more than one location selected
if (strLocationValue.Length > 1)
{
foreach (var item in strLocationValue)
{
//Set string location to each location value in loop
var strlocation = item;
//Add values to parameter list
var myParams = new List<string>
{
strModelTypeValue,
strFileName,
strlocation,
strLowerLimitValue,
strUpperLimitValue,
strAreaValue,
strProductivityValue,
strFobValue,
libraryLocation
};
//Set myParams as arguments to be sent to r script
engine.SetCommandLineArguments(myParams.ToArray());
engine.Evaluate("source('" + rScriptPath + "')");
}
}
//Only one location specified, no need to loop
else
{
foreach (var item in strLocationValue)
{
//Set string location to each location value in loop
var strlocation = item;
var myParams = new List<string>
{
strModelTypeValue,
strFileName,
strlocation,
strLowerLimitValue,
strUpperLimitValue,
strAreaValue,
strProductivityValue,
strFobValue,
libraryLocation
};
engine.SetCommandLineArguments(myParams.ToArray());
engine.Evaluate("source('" + rScriptPath + "')");
}
}
//engine.Evaluate("source('" + rScriptPath + "')");
//engine.Dispose();
return Json("success", JsonRequestBehavior.AllowGet);
}
have you done any debugging by attaching process .
I am trying to programatically build a list of files in a folder, with certain attributes like file size and modified date.
I can return the file name, but any other attribute throws an error: System.IO.IOException: The filename, directory name, or volume label syntax is incorrect.
What am I missing here?
private void BuildDocList()
{
var files = Directory.GetFiles(Server.MapPath(FilePath));
foreach (var f in files)
{
var file = new FileInfo(FilePath + f);
var fileItem = new ListItem();
// this line works fine
fileItem.Text = file.Name.Split('.')[0] + ", ";
// this line causes the runtime error
fileItem.Text = file.CreationTime.ToShortDateString();
FileList.Items.Add(fileItem);
}
}
You're trying to use the wrong filename for the FileInfo - you're using the unmapped path. You should use something like this:
string directory = Server.MapPath(FilePath);
string[] files = Directory.GetFiles(directory);
foreach (string f in files)
{
FileInfo file = new FileInfo(Path.Combine(directory, f));
// Now the properties should work.