I am using ASP.NET VB.NET, CSV File upload and FileHelpers, ASP.NET control is File Upload
Code to read CSV File
Using sr As New StreamReader(FileUpload1.PostedFile.InputStream)
Dim engine As New FileHelperEngine(GetType([MyClass]))
For Each entry As [MyClass] In engine.ReadStream(sr)
Next
End Using
FileHelpers
<DelimitedRecord(",")> _
<IgnoreFirst(1)> _
<IgnoreEmptyLines> _
Class [MyClass]
<FieldTrim(TrimMode.Both)> _
Public Prop1 As String
<FieldTrim(TrimMode.Both)> _
Public Prop2 As String
End Class
Question
There is some comma in data and due to that It is crashing.
VB.net provides exactly what you are looking for in Microsoft.VisualBasic.FileIO. You can use it in C# however, as shown below:
TextFieldParser parser = new TextFieldParser(pFileName);
parser.SetDelimiters(",");
parser.TextFieldType = FieldType.Delimited;
while (!parser.EndOfData)
{
string current = parser.ReadFields();
}
Edit
Just noticed you are using VB - basic premise remains the same, but don't know enough about VB.net to rewrite it. It will work though, as I had the exact same problem and it fixed it.
If your CSV looks like:
value1,value2,valu,e3
it always will be 4 columns not 3
But if CSV looks like:
value1,value2,"valu,e3"
I recomended use regexp split like:
var result = Regex.Split(csvline, ",(?=(?:[^']*'[^']*')*[^']*$)");
In this case,the file has errors and it's pretty much impossible to know what's wrong with an individual line except to say "there are too many fields when split by comma". There is no sane way your code can fix this error, you have to throw it back to the user who is uploading the broken file.
You can set the FileHelper's ErrorMode setting to SaveAndContinue (see here) so that it doesn't throw on error, and after your loop check the Errors property and if it's not empty, report the problem to the user, listing out each ErrorInfo in the Errors collection.
Related
I'm trying to do some simple webscraping in OpenOffice (I usually work in Excel but I'm trying to port something over for a coworker that doesn't have Excel). However, when I try to run something very similar to this, it keeps giving me this BASIC runtime error 1.
Here's the relevant code, I'm not sure what I'm supposed to do to make this work.
Sub Macro1
Dim explorer As Object
Set explorer = CreateObject("InternetExplorer.Application")
explorer.Visible = True
explorer.navigate("www.yahoo.com")
Const READYSTATE_COMPLETE As Long = 4
Do While explorer.Busy Or explorer.readyState <> READYSTATE_COMPLETE
Loop
dim page as object
set page = explorer.Document
dim mailButton as object
set mailButton = page.GetElementByID("ybar-navigation-item-mail") 'this is the line the error occurs on
mailButton.Click
End Sub
Do you know that you can save script in vbs file (you have to delete types in variables declarations) and run it directly by double click without using office application? I recommend you to use this way.
I need to change dataset connectionstrings to point to different DBs at run time.
I've looked at a number of solutions however they all seem to be related to WinForms or web application projects or other technology slightly different than what I'm using, so I haven't figured out how apply them.
The application is like a discussion. It's a web site project based on code originally written under VS2005, and there's no budget (or personal talent!) for major changes at this time. The app is written in vb.net; I can understand answers in c#. I'm working in VS2013.
The app has three typed datasets pointing to one MDF, call it "MainDB.mdf". There are dozens of tableadapters among the three datasets.
I'm going to deploy the app it as an "alpha/demo" version. I would like to use the same code base for all users, and a separate physical version of MainDB for each user, to reduce chances that the users crash each other.
The initial demo access URL will contain query string information that I can use to connect the user with the right physical database file. I should be able to identify the database name and thus the connection string parameters from the query string information (probably using replace on a generic connection string). If necessary I could use appsettings to store fully formed connection strings, however, I would like to avoid that.
I would like to be able to change the connection strings for all the datasets at the time that the entry point pages for the app are accessed.
Changing the tableadapter connection strings at each instantiation of the tableapters would require too much code change (at least a couple of hundred instantiations); I'd just make complete separate sites instead of doing that. That's the fall back position if I can't dynamically change the connectionstrings at runtime (or learn some other way to make this general scheme work).
Any suggestions on how to approach this would be appreciated.
Thanks!
UPDATE: Per comments, here is a sample instantiation of tableadapter
Public Shared Sub ClearOperCntrlIfHasThisStaff( _
varSesnID As Integer, varWrkprID As Integer)
Dim TA As GSD_DataSetTableAdapters.OPER_CNTRLTableAdapter
Dim DR As GSD_DataSet.OPER_CNTRLRow
DR = DB.GetOperCntrlRowBySesnID(varSesnID)
If IsNothing(DR) Then
Exit Sub
End If
If DR.AField = varWrkprID Then
DR.AField = -1
TA.Update(DR)
DR.AcceptChanges()
End If
End Sub
UPDATE: Below is the test code I tried in a test site to modify the connectionString in a single instantiation of a tableadapter. It feeds a simple gridview. I tried calling this from Page_Load, Page_PreLoad, ObjectDataSource_Init, and Gridview_Databind. At the concluding response.writes, the wrkNewConnString looks changed to TestDB2, and the TA.Connection.ConnectionString value looks changed to TestDB2, but the displayed gridview data is still from TestDB1. Maybe it needs to be called from somewhere else?
Sub ChangeTableAdapter()
Dim wrkNewConnStr As String = ""
Dim wrkSel As Integer
wrkSel = 2
wrkNewConnStr = wrkNewConnStr & "Data Source=.\SQLEXPRESS;"
wrkNewConnStr = wrkNewConnStr & "AttachDbFilename=D:\9000_TestSite\App_Data\TESTDB1.MDF;Integrated Security=True;User Instance=True"
Select Case wrkSel
Case 1
wrkNewConnStr = wrkNewConnStr.Replace("TESTDB1", "TESTDB1")
Case 2
wrkNewConnStr = wrkNewConnStr.Replace("TESTDB1", "TESTDB2")
Case 3
wrkNewConnStr = "Data Source=localhost; Initial Catalog=test01;"
wrkNewConnStr = wrkNewConnStr & " User ID=testuser1; Password=testuserpw1"
End Select
Try
Dim TA As New DataSetTableAdapters.NamesTableAdapter
TA.Connection.ConnectionString = wrkNewConnStr
Response.Write("1 - " & wrkNewConnStr)
Response.Write("<br/>")
Response.Write("2 - " & TA.Connection.ConnectionString)
Catch ex As Exception
Dim exmsg As String = ex.Message
Response.Write(exmsg)
End Try
End Sub
The connection string:
<add name="TestDB1ConnectionString"
connectionString="Data Source=.\SQLEXPRESS;
AttachDbFilename=D:\9000_TestSite\App_Data\TESTDB1.MDF;Integrated Security=True;User Instance=True"
providerName="System.Data.SqlClient" />
UPDATE: the following post has lots of solutions, however, they seem to focus on web application projects, that have a project file with settings, which this web site project does not.
link with possible solutions
UPDATE: this next link was brought to my attention, and in working on it I did get it to work, however, it still relies either on having a web application project (with project file) or modifying each table adapter as they are instantiated. So, while I'm not going to implement it, I believe that is the technical answer.
modifying connection strings
sorry if this answer is too late, but I have exactly the same problem and eventually came up with a solution using Reflection.
My solution was to "save" a new default value for the connection string in the settings at run time, which means any further use of the table adapter uses the the new connection string.
It should be noted the term "save" is misleading as the new value is lost when the application closes.
Have tested and worked perfectly.
public void ChangeDefaultSetting(string name, string value)
{
if (name == null)
throw new ArgumentNullException("name");
if (value == null)
throw new ArgumentNullException("value");
Assembly a = typeof({Put the name of a class in the same assembly as your settings class here}).Assembly;
Type t = a.GetType("{Put the full name of your settings class here}");
PropertyInfo propertyInfo = t.GetProperty("Default");
System.Configuration.ApplicationSettingsBase def = propertyInfo.GetValue(null) as System.Configuration.ApplicationSettingsBase;
//change the "defalt" value and save it to memory
def[name] = value;
def.Save();
}
I want to load XML in a single table of a dataset. I use following code
string val = getAbonentInfoParametr(ai,"abonentDescription");
DataSet ds = new DataSet();
ds.ReadXml(new StringReader(val));
but when I do this, I got three tables because in one node of XML file I now get HTML code that I want to have like a string field in my only table. What should I do?
Also I prefer not to use scheme files because the structure of that xml file can be changeable except several field that I use, please suggest me something.
Use CDATA to wrap around the HTML, otherwise there is no way to differentiate HTML from XML.
How should I go about providing download functionality on an asp.net page to download a series of rows from a database table represented as a linq2sql class that only has primitive types for members (ideally into a format that can be easily read by Excel)?
E.g.
public class Customer
{
public int CustomerID;
public string FirstName;
public string LastName;
}
What I have tried so far.
Initially I created a DataTable, added all the Customer data to this table and bound it to a DataGrid, then had a download button that called DataGrid1.RenderControl to an HtmlTextWriter that was then written to the response (with content type "application/vnd.ms-excel") and that worked fine for a small number of customers.
However, now the number of rows in this table is >10,000 and is expected to reach upwards of 100,000, so it is becoming prohibitive to display all this data on the page before the user can click the download button.
So the question is, how can I provide the ability to download all this data without having to display it all on a DataGrid first?
After the user requests the download, you could write the data to a file (.CSV, Excel, XML, etc.) on the server, then send a redirect to the file URL.
I have used the following method on Matt Berseth blog for large record sets.
Export GridView to Excel
If you have issues with the request timing out try increasing the http request time in the web.config
Besides the reasonable suggestion to save the data on server first to a file in one of the answers here, I would like to also point out that there is no reason to use a DataGrid (it’s one of you questions as well). DataGrid is overkill for almost anything. You can just iterate over the records, and save them directly using HtmlTextWriter, TextWriter (or just Response.Write or similar) to the server file or to a client output stream. It seems to me like an obvious answer, so I must be missing something.
Given the number of records, you may run into a number of problems. If you write directly to the client output stream, and buffer all data on server first, it may be a strain on the server. But maybe not; it depends on the amount of memory on the serer, the actual data size and how often people will be downloading the data. This method has the advantage of not blocking a database connection for too long. Alternatively, you can write directly to the client output stream as you iterate. This may block the database connection for too long as it depends on the download speed of the client. But again; it your application is of a small or medium size (in audience) then anything is fine.
You should definitely check out the FileHelpers library. It's a freeware, excellent utility set of classes to handle just this situation - import and export of data, from text files; either delimited (like CSV), or fixed width.
It offer a gazillion of options and ways of doing things, and it's FREE, and it works really well in various projects that I'm using it in. You can export a DataSet, an array, a list of objects - whatever it is you have.
It even has import/export for Excel files, too - so you really get a bunch of choices.
Just start using FileHelpers - it'll save you so much boring typing and stuff, you won't believe it :-)
Marc
Just a word of warning, Excel has a limitation on the number of rows of data - ~65k. CSV will be fine, but if your customers are importing the file into Excel they will encounter that limitation.
Why not allow them to page through the data, perhaps sorting it before paging, and then give them a button to just get everything as a cvs file.
This seems like something that DLinq would do well, both the paging, and writing it out, as it can just fetch one row at a time, so you don't read in all 100k rows before processing them.
So, for cvs, you just need to use a different LINQ query to get all of the rows, then start to save them, separating each cell by a separator, generally a comma or tab. That could be something picked by the user, perhaps.
OK, I think you are talking too many rows to do a DataReader and then loop thru to create the cvs file. The only workable way will be to run:
SQLCMD -S MyInstance -E -d MyDB -i MySelect.sql -o MyOutput.csv -s
For how to run this from ASP.Net code see here. Then once that is done, your ASP.Net page will continue with:
string fileName = "MyOutput.csv";
string filePath = Server.MapPath("~/"+fileName);
Response.Clear();
Response.AppendHeader("content-disposition",
"attachment; filename=" + fileName);
Response.ContentType = "application/octet-stream";
Response.WriteFile(filePath);
Response.Flush();
Response.End();
This will give the user the popup to save the file. If you think more than one of these will happen at a time you will have to adjust this.
So after a bit of research, the solution I ended up trying first was to use a slightly modified version of the code sample from http://www.asp.net/learn/videos/video-449.aspx and format each row value in my DataTable for CSV using the following code to try to avoid potentially problematic text:
private static string FormatForCsv(object value)
{
var stringValue = value == null ? string.Empty : value.ToString();
if (stringValue.Contains("\"")) { stringValue = stringValue.Replace("\"", "\"\""); }
return "\"" + stringValue + "\"";
}
For anyone who is curious about the above, I'm basically surrounding each value in quotes and also escaping any existing quotes by making them double quotes. I.e.
My Dog => "My Dog"
My "Happy" Dog => "My ""Happy"" Dog"
This appears to be doing the trick for now for small numbers of records. I will try it soon with the >10,000 records and see how it goes.
Edit: This solution has worked well in production for thousands of records.
I want to write some stats to a text file every time a person loads a page. But every once in awhile I am getting at 'Could Not Open File, Already in use' type of error. I can not 100% replicate this error it is very erratic. My code is
Public Sub WriteStats(ByVal ad_id As Integer)
Dim ad_date As String = Now.Year & Now.Month
Dim FILENAME As String = Server.MapPath("text/BoxedAds.txt")
Dim objStreamWriter As StreamWriter
objStreamWriter = File.AppendText(FILENAME)
objStreamWriter.WriteLine(ad_id & ";" & ad_date)
objStreamWriter.Close()
End Sub
My question is, how can I lock and unlock the file so I stop getting the erratic errors?
Thanks
If two or more requests hit your web server at roughly the same time, they will all try to open the same file. You will need to create unique file names for each request.
Public Sub WriteStats(ByVal ad_id As Integer)
Dim ad_date As String = Now.Year & Now.Month
Dim FILENAME As String = Server.MapPath("text/BoxedAds.txt")
Dim index As Integer
Using fs As New IO.FileStream(FILENAME, IO.FileMode.Append, IO.FileAccess.Write, IO.FileShare.ReadWrite), _
tl As New TextWriterTraceListener(fs)
index = Trace.Listeners.Add(tl)
Trace.WriteLine(ad_id & ";" & ad_date)
Trace.Listeners(index).Flush()
Trace.Flush()
End Using
Trace.Listeners.RemoveAt(index)
End Sub
Three important things here:
Use of IO.FileShare.ReadWrite to allow multiple writers on the file at once.
The Using statement to make sure the stream is closed immediately, even if an exception occurs. This will minimize collisions
The TextWriterTraceListener will create a temp file for your if it can't open the file you request, to make sure the message isn't lost.
You will have to handle the exception and build some handling to re-try writing to the file after a short random interval.
If you get too much contention then it might make more sense to log it to a table in a database and create a process to export to a file (if its still needed)
I haven't had any trouble with short info using:
File.AppendAllText(path, info);
Regarding the comment on it causing locks, from reflector it uses the same options explained very well by Joel. It does not use the trace writer, so it won't output to a temp file in the case of high load / large content causing trouble.
If the info is large, you really want separate files. For high load, I would go with Joel's suggestion and create a temp file, which can be alternatively done by catching the exception on File.AppendAllText, and using the same File.AppeandAllText with a unique filename.