I am working on Windows Application development using c#. I want to read a csv file from a directory and imported into sql server database table. I am successfully read and import the csv file data into database table if the file content is uniform. But I am unable to insert the file data with invariant form ex.Actually my csv file delimiter is tab('\t') and after getting individual fields I have a field that contains data like dcc
Name
----
xxx
xxx yyy
xx yy zz
and i rerieved data like xxx,yyy and xx,yy,zz so the insertion becomes problem.
How could i insert the data uniformly into a database table.
It's pretty easy.
Just read file line-by-line. Example on MSDN here:
How to: Read Text from a File
For each line use String.Split Method with your tab as delimiter. Method documentation and sample are here:
String.Split Method (Char[], StringSplitOptions)
Then working insert your data.
If a CSV (or TSV) value contains a delimiter inside of it, then it should be surrounded by quotes. See the spec for more details: https://www.rfc-editor.org/rfc/rfc4180#page-3
So your input file is incorrectly formatted. If you can convince the input provider to fix this issue, that will be the best way to fix the problem. If not, other solutions may include:
visually inspecting and editing the file to fix errors, or
writing your parser program to have enough knowledge of your data expectations that it can correctly "guess" where the real delimiters are.
If I'm understanding you correctly, the problem is that your code is splitting on spaces instead of on tabs. Given you have read in the lines from the file, all you need to do is:
string[] fileLines;//from the file
foreach(string line in fileLines)
{
string[] lineParts=line.Split(new char[]{'\t'});
}
and then do whatever you want with each lineParts. The \t is the tab character.
If you're also asking about writing the lines to a database file...you can just read in tab-delimited files with the Import Export Wizard (assuming you're using Sql Server Mgmt Studio, but I'm sure there are comparable ways to import using other db management software).
Related
What would be the best practice - if there is any - to parse multiple config files?
I want to parse the mysql server configuration and also write the configuration again.
The configuration allows to issue multiple lines like:
!includedir /etc/mysql.d/
So the interesting thing is, that some configuration may be located in the main file but other may be located in a sub file.
I think pyparsing only works on ONE single file or one content string.
So I probably first need to read all files and maybe restructures the contents like adding headers for the different files...
====main file====
[mysql]
....
!includedir /etc/mysql.d/
====/etc/mysql.d/my.cnf====
[client]
.....
I would only have one pyparsing call.
Then I could parse everything into one big data object, group the file sections and have the file names as keys. This way I could also write the data back to the disk...
The other possibility would be to parse the main file and programmatically parse all other files that were found in the main file.
Thus I would have several pyparsing calls.
What do you think?
In your pyparsing code, attach a parse action to the expression that matches the include statements, have it parse the contents of the referenced files or directory of files, then merge those results into the current parse output. The parse action would make the successive calls to parseString, your code would only make a single call.
See this new example added to the pyparsing examples directory: https://github.com/pyparsing/pyparsing/blob/master/examples/include_preprocessor.py
Actually my intension is talking with non-conventional database like ms=excel,txt files. When i'm trying to create microsoft text driver in data source pointing to some .csv file and also trying to some file in my drives.. in that select Directory file is not visible to pointing it is in hidden mode What is the problem.
You must point to the directory in ODBC first, then set the options for the defined column header format. The ODBC connection will recognize all text files in the directory, and follow the header formats for the columns you must specify
How do I add an html entity to my CSV?
I have an asp.net, sql server that generates html, excel, and csv files. Some of the data needs to have the ‡ entity in it. How do I get it to output to my CSV correctly? If I have it like this: ‡, then it gets screwed up but if I output it with the entity code, the CSV outputs that text.
Non-printable characters in a field are sometimes escaped using one of several c style character escape sequences, ### and \o### Octal, \x## Hex, \d### Decimal, and \u#### Unicode.
So just escape your non-ascii character C#-style and you'll be fine.
I'm not sure what you mean by "it gets screwed up".
Regardless, it is up to the receiving program or application to properly interpret the characters.
What this means is that if you put ‡ in your csv file then the application that opens the CSV will have to look for those entities and understand what to do with them. For example, the opening application would have to run an html entity decoder in order to properly display it.
If you are looking at the CSV file with notepad (for example) then of course it won't decode the entities because notepad has no clue what html entities are or even what to do when it finds them.
Even Internet Explorer wouldn't convert the entities for display when opening a CSV file. Now if you gave it a .html extension then IE would handle the display of the file with it's html rendering engine.
At the moment i get file extension of the file like :
string fileExt = System.IO.Path.GetExtension(filUpload.FileName);
But if the user change the file extension of the file ( for example user could rename "test.txt" to "test.jpg" ), I can't get the real extension . What's the solution ?
You seem to be asking if you can identify file-type from its content.
Most solutions will indeed attempt the file extension, but there are too many different possible file types to be reliably identifiable.
Most approaches use the first several bytes of the file to determine what they are.
Here is one list, here another.
If you are only worried about text vs binary, see this SO question and answers.
See this SO answer for checking if a file is a JPG - this approach can be extended to use other file headers as in the first two links in this answer.
Whatever the user renames the file extension to, that is the real file extension.
You should never depend on the file extension to tell you what's in the file, since it can be renamed.
See "how can we check file types before uploading them in asp.net?"
There's no way to get the 'real' file extension - the file extension that you get from the filename is the real one. If file content is your concern, you can retrieve the content type using the .ContentType property and verify that it is a content type that you are expecting - eg. image/jpg.
I need to export huge amount of data from ado.net datatable(which i get by db query) to excel.
I tried the following way : 1. Create excel object with workbook/worksheet # server side...and use memory stream to write whole document to client side.
But this gave me "out of Memory exception". bcoz my memory stream was was so huge.
So I replaced this with a new way - as follows :
Writing each row from datatable as a coma seperated string to client side. So, as and when we get each row ...we can write to client side ..no memory is used.
But by this way we can write to csv file...not to excel...
Does anybody know how to handle this situation.
Can I use silverlight to get data row by row from server, pass it to client side...build excel at client side.?
You should create it on the server, then copy it to the client in chunks.
For an example, see this answer.
If this is for XL 2007, then the workbook is basically in an OPEN XML file format.
If you can format the data in your datatable to conform to the OPEN XML, you can save the file and then just download the entire file.
Read up on OPEN XML at http://msdn.microsoft.com/en-us/library/aa338205.aspx