This is my problem:
I'm reading data from an Excel file on a .NET MVC app, what I'm doing is to read all data from the excel and then loop over each record inserting the data contained in the record into my business model.
All works perfectly. However, I've found that one field, sometimes, return an empty string when retrieved from the excel. Curiously this field can contain a simple string or a string that will be treated as an array (it can include '|' characters to build the array) on some excel files the field returns empty when the '|' char is present and in others when it isn't, and this behaviour is consistent all along that file.
There are other fields that can receive the separator and work always ok. The only difference between both fields are that the working ones are pure strings and the one that's failing is a string of numbers with possibles '|' separating them.
I've tried to change the separator character (I tried with '#' with same results) and to specifically format the cells as text without any success.
This is the method that extracts data from the excel
private DataSet queryData(OleDbConnection objConn) {
string strConString = "SELECT * FROM [Hoja1$] WHERE NUMACCION <> ''";
OleDbCommand objCmdSelect = new OleDbCommand(strConString, objConn);
OleDbDataAdapter objAdapter1 = new OleDbDataAdapter();
objAdapter1.SelectCommand = objCmdSelect;
DataSet objDataset = new DataSet();
objAdapter1.Fill(objDataset, "ExcelData");
return objDataset;
}
I first check the fields from the excel with:
fieldsDictionary.Add("Hours", table.Columns["HOURS"].Ordinal);
And later, when looping through the DataSet I extract data with:
string hourString = row.ItemArray[fieldsDictionary["Hours"]].ToString();
This hourString is empty in some records. In some Excel files it's empty when the record contains '|', on others it's empty when it doesn't. I haven't found yet a file where it returns empty on records of both classes.
I'm quite confused about this. I'm pretty sure it has to be related to the numerical nature of field data, but cannot understand why it doesn't solve when I force the cells on the excel file to be "text"
Any help will be more than welcome.
Ok. I finally solved this.
It seems like Excel isn't able to recognize a whole column as same data type if it contains data of possibly different classes. This happens even if you force the cell format to be text on the workbook, as when you query the data it will recognize the field as a determinated type according to the first record it receives; that was the reason why different files emptied different type of records, files starting with a plain text emptied numeric values and vice versa.
I've found a solution to this just changing the connection string to Excel.
This was my original connection string
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=pathToFile;Extended Properties="Excel 8.0;HDR=Yes;"
And this the one that fixes the problem
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=pathToFile;Extended Properties="Excel 8.0;HDR=Yes;IMEX=1"
The parameter IMEX=1 states to excel that it must manage all mixed data columns as plain text. This won't work for you if you need to edit the excel file, as this parameter also opens it on read-only mode. However it was perfect for my situation.
Related
I'm working in a program to download all the attachments linked to one record on database, the record on database is stored like "attach1.pdf attach2.docx".
How could i separate in two differents strings?
Something like this:
string1=Attach1.pdf
string2=attach2.docx
Dim fileNames As String() = String.Split({" "c})
Space delimiting file names is incredibly dumb since most filesystems consider the space character to be a valid path character.
I am using LinqToExcel to easily import Excel data into SQL Server.
var fileName = ConfigurationManager.AppSettings["ExcelFileLocation"];
var excel = new ExcelQueryFactory(fileName);
var employees = excel.Worksheet<Employee>().ToList();
Everything's fine, there is only 1 problem, the fields mapped exactly to the database table fields, and in the database they are NOT NULL.
Having said that, if you look at this screenshot of Excel file, some rows below row 3 are actually not empty. There are no spaces, but somehow LinqToExcel reads them as well and of course I get exception thrown by EntityFramework saying the field cannot be null.
I need to select all the blank rows below 3 up to 8980 something, and delete them.
Only then I can use LinqToExcel not trying to import blank rows.
Any idea how to solve the problem?
Thanks.
You can add a condition to the LINQ statement so the empty rows are not included.
var employees = excel.Worksheet<Employee>().Where(x => x.VS2012 != null).ToList();
And if checking for not null does not work, then you can check for an empty string
var employees = excel.Worksheet<Employee>().Where(x => x.VS2012 != "").ToList();
but somehow LinqToExcel reads them as well
This is a quirk of Excel. It remembers how many rows and columns where used when the Sheet was at it largest size. You can see this by typing Ctrl-End. This will select the cell in the last row and column ever used.
Office support describes how to reset the last cell: Locate and reset the last cell on a worksheet
Basically, you delete excess rows and columns, clear formatting, save the workbook and reopen it.
This work-around could be useful if you have Excel-files waiting to be imported and no time to deploy your fixed Linq2Excel client.
I have a sqlite database from which I want to extract a column of information with the datatype BLOB. I am trying this:
SELECT cast(data as TEXT) FROM content
This is obviously not working. The output is garbled text like this:
x��Uak�0�>�8�0Ff;I�.��.i%�A��s�M
The data in the content column is mostly text, but may also have images (which I recognized could cause a problem if I cast as TEXT). I simply want to extract that data into a usable format. Any ideas?
You can use
SELECT hex(data) FROM content
or
SELECT quote(data) FROM content
The first will return a hex string (ABCD), the second quoted as an SQL literal (X'ABCD').
Note that there's (currently) no way of converting hexadecimal column information back to a BLOB in SQLite. You will have to use C/Perl/Python/… bindings to convert and import those.
You can write some simple script which will save all blobs from your database into files. Later, you can take a look at these files and decide what to do with them.
For example, this Perl script will create lots of files in current directory which will contain your data blob fields. Simply adjust SELECT statement to limit fetched rows as you need:
use DBI;
my $dbh = DBI->connect("dbi:SQLite:mysqlite.db")
or die DBI::errstr();
my $sth = $dbh->prepare(qq{
SELECT id, data FROM content
});
$sth->execute();
while (my $row = $sth->fetchrow_hashref()) {
# Create file with name of $row->{id}:
open FILE, ">", "$row->{id}";
# Save blob data into this file:
print FILE $row->{data};
close FILE;
}
$dbh->disconnect();
I'm trying to get the list of field names for a given table, to turn them into a string which I can to post back as a variable to another function.
I've Googled loads of stuff regarding GetSchemaTable but all I seem to be able to output are field parameters, but not the actual value of these parameters (ie ColumnName, which is the one I actually want!)
Found this page; What is the SQL command to return the field names of a table?
But all the queries on there give me an error "You do not have access to Table 'Columns'"
I feel sure this is pretty simple, can someone give me a little function that will simply give me
fieldNames = "fieldName1, fieldName2, etc"
I am using a MySQL server and ODBC connections ASP.NET using VB.
I don't know if this will work, but give it a try.
Instantiate a OdbcCommand with something like select * from yourtable limit 0;
Load a DataTable with the datareader returned from cmd.ExecuteReader.
DataTable dt as new DataTable()
dt.Load(cmd.ExecuteReader())
Now iterate through the columns of dt to find out what the column names are.
This is just and idea. Don't know if it will work or not.
I'm using this code to store korean string in my database:
Dim username As String = Request.QueryString.Get("Some Korean String")
Using dg As New DataContext()
Dim newfriend As New FriendsTable With {.AskingUser = User.Identity.Name, .BeingAskedUser = username, .Pending = True}
dg.FriendsTables.InsertOnSubmit(newfriend)
dg.SubmitChanges()
end using
Checking my database, the username stored is a string"????"...
anybody got an idea how this happened or any workarounds?
What is your database collation? Are you able to store Korean strings with any other data access technology? What is the type of the username column, and is it accurately mapped in LINQ to SQL?
I suspect that something in the database isn't set up correctly to allow full Unicode. I very much doubt that this has anything to do with LINQ itself.
The other thing to check is that you're actually getting the right data in the first place. There are often several places where things can go wrong - you need to validate each place separately to see where the data is being corrupted. I have a short article on this which you may find helpful.
It sounds like you are storing Korean text in a varchar/text column which is not using a Korean collation. Thea easiest fix is to change the column type to nvarchar/ntext.
The nchar column types store Unicode data, whereas the char and varchar types store single byte characters in the specified collation.