How to use BizTalk To disassemble Excel File .. Then Save these data in Database?
Can anyone provide me detailed steps of how to achieve this or any existing link for the same.
Wow - this is pretty open ended!
The steps you would generally take are:
1) Generate a Flat File schema that represents your excel file structure. As it's excel I'm assuming that your file is actually a CSV?
2) Create a custom pipeline that implements a flat file disassembler to convert CSV to Xml.
3) Using the WCF-LOB adapter, generate schemas for the Table you want to insert to. You might want to front this with a stored proc. I'm assuming an SQL or ORACLE database as you don't say what DB you are using!
4) Map your input Xml file to your Table/SP Schema.
5) Send your insert request to your DB (advise using Composite operations or a User defined table parameter to to avoid looping through your Xml and sending line-by-line!)
This is pretty high-level but frankly you are asking quite a few questions in one go!
HTH
In case it isn't not a CSV flat file as assumed by teepeeboy, and it is actually a XLS file you want to parse you will need something like FarPoint Spread for BizTalk. We've successfully used this to parse incoming XLS files attached to e-mails. The other option would be to write your own Pipeline component to do it but would be a lot of work. Other than that the steps that teepeeboy outline are what I would do as well.
Related
I'm currently working on a project which one of its modules is to upload a custom excel file then my code should recognize the excel file's template.
then Users should be able to determine which data is put in which excel field.
I am writing this with asp.net mvc. and so far I'm screwed. anyone knows where should I start? or what library I should Use? I'd really appreciate the help
For Fetching Excel File data in c# , I recommend you to study about Aspose.Cells, It is a third party Dll, There API is Easy to understand and Very Easy to implement.
In that you can Fetch Your Excel Files as
Workbook - Full Excel File
Worksheets - Sheets Inside Excel File
Cells - Group of Excel Cell
Cell - A single Cell which you can access in the form of 2D Array like A1 Cell of Excel Sheet Will be Cell[0][0];
It also gives you facility to put formulaes, Sorting, Blank Row Deletion and many more Things
Here you can easily Export your Excel sheet in a DataTable and vice Versa
Have a look at it is Easier to use.Thanks
https://github.com/closedxml/closedxml/wiki/Showcase
this library was close to what I needed.
There are manay examples of new using modern Get Data feature while connecting to SQL Server. However, I can't find any examples of importing data from multiple flat files (csv/txt) located in one folder.
How should I make an initial connection to the data source? Whether it should be a connection to a folder or to one of the files? How should I buid the query chaing (query M).
It seems that the way I do it in Excel does not work.
I would be gratefull for any tips.
Here are two good examples of how to import multiple text files with Power Query in Power BI.
Import all CSV files from a folder with their filenames in Power BI
https://powerpivotpro.com/2016/12/import-csv-files-folder-filenames-power-bi/
Power BI – Combine Multiple CSV files,Ignore Headers and Add FileName as a Column
http://analyticsavenue.com/power-bi-combine-multiple-csv-filesignore-headers-and-add-filename-as-a-column/
There are several ways to do this with standard SSIS tasks also, but I think the most flexible way is to use a Foreach Loop Container to read all the files in a folder. In the properties of the Foreach Loop Container you specific the folder and a file name pattern (i.e. *.csv) of the files you want to import.
You create a variable to hold the name of the current file, and use that variable to change the Connection String property of the Flat File Source each iteration of the loop.
Here's a decent example of how to do this that covers most of the setup, and provides a downloadable example.
Loop through Flat Files in SQL Server Integration Services
https://www.mssqltips.com/sqlservertip/2874/loop-through-flat-files-in-sql-server-integration-services/
I want to use XML file as a data source for my application.
What approach should I take any example??
Thanx
This tutorial will show you how to use LINQ to XML to read and also add data to an external XML file, in C#, LINQ to XML C# Tutorial.
Another tutorial regarding LINQ to XML which explains LINQ a bit more can be found here
Also why are you opting for an XML file as data storage? Hope some of this will help though!
P.S. These tutorials are not mine thus credit goes out to the authors.
XML is very inefficient at the operations you mention. Manipulating XML files without reading the whole file into memory, changing it, and writing it back out again is very likely not worth the effort.
The better bet would be to use a real database, perhaps SQLite if you need something simple and file-based. Then you can write a simple routine to dump this data to XML whenever you require.
I want to import excel data to oracle DB. I got enough help for Excel part, can you guys help me in Oracle side?
Is it possible to import to oracledb with filehelpers? Please provide some sample code for reference.
Thanks
Dee
If you save the spreadsheet data as .csv files it is relatively straightforward to import it into Oracle using SQLLoader or external tables. Because we can use SQL with external tables they are generally eaiser to work with, and so are preferable to SQLLoader in almost all cases. SQL*Loader is the betterchoice when dealing with huuuuge amounts of data and when ultra-fast loading is paramount.
Find out more about external tables here. You'll find the equivalent reference for SQL*Loader here.
A simple and relatively fool-proof way to do that for one-off tasks is to create a new column in the excel sheet, containing a formula like that:
="insert into foobar values('"&A1&"','"&A2&"');"
Copy that to all rows, then copy the whole column into an editor and run it in SQL*Plus or sql developer.
I am using Sqlloader to upload data from CSV to Oracle DB.
Anyone know of some code out there that does this already? I have a bunch of pages with data grids on them in an admin website they want to export them to Excel, was hoping someone had this written already - or if not I'll post mine when I am done.
Excel can open files in the format of CSV and XML - it can also generate the schema file.
if you have data grids then the chances are you have data sets.
There is a method on the dataset to output as XML.
you probably have to use an xmlWriter to save the file.