I am populating a DataSet with data from an excel sheet. I now need to insert this data into a table in an Access database that has identical structure.
Convention would dictate that I iterate over the rows of the DataTable and make an INSERT query for each, but I was wondering if there is a more efficient way to accomplish this? Perhaps something analogous to SqlBulkCopy for SQL Server?
Here is the code I have so far:
Dim connection As OleDbConnection = New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & Server.MapPath(path) & ";Extended Properties=Excel 12.0;")
Dim adapter = New OleDbDataAdapter("SELECT * FROM [Sheet1$] WHERE EmpID IS NOT NULL", connection)
Dim results = New System.Data.DataSet
connection.Open()
adapter.Fill(results)
connection.Close()
connection = New System.Data.OleDb.OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & Server.MapPath("~/App_Data/Leaves.accdb"))
`Now, short of a loop, how can I insert results.Tables(0) into my database?
Thanks for your help.
You can use ADO, for example:
INSERT INTO Table1 ( ADate )
SELECT SomeDate FROM [Excel 8.0;HDR=YES;DATABASE=Z:\Docs\Test.xls].[Sheet1$a1:a4]
Where the connection is to the Access database. If you wish to use the whole sheet, refer to [Sheet1$] as your table, if you wish to use a named range, just refer to it by name.
Related
I want to import data (not structure) from Access to SQLite via ODBC
and it works, but there is one point that is annoying:
I don't want to create a new table automatically - I want to import data into my existing table and I get the following error:
ODBC--call failed
table "x" already exists(1)(#1)
I want to find a way to avoid creating new table in sqlite.
I have also tried another solution for importing my data from Access to SQLite via csv. However, my encoding is utf-8 but it shows data as question marks!
There are many ways to move data around using Access.
When appending data to ODBC tables, I recommend the following steps:
Create a linked table to the ODBC table you want to import to in Access using Import -> ODBC database
Use an append query to move the data from the local table to the ODBC table
(optional) Remove the linked table.
Alternatively, Access can directly run append queries to ODBC data sources without creating linked tables, but this requires some SQL knowledge. Syntax is the following:
INSERT INTO [ODBC;MyDSNOrConnectionString].[MyRemoteTable] (Field1, Field2, FieldN)
SELECT Field1, Field2, FieldN
FROM MyLocalTable
There is another way of exporting Access table records into an exisitng ODBC table with Adobe connection, if the tables on both side have the same name and the same structure and if the table structure is stored in a system table.
Dim Str_Sql As String
Dim Conn As New ADODB.Connection
Dim myDSN As String
Dim UserName As String
Dim Password As String
Dim Rs_Tabl As DAO.Recordset
Dim Rs_TablStru As DAO.Recordset
myDSN = ...
UserName = ...
Password = ...
Conn.Open "dsn=" & myDSN & ";uid=" & UserName & ";pwd=" & Password
Set cmd = CreateObject("ADODB.Command")
Set cmd.ActiveConnection = Conn
Set Rs_Tabl = CurrentDb.OpenRecordset(TablName)
Set Rs_TablStru = CurrentDb.OpenRecordset("select * from Systable where TableName=TableName order by ColumnOrder")
Str_Sql = "truncate table TableName"
Conn.Execute (Str_Sql)
Str_Sql = "insert into TableName values ('"
Do Until Rs_Tabl.EOF
Str_Sql = "insert into TableName values ('"
Rs_TablStru.MoveFirst
Do Until Rs_TablStru.EOF
Str_Sql = Str_Sql & Rs_Tabl.Fields(Rs_TablStru.Fields("AAA")) & "','"
Rs_TablStru.MoveNext
Loop
Str_Sql = Mid(Str_Sql, 1, Len(Str_Sql) - 2) & ")"
Debug.Print Str_Sql
Conn.Execute (Str_Sql)
Rs_Tabl.MoveNext
Loop
Rs_Tabl.Close
Rs_TablStru.Close
Set Rs_Tabl = Nothing
Set Rs_TablStru = Nothing
Conn.Close
*AAA:A column from system table that stores column names from all tables.
I am trying to get multiple rows into a table hence my attempt to get the row number and put it into a for loop, the countC is exactly the same number of rows as the select statement, so the issue is not there
I'm using an oledb connection as my code is in vb asp.net but my database is in ms access 2003
For c As Integer = 1 To countC
Dim cmdstring As String
cmdstring = " INSERT INTO [KN - ProductionMachineAllocation] (BatchNo, ComponentID)
SELECT POH.BatchNo, SSCDD.ComponentID
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY BatchNo ASC) AS rownumber
([KN - ProductionOrderHeader] AS POH
INNER JOIN [FG - End Product Codes] AS EPC
ON POH.ProductID = EPC.ProductID)
INNER JOIN ([KN - ProductionOrderDetails] AS POD
INNER JOIN [FG - Style Size Comp Def Details] AS SSCDD
ON POD.SizeID = SSCDD.SizeID)
ON (POH.BatchNo = POD.BatchNo)
AND (EPC.StyleID = SSCDD.StyleID)
WHERE POH.BatchNo = '" & BatchNo & "'
) AS temptablename
WHERE rownumber IN (" & c & ");"
Dim con As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=|DataDirectory|\Shantara Production IT.mdb")
Dim cmd As New OleDbCommand(cmdstring)
cmd.CommandType = CommandType.Text
cmd.Connection = con
cmd.Connection.Open()
cmd.ExecuteNonQuery()
cmd.Connection.Close()
Next
I found out that ms access doesn't support ROW_NUMBER() so I need to find another going through each row since ms access doesn't support multi row insert by insert into select statement such as mine, any suggestions around my problem?
Most databases are able to do all this work much more efficiently entirely in the database. Certainly in SQL Server I could get entire thing down to a single query. Access is a little different, since vbscript is its procedural language, rather than something more like t-sql. There's still probably a way to do it, but since what you have works, we can at least focus on making that better.
GridViews are visual constructs that will use up extra memory and resources. If Access won't do a real INSERT/SELECT, you can at least read direct from the previous result set into your insert. You can also improve on this significantly by using parameters and re-using a single open connection for all the inserts:
Dim cnString As String = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=|DataDirectory|\Shantara Production IT.mdb"
Dim SQLDown As String = _
"SELECT DISTINCT POH.BatchNo, SSCDD.ComponentID
FROM ([KN - ProductionOrderHeader] AS POH
INNER Join [FG - End Product Codes] AS EPC
On POH.ProductID = EPC.ProductID)
INNER Join([KN - ProductionOrderDetails] AS POD
INNER Join [FG - Style Size Comp Def Details] AS SSCDD
On POD.SizeID = SSCDD.SizeID)
On (POH.BatchNo = POD.BatchNo)
And (EPC.StyleID = SSCDD.StyleID)
WHERE POH.BatchNo = ? "
Dim SQLUp As String = _
" INSERT INTO [KN - ProductionMachineAllocation]
(BatchNo, ComponentID)
VALUES( ?, ? )"
Dim dt As New DataTable
Using con As New OleDbConnection(cnString), _
cmd As New OleDbCommand(SQLDown, con)
'Guessing at parameter type/length here.
'Use the actual column type and size from your DB
cmd.Parameters.Add("#BatchNo", OleDbType.VarWChar, 10).Value = BatchNo
con.Open()
dt.Load(cmd.ExecuteReader())
End Using
Using con As New OleDbConnection(cnString), _
cmd As New OleDbCommand(SqlUp, con)
'Guessing at parameter types/lengths again
cmd.Parameters.Add("#BatchNo", OleDbType.VarWChar, 10)
cmd.Parameters.Add("#ComponentID", OleDbType.Integer)
'Connection is managed *outside of the loop*. Only one object created, only one negotiation with DB
con.Open()
For Each row As DataRow In dt.Rows
cmd.Parameters(0).Value = row(0)
cmd.Parameters(1).Value = row(1)
cmd.ExecuteNonQuery()
Next
End Using
Normally, with any ADO.Net provider you do not re-use your connection or command objects. You want a new connection object for every query sent to the DB to allow connection pooling to work correctly. Using the connection in a tight loop like this for the same query is one of the few exceptions.
I might be able to improve further by sticking with the active DataReader, rather than first loading it into a DataTable. That would allow us to avoid loading the entire result set into memory. You would only ever need one record in memory at a time. Certainly this would work for Sql Server. However, Access was designed mainly as a single-user database. It doesn't really like multiple active connections at once, and I'm not sure how it would respond.
It would also be nice to be able to do all of this work in a transactional way, where there's never any risk of it failing part way through the loop and getting stuck with half the updates. Sql Server would handle this via a single INSERT/SELECT query or with an explicit transaction. But, again, this isn't the kind of the Access is designed for. It probably does have a way to do this, but I'm not familiar with it.
OK SO I finally found a way around it, it's a bit of a long process but basically I loaded the SELECT statement(with multiple rows) into a gridview table and the used a for loop to insert it into my insert into statement. bellow is my code:
Displaying into a table
Dim Adapter As New OleDbDataAdapter
Dim Data As New DataTable
Dim SQL As String
Dim con As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=|DataDirectory|\Shantara Production IT.mdb")
Dim cmd As New OleDbCommand()
grdvmachincomp.Visible = false
SQL = "SELECT DISTINCT POH.BatchNo, SSCDD.ComponentID
FROM ([KN - ProductionOrderHeader] AS POH
INNER Join [FG - End Product Codes] AS EPC
On POH.ProductID = EPC.ProductID)
INNER Join([KN - ProductionOrderDetails] AS POD
INNER Join [FG - Style Size Comp Def Details] AS SSCDD
On POD.SizeID = SSCDD.SizeID)
On (POH.BatchNo = POD.BatchNo)
And (EPC.StyleID = SSCDD.StyleID)
WHERE POH.BatchNo = '" & BatchNo & "'"
con.Open()
cmd.Connection = con
cmd.CommandText = SQL
Adapter.SelectCommand = cmd
Adapter.Fill(Data)
grdvmachincomp.DataSource = Data
grdvmachincomp.DataBind()
cmd.Connection.Close()
Insert into through for loop
For c As Integer = 0 To grdvmachincomp.Rows.Count - 1
Dim cmdstring As String
cmdstring = " INSERT INTO [KN - ProductionMachineAllocation] (BatchNo, ComponentID) VALUES('" & grdvmachincomp.Rows(c).Cells(0).Text & "', " & grdvmachincomp.Rows(c).Cells(1).Text & ");"
Dim con As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=|DataDirectory|\Shantara Production IT.mdb")
Dim cmd As New OleDbCommand(cmdstring)
cmd.CommandType = CommandType.Text
cmd.Connection = con
cmd.Connection.Open()
cmd.ExecuteNonQuery()
cmd.Connection.Close()
Next
What I am missing to get data from temporary table. This is showing error like Invalid object name #emp. Please help me
I am using Asp.net.
Dim sqlcmd = New SqlCommand("select * into #emp from employees", conn)
sqlcmd.ExecuteNonQuery()
sqlcmd = New SqlCommand("select * from #emp", conn)
Dim dr As SqlDataReader = sqlcmd.ExecuteReader
See Above query is working fine and data is going into temporary table. but it is not selecting again through second one query.
Thanks
Try to use Global Temporary table instead of Local Temporary tabel like.. ##emp
or
You can just use a stored procedure which has all the SQL statement you want to execute and return your desired recordset.
Sorry my Enghist is not good!
How to check excel fie, if a row existing in table do'nt import this row
My function work well for import (not check existing)
string excelConnectionString = "";
string strFileName = DateTime.Now.ToString("ddMMyyyy_HHmmss");
string strFileType = System.IO.Path.GetExtension(fileuploadExcel.FileName).ToString().ToLower();
fileuploadExcel.SaveAs(Server.MapPath("~/Import/" + strFileName + strFileType));
string strNewPath = Server.MapPath("~/Import/" + strFileName + strFileType);
//Connection String to Excel Workbook
if (strFileType.Trim() == ".xls")
{
excelConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + strNewPath + ";Extended Properties=Excel 8.0;Persist Security Info=False";
}
else if (strFileType.Trim() == ".xlsx")
{
excelConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + strNewPath + ";Extended Properties=Excel 12.0;Persist Security Info=False";
}
//Create Connection to Excel work book
OleDbConnection excelConnection =new OleDbConnection(excelConnectionString);
//Create OleDbCommand to fetch data from Excel
OleDbCommand cmd = new OleDbCommand("Select [account_id], [type],[first_name], [last_name], [title], [birthday], [tel],[phone], [email_1],[email_2], [remark],[del_if] from [Sheet1$]",excelConnection);
excelConnection.Open();
OleDbDataReader dReader;
dReader = cmd.ExecuteReader();
SqlBulkCopy sqlBulk = new SqlBulkCopy(strConnection);
//Give your Destination table name
sqlBulk.DestinationTableName = "contact";
sqlBulk.WriteToServer(dReader);
I don't know if this is going to be possible using SqlBulkCopy. What you could do, is write the data to a staging table, and only copy across the records that don't already exist. Then you can clear out the staging table once your copy is complete.
Edit:
Given your comment about not wanting to use a staging table, you could possibly find a distinct list of key values (possibly account_id from your example) from your "contact" table before reading data from the excel file, and then use the list to exclude records you read from the excel file. The benefits to this are you aren't reading data from the excel file that you don't need to. The downside is that this might not be all that efficient if your "contact" table is huge or if the key you'd use to compare the data isn't something simple like account_id.
Another alternative could be, if you're excel file isn't huge, to insert records row by row using a where clause on the insert to ensure the insert won't create unnecessary duplicates. This will be significantly slower than SqlBulkCopy, but it'll work.
I have a AdvWebGrid where the 7th column is DynEdit where user will enter the value. Now I have to take the entered value and insert it into the SQL table.
For example I have 7 records in the grid, the user will enter some comments for the first three records and save. Now I want to insert/ update the first three comments in the table.
If you are able to get the info in the 7th column you could use a datable with the rows you need, the use a sqldataadapter to fill the info into the sql server, i feel this is the best way to do it.
Other way is create a stored procedure in your SQL Server then invoque it from your .net program using SqlCommand and SqlDataReader...
Here is an example:
Dim val as integer = value you want to insert
Dim comi As New SqlCommand
Dim dr As SqlDataReader
Dim _con as sqlconnection
_con.ConnectionString = _strcon ' connection string
comi.CommandType = CommandType.StoredProcedure
comi.CommandText = sp_name ' your stored procedure is sp_name this inserts a value into the table x
comi.Connection = _con
comi.Parameters.AddWithValue("val",val)
dr = comi.ExecuteReader
dr.Close()
This should do the trick ...
Greetings !