I am able to get sqlbulkcopy working well on iis6(testing server) and iis7(live server) but when uploading the data in iis7, the rows are cut off. It only imports ~190,000 out of 250000 in iis7. In iis 6 using the same code/page it loads all 250000 records without issues. Also both servers are updating the same SQL DB.
I have searched all over for a solution. Any help is much appreciated.
Sub UpdateData()
Dim sSQLTable As String = "Table1"
Dim sExcelFileName As String = savePath
Dim sWorkbook() As String = GetExcelSheetNames(sExcelFileName)
'Create our connection strings
Dim sExcelConnectionString As String = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & sExcelFileName & ";Extended Properties=""Excel 12.0;HDR=YES;"""
'Execute a query to erase any previous data from our destination table
Dim sClearSQL = "DELETE FROM " & sSQLTable
Dim SqlConn As SqlConnection = New SqlConnection(ConnString)
Dim SqlCmd As SqlCommand = New SqlCommand(sClearSQL, SqlConn)
SqlConn.Open()
SqlCmd.ExecuteNonQuery()
SqlConn.Close()
'Series of commands to bulk copy data from the excel file into our SQL table
Dim OleDbConn As OleDbConnection = New OleDbConnection(sExcelConnectionString)
OleDbConn.Open()
Dim OleDbCmd As OleDbCommand = New OleDbCommand(("SELECT * FROM [" & sWorkbook(0) & "]"), OleDbConn)
Dim dr As OleDbDataReader = OleDbCmd.ExecuteReader()
Dim bulkCopy As SqlBulkCopy = New SqlBulkCopy(ConnString, SqlBulkCopyOptions.UseInternalTransaction)
bulkCopy.BulkCopyTimeout = 2000
bulkCopy.DestinationTableName = sSQLTable
'DEMO bulkCopy.ColumnMappings.Add("Excel", "SQL")
bulkCopy.ColumnMappings.Add("Material", "Material")
bulkCopy.ColumnMappings.Add("Plnt", "Plant")
bulkCopy.ColumnMappings.Add("SLoc", "SLoc")
bulkCopy.ColumnMappings.Add("S", "S")
bulkCopy.ColumnMappings.Add("Batch", "Batch")
bulkCopy.ColumnMappings.Add("Special Stock Number", "SpecialStockNumber")
bulkCopy.ColumnMappings.Add("Material Description", "MatDesc")
bulkCopy.ColumnMappings.Add("Typ", "Type")
bulkCopy.ColumnMappings.Add("StorageBin", "StorageBin")
bulkCopy.ColumnMappings.Add("Available stock", "AvailStock")
bulkCopy.ColumnMappings.Add("BUn", "BUn")
'bulkCopy.ColumnMappings.Add("GR Date", "GRDate")
bulkCopy.WriteToServer(dr)
OleDbConn.Close()
End Sub
Thanks Andy for the ExcelDataReader tip!
Got it to work with ExcelDataReader. Looks like it was a memory issue with OleDB. Posting my code in case it will help others in future.
Sub updateData()
Dim sSQLTable As String = "Table1"
'Execute a query to erase any previous data from our destination table
Dim sClearSQL = "DELETE FROM " & sSQLTable
Dim SqlConn As SqlConnection = New SqlConnection(ConnString)
Dim SqlCmd As SqlCommand = New SqlCommand(sClearSQL, SqlConn)
SqlConn.Open()
SqlCmd.ExecuteNonQuery()
SqlConn.Close()
Dim stream As FileStream = File.Open(savePath, FileMode.Open, FileAccess.Read)
'1. Reading from a binary Excel file ('97-2003 format; *.xls)
'Dim excelReader As IExcelDataReader = ExcelReaderFactory.CreateBinaryReader(stream)
'2. Reading from a OpenXml Excel file (2007 format; *.xlsx)
Dim excelReader As IExcelDataReader = ExcelReaderFactory.CreateOpenXmlReader(stream)
'3. DataSet - The result of each spreadsheet will be created in the result.Tables
'Dim ds As DataSet = excelReader.AsDataSet()
'4. DataSet - Create column names from first row
excelReader.IsFirstRowAsColumnNames = True
Dim ds As DataSet = excelReader.AsDataSet()
Dim sourceData As New DataTable()
sourceData = ds.Tables(0)
''5. Data Reader methods
'Using destinationConnection As New SqlConnection(ConnString)
' open the connection
'destinationConnection.Open()
'Using bulkCopy As New SqlBulkCopy(destinationConnection.ConnectionString)
' column mappings
Dim bulkCopy As SqlBulkCopy = New SqlBulkCopy(ConnString, SqlBulkCopyOptions.UseInternalTransaction)
bulkCopy.BulkCopyTimeout = 2000
bulkCopy.DestinationTableName = sSQLTable
'DEMO bulkCopy.ColumnMappings.Add("Excel", "SQL")
bulkCopy.ColumnMappings.Add("Material", "Material")
bulkCopy.ColumnMappings.Add("Plnt", "Plant")
bulkCopy.ColumnMappings.Add("SLoc", "SLoc")
bulkCopy.ColumnMappings.Add("S", "S")
bulkCopy.ColumnMappings.Add("Batch", "Batch")
bulkCopy.ColumnMappings.Add("Special Stock Number", "SpecialStockNumber")
bulkCopy.ColumnMappings.Add("Material Description", "MatDesc")
bulkCopy.ColumnMappings.Add("Typ", "Type")
bulkCopy.ColumnMappings.Add("StorageBin", "StorageBin")
bulkCopy.ColumnMappings.Add("Available stock", "AvailStock")
bulkCopy.ColumnMappings.Add("BUn", "BUn")
'bulkCopy.ColumnMappings.Add("GR Date", "GRDate")
'bulkCopy.DestinationTableName = sSQLTable
bulkCopy.WriteToServer(sourceData)
'6. Free resources (IExcelDataReader is IDisposable)
excelReader.Close()
End Sub
Related
I'd like to ask you for some help with my problem.
I'm trying to automate sending emails with attachement separately using VB
There is a folder with few PDF files to attache to the mail
the file name start with a code of 4 digits XXXX_*
this code is the one to use to get the email from the DATABASE and every time I try to send an email it sends for all the recipient all the files
Try
Dim Smtp_Server As New SmtpClient
Dim e_mail As New Net.Mail.MailMessage()
Smtp_Server.UseDefaultCredentials = False
Smtp_Server.Credentials = New Net.NetworkCredential(TextBox3.Text, TextBox4.Text)
Smtp_Server.Port = 587
Smtp_Server.EnableSsl = True
Smtp_Server.Host = TextBox1.Text
e_mail = New Net.Mail.MailMessage()
e_mail.From = New MailAddress(TextBox3.Text)
'get users emails
Dim SQL As String = "SELECT EMail , code FROM [SagePaieSQL].[dbo].[T_SAL] WHERE EMail IS NOT NULL and MatriculeSalarie IS NOT NULL"
Dim dir As String = "C:\Users\Administrateur\Desktop\fiche_de_ paie\"
Dim output As New List(Of String)()
Dim cmd As New SqlCommand
Dim sqLdr As SqlDataReader
Dim dr As DataRow
ConnServer()
cmd.Connection = con
cmd.CommandText = SQL
Using sda As New SqlDataAdapter(cmd)
Using ds As New DataTable()
sda.Fill(ds)
sqLdr = cmd.ExecuteReader()
For i = 0 To ds.Rows.Count - 1
dr = ds.Rows(i)
Console.WriteLine(i)
e_mail.To.Add(dr("EMail").ToString())
e_mail.Subject = TextBox6.Text
e_mail.IsBodyHtml = False
e_mail.Body = (TextBox7.Text)
Dim files As String() = IO.Directory.GetFiles(dir, dr("code") & "_*")
If files.Any = False Then
MsgBox("bulltin non trouver ")
Exit Sub
Else
Button1.Enabled = False
For Each file As String In files
Dim fileName As String = IO.Path.GetFileName(file)
Dim attach As New Net.Mail.Attachment(file)
e_mail.Attachments.Add(attach)
If e_mail.Attachments.Any = True Then
MsgBox("file attached")
Smtp_Server.Send(e_mail)
MsgBox("Mail Sent")
Else
MsgBox("bulltin non attacher ")
End If
Next
End If
Next
End Using
End Using
Catch error_t As Exception
MsgBox(error_t.ToString)
End Try
I have to create crystal report dynamically by using ParameterFields. - In that a single parameter have to call particular row from MySQL database dynamically. Now I called multiple values from MySql database static and single.
I need to call particular row(dynamically) from MySQL database.
Dim sql_Sel As String
Dim hIssId As Integer = 0
Dim connectionInfo1 As New ConnectionInfo()
Dim SqlCon As String = "Data source= *****"
hIssId = Request.QueryString("patID")
Try
CrystLabl.ReportSource = Nothing
Me.SqlCon.ConnectionString = System.Configuration.ConfigurationManager.ConnectionStrings("xyz").ConnectionString
Me.SqlCon.Open()
Dim crystalReport As New ReportDocument()
sql_Sel = "exec DI_Details #opt=1"
Dim DA As SqlDataAdapter = New SqlDataAdapter(sql_Sel, SqlCon)
Dim DG As New DataSet
DG.DataSetName = "DataSet.xsd"
DA.Fill(DG, "DataTable1")
Dim paramFields3 As New CrystalDecisions.Shared.ParameterFields()
For Each i As String In sql_Sel
Dim paramField31 As New CrystalDecisions.Shared.ParameterField()
Dim discreteVal1 As New CrystalDecisions.Shared.ParameterDiscreteValue()
discreteVal1 = New ParameterDiscreteValue()
paramField31.ParameterFieldName = "My Parameter"
discreteVal1.Value = ((DG.Tables(0).Rows(0).Item(1)) & (DG.Tables(0).Rows(0).Item(2)) & (DG.Tables(0).Rows(0).Item(3)) >(DG.Tables(0).Rows(0).Item(6)))
paramField31.CurrentValues.Add(discreteVal1)
paramFields3.Add(paramField31)
Next
CrystLabl.ParameterFieldInfo = paramFields3
CrystLabl.ReportSource = "CrystalReport.rpt"
CrystLabl.RefreshReport()
Catch ex As Exception
Finally
End Try
End Sub
If I Debug this I just get a Invalid Columnname Error("Name of the Object"). I am using a SQL database.
Protected Sub ddlKunden_SelectedIndexChanged(ByVal sender As Object, ByVal e As System.EventArgs) Handles ddlKunden.SelectedIndexChanged
Dim strSql As String
Dim kontakt As String = ddlKunden.SelectedItem.Value
Dim dtbP As DataTable
Using connection As OleDbConnection = New OleDbConnection(strConnection)
connection.ConnectionString = strConnection
connection.Open()
'Kontaktpersonen laden
strSql = "SELECT * FROM Kontaktpersonen WHERE Nr =" & Chr(34) & kontakt & Chr(34)
dtbP = New DataTable()
Using dad As New OleDbDataAdapter(strSql, connection)
dad.Fill(dtbP)
End Using
ddlKontaktperson.Items.Clear()
ddlKontaktperson.DataSource = dtbP
ddlKontaktperson.DataTextField = "AP_Nam"
ddlKontaktperson.DataValueField = "ID"
ddlKontaktperson.DataBind()
End Using
ddlKontaktperson.Visible = True
End Sub
The Error pops at
dad.fill(dtbP)
It should select all rows Where Nr="SELECTED VALUE" and you select it in a dropdownlist. And all these rows should be saved in a Datatable then and are used in another dropdownlist.
It works when I try the exact same thing without the where.
Example:
'Kunden laden
strSql = "SELECT * FROM Kontakte"
dtbK = New DataTable()
Using dad As New OleDbDataAdapter(strSql, connection)
dad.Fill(dtbK)
End Using
ddlKunden.Items.Clear()
ddlKunden.DataSource = dtbK
ddlKunden.DataTextField = "Nr"
ddlKunden.DataValueField = "Nr"
ddlKunden.DataBind()
Please try following code
You do not need to use "'" around parameters
Just add these values as a parameter to the oledbcommand with its value and type
Otherwise, your sql command will be vulnerable to sql injection
Dim strConnection As String = "Provider=sqloledb;Data Source=(local);" &
"Initial Catalog=kodyaz;" &
"User Id=sa;Password=sa"
Dim kontakt As Int32 = 1
Dim dtbP As DataTable
Using connection As OleDbConnection = New OleDbConnection(strConnection)
connection.Open()
Dim cmd As New OleDbCommand()
cmd.Connection = connection
cmd.CommandText = "SELECT * FROM Kontaktpersonen WHERE Nr = ?"
cmd.Parameters.AddWithValue("Nr", kontakt)
dtbP = New DataTable()
Using dad As New OleDbDataAdapter(cmd)
dad.Fill(dtbP)
End Using
End Using
I am filling data from a sheet of Excel book to datatable . The excel sheet contains about 160 columns. It gives error message "to many fields"
Dim largespan1 As String
Dim larg As String
Dim str, str5, str6 As String
largespan1 = "C:\shivam.xls"
Dim sConnectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & largespan1 & ";Extended Properties=""Excel 8.0;HDR=yes;IMEX=1"""
Dim len, len1, i As Integer
Dim cmd, cmd1 As New SqlCommand()
Dim objConn1 As New OleDbConnection(sConnectionString)
Dim objCmdSelect1 As New OleDbCommand("SELECT * FROM [Master Schedule1$]", objConn1)
objConn1.Open()
Dim objAdapter2 As New OleDbDataAdapter()
objAdapter2.SelectCommand = objCmdSelect1
Dim dt5 As New DataTable
objAdapter2.Fill(dt5)
objConn1.Close()
Any one can suggest why it happens
Thanks in advance
I have access db with 3 different tables,I want to load the whole database into dataset so I will be able to work with the data without load the db serval times.
all the examples of working with dataset are showing how to get part of the database using ".fill"
for example :
OleDbCommand CommandObject = new OleDbCommand ("Select * from employee");
OleDbAdapter myDataAdapter = new OleDbAdapter (null, con);
myDataAdapter.SelectCommand = CommandObject;
myDataAdapter.Fill (myDataSet, "EmployeeData");
this example load only from employee but how can I etrieve the all tables in once into dataset?
in xml for instance there is command to load all the document to dataset with:" dataset.ReadXml"
How can I achive it in access db?
Thanks for any help
Baaroz
Protected Function getDataSetAndFill(ByRef connection As OleDb.OleDbConnection,
Optional ByVal isExportSchema As Boolean = True) As DataSet
Dim myDataSet As New DataSet
Dim myCommand As New OleDb.OleDbCommand
Dim myAdapter As New OleDb.OleDbDataAdapter
myCommand.Connection = connection
'Get Database Tables
Dim tables As DataTable = connection.GetOleDbSchemaTable( _
System.Data.OleDb.OleDbSchemaGuid.Tables, _
New Object() {Nothing, Nothing, Nothing, "TABLE"})
'iterate through all tables
Dim table As DataRow
For Each table In tables.Rows
'get current table's name
Dim tableName As String = table("TABLE_NAME")
Dim strSQL = "SELECT * FROM " & "[" & tableName & "]"
Dim adapter1 As New OleDb.OleDbDataAdapter(New OleDb.OleDbCommand(strSQL, connection))
adapter1.FillSchema(myDataSet, SchemaType.Source, tableName)
'Fill the table in the dataset
myCommand.CommandText = strSQL
myAdapter.SelectCommand = myCommand
myAdapter.Fill(myDataSet, tableName)
Next
''''''''''''''''''''''''''''''''''''''
'''' Add relationships to dataset ''''
''''''''''''''''''''''''''''''''''''''
'First, get relationships names from database (as well as parent table and child table names)
Dim namesQuery As String = "SELECT DISTINCT szRelationship, szReferencedObject, szObject " & _
"FROM MSysRelationships"
Dim namesCommand As New System.Data.OleDb.OleDbCommand(namesQuery, connection)
Dim namesAdapter As New System.Data.OleDb.OleDbDataAdapter(namesCommand)
Dim namesDataTable As New DataTable
namesAdapter.Fill(namesDataTable)
'Now, get MSysRelationship from database
Dim relationsQuery As String = "SELECT * FROM MSysRelationships"
Dim command As New System.Data.OleDb.OleDbCommand(relationsQuery, connection)
Dim adapter As New System.Data.OleDb.OleDbDataAdapter(command)
Dim relationsDataTable As New DataTable
adapter.Fill(relationsDataTable)
Dim relationsView As DataView = relationsDataTable.DefaultView
Dim relationName As String
Dim parentTableName As String
Dim childTablename As String
Dim row As DataRow
For Each relation As DataRow In namesDataTable.Rows
relationName = relation("szRelationship")
parentTableName = relation("szReferencedObject")
childTablename = relation("szObject")
'Keep only the record of the current relationship
relationsView.RowFilter = "szRelationship = '" & relationName & "'"
'Declare two arrays for parent and child columns arguments
Dim parentColumns(relationsView.Count - 1) As DataColumn
Dim childColumns(relationsView.Count - 1) As DataColumn
For i As Integer = 0 To relationsView.Count - 1
parentColumns(i) = myDataSet.Tables(parentTableName). _
Columns(relationsView.Item(i)("szReferencedColumn"))
childColumns(i) = myDataSet.Tables(childTablename). _
Columns(relationsView.Item(i)("szColumn"))
Next
Dim newRelation As New DataRelation(relationName, parentColumns, childColumns, False)
myDataSet.Relations.Add(newRelation)
Next
If isExportSchema Then
Dim schemaName = GetXmlSchemaFileName()
If File.Exists(schemaName) Then File.SetAttributes(schemaName, FileAttributes.Normal)
myDataSet.WriteXmlSchema(schemaName)
End If
Return myDataSet
End Function
You should just call the OleDbDataAdapter.Fill method with different SelectCommands and pass the same DataSet but different table names inside. In this case, your dataSet will contain different filled tables.