I have an excel file which gets updated every 10 seconds through an automated process. I need excel data to be updated in MY-SQL database which is located on a remote server.
How do I do that?
I have thought of following option:
1) Every 11 seconds, an Excel macro will run and will "Save as" excel as CSV file. (not sure whether this can be done by macro...just thinking)
2) This CSV file we will FTP to remote server using Windows Service.
3) On remote server, we will parse the csv file and Update MYSQL database.
Is this approach fine? Or do you have a better approach which requires less time to update the database?
Thanks!
I found following links to be more useful:
http://www.heritage-tech.net/908/inserting-data-into-mysql-from-excel-using-vba/
http://vbaexcel.eu/vba-macro-code/update-mysql-database-php
I hope this helps someone having similar problem as mine.
You can connect to the Excel spreadsheet using ODBC connection, read the data, and post it to the MySQL database, maybe through some sort of web service access, or via a saved CSV file?
Related
I have a remote MonetDB server running and I want to bulk upload a csv file as it is much faster.
Based on the params in MonetDB.R, there is a csvdump=TRUE option but I don't think it works when you are trying to do this against a remote server. The server has to be local.
https://rdrr.io/github/MonetDB/monetdb-r/man/dbWriteTable.html
First, am I correct that I can't do this and if not, is there a workaround? I have a dataframe with +5M rows so it takes a long time with insert statements rather than using COPY INTO.
When I try using csvdump=TRUE against the remote server, it can't find the csv file because it is local to computer that called the dbWriteTable command.
I think you are right. As a workaround either use explicit COPY INTO ON CLIENT SQL statements or first use some file transfer tool to copy the file to the remote server before calling dbWriteTable.
It reads from MonetDB's documentation on COPY INTO:
FROM files ON SERVER
With ON SERVER, which is the default, the file name must be an
absolute path on the system on which the database server (mserver5) is
running. ...
Interestingly enough pymonetdb, the Python driver for MonetDB, uses ON CLIENT for bulk loads. From the pymonetdb's doc:
File Uploads and Downloads
Classes related to file transfer requests as used by COPY INTO ON
CLIENT.
You might want to file an issue for the MonetDB R-driver project to have similar behavior as pymonetdb.
I Have a Excel file(xlsl) with multiple worksheets in Sharepoint server which I should read in informatica and load the data into different tables.
Informatica is hosted on unix server
Currently I am thinking of the below work around but I have challenges here:
1.Copying the excel file into unix.(Once i copy the file from sharepoint server to unix using the "curl" command, the format of the file is getting changed to html. I can I retain the original excel format ,I can't install any excel utility on our server)
2.Convert them into multiple CSV files using some script (How can I do this, As I mentioned earlier I don't have any utilities like xls2csv, unoconv )
3.And read the CSV file and load them into tables.
Please let me know if there is any better approach than this.
You can try using wget to download the excel file (or set of files) from the sharepoint to Informatica file server location. It will allow you to specify the directory and target file name as well.
We have a CSV files being loaded automatically in Unix machine.
Requirement: We need to load the csv file from that remote server to my oracle DB. We do have an ODI as our ETL tool. Can someone advice on how to proceed further. What is the way to load the CSV from Unix server to Oracle DB.Please help us with some document if this case is possible.
Thanks ,
Gowtham Raja S
Oracle is providing some tutorials (Oracle By Example), one of them explains how to load a flat file to an Oracle table with ODI 12c : https://apexapps.oracle.com/pls/apex/f?p=44785:112:::::P112_CONTENT_ID:7947
You will just need to change the field delimiter to a comma instead of a tab.
The other tutorials can be found on the product page : http://www.oracle.com/technetwork/middleware/data-integrator/learnmore/index.html
if you know how to create a data store for file, in ODI we have a LKM called LKM file to Oracle (SQLLDR) OR LKM file to Oracle (external table), both can be used to load data quickly, or if you feel like this is bit difficult, however since you have the Sqlldr to load data manually from file to DB, what ever the command you are using to start sqlldr place it in ODI procedure with technology as OSCommand whihc loads data automatically.
let me know if any other suggestios are required.
I'm building an application that takes as an input data stored in an Excel sheet. I want the user to be able to select the file they want to load data from, have the application connect to and read from the Excel file stored on the client's machine, and close the connection. Can I do this without uploading the Excel file to the server? I'm able to do everything except for selecting the file using a filedialog box and passing the path & file name to the procedure that connects to the Excel file and processes. I've tried using the file input control but I'm unable to pass the path & file name to the connection string. Any suggestions as to what other routes I might take?
EDIT:
The application essentially takes user input, either via single inputs into textboxes on the page or a bulk upload via an Excel spreadsheet, processes the inputs and spits out a report in Excel format. The only thing displayed on the page are the loaded inputs (via the 2 methods just described) in a listbox that the user can either add or remove additional items.
The short answer is "Only with an ActiveX control in IE unless you write your own plugin for another browser." My opinion is, "you shouldn't."
There is a good discussion already on Stack Overflow: How to read an excel file contents on client side?
The long answer, given the rest of the information you have provided, is that I would recommend the following:
1) Upload the spreadsheet to the server.
2) Extract the data on the server.
3) Return the data to the client in whatever form suits your situation.
4) Clean the original file on the server.
Some further recommendations for you, since you provided so much detail in your comments:
Rather than using a GUID to generate your server-side filename, use a timestamp in the format YYYY-MM-DD-HH-MM-SS-Ticks followed by the original filename.
Instead of deleting the data files immediately, each time you add a file, remove any files older than N days.
This way if you have any issues processing your files in the future, you'll be able to retrieve the uploaded file to your development environment and debug it there. Assuming the data isn't personal information or sensitive in some other way, of course.
Cheers!
I have software for STOCK and SALES that can give me an Excel price list, I wanna write a small web app that will enable users to check prices, I don't have any access to that software's database, so I wanna import that Excel data (one worksheet, 5 columns, almost 10000 rows) into an MS SQL database table, keep in mind that the price list will change few time a month so I need to re-import that price list once its been updated.
Any suggestions? or maybe I should skip using an MS SQL database and directly use the Excel sheet? if so, then how?!
I'd go directly to SQL Server. Let it do the heavy lifting. If one of your users wants to export data to Excel, by all means provide such a feature. But multi-user changes to the data should be done in SQL Server via the web. Excel should be a local, read-only, personal copy to download and nothing else.
SQL Server can easily import an Excel spreadsheet or .csv file, especially if we're talking about a single table with five columns. I'd still have SQL Server do all the work on the web side.
If you save the Excel file as a CSV, it doesn't take much of a program to convert that into SQL statements.
You can use bulk upload to upload data from excel to sql server. Please look at the link below
http://www.gyansangrah.com/ArticleContent.aspx?ID=bulk_upload_in_asp_net
Please let me know in case of any issues.