Is web automation with Openoffice BASIC even possible? - web-scraping

I'm trying to do some simple webscraping in OpenOffice (I usually work in Excel but I'm trying to port something over for a coworker that doesn't have Excel). However, when I try to run something very similar to this, it keeps giving me this BASIC runtime error 1.
Here's the relevant code, I'm not sure what I'm supposed to do to make this work.
Sub Macro1
Dim explorer As Object
Set explorer = CreateObject("InternetExplorer.Application")
explorer.Visible = True
explorer.navigate("www.yahoo.com")
Const READYSTATE_COMPLETE As Long = 4
Do While explorer.Busy Or explorer.readyState <> READYSTATE_COMPLETE
Loop
dim page as object
set page = explorer.Document
dim mailButton as object
set mailButton = page.GetElementByID("ybar-navigation-item-mail") 'this is the line the error occurs on
mailButton.Click
End Sub

Do you know that you can save script in vbs file (you have to delete types in variables declarations) and run it directly by double click without using office application? I recommend you to use this way.

Related

Run a Windows command line from ASP Classic and get results

I'm trying to improve security on an old ASP Classic site. I need to generate (truly) random numbers, and ASP itself doesn't seem to have that capability directly. So it occurred to me that I might run a Windows CLI script and use the results.
I've figured out how to run a script, but not how to return the results of that script to ASP.
Dim oShell, sCommand
sCommand = "C:\scripts\myscript.bat /foo"
Set oShell = Server.CreateObject("WScript.Shell")
Set oExec = oShell.exec(sCommand)
...?
Set oShell = Nothing
Can anybody help me with that last piece of the puzzle.
(Or... if you know a better way to generate truly random numbers in ASP Classic, that would be excellent)
Make the output of command redirected to text file. For example "dir >test.txt". Then read the file.

DotNetZip download works in one site, not another

EDIT - RESOLVED: the difference was that in the "main" case the download was initiated via a callback cycle, and in the "test" case it was initiated through a server side button click function. My guess is that the download request and the callback cycle interfered with each other, both stopping the download and causing the page to become inactive (as described below). When I rewired the download on the main page to start with a submit instead of a callback, it did initiate the download.
This is in VS2013 Ultimate, Win7Pro, VB.Net, websites (not projects),IISExpress.
I built a test site to develop functionality for creating OpenXML PPTX and XLSX memorystreams and zipping and downloading them using DotNetZip. Got it to work fine. I then merged all that code into my "main" site. Both sites are on the same machine; I can run the test site and the main site at the same time. The main site processing is somewhat more complicated, but only in terms of accessing and downloading more files.
However, the Zip and Download function (below) works fine in the test site, but the exact same code doesn't work in the main site (with or without the test site up and running).
There's an error trap (see below) around the Zip.Save function where the download occurs but no error shows up.
Same overall behavior in Chrome, Firefox and IE11.
One peculiarity that might be a clue is that when the main site download fails, the server side functionality "goes dead". Local JS functions work, but the app doesn't respond to callbacks. When I do an F5 on the browser it works again.
I did a refresh on the DotNetZip package in the main site. The Zip object appears to be working properly, because it generates an error on duplicate file names.
I thought it might be the download function as written, however, it works in the test site. Also, another piece of the main site does a non-zipped download of a memory stream (included as the second code block below) and that works fine.
I thought it might be the data. So I kludged the main site to access, convert to memorystream and download the same file that the is accessed and downloaded in the test site. Still the main site download doesn't work.
When I compare the watch values on the Zip object in the two sites, they look identical. The length of the wrkFS.ContentStream is identical in both cases. The file names are different, however, they are:
Test_2EFVG1THK5.xlsx (main)
6-18_12-46-28_0.xlsx (test)
which are both legal file names.
EDIT: I saved the zip file to disk from the main program, instead of trying to download it, using this:
wrkFilePath = "D:\filepath\test.zip"
wrkZip.Save(wrkFilePath)
And it worked fine. So that possibly isolates the problem to this statement
wrkZip.Save(context.Response.OutputStream)
EDIT: Base on help I received here:
Convert DotNetZip ZipFile to byte array
I used this construct:
Dim ms as New MemoryStream
wrkZip.Save(ms)
wrkBytes = ms.ToArray()
context.Response.BinaryWrite(wrkByteAr)
to get around the ZipFile.Save(to context), and that didn't work either; no download, no error message, and page goes dead. However, at least I can now assume it's not a problem with the ZipFile.Save.
At this point I'm out of ways to diagnose the problem.
Any suggestions would be appreciated.
Here is the code that works in the test site but not in the main site.
Public Sub ZipAndDownloadMemoryStreams(ByVal context As HttpContext) _
Implements IHttpHandler.ProcessRequest
Dim rtn As String = ""
Try
Dim wrkAr As ArrayList
wrkAr = SC.ContentArrayForDownLoad
If wrkAr.Count = 0 Then
Dim wrkStop As Integer = 0
Exit Sub
End If
Dim wrkFS As ZipDownloadContentPair
Using wrkZip As New ZipFile
'----- create zip, add memory stream----------
For n As Integer = 0 To wrkAr.Count - 1
wrkFS = wrkAr(n)
wrkZip.AddEntry(wrkFS.FileName, wrkFS.ContentStream)
Next
context.Response.Clear()
context.Response.ContentType = "application/force-download"
context.Response.AddHeader( _
"content-disposition", _
"attachment; filename=" & "_XYZ_Export.zip")
'---- save context (initiate download)-----
wrkZip.Save(context.Response.OutputStream)
wrkZip.Dispose()
End Using
Catch ex As Exception
Dim exmsg As String = ex.Message
Dim wrkStop As String = ""
End Try
End Sub
Below is the non-zip download function that works fine in the main site.
It might be possible to convert the Zip content to a byte array and try the download that way, however, I'm not sure how that would work.
(SEE EDIT NOTE ABOVE --- I implemented a version of the below, i.e. try to download byte array instead of directly ZipFile.Save(), however, it didn't help; still doesn't download, and still doesn't give any error message)
Public Sub DownloadEncryptedMemoryStream(ByVal context As HttpContext) _
Implements IHttpHandler.ProcessRequest
Dim wrkMemoryStream As New System.IO.MemoryStream()
wrkMemoryStream = SC.ContentForDownload
Dim wrkFileName As String = SC.ExportEncryptedFileName
wrkMemoryStream.Position = 0
Dim wrkBytesInStream As Byte() = New Byte(wrkMemoryStream.Length - 1) {}
wrkMemoryStream.Read(wrkBytesInStream, 0, CInt(wrkMemoryStream.Length))
Dim wrkStr As String = ""
wrkStr = Encoding.UTF8.GetString(wrkMemoryStream.ToArray())
wrkMemoryStream.Close()
context.Response.Clear()
context.Response.ContentType = "application/force-download"
context.Response.AddHeader("content-disposition", "attachment; filename=" & wrkFileName)
context.Response.BinaryWrite(wrkBytesInStream)
wrkBytesInStream = Nothing
context.Response.End()
(Per the note now at the top of the question): The difference was that in the "main" case the download was initiated via a callback cycle, and in the "test" case it was initiated through a server side button click function. My guess is that the download request and the callback cycle interfered with each other, both stopping the download and causing the page to become inactive (as described below). When I rewired the download on the main page to start with a submit instead of a callback, it did initiate the download.

FileUpload control in asp/VB.net cannot access the file because it is being used by another process

Error message:
The process cannot access the file 'C:\SampleProjectName\mytestcsv.csv' because it is being used by another process.
I am trying to read numerous files (CSV, XML, HTML) in asp/VB.net using the fileupload (file upload) control.
I'm saving the file using Server.MapPath so I can process the file in another procedure. It's very odd, but sometimes I can browse and upload the same file over and over with no issues, but sometimes it immediately fails.
I've found that I can ultimately kill the WebDev.WebServer40.exe it releases whatever lock is present. This is annoying, but fine for my debugging... but unacceptable for endusers.
My fileupload code:
If fuImport.HasFile Then
If (System.IO.File.Exists(Server.MapPath("myhtml.html"))) Then
System.IO.File.Delete(Server.MapPath("myhtml.html"))
End If
Dim dtFromHTML As New Data.DataTable
Dim dtFromSQL As New Data.DataTable
Try
fuImport.SaveAs(Server.MapPath("mytestcsv.csv"))
'Process data here
ProcessCSVData(Server.MapPath("mytestcsv.csv"))
Catch ex As Exception
Response.Write("error: " & ex.Message)
Finally
fuImport.PostedFile.InputStream.Flush()
fuImport.PostedFile.InputStream.Close()
fuImport.FileContent.Dispose()
End Try
'Other things happen here
Else
Response.Write("no file...")
End If
Any ideas would be appreciated.
Use FileShare.Read to read a file even if it is opened exclusively by an another process.
You may not be releasing the file in your code for access. You should use the keyword "using".
Read this post:
The process cannot access the file because it is being used by another process - using static class
and this:
http://msdn.microsoft.com/en-us/library/htd05whh.aspx
I am not a VB.NET coder but try something along the lines of:
Using {fuImport.SaveAs(Server.MapPath("mytestcsv.csv")) }
ProcessCSVData(Server.MapPath("mytestcsv.csv"))
End Using
The next procedure that uses file...
Try closing input stream before accessing the file:
UPDATED To use temp filename
Dim sTempName = Path.GetRandomFileName
fuImport.SaveAs(Server.MapPath(sTempName))
'close before accessing saved file
fuImport.PostedFile.InputStream.Close()
'Process data here
ProcessCSVData(Server.MapPath(sTempName)

Trying to retrieve spreadsheet from asp site without any prompts

I have a company site that publishes large reports which I pull down and split up into what I need. I can get the webpage open and get the link to the spreadsheet I need opened, but then I get one IE pop-up for open/save/cancel and, when I click to open the spreadsheet, I get a second pop-up (this one from Excel) saying that the spreadsheet is in a different format than specified by the extension." I have no idea how, if possible, to get the first pop-up to away; the only way I know to normally prevent pop-ups with excel is with DisplayAlerts=False, but adding that doesn't seem to have any effect. The code for the start of this automation project follows:
[EDIT] I have edited and replaced the original coding to reflect the most recent attempt at this. I have managed to get past the first file download pop-up (though with the use of sendkeys) and am now working on getting the second pop-up taken care of (which I suppose is a matter of shifting focus back to excel). Any suggestions on replacing the sendkeys portion are definitely welcomed, as well as advice for returning focus to excel so that I can use excel to control the second pop-up (if that's the best way to do it). The updated code follows:
Private Declare Function FindWindow Lib "user32" Alias "FindWindowA" _
(ByVal lpClassName As String, ByVal lpWindowName As String) As Long
Sub Automation()
Dim IeApp As Object
Dim IeDoc As Object
Dim URL, URL2 As String
Application.DisplayAlerts = False
URL = "https://companysite.com/directorypage/default.aspx"
URL2 = "https://companysite.com/directorypage/Reports/MyReport.aspx?Format=Excel"
Set IeApp = CreateObject("InternetExplorer.Application")
IeApp.Visible = True
IeApp.Navigate URL
While IeApp.Busy Or IeApp.ReadyState <> 4: DoEvents: Wend
IeApp.Document.All.Item("MainContent_btnAuthenticate").Click
While IeApp.Busy Or IeApp.ReadyState <> 4: DoEvents: Wend
Set IeApp = CreateObject("InternetExplorer.Application")
IeApp.Visible = False
IeApp.Navigate URL2
Do Until thewindow <> 0 'wait for the "File Download" popup window to appear
thewindow = FindWindow(vbNullString, "File Download")
Loop
SendKeys "{LEFT}"
Application.Wait Now + TimeValue("00:00:01")
SendKeys "{LEFT}"
Application.Wait Now + TimeValue("00:00:01")
SendKeys "{ENTER}"
End Sub
You're not going to be able to suppress that message in your code, it's a setting on the user's workstation to prevent harm from malicious files.
The user would have to either edit their Windows registry or have it controlled via a group policy setting. http://support.microsoft.com/kb/948615
I ran across this same issue with Excel documents generated on on the server. The only workaround I made was to create files using the Open XML SDK.
EDIT: I read your question again and noticed it's more focused on the first pop-up, and JMax linked answer (How to disable file download popup in Internet Explorer?) should fix that.
Jon, my two cents on this.
If you have administrative access to your pc then what JMax suggested will sort it out. I would discourage editing the registry if you are an administrator until and unless you know what you are doing.
If it is say an office pc then editing the registry is not even an option. You will have to contact the IT dept. They can either login to your pc as an administrator and then turn the option off for you or they can tweak the GP as zeroef suggested. My best guess is that if it is an office pc then they will login as an administrator and make the changes for you individually instead of making the change in the GP (unless you are the only member of that group). In a corporate environment it is really difficult to get that pulled off via GP as it affects lot of users. There is also a possibility that your request might not be agreed upon if it is an Office pc.
BTW, from what I see, your question has nothing to do with it being an Excel problem :)
HTH
Sid
You could just download the file directly...
Private Declare Function URLDownloadToFile Lib "urlmon" Alias _
"URLDownloadToFileA" (ByVal pCaller As Long, _
ByVal szURL As String, ByVal _
szFileName As String, ByVal dwReserved As Long, _
ByVal lpfnCB As Long) As Long
Sub DownloadFile(sURL, sSaveAs)
Dim rv As Long
rv = URLDownloadToFile(0, sURL, sSaveAs, 0, 0)
If rv <> 0 Then
MsgBox "Error with download!"
End If
End Sub
Don't know if the https will be an issue here.
Another thing to try is just:
Workbooks.Open "https://companysite.com/directorypage/Reports/MyReport.aspx?Format=Excel"

Having problem opening/writing to a text file in ASP.NET

I want to write some stats to a text file every time a person loads a page. But every once in awhile I am getting at 'Could Not Open File, Already in use' type of error. I can not 100% replicate this error it is very erratic. My code is
Public Sub WriteStats(ByVal ad_id As Integer)
Dim ad_date As String = Now.Year & Now.Month
Dim FILENAME As String = Server.MapPath("text/BoxedAds.txt")
Dim objStreamWriter As StreamWriter
objStreamWriter = File.AppendText(FILENAME)
objStreamWriter.WriteLine(ad_id & ";" & ad_date)
objStreamWriter.Close()
End Sub
My question is, how can I lock and unlock the file so I stop getting the erratic errors?
Thanks
If two or more requests hit your web server at roughly the same time, they will all try to open the same file. You will need to create unique file names for each request.
Public Sub WriteStats(ByVal ad_id As Integer)
Dim ad_date As String = Now.Year & Now.Month
Dim FILENAME As String = Server.MapPath("text/BoxedAds.txt")
Dim index As Integer
Using fs As New IO.FileStream(FILENAME, IO.FileMode.Append, IO.FileAccess.Write, IO.FileShare.ReadWrite), _
tl As New TextWriterTraceListener(fs)
index = Trace.Listeners.Add(tl)
Trace.WriteLine(ad_id & ";" & ad_date)
Trace.Listeners(index).Flush()
Trace.Flush()
End Using
Trace.Listeners.RemoveAt(index)
End Sub
Three important things here:
Use of IO.FileShare.ReadWrite to allow multiple writers on the file at once.
The Using statement to make sure the stream is closed immediately, even if an exception occurs. This will minimize collisions
The TextWriterTraceListener will create a temp file for your if it can't open the file you request, to make sure the message isn't lost.
You will have to handle the exception and build some handling to re-try writing to the file after a short random interval.
If you get too much contention then it might make more sense to log it to a table in a database and create a process to export to a file (if its still needed)
I haven't had any trouble with short info using:
File.AppendAllText(path, info);
Regarding the comment on it causing locks, from reflector it uses the same options explained very well by Joel. It does not use the trace writer, so it won't output to a temp file in the case of high load / large content causing trouble.
If the info is large, you really want separate files. For high load, I would go with Joel's suggestion and create a temp file, which can be alternatively done by catching the exception on File.AppendAllText, and using the same File.AppeandAllText with a unique filename.

Resources