Using Selenium to scrape from Azure web app - asp.net

I have successfully used Selenium from the localhost on my PC and retrieved data from the target URL but I need to be able to do this from a web app hosted in Azure. The problem I encounter is the message that ChromeDriver.exe cannot be located. If I look at the results of my build, there is a copy located in the bin folder but I don't know if this is published to the server and how I would tell the server where it was.
If the app could find the ChromeDriver, I anticipate the next error would be 'cannot locate binary'. I have commented out the binary location because I don't know how I would make Chrome available on Azure.
I can't find any examples of anyone else employing the same environment. I include a code extract.
Imports System.Net
Imports OpenQA.Selenium
Imports OpenQA.Selenium.Chrome
Imports OpenQA.Selenium.Support.UI
Public Class Scraper
Inherits System.Web.UI.Page
Public Function ScrapeToChrome(ByVal url As String) As String
Dim LinkList As List(Of String) = New List(Of String)()
Dim options = New ChromeOptions() 'With {
' .BinaryLocation = "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"
' }
options.AddArguments(New List(Of String)() From {
"headless",
"disable-gpu"
})
Dim browser = New ChromeDriver(options)
Dim wait As WebDriverWait = New WebDriverWait(browser, TimeSpan.FromSeconds(10))
browser.Navigate().GoToUrl(url)
browser.FindElement(By.Id("txtSurname")).SendKeys("Mahony")
browser.FindElement(By.Id("txtPhoenixID")).SendKeys("5258242")
browser.FindElement(By.XPath("//*[#id='divSearchBox']/div/div/div[1]/form")).Submit() 'Id("btnSearch"))
Dim elementToBeDisplayed As IWebElement
wait.Until(Function(condition)
Try
elementToBeDisplayed = browser.FindElement(By.Id("divSearchResults"))
Return elementToBeDisplayed.Displayed
Catch __unusedStaleElementReferenceException1__ As StaleElementReferenceException
Return False
Catch __unusedNoSuchElementException2__ As NoSuchElementException
Return False
End Try
End Function)
Return elementToBeDisplayed.Text
End Function
End Class

Related

Check to see if currently connected to work network (ldap server)

Firstly, I apologize if i have raised this incorrectly or not quite to the rules. It is my first post.
Can someone please help with my issue.
I have an web application that is fully functioning and authorizes the user.
my problem is that i want to add a check when the site loads to see if the PC is currently connected to the work network. EG. when at home and not connected via VPN I want to redirect to an error page stating not connected to the AD Domain Network.
Currently the app just crashes with exception of "ldap server cannot be reached".
I hope this makes sense.
CODE ADDED
Imports Microsoft.VisualBasic
Imports System.Security.Principal
Imports System.DirectoryServices
Imports System.DirectoryServices.AccountManagement
Imports System.Configuration.ConfigurationManager
Public Class Gen
Dim DomUser As System.DirectoryServices.AccountManagement.UserPrincipal = System.DirectoryServices.AccountManagement.UserPrincipal.Current
Public ADUser As String = AppSettings("DomainPrefix").ToString & DomUser.SamAccountName
Public ADEmail As String = DomUser.EmailAddress
Public ADForname As String = DomUser.GivenName
Public ADSurname As String = DomUser.Surname
Public ADFullName As String = ADForname & " " & ADSurname
Public wi As WindowsIdentity = HttpContext.Current.User.Identity
End Class
Culprit line is
Dim DomUser As System.DirectoryServices.AccountManagement.UserPrincipal = System.DirectoryServices.AccountManagement.UserPrincipal.Current
The best approach is to use the proper OOP paradigm.
Public Function CheckLDAP() As Boolean
Dim DomUser As System.DirectoryServices.AccountManagement.UserPrincipal
Dim blnLDAPOk As Boolean = False
Try
DomUser = System.DirectoryServices.AccountManagement.UserPrincipal.Current
blnLDAPOk = True
Catch ex as Exception
' Perhaps Throw an exception to be handled
End Try
Return blnLDAPOk
End Function
From the class Gen, instantiate the class
Dim gLDAP = New Gen
If (gLDAP.CheckLDAP()) Then
' We are in
Else
' Whoops, no LDAP found
End If

Rename Sharepoint folder using ClientContext

been stuck for this for a while. I need to rename a SharePoint folder using ClientContext. I created a function like so:
Public Function renameFolder(_folders As ListItemCollection, _newFolderName As String) As Boolean
Try
Using _clientContext As New ClientContext(vSharepointSite)
AddHandler _clientContext.ExecutingWebRequest, AddressOf vClaimsHelper.clientContext_ExecutingWebRequest
Dim _folder = _folders(0)
_folder.Item("Title") = _newFolderName
_folder.Item("FileLeafRef") = _newFolderName
_folder.Item("DisplayName") = _newFolderName
_folder.Update()
_clientContext.ExecuteQuery()
End Using
Return True
Catch ex As Exception
Return False
End Try
End Function
This function takes a folder collection (actually I pass a collection of only 1 folder) and the new folder name. The function executes well. Inspecting the _folder after the ExecuteQuery, everything looks as expected. However nothing happens in SharePoint, meaning that the folder name remains the original name.
Any suggestions?
Best regards and....HAPPY NEW YEAR!!!!
Ariel
Make sure List Item ( _folder variable in your example) is associated with Folder object.
How to determine whether List Item is associated with a Folder object
Using ctx As New ClientContext(webUrl)
Dim list = ctx.Web.Lists.GetByTitle(listTitle)
Dim item = list.GetItemById(itemId)
ctx.Load(item.Folder)
ctx.ExecuteQuery()
Dim isFolderItem = Not item.Folder.ServerObjectIsNull.Value
End Using
How to rename Folder using SharePoint CSOM
The following example demonstrates how to rename a Folder:
Public Sub RenameFolder(folder As Folder, folderName As String)
Dim ctx = folder.Context
Dim folderItem = folder.ListItemAllFields
folderItem("FileLeafRef") = folderName
folderItem("Title") = folderName
folderItem.Update()
ctx.ExecuteQuery()
End Sub
Usage
Using ctx As New ClientContext(webUrl)
Dim folder = ctx.Web.GetFolderByServerRelativeUrl(folderUrl)
RenameFolder(folder, "Orders")
End Using
Use the BaseName field to rename the folder.
_folder.Item("BaseName") = _newFolderName

IO.File.Delete works in debug mode but throws security exception in run mode

Has anyone ever seen an issue where IO.File methods work with the debugger attached but not at normal run-time?
IO.File.Delete gives this exception at run-time but not if the debugger is attached via VS (which runs in admin mode).
"Access to the path 'C:\AppName\App_Data\BodyPart_7416da26-4b8f-4d08-9a4a-fd3a9bf02327' is denied."
I have verified IIS_IUSRS has full permissions on the \App_Data directory. BodyPart_* is a file name generated by ASP.Net, not a sub-directory.
One other person having this issue on StackOverflow but no fixes as of yet. (File.Delete() not working in run mode but only works in debug mode)
My code:
''' <summary>
'''Post file(s) and store it via VDocument WS
''' </summary>
<System.Web.Http.HttpPost>
<ActionNameAttribute("PostFiles")> _
Public Function PostFiles(<FromUri> fileGroupGuid As Guid) As HttpResponseMessage
Dim newFileIds As New List(Of Integer)
Dim filesToDelete As New List(Of String)
' Check if the request contains multipart/form-data.
If Not Request.Content.IsMimeMultipartContent() Then
Throw New HttpResponseException(HttpStatusCode.UnsupportedMediaType)
End If
Dim root As String = HttpContext.Current.Server.MapPath("~/App_Data")
Dim provider = New MultipartFormDataStreamProvider(root)
' Read the form data.
Request.Content.ReadAsMultipartAsync(provider)
For Each file As MultipartFileData In provider.FileData
'Store to VDoc Server
Dim vdocService As New wsdocument.vdocument
Dim vdocId As String
Dim sOrigFileName As String = "/" & file.Headers.ContentDisposition.FileName.Replace("""", "")
vdocId = vdocService.savedocument(IO.File.ReadAllBytes(file.LocalFileName), sOrigFileName, _
"FS Upload", "0", "0", "0", "0")
' Store the posted file reference in the database
Dim fileId As Integer = New Answer().StoreAnswerFileWithVDocumentId(fileGroupGuid.ToString, sOrigFileName, 0, file.Headers.ContentType.ToString, New Byte(-1) {}, 0, _
0, FSFileMode.RespondentAnswer, Convert.ToInt32(vdocId))
newFileIds.Add(fileId)
filesToDelete.Add(file.LocalFileName)
Next
For Each tempFile As String In filesToDelete
'delete the temp file
IO.File.Delete(tempFile)
Next
Return Request.CreateResponse(HttpStatusCode.Accepted, newFileIds)
End Function
In debug mode a new thread is not getting created for the async call 'ReadAsMultipartAsync' and so the thread is blocked until that method completes. In release mode, it's using a new thread for the async call and since that method is running on a seperate thread, the rest of your code is still processing on the current thread. When you get to deleting the files, the files are still locked by the 'ReadAsMultipartAsync' method on the additional thread. Since the files are still locked, the delete will fail. You need to await the 'ReadAsMultipartAsync', so that it completes before you continue processing.
Try this:
await Request.Content.ReadAsMultipartAsync(provider)

Can't share isolated storage file between applications in different app pools

I've got various web apps (containing WCF services) in IIS under the default website. As long as they are all running in the same app pool they can access a shared isolated storage file no problem.
However, once I move them to different app pools I get "System.IO.IsolatedStorage.IsolatedStorageException: Unable to create mutex" when one tries to access a file created by another. They are all running under NetworkService user. I tried GetUserStoreForAssembly and GetMachineStoreForAssembly all with the same result. Any ideas why they couldn't use a shared file?
I made sure to close the stream and even dispose it in case one was holding onto it, but I am running a simple test where one service writes it, then another tries to read from it later, and it always fails.
Also, I am accessing the isolated store from a signed assembly.
Does anybody have any ideas?
Here is the code:
Private Sub LoadData()
Dim filename = FullFilePath(_fileName)
Dim isoStorage As IsolatedStorageFile = IsolatedStorageFile.GetUserStoreForAssembly()
' Tried GetMachineStoreForAssembly, same failure
isoStorage.CreateDirectory(ROOT_DIRECTORY)
If (isoStorage.GetFileNames(filename).Length = 0) Then
Return
End If
Dim stream As Stream = New IsolatedStorageFileStream(filename, FileMode.OpenOrCreate, isoStorage)
If stream IsNot Nothing Then
Try
Dim formatter As IFormatter = New BinaryFormatter()
Dim appData As Hashtable = DirectCast(formatter.Deserialize(stream), Hashtable)
Dim enumerator As IDictionaryEnumerator = appData.GetEnumerator()
While enumerator.MoveNext()
Me(enumerator.Key) = enumerator.Value
End While
Finally
stream.Close()
stream.Dispose()
stream = Nothing
End Try
End If
End Sub
Public Sub Save()
Dim filename = FullFilePath(_fileName)
' Open the stream from the IsolatedStorage.
Dim isoFile As IsolatedStorageFile = IsolatedStorageFile.GetUserStoreForAssembly()
' Tried GetMachineStoreForAssembly, same failure
Dim stream As Stream = New IsolatedStorageFileStream(filename, FileMode.Create, isoFile)
If stream IsNot Nothing Then
Try
Dim formatter As IFormatter = New BinaryFormatter()
formatter.Serialize(stream, DirectCast(Me, Hashtable))
Finally
stream.Close()
stream.Dispose()
stream = Nothing
End Try
End If
End Sub
Looks like it was a trust issue.
After adding the assembly accessing the isolated storage file to the gac it magically worked as everything in the gac has full trust set automatically.
This works for me, but it might not always be an option to do this for other solutions. Check out the .NET Framework caspol utility if this is the case.
Hope this helps somebody! It was a huge pitafor me.

Crystal Report convert to PDF (Error The logon to the database failed. ((0x8004100f)))

In my application I have some code to convert to PDF. In debug mode all is good and working but when I test it on then server I keep getting the logon to the database failed. I have no idea why I get the error becase the login and password are 100% ok.
tried 2 ways for the server of sending the report
SetCrystalReportFilePath(Server.MapPath("~/MemberPages/Report.rpt"))
SetPdfDestinationFilePath(Server.MapPath("~/MemberPages/Report_" & Report & ".pdf"))
SetRecordSelectionFormula("{Report.Report_id} =" & ID)
Transfer()
SetCrystalReportFilePath("C:\inetpub\wwwroot\Werkbon.rpt")
SetPdfDestinationFilePath("C:\inetpub\wwwroot\Werkbon_" & werkbonnr & ".pdf")
SetRecordSelectionFormula("{werkbon.werkbon_id} =" & werkbonnr)
Transfer()
Dim ConInfo As New CrystalDecisions.Shared.TableLogOnInfo
Dim oRDoc As New ReportDocument
Dim expo As New ExportOptions
Dim sRecSelFormula As String
Dim oDfDopt As New DiskFileDestinationOptions
Dim strCrystalReportFilePath As String
Dim strPdfFileDestinationPath As String
Public Function SetCrystalReportFilePath(ByVal CrystalReportFileNameFullPath As String)
strCrystalReportFilePath = CrystalReportFileNameFullPath
End Function
Public Function SetPdfDestinationFilePath(ByVal pdfFileNameFullPath As String)
strPdfFileDestinationPath = pdfFileNameFullPath
End Function
Public Function SetRecordSelectionFormula(ByVal recSelFormula As String)
sRecSelFormula = recSelFormula
End Function
Public Function Transfer()
oRDoc.Load(strCrystalReportFilePath)
oRDoc.RecordSelectionFormula = sRecSelFormula
oDfDopt.DiskFileName = strPdfFileDestinationPath
expo = oRDoc.ExportOptions
expo.ExportDestinationType = ExportDestinationType.DiskFile
expo.ExportFormatType = ExportFormatType.PortableDocFormat
expo.DestinationOptions = oDfDopt
oRDoc.SetDatabaseLogon("databasename", "password")
oRDoc.Export()
End Function
Crystal is very particular when swapping in and out connection information (especially with sub-reports if you have any). It's always been a sore spot that they haven't really made any better over the years. I have a blog entry I've shared that has extension methods I created (in VB.Net) to easily load and swap in new connections (typically I use ADO for the connection in the report). The extension methods can be found in this link (and below is an example of how to use them, ignore the System.Diagnostics line if you're using this from ASP.NET). Hope this helps.
http://www.blakepell.com/Blog/?p=487
Using rd As New ReportDocument
rd.Load("C:\Temp\CrystalReports\InternalAccountReport.rpt")
rd.ApplyNewServer("serverName or DSN", "databaseUsername", "databasePassword")
rd.ApplyParameters("AccountNumber=038PQRX922;", True)
rd.ExportToDisk(ExportFormatType.PortableDocFormat, "c:\temp\test.pdf")
rd.Close()
End Using
System.Diagnostics.Process.Start("c:\temp\test.pdf")

Resources