How to get _RequestVerificationToken from a dynamically loaded html website? - web-scraping

I am making a request to a website which requires _RequestVerificationToken, a value that in that specific website, is loaded dynamically and found in the html code (not in the page source) when the website is loaded in the browser.
How can I find this value, without browser automation libraries like selenium?

Related

CSS changes automatically applied on local IIS pages

I'm hosting an ASP.NET website on Local IIS (not IIS Express), and as soon as I save a change to a .css file in Visual Studio, the change immediately appears in browser windows that use that file (or after mousing over the window in Chrome), without clearing caches and refreshing.
Why do the changes appear immediately?
Opening the .css file itself (not a page using the file) in the browser shows a more expected result: saving the file in Visual Studio does not change what I see in the browser until I refresh the .css file.
As it turns out, I had Browser Link enabled in Visual Studio, and with it, CSS Auto-Sync. This opens up a port on the local machine and uses SignalR to communicate with the browser window about 400 times per second, including any CSS changes needed.
For more information, see these topics:
.net localhost website consistently making get arterySignalR/poll?transport=longPolling&connectionToken= calls
How can I disable __vwd/js/artery in VS.NET 2013?
This probably happens due to caching. when you open the css itself, it retrieves a new copy from the server, but when you open a page that uses the css file, the css file is being cached as the page's resource and the browser just shows the cached resources until you force it to reload them.
a trick i learned to fix the issue, is to link the css file to the aspx page and include a random query string to the linking, that way it tricks the browser to think that its a new resource and reload it from the server anyway.
like this:
<link href="../stylesheets/MyCSS.css?<%=DateTime.Now%>"
rel="stylesheet" type="text/css" />
we use the aspx preprocessor directive <%=DateTime.Now%> to append the current time as a query string, to ensure the link is always different.
Dont forget the question mark between the css filename and the preprocessor directive

Injecting script into iframe before load in node-webkit

I'm Trying to make a simple web browser in node-webkit, to polyfill features that Chromium doesn't support yet (time element, etc). I have had success in listening for the iframe.onload event and then appending a script tag with the polyfills, but this still means that features that I've polyfilled won't be detected by Modernizr or other feature detention.
I've tried loading the page using the http node module, appending a script tag and then turning the page source into a data URI for the frame but data uris essentially turn external pages into static html with no scripting, which renders many web pages unusable.
Also, loading a page through node's http module is proving extremely slow compared to loading through an iframe.
So, is there any other way? Ideally I run a script in the iframe before any other scripts are run.
Yes, I am using nwfaketop and nwdisable on the iframe.
The 'document-start' event should be helpful. See https://github.com/rogerwang/node-webkit/wiki/Window#document-start
See also Window.eval() in https://github.com/rogerwang/node-webkit/wiki/Window#windowevalframe-script

Why content from some url's can't be loaded in AIR application?

I was trying to show Google Plus page for certain place in my AIR application using HTML control but HTML control displays page with error code 400. Same url can be opened in browser without any errors. I have also tried to load content using URLLoader and got same error 400.
What can be different between browser and AIR? Is it possible that Goole can detect out of browser http requests and prevent them?
The website may be blocking the following:
Frame attempts
Specific User Agents
Unknown User Agents
References
Are you using a robots.txt File?
Clickjacking Security Advisory

Load a Silverlight App from an ASPX loaded assembly

We're using an MVC-based plugin architecture using ASPX pages (because Razor ones need to be precompilated). Thus we're using a custom AssemblyResourceProvider to get the custom streams for the pages.
The problem comes when we try to load a Silverlight Application on the ASPX page that's on the same assembly as the page, as an embedded resource (because we want a full plugin architecture, a single dll should be enough for everything). Even with using a physical .XAP we can't get it to work, we suppose that as the path of the page is virtual, it won't be able to search the file thus generating a 2104 silverlight error.
We can use a http uri to get the .xap, and it'll be out last resort, but we'd like to do something similar as creating a custom AssemblyResourceProvider for ASPX pages and getting the stream from the assembly.
Is that even possible?

How can I know when .aspx call is finished?

I am building a Flex Application that calls a .aspx page on the same webserver which builds a PDF report using SQL Reporting Services. When the report is built it prompts the user to open or save the PDF.
We are trying to find a way to display a Progress Bar to let the user know that the report they requested is being built, and then destroy the Progress Bar once the report is finished being built.
I've tried opening a new window using JavaScript and trying to catch when the window closes, as well as trying XMLHTTPRequest, but nothing to seems to work.
Does anyone have any suggestions?
There are 2 options:
Use the FileReference class in Flex to programmatically invoke your aspx file. You will be able to track the progress of the call from within Flex by listening to its events. But the users can only save the PDF, not open it.
Have an intermediate HTML page that displays a loading icon and then refresh itself to your PDF generating ASPX page. Encode your aspx url along with parameters etc and set it as a parameter to this intermediate page so it knows what to load.
If you don't have control over the page to be able to put JavaScript on it to hit a URL (or call back to the parent/opener), then you might consider whipping up an aspx page of your own to host a ReportViewer control, and display the report inside of that. This would require you to create a .NET website with a page and a web.config - you wouldn't need to do more than make it receive any parameters your report needs, and it would be do-able via inline-to-the-aspx code as opposed to requiring in-depth .NET knowledge.
Or, you could hit the SS-RS API and render the report directly. Here, you'd craft a URL with parameters for the report on the SS-RS API site to accept. I think, though I don't know for sure, that the SS-RS UI uses the API itself behind the scenes. By default the API is hosted in a site called "reportserver" - you might sniff HTTP traffic while the report is being rendered to get you started with the URL that you'd need to hit.
Another option not mentioned here is to create a .Net webservice, add it to your flex project and when it hits the result handler you know the file is created at that point.

Resources