I am trying to find the full webpage address for a form generated by a website. The website is https://treasurer.maricopa.gov/Parcel/?Parcel=50427029
Once you get there I want to see the web address for the Redemption Statement. You click on it and then have to click on another link to get the form showing all the information I want to scrape.
Here is the problem. The web address says https://treasurer.maricopa.gov/Parcel/RedemptionStatement.aspx
I know this is not the full address and is being hidden by the aspx suffix at the end. However, I can't find a way to determine the complete web address url.
Any thoughts?
Thanks.
That is the 'web address url'. That page is using parameters that are being passed in with a POST when you click on 'see redemption statement' as part of the view state. If you install and run Fiddler you can see the information being sent in with the POST. However, it is not this page that you should be interested in, but the RedemptionStatement.aspx page. From here, it looks like it's a POST with a pretty simple JSON payload.
{
"parcelNumber": "50427029",
"paymentDate": "4/3/2018"
}
Install and run Fiddler / Postman to inspect and formulate the POSTs needed.
Related
I am developing a web scraper and I need to download a .pdf file from a page. I can get the file name from the html tag, but can't find the complete url (or request body) that downloads the file.
I have tried to sniff the traffic with the chrome and firefox network traffic tool and with wireshark, with no success. I can see it make a post request to the exact same url as the page itself, and so I can't understand why this happens. My guess is that the filename is being sent inside the POST request body, but I also can't find that information in those tools. If I could see the variable name in the body, I could create a copy of the request and then get the file.
How can I get that information?
Here is the website I am talking about: http://www2.trt8.jus.br/consultaprocesso/formulario/ProcessoConjulgado.aspx?sDsTelaOrigem=ListarProcessos.aspx&iNrInstancia=1&sFlTipo=T&iNrProcessoVaraUnica=126&iNrProcessoUnica=1267&iNrProcessoAnoUnica=2010&iNrRegiaoUnica=8&iNrJusticaUnica=5&iNrDigitoUnica=24&iNrProcesso=1267&iNrProcessoAno=2010&iNrProcesso2a=0&iNrProcessoAno2a=0
EDIT: for those seeking to do something similar, take a look at this website: http://curl.trillworks.com/
It converts a cURL to a python requests code. Very useful
The POST data used for the request is encoded content generated by ASP.NET. It contains various state/session information of the page that the link is on. This makes it difficult to directly scrape for the URL.
You can examine the HAR by exporting it from the Network tab in Chrome DevTools:
The __EVENTVALIDATION data is used to ensure events raised on the client originate from the controls rendered on the page from the server.
You might be able to achieve what you want by requesting the page the link is on first, then extract the required POST data from the response (containing the page state and embedded request for file), and then make a new request with this information. This assumes the server doesn't expire any sessions in the meantime.
I'm building an emailshot tracking function in my ASP.NET site which will serve up a linked image when the email is opened, so allowing me to count the email opens.
It's all above-board, opted-in etc, etc, and I've tried two methods:
calling a .aspx page that serves (direct to the Response stream) a Base64-encoded image stored in a database
calling a .ashx HTTP handler that serves an image from a physical folder
Both of these techniques work absolutely fine - the image is served, and I can process the hit in the page code - if I use my POP email or my Gmail accounts.
However, Hotmail blocks them both. It doesn't even give the option of displaying images - there's no trace of the original link if I View Source on the browser email display.
Links to images only seem to display correctly in the Hotmail browser window if I:
Use an actual domain name in the link (rather than an IP address or localhost for testing) AND
Link to physical files (rather than pages that serve them) using <img src="http://domain/imagefilename.ext"/>.
Is there an alternate approach that will at least give the option of viewing the image? I'm not trying to conceal the tracking in any way - there will usually be other linked images in the emails anyway.
Nor does the thing being served have to be an image - is there anything else that can be served (and hence counted) that isn't treated as suspicious by over-zealous mail servers such as Hotmail?
Cheers.
You can add a querystring value at the end of the url. And then configure IIS to handle that extension.
WELL what are you asking is a nightmare for email server.this same technique is used by spammers to get valid emailid out of junk emailids. For now i can assure you tracking opening of email is impossible using any white hat or gray hat techniques on any reputiable servers.
plese see these links for more details:-
http://emailuniverse.com/ezine-tips/?id=502
https://webapps.stackexchange.com/questions/22662/way-to-ensure-that-my-sent-email-was-read
However dont take myword 'impossible' for granted, there are some protected techineques available but i am not sure about it
http://mailchimp.com/
good luck
I have ssl in my e-commerce web site. At first, browser always asking "do you want to show this web site's content" in all page and when I redirect to mycart page browser shows the same alert like that "This webpage contains content that will not be delivered using a secure HTTPS connection, which could comprise the safety of the entire webpage....Yes...No....". After I clicked to yes, all my sessions get null. Do you have any suggestions for me?
KR,
Çağın
The problem is your secure page is accessing information (scripts, images, etc.) from pages that are not secure. For example if you reference a javascript file (say jQuery) from a nonsecure site (say Google) then certain browsers (like IE) will display this message. You need to search through your references and find these. In other words searching src="http or something along those lines will pull up the nonsecure references.
Depending on what you are referencing you can move those items to your site so that they are now "secure". Also, in some cases changing your reference from src="http to src="https can resolve the problem.
Once you resolve this alert you can check again to see if you are having sessions issues as you could have some other issues to address.
I searched google for a good number of hours. Maybe I searched for the wrong keywords.
Here is what I want to do.
I'm posting data to a website which then makes a HTTP POST request and returns a .CGI webpage. I want to know the parameters the web page uses to send that HTTP POST request so that I can directly link a page from my Webpage to the final .CGI webpage by making the user enter the data on my own webpage.
How do I achieve it?
Usually the POST body is piped into STDIN, just read it as a normal file
I am setting up a website using drupal 6 and trying to be minimalistic. I use contact form, and it works pretty well. Once the message is sent, it is redirected to the front page of my site with a message "Your message has been sent.".(with a div messages status). How can one change it like,
customize the message content
instead of redirecting to the home page, display a node (another page)
thanks for any suggestion.
For more advanced forms and customisation like you mention above, you should try webform. Webform has the ability to redirect a successful submission to another URL and have a custom status message as well.
Change the string for the message. To do that, you can either turn on the locale.module or get string overrides module.
You can create a small custom module to redirect the user using the #redirect property of the Forms API. See here for an example (and check the comments for potential issues).
Alternatively, consider using one of these modules:
Node Destination
Custom (Form) Destination
Rules
Page manager redirect