I'm trying to scrape some data of a website using python request. I have to login to the website though first. So I found the /acounts/login/ url and when i try to post to this URL with a payload containing my username and password I get redirected to a page where I have to give consent to cookies by pressing a big green button.. How can I do this with python request? Even when I just do s.get and print the url it is already redirected to this Cookie Consent Page..
import requests
s = requests.Session()
res = s.get('https://example.nl/accounts/login/')
print(res.url)
When I inspect this cookie consent page I find the following relevant lines of HTML:
<form action="/consent?token=f1ad267c-377b-486a-b286-67ff63e3b046" method="post">
<input type="hidden" name="permissions" value="functional">
<input type="hidden" name="permissions" value="analytics">
<input type="hidden" name="permissions" value="socialMedia">
<input type="hidden" name="permissions" value="advertising">
<input type="hidden" name="permissions" value="personalization">
<input type="hidden" name="referrer" value="">
<input type="hidden" name="_csrf" value="">
<div class="buttons">
<input type="submit" data-track-click="{"contentInteractionName": "consentChange","cookiewallInitiator":"f1tm","consentChoice": {"permissions": { "advertising" : true,"analytics": true,"functional": true,"personalization": true,"socialMedia": true},"consentScreen": "front"}}" value="Ja, ik ga akkoord" data-track-click-has-click-listener="true">
Voorkeuren beheren
</div>
</form>
I don't know what to do with this.. I would like to "press" the submit button to continue to the login page..
Related
I have a Paypal account with set subscription button on it. After a subscription is set by a buyer I am sending him to a receipt page. On this page I need to retrieve variables from Paypal to update by database.
form is working
subscription saved on Paypal
buyer correctly redirected to receipt page
but variable are not retrieved.
here is what I have tried so far:
<form action="https://www.paypal.com/cgi-bin/webscr" method="post" target="_top">
<input type="hidden" name="cmd" value="_xclick-subscriptions">
<input type="hidden" name="business" value="mybusinessname">
<input type="hidden" name="lc" value="IE">
<input type="hidden" name="item_name" value="Monthly Subscription">
<input type="hidden" name="no_note" value="1">
<input type="hidden" name="no_shipping" value="1">
<input type="hidden" name="a1" value="0.00">
<input type="hidden" name="p1" value="1">
<input type="hidden" name="t1" value="M">
<input type="hidden" name="src" value="1">
<input type="hidden" name="a3" value="1.50">
<input type="hidden" name="p3" value="1">
<input type="hidden" name="t3" value="M">
<input type="hidden" name="currency_code" value="EUR">
<input type="hidden" name="bn" value="PP-SubscriptionsBF:btn_subscribeCC_LG.gif:NonHosted">
<input type="hidden" name="custom" value="test_value">
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_subscribeCC_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!">
<img alt="" border="0" src="https://www.paypalobjects.com/en_US/i/scr/pixel.gif" width="1" height="1">
</form>
Instant Payment Notification (IPN) details on Paypal are enabled and point to:
https://www.mydomain/payments/
IPN history shows that notifications from Paypal are correctly sent to Notification URL (above)
However, the receipt page shows
Error
Undefined index: custom
to get the custom variable I tried GET, POST and even REQUEST method using something like
test 1: <?php echo $_GET["custom"]; ?>
test 2: <?php echo $_POST["custom"]; ?>
Then I tried
<?php
if (isset($_POST['custom'])) {
$custom_display = $_POST['custom'];
}
?>
but this results in: Undefined variable: custom_display
All I need is to get custom variable.
I am guessing that I am doing something wrong, but after many hours of searching for an answer and trying different settings (e.g. sending dynamically notification url within the form, changing settings in paypal, using different urls etc) nothing seems to be working for me and I am stuck
Any help would be greatly appreciated!
I ran into the same issue (?) a few months back. The answer seems to be: PayPal does not pass custom in IPN anymore (nor does it honor the return value). Both of these stopped working sometime last year.
My workaround was to use the subscr_id value to lookup the customer profile. YMMV.
I've an aspx form with a form tag as below and this form tag contains some sensitive info with hidden fields
<form id="payment_confirmation" target="myFrame" action='https://testsecureacceptance.cybersource.com/embedded/pay' method="post"/>
<input type="hidden" name="access_key" value="sensitivevalue1">
<input type="hidden" name="profile_id" value="sensitivevalue2">
<input type="hidden" name="transaction_uuid" value="<% Response.Write(getUUID()); %>">
<input type="hidden" name="signed_field_names" value="sensitivevalue3">
<input type="hidden" name="unsigned_field_names" value="card_type,card_number,card_expiry_date">
<input type="hidden" name="signed_date_time" value="<% Response.Write(getUTCDateTime()); %>">
<input type="hidden" name="locale" value="en">
<input type="submit" id="submit" name="submit" value="Submit"/>
</form>
When i click "Submit" it successfully post the values and user is "redirected" into the "https://testsecureacceptance.cybersource.com/embedded/pay" form but those hidden fields values are exposed to users (when they inspect the page). I can encrypt them but is there any other way i can post values and redirect from backend where values will not be exposed to users?
Thanks
In short, no. Hidden fields are only hidden from view on the screen and will always be available be available when viewing the source of the page when you are using web forms like this. You can use the viewstate, which will encode the fields and make them harder to read, but if it is truly sensitive information, you will need to properly encrypt that information.
Hope that helps.
I'm pretty new to using wufoo forms and have been searching for a few days and can't quite find what I'm looking for.
I did find a number of articles about 'URL Modification' but not sure how to implement this for what I need.
We have a simple single wufoo form which is being used across 6 iterations of a client's domains (they are sector specific).
We want (in the email notification and response entry on wufoo) to record which site was used to complete the form (for analytical purposes).
In other words the email to the client should list:
Name: John Smith
Email: Johnsmith#mail.com
Phone: 555-123-1234
From: www.websiteversion1.com
The form is being integrated on Wordpress sites.
Any help would be appreciated!
You can copy the form HTML to your site's templates and modify the form, using PHP to fill in the value of the site url. I don't think WuFoo will automatically fill that field in for you.
First of all, in your WuFoo account forms manager, add a website (url) field and make it visible for admins only (this is a Wufoo option).
Then copy the generated form into your own template.
Now modify your form template so that it grabs the site URL and fills it in for the value of the website field where you want it.
Your form template might look something like this:
<form class="wufoo-form" id="form3" name="form3" accept-charset="UTF-8" autocomplete="off" enctype="multipart/form-data" method="post" novalidate action="#">
<div class="form-group">
<label for="Field1">Name</label>
<input id="Field1" name="Field1" type="text" placeholder="" value="">
</div>
<div class="form-group">
<label for="Field2">Email</label>
<input id="Field2" name="Field2" type="email" placeholder="" value="">
</div>
<div class="form-group">
<label for="Field3">Phone</label>
<input id="Field3" name="Field3" type="tel" placeholder="" value="">
</div>
<div class="form-group hidden">
<label for="Field4">From</label>
<input id="Field4" name="Field4" type="url" class="form-control" placeholder="" value="<?php esc_url( home_url() ); ?>">
</div>
<div class="form-group">
<button id="saveForm" name="saveForm" type="submit" name="submit" class="btn btn-primary btn-lg">Let's talk!</button>
<input type="hidden" id="idstamp" name="idstamp" value="***the_id_for_your_form_wufoo***" />
</div>
</form>
I am trying to put a button on my shiny app which calls out external website though a form/post request. The input values will change but here is a real example.
Example:
<form action="http://toppgene.cchmc.org/CheckInput.action" method="post">
<input type="hidden" name="query" value="TOPPFUN">
<input type="hidden" id="type" name="type" value="HGNC">
<input type="hidden" name="training_set" id="training_set" value="EGFR">
<input type="submit" class="mybutton">
</form>
PS: cross posted on shiny-discuss google group link
Thanks!
-Abhi
i want to use https only for login form and after the user logs in send it to http .
login form is in page "a.php" and the action of login form is "b.php" .
my question is:
which one should i call: https://mysite.com/a.php or http://mysite.com/a.php
the action should be : https://mysite.com/b.php or http://mysite.com/b.php
i mean post data is send according to form page or action page protocol??
The page with input fields (a.php) should be on HTTP, while the page with authentication logic (b.php) should be on HTTPS.
In a.php there will be something like this:
<form method="post" action="https://[domain-name]/b.php">
<input type="text" name="username" />
<input type="password" name="password" />
<input type="submit" value="login" />
</form>