I am trying to scrape data from my company website which requires a username, password and a one time password provided by the user in each session. I am able to get to the login page get the username and password but unable to get the one time password and pass it though the website to login and scrape the data.
Try PyOTP
It can be used to implement two-factor (2FA) or multi-factor (MFA) authentication methods in web applications and in other systems that require users to log in.
github ripo Learn more about it pyotp
Related
I'm new to web scrapping, is it possible to get user profile data by scraping linkedIn without entering our login credentials?
No i don't think that this is possible. You cannot bypass the authwall and when logged in, there's a pretty fair chance that they will ban your account.
https://www.reddit.com/r/scrapy/comments/gyyx3f/advice_scraping_linkedin_why_does_my_script_only/
I am working on a .NET application and I have set up an external login with facebook.
Currently, when the user uses the external login functionality, signs in to Facebook and my app recieves his email address, I create a new user account and consider the email address verified. (otherwise he could not login)
Is this a good practice though? Is it possible that some attacker would change the email address midway or something? What is the best practice for letting users sign in using external login providers?
Any help would be much appriciated, thanks.
Edit: In this tutorial the guy sends a confirmation email to the email address he recieves from the external login providers. However this seems impractical to me. It kind of defeats the purpuse of simplifying the log in/sign in process, moreover I don't think I was ever asked to confirm my email when I had used external login providers to log in myself.
Is it possible that some attacker would change the email address midway or something?
No, because you are using facebook which implements openid or oauth2.
In oauth2, mail and its password are safe because you do not manage them. Those are managed by your oauth2 provider (facebook in your case)
Also according to the oauth2 flow which is the same in google, facebook, linkedin, etc the provider don't send you the email. It sends you the authorization code:
use go to your web.com
user is redirected to https://www.facebook.com/v8.0/dialog/oauth?client_id={app_id}&redirect_uri={redirect_uri}
oauth2 provider prompts a login if user was not logged in previously
user accepts the consent form (next next)
oauth2 provider (facebook in your case) at the end, perform a final redirect to your web.com (using the callback url previously registered) sending the authorization code: https://web.com?code=196da272-083c
this code is required to generate the access_token and can be used just one time(another http invocation)
the access_token is required to get the email (another http invocation)
The only way to attack could be try to send fake authorization codes to https://web.com?code=**** but in the next step (exchange auth code for a new access_token), facebook will return you an error because the attacker cannot create real authorization codes.
Confirmation email
As you said, if your web allows the user to login with some social network, add a new step with email confirmation is impractical. Is more, facebook allows the use of phone number instead of mail.
But there are some scenarios (not in the authentication) in which mail could be your ally:
Offer an option for alert the user with something like this: Hi Bob, a new account was created with your social network... If you didn't, please click on the following...
confirm an email to be used in future notifications
i want to implement a REST webservice in my wordpress blog. Visitors or other users can select images from my OneDrive Account and order them.
My problem is when i get the authentication code i must enter my password für OneDrive. After that i get the access_token. The access_token has only a runtime for 1 hour.
When i order i new access_token the code is expired and i must get a new authentication code with my user password.
How i can implement a webservice for OneDrive without enter my own password everytime?
Thanks for your help.
When you get the access_token through the code flow when authing you should also see a refresh_token come back (provided you requested the wl.offline_access scope). You should be able to use that to get a new access_token by using the this process without requiring user interaction.
We are partner to another company. We developed an asp.net site with membership as the provider. Our partner company requests the following. The users would log in to their site through their UI. Once the user is logged in, there is a link to our site. They want to bypass our login screen and let the users get into our site after they authenticate the user. Is there any standard for this? I checked openid, but it didn't seem like a quite fit. As far as I understood, in openid, there needs to be a link in our login page saying "Login using Company X". Then when the user clicks that the user is taken to Company X login page. Once the user is authenticated is redirected to the link we provided and the user is authenticated. Our partner company doesn't want the user to come to our login page. I still want to use everything that membership provides such as roles.
You might want to look at Claim Based Authentication.
Basically, you can redirect a user to your Partner Site. The user logins at their site, and then your Partner issues security token.
You can download the book - A Guide to Claims-Based Identity and Access Control, Second Edition - Book Download
Source: about slide is from PluralSight - Identity and Access Control in ASP.NET 4.5
I am working on an old ASP.NET application whose end users authenticate with a local Windows account (no domain). We want to switch to using membership (via aspnet_regsql), and move the application to a new server. What's the best way to seamlessly do this?
My biggest concern is I don't want every single user to have to go through the "forgot password" process. Regarding passwords. As I understand, there's no supported way to get at the passwords in order to convert them to membership users. Is there an unsupported way to grab the passwords?
There are about 1000 users, and they log into the system rarely -- maybe a few times per year. Point being: there will be a huge % of users that won't log in until the old system is dead and gone.
My current thought is that we will need to implement two workflows:
Before migration to the new server, anyone that logs in will be authenticated with Windows, and their account automatically converted to a membership user. We will have their password from our custom login page.
After migration, anyone logging in that hasn't been converted to membership will have to go through the "forgot password" routine. We would present a message to them explaining why.
So bottom line: Is there a way to do this without requiring users to go through the forgot password process at all?
I would expect you cannot get the passwords from the windows accounts, you could import the users, create a password and send out an email to users with their new password or you could send a link for users to login and create a password or request a password reset given their email address?