I'm writing a simple log in to website, navigate to page and parse app for iOS. I've managed to generate a post request, but it doesn't seem to want to log in. I don't know why it could be failing, because I get status 200. Admittedly I get this even if I deliberately enter wrong credentials. Any ideas? (code is in Swift)
var url = NSURL(string:"https://www.example.com/ps/signin.html")
var request = NSMutableURLRequest(URL: url)
request.HTTPMethod = "POST"
var dataString = "timezoneOffset=-600&userid1=xxx&userid=xxx&pwd=xxx&x=31&y=12"
let data = (dataString as NSString).dataUsingEncoding(NSUTF8StringEncoding)
request.HTTPBody = data
var connection = NSURLConnection(request: request, delegate: self, startImmediately: false)
println("sending request : \(request)")
connection.start()
So I get a response back, with status 200, and seemingly the html code for the login page again.
The problem is that you are POSTing to the login form: https://www.example.com/ps/signin.html. You need to POST to the same place the HTML form posts to, namely: https://www.example.com/psp/ps/?cmd=login&languageCd=ENG
Try changing your URL:
var url = NSURL(string:"https://www.example.com/psp/ps/?cmd=login&languageCd=ENG")
Related
I used the following Python code to retrieve a web page behind a login page successfully for some years:
username = 'user'
password = 'pass'
login_url = 'https://company.com/login?url='
redirect_url = 'https://epaper.company.com/'
data = { 'email' : username, 'pass' : password }
initial_url = login_url + quote(redirect_url)
response = requests.post(initial_url, data=data)
Then something changed at company.com about 2 months ago, and the request returned status code 400. I tried changing the data parameter to json (response = requests.post(initial_url, json=data)) which gave me a 200 response telling me a wrong password was provided.
Any ideas what I could try to debug?
Thanks,
Jan
Update: I just tried using a requests session to retrieve the csrf_token from the login page (as suggested here), so now my code reads:
with requests.Session() as sess:
response = sess.get(login_url)
signin = BeautifulSoup(response._content, 'html.parser')
data['csrf_token'] = signin.find('input', {'name':'csrf_token'})['value']
response = sess.post(initial_url, data=data)
Unfortunately, the response is still 400 (and 200/wrong password with the json parameter).
First: When you send data=data, used {"Content-Type":"application/x-www-form-urlencoded"}; if you send json=data, in headers response should be used {"Content-Type":"application/json"}
Second: Perhaps redirects have been added. Try to add:
response = sess.post(url, data=data)
print("URL you expect", url)
print("Last request URL:", response.url)
Be sure to check:
print(sess.cookies.get_dict())
print(response.headers)
If you get an unexpected result when checking, change the code like this:
response = sess.post(url, data=data, allow_redirects=False)
I'm creating client side Blazor app with Microsoft.OData.Client. When I create new entity like this:
var dataServiceContext = this.ClientFactory.CreateClient<Container>(new Uri("http://localhost:5000/odata"));
var newAsset = new CreateAssetDto()
{
TechnicalName = "from_client_4",
DisplayNameFormat = "format from client",
Icon = "client/icon",
InheritedFrom = Guid.NewGuid(),
IsActive = true,
Translation = new AssetTranslationDto
{
Title = "Client Asset",
Language = "en",
Description = "This is asset from client"
}
};
dataServiceContext.AddToAssets(newAsset);
await dataServiceContext.SaveChangesAsync();
I get an exception stating that response to this POST request is missing Location header. When I run fiddle to see what's going on I can see that it actually made 2 requests.
The first request is POST but doesn't include the body and recieves 204 response.
The second request is the one that actually contains the data creating new Asset and response contains Location header as it should.
I guess OData Client is complaining about Location header missing in the response for the first request (since response for second request does contain the header). But why is it even making the first request?
Any idea how to deal with this problem?
It's possible that the first request is a preflight request sent by the browser. But normally CORS preflight requests are sent using OPTIONS method, not POST. So this case is curious.
I am a contributor to the project but do not have enough reputation to add comments here to get clarifications. Could you create an issue on https://github.com/OData/odata.net ?
So I got a template of a Flutter app that retrieves all its data from a website using HTTP get requests.
I have the following method that gets the list of resturaunts:
Future<Stream<Restaurant>> getNearRestaurants(LocationData myLocation, LocationData areaLocation) async {
String _nearParams = '';
String _orderLimitParam = '';
if (myLocation != null && areaLocation != null) {
_orderLimitParam = 'orderBy=area&limit=5';
_nearParams = '&myLon=${myLocation.longitude}&myLat=${myLocation.latitude}&areaLon=${areaLocation.longitude}&areaLat=${areaLocation.latitude}';
}
final String url = '${GlobalConfiguration().getString('api_base_url')}restaurants?$_nearParams&$_orderLimitParam';
final client = new http.Client();
final streamedRest = await client.send(http.Request('get', Uri.parse(url)));
return streamedRest.stream.transform(utf8.decoder).transform(json.decoder).map((data) => Helper.getData(data)).expand((data) => (data as List)).map((data) {
return Restaurant.fromJSON(data);
});
}
However when I swap the template's url variable for my own website, the app gets stuck since it cannot retrieve the same information from my website.
What could I be missing? Is the problem in the flutter code or the website?
Update 1:
I surrounded it with a try/catch block and it gave me a "bad certificate exception.". This might be because my website does not have a SSL certificate, so I added an exception to the HttpClient for my self-certified website:
bool _certificateCheck(X509Certificate cert, String host, int port) =>
host == '<domain>';
HttpClient client2 = new HttpClient()..badCertificateCallback = (_certificateCheck);
HttpClientRequest request = await client2.getUrl(Uri.parse(url));
var response = await request.close(); // sends the request
// transforms and prints the response
response.transform(Utf8Decoder()).listen(print);
This code showed a Error 404: Not found on the page that I need to get my JSON data from.
I also installed postman and checked my website with the GET statement for the same list of restaurants I try to retrieve in the flutter code posted above and see this:
Postman GET screenshot
Update 2:
So I configured SSL on my website and the problem still persists. I tried testing the GET request via postman and it returns a error 404 page as well. I have tried going through my server files and laravel logs and nothing did the trick.
Its as if my website cannot route to the specific pages in my API folder. BUt they are all defined in api.php.
I am trying to login to website using python requests library; I get the 200 status code however I just get redirected back to the login page.
I have tried many variations of the various post function variables including:
data = formData, form data includes the correct username, password and _csrf token
data = json.dumps(formData)
headers = headers (which included the "user-agent")
auth = HTTPBasicAuth(username,password) where username and password are the correct string variables for the site login
with requests.Session() as c:
loginPage = c.get(LoginURL)
loggedInPage = c.post(LoginURL,
data=formData,
headers=headers,
auth=HTTPBasicAuth(username,password)
)
I am having to re-write an existing REST API using .NET (originally written with Ruby). From the client's perspective, it has to work exactly the same way as the old API - i.e. the client code mustn't need to change. The current API requires Basic Authentication. So to call the old API, the following works perfectly:-
var wc = new System.Net.WebClient();
var myCache = new CredentialCache();
myCache.Add(new Uri(url), "Basic", new NetworkCredential("XXX", "XXX"));
wc.Credentials = myCache;
var returnBytes = wc.DownloadData("http://xxxx");
(I have had to ommit the real URL / username / password etc for security reasons).
Now I am writing the new API using ASP.Net Web API with MVC4. I have a weird problem and cannot find anybody else with exactly the same problem. In order to support Basic Authentication, I have followed the guidelines here:
http://sixgun.wordpress.com/2012/02/29/asp-net-web-api-basic-authentication/
One thing, I put the code to "hook in the handler" in the Global.asax.cs file in the Application_Start() event (that wasn't explained so I guessed).
Anyway, if I call my API (which I have deployed in IIS) using the above code, the Authorization header is always null, and the above fails with 401 Unauthorized. However, if I manually set the header using this code, it works fine - i.e. the Authorization header now exists and I am able to Authenticate the user.
private void SetBasicAuthHeader(WebClient request, String userName, String userPassword)
{
string authInfo = userName + ":" + userPassword;
authInfo = Convert.ToBase64String(Encoding.Default.GetBytes(authInfo));
request.Headers["Authorization"] = "Basic " + authInfo;
}
.......
var wc = new System.Net.WebClient();
SetBasicAuthHeader(request, "XXXX", "XXXX");
var returnBytes = wc.DownloadData("http://xxxx");
Although that works, it's no good to me because existing users of the existing API are not going to be manually setting the header.
Reading up on how Basic Authentication works, the initial request is meant to be anonymous, then the client is returned 401, then the client is meant to try again. However if I put a break point in my code, it will never hit the code again in Antony's example. I was expecting my breakpoint to be hit twice.
Any ideas how I can get this to work?
You're expecting the right behavior. System.Net.WebClient does not automatically include the Authorization headers upon initial request. It only sends them when properly challenged by a response, which to my knowledge is a 401 status code and a proper WWW-Authenticate header. See here and here for further info.
I'm assuming your basic authentication handler is not returning the WWW-Authenticate header and as such WebClient never even attempts to send the credentials on a second request. You should be able to watch this in Fiddler or a similar tool.
If your handler did something like this, you should witness the WebClient approach working:
//if is not authenticated or Authorization header is null
return base.SendAsync(request, cancellationToken).ContinueWith(task =>
{
var response = task.Result;
response.StatusCode = HttpStatusCode.Unauthorized;
response.Headers.Add("WWW-Authenticate", "Basic realm=\"www.whatever.com\"");
return response;
});
//else (is authenticated)
return base.SendAsync(request, cancellationToken);
As you noticed, if you include the Authorization headers on every request (like you did in your alternative approach) then your handler already works as is. So it may be sufficient - it just isn't for WebClient and other clients that operate in the same way.