I'm trying to scrap a webpage for data but came across the problem of needing to log in.
Connection.Response loginForm = Jsoup.connect("http://www.rapidnyc.net/users/google_login")
.method(Connection.Method.GET)
.execute();
Document document = Jsoup.connect("http://www.rapidnyc.net/users/google_login")
.data("Email", "testEmail")
.data("Passwd", "testPass")
.... //other form data
.cookies(loginForm.cookies())
.post();
This gives me the org.jsoup.HttpStatusException: HTTP error fetching URL. Status=400
I used chrome developer tool to look at the Form data being posted but nothing I post works.
1. Have you submitted ALL input fields? Including HIDDEN ones.
2. I see the website requires "captcha-box" authentication, which is to prevent web crawlers from logging in. I highly doubt you will be able to log in with your program.
I say the 400 status is coming from your program not being able to provide the value for "captcha" authentication.
Related
I would like to understand better Firebase Dynamic Links because i am very new to this subject.
What i would like to know :
FirebaseDynamicLinks.instance.getInitialLink() is supposed to return "only" the last dynamic link created with the "initial" url (before it was shorten) ?
Or why FirebaseDynamicLinks.instance.getInitialLink() doesn't take a String url as a parameter ?
FirebaseDynamicLinks.instance.getDynamicLink(String url) doesn't read custom parameters if the url was shorten, so how can we retrieve custom parameters from a shorten link ?
My use case is quite simple, i am trying to share an object through messages in my application, so i want to save the dynamic link in my database and be able to read it to run a query according to specific parameters.
FirebaseDynamicLinks.instance.getInitialLink() returns the link that opened the app and if the app was not opened by a dynamic link, then it will return null.
Future<PendingDynamicLinkData?> getInitialLink()
Attempts to retrieve the dynamic link which launched the app.
This method always returns a Future. That Future completes to null if
there is no pending dynamic link or any call to this method after the
the first attempt.
https://pub.dev/documentation/firebase_dynamic_links/latest/firebase_dynamic_links/FirebaseDynamicLinks/getInitialLink.html
FirebaseDynamicLinks.instance.getInitialLink() does not accept a string url as parameter because it is just meant to return the link that opened the app.
Looks like there's no straightforward answer to getting the query parameters back from a shortened link. Take a look at this discussion to see if any of the workarounds fit your use case.
I have an application which is successfully sharing to LinkedIn as per the official documentation here:
https://developer.linkedin.com/docs/share-on-linkedin
When I get a 201 created response the content is being shared to LinkedIn which is great...but in the response under the updateUrl property the URL I always get is bringing back a page not found URL, for example:
https://www.linkedin.com/company/3728030/comments?topic=6368780249068507136&type=U&scope=3728030&stype=C&a=_YH0
When I visit the actual post on LinkedIn and get the post URL it gets this: https://www.linkedin.com/feed/update/urn:li:activity:6368780249068507136
and this URL actually works. In the documentation it says:
The updateURL value is a direct link to the newly shared content on LinkedIn.com that you can direct the user's web browser to.
I have tried this on multiple LinkedIn accounts and I am getting the same issue. I could parse my updateUrl property and transform it into a working URL but as per the documentation I shouldn't have to do this and it seems like a bug...
The same thing applies to newly created company shares: https://developer.linkedin.com/docs/company-pages#company_share
The returned value is e.g.
{
"updateKey": "UPDATE-c111111-11111111",
"updateUrl": "https://www.linkedin.com/company/111111/comments?topic=11111111&type=U&scope=18464510&stype=C&a=ls_e"
}
That URL directs you to a non-existing page. For proper use, the "updateKey" needs to be parsed and the link manually created e.g. "https://www.linkedin.com/feed/update/urn:li:activity:11111111"
Note, the URLs and keys are changed.
I'm also wondering the purpose of the "updateUrl", since its undocumented on company shares, and seemingly misleading on shares documentation.
Thanks :)
I'm in a situation in which i need to be able to create a topics using self created bot. My forum has a special category and user, which can create a topics in that category.
Technology i'm using to create that bot is ruby + mechanize gem but it's not important right now. That bot works in a following way:
Sign in as previously mentioned user:
visit - /ucp.php?mode=login
fill the sign-in form using user credentials
if, after submiting a form, there is a sign-out link somewhere on the page, threat this whole process as "successfull"
Create a topic
visit - /posting.php?mode=post&f=21
fill subject field with desired subject
fill message field with desired message
submit a form using Submit button
And now, while first point works just great, the second one behaves in a strange way. After submiting a form, there is no error message or anything like that, i'm just getting redirected to /viewforum.php?f=21 (log's says that it's 302 Moved Temporarily status) page and the topic is not there.
Can anyone tell me what such behaviour means? Is there any security mechanism i don't know about? Please also note that the new topic form has form_token and creation_time fields filled correctly while form is beeing submited.
Thanks in advance for any clues.
According to this thread: https://stackoverflow.com/a/11713867/552936, user is treated as a bot if he sends a form without any delay (and it end's up with a 302 redirect). I can't find any info on google what's the exact delay, carck3r says that it's 8 seconds but for me, it was 2 seconds.
I want to download HTML default report for a test run from Performance Center storage (using Rest API). Actually I need just summary.html file.
I was using the following steps in PC 11.5:
Request test scenarios:
http://{server:port}/qcbin/rest/domains/{domain}/projects/{project}/tests?fields=id,last-modified,name,owner&query={subtype-id[=PERFORMANCE-TEST]}&page-size=max
Let user choose the scenario (id) and request all its runs:
http://{server:port}/qcbin/rest/domains/{domain}/projects/{project}/runs?page-size=max&fields=id,owner,pc-start-time,duration,status,test-id&query={test-id[=234]}
Let user choose the run (id) and request Report (result entity):
http://{server:port}/qcbin/rest/domains/{domain}/projects/{project}/results?page-size=max&query={run-id[=123];name[=Reports]}&fields=id,name
Request "summary.html" file using file-id taken from previous step response:
http://{server:port}/qcbin/rest/domains/{domain}/projects/{project}/results/{file-id}/storage/report/summary.html
However it is not working with Performance Center 11.0. It fails at last step:
qccore.general-error
Not Found
I guess it is because the path of report was changed.
Can someone tell the path for summary.html for Performance Center 11.0?
I've been able to have a little bit of success with this. Rather than use the request you are using above I used the following:
http://{server:port}/qcbin/rest/domains/{domain}/projects/{project}/results/{file-id}/logical-storage/
This gave me a zip file, which contained the report inside it.
I am trying to use PHP to fire hits at Google to track newsletter opens and clickthroughs. I want to use the same technique for both clickthroughs and opens since the clickthroughs will go to sites outside of my own control - I want to be able to report on the clickthrough rates of the newsletters regardless of where the clicks go to. I was thinking of trying code.google.com/p/php-ga/ but there is little in the way of example code/support docs to start with so I am hesitant.
Here is my url to __utm.gif broken up over the lines for clarity:
utm.gif?utmac=MO-xxx31982-1">http://www.google-analytics.com/_utm.gif?utmac=MO-xxx31982-1
&utmhn=myfake.com
&utmcc=_utma%3D7042858245.1436153422.1214501708.1340117181.1340117181.1%3B%2B_utmz%3D1.1340117181.1.1.utmcsr%3D%28direct%29%7Cutmccn%3D%28direct%29%7Cutmcmd%3D%28none%29%3B
&utmwv=1
&utmr=click
&utm_source=Emails
&utm_medium=Newsletter
&utm_campaign=tet+2012-06-19+10%3A41%3A30
&utmp=%2FMYZZ%2FNEWSLETTERS
&uservar=16430
Does the utmhn need to be a legit URL or one associated with the account? What about utmr? I was using that to contain 'click' or 'open' so I could differentiate.
When I click a link in the newsletter I get the expected pixel image returned so Google is getting something anyway even if ignoring my querystring. In my Google Analytics account where should I see the data relating the the __utm.gif hits? So far I see none when using this technique.
P.S. I got this technique from here
Follow up:
I changed a few things and my url now looks like so:
utm.gif?utmwv=1">http://www.google-analytics.com/_utm.gif?utmwv=1
&utmn=8391432847
&utmsr=click
&utmsc=-
&utmul=-
&utmje=0
&utmfl=-
&utmdt=-
&utmhn=myfake.com
&utm_source=my_newsletter
&utm_medium=Emails
&utm_campaign=tet 2012-06-19 10:41:30
&utmr=my_newsletter
&utmp=images/google/click
&utmac=MO-xxx31982-1
&utmcc=_utma%3D24820632.1925394567.1340121629.1340121629.1340121629.2%3B%2B_utmb%3D24820632%3B%2B_utmc%3D24820632%3B%2B_utmz%3D24820632.1340121629.2.2.utmccn%3D(direct)%7Cutmcsr%3D(direct)%7Cutmcmd%3D(none)%3B%2B__utmv%3D24820632.6430%3B
and nothing happens except when I paste that link into my browser then Google gets it, so why does it not work when called from the PHP line $handle = fopen ($urchinUrl1, "r");?
ok, nevermind. I changed my app around so the utm.gif is just included in the email and in a redirect page rather than called from the script. Should have done it that way but got caught up in the fancier idea of calling the url from php.
Even though this question is over a year old...the GA measurement protocol can be used to send hits back to GA for newsletter opens and clickthroughs.
To answer your initial question, it looks like you're missing the utmhid and utmn parameters.