how to get the link names of a webpage using cURL command - http

I am using Postman, a google Chrome add on that makes cURL commands, and I make a GET command with a website url. My question goes with an example: on a website like google, if i type "stackoverflow" and search, I take this url and make my cURL command, how can I get the names of each link? Is that possible? By example, for this page there would be "Stack Overflow" ... "Stack Overflow - Wikipédia"...

I found out the answer by myself.
I installed Postman Interceptor, activated it and entered the website on google chrome. The interceptor sent all the requests done to get that website to my postman, so all i had to do is look at them and I found the information I looked for in one of them.

Related

Slack API - Link to custom URI scheme

I have a (working) custom URI scheme that opens an app in my local machine. It looks somewhat like this:
customscheme://?ip=xx.xx.xx.xx?platform=xx
I am trying to send the user a message using the Slack API including a link that opens the custom URI. It seems to work with mailto: links but not with mine.. The syntax I am trying is the following:
<customscheme://?ip=xx.xx.xx.xx?platform=xx|Open Uri>
or
[Open Uri](customscheme://?ip=xx.xx.xx.xx?platform=xx)
without success.
The thing is, I am able to create the link from the UI using the Link (Ctrl+Shift+U) functionality, pasting the exact same link from above. The outcome is exactly what I want, the text with the custom URI as the link (and it opens it after a security check).
Is there some security concern that keeps me from getting this done?
I found that inserting some word before ? then will work well
customscheme://foo?ip=xx.xx.xx.xx?platform=xx

Making an HTTP request with a blank user agent

I'm troubleshooting an issue that I think may be related to request filtering. Specifically, it seems every connection to a site made with a blank user agent string is being shown a 403 error. I can generate other 403 errors on the server doing things like trying to browse a directory with no default document while directory browsing is turned off. I can also generate a 403 error by using a tool like Modify Headers for Google Chrome (Google Chrome extension) to set my user agent string to the Baidu spider string which I know has been blocked.
What I can't seem to do is generate a request with a BLANK user agent string to try that. The extensions I've looked at require something in that field. Is there a tool or method I can use to make a GET or POST request to a website with a blank user agent string?
I recommend trying a CLI tool like cURL or a UI tool like Postman. You can carefully craft each header, parameter and value that you place in your HTTP request and trace fully the end to end request-response result.
This example straight from the cURL docs on User Agents shows you how you can play around with setting the user agent via cli.
curl --user-agent "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL]
In postman its just as easy, just tinker with the headers and params as needed. You can also click the "code" link on the right hand side and view as HTTP when you want to see the resulting request.
You can also use a heap of hther HTTP tools such as Paw and Insomnia, all of which are quite well suited to your task at hand.
One last tip - in your chrome debugging tools, you can right click the specific request from the network tab and copy it as cURL. You can then paste your cURL command and modify as needed. In Postman you can import a request and past from raw text and Postman will interpret the cURL command for you which is particularly handy.

Find Hidden Webpage Url Address

I am trying to find the full webpage address for a form generated by a website. The website is https://treasurer.maricopa.gov/Parcel/?Parcel=50427029
Once you get there I want to see the web address for the Redemption Statement. You click on it and then have to click on another link to get the form showing all the information I want to scrape.
Here is the problem. The web address says https://treasurer.maricopa.gov/Parcel/RedemptionStatement.aspx
I know this is not the full address and is being hidden by the aspx suffix at the end. However, I can't find a way to determine the complete web address url.
Any thoughts?
Thanks.
That is the 'web address url'. That page is using parameters that are being passed in with a POST when you click on 'see redemption statement' as part of the view state. If you install and run Fiddler you can see the information being sent in with the POST. However, it is not this page that you should be interested in, but the RedemptionStatement.aspx page. From here, it looks like it's a POST with a pretty simple JSON payload.
{
"parcelNumber": "50427029",
"paymentDate": "4/3/2018"
}
Install and run Fiddler / Postman to inspect and formulate the POSTs needed.

Getting error while testing website in google page speed

Hi I am trying to test my website on page speed google but getting an error.
"Attempting to load the page reached the limit of 3 client redirects. The last URL fetched was http://www.example.com/. This may indicate the page is redirecting to itself, or has a loop of redirects."
Could you please tell me exactly whats the issue.
It looks like your page is redirecting to itself. Execute the following command in your terminal.
curl -I http://www.example.com
Make sure the URL you are testing with returns HTTP/1.1 200 OK.
You can also use some online tools to find the final destination and use that one for testing the page speed. Check this one - http://redirectdetective.com/
Hi thanks for the reply I have tested the url at "http://redirectdetective.com/" and got no redirects found also I have execute the curl -I http://www.example.com command and got HTTP/1.1 200 OK response.
But still I am getting an error on page speed google.

How to make POST request using OAuth via Youtube API?

I am trying to get this thing to work for a couple days since it's my first time working with the OAuth system without any luck.
I have been experimenting here: https://developers.google.com/youtube/v3/docs/subscriptions/insert#try-it
With the following settings:
http://i.gyazo.com/5cd28f1194d5dfebee25d07bc0db965e.png
When I execute the code it successfully subscribes to the specified channelIdaccount with the authorized account.
I have tried to copy paste the shown POST URL into my browser without any luck. The plan was just to test it as I would like to implement this in PHP.
Now to my questions:
The {YOUR_API_KEY}, is this where I am supposed to type in the access token? If so, do I need the &mine=true tag at all?
I just realized that there are no ID's in the URL but there is an JSON-object in the request box example. Am I supposed to convert a string to JSON-object and pass it to the $fields= tag?

Resources