I am using Chrome for some web scraping. How can I find manually the call to API endpoints url ?
The way I do it now it going to Network and look link by link. Is there a better way to quickly identify the call to database endpoints ?
Thanks
You can use fiddler or similar network sniffers.
You start fiddler. Then as you navigate to your website of interest, fiddler is going to show you all the url's that are sent to the website, as well as the website's responses.
Related
I was able to pass a simple URL (thousands) 6 days ago to the vision client and obtain an output. However,right now, it is not possible. What changed? I don't like the idea of a simple workaround by downloading the data and then placing that as input.
What happened Google? Why did this changed? Should we move to Azure, AWS?
Anybody got the URL working without the workaround
If you are using http URLs as input, it is possible that the Google crawling bot fails to fetch the URL you provided at that specific moment.
There are many reasons for this, for example:
The site of your URL is down, or it temporarily blocked Google bot, or it updated robot.txt to stop the Google bot to crawl URLs.
You can retry after a while, or you can download those URLs by yourself.
I guess if you try again today, it may recover to work.
Why don't you try to use base64 encoded string instead of image URL.
I had the same experience to get a response from Google OCR API. It failed two or three times out of ten API calls. It really got me stressed because I got no clue.
Since I've changed my code to base64 accidentally, It worked well. 100% response. Never fail!!
Good luck!
I try to find a internet place where I could find support about the Skype Web Control (dedicated support web site, forum, chat, documentation). A place where I can report problem and find help.
Here are my issues, maybe someone has a solution:
I use the Skype Web Control with a Microsoft Chatbot (Azure, LUIS) and it works pretty well.
But the smileys are not displayed in the conversion when the user send one. Space are taken to display the smiley but no smiley in there. If the user is connected, the conversation in the Skype application displays well the smileys.
And when the bot answer with a smiley, it is displayed as text :) not replaced by an image. Is there a way to do it?
I also have the following error:
Cross-Origin Request Blocked:
The Same Origin Policy disallows reading the remote resource at
https://browser.pipe.aria.microsoft.com/Collector/3.0/?qsp=true&content-type=application%2Fbond-compact-binary&client-id=NO_AUTH&sdk-version=ACT-Web-JS-2.9.0&x-apikey=xxx.
(Reason: CORS request did not succeed).
Does anyone know how to fix it?
There are multiple questions. To enable emojis you can customize webchat control. We did it and enabled emojies and other features too. Below is the link of source of webchat:
https://github.com/Microsoft/BotFramework-WebChat
For CORS issue (cross domain security issue) have you tried placing the code on ms azure app. We faced the issue, with our own server, but not on azure app. CORS can be configured using web.config too.
When sharing the URL on LinkedIn, the OpenGraph data won't show up. I did validate the data. Is this maybe a domain issue?
The url i tried to share was:
https://overons.kpn/en/news/2017/kpn-and-major-suppliers-aim-for-circular-operation-by-2025
edit: following error was thrown on just the domain: i18n_url_preview_error
So LinkedIn share was working fine for me.
Today I started facing this error after moving on HTTPS.
Here is what I did :
While sharing URL, use http instead of https on LinkedIn and it will work (assuming that http redirects to https on your website).
I tried it for my site CoverMyStartup today. If this doesn't work, let me know.
The LinkedIn system does not recognize many of the new top level domains as being valid, as many of them have become a huge source of spamming and scamming.
Works perfectly fine for me. As usual, make sure you are URL-encoding your parameters...
https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.overons.kpn%2Fen%2Fnews%2F2017%2Fkpn-and-major-suppliers-aim-for-circular-operation-by-2025
See:
Maybe it was rejecting .kpn domains previously, but it seems to be good and working now!
I've been trying to send data to Google Analytics through url, but it's not working.
Here is the url:
http://www.google-analytics.com/__utm.gif?utmac=MY-UA
&utmn=0.57942900+13058623924dd5e0f88d784
&utmp=%2Fmypage.html
&utmr=http%3A%2F%2Fpp.test%2Fmypage-ref.html
&utmdt=test+ga&utmul=en-us
&utmhn=pp.test
&utmcc=__utmz%3D1.1304060743.1.1.utmcsr%3D%28direct%29%7Cutmccn%3D%28direct%29%7Cutmcmd%3D%28none%29%3B+__utma%3D1.100229728.1304060743.1304060743.1304060743.1%3B+__utmz%3D185677488.1304391672.1.1.utmcsr%3D%28direct%29%7Cutmccn%3D%28direct%29%7Cutmcmd%3D%28none%29%3B+__utma%3D185677488.2126025763.1304391672.1304667350.1305106191.6%3B+PHPSESSID%3Dpghin7v4q3g5lssq5c5mjchfj6
&utmcn=1
&utmcr=1
&utmwv=1
&utmsr=1280x1024
&utmsc=32-bit
&utmcs=-
&utmje=0
I have been trying follow the parameter here http://code.google.com/apis/analytics/docs/concepts/gaConceptsOverview.html
I call the url by curl and tested through the address bar, but not working, no data in my Google Analytics :(
Any ideas please?
It's possible, although if you're trying to do it you're probably doing something wrong.
But there's more to it than you can see at first sight. The best way to achieve it is to use a mobile library for Google Analytics. Google already provides this for a couple of languages. These libraries are built to send data to Google Analytics via your server. And if you check the server code you'll see that there are some calculations involved. You can possibly change one of these libraries to be run without a request. They're not complicated.
http://code.google.com/mobile/analytics/docs/web/
Here's the scenario:
I have a mailing list that contains a PDF download link. The PDF contains ads with clickable links. I need to get analytic data on the link clicks - preferably via Google Analytics (due to the richness of information available).
The solution I have in mind is for the link to go to a web page that I host with some sort of ad-specific token. GA records the request and then I use a client-side technique to redirect to the actual target URL. The redirect page serves no purpose other than to track the click and so I'm not worried about it being perceived as cloaking by search engines.
What I want to know is:
Are there any alternative ways to achieve the tracking without using an intermediate redirect page (could I perhaps call GA server-side somehow)?
If I do use the redirect page approach, what are potential pitfalls could I encounter?
Thanks in advance for any advice.
dunno what server-side environment/language you use but for instance in php you can use cURL to send an image request to google, with the custom code appended to the url. Easiest way to do it is to output the code with javascript with your custom code and then capture the image request url with a sniffer, so you can replicate the format for your cURL request. Make sure to send header info, including fake browser info so GA doesn't weed it out as a bot. Then forward to the ad url. That way you don't need to output a page.
Yeah you still have a 'redirect' happening but you cut out having to have the client download a page or worry about javascript being disabled, etc...
unfortunately there really isn't anything better you can do.