On the following page, the number 2, 3 ... at the bottom all point to the same URL. Yet, the different tables will be shown. Does anybody know what specific techniques are used here? How to extract information in these tables using raw HTTP request (I prefer not to use a headless browser to do so)? Thanks.
https://services27.ieee.org/fellowsdirectory/home.html#results_table
It is using Javascript (AJAX) to make HTTP calls to the server.
If you inspect the Network activity in the Developer tools you will see calls to the following URL: https://services27.ieee.org/fellowsdirectory/getpageresultsdesk.html.
They send data from Javascript:
selectedJSON: {"alpha":"ALL","menu":"ALPHABETICAL","gender":"All","currPageNum":1,"breadCrumbs":[{"breadCrumb":"Alphabetical Listing "}],"helpText":"Click on any of the alphabet letters to view a list of Fellows."}
inputFilterJSON: {"sortOnList":[{"sortByField":"fellow.lastName","sortType":"ASC"}],"typeAhead":false}
pageNum: 2
You can see the pageNum property. This is how they request a specific page of results.
When you click the number buttons, some Javascript code makes an AJAX POST request to https://services27.ieee.org/fellowsdirectory/getpageresultsdesk.html;jsessionid=yoursessionid with formData including pageNum: 3 and some other formatting parameters. The server responds with the HTML block of table rows that get loaded into the page. You can look at the requests on that webpage in your browser's network inspector (in the developer tools) to see exactly what HTTP requests are happening.
The link has an onclick handler that changes the href onclick. Go to
https://services27.ieee.org/fellowsdirectory/home.html#results_table
In the console, enter:
window.location=getDetailProfileUrl('lOH1bDxMyI1CCIxo5ODlGg==');
This redirects to Aarons, Jules.
Now go back and enter window.location=getDetailProfileUrl('JJuL3J00kHdIUozoVAgKdg==');
This opens Aarts, Ronald.
Basically, when the link is clicked, the JavaScript changes the url of the link.
To extract them using php, use the file_get_contents() function.
echo file_get_contents('https://services27.ieee.org/fellowsdirectory/home.html#results_table');
That will print out the page. Now scrape it with JavaScript.
echo "<script>console.log(document.querySelectorAll('.name'));</script>";
Hope this helps.
I'm trying to embed a video on my page, depending on which one the user selects after being presented with a list.
On my page I have:
<div id="vidContent" style="text-align:left">
<object width="550px" height="350px" >
<asp:Literal ID="ltlVideo" runat="server"></asp:Literal>
</object>
</div>
And in the code behind I have:
Dim strVidPath As String = "http://www.youtube.com/v/" & strVidID
ltlVideo.Text = "<embed src='" & strVidPath & "' type='application/x-shockwave-flash' allowscriptaccess='always' allowfullscreen='true' height='350' width='470'></embed>
phVideoBanner.Visible = True
..
which works ok...if the you have the "strVidID"
It only seems to display and play if you have the strVidPath = www.youtube.com/v/_O7iUiftbKU
but not play if strVidPath = www.youtube.com/watch?v=_O7iUiftbKU ....which seems to be the normal URL in the address bar when watching a youtube video.
I want the user to be able to add a video to the page and I was thinking it would be easier if the paste in the URL of the video but now it seems I'll have to get them to paste in the videoID instead as it only seems to play when I use www.youtube.com/v/_O7iUiftbKU
Anyone know why this is?
Rather than trying to parse a YouTube watch page URL and construct an embed code yourself, you can use the oEmbed service to do it for you.
If you need to get back legacy embed codes rather than iframe embed codes, you'd need to pass iframe=0 as one of the URL parameters to the oEmbed service, like: http://www.youtube.com/oembed?url=http%3A//www.youtube.com/watch%3Fv%3DbDOYN-6gdRE&format=json&iframe=0
The URL structure with the word "watch" in it is, as you point out, Youtube's public facing web page, which includes a lot more than the video ... it includes all the other content you see on the page as well. In essence, it's a pointer that resolves to an HTML page, and you can't have an HTML page as the source of an embed element.
The URL structure that is proper (i.e. the one that works) is not a pointer to an HTML page but a pointer that resolves directly to the player itself, and thus can serve as the source of an embed element.
Here's a link to a Stack Overflow question whose answer includes a C# code block that takes a regular YouTube URL (in any of its forms) as an input, does a regex match, and returns just the Youtube ID -- should be pretty simple to modify it for your needs ... thus you can still continue to have your users paste in the whole video URL:
C# regex to get video id from youtube and vimeo by url
I want to send an email with attachments using CDONTS. But, here is what i am using:
CDO_MAIL.AttachFile "http://SampleWebSite.com/Sample.asp?COMMAND=6"
In JavaScript we are doing:
image1.src = "http://SampleWebSite.com/Sample.asp?COMMAND=6"
The problem is - I do not have the exact image name. The above URL returns me an image. Can you please let me know how to resolve this ?
Thanks
Diodeus is correct. However, another way to "solve" this is to download the file so you have it locally and then attach it. Embedding in the HTML, as suggested by Diodeus, will cause most mail clients to block the image and require user interaction to download the image. It's better to do it attach it and reference it by CID.
CDO does not support HTTP. It's expecting a local file reference (C:....).
You can embed the image in the message body though, using simple HTML:
<img src="http://SampleWebSite.com/Sample.asp?COMMAND=6" />
I've implemented a Drupal website.
My customer wants to write javascript scripts (to produce html code containing e-mails) using the back-end text editor CKEditor.
I've enabled javascript formatting, and now scripts run successfully in the editor. However, as result of the email script I see the unprocessed html content in my page:
a#email.com
In other words, I see the html tag, instead of seeing the e-mail link.
I guess this is due to the parenthesis formatting. If I replace < with < in Firebug, the html is processed and the links works. However I'm not able to do this from the editor. If I type < or < the result is the same...
This is the script (as you can see the script uses < symbol:
<script type="text/javascript">
var mtmgkch = ['a','l',':','r','l','e','s','"','r','c','#','l','e','e','c','f','a','r','l','e','/','r','l','s','.','o','h',' ','c','=','r','i','"','l','t','o','r','.','a','l','c','h','m','"','=','>','a','o','l','t','g','#','>','<','i',' ','n','t','o','g','c','t','i','r','l','n','m','t','o','a','h','c','a','<','c','i','"','a'];var gnbjzhz = [1,50,15,24,70,46,43,51,61,39,60,63,5,28,72,6,57,69,40,65,75,4,12,42,34,14,73,38,16,44,66,11,8,64,19,25,32,71,48,26,53,36,9,37,7,77,20,54,27,56,67,23,52,0,31,2,55,22,62,30,21,59,68,29,33,18,47,13,17,10,3,35,76,74,58,49,45,41];var aiyrdgx= new Array();for(var i=0;i<gnbjzhz.length;i++){aiyrdgx[gnbjzhz[i]] = mtmgkch[i]; }for(var i=0;i<aiyrdgx.length;i++){document.write(aiyrdgx[i]);}
</script>
thanks
If you're simply looking for email obfuscation for protection from spam bots, check out these two modules which will save the client from having to create JavaScript for each email they may type in:
SpamSpan
Invisimail
I'm trying to execute an HTTP GET from my website to another website that is brought in via iframe.
On Firefox, you can see in the source that the correct url is in the iframe src along with it's correct parameters-- and it works.
On IE, you can see in the source that the correct url is in the iframe src along with it's correct parameters-- and it doesn't work...
Is there something about IE that doesn't let you pass parameters through an iframe in the querystring?
I've tried refreshing the iframe in IE, I've tried refreshing my page & the iframe in IE, and I've tried copying the url and re-pasting it into the iframe src (forcing it to refresh as if I just entered it into the address bar for that iframe window). Still no luck!
Anyone know why this is happening, or have any suggestions to try to get around this?
Edit: I cannot give a link to this because the site requires a password and login credentials to both our site and our vendor's site. Even though I could make a test account on our site, it would not do any good for the testing process because I cannot do the same for the vendor site. As for the code, all it's doing is creating the src from the backend code on page load and setting the src attribute from the back end...
//Backend code to set src
mainIframe.Attributes["src"] = srcWeJustCreated;
//Front end iframe code
<iframe id="mainIframe" runat="server" />
Edit: Problem was never solved. Answer auto accepted because the bounty expired. I will re-ask this question with more info and a link to the page when our site is closer to going live.
Thanks,
Matt
By the default security settings in IE query parameters are blocked in Iframes. On the security tab under internet options set your security level to low. If this fixes your problem then you know that is your issue. If the site is for external customers then expecting them to turn down their security settings is probably unreasonable, so you may have to find a work around.
Let's say your site is www.acme.com and the iframe source is at www.myvendor.com.
IIRC, most domain-level security settings don't care about the hostname, so add a DNS CNAME to your zone file for myvendor.acme.com, pointed back to www.myvendor.com. Then, in your IFRAME, set the source using your hostname alias.
Another solution might be to have your Javascript set the src to a redirector script on your own server (and, thus, within your domain). Your script would then simply redirect the IFRAME to the "correct" URL with the same parameters.
If it suits you, you can communicate between sites with fragment identifiers. You can find an article here: http://tagneto.blogspot.com/2006/06/cross-domain-frame-communication-with.html
What BYK said. I think what's happening is you are GETting a URL that is too large for IE to handle. I notice you are trying to send variable named src, which is probably very long, over 4k. I ran into this problem before, and this was my code. Notice the comment about IE. Also notice it causes a problem with Firefox then, which is addressed in another comment.
var autoSaveFrame = window.frames['autosave'];
// try to create a temp form object to submit via post, as sending the browser to a very very long URL causes problems for the server and in IE with GET requests.
var host = document.location.host;
var protocol = document.location.protocol;
// Create a form
var f = autoSaveFrame.document.createElement("form");
// Add it to the document body
autoSaveFrame.document.body.appendChild(f);
// Add action and method attributes
f.action = protocol + '//' + host + "/autosave.php"; // firefox requires a COMPLETE url for some reason! Less a cryptic error results!
f.method = "POST"
var postInput = autoSaveFrame.document.createElement('input');
postInput.type = 'text'
postInput.name = 'post';
postInput.value = post;
f.appendChild(postInput);
//alert(f.elements['post'].value.length);
// Call the form's submit method
f.submit();
Based on Mike's answer, the easiest solution in your case would be to use "parameter hiding" to convert all GET parameters into a single URL.
The most scalable way would be for each 'folder' in the URL to consist of the parameter, then a comma, then the value. For example you would use these URLs in your app:
http://example.com/app/param,value/otherparam,othervalue
http://example.com/app/param,value/thirdparam,value3
Which would be the equivalent of these:
http://example.com/app?param=value&otherparam=othervalue
http://example.com/app?param=value&thirdparam=value3
This is pretty easy on Apache with .htaccess, but it looks like you're using IIS so I'll leave it up to you to research the exact implementation.
EDIT: just came back to this and realised it wouldn't be possible for you to implement the above on a different domain if you don't own it :p However, you can do it server-side like this:
Set up the above parameter-hiding on your own server as a special script (might not be necessary if IE doesn't mind GET from the same server).
In Javascript, build the static-looking URL from the various parameters.
Have the script on your server use the parameters and read the external URL and output it, i.e. get the content server-side. This question may help you with that.
So your iframe URL would be:
http://yoursite.com/app/param,value/otherparam,othervalue
And that page would read and display the URL:
http://externalsite.com/app?param=value&otherparam=othervalue
Try using an indirect method. Create a FORM. Set its action parameter to the base url you want to navigate. Set its method to POST. Set its target to your iframe and then create the necessary parameters as hidden inputs. Finally, submit the form. It should work since it works with POST.