Selenium has a function called getPageSource that will get the source code of the page. I'm looking for the equivalent in Cypress.
I've tried
cy.get('html:root')
.eq(0)
.invoke('prop', 'outerHTML')
but, it doesn't really return the source of the page. As an example, the source contains ©, but the above cypress command shows it as ©. I want to see the actual source of the page, what I would see if I were to go to the server and open the file up in any plain text editor like notepad.
The naïve answer is to use string.replaceAll()
cy.get('html:root')
.eq(0)
.invoke('prop', 'outerHTML')
.then(pageSource => pageSource.replaceAll('©', '©'))
See dev.w3.org/html5/html-author/charref for a reference chart.
But I have a feeling there is a conversion function somewhere that does any and all occurrences.
Getting the equivalent of "View page source"*
cy.request(my-url)
.its('body') // NB the response body, not the body of your page
.then(content => {
// send content to validator.w3.org
// you can probably cy.visit('validator.w3.org') and manipulate the
// validation page, pasting in content value as required
})
Related
I am trying to scrape this website: https://www.casablanca-bourse.com/bourseweb/Societe-Cote.aspx?codeValeur=12200 the problem is i only want extract data from the tab "indicateurs clès" and i can't find a way to access to it in code source without clicking on it.
Indeed, i can't figure out the URL of this specific tab... i checked the code source and i found that there's a generated code that changed whenver i clicked on that tab
Any suggestions?
Thanks in advance
The problem is that this website uses AJAX to get the table in the "Indicateurs Clès", so it is requested from the server only when you click on the tab. To scrape the data, you should send the same request to the server. In other words, try to mimic the browser's behavior.
You can do it this way (for Chromium; for other browsers with DevTools it's pretty much similar):
Press F12 to open the DevTools.
Switch to the "Network" tab.
Select Fetch/XHR filter.
Click on the "Indicateurs Clès" tab on the page.
Inspect the new request(s) you see in the DevTools.
Once you find the request that returns the information you need ("Preview" and "Response"), right-click the request and select "Copy as cURL".
Go to https://curl.trillworks.com/
Select the programming language you're using for scraping
Paste the cURL to the left (into the "curl command" textarea).
Copy the code that appeared on the right and work with it. In some cases, you might need to inspect the request further and modify it.
In this particular case, the request data contains `__VIEWSTATE` and other info, which is used by the server to send only the data necessary to update the already existing table.
At the same time, you can omit everything but the __EVENTTARGET (the tab ID) and codeValeur. In such a case the server will return page XHTML, which includes the whole table. After that, you can parse that table and get all you need.
I don't know what tech stack you were initially going to use for scraping the website, but here is how you can get the tables with Python requests and BeautifulSoup4:
import requests
from bs4 import BeautifulSoup
params = (
('codeValeur', '12200'),
)
data = {
'__EVENTTARGET': 'SocieteCotee1$LBFicheTech',
}
response = requests.post('https://www.casablanca-bourse.com/bourseweb/Societe-Cote.aspx', params=params, data=data)
soup = BeautifulSoup(response.content)
# Parse XHTML to get the data you exactly need
I can get basic html text to flip 180*, but I'd like to know how to get a whole Doc in my Drive file to flip using a standalone script (so I can do it repeatedly). I'm aware I can get a doc, open the scripts editor and then use my flippin' project to flip the doc I called, but I don't know what the syntax looks like. My first flippin' success was pasting text into the .html file as simply as possible and using:
function doGet() {
return HtmlService.createHtmlOutputFromFile('Page');
getContent()
}
I just test ran it from the dialog box as a web app. But I'm interested in building this one command feature out into several different domains to get experience with the variety of possibilities available in GAS. Anyone care to tutor me? Please?!...
Is there anyway we can add data like in php echo "something" in the first html page. I want to know the server's timestamp to format a document created time like 2 hours ago, the document already has a property createdTime. When I use Meteor.Collection.find, I cannot add the server time by using transform.
I can use Meteor.method but I may have to format time before the result arrives.
Thank you.
Well, after digging around the code, here is the answer.
You can use the global variable __meteor_runtime_config__ to add more information to the first downloaded html file. In my case, in a server side javascript file, I add __meteor_runtime_config__.now = new Date().getTime() and this value will be available on the client side
the __meteor_runtime_config__ approach is run-once; that is, only changes made at package load time (not Meteor.startup()) are taken into account, and then the __meteor_runtime_config__ snippet is frozen.
To pass run-time (per-page) metadata to the page, it looks like the only option is to set a custom tag on the <html> element using the (public, but undocumented) WebApp.addHtmlAttributeHook API.
I have written a GUI extension which adds an additional tab to many of the Item views in the SDL Tridion CME (e.g. Component, Page and Schema etc.). I have also written some JavaScript which loads that tab directly if when the view is loaded with a tab name is specified in the URL.
The result is that if a page is loaded with the tab name added as follows:
http://localhost/WebUI/item.aspx?tcm=64#id=tcm:1-48-64&tab=InfoTab
Rather than the default of
http://localhost/WebUI/item.aspx?tcm=64#id=tcm:1-48-64
The Info Tab will be loaded on top, instead of the General Tab. This is performed with the following code snippet and works very well:
$evt.addEventHandler($display, "start", onDisplayStarted);
// This callback is called when any view has finished loading
function onDisplayStarted() {
$evt.removeEventHandler($display, "start", onDisplayStarted);
var tabname = $url.getHashParam("tab");
if (tabname != '') {
var tabControl = $controls.getControl($("#MasterTabControl"), "Tridion.Controls.TabControl");
tabControl.selectItem(tabname);
}
}
Now I would like to make a context menu item to open items and link to the tabs using my new functionality. My first thought was to construct the Item URL myself and simply open a new window in my execute method. So I looked at the default functionality in the standard Open.prototype_execute() functionality of the GUI. This is stored in the navigation.js file of the CME, and is performed by the Tridion.Cme.Commands.Open.prototype._execute method. The code is a lot more complicated than I had anticipated as it deals with shared items, and permissions etc.
Rather than just copying all of this code to my own function, I was wondering if there is a way to elegantly extend the existing Open.prototype_execute() function and append my “&tab=MyTab” to the $cme.Popups.OPEN_ITEM_OPTIONS.URL constant for my own functions.
Any advice would be greatly appreciated.
At the end the Open command uses $config.getEditorUrl(item_type) to get the url for the item view (item_type - $const.ItemType.COMPONENT, etc). There are no extension points for this part of the functionality, but you could always try to overwrite it on your own risk.
I'm trying to import nodes from my forum to drupal 7. Not in bulk, but one by one so that news posts can be created and referenced back to the forum. The kicker is that I'm wanting to bring image attachments across as well...
So far, using the code example here http://drupal.org/node/889058#comment-3709802 things mostly work: Nodes are created, but the images don't go through any validation or processing.
I'd like the attached images to be validated against the rules defined in the content type. in particular the style associated with my image field which resizes them to 600x600.
So, instead of simply creating the nodes programatically with my own form, i decided to modify a "new" node using hook_node_prepare and using the existing form to create new content (based on passed in url args). This works really well and a create form is presented pre-filled with all my data. including the image! very cute.
I expected that i could then hit preview or save and all the validation and resizing would happen to my image, but instead i get the error:
"The file used in the Image field may not be referenced."
The reason for this is that my file doesn't have an entry in the file_usage table.. *le sigh*
so, how do i get to all the nice validation and processing which happens when i manually choose a file to upload? like resizing, an entry in the file_usage table.
The ajax upload function does it, but i can't find the code which is called to do this anywhere in the api.
What file upload / validation functions does Drupal call which i'm not doing?
Anybody have any experience with the file/image api for Drupal 7 who can help me out?
For getting the usage entry (in essence, checking out a file to a specific module so that it doesn't get deleted while its in use) look up the Drupal function 'file_usage_add()'
For validating incoming images, I got this example from user.module (if you're comfortable with PHP, you can always look at the core to see how something is done the 'Drupal way'):
function user_validate_picture(&$form, &$form_state) {
// If required, validate the uploaded picture.
$validators = array(
'file_validate_is_image' => array(),
'file_validate_image_resolution' => array(variable_get('user_picture_dimensions', '85x85')),
'file_validate_size' => array(variable_get('user_picture_file_size', '30') * 1024),
);
// Save the file as a temporary file.
$file = file_save_upload('picture_upload', $validators);
if ($file === FALSE) {
form_set_error('picture_upload', t("Failed to upload the picture image; the %directory directory doesn't exist or is not writable.", array('%directory' => variable_get('user_picture_path', 'pictures'))));
}
elseif ($file !== NULL) {
$form_state['values']['picture_upload'] = $file;
}
}
That function is added to the $form['#validate'] array like so:
$form['#validate'][] = 'user_validate_picture'