How to extract source from website as we see in inspect element - web-scraping

I would like to extract the complete html source of website as we get it on chrome inspect element.
I am trying to the scrap the comments of disqus from http://bossip.com/1190557/cops-kill-people-mississippi-cop-under-investigation-for-strangling-unarmed-black-man-jonathan-sanders-to-death-43081/
Initially I tried with java and Jsoup but I am unable to extract the complete html source.
Then as suggested in How to get html that I see in inspect element? I tried using HTML Unit but unfortunately HTML unit also did not extract as i expected.
Is there any solution around for this problem?

In Chrome,
in normal browser document: right-click on element, select "Inspect Element" from pop-up menu. You will go to the "developer tools" window.
in developer tools window: right-click on the HTML element and select "Edit as HTML" from pop-up menu.
select all HTML text in the editor (press CTRL-A) and copy to clipboard (press CTRL-C)

Related

How to Inspect element of non-inspect able sites?

I want the id or class or CSS selector name of sign-in Gmail button.
I have downloaded the page but can someone tell me how to find the name in these files?
https://auris.ahduni.edu.in/core-emli/code/my_home/
In chrome, f12 opens the development console and you can click over to Elements to view the page's HTML with highlighting as chrome normally does with inspect element.

How to Copy HTML and CSS of a particular webite from google chrome

I have done many Moves to get Get Direct CSS form website in google chrome but i failed. So tell me how to get it.
use view page source and in the opening window find the linking to the css right click it and open it in a new window you can copy it from there
for example in stackoverflow
<link rel="stylesheet" type="text/css" href="//cdn.sstatic.net/stackoverflow/all.css?v=c9958f49ae3f">
I'm using a mac so it may be different for you. Try right-clicking on the page. Then select 'view page source' from the list. The chrome will open a new tab with the source code inside. From there you can even open the external css and js files. At this point just copy and paste the code. To view a certain elements source code just choose 'inspect element' and click on the element of which you wish to see the css. Then (even though the styles are beside checkboxes it still lets you highlight them) copy and paste those.

chrome developer mode style sidebar linked css no more

In my Google Chrome version 38.0.2125.104, stylesheet filename/linked css has been disappeared. I'm not sure if updated version removed this or there is somewhere to set this up again, but when i restored to default, still it does not appear.
To make it clear to you guys, This is an example of old version Style Panel which have the link to the related stylesheet (filename & line) next to the class name.
http://i.pictr.com/7g52x66rw8.png (i cant attach images here yet)
and below is the latest Style Panel. Each classes usually have link to the CSS stylesheet .
http://i.pictr.com/sks0tp1sem.png
Any idea on how to enable back the stylesheet link?
Thanks in advance.
when you open the inspect element on the top right side of the dev eloper tools panel you have a cog wheel (settings). click that and go to general setting s tab and check out the elements section and sources section and see if all the boxes that needs to be checked are checked.... in mine i have the source maps box checked and enabled

How to enable real time CSS editing in chrome?

I have seem a lot of videos in which developers are changing CSS on the fly in chrome. I tried the same thing but chrome did not allow me to change the code. I can't write on the style sheet.
Is there any specific setting to do this? Kindly help.
EDIT: To edit the CSS, I right click on an element, select inspect element. It will open the console. I select the id of the element and go to style.css in Resources and try to change the CSS. It does not allow me to write there.
You are doing it wrong... the resources panel is not there for live edit, if you want to change the css associated with an HTML element, right click on that element and then in the right panel you will see the css styles associated with the selected element. You can edit that rules and you will see the changes in real time.
Maybe you can check some videos to learn some basics about the Chrome Developer Tools, and after that if you want to learn more, you can check this question:
Chrome Developer Tools: Best resource for learning advanced features?
Here is a great tool for Google Chrome called Stylebot.
In this you can change the style sheet and save your own styles to any website for your own custom website theme!
Here is the link for Stylebot
Check it out and to put the icing on the cake, it's free!
This should not be used to work on your own website projects since the CSS file saves local on your browser!
In Chrome, clicking on something like "all.css:1" in the Styles pane of the Elements tab of DevTools takes one to the Sources tab of DevTools. If you're looking at code on remote server, the CSS rules in this source view are not live-editable (unlike the live-editing Style Editor tab of Firefox*) unless you're:
viewing the "inspector-stylesheet" -- a temporary stylesheet containing new style rules you created with the "+" button in the Styles pane of the Elements tab. Clicking on a new rule's "inpector-stylesheet:1" link will take you to the editable source of the temporary rules you've created.
viewing a persistent local workspace. Setting this up takes a few extra steps, described here: "Set Up Persistence with DevTools Workspaces" .
Basically, you make a local folder on your machine where you can save local copies that you direct Chrome to use in in lieu of the version on the internet. See the instructions at that link. Note that, as it says there, "If you are mapping files from a remote server instead of a local server, when you refresh the page, Chrome reloads the page from the remote server. Your changes still persist to disk and are reapplied if you continue editing in Workspaces." (So just type a space character into the source local CSS file to see your alterations applied again, if you've refreshed or navigated to a different page that uses the same stylesheet.)
* In Firefox, if you right-click on an element on a remote webpage, select Inspect Element, then in the Rules pane of the Inspector tab, click on a link on the right like "all.css:1", you are taken to a "Style Editor" tab where you can immediately live-edit, in contrast to Chrome's requirement of making you map to a local file. This may cause some people some confusion, if they expect the same behavior from Chrome's DevTools.
Another Chrome extension that is similar to Stylebot is Code Cola. It has an inspector that allows selecting elements, and a visual editor which does not require typing the CSS by hand. To see the generated CSS code click the curly brackets icon in the toolbar.

Inject CSS with chrome developer tool?

Where can I add CSS to the page I'm viewing? I don't want to add style to one element directly, I want to add a 'document' to a page to debug changes before editing the site's style.css.
Note, there are lots of questions here about 'injecting CSS from a chrome extension', but specifically I want to do it via the 'Chrome Developer Tools' thingy.
I'm not sure if it works, but you'd have to try.
Pressing F12/ (Cmd + opt + I on Mac) to open up the Developer Console and go to the Console tab.
Copy paste the following code (edit the path):
$(document.head).append('<link rel="stylesheet" href="path_to_my_css">');
Alternatively, you could edit one property so the inspector-stylesheet.css is created by Chrome, and copy past your CSS source there.
There are multiple ways to do that, and they are also very easy.
First way, inspector-stylesheet:
Open Inspect Element (F12 Or Ctrl+ Shift+I)
Go to Elements tab (Ctrl+ Shift+ P and type show elements), if you are not already there, see the Styles tab, now see on right corner, there would be a + icon, click it (or long press that icon if it doesn't automatically add inspector-stylesheet), it will add selector of currently highlighted element, just next to the selector, there will a link/button inspector-stylesheet, click it, it will take you a window, where you can add styles.
Second way, Edit as HTML
Open Inspect Element (F12 Or Ctrl+ Shift+I)
Go to element panel (Ctrl+ Shift+ p and type show element panel).
Scroll to the head element right click on the element and choose Edit as HTML, go to the bottom of that element (Ctrl+ End), add your <style></style> element there add your style in that element, and hit Ctrl+ Enter.
Third way, Snippet:
Open Inspect Element (F12 Or Ctrl+ Shift+I)
Go to the Source tab, go to the Snippets tab, click on the + New snippet, name it whatever you want, and add this:
Create new snippet Ctrl+ Shift+ P type Create new snippet hit Enter , rename the snippet if you want to, and add this (edit the style) :
(function(){
let style = `<style>
/*change your style here*/
body{
display:none;
}
</style>`;
document.head.insertAdjacentHTML("beforeend", style);
})();
Save it, run it (Ctrl+Enter).
You can also run the snippets by doing this: Ctrl+ p type ! it will show your saved snippets, choose the one you want to run.
To edit or see your snippets, Ctrl+ Shift+ P type show snippets.
In FireFox it's even easier:
Open Inspect Element (F12)
Go to Style Editor, click the + icon and you will be able to edit the style; You can also, import styles, just by clicking the icon next to the +.
To begin with, this is one reason why I use Firefox for teaching and my own development work. The answer is built in.
As a variation to the above answers, using modern JavaScript, you can add a hard-coded style as follows:
document.head.insertAdjacentHTML('beforeend','<style> … </style>');
or you can add an external style sheet as follows:
document.head.insertAdjacentHTML('beforeend','<link rel="stylesheet" href="…">');
The beforeend argument is to help the injected CSS to override previously loaded styles.
If you’re going to do this repeatedly, you can then add it as a bookmarklet, using something like Bookmarklet Crunchinator.
The technique is similar to one I teach in a JavaScript class, where I use afterbegin to inject some default CSS, but allow user style sheets to override the defaults.
Why not a kind of simple framework agnostic one-liner like this?
document.body.appendChild(function() {var el = document.createElement('link'); el.setAttribute('rel', 'stylesheet'); el.setAttribute('href', 'http://domain/print.css'); return el;}())
Seems to work like a charm...
Since 2018 in Chrome (65) the browser's integrated DevTools has a feature called Local Overrides 1. As such, there is no need for an add-on or extension like StyleBot, Stylish or Greasemonkey.
Local Overrides allow rewrites of CSS, JS and DOM on any live site. Changes are saved in a local folder and they override the contents of the live environment.
This can be accessed under Developer Tools > Sources >> Overrides
This allows you to select a custom local folder that will contain CSS and JS that will override the current website's own CSS and JS.
This should work (paste into console, edit arguments in the last line as needed):
(function(i,n,j,e,c,t,css){
css=i.createElement(n);t=i.getElementsByTagName(c)[0];css.href=j;css.rel=e;
t.insertAdjacentElement('beforeend',css);})
(document,'link','//fonts.googleapis.com/css?family=Roboto','stylesheet','head');
This will insert a <link>
with an href //fonts.googleapis.com/css?family=Roboto
before the closing </head>
If there's no head tag in the document you're trying to add a css file to, try body as the last argument:
(document,'link','//fonts.googleapis.com/css?family=Roboto','stylesheet','body');
You can inject CSS using snippets in Chrome Devtools. Save and execute the snippet and then invoke it in the console or in another snippet:
function insertCss(code) {
var style = document.createElement('style');
style.type = 'text/css';
if (style.styleSheet) { // IE
style.styleSheet.cssText = code;
} else { // Other browsers
style.innerHTML = code;
}
document.getElementsByTagName("head")[0].appendChild( style );
}
// run the snippet as follows:
insertCss('span { color: red !important; }');
Go to the sources tab in dev tools and right click in the left column, then add folder to workspace and use file explorer to select the folder that contains your css file. You will have to allow to make changes, once you do this you will see your folder in the sources tree(MAKE SURE YOU SELECT FILESYSTEM TAB UNDER SOURCES TAB), open your folder find the file and right click on the your css file and select map to network resource. Once you map the file you can open and see it in the workspace and from that file any change made will affect the page styles. So basically your styles will over ride the served styles.
Is this what you're after?: "How to Edit Source Files Directly in Chrome" http://www.sitepoint.com/edit-source-files-in-chrome/
From that article:
Step 1: Launch Developer Tools. Go to View -> Developer -> Developer Tools. Navigate to "Sources"
Step 2: Click the Filesystem tab, then click + Add folder to workspace. You’ll be prompted to locate your work folder and Chrome will ask you to confirm that you Allow access.
Step 3: Edit and Save Your Code and refresh the browser to see your changes

Resources