I created a page on a Wordpress site that was for internal use only and triggers some backend code. Within a few days I started seeing hits on that page from "bingbot".
I'm not using any kind of sitemap plugin. How are crawlers finding this page?
I know the robots.txt file can block them but I want to make sure they don't show up for crawlers that don't respect this. I still want to have the page publicly accessible if someone types in the URL.
What needs to be done in Wordpress to make sure a page can't be discovered except by typing in the URL?
Any given URL is potentially "discovered" once the post is published and if there's a link to it from elsewhere on your site. There's no guaranteed way to prevent search engines from indexing a URL.
Related
I am working on a website "https://datasiplus.com".
When i type datasiplus on google, i get as 3rd result this url "https://datasiplus.com.cutestat.com/".
Is it normal ?
Can it be the cause for my website having unwanted popup ads ?
How to check search results of website on google?
You can see all indexed pages from your website (domain) if you go to google search and type the following
site:datasiplus.com
cutestat.com is it normal?
This page is a tool to get information about a specific domain. It's estimating the value, traffic and lot more. Either this tool has automatically crawled your page or someone searched for your domain with it.
There is a form on their site, where you can request to remove your domain from cutestat.com here
So yes, it's normal that this is in google index because it's like a subpage of their tool and datasiplus is a keyword for both sites, yours and datasiplus.com.cutestat.com
If you go to google now and search for datasiplus, then you can already see your own question there.
Can it be the cause for my website having unwanted popup ads?
No, this page will not cause unwanted popup ads on your page (or any other page).
Popups like this is most probably caused by malware on your page. This may be introduced through some security holes in wordpress and / or from one of the plugins you are using.
To get started to search and remove such malware, you can start at this SO question
Working on a live Wordpress website deleted a page that has been published by mistake:
1- Turned the page into a Draft page, but it was still showing
2- Moved the page to Trash on the Wordpress dashboard and then deleted it permanently from the bin, but it still showing in the google search.
3-Cleared the catch, but it still showing up in google search and the link returns a 404 error.
Following up Google instruction:
Make removal permanent:
Remove or update the actual content from your site (images, pages,
directories) and make sure that your web server returns either a 404
(Not Found) or 410 (Gone) HTTP status code. Non-HTML files (like PDFs)
should be completely removed from your server. (Learn more about HTTP
status codes)
Block access to the content, for example by requiring a password.
Indicate that the page should not to be indexed using the noindex meta
tag. This is less secure than the other methods.The Remove URLs tool is
only a temporary
The first part is easy and shows the 404 error.
The second part:
Turning the page to a password protected page, now it takes me to password protected page and it is still showing on the Google search.
Also for noindex I had the following options:
Any idea why or any recommendation?
Thank you
Google search is based on indexing, it takes time to update your website's content, pages, and dependencies. Search updates are based on mechanism of Google crawlers (so called spiders). These spiders crawls your website's content and follow your meta tags and robots.txt file.
Generally, it will take about 1-3 days to get your page removed from the search results. There are no quick ways to do it as it is based on indexing. Make sure you update your Google webmasters account (just check for errors on your account which relates to a page not found).
My website was recently hacked and Google mentioned "This site may be hacked" so I removed the entire wordpress website and changed server. I installed a new wordpress website however, Google is still crawling the old pages. It runs into 20+ Google pages meaning there's over 200 links generated by the hack.
Now, I would like to do a 301 Redirect to all this links using htaccess so Google cache faster and remove these links.
How do I list all this links displayed in the searched result? or is there a better way to do? Yes, I have asked Google for Review but they said it will take several weeks :(
You can see all youre indexed pages by writing this in Google search
site:example.com
Just write youre domain name instead of example.com
I need to test in terms of SEO a new page which was exported from wordpress to single page application. I want to keep my rank in search results. Unfortunately I am not sure how to do it properly.
Could you please give me any advice?
I know that I need to verify sitemap and broken links. Could you please recommend any tools to do it automatically?
The most important thing to do is to use the same URLs you have from your old website, or use 301 redirect method to redirect the old URLs to the new one.
Regarding the one pages, its not good for SEO since all your content will be in the same page, and here you can't target many keywords.
Still you can use one page website with advanced techniques to rank one many keywords but here it will affect the UX.
Regarding the XML site, since you have one page in your new website, you have to redirect all the pages to the new one using 301 redirect or you will have many 404s in the webmaster tool.
first i like know how can i see what are the pages are cached of my web site. say my web site is www.mysite.com
i am going to change few urls of my site but there is one problem that i may loose SEO. suppose google cached this page of my url like
www.mysite.com/detailproduct.aspx?id=200 now i have change the location and name of the page. say now detail product name change to product and url looks like
www.mysite.com/catalog/products.aspx?id=200 so when people search google and if this link www.mysite.com/detailproduct.aspx?id=200 comes in google search and if user click on this link then no relevant page will display. so first of all i need to know what are the different pages has been cached of my web site by google if i know then i can write permanent redirection logic as a result google cache pages url will be change..i guess.
if anyone know any best practice to handle this situation then please discuss in details. the situation is few page name and location has been change and if user search google and if old page url comes then no page will display when user click on those link. i want to handle this situation in best way....what is all of your suggestion. thanks