I have a root domain website www.example.com and multiple subdomains - sub1.example.com, sub2.example.com, sub3.example.com.
All these subdomains have sitemap index file of their own, sub1.example.com/sitemap_index.xml, sub2.example.com/sitemap_index.xml, etc.. and it is being indexed properly.
Is it required for me to list all the sitemap index files in robots.txt of the root domain (www.example.com/robots.txt)? Or is it not mandatory?
Please explain how sitemap listing works across subdomains. Thank you.
I'm wondering the same thing too. According to Sitemap.org
"Note: A Sitemap index file can only specify Sitemaps that are found on the same site as the Sitemap index file.
For example:
http://www.yoursite.com/sitemap_index.xml can include Sitemaps on http://www.yoursite.com but not on http://www.example.com or http://yourhost.yoursite.com. As with Sitemaps, your Sitemap index file must be UTF-8 encoded."
Related
I have a Wordpress site (single page) on the root domain and another Wordpress site on a subfolder for my blog. The single page on the root domain does not link to the blog in the subfolder, nor do I want it to.
The question is, should I have 2 site maps, one in the root folder and one in the blog subfolder? If I only need one in the root domain, how do I go about making sure that the blog subfolder is included in the root sitemap?
And should I have both the root URL and the blog subfolder URL showing separately in Google Webmaster tools?
Would appreciate some input on this as I'm really confused. Thanks!
“I have a Wordpress site (single page) on the root domain and another Wordpress site on a
subfolder for my blog.” - ltm778
Seems like a lot of work. You should try setting your root domains WordPress install, custom structures option to something like "blog". Considering your root urls WordPress install consists of one page, you could write "blog post's" from within that root urls WordPress install that look a little something like this www.thisismydomain.com/blog/hello-world/.
“The question is, should I have 2 site maps, one in the root folder and one in the blog subfolder?” - ltm778
I am not too sure on how all search engine robots handle looking for sitemaps, if whether they only look in the root directory or also search sub-path directory's. There are tools however that help aide you in submitting multiple sitemaps. You can have sitemaps in different sub-path directory's, but it doesn't make it good practice. You can even submit multiple sitemaps that are found under one domain to Google Webmaster.
It's extra work running two WordPress sites, but you can have two different installs, and you can have each install have it's own sitemap that you would later submit to Google Webmsater Tools. Google XML Sitemap Generator for WordPress by Arne Brachhold http://www.arnebrachhold.de/redir/sitemap-home/ is a great sitemap generator. I have not noticed any difference in SEO by having two separate WordPress installs, with two separate sitemaps. To Googlebot, it's all the same website.
What sort of sitemaps do you plan on having? An XML file? An XML.GZ file? A static webpage with sitemap hyperlinks?
“And should I have both the root URL and the blog subfolder URL showing
separately in Google Webmaster tools?” - ltm778
ANSWER:
You do not have to add sub-path urls to Google Webmaster tools, as the root url, once added, covers all urls of that root domain.
There is soooo much information about htaccess redirect that I think I'm lost.
Here is my current structure
/web/
/web/CMS/
In the root I have a vBulletin forum and in /CMS/ I have wordpress. (Ideally I should have vBulletin in a /FORUM/ folder maybe?)
I'd like to setup something like this:
www.domain.com OR domain.com goes to /web/CMS/
forum.domain.com OR forums.domain.com goes to /web/
Then any other subdomain goes to /web/CMS/ (as I'll be using the multi site network of Wordpress)
I have 2 questions:
What would be the best structure for this? Maybe I should have my CMS as root and my forum in a folder?
How can I do an htaccess that does the job? (do I also need one in /web/CMS/?)
Thank you so much in advance.
If you have vBulletin in your root and you want it to go to forums, you should consider moving it into it's own directory called /forums. Inside the /forums directory, you'll have a .htaccess file generated by vBulletin.
You can keep your WordPress inside /CMS. WordPress will and should have its own htaccess file.
Now if you want to set a subdomain, you'll have to look into your DNS settings and at a minimum setup an A record. If you're on cPanel, this is pretty simple to do inside your DNS Zone section.
I have magento on my root domain with wordpress integration, so currently I have url like www.myurl.com/blog and I don't to display /blog in my url. the result url should be www.myurl.com/
is there a way to do this?
You need to go to settings>general and change the site url to www.myurl.com and the wordpress url to www.myurl.com/blog then COPY not move the .htacess and index.php files into the root directory. This should use wordpress as the site for the domain now. In order to access the backend you will still need to go to www.myurl.com/blog/wp-admin
I have an apache server with a WordPress website set up at root and a WordPress website in a subdirectory. I would like for the website in the subdirectory to be the default website. For example, when someone goes to www.example.com I would like for them to see the website that is actually located at www.example.com/newsite but the URL needs to stay the same, www.example.com.
Is doing something like that possible with say a 301 redirect or some other method?
check this out, you just need some htaccess changes:
https://codex.wordpress.org/Giving_WordPress_Its_Own_Directory
You can simply move/copy the files to the root folder in case you are having root access.
cp -pR /path/to/rootfolder/subdirectory(wordpress)/* /path/to/rootfolder/
Thats it. Don't complicate by putting redirects.
In case you are having a control panel like cPanel, Its easy to put redirects. But as far as I concern moving/copying the files are more reliable.
I have a domain like domaindev.com.
I have set the robots.txt file on this domain to block everything:
User-agent: *
Disallow: /
Just like that. So that blocks everything from being crawled.
Now here is where it gets interesting. We also have several subdomains.domaindev.com hosted on our server and I want to block all of those subdomains as well from being crawled. I want a simple way to block any old subdomains and any new subdomains. Is there a line I can add to the www.domaindev.com that will prevent any subdomains under the domaindev.com?
Is the best way to make a default robots.txt and just drop it in all of the subdomain folders manually.
I'd really like a definite solution so as not to have to continually do same.
We use WordPress and in the wp-admin we have set it to not let spiders crawl our websites. But somehow these websites are finding there way into Google.
How do I go about it?
I've searched the site and found this line to add to my .htaccess file:
Header set X-Robots-Tag "noindex, nofollow"
I'm going to do that as well.
I also saw that it is part of the standard that each subdomain would need its own robots.txt file.
Put an Alias directive in your httpd.conf file outside of any VirtualHost section to catch any "robots.txt" requests.
Alias /robots.txt /home/www/robots.txt