SEO: Duplicated URLs with and without dash "/" and ASP.NET MVC - asp.net

after reading this article "Slash or not to slash" (link: http://googlewebmastercentral.blogspot.com/2010/04/to-slash-or-not-to-slash.html) on Google Webmaster Central Blog (the oficial one) I decided to test my ASP.NET MVC app.
For example:
http://domain.com/products and http://domain.com/products/ (with "/" in the end), return the code 200, which means: Google understands it as two different links and likely to be a "duplicated content". They suggest to choose the way you want... with or without dash and create a 301 permanent redirect to the preferred way.
So if I choose without dash, when I try to access http://domain.com/products/ it will return a 301 to the link without dash: http://domain.com/products.
The question is, how can I do that with ASP.NET MVC?
Thanks,
Gui

If your using IIS 7 you could use the URL Rewrite Extension ScottGu has a blog post about it here.
Alternatively if you want to do it in code you could inherit from PerRequestTask. Here some sample code the removes the www from an address - this is from Shrinkr:
public class RemoveWww : PerRequestTask
{
protected override TaskContinuation ExecuteCore(PerRequestExecutionContext executionContext)
{
const string Prefix = "http://www.";
Check.Argument.IsNotNull(executionContext, "executionContext");
HttpContextBase httpContext = executionContext.HttpContext;
string url = httpContext.Request.Url.ToString();
bool startsWith3W = url.StartsWith(Prefix, StringComparison.OrdinalIgnoreCase);
bool shouldContinue = true;
if (startsWith3W)
{
string newUrl = "http://" + url.Substring(Prefix.Length);
HttpResponseBase response = httpContext.Response;
response.StatusCode = (int) HttpStatusCode.MovedPermanently;
response.Status = "301 Moved Permanently";
response.RedirectLocation = newUrl;
response.SuppressContent = true;
response.End();
shouldContinue = false;
}
return shouldContinue ? TaskContinuation.Continue : TaskContinuation.Break;
}
}
You would just need to check for the url ending with a / in your code.
** Note this does use a 3rd party dll - System.Web.MVC.Extensibility namespace. **

It dosnt matter really for Google, but what does matter is if both urls'
http://domain.com/products and http://domain.com/products/ show the same page, you also need to watch with windows servers that links to your site like from external pages where the user has typed http://domain.com/PRODUCTS/ will aloso be seen as a diffrent page as the web is case sensitive.
There is away round this with the use of canonical url meta tag, it tell s google what the page name is really, so will avoid duplicate pages which ant really diuplicate
http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html

you need to check the URI in the INIT event and check the URI to see if it coming in with the slash, if it is, simply do a redirect and add the 301 header to the output response.

Related

.Net Core 3.1 Razor Pages : autoredirect to culture

I try to accomplish a similar behaviour with MS Docs.
For example, if you visit https://learn.microsoft.com/, you will be redirected to your culture, in my case I'm being redirected automatically to https://learn.microsoft.com/en-gb/.
Same goes for inner pages if you access them without the culture in the URL.
For instance, by accessing:
https://learn.microsoft.com/aspnet/core/razor-pages/?view=aspnetcore-3.1&tabs=visual-studio
it will be automatically redirect you to:
https://learn.microsoft.com/en-gb/aspnet/core/razor-pages/?view=aspnetcore-3.1&tabs=visual-studio
I have a small demo app where I conduct my localisation experiment for .NET Core 3.1 and Razor Pages here.
I have set options.Conventions here, and I have created CustomCultureRouteRouteModelConvention class here, but I'm fairly novice with .NET Core and I'm kind of stuck on how to implement the above-described functionality.
Thank you all in advance!
You should use existing Rewriting Middleware to do redirects: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/url-rewriting?view=aspnetcore-3.1
In the simplest form, you can tell rewrite middleware to redirect if it does not see a locale pattern at the beginning of the URL path, maybe
new RewriteOptions() .AddRedirect("^([a-z]{2}-[a-z]{2})", "en-US/$1")
(regex not tested) or do full redirect class with more detailed rules when and to what locale you want to redirect. Example in that aspnet document references RedirectImageRequest which you can use to get an understanding of how custom redirect rules works. Adapting to your case as a proof of concept, I reused most of the logic in your existing RedirectUnsupportedCulture:
public class RedirectUnsupportedCultures : IRule
{
private readonly string _extension;
private readonly PathString _newPath;
private IList<CultureInfo> _cultureItems;
private string _cultureRouteKey;
public RedirectUnsupportedCultures(IOptions<RequestLocalizationOptions> options)
{
RouteDataRequestCultureProvider provider = options.Value.RequestCultureProviders
.OfType<RouteDataRequestCultureProvider>()
.First();
_cultureItems = options.Value.SupportedUICultures;
_cultureRouteKey = provider.RouteDataStringKey;
}
public void ApplyRule(RewriteContext rewriteContext)
{
// do not redirect static assets and do not redirect from a controller that is meant to set the locale
// similar to how you would not restrict a guest user from login form on public site.
if (rewriteContext.HttpContext.Request.Path.Value.EndsWith(".ico") ||
rewriteContext.HttpContext.Request.Path.Value.Contains("change-culture"))
{
return;
}
IRequestCultureFeature cultureFeature = rewriteContext.HttpContext.Features.Get<IRequestCultureFeature>();
string actualCulture = cultureFeature?.RequestCulture.Culture.Name;
string requestedCulture = rewriteContext.HttpContext.GetRouteValue(_cultureRouteKey)?.ToString();
// Here you can add more rules to redirect based on maybe cookie setting, or even language options saved in database user profile
if(string.IsNullOrEmpty(requestedCulture) || _cultureItems.All(x => x.Name != requestedCulture)
&& !string.Equals(requestedCulture, actualCulture, StringComparison.OrdinalIgnoreCase))
{
string localizedPath = $"/{actualCulture}{rewriteContext.HttpContext.Request.Path.Value}";
HttpResponse response = rewriteContext.HttpContext.Response;
response.StatusCode = StatusCodes.Status301MovedPermanently;
rewriteContext.Result = RuleResult.EndResponse;
// preserve query part parameters of the URL (?parameters) if there were any
response.Headers[HeaderNames.Location] =
localizedPath + rewriteContext.HttpContext.Request.QueryString;
}
}
and registered it in Startup.cs with
// Attempt to make auto-redirect to culture if it is not exist in the url
RewriteOptions rewriter = new RewriteOptions();
rewriter.Add(new RedirectUnsupportedCultures(app.ApplicationServices.GetService<IOptions<RequestLocalizationOptions>>()));
app.UseRewriter(rewriter);
Improvement:
After using the above code I bumped on a bug that in case the culture is not supported by the application, the redirection will end up with infinite culture paths. For example, if I support the cultures en (default) and gr, if instead of either /en/foobar or /gr/foobar I would write /fr/foobar, I would end up getting /en/fr/foobar then /en/en/fr/foobar and etc.
I added private readonly LinkGenerator _linkGenerator; to the class, which I initialise it in the constructor. I removed that line string localizedPath = $"/{actualCulture}{rewriteContext.HttpContext.Request.Path.Value}"; and the code after that line looks like this:
rewriteContext.HttpContext.GetRouteData().Values[_cultureRouteKey] = actualCulture;
HttpResponse response = rewriteContext.HttpContext.Response;
response.StatusCode = StatusCodes.Status301MovedPermanently;
rewriteContext.Result = RuleResult.EndResponse;
// preserve query part parameters of the URL (?parameters) if there were any
response.Headers[HeaderNames.Location] =
_linkGenerator.GetPathByAction(
rewriteContext.HttpContext,
values: rewriteContext.HttpContext.GetRouteData().Values
)
+ rewriteContext.HttpContext.Request.QueryString;
As decribed in Microsoft docs localization middleware; each the localization request initializes a list of RequestCultureProvider and is enumerated by the below order :
QueryStringRequestCultureProvider : e.g. http://localhost:1234/Index?culture=en
CookieRequestCultureProvider : Looks for the culture cookie, and it will be null if you haven't set it manually.
AcceptLanguageHeaderRequestCultureProvider : This one depends on the browsers cultures adn this is what you need to look for.
To make sure how it works, delete the culture cookie and change the browser language preferences by moving the desired language to the top, you will see that the language is selected according to the browser preferences.

ASP.NET: Tracking dynamic subdomaiins

My company wants to mail out postcards asking the recipient to visit a website for add'l information. The url on the postcard will contain a unique subdomain, for tracking purposes.
So for example, John Smith's url will look like johnsmith.mysite.com. Amy Johnson's postcard url will be amyjohnson.mysite.com, etc.
So, 2 questions. One, can url's be setup in this fashion? We would be sending thousands of unique postcards, so manually setting up subdomain's on our web host's admin section isn't realistic. And two, how in asp.net, could I capture just the subdomain?
Thanks
I took an alternate approach to resolving my problem. Instead of trying to handle this entirely programatically, I created a "wildcard DNS" on my domain with my web host (GoDaddy). There's some info for it at this link.
Once my domain was setup to allow wildcard subdomains, I was able to go into my code, and use the following, which allows me to extract the name of the subdomain:
Uri url = new Uri(System.Web.HttpContext.Current.Request.Url.AbsoluteUri);
string subDomain = GetSubDomain(url);
And...
public string GetSubDomain(Uri url)
{
if (url.HostNameType == UriHostNameType.Dns)
{
var host = url.Host;
if (host.Split('.').Length > 2)
{
var lastIndex = host.LastIndexOf(".");
var index = host.LastIndexOf(".", lastIndex - 1);
return host.Substring(0, index);
}
}
return null;
}

RouteCollection Querysting

I am trying to define MapPageRoute on the application global.asax but my problem is that I can not route the specific URL to a physical file with a query string.
For example I want to redirect http://mysite.com/Apple to http://mysite.com/product.aspx?id=95.
What I managed to achieve so far is if a user ask for ./Apple he will be redirected to ./product.aspx but I can not pass the query string.
Looking forward for your comments.
Try this:
if (Page.RouteData.Values["Apple"] != null)
{
int appleID = Convert.ToInt32(Page.RouteData.Values["Apple"]);
Response.Redirect("~/product.aspx?id=" + appleID.ToString(), true);
}

asp.net url minus pagename and querystring

I've been looking around the web for a simple and straight forward solution for the following problem but I cant seem to find anything that suits my needs.
I have an asp.net site with many subdirectories as follows.
http://mysite.com/dir1/subdir1/
http://mysite.com/dir1/subdir2/
http://mysite.com/dir2/
http://mysite.com/dir3/subdir1/
etc...
On each of my sites pages I need to extract the URL to the page minus the pagename and querystring.
So if the page name was http://mysite.com/dir1/subdir2/mypage.aspx?param=5&param2=9
I would need the following http://mysite.com/dir1/subdir2/ I cant find any properties of the httprequest object that make this URL format readily available.
Take a look at this. It should give you everything you need, especially Url.Segments.
This works as well:
System.IO.Path.GetDirectoryName(url).Replace(#"\","/");
You're right, such thing is not ready so you need to make it yourself. One recipe is:
public string GetSubFolderURL()
{
string url = "http";
if (string.Equals(Request.ServerVariables["HTTPS"], "ON", StringComparison.CurrentCultureIgnoreCase))
url += "s";
url += "://";
url += Request.ServerVariables["SERVER_NAME"];
int port;
if (Int32.TryParse(Request.ServerVariables["SERVER_PORT"], out port) && port != 80)
url += ":" + port;
url += Request.ServerVariables["SCRIPT_NAME"];
return url.Substring(0, url.LastIndexOf("/") + 1);
}

Find application root URL without using ~

I need to construct the URL of a page in a String, to send it an email (as part of an email verification system). If i use the ~ symbol to denote the app root, it is taken literally.
The app will be deployed on a server on three different sites (on different ports) and each site can be accessed via 2 different URLs (one for LAn and one for internet).
So hardcoding the URL is out of question. I want to construct the url to verify.aspx in my application
Please help
You need this:
HttpContext.Current.Request.ApplicationPath
It's equivalent to "~" in a URL.
http://msdn.microsoft.com/en-us/library/system.web.httprequest.applicationpath.aspx
Unfortunately none of the methods listed generated the full url starting from http://---.
So i had to extract these from request.url. Something like this
Uri url=HttpContext.Current.Request.Url;
StringBuilder urlString = new StringBuilder();
urlString.Append(url.Scheme);
urlString.Append("://");
urlString.Append(url.Authority);
urlString.Append("/MyDesiredPath");
Can someone spot any potential problems with this?
Try:
HttpRequest req = HttpContext.Current.Request;
string url = req.Url.GetComponents(UriComponents.SchemeAndServer, UriFormat.SafeUnescaped)
+ ((req.ApplicationPath.Length > 1) ? req.ApplicationPath : "");
You need to put the URL as part of your web application's configuration. The web application does not know how it can be reached from the outside world.
E.g. consider a scenario where there's multiple proxies and load balancers in front of your web server... how would the web server know anything but its own IP?
So, you need to configure each instance of your web application by adding the base URL e.g. as an app setting in its web.config.
You can use HttpRequest.RawURL (docs here)property and base your URL on that, but if you are behind any kind of redirection, the RawURL may not reflect the actual URL of your application.
I ended up with this. I take the request url, and use the position of Request.ApplicationRoot to discover the left part of the uri. Should work with applications hosted in a virtual directory "/example" or in the root "/".
private string GetFullUrl(string relativeUrl)
{
if (string.IsNullOrWhiteSpace(relativeUrl))
throw new ArgumentNullException("relativeUrl");
if (!relativeUrl.StartsWith("/"))
throw new ArgumentException("url should start with /", "relativeUrl");
string current = Request.Url.ToString();
string applicationPath = Request.ApplicationPath;
int applicationPathIndex = current.IndexOf(applicationPath, 10, StringComparison.InvariantCultureIgnoreCase);
// should not be possible
if (applicationPathIndex == -1) throw new InvalidOperationException("Unable to derive root path");
string basePath = current.Substring(0, applicationPathIndex);
string fullRoot = string.Concat(
basePath,
(applicationPath == "/") ? string.Empty : applicationPath,
relativeUrl);
return fullRoot;
}
This has always worked for me:
string root = Request.Url.AbsoluteUri.Replace(Request.Url.PathAndQuery, "");

Resources