As at 2013 what is the best practice for multi language multi culture localization of web pages.
Should I store all the translations in a database? for language etc?
Should I build resx files based on a database query or should I simply create resx files?
The application will be used with a browser interface across multiple devices and platforms e.g. Windows/IOS/Android. Are there any additional things I should think about.
best regards
Shrekito
There are a few types of global content you will usually have to deal with:
Application templates such as HTML pages, menus, and emails sent by your app
Dynamic content stored in your database
App templates
Templates are typically handled using your own framework's internationalization (i18n) framework. In the case of asp.net, the framework includes libraries to mark strings and export them to (English) .resx files. This supports a lot of built-in features such as variables, plurals, and developer comments. So unless you're facing some actual blockers, you typically want to go with it instead of building your own i18n framework (eg. in-database, or in-DB with .resx export).
Nowadays, the workflow decisions come when you actually want to get those files localized. In the old days you'd exchange the files with an agency using email. In 2013, web apps have a fast-paced development cycle with multiple releases per week (or per day), so there is a much bigger need for automation. Teams are switching away from using a VCS, FTP or Dropbox as a temporary storage for their files to modern localization (L10n) management platforms like Transifex (discl: I'm the founder) to manage their localization process.
The best way to remove the L10n pain from developers is to integrate your L10n platform with your build tools, so whenever you commit something, the English files are sent automatically to the L10n system, which will detect the changes and notify the right translators. When you're ready to deploy, your integration will pull the fresh translated files automatically.
With Transifex, you can use the Tx Client app with a git-commit hook, your build/CI system or your ol' good deploy script:
tx push --source
tx pull -l de,fr,it --mode=reviewed --minimum-perc=90
Dynamic and External content
This type of content is typically not handled using .resx files since it's easier to manage it in your DB itself. Typically, you want to store this content along with your template content so that translators have a single place to look at everything.
The platform you choose will need to have a modern API to work with. Check out the Transifex API on how these look like.
Other tips and tricks
Choose a platform which has a strong built-in web-based translation editor. Translators need solid features such as Translation Memory, Glossary, Machine Translation to deliver quality and consistent translations.
Follow good i18n practices such as good date formatting and avoiding concatenation.
Love your translators. They're the ones who make it all happen.
Related
Here is what I think my website should be able to provide to user.
Ability to upload file to the system. It should not blocking, user should be able to surf other pages of the website while upload is ongoing. Once upload is done user will get notified about upload.
User should be able to view of his/her uploaded files in website.
Ability to edit files in web browsers using third party APIs
Number of user are going to be around 5000, and all of them might upload files at the same time so performance should not decrease.
Where should I store this files? How to make sure that read and write of files on this directory should handle concurrent user request?
Considering above points. What should be the best way to architect this website?
Are there any existing web framework that play along with this type of architecture like rails, express?
If you want to have the ability to browse the site while a file is uploading, you'll want to use something on the front end that overrides anchor tags and asynchronously fetches the next page - there might be a library or something to accomplish this but it should be easy to implement yourself with jQuery.
To make this easier (and for many other reasons), you'll almost definitely want to structure your site with an MVC (Model View Controller) architecture. Rails is structured this way, as is almost any web framework. It doesn't sound like what you're describing is better suited to Rails over PHP or Python etc so just use whatever language or framework you (or your developers) feel most comfortable with. You might want to do some research into available plugins for editing files (it really depends on what type of files you want to edit and how) and using those to influence your decision on which language to choose as well.
With regards to storing files on your server, any logical system should suffice. Perhaps:
/username/year/month/day/myFile.txt
You'll want to do something to ensure filenames don't clash as well. And obviously you'd want a database storing the information linking files to users.
Here it is my problem:
I developed an MVC site and currently using standard method to publish which will placed files & folders inside the server. All dll files go under BIN folder and so on with the Content and Views go to Content & Views folder.
The problem is this website is an admin panel designed for commercial hardware device (embedded windows OS), so exposing the views and content as a plain text file can't be an option since it will open vulnerability of hijacking/code stealing. Even the device will be packed in a sealed box, anyone who buy it can broke the case and when they are knowing that the device run in windows environment then anything of security breach may happen, including stole the views code to be copied/changed for any purpose.
So I would need to secure the MVC files. I imagine if MVC can be published in secure files, e.g put all the content and views inside dll files.
By default there is an assumption that whoever has access to your views and DLLs is trusted. If they have your files, they can do whatever they want with them.
By the nature of HTML, there is no point in trying to conceal your content files such as javascript and CSS. These files are served to the client regardless, so they are always retrievable.
If you want to put your views into DLLs, you can look into RazorGenerator.
A Custom Tool for Visual Studio that allows processing Razor files at design time instead of runtime, allowing them to be built into an assembly for simpler reuse and distribution.
Please note that what you're doing is known as security through obscurity.
[ ... ] security through obscurity is the use of secrecy of the design or implementation to provide security
Security through obscurity is discouraged and not recommended by standards bodies.
MVC views should never contain business logic, only formatting logic and that is it. Moreover, since C# code is compiled into Intermediate Language (IL), anyone can reverse the process and get the source code.
In such case, you need an obfuscator to mingle the IL to make it difficult to hack, but that this not 100% guaranteed to prevent hackers from reverse engineer you IL (DLL and exe).
The best thing to do is to establish a comprehensive way of testing the admin panel and to facilitate a robust update process, so in case anything went wrong, you push your updates as quickly as possible.
At my office, we have had a long-standing debate about Localization/Globalization and how to handle it. One side pushes for the Resource (.resx) file route built in to ASP.NET, one side pushes for a database driven solution. A third group believes in rolling a custom solution.
Of course, each method has its own unique benefits and disadvantages - and we've discussed it over and over, without ever coming to a real consensus.
So, I pose it to the community: in your experience, which method provides the best mix of the following as the application grows:
Maintainability
Extensibility
Performance / Scalability
In addition to just advice, we'd also be interested in any open source projects which might help to simplify the question, as well. Thanks!
Rick Strahl (An MS MVP) has a great tool kit for managing localization via the DB - offer the ability to update and modify on demand through a controlled environment and does much of the heavy lifting for you. Histoolkit offer the following features:
Data Driven Localization Resource Provider
Database driven Localization lets you store resources in a SQL Server database.
Interactive Web based Resource Adminstration provides a live Web based adminstration for that can edit and update resources while the app is running
Resource Editing Control associates icons with each localizable control and allows jumping directly to the administration form with the current resource id and locale selected.
Resx Import and Export lets you import existing Resx resources, interactively edit them with the data driven provider, then export them back out as Resx resources.
Localization Utilities like a JavaScript Resource Handler, functions to embed localized script values and much more.
He also summarises the issues very well here (Ive pasted some good bits here - not my own work!)
To Resx or not to Resx
The default resource storage mechanism in .NET
uses Resx based resources. Resx refers to the file extension of XML
files that serve as the raw input for resources that are native to
.NET. Although XML is the input storage format that you see in Visual
Studio and the .Resx files, the final resource format is a binary
format (.Resources) that gets compiled into .NET assemblies by the
compiler. These compiled resources can be stored either alongside with
code in binary assemblies or on their own in resource satellite
assemblies whose sole purpose is to provide resources. Typically in
.NET the Invariant culture resources are embedded into the base
assembly with any other cultures housed in satellite assemblies stored
in culture specific sub-directories.
If you’re using Visual Studio
the resource compilation process is pretty much automatic – when you
add a .Resx file to a project VS.NET automatically compiles the
resources and embeds them into assemblies and creates the satellite
assemblies along with the required directory structure for each of the
supported locales. ASP.NET 2.0 expands on this base process by further
automating the resource servicing model and automatically compiling
Resx resources that are found App_GlobalResources and
App_LocalResources and making them available to the application with a
Resource Provider that’s specific to ASP.NET. The resource provider
makes resource access easier and more consistent from within ASP.NET
apps.
The .NET framework itself uses .Resx resources to serve
localized content so it seems only natural that the tools the
framework provides make resource creation tools available to serve
this same model.
Resx works well enough, but it’s not very flexible
when it comes to actually editing resources. The tool support in
Visual Studio is really quite inadequate to support localization
because VS doesn’t provide an easy way to cross reference resources
across multiple locales. And although ASP.NET’s design editor can help
with generating resources initially for all controls on a page – via
the Generate Local Resources Tool – it only works with data in the
default Invariant Culture Resx file.
Resx Resources are also static
– they are after all compiled into an assembly. If you want to make
changes to resources you will need to recompile to see those changes.
ASP.NET 2.0 introduces Global and Local Resources which can be stored
on the server and can be updated dynamically – the ASP.NET compiler
can actually compile them at runtime. However, if you use a
precompiled Web deployment model the resources still end up being
static and cannot be changed at runtime. So once you’re done with
compilation the resources are fixed.
Changing resources at runtime
may not seem like a big deal, but it can be quite handy during the
resource localization process. Wouldn’t it be nice if you could edit
resources at runtime, make a change and then actually see that change
in the UI immediately?
Using Database Resources
This brings me to storing resources in a
database. Databases are by nature more dynamic and you can make
changes to data in a database without having to recompile an
application. In addition, database data is more easily shared among
multiple developers and localizers so it’s easier to make changes to
resources in a team environment.
When you think about resource
editing it’s basically a data entry task – you need to look up
individual resource values, see all the different language variations
and then add and edit the values for each of the different locales.
While all of this could be done with the XML in the Resx files
directly it’s actually much easier to build a front end to a database
than XML files scattered all over the place. A database also gives you
much more flexibility to display the resource data in different views
and makes it easy to do things like batch updates and renames of keys
and values.
The good news is that the resource schemes in .NET are
not fixed and you can extend them. .NET and ASP.NET 2.0 allow you
create custom resource managers (core .NET runtime) and resource
providers (ASP.NET 2.0) to serve resources from anywhere including out
of a database.
As you perhaps know, default method (which is actually industry best practice) for Localizing .Net Applications is using resource files (.resx in this case). If you want to use database, you would have to write your own ResourceManager.
From this, the answer should be obvious: use standard and do not reinvent the wheel.
You might be wondering why Localization via resource files became industry-wide standard. Well, there are many reasons (too many to mention here), most of them regard to Localization process. The key one is, it is painfully hard to update (i.e. fix or install) translations for database driven Localization. Just think of what you need to install it - some SQL script. You know what will happen if you send out this for translation? Or even mistakenly update it? These kind of files are not very safe to work with (and they tend to be very large), so either you would need to create some kind of generator (with resource-like file as an input, which totally bits the purpose...) or you would need to be very careful (and pray that a translator won't break the file).
That is to say, database-driven Localization is sometimes the only sensible way of doing things - this is when you need to implement so-called dynamic Localization, that is allow users to translate things or add their contents in multiple languages.
For static Localization (typical scenario) use resource files.
Localizing user interface should not be stored in database, it is preferable to use the standard resx method because this will give you the flexibility to customize the user interface of front end for each client/deployment, without the need to change the back end or store much data about each client customization in database.
Regarding data (bi-lingual data or multi-lingual data) store them in database and use whatever technique suitable for the context (table per language, or duplicate columns for each language).
using resx is the best approach for some static values that needs not to be manipulated via UI of the app but if your values needs to be updated DB driven would be the best for it. For me its still a case to case basis. But one of the blogs I have seen in the internet made the resx files updateable via user interface.. http://sandblogaspnet.blogspot.com/2009/11/updating-resource-file.html.. hope this would help you.
As all the above are true, I want to add some additional insights.
I tend to use .resx based localisation, when working on "static" projects/websites like Dashboards or other small websites, which are focused on a specific usergroup.
When working on larger and more "dynamic" projects like shops, service-offerings, etc. (esp. when content is localized - not only labels) I like to use database localisation.
When you are developing on larger projects each language is maintained by another person, who is not necessarily in your project (especially in community-projects). Thus maintenance of different languages becomes a real hassle.
On the other side providing users some good/easy UI to update their language is time-consuming as well. So try to find a good path for your project.
I am working on a web forms application that needs to support multiple languages based on a user's preference. Here are some considerations to keep in mind about the needed solution:
I want to avoid using resource files to store the different text translations because I'd like the ability to change them without having to recompile and deploy the application.
Also the translations ideally need to be adminstratable.
It seems its a considerable amount of effort to add this support to an existing application. Any suggestions would be appreciated. Thanks.
Rick Strahl has and awesome solution that is free for personal use, src available here via SVN:
West Wind Web & Ajax Toolkit for ASP.NET
Data Driven Localization Resource Provider
Database driven Localization lets you store resources in a SQL Server database.
Interactive Web based Resource
Adminstration provides a live Web
based adminstration for that can edit
and update resources while the app is
running
Resource Editing Control associates
icons with each localizable control
and allows jumping directly to the
administration form with the current
resource id and locale selected.
Resx Import and Export lets you
import existing Resx resources,
interactively edit them with the data
driven provider, then export them
back out as Resx resources.
Localization Utilities like a
JavaScript Resource Handler,
functions to embed localized script
values and much more.
You'll need languge tables for each supported language. You may need to start with an "en" table as your base language for the app.
Then you can add tables for en-gb, en-us and if required you can even go brand specific using custom cultures like en-gb-MyApp1, en-gb-myApp2
Make extensive use of caching and write a custom resource provider.
I blogged about my design of an E-commerce store where products, pages etc. are stored in database, and saved in multiple languages by storing column data in XML format.
We've recently finished internationalizing a website and are ready to begin the, im sure, arduous task of localization / translation. The site is written in ASP.NET, and im looking for recommendations from people who have done it. CMS or Resource Files? If resource files, is there a recommended management tool? Budget, within reason, is not a factor.
Resource files for sure.
I recently finished a localized website and the tools in ASP.NET for localization are great.
Basically, you have two choices. Implicit or explicity localization. Implicit means the website reads the culture selection of the web browser and serves the correct language and explicit means the user chooses which language they want by clicking on flags or similar technique.
Everything I needed for the project was learned right here:
http://msdn.microsoft.com/en-us/library/fw69ke6f(VS.80).aspx
If there are business users who management content on the site then you'll need to think about a CMS.
If it's purely developer driven then resource files should be fine.