This may seem a bit trivial...but how do you go about transforming the db connection for a nopcommerce app as it is deployed to various environments.
The db connection is set in app_data\datasettings.json.
Normally this type of stuff is handled with web.config transforms.
How do you go about setting up build transforms for different environments (dev, test, prod)?
I am also looking around this topic.
In my humble opinion, the nopCommerce config is a pain, because it makes it really hard to do proper Continuous Integration/Continuous Delivery while keeping secrets safe.
At initial deployment you are greeted with the install page. The problem is that the installation process writes a a bunch of files to on server, including datasettings.json, where the connection string to the DB is hard-coded.
This means that when I deploy nopCommerce to Azure App Service, for deployments after installation, I have to make sure NOT to delete "additional files on the server" or the config will be deleted, since these config files written by the installer, are not in source control.
It is really impractical not to be able to use standards ASP.NET connection strings, environment variables or KeyVault.
To answer your question on how you do transformation on the config file, one possibility is to use a PowerShell script to read, transform, and write the config file directly on the App Service instance. There is an API for that.
https://blogs.msdn.microsoft.com/gabeshapiro/2017/01/01/samples-for-using-the-azure-app-service-kudu-rest-api-to-programmatically-manage-files-in-your-site/
https://github.com/projectkudu/kudu/wiki/REST-API
Alternatively, you can modify the source to read from Web.Config:
Change the connection string of nopCommerce?
Related
I am starting to play around with MVC 6 and I am wondering, with the new config.json structure... are my connection strings safe in the config.json file?
Also, I was watching a tutorial video and I saw the person only put their connection strings in their config.dev.json file, not just the config.json. This will mean the application will not have the connection strings while on the production side, correct? He must have meant to put them in both.
Thanks a lot for the help!
I think the Working with Multiple Environments document sums it up pretty well.
Basically, you can farm secret settings such as connection strings out into different files. These files would then be ignored by your source control system and every developer will have to manually create the file on their system (it might help to add some documentation on how to setup a project from a fresh clone of SCC).
For production, the compile will include the production settings. Typically, these are provided by a build server where they are locked away from developers. I'm not sure if that is totally automatic with MVC core or you have to add some kind of build step to do it, but that is how it is normally done.
If you are worried about storing connection strings in the production environment securely, you can extend the framework with your own configuration provider.
I've collected a (hopefully useful) summary of the ways I've researched to accomplish the subject of this post, as well as the problems I have with them. Please tell me if you've found other ways you like better, especially if they resolve the problems that the methods I mention do not.
Leave connection strings in web.config and use XDT/msdeploy transformation to replace them with settings according to my active build configuration (for example, a web.PublicTest.config file). My problem with this is I merge and bury a few server-specific settings into an otherwise globally identical file with many configuration elements. Additionally, I cannot share connection string definitions among multiple peer-level applications.
Specify a configSource="DeveloperLocalConnectionStrings.config" value for connection strings in web.config, and XDT transform this value to point to one of the multiple environment-specific files in my code-base. My problem with this is I send passwords for all my environments to all destinations (in addition to SVN, of course) and have unused config sections sitting on servers waiting to be accidentally used.
Specific connection strings in the machine.config file rather than web.config. Problem: who the heck expects to find connection strings in the machine.config, and the probability of surprise name collisions as a result is high.
Specify a configSource="LocalConnectionStrings.config", do not transform the value, and edit the project xml to exclude deployment of the connection string config. http://msdn.microsoft.com/en-us/library/ee942158.aspx#can_i_exclude_specific_files_or_folders_from_deployment - It's the best solution I've found to address my needs for a proprietary (non-distributed) web application, but I'm paranoid another team member will come one day and copy the production site to test for some reason, and voila! Production database is now being modified during UAT. (Update: I've found I can't use one-click publish in this scenario, only msdeploy command line with the -skip parameter. Excluding a file as above is the same as setting it to "None" compile action instead of "Content", and results in the package deleting it from the deployment target.)
Wire the deployment package up to prompt for a connection string if it isn't already set (I don't know how to do this yet but I understand it is possible). This will have similar results to #4 above.
Specify a configSource="..\ConnectionStrings.config". Would be great for my needs, since I could share the config among the apps I choose, and there would be nothing machine-specific in my application directory. Unfortunately parent paths are not allowed in this attribute (like they are for 'appSettings file=""' - note also that you can spiffily use file= inside a configSource= reference).
p.s. some of these solutions are discussed here: ASP.Net configuration file -> Connection strings for multiple developers and deployment servers
When using SQL Server, you can also use Integrated Security / SSPI and add the WebServer Computer Login to the Sql Server.
That way you dont have to expose anything in the web.config and you can grant roles to that login like you would to any other DB user.
Though you have to understand the implications and security considerations to be taken, because any malicious code executed as THAT machine will have access to the Sql Server.
with regards
Ole
Use the hostname as key for the connectionstring, that way you can choose the datasource automagically. Make sure the choosing routine is not buggy (change hostname - test!)...
Don't put it in the web.config, write an ini file, that way there is no XML encoding.
Encrypt the password therein, with private/public key (RSA/PGP). Don't ever use cleartext, or a symmetric key, which is just as bad.
Check my following blog post: Protecting asp.net machine keys and connection strings
If you do use Quandary's answer, use a key that's not in the site's folder, just like asp.net does with protected config sections.
We manually approve changes to the web.config that go into staging/production. We use integrated instead of username based where possible, but an option we've used in the later case is to just have placeholders for the username/passwords in SVN.
We've used separate config files in the past, but we have run into other type of issues with web.config modifications, so we have been locking it in a single file lately.
Currently I can easily setup Web.config transform based on build configuration, e.g. use connectionString=server;.. for Debug and connectionString=./SQLExpress;.. for Release.
But is it possible to do some Web.config transformation basing on web publish profile? I.e. use connectionString=server1;.. for profile Server1 and connectionString=server2;.. for Server2 ?
We keep all machine/profile specific configuration in separate config files, then use configSource to include them like so...
<connectionStrings configSource="cstrings.config"/>
This way Web.config is the same and doesn't require any transformations. We do this for connection strings, smtp settings and app settings.
We version control Web.config and "machine specific" files such as cstrings.config.production, cstrings.config.staging, etc.
Once you have this structure it's easy to generate images for different profiles. We have deployment scripts on each machine that read an environment variable and deploy appropriately. For example, the staging server build script copies cstrings.config.staging to cstrings.config, etc.
There could be a slightly different way to do this.
On your production servers create a dummy entry, for customdb, in the c:\windows\system32\drivers\etc\hosts file on each of the production machines. Each one pointing to the database that you want that machine to use. Then you only have to point to the connectionString=customdb; for all your production servers.
Only downside of this would be that you would need access to the hosts file and it would require you to use a db.
Hope this helps
I believe that the publishing profiles are independent of the build profiles, which is a bit of a shame, as you could easily accidentally deploy a debug configuration to your production servers.
However, if you're using MSDeploy, there are ways to modify the web.config in there. See MSDeploy - Changing Connection string parameter after deploying the package for more details.
I have a team of three developers, two of whom use a standard local test database, one of whom uses his own database and there is also a server environment with a production database and a testing database.
This amounts to multiple connection strings required.
The web.config file periodically gets updated and keeps having to be changed by each developer when a source control update is performed, as well as the fact that sometimes a developer accidentally checks in his personal web.config file change with his connection string, which temporarily interrupts us after running a subsequent update from source control.
What can I change so that the web.config file can be committed/updated as often as is necessary and not result in broken connection strings for other developers or require modifications before it can be uploaded to the testing or production environments?
You might want to look at replacing the ConnectionStrings section of the Web.Config with a Config Source file, commit the LIVE set of connection strings into source control, but not include them in the actual solution.
Information on using the configSource attribute can be found in the section "Using External Configuration Files" in the document:
Connection Strings and Configuration Files
This has the advantage that developers could have their own settings that don't get checked into the main Web.Config. The potential downside is that it's yet another file that you have to worry about, and depending on how you're getting the latest version out of source control might not help (i.e. if you go to your SC client and say "Get latest" that would overwrite the file, whereas if you are in Visual Studio, right click on the solution/project and select "Get Latest" that will only get files in the projects).
For the question regarding the testing and production environments, you can make use of the Visual Studio Web Deployment Projects. With that, you can define sections in the web.config that will be replaced when you build the deployment package for the test / prod server.
You could use the Enterprise Library Data Access Block to handle your database connections. You can then define as many connection strings as you want and simply change the use key to whichever one is required. See http://www.devx.com/dotnet/Article/30910 for more information.
Right now our test and production databases are on the same server, but with different names. Deploying has meant editing Web.config to change all the connection strings for the correct database. A step which I forget all too frequently...
We've finally created a new database server for testing, and I'm moving the databases over... but now the server will be different and we'll still need to deal with connection string issues.
I was thinking of managing it via a hosts file, but the thought of switching that on my desktop machine whenever I need to test against production data seems cumbersome at best.
So I'm just wondering if there's a better way out there. Something that would build with a "production" web config for deployment would be ideal...
Use a Web Deployment Project and update the wdproj file (it's just an MSBuild file) with some post build tasks to output the correct .config file. I keep a web.config and web.release.config then use this in the wdproj file:
<Target Name="AfterBuild">
<Copy Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' " SourceFiles="$(SourceWebPhysicalPath)\web.release.config" DestinationFiles="$(OutputPath)\web.config" />
<Delete Files="$(OutputPath)\web.release.config" />
</Target>
More information
A simpler solution some like is using configSource property of appSettings and connectionStrings and then never overwriting that file on the production server.
I usually have three separate web configs: one for my development machine, one for QA, and one for production. The development one connects to my local SQL database (which is firewalled from outside) and it is the default web.config. The others are named web-prod.config and web-qa.config. After publishing I delete the two that I don't need and rename the correct one to web.config. If I forget, the app breaks the first time it attempts to access the database, since the default config references one it can't get to.
Since IIS refuses to serve up a file named .config, I make sure they all end in .config instead of say web.config-prod or web.config-qa.
Here's another thing you can try:
Using SQL Server Configuration Manager, make a db Alias for your development database so that the web.config file can be the same on both your development box and the production server.
I create a database alias on each server to point to the database. I then use this alias in my web.config files. If I need to change which database the application points to, then I change the alias and not the web.config.
For SQL Server, go to SQL Server Configuration Manager > SQL Native Client Configuration > Aliases > Create New Alias.
You can do the same thing with Oracle with the tnsnames file.
have environment folders with separate configs for each environment
deploy out the correct one for the environment
I did this so often, I made the web.config on the production server read-only.
I've been in a few places now that store them in the registry.
There's probably more elaborate ways to do it now but a lot of code I've worked on with a 1.0/1.1 heritage store the strings in the registry.
The registry has a few advantages
It keeps people from deploying the code to the wrong places since machines not configured properly will lack the keys
It eliminates the problem wherein a developer will accidentally package a web.config file with the development connection strings in it (followed by a frantic phone call in the middle of the night wherein it is revealed that the late night sysadmin did not back up the previous web.config and the developer does not know or recall the production strings)
It limits the possibility of a hacker being able to get the connection string by fetching the web.config off of the machine. Plus the registry has more levels of security than the filesystem.
We drive our a deployments from our CI server. We usualy have a seperate file for each location and have the CI server switch to the appropriate config depending on the arguments passed ot it. All the file editing is done in NAnt scripts, so develops can run the sam build on their machine to get their own settings.
I'll put my connection strings in the machine.config on our QA and Production boxes. I'll keep them in the web.config on my dev box for flexibility, though. Then, I'll use a web deployment project to overwrite my dev connection strings with nothing (no connection strings) when deploying to QA. Therefore the QA site relies on the connection strings in machine.config. I still deploy to Production manually to make sure everything succeeds. I do this by manually copying everything from QA (except for web.config) to production.
This kind of task is exactly what build events are designed to address. Think of building as building for a specific target, any target specific configuration should be done there. (with the normal caveat that there are always some exceptions to the rule)
I've recently been leaning towards config manipulation on the continuous integration server. That's because we've had problems with multiple web.config, web.qa.config, web.production.config keeping the 95% of the file that should be the same in sync.
In a nutshell: there's only the one web.config in source control and it's the development configuration (debug friendly, local db, etc.). The build server does the compile, then a deploy to the canary site, then the package for release candidate.
We're using nant, so it's the .build file that has xmlpoke to set debug="false", alter connection strings, and whatever else needs to change in the canary copy and the packaging copy of the web.config.
The build machine's deploy is called "canary" because it's the first thing to die if there's a problem.