I've collected a (hopefully useful) summary of the ways I've researched to accomplish the subject of this post, as well as the problems I have with them. Please tell me if you've found other ways you like better, especially if they resolve the problems that the methods I mention do not.
Leave connection strings in web.config and use XDT/msdeploy transformation to replace them with settings according to my active build configuration (for example, a web.PublicTest.config file). My problem with this is I merge and bury a few server-specific settings into an otherwise globally identical file with many configuration elements. Additionally, I cannot share connection string definitions among multiple peer-level applications.
Specify a configSource="DeveloperLocalConnectionStrings.config" value for connection strings in web.config, and XDT transform this value to point to one of the multiple environment-specific files in my code-base. My problem with this is I send passwords for all my environments to all destinations (in addition to SVN, of course) and have unused config sections sitting on servers waiting to be accidentally used.
Specific connection strings in the machine.config file rather than web.config. Problem: who the heck expects to find connection strings in the machine.config, and the probability of surprise name collisions as a result is high.
Specify a configSource="LocalConnectionStrings.config", do not transform the value, and edit the project xml to exclude deployment of the connection string config. http://msdn.microsoft.com/en-us/library/ee942158.aspx#can_i_exclude_specific_files_or_folders_from_deployment - It's the best solution I've found to address my needs for a proprietary (non-distributed) web application, but I'm paranoid another team member will come one day and copy the production site to test for some reason, and voila! Production database is now being modified during UAT. (Update: I've found I can't use one-click publish in this scenario, only msdeploy command line with the -skip parameter. Excluding a file as above is the same as setting it to "None" compile action instead of "Content", and results in the package deleting it from the deployment target.)
Wire the deployment package up to prompt for a connection string if it isn't already set (I don't know how to do this yet but I understand it is possible). This will have similar results to #4 above.
Specify a configSource="..\ConnectionStrings.config". Would be great for my needs, since I could share the config among the apps I choose, and there would be nothing machine-specific in my application directory. Unfortunately parent paths are not allowed in this attribute (like they are for 'appSettings file=""' - note also that you can spiffily use file= inside a configSource= reference).
p.s. some of these solutions are discussed here: ASP.Net configuration file -> Connection strings for multiple developers and deployment servers
When using SQL Server, you can also use Integrated Security / SSPI and add the WebServer Computer Login to the Sql Server.
That way you dont have to expose anything in the web.config and you can grant roles to that login like you would to any other DB user.
Though you have to understand the implications and security considerations to be taken, because any malicious code executed as THAT machine will have access to the Sql Server.
with regards
Ole
Use the hostname as key for the connectionstring, that way you can choose the datasource automagically. Make sure the choosing routine is not buggy (change hostname - test!)...
Don't put it in the web.config, write an ini file, that way there is no XML encoding.
Encrypt the password therein, with private/public key (RSA/PGP). Don't ever use cleartext, or a symmetric key, which is just as bad.
Check my following blog post: Protecting asp.net machine keys and connection strings
If you do use Quandary's answer, use a key that's not in the site's folder, just like asp.net does with protected config sections.
We manually approve changes to the web.config that go into staging/production. We use integrated instead of username based where possible, but an option we've used in the later case is to just have placeholders for the username/passwords in SVN.
We've used separate config files in the past, but we have run into other type of issues with web.config modifications, so we have been locking it in a single file lately.
Related
This may seem a bit trivial...but how do you go about transforming the db connection for a nopcommerce app as it is deployed to various environments.
The db connection is set in app_data\datasettings.json.
Normally this type of stuff is handled with web.config transforms.
How do you go about setting up build transforms for different environments (dev, test, prod)?
I am also looking around this topic.
In my humble opinion, the nopCommerce config is a pain, because it makes it really hard to do proper Continuous Integration/Continuous Delivery while keeping secrets safe.
At initial deployment you are greeted with the install page. The problem is that the installation process writes a a bunch of files to on server, including datasettings.json, where the connection string to the DB is hard-coded.
This means that when I deploy nopCommerce to Azure App Service, for deployments after installation, I have to make sure NOT to delete "additional files on the server" or the config will be deleted, since these config files written by the installer, are not in source control.
It is really impractical not to be able to use standards ASP.NET connection strings, environment variables or KeyVault.
To answer your question on how you do transformation on the config file, one possibility is to use a PowerShell script to read, transform, and write the config file directly on the App Service instance. There is an API for that.
https://blogs.msdn.microsoft.com/gabeshapiro/2017/01/01/samples-for-using-the-azure-app-service-kudu-rest-api-to-programmatically-manage-files-in-your-site/
https://github.com/projectkudu/kudu/wiki/REST-API
Alternatively, you can modify the source to read from Web.Config:
Change the connection string of nopCommerce?
I am starting to play around with MVC 6 and I am wondering, with the new config.json structure... are my connection strings safe in the config.json file?
Also, I was watching a tutorial video and I saw the person only put their connection strings in their config.dev.json file, not just the config.json. This will mean the application will not have the connection strings while on the production side, correct? He must have meant to put them in both.
Thanks a lot for the help!
I think the Working with Multiple Environments document sums it up pretty well.
Basically, you can farm secret settings such as connection strings out into different files. These files would then be ignored by your source control system and every developer will have to manually create the file on their system (it might help to add some documentation on how to setup a project from a fresh clone of SCC).
For production, the compile will include the production settings. Typically, these are provided by a build server where they are locked away from developers. I'm not sure if that is totally automatic with MVC core or you have to add some kind of build step to do it, but that is how it is normally done.
If you are worried about storing connection strings in the production environment securely, you can extend the framework with your own configuration provider.
I was wondering if people could post their solution to the ongoing problem of local databases and different connectionstrings among many developers in one project within source control?
More specifically, I'm talking about the problem where a project that is in source control and has many developers with a local database each. Each developer has their own connection string (named instance, default instance, machine name, username, password, etc). Every check in overrides the previous version and pulling the latest version results in using someone else's connection string.
So, people, which solution are you using for this problem? Extra points for explanations on why their solution works, pros, and cons.
EDIT
Keep in mind that this answer shouldn't be targeted only to an enterprise environment where you have full control of the setup. The right solution should work for everyone: enterprise, startup, and open source devs.
Thanks!
To me, your question seems to imply one of two outcomes:
A connection string is specified in the Web.config file that is generic enough to work for all local versions of the database. You've indicated that this isn't an ideal setup in environments where you don't have complete control.
Each developer is required to supply his or her own connection string that is never checked into source control.
A few others have already covered the first scenario. Use localhost and follow a convention for the database name. For option 2, I'd recommend specifying a config source that doesn't get checked into source control:
<configuration>
<connectionStrings configSource="connectionStrings.config"/>
</configuration>
EDIT:
connectionStrings.config
<connectionStrings>
<add name="Name"
providerName="System.Data.ProviderName"
connectionString="Valid Connection String;" />
</connectionStrings>
From: http://msdn.microsoft.com/en-us/library/ms254494(v=vs.80).aspx
connectionStrings.config would be a file in the root of the project that you specifically excluded from source control. Each developer would be required to provide this file when working locally. Your production connection string could be substituted via a Web.config transformation on build / deployment.
All of our development stations are setup somewhat the same.
We use integrated authentication to the database, so no user/password needs to be stored.
Everyone uses the default instance.
Since they are all local databases, you can user localhost.
Same database name
So a connection string might end up being:
Data Source=localhost;Initial Catalog=TheDatabaseName;Integrated Security=SSPI;
And everyone uses that.
We have a test database that runs on it's own server. It is just test data actually replicated from the live server every night.
All developers use this for testing. It is true data but non critical as it is a day old.
We store the connection strings in the web.config of course. I have a function to get the connection string from the web.config and I just look if it is localhost return the test connection string. if not then return the live connection string.
This works for us. And we just don't use local databases.
I've seen various methods used.
i) You only maintain a live connection string and developers make modifications to the relevant module and never check it in.
ii) An app.config is maintained with the connection strings and it is a shared file as far as source control is concerned.
iii) In debug config, connection string details are passed in the command line from the IDE.
I want to do some unit testing on one of my projects. This is a web project, and there will only be one copy of this program running aside from development copies.
I want to write some unit tests that will use the web.config. I understand that ordinarily, a tester would stub out this external dependency because he wants to test the code without the test depending on the web.config holding certain values.
However, the web.config in my project is supposed to always hold certain values and I want to have a unit test that will fail if they are set to invalid values. For example, one of the values is a SQL connection string.
I want to write a test that will read the connection string from the web.config. I envision that the test could connect to a server with the connection string and perhaps perform a very simple command like SELECT system_user;. If the command executes successfully and returns something the test passes. Otherwise, it fails. I want the connection string to be read from the web.config in the project I'm testing.
Of course, the ConfigurationManager will not ordinarily look for a web.config in another project. I could manually copy the web.config from the original project to the test project, but I would have to do that before every test and there is no way I could count on anyone else to do that.
How do I make my test project read the web.config from another project?
It sounds like you are trying to validate settings in web.config, which is a deployment-level concern and is different from unit testing.
Unit testing tells you that your core logic is performing as expected; deployment verification tells you that the application was installed and configured properly and is safe to use. Unit tests are meaningful to developers, deployment verification is meaningful to the end user or administrator that is deploying the app.
In situation like this I like to build a "system console" into my apps. This console contains a number of self-diagnostic checks such as:
Ensuring the connection string(s) are configured properly
Ensuring that any 3rd party services are available and functioning
Ensuring that all configuration settings are valid and won't cause runtime errors (e.g. paths exist, the web user account has read/write access where needed, etc)
I strongly recommend you consider separating this sort of configuration and deployment verification from your unit test suite. Not only will it simplify your work (because you won't have to load a config file from another project) but it's also the sort of tool that customers really, really like :)
You can load and explore other config files with the ConfigurationManager.OpenXXX() methods.
The WebConfigurationManager class specifically has a method for opening web.config files, and the documentation page I linked to has some more code examples. Once you have your configuration object loaded, you can explore it for sections and keys.
var cfm = new ConfigurationFileMap("path/to/web.config");
var config = WebConfigurationManager.OpenMappedWebConfiguration(cfm);
I asked a similar question that you might want to check out:
How do I test that all my expected web.config settings have been defined?
I ended up getting it working but the annoying part is my source control constantly locking the config file that is copied over. You can also rename the web.config to app.config so that it will compile into a non-web project.
It sounds like you're trying to squash a mosquito with a sledgehammer. Why not do this manually, as part of the deployment checklist; have a task to manually confirm the connectionString.
Or if you want to automate it, write a program to check the connectionString, attach it to your Continuous Integration server (assuming you have one) and fail the build if the connectionString is wrong.
Use Unit Tests for what they're intended for testing code, not configuration.
If you want to use original web.config file from your website to your Unit Testing project without copying it then you can Modify VS Local-Test-Settings.
Here is a step by step procedure to use ASP.net website configuration file under Unit Testing project. Follow the link http://forums.asp.net/t/1454799.aspx/1
There is a commercial tool called CheckMyConfig for validating .NET config files, which identifies settings within a given config file, and attempts to validate them.
Possible setting types include database connection strings, files, folders, IP addresses, hostnames and URLs.
The tool allows you to perform a number of checks including opening database connections, accessing folders, requesting a particular URL etc.
There is a standalone version, but the tool also has Visual Studio integration and a simple API that you can use to embed the tool within your own apps, in order
to perform a config 'sanity check' at app startup time.
In your unit testing project, add an app.config file and add the settings from the web.config file that you would like to use for your tests.
I have a team of three developers, two of whom use a standard local test database, one of whom uses his own database and there is also a server environment with a production database and a testing database.
This amounts to multiple connection strings required.
The web.config file periodically gets updated and keeps having to be changed by each developer when a source control update is performed, as well as the fact that sometimes a developer accidentally checks in his personal web.config file change with his connection string, which temporarily interrupts us after running a subsequent update from source control.
What can I change so that the web.config file can be committed/updated as often as is necessary and not result in broken connection strings for other developers or require modifications before it can be uploaded to the testing or production environments?
You might want to look at replacing the ConnectionStrings section of the Web.Config with a Config Source file, commit the LIVE set of connection strings into source control, but not include them in the actual solution.
Information on using the configSource attribute can be found in the section "Using External Configuration Files" in the document:
Connection Strings and Configuration Files
This has the advantage that developers could have their own settings that don't get checked into the main Web.Config. The potential downside is that it's yet another file that you have to worry about, and depending on how you're getting the latest version out of source control might not help (i.e. if you go to your SC client and say "Get latest" that would overwrite the file, whereas if you are in Visual Studio, right click on the solution/project and select "Get Latest" that will only get files in the projects).
For the question regarding the testing and production environments, you can make use of the Visual Studio Web Deployment Projects. With that, you can define sections in the web.config that will be replaced when you build the deployment package for the test / prod server.
You could use the Enterprise Library Data Access Block to handle your database connections. You can then define as many connection strings as you want and simply change the use key to whichever one is required. See http://www.devx.com/dotnet/Article/30910 for more information.