Is there a way to get a web.config transformation in Visual Studio 2015+ to include the contents of an externally-referenced secrets file as described here?
This works well when developing locally, and doesn't include the file in source control, but when I go to deploy the web app to the server, I want it to include the username and password for another web service which it connects to. If I enter the settings manually in IIS on the server, they are lost each time I publish the app.
If I omit the keys in the appSettings block in my web.config (so that they are only referred to in the secrets file), the manually-entered settings in IIS on the server are removed completely whenever I publish the app.
Being able to refer to certain things in an external file for the sake of better security and not checking in passwords to source control isn't turning out to be a very good idea IMHO because my deployment is now a nightmare. I don't want to manually enter the passwords in the web.config file on the server after every deployment.
I'd rather not look into encryption, either, because I would have to do that for each server I deploy to so that the relevant machine key is used for each web.config file.
I've only recently thought about removing this password from source control, in response to a recent push to improve security practises at work - which I well understand and agree with - but I can also see why security is so poorly considered because the life of the ordinary developer becomes extremely unpleasant if there the tools available don't make it easy.
Surely there's a way without resorting to encryption?
Thanks.
Without a CI system I think your best option is probably a pre/post build action that executes a script?
I'd suggest you replace the actual values with tokens for your sensitive web.config values (something unique/easy to find like MY_PRODUCT_DATABASE_PASSWORD etc). Your web.config can then be checked into source control safely.
In VS you can add a build action to run a custom powershell or exe to basically perform a find-and-replace on the tokens with actual values before you zip & deploy as normal.
Exactly how/where you store the real values and how the script works is up to you. you could easily find a file on your deployment machine or a row in some database based on data passed to the script/exe from vs or from data within web.config itself (or embedded as a comment in web.config even).
Here's details of the variables available from vs you could pass to your exe or script in a build action: https://msdn.microsoft.com/en-us/library/42x5kfw4.aspx
It you wanted to use PowerShell you could read/replace/write values to a web.config(or any text file) like this answer: How can I replace every occurrence of a String in a file with PowerShell?
Following the Microsoft Docs example you've linked to, putting this XDT transformation inside Web.Release.config should do the trick:
<appSettings file="..\..\AppSettingsSecrets.config" xdt:Transform="SetAttributes">
<add key="mailAccount" xdt:Locator="Match(key)" xdt:Transform="Remove" />
<add key="mailPassword" xdt:Locator="Match(key)" xdt:Transform="Remove" />
<add key="TwilioSid" xdt:Locator="Match(key)" xdt:Transform="Remove" />
<add key="TwilioToken" xdt:Locator="Match(key)" xdt:Transform="Remove" />
<add key="TwilioFromPhone" xdt:Locator="Match(key)" xdt:Transform="Remove" />
<add key="GoogClientID" xdt:Locator="Match(key)" xdt:Transform="Remove" />
<add key="GoogClientSecret" xdt:Locator="Match(key)" xdt:Transform="Remove" />
</appSettings>
You dont need xdt:Locator attribute on <appSettings> element itself, because there is only one appSettings.
Edit: I've misunderstood the original question. The goal is to include contents of referenced file, which is not possible using XDT. There has to be another way.
We are storing session data in the database, ideally each developer wants a session database on their own pcs, or possibly 1 developer might use inproc, another a mssql connection, another Oracle (if we get it working)
Is this possible? Can you access the session key in code and change it in the application startup? Or is there a file which could be merged with the web.config file that wouldn't get checked in?
Or option C which is easier but which I haven't thought of :-)
thanks
(Edit) Just found this which goes into this in some detail
developer specific app.config/web.config files in Visual Studio
(Answer). This came from a mix of Andrew Barber in the comments and the above
(1) In the web.config have this
<sessionState configSource="SystemWeb.config" />
(2) Make a file called SystemWebDefault.config which holds something like this:
<sessionState mode="SQLServer" allowCustomSqlDatabase="true" etc
(3) Each developer has to copy the default into a file called SystemWeb.config, changing it to suit themselves. This file should be explicitly ignored in subversion or whatever source control system you use.
(4) The build box needs a copy step
Something that might do what you want is NConfig (NuGet link). I've used it on projects to let multiple developers in our team have different connection strings for their local DB copies. It lets you define a machine specific config file to replace your default config values. For example you can define a {MachineName}.Custom.config that might look like this:
<?xml version="1.0"?>
<configuration>
<connectionStrings>
<add name="main" connectionString="data source=(local)\SQLExpress;uid=username;pwd=password;initial catalog=mydb" providerName="System.Data.SqlClient" />
</connectionStrings>
</configuration>
I've never tried it with the session settings, but I can't find anything that says it won't work.
I am getting the following error while debugging my visual studio 2010 website:
An attempt to attach an auto-named database for file C:\Users...\Desktop\Dpp2012New\App_Data\dppdatabase.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Data.SqlClient.SqlException: An attempt to attach an auto-named database for file C:\Users...\Desktop\Dpp2012New\App_Data\dppdatabase.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
Here is my connection string from my web.config:
<connectionStrings>
<add name="ApplicationServices"
connectionString="data source=.\SQLEXPRESS;Integrated Security=SSPI;AttachDBFilename=|DataDirectory|\aspnetdb.mdf;User Instance=true"
providerName="System.Data.SqlClient"/>
<add name="ConnectionString"
connectionString="Data Source=.\SQLEXPRESS;AttachDbFilename=|DataDirectory|\dppdatabase.mdf;Integrated Security=SSPI"
providerName="System.Data.SqlClient"/>
</connectionStrings>
And I access it from my website as:
Dim connectionString As String = ConfigurationManager.ConnectionStrings("ConnectionString").ConnectionString
The stacktrace shows the error line as:
Dim conn As New SqlConnection(connectionString)
Dim dr As SqlDataReader
conn.Open() 'This is the error line as per stacktrace
I have given the needed permissions to the above folder so it can not be the " or specified file cannot be opened" issue. If we look at all the posts related to the same error in this forum, clearly the profoundness of this error can be found out. However, none of the solutions solves my issue.
Some of the resources which I have tried are:
http://weblogs.asp.net/scottgu/archive/2005/08/25/423703.aspx
http://blogs.msdn.com/b/webdevelopertips/archive/2010/05/06/tip-106-did-you-know-how-to-create-the-aspnetdb-mdf-file.aspx
http://forums.asp.net/t/1033225.aspx
I eagerly await a solution.
I had this problem also and it was a pain. I configured my connection string and managed to solve the problem. In the connection string I replaced the value |DataDirectory|\dbfilename.mdf for AttachDbFilename property, with the path to the file. |DataDirectory| can only be used if the database file is in the App_Data folder inside same project.
So changing the AttachDbFilename property to the direct path of the mdf file, fixed my issue.
AttachDbFilename=C:\MyApp\App\DAL\db.mdf
I hope this works for you.
Try to create your connection string as below:
<add name="ECommerce" connectionString="Data Source=(localdb)\v11.0;Initial Catalog=C:\USERS\dL\DESKTOP\DATABASE\MYSHOP.MDF;Integrated Security=True;Connect Timeout=15;Encrypt=False;TrustServerCertificate=False" providerName="System.Data.SqlClient"/>
It's working for me.
<add name="Connections" connectionString="Data Source=.\SQLEXPRESS;AttachDbFilename=|DataDirectory|\Database.mdf;Integrated Security=True;User Instance=True" providerName="System.Data.SqlClient"/>
I had a similar error, but using a connection configured in code, rather than a config file. The solution for me was simply to add the "Initial Catalog=MyDatabase" part. My code now looks like...
DbConnection connection = new SqlConnection(String.Format(#"Data Source=.\SQLEXPRESS; Integrated Security=SSPI; AttachDbFilename={0}; User Instance=True; Initial Catalog=IPDatabase;", Location));
Location is a full path to an mdb file, that does not have to exist yet.
One of the most annoying errors when developing web apps in .Net. I had this issue on a project I was doing a year ago.
This is what worked for me: I've logged in with sa account to SQL Management Studio and attached the database to the object explorer (Databases -> Right Click -> Attach).
Then I've opened Security->Logins->[myWindowsUsername] and edited User Mappings tab, giving dbowner rights to the attached database.
I'm sure you can do those things without Management Studio, with console commands, but I'm not that good with T-SQL and can't give you advice on that.
Mine was an EF code-first project and it comes up when the file has been deleted. Running Update-Database from the Package Manager Console makes the DB and its fine.
The error occurred because the AttachDbFilename parameter is set to an absolute path, which points to the wrong file location.
It's important to know, that the database .mdf file itself will be copied to the build target folder on every build by default or, if configured to Copy if newer using the file Property Explorer, only if the file version has changed (it is recommended to use Copy if newer to prevent the database from being overwritten on every build).
To fix the issue, the AttachDbFilename parameter should point to the current execution folder. It's best to use the Visual Studio environment variable |DataDirectory| to define the relative path.
A fixed connection string could look as follows:
"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\MyDatabase.mdf;Integrated Security=True;Asynchronous Processing=True;User Instance=False;Context Connection=False"
check your web-config file, there may be more than one connection string having same name
Please check the path for your database on the connection string. I have the same error message and I later found out that the problem is a misspelled path.
I've been trying mdf portability in my application, so I've been using
public DbContext IoDatabase()
{
var directory = System.IO.Directory.GetCurrentDirectory();
var connectionString = #"Data Source=(localdb)\mssqllocaldb;AttachDbFilename=" + directory + "\\App_Data\\ITIS.mdf;Integrated Security=True;Connect Timeout=30;";
return new NerdDinners(connectionString);
}
with nerdDinners, I can't remember where I found it, being
public class NerdDinners : DbContext
{
public NerdDinners(string connString)
{
Database.Connection.ConnectionString = connString;
}
}
Then I can use the return type as a DbContext.
The main thing I ran into was that GetCurrentDirectory() gets the compiled file path, not the project file path, so deployment should be set to GetCurrentDirectory and the mdf should be copied on compile, not reference the actual App_Data. Include your mdf in your project and go properties on it. Set the copy to output directory.
I had the same problem with EF. The absolute path solution worked. But it is not a nice solution if you want to move your program to a new location. For me this was the problem.
VS searches the mdf file in the debug folder
There is an .mdf file in the project folder, sometime VS sync the two
files
For some reason the .mdf file was not synced into Debug
My workaround:
Copy the .mdf and .ldf file to a temp folder
Make a connection for it in vs/server explorer
When you add the ADO.NET use the new connection
VS offers to copy the .mdf file into the project folder, accept it
Now the .mdf file will appear in the debug folder as well
Delete the connection for the temp folder, create a new one to the project
folder
Just change your path with a new location.
How you will get a path?
Go to database >right click > go to the property > on right-side panel data source is there, copy that data source location and update into a web.config file (the only path has to take care and to look for in the AttachDbFilename in the database).
Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=E:\manish_data\project_practice\sqlbasedprojects\sqlbasedproject\realease\mainproject\App_Data\cruddatabase.mdf;
I also got the same error and I fixed the following (I use Visual Studio 2019):
Click to Debug > [project name] debug properties, select the build item in the left sidebar. On the right sidebar, navigate to the output path entry and change bin\debug to .\
Error was fixed for me after I changed this
ConfigurationManager.ConnectionStrings[0].ConnectionString
to this
ConfigurationManager.ConnectionStrings["myconnection"].ConnectionString
I have changed the installation folder from program files to C directory while I am installing the app. then my problem is solved
well I think reason is that in your database directory, contains a .mdf file which has the same name. So best thing is to give the full path you can just replace the AttachDbFilename. just simply get the fullpath of the database file and assign it into the AttachDbFilename which is in app.config file.
In most cases, the unit test project is separated from the main project.
This causes the generated connection string to be invalid as it fails to find an app_data folder, if your db is local to the solution.
You can just replace the |DataDirectory| with a relative path to your db in your main project if you need to connect to it from your unit test project.
what I've found so far to solve the problem, if you create mdf file in the below way, it would work just fine for visual studio.
Right-click on the data connection in the server Explorar->add connection->Select Microsoft SQL Server Database File and choose database name and select okay. Then this prob doesn't arise.
Follow the video:
youtube link
Looks like there are lots of causes for this...Here was mine.
The clue was on the last line of the error message...
System.Data.SqlClient.SqlException: 'An attempt to attach an
auto-named database for file Q:\Folder\FileName.mdf failed. A database
with the same name exists, or specified file cannot be opened, or it
is located on UNC share.'
...or it is located on UNC share.
In trying to replicate a production environment...
I originally had physical drive Q:. After my dev machine tanked..I rebuilt and just created a mapped drive.
Started getting this error...and then went back and just created another Q: partition and the problem was solved.
I had a similar problem after i moved my db to a folder within the same project.
All i had to do finally was to to properties of the database, copy the path listed under 'identity' and replaced the path at 'AttachDbFilename='.
Worked just fine.
Try this, it works for me
try {
String ConnectionString = #"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\App_Data\Test.mdf;Integrated Security=True;Connect Timeout=15;Encrypt=False;TrustServerCertificate=True";
SqlConnection connection = new SqlConnection(ConnectionString);
connection.Open();
MessageBox.Show("Connection is opened.!!");
connection.Close();
} catch(Exception ex)
{
MessageBox.Show(ex.Message);
}
TLDR: May need to make the change as mentioned by HasanG in multiple files, such as App.config and Settings.Designer in my case.
To add on to HasanG's answer, there may be multiple locations that you need to change the path for your ConnectionString. In my case, I wasn't getting data persistence between two runs of the debugger and changing the ConnectionString defined in my Settings.Designer file fixed this. However, after closing and re-opening Visual Studio and running the debugger again I found that I wasn't getting persistence after Visual Studio was closing. I would either end up with an empty DataGridView or would get an error at build-time stating the following:
*An attempt to attach an auto-named database for file <database file path in debug folder> failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.*
I then searched my entire project solution for instances of that file path and found that it was also being used in the App.config file. Once I changed the path in that file as well as in the Settings.Designer file I was able to have data persistence in my DataGridView between multiple runs of the debugger, and across multiple runs of Visual Studio itself. Hope this helps someone! I pulled out a lot of hair over this one.
EDIT: Make that three files, the Settings.Settings file had to be updated as well.
In .net core 5 for create Db.mdf file in Data folder :
connection string in appsettings.json :
"DefaultConnection": "Data Source=(LocalDB)\\MSSQLLocalDB;AttachDbFilename=projectPath\\Data\\Db.mdf;Integrated Security=True;Connect Timeout=30"
options.UseSqlServer in startup.cs :
var projectPath = Directory.GetCurrentDirectory();
services.AddDbContext(options =>
options.UseSqlServer(
Configuration.GetConnectionString("DefaultConnection").Replace("projectPath", projectPath)));
I had the same problem,and i think i figured it out.in the connection string make sure that the connection string looks like this:
<connectionStrings>
<add name="ApplicationServices"
connectionString="data source=.\SQLEXPRESS;
Integrated Security=SSPI;
AttachDBFilename=|DataDirectory|\aspnetdb.mdf;
User Instance=true"
providerName="System.Data.SqlClient" />
<add
name="NorthwindConnectionString1"
connectionString="Data Source=localhost;
Initial Catalog=Northwind;
Integrated Security=True"
providerName="System.Data.SqlClient" />
</connectionStrings>
Now, the first
<add
name="ApplicationServices"
connectionString="data source=.\SQLEXPRESS;
Integrated Security=SSPI;
AttachDBFilename=|DataDirectory|\aspnetdb.mdf;
User Instance=true"
providerName="System.Data.SqlClient" />
is what appears by default in the web.config. Leave it unchanged,and add another connectionstring
<add
name="NorthwindConnectionString1"
connectionString="Data Source=localhost;
Initial Catalog=Northwind;
Integrated Security=True"
providerName="System.Data.SqlClient" />
with your parameters. This is working for me. Hope it helps.
I usually put my connection string to a separate file, adding something like this in web.config:
<connectionStrings configSource="WebConnection.config" />
I've just installed VS 2012 and it automatically picked up my existing publish settings.
However, when I do a webpublish it now adds two connections strings by itself, so my web.config on the deployment target now looks like that:
<connectionStrings configSource="WebConnection.config">
<add name="EF.Model.DbContext" connectionString="EF.Model.DbContext_ConnectionString" providerName="System.Data.SqlClient" />
<add name="Migrations.Db.MigrationDb" connectionString="Migrations.Db.MigrationDb_ConnectionString" providerName="System.Data.SqlClient" />
</connectionStrings>
certainly, that produces an error (node contents must be empty when using configSource).
I noticed, that in newly generated .pubxml files (where publish settings are now stored) there are following lines:
<ItemGroup>
<_ConnectionStringsToInsert Include="EF.Model.DbContext" />
<_ConnectionStringsToInsert Include="Migrations.Db.MigrationDb" />
</ItemGroup>
How can I remove them? :) If I delete them from file, Web-publish dialog adds them anytime I edit the publish settings.
I suddenly resolved that by going to project properties, "Package/Publish Web" and checking the mark "Include all databases configured in P/P SQL tab" (and I don't have any DB configured there :)).
After doing this and deleting the mentioned lines from .pubxml everything went fine.
Seems like a hack, but it was a way to go for me :)
#Sayed, thanks for confirming it's a bug, hope it'll be resolved!
I came up with a (possibly) less hacky solution for bypassing the bug in publish that forces discovered Entity Framework code first db contexts to have a connection string. This is still an issue that I'm having in VS 2013.
In your web.config, add a dummy version of the connection string:
<add name="DbContextName" connectionString="This is a dummy connection string to bi-pass publish bug." providerName="System.Data.SqlClient" />
Now, setup a transform for the configuration you want to create a publish package for. Read more about it here.
In your web.config.{configuration} file, use the following transform to remove the connection string:
<connectionStrings>
<add name="DbContextName" xdt:Transform="Remove" xdt:Locator="Match(name)"/>
</connectionStrings>
This transform runs AFTER the publish transform in your pubxml runs, so it clears out the unwanted connection string.
On the Settings tab of the publish profile, clear the Use this connection string at runtime check box and the Apply Code First migrations check box. Make sure that migrations is enabled, or the Use this connection string box won't stay cleared, and even then you may have to clear it again each time you open the profile.
I'm entering a parallel test and dev stage where I need to use one db for test and a different one for dev. How can I have the app choose which connection string to implement based on which physical folder it (the app) sits in?
I know there are SVN strategies to consider but this is small-scale enough to avoid 2 sperate code-bases. Would like to be able to publish the same VS project to either of my 2 directories without having to remind myself to change the connection string.
I'm running under IIS7 so perhaps it offers better control than conditionals in (and overrides) web.config. (or not)
thankx!
A word of advice:
I wouldn't base your connection string on your published folder. Down the road, the folder might change, and folks may not be aware that that determines which connection string you're using.
Instead, control it with a setting in your web.config file. Just add a setting that allows you to switch between production and dev databases. In fact, you could simply test for the presence of a debug mode setting. If that setting is there, you're targeting the development database; otherwise, you're targeting production.
The nice thing about that solution is that it doesn't depend on where you deploy the site, and you can document the setting in the Web.config file.
Hope this helps.
Edit for Clarity: By "a debug mode setting" I mean a setting that determines which database you're targeting, dev/production. Not whether your application is running in Debug mode, since the Framework already provides a function that does that. Also, you wouldn't necessarily remove the setting, since you'd want to keep it for documentation purposes. Rather, you'd comment it out.
You could e.g. create a <connectionStrings> container that contains a connection string for each folder your app could be in:
<connectionStrings>
<add name="Folder1" connectionString=".....(conn str. #1)...:" />
<add name="Folder2" connectionString=".....(conn str. #2)...:" />
....
<add name="Folder-n" connectionString=".....(conn str. #n)...:" />
</connectionStrings>
and then just pick the right one, depending on where your app starts up from.
Marc
Get a unique string for the application, perhaps something like:
string folder = Regex.Match(Server.MapPath("~"), #"\(.+?)\$").Groups[0].Value;
Then use the string to get a value from the web.config:
ConnectionStringSetting connectionString = ConfigurationManager.ConnectionStrings["ConnectionString." + folder] ?? ConfigurationManager.ConnectionStrings["ConnectionString"];
In the web.config you can add several connection strings with the folder names appended for each dev site, and a default connection string with just the name and no folder.
I usually put the connection strings into a separate config file and reference them from the main web.config using configSource:
In web.config:
<?xml version="1.0"?>
<configuration>
<!-- connection strings are located in an external config
file to facilitate deployment in various environments -->
<connectionStrings configSource="connections.config"></connectionStrings>
</configuration>
In connections.config:
<?xml version="1.0"?>
<connectionStrings>
<add name="ConnectionName" connectionString="[your connection string]"/>
</connectionStrings>
Once deployed, I usually exclude connections.config from future deployments (unless it should be changed, that is).