Excluding schema-level permissions in SSDT comparisons - sql-server-data-tools

TL;DR: Is it possible to exclude permissions granted on a schema when doing a schema comparison in Visual Studio 2013?
I have a user account that is only present on production servers, and an identical user that is only on non-production servers. These users are granted execute at the database schema level.
I run a schema comparison in Visual Studio 2013 (using SSDT version 12.0.50927.0) and when presented with the differences I exclude all of them:
VS2013 schema comparision
But the change script generated always wants to force a grant/revoke on these users:
GO
PRINT N'Dropping Permission...';
GO
REVOKE EXECUTE
ON SCHEMA::[dbo] TO [NonProductionUser] CASCADE;
GO
PRINT N'Creating Permission...';
GO
GRANT EXECUTE
ON SCHEMA::[dbo] TO [ProductionUser];
After doing some research, I found that it was possible to exclude these statements from the update script in previous versions. When running the same schema comparison in Visual Studio 2010 (using SSDT version 10.3.31009.2), I'm presented with the option of excluding differences on the schema:
VS2010 schema Comparision
When I exclude the schema differences, the script generated does not include the REVOKE/GRANT statements. Saving the schema comparison and opening the scmp file shows the following XML has been generated:
<ExcludedSourceElements>
<SelectedItem Type="Microsoft.Data.Tools.Schema.Sql.SchemaModel.SqlPermissionStatement, Microsoft.Data.Tools.Schema.Sql, Version=10.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Name>Grant.Execute.Schema</Name>
<Name>ProductionUser</Name>
<Name>dbo</Name>
<Name>dbo</Name>
</SelectedItem>
</ExcludedSourceElements>
<ExcludedTargetElements>
<SelectedItem Type="Microsoft.Data.Tools.Schema.Sql.SchemaModel.SqlPermissionStatement, Microsoft.Data.Tools.Schema.Sql, Version=10.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Name>Grant.Execute.Schema</Name>
<Name>NonProductionUser</Name>
<Name>dbo</Name>
<Name>dbo</Name>
</SelectedItem>
</ExcludedTargetElements>
I can take this scmp file and open it in VS2013 with no problem, but the result is the same: The option to exclude permissions granted on the schema is not available. What's really strange is that if I save the scmp file using VS2013, the value listed for "Version" is changed from 10.3.0.0 to 12.0.0.0. It's as if SSDT is claiming that this feature is supported, but it appears that it isn't. Or am I missing something?

You can use my deployment contributor to do it:
https://the.agilesql.club/Blogs/Ed-Elliott/HOWTO-Filter-Dacpac-Deployments
Use the filter IgnoreSecurity or you can filter on the specific object.
Ed

In the schema compare results, the differences in these permissions will appear under Database Options > Permissions (and not under the user). You should be able to exclude these changes by unchecking those items under Database Options.
Alternately you can disable comparison of permissions by unchecking Database Options, Permissions and Extended Properties in the Object Types tab of the Schema Compare Options dialog.

Related

Where does BizTalk keep dlls?

I am having some trouble with BizTalk saying a schema doesn't exist when it does exist.
I can see it in the BizTalkMgmtDb using
select * from bt_documentspec where msgtype like '%myschema%'
I can see it in the BizTalk Server Administration Console > <All Artifacts> > Schemas > My Schema.
This paragraph is optional. I'm only including it so you know I already tried the answer you get when you google this problem.
The exact error message I get is something like "This Assembler cannot retrieve a document specification using this type" which if you google that tells you the schema is either not deployed or the root element you're using exists in two separate namespaces so you should supply a DocSpecName on the SendPorts XMLTransmit 1 2.
I've also tried reinstalling the dll that contains the problem schema into the GAC as per the answer to this question. That didn't work either but now I have more questions anyways.
if I do gacutil -l none of my assemblies are listed there.
So I was wondering what does this even do?
Because those assemblies are not listed under gacutil -l. Also when you look at those assemblies
it shows a source location and a destination location.
Why? Why not just THE location? I can't even find %BTAD_InstallDir% on my computer so there are a minimum of 4 possible locations where this schema dll might be.
source location
destination location (%BTAD_InstallDir%)
"the GAC"
wherever it actually is because there doesn't seem to be a %BTAD_InstallDir%.
So if I want to "update the GAC" to make sure this schema is there how do I do that? What dll is BizTalk ACTUALLY using? The one in %BTAD_InstallDir% or the one in the GAC?
BizTalk 2010.
It has nothing to do with where the DLL is, it has to do with the fact that either
The XML payload namespace and root node does not match any schema
You have two or more schemas with exactly the same namespace and root node that match.
If you don't have any namespace on the XML, then it will try and match on Root node only, but again if you have more than one schema with the same root node, it will fail.
What you need to do is look at the suspended message and look at it's context properties and see what it's Message Type is, and compare that against the schema in BizTalk. There should be exactly one that matches.
vs
But to answer your question,
The source location is usually just the path of where the DLL was when it was initially imported in Dev.
the %BTAD_InstallDir% is the directory you chose when importing a MSI, the default location is C:\Program Files (x86)\Generated by BizTalk. However that is just where it puts it initially
Yes, in the end the actual one it uses will be in the GAC, which is under C:\Windows\Microsoft.NET\assembly\GAC_MSIL
Note: If the DLL has been previously been deployed and you are deploying a new one then make sure you restart the host instances afterwards, as it will cache DLLs in memory for a period.

Unable to compile SSDT Database Project with a view that has fully qualified name of table in view definition

We have a SQL 2019 database where all table names are fully qualified in views starting with the database name. We do NOT have the option of avoiding the fully qualified reference as the view definition is auto-generated (otherwise I would simply not fully qualify them). When views are defined by referencing tables within the same database as the view, the SSDT project complains that it has an unresolved reference.
Visual Studio does not allow adding a database reference to itself. The only way I can get it to compile is to create a DACPAC of the same database and then add that as a reference along with removing the database variable ($Name).
Is there any other method of providing fully qualified table names in views without having to create a DACPAC in SSDT project?
Only way I'm aware of would be to take the view code out of the project and handle in post-deploy scripts. This is done by design, because the database name may not be what was defined in the original code.
You can't use 3-4 part naming in SSDT normally. You can workaround this by using variables in the code. So let's say, that you have [localhost].[reports].[dbo].[your_table] you'll need to use [$(ReportServer)].[$(ReportDatabase)].[dbo].[your_table].
I have a DacPac project containing objects which use three part naming to refer to the containing database (hundreds of instances such as [thisDb].[dbo].[obj]* exist). I need compare and update this database, but the db project fails to build due to 200+ sql71561 errors.
I did not want to remove the unnecessary database name part or switch to using a database name variable. To successfully build (or compare, and then update) a database using three part naming or fully qualified naming to refer to itself, there is a way I found to pacify visual studio. It's not what I'd prefer, but it works.
Create a copy of the original db project.
In the copy db project, update all local database object references to use just two part names ([dbo].[obj]) instead of three part names (I used find & replace).
Make sure the copy db project targets the same SQL server version and builds successfully.
Reference the copy db project from the original db project (whether via database variable, database name only, or dacpac).
The original db project can now build because its references can be resolved. You'll end up with a dacpac for both the original and the copy, but at least the errors are gone and it compiles.

Dynamic database reference in SSDT (dacpac) project

Is there a way where the database references in a SQL Server Database project (Dacpac) can be derived at run time?
We're building a product which uses Dacpac to deploy database objects.
Our product implementation teams also use Dacpac projects by adding database reference to the product Dacpac file and then adding their own additional objects to the project.
The problem we're facing is - every time the implementation needs to point to a newer product release version, the parent dacpac references in the implementation dacpac have to be changed manually to refer to the new file path of the new product dacpac (in the newer release). We've mutiple implementation teams and multiple database projects in each implementation.
Is there any way the database references (*.dacpac) in a database project can be derived at run time using a variable or parameter or something of that sort?
My understanding of your question is as follows:
You have a SSDT database project (see example image below), that has a database reference (similar to #1 below) with stored procedures and other db objects that access the reference database (similar to file #2 and code #3). You've 'built' the solution resulting in a DACPac and now you want to take this DACPac and reference other databases at deploy-time. Hopefully I understood your question correctly.
If so, then you can use the Publish Profiles feature to achieve this (similar to #4 below).
The code for this can be seen in my notes from my SSDT talk github project.
If you look specifically at the demo04 solution file, you will see that I have a DEV_MJE.deploy.ps1 PowerShell file and a DEV_MJE2.deploy.ps1 file. These scripts run MSBuild to build the DACPac and then use SqlPackage to publish DEV_MJE.publish.xml or DEV_MJE2.publish.xml respectively. Note, if you run this example on your machine, you'll need to add MSBuild.exe and SqlPackage.exe to your path, as well as modifying the TargetConnectionString in the xml files to an existing development database.
As an example of how this works...When I use the Publish Profile DEV_MJE.publish.xml, the resulting GetDataFromAnotherTable.sql file contains:
SELECT [SomeData] FROM [AnotherDb_MJE].[dbo].[AnotherTable]
Whereas when I use DEV_MJE2.publish.xml the resulting GetDataFromAnotherTable.sql file contains:
SELECT [SomeData] FROM [AnotherDb_MJE2].[dbo].[AnotherTable]
Notice the database reference in the second has changed to AnotherDb_MJE2.
For a good description of how Publish Profiles relate to DACPacs and SSDT Database Projects, see this web page. It also contains information on alternative ways to deploy beyond SqlPackage.exe.
Something else to consider
Note, that using file paths to version control a DACPac is not really the best practice. Consider the DACPac artifact as similar to a .Net DLL. It is the biproduct of a build.
As such, a better approach is to use NuGet and tools like Octopus Deploy to store, track, and deploy DACPacs. See the stackoverflow answer for a good description of how this works.
Hope that this helps,
Michael
Thanks for the followup comment, I think what you are trying to do is when you write and deploy your code be able to use different dacpacs depending on the project?
Each implementation team might have a different version of the shared dacpac deployed so you can't just put the files in a shared location and call the dacpac "Product_Latest.dacpac" or something, so everyone always gets the latest version.
".sqlproj" files are standard msbuild scripts and references can be managed using msbuild properties so you can technically change the reference at runtime. If you edit your .sqlproj file and add a property in the first <PropertyGroup> tag, I used:
<ProdDacpacVersion Condition="'$(ProdDacpacVersion)' == ''">v1</ProdDacpacVersion>
v1 is the unique name for the version folder - you just need something to identify the dacpac you want.
I put the property just after TargetDatabaseSet and IncludeCompositeObjects.
If you then find the reference to the dacpac and instead of
<ArtifactReference Include="..\..\..\..\..\Desktop\prod\v1\Database2.dacpac">
<HintPath>..\..\..\..\..\Desktop\prod\v1\Database2.dacpac</HintPath>
<SuppressMissingDependenciesErrors>False</SuppressMissingDependenciesErrors>
</ArtifactReference>
Use the property from above:
<ArtifactReference Include="..\..\..\..\..\Desktop\prod\$(ProdDacpacVersion)\Database2.dacpac">
<HintPath>..\..\..\..\..\Desktop\prod\$(ProdDacpacVersion)\Database2.dacpac</HintPath>
<SuppressMissingDependenciesErrors>False</SuppressMissingDependenciesErrors>
</ArtifactReference>
Then the reference will use the property to get the path of the dacpac. There are a few ways you can set the property, you could edit the .sqlproj file when you want to take a new version of read the property from a file or environment variable or something (i'll leave the msbuild fun to you!).
I would, as a standard, everytime the reference was changed either reload the project or restart visual studio - it won't take long and it will save lots of cursing :)
When you deploy the dacpac's, the deploy looks in the same folder for references first so just make sure you copy the right one into the bin folder when you deploy.

SSDT Project Structure when updating using SchemaCompare

I have set up my sqlproj project structure based on object type using the import Database wizard and using the Object Type as the folder structure. ie the same view as you'd get in SSMS or SQL Server Object Explorer
Yet when using ssdt schema compare to update the project, objects are always imported into Schema\Object Type structure, causing the project to turn into a mess of mixed structure.
I cant find anywhere I can change the behaviour of the Schema Compare update to continue to use the Object Type structure?
using ssdt 14.0.51215 (Dec 2015)
I would suggest submitting this feedback as a suggested improvement to Microsoft at https://connect.microsoft.com/SQLServer/feedback/CreateFeedback.aspx using the category "Developer Tools (SSDT, BIDS, etc.)"
you'll have to update the sqlproj file directly...
<PropertyGroup>
...
<DefaultFileStructure>BySchemaType</DefaultFileStructure>
...
</PropertyGroup>
More info on this link... https://social.msdn.microsoft.com/Forums/sqlserver/en-US/e6bfd561-3e7d-4052-be9c-631037681c3e/default-file-structure?forum=ssdt

Best practice for using entity framework for post deployment database changes

I am working on a ASP.NET Web Forms project which is expected to experience lots of database changes even after deployment.
Our preference was to use Entity Framework 5, and the Database First design paradigm. However, since we have to make lots of changes to the database even after deployment, if I use database first approach, then every time I update my database, I will have to delete my entire model and regenerate it. Is there any best practices to make this process less painful?
You should use Code First, so that you can use Migrations.
More specifically I'd use manual Migrations.
With manual Migrations you can create Migrations at any point in time. A migration has the following information:
a snapshot of the current model
a set of "instructions" to upgrade from the previous migration
a set of "instructions" to downgrade to the previous migration
Apart from the necessary Migrations, you should add a new Migration when you deploy your app. For example, you can create a migration called "Version 1.0", when you deploy the version 1.0 of your app.
When you finish each new stable version, you simply add a new migration, for it, like "Version 1.1" or "Version 1.2".
The interesting part of migrations shows up when you have a deployed application version and you need to upgrade (or downgrade) to a new version.
There are commands that let you upgrade (or downgrade) a DB from one particular version
to another particular version. You can do it directly specifying a connection to the DB, or create a SQL script which will apply the changes to the DB. For example, if you deployed the version 1.0 in a customer server, and you need to upgrade the software to version 1.2, you can do this:
Update-Database -SourceMigration "Version 1.0" -TargetMigration "Version 1.2"
-Script
This will create a SQL script which can be run on the DB to upgrade from version 1.0 to 1.2.
If you need help about any of the Migrations commands, simply type:
get-help Update-Database -full
(Update-Database is a command name, you can specify any other like Add-Migration)
It's possible that you need to specify the project where the model is in, the connection string name, the name of the project with the .config file or some other things, depending on the specified parameters, and the structure of the projects in your solution.
To get more info on migrations, read MSDN EF Code First Migration.
NOTE ADDED IN EDITION: there is a new DB initializer that can automatically migrate to the latest version when the application runs. I've worked in a real application, and it works like a charm.
ALTERNATIVE SSDT
If you don't want to follow the advice, you can use SQL Server Data Tools (which can be installed inside VS, or work as an independente application, depending on the version you're using).
The idea of this tool is that you can compare projects (which are DB schema snapshots) to existing DBs, and generate the scripts to change the DB to match the schema in the project. (In fact you can compare any combination ofr project and DB)
If you keep versions of your project in a CVS you can even check the changes from one verison of the project to other version of the project.
NOTE ADDED IN EDITION: a SSDT project is a set of scripts that can build the whole DB, including all the objects in its schema. You can create it from an existing DB or viceversa. Then you can keep comparing any combination of DBs and SSDT projects, as soruce or target, and create an apply the scripts neccesary to change each particular object which has changes. (The scripts are created to change the target so that it looks like the source, but you can swap source and target easyly)
This is an alternative solution, but Migrations are much more powerful because you can customize them, for example to fill a "master table" when creating it, to set the initial value of a new column, and so on. If you use SSTD you'll have to do all of that by hand, and carefully keep track of it. So I highly recommend using Migrations.

Resources