How I make the package
I make the msdeploy package like this:
msdeploy.exe -verb:sync -source:iisApp=c:\content\ -dest:package=c:\pkg.zip
The c:\content directory has a single index.html file.
Result
The output looks like this:
Info: Adding package (package).
Info: Adding child iisApp (c:\content\).
Info: Adding child createApp (c:\content\).
Info: Adding child contentPath (c:\content\).
Info: Adding child dirPath (c:\content\).
Info: Adding child filePath (c:\content\index.html).
Total changes: 6 (6 added, 0 deleted, 0 updated, 0 parameters changed, 0 bytes copied)
If I extract the content of c:\pkg.zip into directory c:\pkg it looks like this:
archive.xml
systemInfo.xml
Content\c_C
Content\c_C\content
Content\c_C\content\index.html
If I dump the package like this:
msdeploy.exe -verb:dump -source:package=c:\pkg.zip -xml
I get:
<output>
<MSDeploy.iisApp>
<iisApp path="c:\content\">
<createApp
path="c:\content\"
isDest="False"
managedRuntimeVersion=""
enable32BitAppOnWin64=""
managedPipelineMode=""
applicationPool=""
appExists="True" />
<contentPath path="c:\content\">
<dirPath
path="c:\content\"
securityDescriptor="D:"
parentSecurityDescriptors=""
attributes="Directory">
<filePath
path="index.html"
size="0"
attributes="Archive"
lastWriteTime="07/07/2011 20:58:00"
securityDescriptor="D:" />
</dirPath>
</contentPath>
</iisApp>
</MSDeploy.iisApp>
</output>
How I want it to be
I don't want the package to depend upon the current location of the site files. I'm going to send the package to a customer, and I don't want any detailes about the packaging process to get shipped with the package. I want the content of the package c:\pkg.zip to be like this:
archive.xml
systemInfo.xml
Content\index.html
I want the package to be able to create an IIS application, so I need a virtual path. I also want to install the package into the default location. So the physical path also has to change. I want the dump to look something like this:
<output>
<MSDeploy.iisApp>
<iisApp path="Default Web Site\Site">
<createApp
path="Default Web Site\Site"
isDest="False"
managedRuntimeVersion=""
enable32BitAppOnWin64=""
managedPipelineMode=""
applicationPool=""
appExists="False" />
<contentPath path="c:\inetpub\wwwroot\site">
<dirPath
path="c:\inetpub\wwwroot\site"
securityDescriptor="D:"
parentSecurityDescriptors=""
attributes="Directory">
<filePath
path="index.html"
size="0"
attributes="Archive"
lastWriteTime="07/07/2011 20:58:00"
securityDescriptor="D:" />
</dirPath>
</contentPath>
</iisApp>
</MSDeploy.iisApp>
</output>
I have changed the iisApp and createApp provider path attributes to be Default Web Site\Site. And I changed the contentPath and dirPath provider path attributes to be c:\inetpub\wwwroot\site.
Questions
How can I accomplish this?
You need to look at MS Deploy replace rules, a useful feature well hidden on the MS Deploy Team Blog.
In your case, you will need to extend your command line with a pile of replace expressions, something like this:
msdeploy.exe
-verb:sync
-source:iisApp=c:\content\
-dest:package=c:\pkg.zip
-replace:objectName=iisApp,targetAttributeName=path,
replace="Default Website\Site"
-replace:objectName=createApp,targetAttributeName=path,
replace="Default Website\Site"
-replace:objectName=contentPath,targetAttributeName=path,
replace="c:\inetpub\wwwroot\site"
-replace:objectName=dirPath,targetAttributeName=path,match="^c:\content",
replace="c:\inetpub\wwwroot\site"
Running this should produce your desired output.
In the above sample, the first 3 replace rules match by tag name (objectName) and attribute name (targetAttributeName), and overwrites with the specified replace string.
The last replace rule will match all path attributes of all dirPath tags beginning with "c:\content" and will replace only that part of the attribute value with the replace string.
Finally, I haven't found a way to avoid having the package zip-file contain the original source folder names. The only workaround would be to package from a neutral, temporary location like "c:\site".
So the procedure is:
Copy your stuff to a neutral, temporary location.
Create your package from here.
Use the verb:dump to see the generated xml.
Create your package again with added replace rules for everything you want changed in the package.
Take a headache pill ;-)
I had more or less the same problems.
First things first:
The long deployment-machine-specific paths
To that effect, I used a trick found at http://sedodream.com/2013/01/13/WebPackagingFixingTheLongPathIssue.aspx
As suggested in the post, one can modify the desired (you can have several) .pubxml file under the Properties/PublishProfiles folder in your project. This is the approach I followed since it allowed me to customize the behavior per publishing profile.
If I'm not mistaken though, I believe you can apply the same modification to the {project-name}.wpp.targets file (which probably doesn't exists yet) on the project root directory. Changes here though, affect the web publishing pipeline (wpp) and thus all publishing profiles found in the project.
However...
This approach is just about to spoil your deployment when it comes time to replace your connection strings with those provided by your publishing profile. The reason: the above trick doesn't affect connection strings since they are being created automatically by the wpp at build time. Buh-huh!
The solution I found for that problem was twofold:
1.) Created a parameters.xml file where I manually declared the connection strings. Ok, maybe I copied them from the parameters.xml file within my package's .zip file since I was deploying to a package. That helped.
They look somegthing like this:
<parameter name="myConnection-Web.config Connection String" defaultValue="" tags="SqlConnectionString"
description="myConnection Connection String used in web.config by the application to access the database.">
<parameterEntry kind="XmlFile" scope="DeploymentPackage\\Web\.config$" match="/configuration/connectionStrings/add[#name='myConnection']/#connectionString" />
</parameter>
2.) Included the following line at the top of the same .pubxml file we modified earlier
<AutoParameterizationWebConfigConnectionStrings>false</AutoParameterizationWebConfigConnectionStrings>
And... VoilĂ !
Create ISS App
With the above approach hopefully you declared several parameters, including the connection strings.
When you create a package, however, regardless of wheter you created a parameters.xml or not, a *.SetParameters.xml template file is created for you. Within it you will see as the very first parameter the "IIS Web Application Name", which will default to whatever you inserted in your publishing profile. You can change that; to whatever you want.
Remember I said template before? I meant it; it's just a template. You're suppose to take that *.SetParameters.xml file and make as many copies of it as needed. What are they for? Environment related parameters. You could have a:
DEV.SetParameters.xml
QA.SetParameters.xml
Staging.SetParameters.xml
Production.SetParameters.xml
... and so on and so forth
and then use the parameters file best suited for the job (or the environment) like so:
{yourProjectName}.deploy.cmd /Y /M:{targetServer} [...] -setParamFile:QA.SetParameters.xml
or its equivalent MsDeploy command line of course.
Now, by default, the manifest created for you at build time, and stashed within your package under the archive.xml file, will use an iisApp provider first and foremost. This is good, cause this provider, unlike the createApp provider, will actually create the directory for you if it doesn't exist. At least according to this note from TechNet:
"Unlike the iisApp provider, if the physical folder for the new application does not exist, the createApp provider does not create a physical folder underneath the folder of the parent site; it only creates a reference in configuration to such a folder. If you want a physical folder created, you will have to create it manually before or after using createApp. For this reason, you should normally use the iisApp provider instead. The iisApp provider is the more appropriate choice because it uses the createApp provider as an initial step in a series of steps that include the creation of the application in configuration, the creation of a physical folder for the application if the folder does not exist, and the copying of content files into the folder of the new application."
I would be happy to include the links... but since I don't have 10+ points I'm allowed only one per post. Go figure! :)
So, in short...
... by the work done on the first part, you probably won't need to do much in order to have the folder created at deploy time in the target server.
In case you do need to override that though, you can either define your own manifest file and deploy off of it (a separate topic)... or you can follow #peter_raven advice and override its value using the -Replace rules from MsDeploy.
Either one would work as a charm.
The package prefix is removed by supplying the kind, scope, and match properties as shown below:
"msdeploy.exe" \
-verb:sync \
-source:iisApp="[Path to your website contents]" \
-declareParam:name="IIS Web Application Name",kind="ProviderPath",scope="IisApp",match="^C:\\path\\to\\your\\site\\folder",defaultValue="Default Web Site/SomeSite" \
-dest:package=[WebDeployPackageName].zip
I've managed to resolve a problem with manually defined connection strings while using the solution from #SkyFighter. Now one can use the auto-parameterization feature and have the connection string parameters with correct scopes.
Fortunately there is a place inside WPP to inject into. Unfortunately I had to use AfterTarget/BeforeTarget rather than SomeTargetDependsOn variables to narrow down the new target's placement.
And here is the target itself:
<Target Name="Replace_WebConfigsToAutoParmeterizeCS_TransformScope"
AfterTargets="PreAutoParameterizationWebConfigConnectionStrings"
BeforeTargets="AutoParameterizationWebConfigConnectionStringsCore"
Condition=" '$(EnableAddReplaceToUpdatePacakgePath)'=='true' ">
<ItemGroup>
<_WebConfigsToAutoParmeterizeCS>
<TransformScope>$([System.String]::Copy('%(TransformScope)').Replace('$([System.IO.Path]::GetFullPath($(WPPAllFilesInSingleFolder)))', '$(PackagePath)'))</TransformScope>
</_WebConfigsToAutoParmeterizeCS>
</ItemGroup>
</Target>
It is driven by the same variables as in the Sayed's sample for fixing long paths. So place this target anywhere those variables already available.
P.S. This trick/hack requires at least MSBuild v3.5 where metadata manipulation was first introduced.
Related
I'm using Octopus as part of our deployment for a .NET Core 3.1 Web API project.
log4net.config exists in .\Utility\Logs.
I'm trying to follow the pattern here:
https://help.octopus.com/t/transformation-best-practice-log4net-config-or-any-non-web-app-config/9906/4
As I understand it, this consists of three parts:
Create Log4Net.DeploymentTransform.config, with the variables in #{name} format - this has been done.
Turn on the "Substitute variables in files" feature, and point at the Log4Net.DeploymentTransform.config transformation file (variable replacement happens before transformation). That would result in the #{LogFileLocation} variable being replaced with whatever value was set for your LogFileLocation variable in the current scope.
This is done and is working.when my app is deployed, Log4Net.DeploymentTransform.config is there as well and the variable has been successfully set in it.
You'd also turn on the configuration transforms feature, and fill out the additional transforms section in the configuration transforms feature to identify your transform file (e.g. Log4Net.DeploymentTransform.config => log4net.config).
This is not working, the content of Log4Net.DeploymentTransform.config is not being copied on top of log4net.config, though they are in the same folder upon deployment.
Here is what I did in our "deploy step"
Which sure looks like what the article is saying to do.
What else should I check? Any idea why step 3 isn't occurring?
Your syntax looks correct for the files - have you checked to ensure that you have the xdt attributes set?
In the example forum post you shared, the log4net and appender elements are tagged with xmlns:xdt, xdt:Transform and xdt:Match attributes that help the XDT layer determine how to transform the files.
A quick example - I created a Log4NetConfigTest package with two files -
Utility/Logs/log4net.config
Utility/Logs/log4net.trasnform.config
I used the same sample code from the forum post as well.
Here's the set up for my package deployment configurations:
With that set up (and my LogFileLocation project variable set), I was able to see the following in my task log for the deployment:
Deploying package: C:\Octopus\Files\Log4NetConfigTest#S1.0.0#20004C95A0E0094490814B5A365DDAD2.zip
Transforming 'C:\Octopus\Applications\Development\Log4NetConfigTest\1.0.0_1\Utility\Logs\log4net.config' using 'C:\Octopus\Applications\Development\Log4NetConfigTest\1.0.0_1\Utility\Logs\log4net.transform.config'.
No matching appSetting, applicationSetting, nor connectionString names were found in: C:\Octopus\Applications\Development\Log4NetConfigTest\1.0.0_1\Utility\Logs\log4net.config
The task log confirmed that Octopus did apply the transformations, and once deployed, I confirmed that my new test log location was present and correct in both the base configuration and the transformation file.
I am adding this item in the ItemGroup in the project wpp.targets file.
<MsDeploySourceManifest Include="setAcl">
<Path>%windir%\TEMP</Path>
<setAclUser>IIS_IUSRS</setAclUser>
<setAclAccess>Read,Write,Modify</setAclAccess>
<setAclResourceType>Directory</setAclResourceType>
<AdditionalProviderSettings>setAclUser;setAclResourceType;setAclAccess</AdditionalProviderSettings>
</MsDeploySourceManifest>
and this on the parameter section
<MsDeployDeclareParameters Include="TempFolderPermissionSetAclParam">
<Kind>ProviderPath</Kind>
<Scope>setAcl</Scope>
<Description>Add read, write permission for IIS_IUSRS to the temp folder.</Description>
<DefaultValue>%windir%\TEMP</DefaultValue>
<Value>%windir%\TEMP</Value>
<Tags>Hidden</Tags>
<Priority>$(VsSetAclPriority)</Priority>
<ExcludeFromSetParameter>True</ExcludeFromSetParameter>
</MsDeployDeclareParameters>
it generate this line in the manifest xml file
but there is an error message says:
Error: A value for the 'setAclUser' setting must be specified when the 'setAcl'
provider is used with a physical path.
what's missing here?
It's been a while since you asked this, but on the surface this looks more or less correct: the key point for specifying an absolute path are that you include setAclUser in the AdditionalProviderSettings, as well as an element for setAclUser.
Ultimately, while your SourceManifest file might appear correct when you build the package, what seems to matter though is the setAclUser settings on the setAcl elements within the archive.xml file contained within the zip that is created. Open that up and ensure that the value you expect for setAclUser is present.
As well, Visual Studio makes liberal use of caching where MSBuild files are concerned; practically speaking it seems that Visual Studio will respect changes made only to pubxml files and project files such as csproj. When making changes to any other target files, you should close Visual Studio and reopen after making each edit. This could explain why your seemingly-correct solution wasn't working: it might have simply been stale target file caches.
Can somebody explain (better than the technet/msdn docs) what the auto provider exactly does, how it works, and when to use it.
This is in regard to Web Deploy. I've seen a lot of documentation specifying -dest:auto and it's not really making sense to me.
The auto provider specifies that the provider on a destination will be
the same as the source provider.
Example
msdeploy.exe -verb:sync -source:appHostConfig="MySite"
-dest:auto,computername=Server1
The auto provider enables you to avoid entering the full path for the
-dest argument when the destination argument is the same as the
-source argument. It also removes the need to copy the manifest file
separately from an archive or package.
The auto provider takes the source that you specify and uses a
corresponding location on the destination computer. For example, if
you specify appHostConfig=Site1 as the source, the destination on the
target computer will be Site1. This is useful when you want to
synchronize a Web site "as is" to a remote machine.
technet docs for auto provider
Example that doesn't make sense:
msdeploy.exe -verb:sync -source:package=myapp.zip -dest:auto
Why would you ever set the destination exactly to the source? What is the point? Aren't you simply overwiting the source with iteslf?
The generated cmd file generated from publishing a package in VS2010 generates something like this:
"C:\Program Files\IIS\Microsoft Web Deploy V2\\msdeploy.exe" -source:package='MySourcePath' -dest:auto"
Doesn't auto mean the source will just overwrite itself? But it doesn't, it actually updates the IIS web site (based on the settings in the manifest)
I've tried using package as the destination and in this case it did update the source package and not the IIS site.
The catalyst for this question is that I'm implementing CI and I 've always used msbuild/xcopy in the past. I want to utilize msdeploy now. I want to understand it rather than simply calling the myproject.cmd that's generated from visual studio.
For example, this SO link specifies using the auto provider for the dest argument.
Thanks
I think you've actually figured out the answer for yourself.
If the source is a package and the destination is auto, that simply means that the components specified inside the package will be "unzipped" and placed on the destination server. In the -source:package -dest:auto syntax, "auto" does not mean that the package itself is the destination.
And, as you observe, if you specify a package as the destination, whatever you specify for the source will indeed be packaged up as a zip file that you can later use as a source to deploy elsewhere.
Simple task, but for some reason no simple solution just yet.
We've all got web.config files - and I haven't worked anywhere yet that doesn't have the problem where someone yells across the room "Sh*t, I've just uploaded the wrong web.config file".
Is there a simple way of being able to auto generate a web.config file that will contain the right things for copying to release? An example of these being:
Swap connection string over to use live database
Change
Switch over to use the live/release logging system, and live/release security settings
(in our case we need to change the SessionState mode to InProc from StateServer - this isn't normal)
If you have others, let me know and I'll update it here so it's easy for someone else to find
Maintaining 2 config files works, but is a royal pain, and is usually the reason something's gone wrong while you're pushing things live.
Visual Studio 2010 supports something like this. Check it out here.
How are you deploying your builds. In my environment, this used to be a pain point too, but now we use cruisecontrol.net and script our builds in nant. In our script, we detect the environment and have different versions of the config settings for each environment. See: http://www.mattwrock.com/post/2009/10/22/The-Perfect-Build-Part-3-Continuous-Integration-with-CruiseControlnet-and-NANT-for-Visual-Studio-Projects.aspx for my blogpost onthe subject of using cruisecontrol.net for build management. Skip to the end fora brief description of how we handle config versions.
In my most recent project I wrote a PowerShell script which loaded the web.config file, modified the necessary XML elements, and saved the file back out again. A bit like this:
param($mode, $src)
$ErrorActionPreference = "stop"
$config = [xml](Get-Content $src)
if ($mode -eq "Production")
{
$config.SelectSingleNode("/configuration/system.web/compilation").SetAttribute("debug", "false")
$config.SelectSingleNode("/configuration/system.web/customErrors").SetAttribute("mode", "off")
$config.SelectSingleNode("/configuration/system.net/mailSettings/smtp/network").SetAttribute("host", "live.mail.server")
$config.SelectSingleNode("/configuration/connectionStrings/add[#name='myConnectionString']").SetAttribute("connectionString", "Server=SQL; Database=Live")
}
elseif ($mode -eq "Testing")
{
# etc.
}
$config.Save($src)
This script overwrites the input file with the modifications, but it should be easy to modify it to save to a different file if needed. I have a build script that uses web deployment projects to build the web app, outputting the binaries minus the source code to a different folder - then the build script runs this script to rewrite web.config. The result is a folder containing all the files ready to be placed on the production server.
XSLT can be used to produce parameterized xml files. Web.config being xml file this approach works.
You can have one .xslt file(having xpath expressions).
Then there can be different xml files like
1. debug.config.xml
2. staging.config.xml
3. release.config.xml
Then in the postbuild event or using some msbuild tasks the xslt can be combined with appropriate xml files to having different web.config.
Sample debug.config.xml file can be
<Application.config>
<DatabaseServer></DatabaseServerName>
<ServiceIP></ServiceIP>
</Application.config>
.xslt can have xpaths referring to the xml given above.
Can have a look at the XSLT transformation This code can be used in some MSBuild tasks or nant tasks and different web.config's can be produced depending on the input config xml files.
This way you just have to manage the xml files.
There is only one overhead that the xslt file which is similar to web.config need to be managed. i.e whenever there is any tag getting added in the web.config the xslt also needs to be changed.
I don't think you can 100% avoid this.
The last years of work ever and ever shows: where human worked, there are fails.
So, here are 3 ideas from my last company, not the best maybe, but better then nothing:
Write an batch file or an C#.Net Application that change your web.config on a doubleclick
Write a "ToDo on Release"-List
Do pair-realesing (== pair programming while realease :))
I am using the built in test framework in VS2008 and I would like be able to write a test that makes sure all the expected web.config settings have been defined so that if by accident one is removed or changed my suite of tests will detect it and not have to be tested in a runtime scenario. How would I set this up?
I do not want to setup a mockup of my web.config since I don't want to maintain two versions and this would make my test invalid anyways since I am really trying to capture the fact that the project's web.config is correct.
Any suggestions, alternatives, hints?
Solution: I ended up using the copy in the pre-build that was suggested with one change. On copy I rename the web.config to app.config so that the test project would automatically pick it up.
I tried to split out the config files as suggested as well but the problem I ran into was when the test project ran, it actually didn't run out of the bin directory (which setting the config files to 'Content' type would copy to) but instead to a results directory that has been pre defined. I could not figure out how to make it copy thos extra files to this results directory so the config files could never be found.
I'am using the pre-build event to copy working web.config to your test project directory.
Set the command line of the pre-build event of test project to string like this:
copy $(SolutionDir)\YourWebAppDir\web.config $(ProjectDir) /y
After that your tests will always run with actual web.config version.
Comment to pcampbell's answer:
I think if you use the configSource attribute you can just set it to the same path in web.config of your web app and app.config of test project and that makes not necessary to use build events.
sorry, I can't leave comments yet.
To expand on bniwredyc's answer, perhaps consider:
refactoring your web.config to reference a new config file like appSettings.config or similar.
modify your project's web.config to:
<appSettings configSource="appSettings.config" />
modify your Unit Test project's app.config to use this file as well.
modify your post or pre-build events to copy just this file.
this also helps ease of deployment in Test/Staging/Prod
Ultimately, the web.config is an XML file. You could generate a schema to validate the sections required are present and that required values have been populated. Obviously, you couldn't contextually validate any sort of business logic that the configuration might contain, but you could use a combination of an XSD validation plus a lightweight class that is used to parse conditions within the file.
Used in conjunction with a copy pre-build event you actually create a very nice test harness for your production quality configurations.