VSTS - Asp.Net Build with Project Dependencies and Git Repositories - asp.net

Since VSTS has sought to bend to the popular Git source control, I have yet to see a good description of building .Net projects located in Git repositories, having project dependencies on one another.
For instance, in Visual Studio, I build a solution that includes projects with dependencies on each other. Then, in VSTS each of those .Net projects are versioned in separate Git repositories.
How, then, do you get a build on VSTS? How do you get the artifacts (read: DLLs) from one project into the project of the other?
UPDATE: 12/18/17
I took #VonC's suggestion and followed-through on a VSTS (Visual Studio Team Services) hosted Nuget package. I was able to make this work. This process makes .Net solution files and project dependencies OBSOLETE.
If you want to reuse a library, you can save the binaries as a NuGet package.
In the downstream project, you simply assign the VSTS url reference to the Nuget package to get the Nuget Restore to find/place the binaries in your build project.
You will have to download and install a Credentials tool that will allow you to push your binaries to VSTS's package location. Additionally, tell your admin to add the Packages functionality from the VSTS Marketplace.
Thanks, #VonC for the great suggestion!
Here are some helpful links:
Create and Publish the Private Nuget Package here
VSTS Marketplace Package Manager here

The idea is, for binary dependencies (DLLs) to not involve a source control tool (like Git) but a binary referential one (like Nuget)
See for instance:
"Package: NuGet"
"NuGet is now fully integrated into MSBuild"
With Visual Studio 2017 and .NET Core, we have improved the NuGet package management experience by introducing the PackageReference feature in MSBuild.
PackageReference brings new and improved capabilities such as deep MSBuild integration, improved performance for everyday tasks such as install and restore, multi-targeting and more.

First, it’s unnecessary to manage the build artifacts (such as dlls) in source control since they're the output files from the source code.
Then to add dependencies (dlls) from other repos to the parent (main) repo’s project, there usually has below options:
Option 1: manage the build artifacts as packages
As Vonc mentioned, you can manage the dlls as nuget packages, and then add nuget packages to your main repo’s project.
Option 2: git submodules
You can also treat other repos as the submodules for the main repo, and both build the projects from the submodules repos and the main repo in the build, then the main repo project can get the dependencies from the submodule repos’ build artifacts.
Commands to add a submodule for the main repo:
# In local main repo
git submodule add <URL for a submodule repo>
git commit -m 'add a submodule'
git push
Note: in VSTS build definition, you should select checkout submodules in Get Sources step.
Details about git submodules, you can refer Submodules.
Option 3: git subree (alternative way for git submodules)
Treat a branch from another repo as a subtree (a folder) in the main repo. Then build the projects both in the main repo and the subtrees, and get dependencies from subtrees for the main repo’s project.
Commands to add a subtree in the main repo:
git submodule add --prefix=submodule1 <URL for sub repo> master
git push
Then it will add a folder submodule1 with the files in the sub repo master branch, and commit the changes in the main repo.
Details about git subtree, you can refer
Git subtree: the alternative to Git submodule.

At any time, if your branch has working code with any version of dependent assemblies, I can't see any reason you need to do anything.
For example of dependencies here:
You can set dependencies in project like:
Also you can add dependencies in solution like :
You can set build order in solution too if your project has multiple project with dependencies.
As long as your current code in branch from which you are build is working (with any version of different assemblies, e.g. Classlibrary1 has version 1.0.0.0, Classlibrary2 has version 1.2.2.1 & so on but is working fine with each other after referencing) this approach will work.
Project dependencies exist for ages in Visual Studio & .Net. As long those project exist in same TFS branch You can add project dependency right in dependent project. Also you can manage Project build order in Solution.
For more complex scenarios like different repositories or branch dependencies you need to modify build workflow but it is also quite possible.
You can also refer
http://dailydotnettips.com/2015/11/25/how-to-identify-the-project-dependencies-in-visual-studio/
what I saw long time ago when I created same sample for test.

Related

How to ensure same nuget packages versions will be used during publish and tests?

I have solution with following projects:
Api
Application
Infrastructure
Tests
Api is WebApplication (entry point) and has ProjectReferences to libraries Application and Infrastructure.
Tests is a xunit test project and has ProjectReferences to Api / Application / Infrastructure.
I want consistent package versions both during publishing main (Api) project and during running tests.
I added following properties to Api.csproj:
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
<RestoreLockedMode Condition="'$(CI)' == 'true'">true</RestoreLockedMode>
And it generated Api/packages.lock.json - and it seems that this file also tracks versions of dependencies of referenced projects.
Here is how I publish application (Api):
RUN dotnet restore ./Api/Api.csproj
RUN dotnet publish Api -c Release -o out --runtime alpine-x64 --self-contained true /p:PublishTrimmed=true
So if CI=true env var is set, then commands above should either restore packages according to package.lock.json or fail.
However before publishing Api I run tests like this:
dotnet test ./Tests/Tests.csproj
My question is how to ensure that exactly same package versions will be used suring testing as in Api/package.lock.json? Because if I add <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile> to Tests project then it will have separate Tests/package.lock.json file which may not be same as the one in Api/package.lock.json, right? On the other hand when Tests project references Api project then from what I understand Api/package.lock.json is ignored (when running Tests project)?
Is it possible to have one package.lock.json for solution (same for all projects in solution)?
I feel a bit bad for making this an answer and possibly getting rep votes, when mu88 beat me by 12 hours in the comments to the question, but Central Package Management is the answer. There's also a blog post about it.
Currently, neither Visual Studio, nor dotnet add package support installing or upgrading packages, so you will need to hand edit all the xml (csproj, props) files. But support should be coming in VS 2022 17.4, .NET SDK 6.0.400.

GitLab package registry Nuget packages per group

In my GitLab I have a multiple dotnet core projects (plugins) placed under a group named Plugins, in each one of these projects I added a CI steps to pack them in nuget packages and push them to the GitLab package registry. I followed the documentation : https://docs.gitlab.com/ee/user/packages/nuget_repository/index.html and in the .gitlab-ci.yaml I placed this config :
image: mcr.microsoft.com/dotnet/sdk:5.0
stages:
- deploy
deploy:
stage: deploy
script:
- dotnet pack -c Release
- dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text
- dotnet nuget push "bin/Release/*.nupkg" --source gitlab
only:
- master
but instead of adding a project level source (which is working for me)"$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json"
I replaced it to group level endpoint "$CI_SERVER_URL/api/v4/groups/{group-id}/-/packages/nuget/index.json" with a deploy token for the authentication because I wanted to have one source for all projects under that group, it shows this error each time the nuget cmd try to push the package
error: ERROR: This version of nuget.exe does not support updating packages to package source my-source
any ideas?
I ran into this too. I found this link https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html
The idea suggested in the link is to create a project in the group, the push all your packages to that project. It's not ideal, pushing directly to the group would be nicer, but it is a workaround that doesn't require multiple Nuget sources in Visual Studio. (as of February 2021, hopefully group-level push is supported in the future)
When pushing to project level source packages will also show in group level.
But beware, gitlab has some fundamental problems with nuget integration - if someone does not have access to one project in group, he won't be able to download any packages from that group.
So it's much better to create separate project for packages and push/consume packages there.

How to pack .NET Core projects recursively without running pack on the entire solution?

I have a solution of a hundred plus .NET Core projects. Not all of them needs to be packed, but only those which are transitive dependencies of a few special projects.
However, when I run dotnet pack it attempts to pack all kinds of projects that it should not and there are errors here and there. I would like instead to run pack on the special projects only in a recursive fashion, so that only them and their transitive dependencies (project references, of course) are packed.
I figured I can implement it by scripting around the dotnet list reference command, but it does not sound right. There must be a better way to do it.
EDIT 1
The solution must work on the command line where we have dotnet and msbuild and possibly nuget, but no VS IDE.
You can modify your project settings to generate *.nupkg file during dotnet build, without explicit dotnet pack call. And as soon as dependencies get builded automatically when "parent" project builds - you will receive nuget packages prepared for all dependencies too when you run dotnet build for "parent" project only.
For each project that should produce nuget package add this lines into csproj file:
<PropertyGroup>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
</PropertyGroup>
Or, instead, you may enable checkbox "Generate NuGet package on build" from Visual Studio, in project properties ("Package" tab) - this will add same line into project file.

Restoring NuGet packages to the cache

I have a .net-core application that works on my machine but when I deploy it on another one, it complains about missing packages and points me to the TheApp.deps.json.
My theory is that on my machine the app looks for packages in some NuGet cache where they were probably installed by the IDE during development because the app's output-dir contains only a couple of internal dlls so the other nuget.org dependecies are definitely missing.
I'm building the app with
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.2</TargetFramework>
<RuntimeIdentifier>win10-x64</RuntimeIdentifier>
</PropertyGroup>
and then xcopy it to the other machine.
Question
Is there a way to restore or install the missing packages to the cache on the target machine based on the *.deps.json file?
dotnet build (and the F5/Build function in Visual Studio) simply build the code that you have provided via your source files (i.e cs, fs, vb, etc.).
Whereas dotnet publish (and the Build > Publish function in Visual Studio) does a full package restore, builds your source code, and resolves any external dependencies before moving the output to a specific directory ready for publishing to another machine.
The description on the dotnet publish command documentation states:
dotnet publish compiles the application, reads through its dependencies specified in the project file, and publishes the resulting set of files to a directory. The output includes the following assets:
Intermediate Language (IL) code in an assembly with a dll extension.
.deps.json file that includes all of the dependencies of the project.
.runtime.config.json file that specifies the shared runtime that the application expects, as well as other configuration options for the runtime (for example, garbage collection type).
The application's dependencies, which are copied from the NuGet cache into the output folder.
dotnet build is only really useful for building on your development machine, and when used in conjunction with dotnet run against a project file.

Where is the nuget executable for dotnet core

Where does dotnet core look for a nuget executable when running restore commands?
Is there a separate executable, or are the nuget functions built directly into the cli tools?
If I already have a nuget executable on my path, can dotnet be configured to use this?
NuGet is no longer an executable that runs for a restore operation, it has become an integrated part of the build tooling and the dotnet CLI.
NuGet operations have been turned into MSBuild tasks which are run during a build. This task would then load some NuGet libraries as needed. There are other tasks that use some NuGet components as well - such as the Pack target or some tasks used to determine framework compatibility (because NuGet knows which net* / netcoreapp* / netstandard* "frameworks" are compatible with another).
The dotnet CLI also uses some library functions of NuGet to execute commands such as dotnet add package or the dotnet nuget commands.
So instead of a single nuget.exe, you will find some NuGet related DLL files inside the SDK's directory and various components used during the build or command line operations will use functionality of these.
This also means that you can't easily replace them with any nuget.exe you have on your PATH.

Resources