In my GitLab I have a multiple dotnet core projects (plugins) placed under a group named Plugins, in each one of these projects I added a CI steps to pack them in nuget packages and push them to the GitLab package registry. I followed the documentation : https://docs.gitlab.com/ee/user/packages/nuget_repository/index.html and in the .gitlab-ci.yaml I placed this config :
image: mcr.microsoft.com/dotnet/sdk:5.0
stages:
- deploy
deploy:
stage: deploy
script:
- dotnet pack -c Release
- dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text
- dotnet nuget push "bin/Release/*.nupkg" --source gitlab
only:
- master
but instead of adding a project level source (which is working for me)"$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json"
I replaced it to group level endpoint "$CI_SERVER_URL/api/v4/groups/{group-id}/-/packages/nuget/index.json" with a deploy token for the authentication because I wanted to have one source for all projects under that group, it shows this error each time the nuget cmd try to push the package
error: ERROR: This version of nuget.exe does not support updating packages to package source my-source
any ideas?
I ran into this too. I found this link https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html
The idea suggested in the link is to create a project in the group, the push all your packages to that project. It's not ideal, pushing directly to the group would be nicer, but it is a workaround that doesn't require multiple Nuget sources in Visual Studio. (as of February 2021, hopefully group-level push is supported in the future)
When pushing to project level source packages will also show in group level.
But beware, gitlab has some fundamental problems with nuget integration - if someone does not have access to one project in group, he won't be able to download any packages from that group.
So it's much better to create separate project for packages and push/consume packages there.
Related
I have solution with following projects:
Api
Application
Infrastructure
Tests
Api is WebApplication (entry point) and has ProjectReferences to libraries Application and Infrastructure.
Tests is a xunit test project and has ProjectReferences to Api / Application / Infrastructure.
I want consistent package versions both during publishing main (Api) project and during running tests.
I added following properties to Api.csproj:
<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
<RestoreLockedMode Condition="'$(CI)' == 'true'">true</RestoreLockedMode>
And it generated Api/packages.lock.json - and it seems that this file also tracks versions of dependencies of referenced projects.
Here is how I publish application (Api):
RUN dotnet restore ./Api/Api.csproj
RUN dotnet publish Api -c Release -o out --runtime alpine-x64 --self-contained true /p:PublishTrimmed=true
So if CI=true env var is set, then commands above should either restore packages according to package.lock.json or fail.
However before publishing Api I run tests like this:
dotnet test ./Tests/Tests.csproj
My question is how to ensure that exactly same package versions will be used suring testing as in Api/package.lock.json? Because if I add <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile> to Tests project then it will have separate Tests/package.lock.json file which may not be same as the one in Api/package.lock.json, right? On the other hand when Tests project references Api project then from what I understand Api/package.lock.json is ignored (when running Tests project)?
Is it possible to have one package.lock.json for solution (same for all projects in solution)?
I feel a bit bad for making this an answer and possibly getting rep votes, when mu88 beat me by 12 hours in the comments to the question, but Central Package Management is the answer. There's also a blog post about it.
Currently, neither Visual Studio, nor dotnet add package support installing or upgrading packages, so you will need to hand edit all the xml (csproj, props) files. But support should be coming in VS 2022 17.4, .NET SDK 6.0.400.
We have Xamarin.Forms solution with iOS and UWP projects. We use Azure pipelines to build the iOS project. Until yesterday everything was working fine.
Now the build fails at the NuGet Restore step with the error:
##[error]The nuget command failed with exit code(1) and error(/Users/runner/work/1/s/"MyProjectName.UWP".csproj : error MSB4057: The target "_IsProjectRestoreSupported" does not exist in the project.
We can see that the problem occurs when trying to restore NuGet packages for the UWP project on the Mac OS build host.
Image: macOS-11
Workaround will be to exclude it from the solution, but we are using it for testing purposes and this is not a good option for us.
We had the same problem yesterday for our iOS (macos-11) and Android (macos-10.15) Pipelines.
The issue 21180 for mono seems to be the root cause, which is also referenced in a pull request to update the mono version for MacOs virtual environments. Regarding to this workflow test we gave msbuild a chance, and it works.
Solution:
Instead of NuGet restore we use directly MSBuild.
- task: MSBuild#1
inputs:
solution: 'App.sln'
configuration: 'Release'
msbuildArguments: /t:restore
Looking at the log file, MSBuild ignores the UWP project. That is the behavior that NuGet had with the older mono version 6.12.0.125. Ignoring the UWP-Project is no problem, because it can only be build on windows environments.
Project "/Users/runner/work/1/s/App.sln" on node 1 (Restore target(s)).
ValidateSolutionConfiguration:
Building solution configuration "Release|Any CPU".
/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin/NuGet.targets(315,5): warning : Skipping restore for project '/Users/runner/work/1/s/App/App.UWP/App.UWP.csproj'. The project file may be invalid or missing targets required for restore. [/Users/runner/work/1/s/App.sln]
Some additional information:
The NuGet restore task uses msbuild to find all referenced packages. If that fails, it will use the packages.config file as fallback, the pipeline step will not fail.
Instead we got an error in the build steps for Android & iOS:
error NETSDK1004: Assets file
'/Users/runner/work/1/s/../obj/project.assets.json' not found. Run a
NuGet package restore to generate this file.
Updated Answer
This can be resolved using MSBuild task instead, as the collogues mentioned below.
However, in my case this still lead to the same error. After investigating the pipeline. The root cause has been identified:
Both in MSBuild#1 and XamariniOS#2 tasks
you should target the specific iOS Project and not the solution file, like so:
"
- task: MSBuild#1
inputs:
solution: 'PathToIosProject/myproject.iOS.csproj'
configuration: '$(BuildConfiguration)'
msbuildArguments: /t:restore
" - task: XamariniOS#2
inputs:
solutionFile: PathToIosProject/myproject.iOS.csproj'
configuration: '$(BuildConfiguration)'
packageApp: true
signingIdentity: $(APPLE_CERTIFICATE_SIGNING_IDENTITY)
Old Answer
We have managed to resolve the issue. The root cause of it seems to be an update to the mono framework in the MacOS pipeline build agent.
In order to fix it, we need a script for downloading and installing an older version of the mono framework as a first step of the build pipeline like so:
link to the pipeline tasks photo
This is the code of the bash scrip used in the task:
#!/bin/bash
set -ex
MONO_MACOS_PKG_DOWNLOAD_URL='https://download.mono-project.com/archive/6.12.0/macos-10-universal/MonoFramework-MDK-6.12.0.100.macos10.xamarin.universal.pkg'
mkdir -p /tmp/mono-install
cd /tmp/mono-install
mono --version
wget -q -O ./mono-installer.pkg "$MONO_MACOS_PKG_DOWNLOAD_URL"
sudo installer -pkg ./mono-installer.pkg -target /
mono --version
ls -alh /Library/Frameworks/Mono.framework/Versions/Current
I’m using the DevOps pipeline to push my dotnet build to an Octopus cloud instance.
It worked fine until I added the dotnet SDK 3 installer to the devops pipeline
- task: UseDotNet#2
displayName: 'Install .net core 3.0.100'
inputs:
packageType: sdk
version: '3.0.100'
installationPath: $(Agent.ToolsDirectory)/dotnet
The octopus part of my azure-pipelines.yml looks like this:
- task: OctoInstaller#4
inputs:
version: 'latest'
- task: OctopusPush#4
inputs:
OctoConnectedServiceName: 'XXX.octopus.app'
Space: 'Spaces-1'
Package: '$(Build.ArtifactStagingDirectory)/$(Build.DefinitionName).$(Build.BuildNumber).zip'
Replace: 'false'
I have tried every combination of OctoInstaller that I can think of including embedded and explicit version numbers. I keep getting this error message with #4.
Failed to push package. The Octo command line tool is too old to run this task. Please use version 6.10.0 or newer, or downgrade the task to version 3.*.
With OctoInstall#3 I get another error:
Failed to push package. The process '/opt/hostedtoolcache/dotnet/dotnet' failed with exit code 150
Has anyone been able to use dotnet 3 with Octopus deploy? One option
might be to install the octo extension for dotnet but I’m not sure how
you would do that from the azure pipeline. Or to use a Windows build
instead of Ubuntu.
I also noticed an error in the build step.
The specified framework 'Microsoft.NETCore.App', version '2.0.0' was not found.
- The following frameworks were found:
3.0.0 at [/opt/hostedtoolcache/dotnet/shared/Microsoft.NETCore.App]
This means that octopus must be trying to use the dotnet 2 framework.
After looking carefully at the logs, I discovered that OctopusPush was failing because it depends on dotnet 2.0.0. I added in a yml task to install that framework and now it pushes to octopus.
It might be neater to use the dotnet octo tool, as described by TrevorBrooks however I don't know how to install dotnet extensions in the DevOps Pipeline.
Use dotnet octo https://octopus.com/blog/octopus-and-netcore
The blog article explains in detail how to make this work.
For Azure Pipelines follow this article: https://octopus.com/docs/packaging-applications/build-servers/tfs-azure-devops/using-octopus-extension
Keep in mind
The Azure DevOps extension tasks require Octo to be available on the
path when executing on a build agent and must have the .net core 2.0.0
runtime or newer installed. This may not always be possible such as
with the Azure DevOps hosted agents. In order to make this work, all
Octopus tasks will automatically attempt to download and use the
latest version of Octo tools unless they're available on the build
agent as specified above. If you would like to avoid any additional
downloads or to use a specific Octo version then you can by adding the
Octo Installer task to the start of your build definition. No attempt
will be made to download Octo if the capability is detected on your
build agent.
Since VSTS has sought to bend to the popular Git source control, I have yet to see a good description of building .Net projects located in Git repositories, having project dependencies on one another.
For instance, in Visual Studio, I build a solution that includes projects with dependencies on each other. Then, in VSTS each of those .Net projects are versioned in separate Git repositories.
How, then, do you get a build on VSTS? How do you get the artifacts (read: DLLs) from one project into the project of the other?
UPDATE: 12/18/17
I took #VonC's suggestion and followed-through on a VSTS (Visual Studio Team Services) hosted Nuget package. I was able to make this work. This process makes .Net solution files and project dependencies OBSOLETE.
If you want to reuse a library, you can save the binaries as a NuGet package.
In the downstream project, you simply assign the VSTS url reference to the Nuget package to get the Nuget Restore to find/place the binaries in your build project.
You will have to download and install a Credentials tool that will allow you to push your binaries to VSTS's package location. Additionally, tell your admin to add the Packages functionality from the VSTS Marketplace.
Thanks, #VonC for the great suggestion!
Here are some helpful links:
Create and Publish the Private Nuget Package here
VSTS Marketplace Package Manager here
The idea is, for binary dependencies (DLLs) to not involve a source control tool (like Git) but a binary referential one (like Nuget)
See for instance:
"Package: NuGet"
"NuGet is now fully integrated into MSBuild"
With Visual Studio 2017 and .NET Core, we have improved the NuGet package management experience by introducing the PackageReference feature in MSBuild.
PackageReference brings new and improved capabilities such as deep MSBuild integration, improved performance for everyday tasks such as install and restore, multi-targeting and more.
First, it’s unnecessary to manage the build artifacts (such as dlls) in source control since they're the output files from the source code.
Then to add dependencies (dlls) from other repos to the parent (main) repo’s project, there usually has below options:
Option 1: manage the build artifacts as packages
As Vonc mentioned, you can manage the dlls as nuget packages, and then add nuget packages to your main repo’s project.
Option 2: git submodules
You can also treat other repos as the submodules for the main repo, and both build the projects from the submodules repos and the main repo in the build, then the main repo project can get the dependencies from the submodule repos’ build artifacts.
Commands to add a submodule for the main repo:
# In local main repo
git submodule add <URL for a submodule repo>
git commit -m 'add a submodule'
git push
Note: in VSTS build definition, you should select checkout submodules in Get Sources step.
Details about git submodules, you can refer Submodules.
Option 3: git subree (alternative way for git submodules)
Treat a branch from another repo as a subtree (a folder) in the main repo. Then build the projects both in the main repo and the subtrees, and get dependencies from subtrees for the main repo’s project.
Commands to add a subtree in the main repo:
git submodule add --prefix=submodule1 <URL for sub repo> master
git push
Then it will add a folder submodule1 with the files in the sub repo master branch, and commit the changes in the main repo.
Details about git subtree, you can refer
Git subtree: the alternative to Git submodule.
At any time, if your branch has working code with any version of dependent assemblies, I can't see any reason you need to do anything.
For example of dependencies here:
You can set dependencies in project like:
Also you can add dependencies in solution like :
You can set build order in solution too if your project has multiple project with dependencies.
As long as your current code in branch from which you are build is working (with any version of different assemblies, e.g. Classlibrary1 has version 1.0.0.0, Classlibrary2 has version 1.2.2.1 & so on but is working fine with each other after referencing) this approach will work.
Project dependencies exist for ages in Visual Studio & .Net. As long those project exist in same TFS branch You can add project dependency right in dependent project. Also you can manage Project build order in Solution.
For more complex scenarios like different repositories or branch dependencies you need to modify build workflow but it is also quite possible.
You can also refer
http://dailydotnettips.com/2015/11/25/how-to-identify-the-project-dependencies-in-visual-studio/
what I saw long time ago when I created same sample for test.
I just did a fresh install of Fedora 25 (followed by an install of dotnet core 1.1) to explore the possibilities of doing some dotnet core development in an linux environment. On attempting to add my first package I received the following error:
No executable found matching command "dotnet-add"
I have seen some similar errors in other SO posts, but nothing that seems to be helping this issue. I attached some images below of the error, dotnet version info, system path, and dotnet location.
Thanks ahead of any tips!
You have installed a version of the dotnet SDK ("CLI" / dotnet-dev package) of the preview2 time, which is still project.json based. This version of the CLI does not contain the dotnet add set of verbs. These were publicly released with the 1.0.0 version. (at the time of writing, the current version of the SDK / CLI is 1.0.4).
Depending on which instructions you followed to install the components, make sure to update to new versions of the SDK / CLI (e.g. microsoft's install instructions for Fedora).
Note that the version of the SDK / CLI is a different one that the version of the runtime(s) on the machine.
Had the same trouble running VS 2017 Version 15.3.5 found a very helpful article
http://thedatafarm.com/data-access/no-executable-found-matching-command-dotnet-ef/
My Basic problem is the tooling is split into two:
One for CLI: Microsoft.EntityFrameworkCore.Tools.DotNet
One for Powershell: Microsoft.EntityFrameworkCore.Tools