Use case
I have a C++ build pipeline that creates my application. The output of this pipeline is a directory which contains my C++ application. I didn't know where to put them, so I installed Sonatype Nexus in order to categorize, and manage these build outputs. (Just to clarify what my initial requirement is)
After installation
After installing Sonatype Nexus I can now configure my build artifact repository. In the configuration I have the choice to use a repository format, but none of the ones listed seems to fit my requirement. I can only find repository formats which seem to be used as an input for my pipeline, but not as an output. Is there anything I miss or which format would fit my need?
The raw format will allow you to store any arbitrary file type in any arbitrary folder structure.
That being said, I'm not at all a C/C++ expert, but the Conan format seems to be dedicated to packaging for your language.
I see a little star next to the format name in your screenshot indicating that this was still a community supported plugin. Hence you are running a nexus version older than 3.22.0.
You should consider upgrading since the conan format is now part of the core nexus and that your version is subject to security vulnerabilities that were fixed in 3.22.0 and 3.22.1. See release notes
Related
I'm currently starting with JFrog Artifactory. Up to now I have only been working with source code control systems not with binary repositories.
Can someone please tell how the versioning of files is done in Artifactory?
I have been trying to deploy a file, then change it and deploy it again.
The checksum has changed, so it's the new file. But it seems that the old version is gone.
So it looks like there are no version of files. If I want that do I have to do it in the filename?
I found versions related to packages.
But I was thinking to use it for other files as well.
Thanks for your help
Christoph
Artifactory, unlike a VCS system, is not managing a history of versions for a given path. When you deploy an artifacts over an existing artifact, it will overwrite it (you can block this by configuring the right permissions).
If you wish to manage permission for generic artifacts (ones which are not managed by a known package manager like npm, Maven etc.), there are a couple of options you can take:
Add the version as part of the artifact name, for example foo-1.0.0.zip
Add the version as part of the artifact path, for example /foo/1.0.0/foo.zip
Combine the 2 above approaches, for example /foo/1.0.0/foo-1.0.0.zip
Use an existing package management tool which is flexible enough to handle generic packages. Many people are using Maven to manage all types of packages beyond Java ones (it comes with its pros and cons)
From the Artifactory point of view there are a couple of capabilities you can leverage:
Generic repositories - aimed at managing proprietary packages which are not managed by a known package manager
Custom repository layout - can be used to define a custom layout for your generic repository and assist with tasks like automatic snapshot version cleanup
Properties - can be used to add version (and other) metadata to your artifacts which can used for searching, querying,resolution and more
Lastly, Conan is another option you should consider. Conan is a package manager intended for C and C++ packages. It is natively supported in Artifactory and can give you a more complete solution for managing your C libraries.
There is a issue that is present with some very specific aspect of applications I build using a Qt library provided in repositories of the system I'm using; this issue isn't present with the same numerical build of Qt that I built myself on this exact system.
I would like to query all information about build environment of Qt, it's configuration flags, etc., anything that could help me determine the difference between two build of Qt library.
I've learned about config.summary file, that is present in mine qtbase sources, but there isn't a counterpart anywhere in the system for pre-built version (I have, besides headers, installed a sources and debug information for the qt library packages).
I also tried looking for some header files that could specify, for example, what features were enabled at the build time, but could't locate anything like that (mostly because I don't know the name or some example contents of such file so that I could search for it or grep it), so if there is indeed such file, please point me to it.
I think the best reply is to quote this answer from Qt mailing list
Unfortunately it isn't that easy to figure out but the data is
available; you just need to know from where to search it :D
Configure options can be found from (qtbase) build log, from qt5.git
integration. E.g for 5.6.3
here:https://testresults.qt.io/coin/integration/qt/qt5/tasks/1505476672
and for 5.9.5 here:
https://testresults.qt.io/coin/integration/qt/qt5/tasks/1523412986
And SW details here: https://wiki.qt.io/Qt_5.6_Tools_and_Versions &
https://wiki.qt.io/Qt_5.9_Tools_and_Versions
You can find the latest builds at https://testresults.qt.io/coin/?project=qt%2Fqt5
If you look inside the "Build" section you have all the system configurations Qt is built for (column "Target") and for each configuration the parameters (column "Configure Arguments").
I prepared a symbol package successfully using dotnet pack's --include-symbols and --include-source switches. Now I wonder how to tell Visual Studio to use that package when trying to step into code of the corresponding non-symbol one.
I tried to place the symbol package to a local folder and configuring solution-level nuget.config file to use this folder as a package source. Idea is that there is maybe some name convention that looks for packages like {name}.symbols in all configured packages sources... but that doesn't work.
Oficial docs (especially the older ones) are talking a lot about configuring "Symbol Servers", but if I understand correctly, that's something different and older, right? If I wanted to set-up an internal symbol server, I wouldn't do that through NuGet. (I really don't want to set up an internal symbol server.)
They are also suggesting to push to smbsrc.net, but I can't do that with internal code, obviously. Also, I can't believe there are hard-coded URLs in .NET toolbox.
I didn't find a way to meaningfully use source included by dotnet build.
There are alternatives though:
SourceLink offers a way to configure mapping between source code build paths and HTTP locations. Unfortunately, that does not work for private repositories without a specific support for source control server authentication method. Bitbucket Server, for example, is not yet supported.
You can embed sources directly into PDBs. I will probably go that way.
Are there more examples of custom build JSON payloads beyond that available at https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API? Or perhaps more in-depth documentation on the “application/vnd.org.jfrog.build.BuildsByName+json” payload?
We have a build that produces both JAR/IVY and RPM files (and some other file types that Artifactory doesn’t really know the content of). Today, we publish those into a generic repository to keep everything together.
What would be ideal is to be able to create my own custom build using the REST API, composed of the JAR files + RPM files, so I can do licensing searches across them.
In the given example, the artifacts composed in the build are referenced by ID/name/hash for reference.
The problem with the current Jenkins/Artifactory/Gradle plugin that we use is that our build is separated amongst many smaller builds, but ultimately, are released as one. This makes making a full report somewhat difficult, and doesn’t have any way for us to easily do license checks including RPM files. We want to be able to publish one build, which contains everything we know in the build.
The current setup also has us uploading our JARs into a Maven repository, which adds time to the builds, given we are also publishing the same content into the Generic repository alongside the RPMS and other content.
Thanks!
The build info JSON is fully documented in the README of this
repository: https://github.com/JFrogDev/build-info
Which is also the repository the holds the code of the build info
engine used by the various JFrog CI/Build plugins. You can definitely
create your own BI JSON, and, if you're going to use Java to do that,
you should check out this project that demonstrates the usage of the
various build info Java APIs:
https://github.com/JFrogDev/project-examples/tree/master/build-info-java-example
Another option you may want to look into is the JFrog CLI, which
recently brought support for associating artifact
deployment/resolution with a build object and deploying it to
Artifactory. This method is completely agnostic to the file types
your build produces or the build tool you are using. Have a look at
the official documentation here:
https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-BuildIntegration
Lastly, if you are using Jenkins, the Jenkins Artifactory Plugin now
has Pipeline APIs that will allow you to collect artifacts and build
information programmatically, and even concatenate multiple build info
objects to deploy them as a single build entity to Artifactory,
which is pretty wicked.
Have a read about this here:
https://wiki.jenkins-ci.org/display/JENKINS/Artifactory+-+Working+With+the+Pipeline+Jenkins+Plugin
In all of my other .net apps my build process (a mixture of nant and custom tasks) automatically updates the [AssemblyVersionAttribute] AssemblyInfo.cs with the current build number before the call to msbuild, stamping in the build number in the version number.
I'm now working on my first BizTalk project and I'd like to do the same thing with the version numbers of the BizTalk assemblies, but I've run into trouble!
First of all the aseembly version numbers are stored in the btproj files, so I did some googling and found www.codeplex.com/biztalk which looked like the answer to my problem, but there is a deeper problem!
I have a project for my schemas and another for my pipelines, the pipelines project references my schemas project as I have a flat file dis/assemblers. The problem comes when I update the version numbers, as updating them even from within visual studio does not update the pipeline components references to the schemas.
So if I update all the version numbers manually in the VS IDE from 1.0.0.0 to 1.1.0.0, the build fails as the pipeline components flat file dis/assemblers still reference the old 1.0.0.0 version of the schemas! They don't automatically update!
Is this really a manual process of updating the version numbers of the BizTalk projects in the property pages, then building the projects and manually updating the references to them in the properties of all the pipeline components that reference them?
This means that I can't have my build process control the build number part of my version numbers!
Or is there a better method of managing the version numbers of the BizTalk assemblies?
I'm sorry to disappoint you but I've been down the exact some road I had to give up. I guess it could be possible to achieve it but it would require a lot of changes to both the binding files and other XML files (as you mentioned and even more if you have published services etc).
Maybe it could be possible to wrap all these necessary changes in a build step (a MSBuild step or similar in other build frameworks) - that would be useful!
Developer- :)
We had the similar problem and we ended up developing a small utility which would change the version number in all the projects i.e. *.csproj (asssemblyinfo.cs), *.btproj accordingly. Apart from this it would open and modify the *.btp files with the new version of schemas. In nutshell, what all you have to do is to configure this utility in your VS.net tools menu and execute it.
I guess its not very difficult to develop such utility in any .net lanagauge.
Caveat: Do not forget to save the files after updates with the same encoding as they were originally.
Cheers!
Gutted, thought that might be the case. Maybe BizTalk 2009 projects will play more nicely when updating references when changing version numbers.
I started to go through and automate it manually, and when I realised what needed to be done, I took a biiig step back when I realised just how many places I'd have to modify to get it working. Thank god for Undo Checkout.
I do have a standard C# class library included in my project (various helper functions), which i am able to update the version number of during my build process, so I'm basically using that one assembly to version the whole application. If anyone wants to know what version is in any environment, check out the version number of that one assembly.
Not ideal, but it's working.
We've done this successfully on our project - I'll see if I can get the developer of the tool to post details...
This problem arises when you perform an integration build to the latest versions of your dependent components as file references (aka schemas here).
Keep in mind that upgrading the assemblyversion must always performed manually, that way you are always in charge of changes to assemblyversions.
A possible solution to solve the buildbreaks issue is to file reference to a specific version of a dependent component build and not to the latest version and use a subst drive and a copy script to get the latest component builds.
For example:
SchemaA, assembly version 1.0.0.0
PipelineA (with pipelinecomponent XMLValidator for example), assembly version 1.0.0.0
PipelineA has a file reference to a subst drive(say R drive, which maps to a workspace D:\MyComponents) and version 1.0.0.0 of SchemaA as follows:
R:\SchemaA\1.0.0.0\SchemaA.dll.
The copy-script copies the buildoutput of SchemaA locally to your R drive.
When schema A updates to version 1.1.0.0 you don't have any issues because you still use version 1.0.0.0 and YOU have the choice to use the 1.1.0.0 version of your schema. When you want to upgrade, you have to alter your copy-script and replace the file reference to R:\SchemaA\1.1.0.0\SchemaA.dll.