Suppose I have a project that uses a third-party library, e.g. scopt which is published under an MIT license.
When using sbt-native-packager to generate a distributable tarball, zip, deb, etc. I am redistributing a copy of all of my dependencies. When redistributing, I am required, by the licensing conditions of my dependencies, to include a copy of that license / copyright notice / licensing conditions (in the license usually called redistribution in binary form). By the way, sbt-native-packager also includes a copy of the Scala standard library as well, which also does not include the appropriate licensing conditions / copyright notice according to its 3-clause BSD style license.
By default, sbt-native-packager does not include such license copies / copyright notices / ... in the generated distribution.
What I am looking for is a convenient way to automatically include all licenses / copyright notices of all (including transitive) dependencies.
... or else, I suppose, I would be breaking the law if I published the raw tarball generated by sbt-native-packager as is on e.g. the GitHub release page, wouldn't I?
edit
I guess, it would be really hard to have a general way for sbt-native-packager to inject all licenses/copyright notices into the distributions created by it, because neither the jar archive nor ivy, maven, sbt, etc. have first class support for adding the license and copyright notice to the archive.
The only recommendation I have seen thus far is by the Apache Software Foundation. They recommend to put the license at the top of the jar archive (cointaining the bare license file) and include a NOTICE file as well (containing all the copyright notices). I deem this recommendation good.
I guess, support for handling all this legal stuff should be better to begin with. Build tools like sbt should support this better and much more visible, e.g. check all generated jar archives and if they do not include both LICENSE and NOTICE files error out of tasks like package.
To answer your question: there's AFAIK no way of handling licences in a general way in sbt-native-packager. E.g. putting them in a separate directory or listing them.
Having said that, I think that this is not really necessary for a couple of reasons. I'm no advocate, so this is just my understanding of this.
If you have your git tarball there is none of your dependencies included. Only a build file that declares them. So you are not distributing any code. So this should be fine
When you build an executable binary the you will include not only the code, but the whole jar, which normally includes all licenese and copyright hints. Technically you have included everything.
I haven't seen any issues around that topic or a way other build tools like maven/gradle handle this. What would a solution look like? If this is a problem?
Update
I found this sbt-license-report plugin which should generate the information you need. If this is what you are looking for, I'm happy to merge a pull request updating the sbt-native-packager docs :)
Related
I recently implemented serverside-rendering react app with code splitting using loadable-components
But it seems that loadable-components itself dependent on webpack, since loadable replaces jsonp_callback with its own reporter.
So what are the alternative options that we can use when using other bundlers like rollup, esbuild?
Do we have to manually walk through the react tree to pre configure which chunk is needed on which component unless there is no specific bundler targeted library like loadable-components when implementing code splitting on serverside rendering?
Is tsconfig typeRoots unnecessary?
First, let us consult the compiler options reference:
The documentation for typeRoots states (emphasis mine):
By default all visible #types packages are included in your compilation. Packages in node_modules/#types of any enclosing folder are considered visible. For example, that means packages within ./node_modules/#types/, ../node_modules/#types/, ../../node_modules/#types/, and so on.
If typeRoots is specified, only packages under typeRoots will be included.
That second line is important: if you don't set typeRoots then tsc defaults to looking for directories under node_modules containing #types in their directory names.
(The documentation doesn't say if it chooses node_modules because of the moduleResolution parameter though. (I suspect I'd need to dig-in tsc's source-code to find out for sure).)
If you do set a value for typeRoots than that overrides tsc's node_modules/**/#types* lookup logic and it will then only look in the specified directories.
As I know, I have to specify the path of the above file into typeRoots option of tsconfig file since typeRoots defaults to look into the node_modules/#types.
Not necessarily. You could also add your extra typings files' locations to the paths parameter and leave the typeRoots parameter blank/un-set, which means tsc will retain the "node_modules/#types-and-ancestor-walking behavior" but will see your .d.ts files just fine.
This scenario is mentioned in this TypeScript GitHub thread: https://github.com/microsoft/TypeScript/issues/13581
So if you're asking about your specific local environment on your machine: and assuming that you're sticking with the normative, typical (I dare say mainstream) TypeScript working idioms (such as using npm) then yes: you can remove the typeRoots parameter because tsc's (current) default behavior is to look for node_modules directories in the same location as your tsconfig.json.
(I understand that VS Code might also be pulling some strings behind-the-scenes to make tsc "aware" of your project files and dependencies and for its language-server process - but that shouldn't matter as you'll notice tsc should work identically from the command-line outside of VS Code).
If you're asking about the fundamental necessity of the typeRoots compiler option, and supposing that you're thinking "that because practically everyone is using npm and node_modules then why is the TypeScript team spending their time supporting unusual development configurations?"_ - well, for many very good reasons: tools shouldn't be dependent on other tools controlled by third-parties1: Consider the possibility that the npm ecosystem and/or NodeJS software could fall out of fashion overnight and then we'd be stuck with tsc's defaults still using node_modules when everyone is rockin' some new cool JS environment: there'd be headaches for many years to resolve the mess (not that the JS ecosystem isn't a mess as it is).
And there are many good reasons to not use npm and node_modules: people could be using TypeScript in an environment without internet access (think: secure software development, the defence industry, national secrets, etc) - those people in those situations might have a network share full of approved or known-trustworthy libraries that won't be using node_modules's naming convention - in which case if those people want to use d.ts files they'll need to manually configure the typeRoots parameter for themselves.
1 I'm aware that npm (which is legally separate from NodeJS, btw) is maintained by npm Inc, which is a subsidary of Microsoft (by way of being acquired by GitHub, also a Microsoft property), so having tsc depend on npm shouldn't be a problem - but that's a very recent thing: Microsoft only acquired npm 18 months ago in March 2020 - and Microsoft could very well spin-off npm Inc - or run it into the ground and everyone switches to yarn. So regardless of the end legal owners of whatever tooling is currently popular, you don't want unnecessary dependencies like that.
I'm currently starting with JFrog Artifactory. Up to now I have only been working with source code control systems not with binary repositories.
Can someone please tell how the versioning of files is done in Artifactory?
I have been trying to deploy a file, then change it and deploy it again.
The checksum has changed, so it's the new file. But it seems that the old version is gone.
So it looks like there are no version of files. If I want that do I have to do it in the filename?
I found versions related to packages.
But I was thinking to use it for other files as well.
Thanks for your help
Christoph
Artifactory, unlike a VCS system, is not managing a history of versions for a given path. When you deploy an artifacts over an existing artifact, it will overwrite it (you can block this by configuring the right permissions).
If you wish to manage permission for generic artifacts (ones which are not managed by a known package manager like npm, Maven etc.), there are a couple of options you can take:
Add the version as part of the artifact name, for example foo-1.0.0.zip
Add the version as part of the artifact path, for example /foo/1.0.0/foo.zip
Combine the 2 above approaches, for example /foo/1.0.0/foo-1.0.0.zip
Use an existing package management tool which is flexible enough to handle generic packages. Many people are using Maven to manage all types of packages beyond Java ones (it comes with its pros and cons)
From the Artifactory point of view there are a couple of capabilities you can leverage:
Generic repositories - aimed at managing proprietary packages which are not managed by a known package manager
Custom repository layout - can be used to define a custom layout for your generic repository and assist with tasks like automatic snapshot version cleanup
Properties - can be used to add version (and other) metadata to your artifacts which can used for searching, querying,resolution and more
Lastly, Conan is another option you should consider. Conan is a package manager intended for C and C++ packages. It is natively supported in Artifactory and can give you a more complete solution for managing your C libraries.
So I've been trying to do some AWS lambda with scala and sbt. And one of the recommendations is
Minimize your deployment package size to its runtime necessities. This
will reduce the amount of time that it takes for your deployment
package to be downloaded and unpacked ahead of invocation. For
functions authored in Java or .NET Core, avoid uploading the entire
AWS SDK library as part of your deployment package.
I've been using sbt-assembly to create fat jars for my code and dependencies, but it seems like sbt-assembly will package all library dependencies when I only use like 10% of the aws-core library which adds a lot of content. Was wondering if there is something I can do to cut down on the number of dependencies to what is actually imported in my code (and their dependencies).
As far as I know, there is no direct and safe way to selectively contains the dependent classes in a fat jar generated by the sbt-compile plugin.
First of all, you should understand that sbt plugins just provide a settings and jar files required to invoke methods in your project. It means that the dependent plugin is brought to your project with the pre-compiled jar file, which is determined by the version that you specified on your build setting (e.g., plugins.sbt in your project dir).
For example, the jars of sbt-assembly are brought to your project from this link when you specify that you want to use the sbt-assembly (although it is brought to your project when you use recent sbt version by default).
Therefore, at least you may have two choices to shrink your jar files.
Compile the jar file from the scratch
For the sbt-aws, its source code is provided on this link, so you may selectively compile the source codes to get the classes that your program is going to use.
Use the tool for shrinking jar file
There are several tools to shrink your jar file based on the dependencies. The most popular tool is proguard; it seems that there is a proguard support for sbt.
Warning
As mentioned in another stack overflow answer, selectively choosing some classes from the jar may cause your program crash depending on the input value and several other conditions. You've said that only 10 percent of the jar file is used, but you cannot ensure whether other classes are required from your code and library that your project depends on. When you use the tool for helping you to shrink the jar file, be careful when the program is security critical one.
Why does every source package that uses a makefile come with a ./configure script, what does it do? As far as I can tell, it actually generates the makefile?
Is there anything that can't be done in the makefile?
configure is usually a result of the 'autoconf' system. It can generate headers, makefiles, really anything. 'Usually,' since some are hand-crafted.
The reason for these things is to allow source code to be compiled in disparate environments. There are many variations on Unix / Linux, with slightly different system headers and libraries. configure scripts allow code to auto-adapt.
The configure step is a sort of meta build. It generates the makefile that will work on the specific hardware / distribution you're running. For instance it determines the name of the C or C++ compiler and embeds that in the makefile.
The configure step will also frequently take a set of parameters, the values of which may determine what libraries need to be linked against. For instance if you compile Apache HTTP with SSL enabled it needs to link against more shared libraries than if you don't. Since linking is handled by the makefile, you need an extra step to create a custom makefile (rather than requiring the make command to require dozens or hundreds of options.
Everything can be done from within the makefile but some build systems were implemented otherwise.
I don't personally use configure files for my projects but I admit I mostly export Erlang & Python based projects.
I don't think of the makefile as a script, I think of it as an input to the make utility.
The configure script (as its name suggests) configures the makefile, including as you say resolving dependencies.
If only from the idea of avoiding self-modifying code, the things in the configure script don't really belong in the makefile.
the point is that autoconf autohdr automake form an integrated system the makes cross platform building on unix relatively str8forward. THe docs are really bad and there are lots of horrible gotchas but on the other hand there are a lot of working samples
When I first came across this stuff I thought - "ha I can do that with a nice clean makefile" and proceeded to rework the source that way. Big mistake. Learn to write and edit configure.ac and makefile.am files, you will be happy in the end
To answer your question. Configure is good for
is function foo available on this platform and if so which include and library do I need
letting the builder choose if they want feature wizzbang included in a nice simple consistent way
I'm mostly a spoiled Windows + Visual Studio (or Borland C++ or whatever, in the past) developer. Although my first contact with Unix was around 20 years ago, and I've used Linux on-and-off for some years, I have only a very limited idea of how to set up a build on a *nix system.
For example, I'm OK with the basics of make - I can get a number of files to compile and link. But I don't really know how to set things up to cope with multiple configurations - how to get all the object files and targets for the release version to go to different folders from the debug version etc etc. Yes, I can RTFM and improvise something, but it's a fair guess that I'd improvise something stupid, overcomplex, fragile and WTF, where it'd make so much more sense to copy a common convention if only I knew what the common conventions are.
Also, I can run a configure script, and I'm vaguely aware that they're associated with autoconf, whatever that is, but I have little idea if/why/how I should set this kind of stuff up in my own projects.
Hopefully, this is enough to give the general idea of what I'm looking for. Of course I could ask/search specific questions here, but that assumes I know all the right questions to ask which I almost certainly don't. So - any pointers?
EDIT
Just thought I'd update this with some longer-term experience.
I tried using premake for a while, but couldn't live with it in the long run. In substantial part there's a dislike of Lua behind that.
Now, I'm using cmake. It generates makefiles/visual studio projects/whatever. It has (so far) handled everything I've needed to do, including support for unit testing and custom build steps. And as I got used to the cmake way of doing things, I found it was a good way, allowing me to easily use multiple sets of tools at once - I can be checking test coverage in MinGW GCC while simultaneously debugging in Visual Studio.
That reveals, of course, that I'm still mostly working in Windows - but switching back and forth is easier than ever.
The downsides of cmake...
Although it generates makefiles/whatever, it can't really be seen as a makefile generator. The resulting makefiles are dependent on cmake being installed. To be honest, I don't really understand why they don't drop makefiles altogether for makefile platforms and just do the building directly, slightly reducing the potential for problems.
It wasn't easy to get started.
The second point has mostly been resolved by asking questions here...
How do I fix this cmake file? - problem linking to imported library
How to apply different compiler options for different compilers in cmake?
For the cmake "include" command, what is the difference between a file and a module?
How to adapt my unit tests to cmake and ctest?
I'd strongly suggest using one of the newer cousins of Make instead of autoconf or makefiles for smaller projects. One of the best ones for you (and the one I mostly love) could be premake4. Why do I suggest it? Because it's extremely simple to use, yet quite powerful, and capable of producing GNU Makefiles, Visual Studio projects, Code::Blocks projects and many more. And the premake files are very clear and readable, using the Visual Studio nomenclature that you're already familliar with.
Here's an example:
-- A solution contains projects, and defines the available configurations
solution "MyApplication"
configurations { "Debug", "Release" }
-- A project defines one build target
project "MyApplication"
kind "ConsoleApp"
language "C++"
files { "inc/**.h", "src/**.cpp", "main.cpp" }
configuration "Debug"
defines { "DEBUG" }
flags { "Symbols" }
configuration "Release"
defines { "NDEBUG" }
flags { "Optimize" }
I would stay away from autoconf until you actually need it as it is pretty complex to use, and in most small projects, make is all you need...
I don't have a tutorial, but I will give you an extremely amazing(and portable between both BSD and GNU make) makefile to start with for small projects. I dug it out of the original BSD 4.3 assembler(as)
HDRS = project.h
OBJS = main.o
LDFLAGS =
project: ${OBJS}
${CC} ${LDFLAGS} ${OBJS} -o project
.c.o: ${HDRS}
${CC} ${CFLAGS} -c $*.c
clean:
rm -f *.o ${OBJS} project *.core a.out errs core
just replace Project with your projects name and such..
edit:
technically, you need a BSD 3 clause license for Berkeley... as per:
* Copyright (c) 1982 Regents of the University of California.
* All rights reserved. The Berkeley software License Agreement
* specifies the terms and conditions for redistribution.