What is a recommended way to write an SBT task that downloads a file from a known URL into a local file?
Older versions of SBT provided IO.download which could be used to download a URL into a local file, but in recent versions (1.2.8) this method is not available and seem to be commented out in the code.
As documented here:
import scala.sys.process._
url("http://...") #> file("somefile") !
Related
I am trying to troubleshot an error I'm getting in Blazemeter for random-csv-data-set.
The btz.log from the Blazemeter Test run shows the below line...
2021-02-10 10:14:52,518 INFO o.j.r.JARSourceHTTP: Downloading: https://search.maven.org/remotecontent?filepath=com/blazemeter/jmeter-plugins-random-csv-data-set/0.7/jmeter-plugins-random-csv-data-set-0.7.jar
Which could be the point it downloads the jar (latest one?) for the required plugins for my test.
And during the test I am getting an exception from one of the classes in the plugin jar.
I have cloned the plugin project (opensource) and started adding some debug lines and compiled a new jar version.
I was advised If I upload the plugin jar along with my tests files to Blazemeter, the uploaded jar should be taken in for the run. But I still see the exception from the old line numbers which means its still referring to the original jar version 0.7.
How can I override this with my version of the plugin?
I believe you should ask this type of questions via BlazeMeter Support as I doubt that everyone here is fully aware of what's going on there
Whatever.
Looking into Taurus documentation it looks like that Random CSV Data Set Config is being detected and automatically downloaded using JMeter Plugins Manager so in order to prevent Taurus from downloading the "vanilla" version of the plugin which doesn't contain your changes you need to add the following line to your Taurus YAML file:
modules:
jmeter:
detect-plugins: false
How will I download all the robolectric dependent jars, to avoid runtime downloading and make it offline? I need to use Robolectric.buildActivity(), which is part of 2.x.x versions.
any idea on this ?
Starting with Robolectric 2.4 they added two system properties to allow you to tell the Robolectric test runner to use local copies of the dependencies. See the Configuring Robolectric page.
The settings are:
robolectric.offline - Set to true to disable runtime fetching of jars
robolectric.dependency.dir - When in offline mode, specifies a folder containing runtime dependencies
One way to figure out which files you need to copy to the dependencyDir, is to run gradlew testDebug -i (or maybe with -d) and watch the output to see which jars are being downloaded at runtime. Then copy them to a known location on your build machine. (Another way to see which files you need, is to look at SdkConfig.java and get the dependency jars mentioned there along with their dependencies.)
For the current Robolectric 3.0-rc2, these are the files it needs:
accessibility-test-framework-1.0.jar
android-all-5.0.0_r2-robolectric-1.jar
icu4j-53.1.jar
json-20080701.jar
robolectric-annotations-3.0-rc2.jar
robolectric-resources-3.0-rc2.jar
robolectric-utils-3.0-rc2.jar
shadows-core-3.0-rc2.jar
sqlite4java-0.282.jar
tagsoup-1.2.jar
vtd-xml-2.11.jar
Copy these files to a known location, like say /home/jenkins/robolectric-files/, and then edit your build.gradle with something like this:
afterEvaluate {
project.tasks.withType(Test) {
systemProperties.put('robolectric.offline', 'true')
systemProperties.put('robolectric.dependency.dir', '/home/jenkins/robolectric-files/')
}
}
Here is how I solved it for org.robolectric:robolectric:3.0
https://gist.github.com/kotucz/60ae91767dc71ab7b444
I downloads the runtime dependencies into the build folder and configures the tests to use it - see setting the system properties.
I had this issue too, and found the cause to be the org.robolectric.Testrunner creating a org.robolectric.MavenCentral object, which declares a Maven repository using an Internet-url (Robolectric 2.3-release). Offline builds will not be able to access that url.
In my case I'm required to use a Maven repository proxy, so I replaced the url pointing to http://oss.sonatype.org with my local Maven repository proxy. That meant subclassing RobolectricTestRunner to org.robolectric.MyRobolectricTestRunner, and creating a custom MavenCentral object for it to use, and overriding the methods where RobolectricTestRunner references its private MAVEN_CENTRAL object.
The source code for RobolectricTestRunner and MavenCentral are available on Robolectric's Github page.
I used Robolectric version 3.0, and the dependency jars were downloaded from my repository, instead of sonatype.
I would like to make a permalink to the latest snapshot version of an artifact in Artifactory. If we are on 1.0-SNAPSHOT, I would like a URL that downloads the latest 1.0-SNAPSHOT JAR. I can find the latest artifact by locating the artifact on our server at http://hostname/artifactory/libs-snapshot/groupId/artifactId/1.0-SNAPSHOT/. Other than checking the timestamps, I can figure out which one if the latest by opening maven-metadata.xml and matching metadata/versioning/snapshot timestamp and buildNumber with a JAR in the same directory. This could be scripted, but ideally Artifactory already has a way to construct a permalink in this manner. Does Artifactory provide such a URL?
Doing the normal query for the entry with artifactId-1.0-SNAPSHOT.jar in the URL name should return automatically the latest snapshot.
See the doc here
One thing: This is base either on the latest creation date if no pom present, or latest creation of the pom if there are some. Mixing pom and non-pom deployment may results in strange results!
I tried using shell script and it worked for me.
Step1: Get an encrypted password for your user account by clicking on user name or create a common user. Go to using your secure password section in the following link
http://www.jfrog.com/confluence/display/RTF/Centrally+Secure+Passwords
Step 2: In your local machine create a temp folder and type this curl(may be wget for windows) command:
curl -o tmp/foo.jar --user <username>:<encrypted_password> <artifactory_url>/list/libs-snapshot-local/com/search/foo/1.0/foo-1.0-SNAPSHOT.JAR
Your foo.jar in tmp folder is latest version. If we dont give timestamp as like above, it will download latest artifact in that version. Hope this helps!
This might be helpful:
How to download the latest artifact from Artifactory repository?
Although there is no permalink ability in the free version of Artifactory, it can be scripted easily as you suggest. I have provided a quick script to do that in the referenced question.
Hope it helps.
Another portable option is to use the maven command line:
mvn org.apache.maven.plugins:maven-dependency-plugin:2.4:get -DartifactId=[artifactId] -DgroupId=[groupId] -Dversion=[version] -Ddest=[dest file]
This works for me (no search API, just direct artifact URL):
curl -O -J --user <username>:<encrypted_password> http://hostname/artifactory/libs-snapshot/groupId/artifactId/1.0-SNAPSHOT/artifactId-1.0-SNAPSHOT.jar
Basically using 1.0-SNAPSHOT in the artifact name downloads the latest version of 1.0-SNAPSHOT snapshot.
I am learning about shared objects (.so) / dynamic link libraries (.dll). Since I'm on Linux only .so files are relevant. Anyway, when I compile a shared library, I get multiple .so files - most of which are only links. Here is a screenshot:
Then I created another Qt console application project to test this SO. I had set up all the header files, copied every .so file from previous screenshot to this new project and added
LIBS += "libAritmeticnoKodiranjeDLL.so"
to project settings. If I compile the project it goes through OK. But when running my test app I get this error:
./DLLTester: error while loading shared libraries: libAritmeticnoKodiranjeDLL.so.1: cannot open shared object file: No such file or directory
Where is the problem and why do I get so many .so files?
Just do:
LIBS += -lAritmeticnoKodiranjeDLL
The specifics of the libraries prefix ("lib") and suffix (".so") will be taken care of by the linker.
Note how all except one are just symbolic links, and the actual file is the one with most precise version. This is to support different versions of the library, the symbolic links determine which exact version is used when version is not fully defined. If you are installing libs to system library folders, this is kind of important, but if you are just distributing one non-shared version along with your application, then you can use the name without version number.
When running application with custom libraries, you need to add the directory to LD_LIBRARY_PATH environment variable, so runtime linker finds them. See https://stackoverflow.com/search?q=LD_LIBRARY_PATH . For troubleshooting, you can run ldd yourprogram to see what shared libraries are actually loaded, and echo $LD_LIBRARY_PATH will let you make sure LD_LIBRARY_PATH is what you think it is.
I can definitely confirm from my current experience that QtCreator 4.0.3 with Qt 5.7 under Linux (Ubuntu 16.04) when building a shared library (using TEMPLATE = lib) then resulting target is libName.so (when CONFIG += unversioned_libname) or libNAME.so.1.0.0 (when CONFIG unchanged). However an application built with the very same environment links correctly against that library (LIBS += -Lfolder -lNAME) during linking but unfortunately during run-time it requests for linName.so.1 because "Application Output" says error while loading shared libraries: libNAME.so.1: cannot open shared object file: No such file or directory (I can see libName.so.1 in the generated binary of the app but NOWHERE else - not in Make files). I have spent 2 working days to figure this all out. The only way is to manually change the name of generated .so file to fit what application requests.
I am having the same issue described in this post on the py2app mailing list.
I have a python application that uses a sqlite database. On my machine, which has all the dependencies installed, there are no issues. However, when I bundle the application with py2app, clicking a menu that causes the database to be accessed results in this error:
Database error: Driver not loaded Driver not loaded
For the Windows installer, the files in \Qt\version\plugins\sqldrivers\*.* can be copied to \myApp\sqldrivers\*
The same files on the Mac can be found in　/opt/local/share/qt4/plugins/sqldrivers (installed via Macports).
However, copying the sqldrivers directory to my application's Resources or Frameworks directories still results in the same error.
How can I add sqlite support into my application that is built using py2app?
Turns out the pyside recipe does have a way to specify which qt-plugins you need...
options=dict(py2app={
'argv_emulation': True,
'qt_plugins' : "sqldrivers",
}
),
This puts all the sqldrivers into the right directory and setups qt.conf correctly.
have you tried what he said
in this post ?
py2app setup.py usage question
it mentioned
you need to include the sqlalchemy.dialects.sqlite as a package
I managed to get this to work as follows:
After building with py2app, inside the application's Contents directory, make a new plugins directory.
Then copy sqldrivers/libqsqlite.dylib into this plugins directory.
Afterwards, install_name_tool has to be used to change the library links in libqsqlite.dylib to point to the Qt libraries in the application's Frameworks directory rather than the system Qt libraries.