I've build an OSS project https://github.com/datlinq/scalafiniti
The Travis-ci pipeline works perfectly, but for 1 final step.
I followed these guides:
http://www.scala-sbt.org/0.13/docs/Using-Sonatype.html
http://www.scala-sbt.org/sbt-pgp/usage.html
https://github.com/xerial/sbt-sonatype
Locally I got all steps working fine and actually published to the Nexus.
In the .travis.yml I import the key before install (Encrypted in travis.ci env)
before_install:
- echo "$PGP_SECRET" | base64 --decode | gpg --import
- echo "$PGP_TRUST" | base64 --decode | gpg --import-ownertrust
The $PGP_PASS is also encrypted in Travis env and available for the build.sbt
I checked it actually gets the key in this command
pgpPassphrase := sys.env.get("PGP_PASS").map(_.toArray)
Now if Travis runs the command
sbt publishSigned
It still prompts for a passphrase for my key
You need a passphrase to unlock the secret key for user:
"com.datlinq.datalabs (Key for Datalabs OSS) "
2048-bit RSA key, ID 305DA15D, created 2017-09-01
Enter passphrase:
I don't know what I should do to make this work
This moment in time is captured:
code:
https://github.com/datlinq/scalafiniti/tree/0d8a6a92bf111bae2a1081b17005a649f8fd00c9
build log:
https://travis-ci.org/datlinq/scalafiniti/builds/271328874
So, the reason it prompted for a password and ignored all sbt based configurations was due to the fact the build script used the local gnupg installation instead of the one packaged with sbt-pgp (bouncy castle).
The local gpg wants you to manually enter the password the first time. A bit hard using TravisCI
So the solution was ignore the local gpg and use the bundled one, that uses the pgpPassphrase setting
Looking back to the documentation:
http://www.scala-sbt.org/sbt-pgp/usage.html
In one of the first lines it actual says:
If you’re using the built-in Bouncy Castle PGP implementation, skip this step.
The first step towards using the GPG command line tool is to make sbt-pgp gpg->aware.
useGpg := true
So the solution was to set useGpg := false
For more details look at the current repo:
https://github.com/datlinq/scalafiniti
Or check this blog (which I found later) https://alexn.org/blog/2017/08/16/automatic-releases-sbt-travis.html
Related
I have keystore in windows as below -
secretkeys.skr
publickeys.pkr
I want to add new pgp pubic keys to above keystore. Can someone help me with the command.
I tried using some tool and gpg commands but no luck as the keys are not updating in the above files.
We use below command to list and encrypt
pkzipc -listcertificates=AddressBook
pkzipc -add -archivetype=pgp -cryptalg=AES,128 -recipient="!encryptionKey!" "!encrptedFileDestination!\%%~nxA" "%%~fA"
Can someone help with any command or tool where I can set the keyring to above file and import the keys to that store.
Thanks,
Arpit
I am trying to establish a SFTP connection in Airflow 1.10.14 with the SFTPOperator from airflow.providers.sftp.operators.sftp or airflow.contrib.operators.sftp_operator.
The contrib operator and the providers package are equivalent ("providers" packages are backported from Airflow 2.0 which do not make use of the contrib operators anymore), and depend on the same Python modules: paramiko, pysftp, and sshtunnel.
My pip freeze:
paramiko==2.7.2 (latest release)
pysftp==0.2.9 (latest release)
sshtunnel==0.1.5 (latest release is 0.4.0)
It works fine with a simple user/password and a private key without a passphrase but it fails when I use an encrypted key protected by a passphrase:
"ERROR - private key file is encrypted" when I set "private_key_passphrase" param alone in the connection
"ERROR - Authentication failed" when I set the "password" alone or both "password" and "private_key_passphrase".
Note that it works well in all cases with the SSHOperator (in this case, the key passphrase is set in the "password" param), thus I believe the problem is in the pysftp module.
Thanks for your help.
How can I get rails to generate a master key? I'm wanting it to decrypt the credentials file in a production environment.
This is how I tried to generate a new key:
config$ rm credentials.yml.enc
config$ EDITOR=vim rails credentials:edit
File encrypted and saved.
In case anyone is looking for a way to generate an encryption key that is exactly 32 characters:
$ bundle exec rails secret | cut -c-32
=> 8d000d7e7233facfc8300b9a3fe0421e
Then save that to your .env as:
# Generated by:
# bundle exec rails secret | cut -c-32
# If one is already setup, you may need to ask someone for it.
RAILS_MASTER_KEY=8d000d7e7233facfc8300b9a3fe0421e
So it turns out I had RAILS_MASTER_KEY set as an environment variable from testing. Rails was referencing that instead of generating a new key. I unset the environment variable and it's now working.
Using the jfrog CLI (jfrog rt s) I can dump file information of my repo to stdout, but this information does not contain the stored checksum. I see a similar question
"Artifactory CLI - Jfrog - How to get binary Hash code (SHA1, SHA256) through jfrog CLI" but the answer is only about searching for a specific checksum. Not being very familiar with jfrog at all, can someone suggest a simple method (has to use jfrog please) for dumping the checksum info for all or a specific file in the repo?
Since version 1.36.0 of JFrog CLI, the search command also returns the SHA1 and MD5 of the files. For example:
$ jfrog rt s repo/path/file
The jfrog rt search command theoretically supports returning sha256 sums if you're using jfrog-cli version 1.36.0 or greater, Artifactory server 5.5 or greater, and a correctly configured database.
Under the hood, the jfrog rt search command utilizes AQL to generate a query which it sends to the server. The default query performs items.find().include(*), which returns all of the supported fields. I guess if the database isn't set up right then SHA-256 sums aren't supported (this seems to be the case at my workplace).
Fortunately, there's an alternative which works even on old versions of the jfrog-cli (I've tested this with 1.26.2). This involves utilizing the jfrog rt curl command to directly grab the metadata from the server. Note that the jfrog rt curl command doesn't support the standard --url --access-token or --apikey parameters, so you'll need to configure a connection to the server using jfrog rt c first (don't forget to use --interactive=false if you're automating this). Once you've done that, the magic incantation which you're looking for is:
jfrog rt curl -XGET "api/storage/your_repo/your_file"
This will return a JSON blob like the following:
'{
"repo" : "your_repo",
"path" : "/your_path/your_file",
"created" : "2020-07-21T21:28:20.663Z",
"createdBy" : "token:your-token",
"lastModified" : "2020-07-21T21:28:27.277Z",
"modifiedBy" : "token:your-token",
"lastUpdated" : "2020-07-21T21:28:27.287Z",
"downloadUri" : "https://your_artifactory_url/artifactory/your_repo/your_path/your_file",
"mimeType" : "application/x-gzip",
"size" : "1198168",
"checksums" : {
"sha1" : "5e288fe94da1fed0b4ce6695c7a984426e9f5a78",
"md5" : "a012711241ba3a5bd4a04e833001d490",
"sha256" : "d22e76b6cc0b4e2e29076d08f8209dec2b7d9c28e71a13d989175a1272ac3da7"
},
"originalChecksums" : {
"sha1" : "5e288fe94da1fed0b4ce6695c7a984426e9f5a78",
"md5" : "a012711241ba3a5bd4a04e833001d490",
"sha256" : "d22e76b6cc0b4e2e29076d08f8209dec2b7d9c28e71a13d989175a1272ac3da7"
},
"uri" : "https://your_artifactory_url/artifactory/api/storage/your_repo/your_path/your_file"
}'
The originalChecksums are from when the artifact was first uploaded. If the artifact has been tampered with on the server then the regular checksums may be different. For this reason I'd recommend validating against the originalChecksums unless you're operating in an environment where the same artifacts are expected to be overwritten.
If you're looking for a quick and dirty way to extract the returned checksums from the JSON blob then try this ugly hack I threw together in bash (note that this won't work if you collapse the whitespace first):
#!/bin/bash
...
checksums=($(sed -n -E "s/^\\s+\\\"sha256\\\"\\s:\\s\\\"(.*)\\\"\$/\\1/p" <<< "$response"))
checksum="${checksums[0]}"
original_checksum="${checksums[1]}"
If you have the option, I'd recommend using a more robust json parser instead.
Here's a transcript so far:
$ sbt new lagom/lagom-scala.g8
[info] Loading global plugins from /Users/abrahma/.sbt/1.0/plugins
[info] Set current project to lagomlife (in build file:/Users/abrahma/Bitbucket/Practice-Scala/LagomLife/)
[info] Set current project to lagomlife (in build file:/Users/abrahma/Bitbucket/Practice-Scala/LagomLife/)
ssh://git#github.com/lagom/lagom-scala.g8.git: Auth fail
I've verified authentication with
GitHub:
$ ssh -T git#github.com
Hi agam! You've successfully authenticated, but GitHub does not provide shell access.
Also verified that I can access the repo in question (i.e. I can do the following, in a separate location):
git clone ssh://git#github.com/lagom/lagom-scala.g8.git
Edit: fwiw I'm able to work around whatever the root cause is here:
git clone ssh://git#github.com/lagom/lagom-scala.g8.git
g8 file:///Users/abrahma/tmp/lagom-scala.g8
I ran into this problem as well, and solved it by removing the following from my ~/.gitconfig:
[url "git#github.com:"]
insteadOf = https://github.com/
Are you using OpenSSH 7.8 or newer, and have you recently created your private key?
If so you might be running into the issue described in “Invalid privatekey” when using JSch aka jsch#129.
The root cause was discovered to be the ssh private key mismatch.
The question has a workaround to convert the key file to an older format if that's the error you're seeing.
You need to add your SSH key to the agent:
ssh-add ~/.ssh/id_rsa