I'm attempting to use Hydra's compose API to launch runs programmatically instead of via CLI. This works for the most part. However, overriding hydra.run.dir to change the base directory doesn't seem to have the effect when using the compose API. I.e.:
with hydra.experimental.initialize_config_module(config_module=config_module):
cfg = hydra.experimental.compose(
config_name=config_name,
overrides=["hydra.run.dir=/tmp/workdir", ...],
return_hydra_config=True
)
hydra.core.hydra_config.HydraConfig.instance().set_config(cfg)
with omegaconf.open_dict(cfg):
del cfg["hydra"]
generates a DictConfig with the appropriate entry for hydra.run.dir, but the working directory is not changed.
The compose API documentation states that not using #hydra.main entails forfeiting Hydra's working directory management. Is there a workaround for this?
The Compose API is stateless and is intentionally not changing the working directory, configuring the logging or changing global state and not integrating with the command line. If you need these functionalities you should consider using #hydra.main().
As a workaround, you can call chdir programmatically on your end (with os.chdir). You will probably also need to mkdir first.
Related
I'm trying to setup multiple environments for my Vue / Firebase Project.
I have two Firebase Projects
1.) Dev
2.) Prod
The project utilizes Stripe Extension which pulls the API Key from an auto-generated file called:
firestore-stripe-payments.env
which contains:
STRIPE_API_KEY=projects/${param:PROJECT_NUMBER}/secrets/firestore-stripe-payments-STRIPE_API_KEY-xxxx/versions/latest
Where xxxx is a random 4 character string.
That line pulls the value of the key from Google Secret Manager.
Let's say Dev is 'dddd'
and Prod is: 'pppp'
The issue is that I can only define either:
firestore-stripe-payments-STRIPE_API_KEY-dddd
or
firestore-stripe-payments-STRIPE_API_KEY-pppp
At first I tried to create a new value within Google Secret Manager simply called:
firestore-stripe-payments-STRIPE_API_KEY
The thought was this should be a simple fix, and it would pull the associated API_KEY for the project currently being used.
but this causes the error:
Error: firestore-stripe-payments: Found 'projects/foo/secrets/firestore-stripe-payments-STRIPE_API_KEY/versions/latest' for secret param STRIPE_API_KEY, but this instance was previously using a different secret projects/fooo/secrets/firestore-stripe-payments-STRIPE_API_KEY-dddd.
Changing secrets is not supported. If you want to change the value of this secret, use a new version of projects/foo/secrets/firestore-stripe-payments-STRIPE_API_KEY-dddd.You can create a new version at https://console.cloud.google.com/security/secret-manager?project=fooo
Also, if there is a better place to ask this question please let me know, couldn't find the 'right' room
For this scenario, could you include a separate env (env.dev) file using the following guidelines
.env # loaded in all cases
.env.local # loaded in all cases, ignored by git
.env.[mode] # only loaded in specified mode
.env.[mode].local
For generating separate keys for each environment, I believe from your example you are using a single Stripe Extension on a single project.
Firebase Extensions can support multiple instances of an Extension per project, this will create a separate "dev" secret for you to use.
Additionally, a separate Firebase project with another "Stripe Extension" installation would be recommended to separate any concerns in development.
I have upgraded my cloud Artifactory to "7.52.0".
Prior to the upgrade I was using System Templates to deploy my pipelines.
Although after the upgrade there is still backward compatibility, The new way to deploy and use System Templates for creating new pipelines is not working for me.
From the release notes I got to this link to configure System Templates in the new way.
https://www.jfrog.com/confluence/display/JFROG/System+Templates
So in my repository A I have 2 files 'pipelines.yml' and 'values.yml'
pipelines.yml is configured as follows:
valuesFilePath: ./values.yml
Include:
template: myTemplates/TestTemplate/1.0.0
My values file contains values for the TestTemplate.
Then I go to https://example.jfrog.io/ui/admin/pipelines/pipelineSources and I try to create a new pipeline from repository A.
Looking at https://example.jfrog.io/ui/pipelines/myPipelines/myPipelines I don't see any pipeline created from the template.
Is that the right way to implement the new System Template?
I have also made sure that the templates are in the Artifactory by checking:
https://example.jfrog.io/ui/pipelines/templates
and also in the Artifactory directory tree.
Currently I am using the REST API in order to CRUD my Template Sources(https://example.jfrog.io/ui/pipelines/sources) and also use the REST API to create a new pipelines sources from a system template (apparently this is the old way).
As after the upgrade creating a source pipeline doesn't sync the old/new templates nor does it create a new pipeline from a system template that is located in the Artifactory.
You need to use the syntax documented in the Global template link.
Using the "jfrog/PublishTemplate" global template documentation
https://www.jfrog.com/confluence/display/JFROG/Global+Templates . I have noticed that in order to create and upload a system template you need to use the following syntax:
valuesFilePath: ./values.yml
include:
template: jfrog/<global_template_name>/<template_version>
According to the system template documentation this is the syntax that got me confused:
valuesFilePath: ./values.yml
Include:
template: jfrog/PublishTemplate/1.0.0
So I have used capital "I" instead of small "i" and bad indentation in order to create a new pipeline from my system template, which failed.
You use the Global template "PublishTemplate" for uploading your system template into your artifactory.
And then use the uploaded templates in order to create your new pipelines.
I'm adding security for my API keys with the Meteor package dotenv: https://github.com/okgrow/meteor-dotenv
as per instructions, I:
1. created a file named ".env" in my root
2. entered two keys in the style "THIS_KEY=BLAH12345" in the file
3. made a meteor call function returning process.env.THIS_KEY for the client side to use.
I'm getting Referenceerror: process.env is not defined. For just plain node there are a lot of answers out there, but not so for Meteor. Did I name my file incorrectly? Need to use a Meteor command to activate something?
I ran into this same problem, and have made it work by putting the
var secretThing = process.env.SECRET_THING
server side, inside the if (Meteor.isServer) and then passing the variable as a parameter to the method that needs to use the secret thing.
Meteor.call("apiCall", secretThing);
Then the receiving method looks like this:
apiCall: function (secretThing) {
console.log(secretThing);
}
DotEnv is designed to read environment variables from a .env file, normally located in the root of your NodeJS application.
DotEnv just makes the variables it finds in the .env file available with the rest of the host systems environment variables through process.env
The problem is, I don't believe meteor supports .env files. This is probably due to the age of the application. I personally believe they should make rectifying this a priority.
The only way you can use process.env in a meteor app is to set the variables you want to use on the host system. If you are using linux it would be something like
export DB_PASSWORD=12345
You should then be able to use process.env to read DB_PASSWORD when your application is running.
// You can only run process.env in server code
if (Meteor.isServer) {
const myDBPassword = process.env.DB_PASSWORD
}
I've seen api-paste.ini as a conf file after installing openstack.
It looks like substituting some prefixes for python implementation but have no clue about this.
Here, my questions are:
What script is it?
it looks like very bizarre grammar like the following:
[composite:metadata]
use = egg:Paste#urlmap
/: meta
How does it work within python script?
See documentation for Paste Deploy.
The api-paste.ini is the configuration for the above web-services framework. Paste.deploy allows you to separate concerns between writing your application and middleware/filters from the composition of them into a web service. You define you WSGI applications and any middleware filters in the configuration file then you can compose pipelines which include the middleware/filters you want into your web-service, e.g. authentication, rate-limiting, etc.
You want to temporarily remove authentication, take it out of your pipeline and restart your web service.
The declaration above is declaring a composite application, but with only one application binding (a bit unnecessary - normally you would expect to see more than one binding, e.g. for different versions of the application). The WSGI application app:meta will be bound to / and you should have a declaration of app:meta later in the file. The implementation of the composite app is declared via the use and the egg:Paste#urlmap is a simple reference implementation.
You load this in your program with paste.deploy.loadwsgi.loadapp().
There is a proposal/recommendation(?) to move away from Paste Deploy/WebOb to WSME/Pecan see OpenStack Common WSGI
Is there a way in Meteor, to add your own commands to the meteor command for the given project? So that on the server side you could have some commands similar to Django's manage commands. Like command for importing/updating data, maybe cron jobs, things like that? Thing is, I would like commands to have access to all the libraries and environment Meteor provides, especially so that I am sure that data stored in the database is compatible with Meteor.
At the moment this isn't directly possible without an edit of meteor's source. If you're using meteorite you can have a localized meteor version which won't affect the rest of your meteor apps.
You might have to add a custom function over at /tools/meteor.js in this forked/modified meteor.
with:
Commands.push({
name: "yourcustomcommand",
help: "..",
func: function (argv) {
//Custom stuff
});