I am trying to connect to mysql database using MySqlHook. Under Admin -> Connections I have defined a new connection of the type mysql with the name myappname_db. I have used this in my code as drupalHook = MySqlHook(conn_name_attr='myappname_db')
However, when I run the dag in local, I see that it picks up the built in default credential from Admin -> Credentials i.e. mysql_default instead of myappname_db
To rectify this do I need to update any setting in airflow.cfg or any other config.
Thanks.
The MySqlHook uses an attribute mysql_conn_id to store the connection id.
conn_name_attr refers to the name of the attribute (mysql_conn_id in this case), which is fetched dynamically in the MySqlHook.
This is quite a long question, but there's quite a lot to it.
It feels like it should be a reasonably common use case, so I'm hoping the Stack Overflow community can provide me with a 'best practice in Symfony2' answer.
The solution I describe below works, but there are several consequences I'd like to avoid:
In my local dev environment, if I have used the wrong db connection the test will work in dev but fail on production
The routes of the ADMIN API are accessible on the PUBLIC API url, just denied.
If I have a mirror of live in my dev environment (3 separate checkouts with the corresponding parameters.yml file) then the feature tests for the other bundles fail
Is there a 'best practice in Symfony2' way to set up my project?
We're running a LAMP stack. We use git/(Atlassian) stash for version control.
We're using doctrine for the ORM and FOS-REST with OAuth plus symfony firewalls to authenticate and authorise the users.
We're committed to use Symfony2, so I am trying to find a 'best practice' solution:
I have a project with 3 applications:
A public-facing API (which gives read-only access to the data)
A protected API (which provides admin functionality)
A set of batch processes (to e.g. import data and monitor data quality)
Each application uses a set of shared models.
I have created 4 bundles, one each for the application and a 4th for the shared models.
Each application must use a different database user to access the database.
There's only one database.
There's several tables, one is called 'prices'
The admin API only must be accessible from one hostname (e.g. admin-api.server1)
The public API only must be accessible from a different hostname (e.g. public-api.server2)
Each application is hosted on a different server
In parameters.yml in my dev environment I have this
// parameters.yml
api_public_db_user: user1
api_public_db_pass: pass1
api_admin_db_user: user2
api_admin_db_pass: pass2
batch_db_user: user3
batch_db_pass: pass3
In config.yml I have this:
// config.yml
doctrine:
dbal:
connections:
api_public:
user: "%api_public_db_user%"
password: "%api_public_db_pass%"
api_admin:
user: "%api_admin_db_user%"
password: "%api_admin_db_pass%"
batch:
user: "%batch_db_user%"
password: "%batch_db_pass%"
In my code I can do this (I believe this can be done from the service container too, but I haven't got that far yet)
$entityManager = $this->getContainer()->get('doctrine')->getManager('api_public');
$entityRepository = $this->getContainer()->get('doctrine')->getRepository('CommonBundle:Price', api_admin');
When I deploy my code to each of the live servers, I put junk values in the parameters.yml for the other applications
// parameters.yml on the public api server
api_public_db_user: user1
api_public_db_pass: pass1
api_admin_db_user: **JUNK**
api_admin_db_pass: **JUNK**
batch_db_user: **JUNK**
batch_db_pass: **JUNK**
I have locked down my application so that the database isn't accessible (and thus the other API features don't work)
I have also set up Symfony firewall security so that the different routes require different permissions
There's also security in the apache vhost to deny access to say the admin api path from the public api directory.
So, I have secured my application and met the requirement of the security audit, but the dev process isn't ideal and something feels wrong.
As background:
We have previously looked at splitting it up into different applications within the same project (like this Symfony2 multiple applications and api centric application. Actually followed this method http://jolicode.com/blog/multiple-applications-with-symfony2) , but ran into difficulties, and in any case, Fabien says not to (https://groups.google.com/forum/#!topic/symfony-devs/yneojUuFiqw). That this existed in Symfony1 and was removed in Symfony2 is enough of an argument for me.
We have previously gone down the route of splitting up each bundle and importing it using composer, but this caused too many development overheads (for example, having to modify many repositories to implement a feature; it not being possible to see all of the changes for a feature in a single pull request).
We are receiving an ever growing number of requests to create APIs, and we're similarly worried about putting each application in its own repository.
So, putting each of the three applications in a separate Symfony project / git repository is something we want to avoid too.
I build some kind CMS, where I can change parameters (parameters.yml) from my admin panel. User (admin) will be able to change database connections (name, host etc.). But I need to handle errors after that and let him rollback or something like that.
What I need:
- make connection to database after parameters change
- check if exists, etc..
What I can do:
- make new instance of \PDO and try / catch etc.
What I prefer:
- some kind symfony/doctrine help for that.
Is there something I can to instead new PDO? OR meyby my thinkinh way is wrong? :)
You can use Doctrine DBAL Connection factory for that. In your controller (or service) you can get the "doctrine.dbal.connection_factory" service from container and create a new connection with the new parameters:
https://github.com/doctrine/DoctrineBundle/blob/master/ConnectionFactory.php#L51
and then try to connect with it:
https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/Connection.php#L350
Here you can catch the errors that may occur.
Hope it helps
Currently I am running puppet from root user. I want to manage Redis using puppet. I found redis module for puppet. I want to run redis-server using "redis" user. So is it possible in puppet to start command/process/script with different user ?
The best way to accomplish this is to make sure your service management framework is starting the service using the correct user.
If you instead want to use Puppet to start the service directly, which is not best practice, then you can use the user parameter of the exec resource type. The documentation for the exec resource type is located at: http://docs.puppetlabs.com/references/latest/type.html#exec
I am getting this error while connecting to IBM MQ. I know that this is because of privileges, but is there any way just to check the connection with IBM MQ?
Please suggest.
The 2035 suggests that your connection is getting to the QMgr. If you had the wrong channel name, host or port you would get back a 2059. The 2035 means that the connection made it to the listener, found a channel of the name that was requested and attempted a connection.
If you want to test past this point it will be necessary to either authorize the ID that you are using to connect or to put an authorized ID in the MCAUSER attribute of the channel.
For a detailed explanation of how the WMQ security works on client channels, see the WMQ Base Hardening presentation at http://t-rob.net/links.
If you enable authorization messages then the 2035 will show up in the event queue. Then you can look at the message and see what ID was used to connect and what options were used too. The 2035 might be because you asked for set authority on the queue manager or something else you aren't supposed to have. The authorization messages wil show you that.
You can also resolve this By setting mcauser('mqm') .. i was able to overcome 2035 error.
Define channel (channel1) chltype (svrconn) trptype (tcp) mcauser(‘mqm’)
Esp thanx to my SENIOR Bilal Ahmad (PSE)
You have to check the privileges with an MQ administrator.
You can use dspmqaut to check the grant.
Below is the sample to give user poc access to Queue Manager QM1 and Queue LQ1
# check the access right of user POC to QM1
dspmqaut -m QM1 -n LQ1 -t q -p poc
# if you want to give access, you should use
setmqaut -m QM1 -n LQ1 -t q -p poc <access Types>
# eg (put everything - in the real live scenario, choose only what you want to grant) :
setmqaut -m QM1 -n LQ1 -t q -p poc +put +get +browse +inq +set +crt +dlt +chg +dsp +passid +setid +setall +clr
Then dont forget to restart QM1 with
endmqm -i QM1
strmqm QM1
Finally, you should be able to proceed without error 2035.
I have been struggling with this for ages too. Eventually I found this solution. (If you can call turning off authentication a solution.)
I am using version
- IBM Websphere 9.1.0.201807091223
From IBM's website they advise turning connection authentication off!!!
Resolving the problem Disable channel authentication
You will need to disable connection authentication, at least
temporarily. There are known issues in FTM for Check with regard to
using MQ connection authorization. These problems are actively being
addressed and fixes will appear in a future fix pack. The target is
fixpack 3.0.0.8.
Steps to disable connection authentication: Open MQ command console
and type runmqsc ALTER
AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) AUTHTYPE(IDPWOS)
CHCKCLNT(NONE) CHCKLOCL(NONE) Restart the queue manager for this
change to take effect.
Source http://www-01.ibm.com/support/docview.wss?uid=swg21962081
On this topic if you are using MQSeries 9.1 in a test or development environment you can disable channel authentication with the following approach :
. Launch MQ command line utility with the following :
runmqsc (for example runmqsc QM1)
. Disable authentication for all channels with the following command
ALTER QMGR CHLAUTH (DISABLED)
For a Q/Q-manager running on Windows, you may have to create the user on the Q/Q-manager machine [i.e. create a user on the Q-machine to match the user on the Q-client machine], and then add that user to the 'mqm' group on that machine.
Steps:
Ensure that the domain user that is being used to create the Q CLIENT [i.e. the user that the Q-client app is running under] also exists on the box with the Q/Q-manager. You may be able to just create a local user on the Q/Q-manager box [, or you may have to do some more complicated creation of an Active Directory user - I can't help you there].
On the Q/Q-manager box, add the user you have just created [or the existing one, if it already exists] to the mqm group. [On a Windows server box you will need to use the Microsoft Management Console (1. 'mmc' from the command line, 2. File > Add/Remove SnapOn > Local Users & Groups, 3. add user to group)]. The 'mqm' group should already exist on the Q/Q-manager machine.
Error MQRC 2035 basically means that your application has been able to connect to the queue manager, however due to certain absence of permissions/authorizations, it was unable to put/get/publish/subscribe messages.
To resolve this, at first, try these steps in order to disable the authorizations from queue manager and channel. Use this only if it isn't a production queue manager.
Always check the queue manager logs. It tells you exactly where you need to look into, and resolve the issue.
In this case, generally, you can issue the following commands after doing a runmqsc on the queue manager :
ALTER QMGR CHLAUTH(DISABLED)
Then set the chckclnt object(under authinfo) to optional
DISPLAY QMGR CONNAUTH
DISPLAY AUTHINFO(name-from-above) ALL //name from the first commands
ALTER AUTHINFO(name-from-above) AUTHTYPE(IDPWOS) ADOPTCTX(YES)
ALTER AUTHINFO(name-from-above) AUTHTYPE(IDPWOS) CHCKCLNT(OPTIONAL)
REFRESH SECURITY TYPE(CONNAUTH)
SET CHLAUTH('*') TYPE(BLOCKUSER) ACTION(REMOVEALL)
This helps remove any blocks that the channel is creating against any user.
SET CHLAUTH(your channel name) TYPE(ADDRESSMAP) ADDRESS('*') USERSRC(CHANNEL)
This should resolve your issue, since we have disabled every authorization that an application has to pass in order to do anything on a queue manager.
Now, in case you are using a production queue manager, NEVER remove authorizations.
Go, and right click on any QM that you have configured in your MQ explorer. Go to the QM authority, and authority records. Click on create new user, and give the same name as the username your application is using. Select all the checkboxes, then copy from the space below all the commands that are given. Namely, setmqaut. Edit with your queue manager name, and issue them!
----Never give up, the answer is where you have not looked yet--------