One of my EMR job relies on getting the AWS access key id and secret access key from the fs.s3.awsAccessKeyId and fs.s3.awsSecretAccessKey properties, respectively. However, when I run EMR cluster using the default EC2 and EMR roles, those properties do not exist.
How do I get these access keys while running the EMR clusters with the default roles, seeing that using IAM roles is a requirement at this point? One way I could do it is set these properties explicitly myself, but I want to see if I missed something else.
Using roles and not hard coded keys is a best practice (http://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). An example of this on EMR is the underlying Hadoop FS calls use the role assigned to the EC2 instance in order to generate temporary security credentials.
Your application can be built to do the same (http://docs.aws.amazon.com/IAM/latest/UserGuide/roles-usingrole-ec2instance.html) such that it fetches a temporary access key and secret with token from the assigned role instead of reading it from a fixed config.
Related
I'm building a Java application which needs to access the DynamoDB. The application is intended to be used by several end users (not all of them are trusted). From my understanding, in order to access the AWS service,the AWS credentials need to be loaded at runtime on end users' machine via several ways described at https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html
However I don't feel it'll be safe to directly hardcode the access key and token into the application code, as this can be easily exposed. Given my end users don't have too much technology background, I don't want to add too many "pre-setup" steps before they can use the application.What will be best/feasible practise to distribute the credential to them?
Thanks for all opinions.
You probably need to be looking at AWS Cognito
An Amazon Cognito user pool and identity pool used together
See the diagram for a common Amazon Cognito scenario. Here the goal is
to authenticate your user, and then grant your user access to another
AWS service.
In the first step your app user signs in through a user pool and
receives user pool tokens after a successful authentication.
Next, your app exchanges the user pool tokens for AWS credentials
through an identity pool.
Finally, your app user can then use those AWS credentials to access
other AWS services such as Amazon S3 or DynamoDB.
I have an application that runs in a Service Fabric(SF) cluster and I wan't to access Key Vault from it.
The cluster hosts a number of applications and I want to give access to a Key Vault for my application without giving access to the other applications. By default an application runs under the same user as the SF cluster, but each applicatiuon has it's own unique name, mine has the name fabric:/application1.
My question is, is it possible to create an Active Directory application account for fabric:/application1 and grant access to the key vault?
I know it is possible to use the RunAs options in the SF manifest, but that requires me storing an encrypted password in the manifest/source code and I want to try and avoid this if possible.
AFAIK,
The only way to have this flexibility is using ClientID & Secret or Service Principal certificates and each application manage their own credentials.
Service Principal Certificate is already integrated to AD, but does not require the application, the user or the Host to be part of the domain, the only requirement is setup an user on AD to grant the permissions on Keyvault.
There are other solutions using AD integration, like Managed identities for Azure resources(Former: Managed Service Identity) but I am not sure if you are able to restrict access per application like you described, because the MI add this as a service in the node, so technically other applicaitons would have access as well, worth a try to validate if you can restrict this.
If you want to try this approach, you can use with Microsoft.Azure.Services.AppAuthentication for implicit authentication of the services running in your cluster, where the nodes are setup with Managed Identities extension like described here.
Something link this:
When you use the Microsoft.Azure.Services.AppAuthentication, the Step 2 will be handled by the library and you won't have to add much changes to your key vault auth logic.
When you run your code on an Azure App Service or an Azure VM with a
managed identity enabled, the library automatically uses the managed
identity. No code changes are required.
The following docs describe other options you can use for KeyVault Authentication.
PS: I've done other KeyVault integrations using Client Secrets and Certificates and they are secure enough, With Certificates you can store it on the managed store or with the application, I would recommend MI only if is a requirement for your solution.
In my MySQL database I have two kind of users
MySQL users, used to connect and access a database directly from a client such as MySQL workbench or any ODBC editor.
ASP.NET membership users + roles : used to provide login for my websites.
My situation:
When I login via a webpage using the ASP.NET user credentials. I want to create a temporary random MySQL user (assign privileges, set password etc) and give it to the currently logged in (asp.net user) to gain temporary access to the MySQL database.
I know, if ASP.NET user has root access to create users he also has complete access to the database. My problem is, I have to give the user name and password so he can connect from another client and access the database directly. (I don't want give him root access or permanent access)
Having said he can access from another client, I would like to restrict this according to his login status in my webpage. like when he logs out of my webpage the newly created MySQL user is marked as expired or deleted.
I assume this should be very possible but I cannot think of any starting point. Could anyone help me?
kind regards
krish
This is possible. You could use the create user and grant syntax to create your user and grant them privileges.
You would however need to maintain a separate table for mapping MySQL users with an expiry date.
Have you thought about a different solution?
it would be vastly more work to complete but in the long run might be a more stable / reliable and scalable solution.
Maybe create your own WCF proxy for MySQL using NET.NCP, you could then have local user accounts / temporary accounts managed by you that authorise against your WCF service rather than the database directly, the proxy would in turn would connect directly to your database. This solution would also work from 3rd party applications.
Using a custom proxy would give you much greater control such as:
Being able to log SQL for specific users Sanitise or restrict any
specific command you don’t want executed on your database Not having
your MySQL server directly contactable on the internet
Split read & writes to different servers or clusters when scaling out
Edit 1: as per comments below:
I wouldn’t recommend relying on an event for the session end as this isn’t always fired. I would suggest you create another table to manage the users and their expiry.
For example, a table that holds the user ID or username and host along with an expiration time. This could be either date time / integer (epoch) depending on your requirements.
You would then need to invoke a query to identify all accounts from this table that have expired, you could then delete the user accounts from MySQL.
Depending on your MySQL version you could wrap a lot of the logic into a few stored procedures which will make the querying and maintenance overhead easier
I'm building an asp.net application that will later be ported to azure.
For the moment, I have all the business tables in one database and a separate database that I use for membership; it's basically the default database that the login control generates.
In the business database, I have a table that contains user profile data and one field is TheUserID (which is an int) and another field that's called TheUserMembership (a string), which will contain the user ID that's generated by the asp.net user management tool.
Once the user logs in, I store TheUserID in the session and the whole app works with the int as the identifier.
Is this a good way to do it? Will this port to azure?
You should be using: http://msdn.microsoft.com/en-us/library/yh26yfzy.aspx
And then yes it will port to Azure and SQL Azure.
Things to watch out for with Azure
Local storage (disk storage). Since you have multiple instances, local storage doesn't work as you will never be able to tell which instance it is on.
Session state must be out of proc, for the same reason as above.
There are many other little things here or there about Azure but those would be the 2 biggest to watch out for when moving across.
the user id is a GUID and it you can find the on the users table, you should not store user id into session since cookie can be stolen. for azure one principal requirement is that all tables have a primary key
Is it possible to use IAM to manage user accounts for EC2-hosted unix hosts by way of a PAM module similarly to LDAP, NIS, etc...?
My goal is to have a means to centralize host authentication on our EC2 hosts without the overhead of setting up a single sign on solution.
AWS IAM is meant to handle access to AWS resources. You can create new users but the basic authentication which EC2 instances get is via key pairs, which are not the same as IAM users.
You might be able to create a system of your own which manages IAM users and also generates a private and public key for them to be used inside the instances being created (maybe even re-using the keys you get when creating a new user in IAM).
All in all its not really meant to be used that way, as far as I understand.
Since you mentioned LDAP, you can use this project:
https://github.com/denismo/aws-iam-ldap-bridge
to sync an LDAP server with IAM.