Using AWS Boto3 Invoke API Gateway from EC2 Instance - python-requests

I am trying to invoke AWS API Gateway endpoint from one of the EC2 instance with IAM Role. I have boto3 library installed on EC2 instance and trying to execute simple gateway API using below code but still getting Authentication missing error.
import boto3
import json
import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth
session = boto3.Session()
credentials = session.get_credentials()
headers = {'params': 'ABC'}
response = requests.get('https://restapiid.execute-api.us-east-1.amazonaws.com/stage/resource_path',
auth=credentials, headers=headers)
This should be very simple from EC2 Instance with IAM Role. Please any advise.

Due to lack of details in your question, (missing instance role details, API gateway policy, unknown headers, or wheather iam_auth is enabled) I can only provide and comment on the python code given.
The python code to use role should be (this is example that I used to verify the code):
import boto3
import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth
session = boto3.Session()
credentials = session.get_credentials()
auth = AWSRequestsAuth(aws_access_key=credentials.access_key,
aws_secret_access_key=credentials.secret_key,
aws_token=credentials.token,
aws_host='fzoskzctgd.execute-api.us-east-1.amazonaws.com',
aws_region='us-east-1',
aws_service='execute-api')
response = requests.get('https://fzoskzctgd.execute-api.us-east-1.amazonaws.com/test', auth=auth)
print(response.content)
I tested this with authorizationType set to AWS_IAM for the resource a tested.
API resource policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456:role/instance-role"
},
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:170576413884:fzoskzctgd/test/*"
}
]
}
instance-role
Does not need to have any api invocation permissions as they are provided through API resource policy. The instance-role must only exist and be attached to the instance.

Related

Can I view the configuration for the Firebase functions used by my website using Firebase/Google Cloud's UI?

I deployed a new version of a web app that included some new Firebase functions. The new Firebase functions were inaccessible upon deployment.
I was able to find the root cause by familiarity with Firebase's configuration: the function requests were intended to route through the main app domain, and be redirected on the Firebase server to their final destination. (This should have been set up with Firebase's 'rewrites' section of firebase.json, but wasn't.)
A partial view of my rewrites section of firebase.json:
"rewrites": [
{
"source": "/getPlaces",
"function": "getPlaces"
},
...
{
"source": "**",
"destination": "/index.html"
}
]
In plain English, this says, "Firebase server: every time you get a request routing to https://my.app/getPlaces, I want you to not route to that address within my app; I want to invoke my Cloud function instead. Otherwise, route normally."
Yet when I look at the Functions tab of my Firebase console, all I see is this:
Under 'Request', it says https://us-central1-my-app.cloudfunctions.net/getPlaces. That gives me just one of the ways to access my function; the other is https://my.app/getPlaces, as defined in rewrites. I need to know all of the addresses that Firebase will respond to, not just the default one using cloudfunctions.net.
Is it possible to see the entire configuration for the deployed Firebase functions anywhere in the web UI, ie console.firebase.com or console.cloud.google.com, where redirects from rewrites like this can be seen?

Need of scope parameter in Microsoft.Identity.Web downstream API

I am using microsoft.Identity.Web package on my .netcore API project which calls Graph API to get the directory objects of the user.
In the appsettings file the downstream api settings are provided as below,
"DownstreamApi": {
"BaseUrl": "https://graph.microsoft.com/v1.0",
"Scopes": "Directory.Read.All"
},
The relevant permission(Directory.Read.All) is setup in the app registration.
But even if I leave the "Scope" parameter blank the API is giving me the directory objects.
So if the settings is of the format below it still works. Then what is the need of this scope parameter?
"DownstreamApi": {
"BaseUrl": "https://graph.microsoft.com/v1.0",
"Scopes": ""
},
The scope claim might not had reflected in the token and so you might not seeing any difference with scope assigned.
user_impersonation is the default delegated permission /scope that exists initially for every Web app or API in Azure AD.
Please make sure to add the required delegated permissions or application permission in portal.And grant consent if required.
In your case add directory.read.all Application permission
ex:I added user.read
Appsettings:
"DownstreamApi": {
"BaseUrl": "https://graph.microsoft.com/v1.0",
"Scopes": "user.read"
},
In startUp.cs
Public void ConfigureServices(IServiceCollection services)
{
string[] initialScopes = Configuration.GetValue<string>("DownstreamApi:Scopes")?.Split(' ');
services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
.AddMicrosoftIdentityWebApp(Configuration)
// acquire a token to call a protected web API
.EnableTokenAcquisitionToCallDownstreamApi(initialScopes)
.AddMicrosoftGraph(Configuration.GetSection("DownstreamApi"))
.AddInMemoryTokenCaches();
//
//othercode
...
}
And in controller we need to specify scopes and send to request headers to get access token for required scopes.
References:
call Microsoft Graph | Microsoft Docs
(OR) active-directory-aspnetcore-webapp-openidconnect-v2 (github.com)
How can I create a new Azure App Registration without the user_impersonation OAuth2Permission? - Stack Overflow
If client_credentials is the grant type you may need to use https://graph.microsoft.com/.default for scope in the application settings which will give you the permissions defined for your app.
"DownstreamApi": {
"BaseUrl": "https://graph.microsoft.com/v1.0",
"Scopes": "https://graph.microsoft.com/.default"
}
Try to use /token endpoint in request and not common
Please see:
ASP.NET Core - Call Graph API Using Azure Ad Access Token - Stack Overflow-Reference

Apache NiFi write to encrypted SQS queue

We are encountering a very strange condition with Apache NiFi and SQS. We are using the AWSCredentialsProviderControllerService to manage our authentication. If we use an unencrypted queue it works fine, however, if using an encrypted queue it doesn't fail but nothing gets written. It doesn't appear to be generating anything in the NiFi or cloud trail logs either. Just wondered if there is anything special that needs to get done to support this condition. If it is failing, we are not able to figure out where that is occurring. Any suggestions or ideas would be greatly appreciated.
I was able to reproduce the silent failure with PutSQS under the following conditions:
SQS Queue configured with server-side encryption using a custom KMS customer master key rather than the default AWS key
AWS credentials used by NiFi had permission to send a message, but not permissions to use the custom KMS key
The solution was to provide NiFi's AWS credential with the permissions to use both SQS and KMS. I found the example policy below documented in What AWS KMS Permissions Do I Need to Use SSE for Amazon SQS?:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "arn:aws:kms:us-east-2:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab"
}, {
"Effect": "Allow",
"Action": [
"sqs:SendMessage",
"sqs:SendMessageBatch"
],
"Resource": "arn:aws:sqs:*:123456789012:MyQueue"
}]
}

Openstack: login as admin and retrieve server data from different tenants via python

I'm writing some cron job in python for Openstack that should read server ids from a database and then get the servers from the API by using the python-novaclient.
In pseudo code things should work like this:
session = login_to_keystone(user="admin", password="something") #or use a token
nova_client = get_nova_client(session)
#the servers array holds dictionaries with server, user and tenant ids as strings
# e.g. {"server_id": "1-2-3", "tentant_id": "456", user_id: "11111-2222"}
for server in servers:
server_obj = nova_client.servers.get(server.server_id)
...do stuff with server_obj (read data from it, delete,...)...
What I've come up with is the following, but it's not right as I get a EndpointNotFound exception. I'm using Devstack with Juno.
from keystoneclient.v2_0 import client as keystone_client
from keystoneclient import session
from novaclient import client as nova_client
#the url is the admin endpoint
keystone = keystone_client.Client(token="my-admin-token",
auth_url="http://192.168.1.1:35357/v2.0",
endpoint="http://192.168.1.1:35357/v2.0")
key_session = session.Session(auth=keystone)
nova = nova_client.Client(2, session=key_session)
#let's assume the servers array is already populated
for server in servers:
server_obj = nova.servers.get(server.server_id) #Exception happens here
I need to run this as admin as servers can belong to any tenant and they might even be deleted by the cron job.
Thanks for any help!
UPDATE:
The information I need was that I can use the admin tenant for retrieving all servers (regardless of their owner). This allows me also to use the publicURL.
My current solution looks like this:
from keystoneclient.auth.identity import v2
from keystoneclient import session
from novaclient import client as nova_client
auth = v2.Password(auth_url="http://192.168.1.1:5000/v2.0",
username="admin",
password="my_secrete",
tenant_name="admin") # the admin's tenant
auth_session = session.Session(auth=auth)
nova = nova_client.Client(2, session=auth_session)
for server in servers:
... do stuff like nova.servers.get("some id")
In order to get a list of servers from all tenants, you need to perform two tasks:
Log in as a user with admin privileges, and
Tell the Nova API that you want a list of servers for all tenants.
Logging in as an admin user
It looks like you're trying to use the admin_token defined in keystone.conf for authentication. This may work, but this mechanism is meant primarily as a means of bootstrapping Keystone. When interacting with the other services, you are meant to log in using a username/password pair that has been defined in Keystone with admin credentials. I think that what #anoop.babu has given you will work just fine:
>>> nova = nova_client.Client('2', USERNAME, PASSWORD,
PROJECT_ID, AUTH_URL)
Where:
USERNAME = admin
PASSWORD = password_for_admin_user
PROJECT_ID = admin
AUTH_URL = http://your_api_server:5000/v2.0
We can test this client out using something like:
>>> nova.hypervisors.list()
[<Hypervisor: 2>]
That tells us we've authenticated successfully.
Listing all servers
If we simply called nova.servers.list(), we are asking for a list of Nova servers owned by the admin tenant, which should generally be empty:
>>> nova.servers.list()
[]
In order to see servers from other tenants, you need to pass the all_tenants search option:
>>> nova.servers.list(search_opts={'all_tenants':1})
[<Server: cirros0>]
And that should get you where you want to be.
Here you have given auth_url as your endpoint. endpoint is actually the publicURL of the service.
You can find the publicURL using the below CLI Command
nova endpoints
and check for keystone details.
You can also get an authenticated keystone object using below api version without using endpoint,
import keystoneclient.v2_0.client as keystone_client
keystone = keystone_client.Client(auth_url = my_auth_url , username = my_user_name , password = my_password , tenant_name = my_tenant_name )
And create a session object as below,
from keystoneclient import session
key_session = session.Session(auth=keystone)
An alternate and simplified approach to get nova client authorized without keystone is,
from novaclient import client as nova_client
nova = nova_client.Client('2', USERNAME, PASSWORD, PROJECT_ID, AUTH_URL)
server_obj = nova.servers.find( name=my_server_name )

how to use the response url recieved from create server api response in openstack

I am using Open-Stack APIs to create a server/instance. I am using a rest client to do it. when I send a request to create a server the request executes successfully and returns me following response:
{
"server": {
"OS-DCF:diskConfig": "AUTO",
"adminPass": "CQH9gWzgkVno",
"id": "324dfb7d-f4a9-419a-9a19-237df04b443b",
"links": [
{
"href": "http://openstack.example.com/v2/openstack/servers/324dfb7d-f4a9-419a-9a19-237df04b443b",
"rel": "self"
},
{
"href": "http://openstack.example.com/openstack/servers/324dfb7d-f4a9-419a-9a19-237df04b443b",
"rel": "bookmark"
}
]
}
}
I don't know how to access the Server/Instance using the URL received in the JSON response. when the URL is clicked it says Unauthorised.
I don't have access to the OpenStack server or Horizon.
Please help!
That URL does not provide a means for you to access your server. The URL is a REST API endpoint that can be used by your REST client to perform actions on the server. Take a look at the REST API documentation. For example, you can GET that URL (with appropriate credentials) to get details about that server, or you can use that URL and a suffix to do things like interact with the server metadata (<url>/metadata) or perform server actions (<url>/action) such as rebooting the server.
If you actually want to log in to the server or access services on the server, you would need to look up the ip addresses that have been assigned to the server by OpenStack. The easiest way to do this is to install the nova command line client, and then just run something like nova list, which will produce output like:
+--------------------------------------+----------+--------+------------+-------------+---------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+---------------------------------+
| 2ed5f06c-66aa-4dc5-b24f-2aec1c4c7411 | myserver | ACTIVE | - | Running | net0=10.0.0.25, 192.168.200.190 |
+--------------------------------------+----------+--------+------------+-------------+---------------------------------+
If for some reason you can't install the nova client, you could replicate the functionality yourself using the REST APIs documented at the above link.

Resources