I'm creating a real-time data fusion pipeline where the Sink is a HTTP plugin call to Vertex AI endpoint in another GCP project. The request body will be provided by a previous step in the pipeline. The http sink plugin being used (HTTP v1.2.2) doesn't seem to support any oauth parameters. what is the best way to make that HTTP call with a dynamically generated token in the headers? any help is appreciated. Thank you
As of now, there is no way to achieve this. I also faced the same issue where my OAuth token expires in X days.
I've had to make a dynamic pipeline that doesn't fail so I have used a custom Argument setter and used the token(macro) that the custom args setter initializes in the HTTP plugin.
You can find the actual open-source code at the https://github.com/data-integrations/argument-setter
Related
How to use a variable directly in Http Request Endpoint?
If you close the activity configuration Wizard, your activity will still be there and you can fine tune the configuration freely in the Properties panel. In your case, configuring the Endpoint by using a variable.
Example
I am trying to migrate a legacy system to use artifactory. However I have two blockers:
the old scripts require PyPixmlrpc, which artifactory doesn't support
they also make use of upload_docs, not supported by artifactory's pypi implementation either
a smaller issue, the old scripts call register and they expect 200 instead of 204 http status code.
Would it be possible for me to write a plugin to implement this?
Looking at https://www.jfrog.com/confluence/display/RTF/User+Plugins I couldn't find a callback for when POST /api/pypi/<index-name> is requested.
If I can make
work for the methods we actually use, to just pretend it deployed docs and to respond with the correct status code I will be happy enough.
As you say, there is no plugin hook for the Pypi API endpoints. It would be possible to use the altResponse endpoint to customize artifact downloads, but then you would be restricted to GET requests with no request body, which is also not a good option for you.
I think the most viable approach would be to define a custom executions endpoint. With this, you can specify the acceptable method, read the body, and set your own response code and body. The main shortcoming with this is that you can't customize the path (it's always /api/plugins/execute/[execution_name]), but this can be worked around.
Execution endpoints can take params in the following form:
/api/plugins/execute/[execution_name]?params=[param_name]=[param_val]
Say your plugin takes a param path, which represents the API path your old scripts are going to call. Then you can set your base URL to /api/plugins/execute/[execution_name]?params=path=/, so that the API path is appended to the param. Alternatively, you can use nginx or another reverse proxy to rewrite the original API path to this form.
(Since you'll be using XML-RPC, I don't suppose you'll need to worry about any of this path stuff, but I'm including it anyway for completeness.)
Some issues with this approach:
Execution endpoints only allow String responses, so sending binary data in the response body might be finnicky. However, no such limitation exists with the request body.
If you need more than one request method, you'll need more than one execution endpoint. This means you'll need to use a reverse proxy to rewrite each method to a separate endpoint. Again, since XML-RPC just uses POST, this probably won't be an issue for you.
Execution endpoints can't customize response headers. Therefore, if your scripts expect a particular Content-Type or other header, you'll need to use a reverse proxy to insert it into the response.
I have a backend which generates three JWT tokens - reference token, access token and refresh token. Reference token stores a reference to the access token, which is used to access API and refresh token is used to reissue access token when it is timed out. The problem is I do not want to pass access token to the client, but want to use nginx to store it in memcached. So, my whole task is to filter the response from the backend, which currently looks as simple as:
{"reference_token":"...","access_token":"...","refresh_token":"..."}
Nginx should filter this response, get access token from this response and store it in memcached. Finally, it should return to the client a new response:
{"reference_token":"...","refresh_token":"..."}
As you can see, there should be no access_token any more. Access token is something which I try to secure and not to show it and even pass it to the client. What I do not know, is what is the best approach to implement this, what Lua block should I use for this task. I know about body_filter_by_lua , but documentation shortly says that:
Note that the following API functions are currently disabled within this context due to the limitations in NGINX output filter's current implementation
So, it seems like body filtering is rather limited and I'm not even sure if it is possible to call memcached API inside this block. So, how can I implement my task in real world? At least, what Lua (openresty) tricks should I use to approach this task?
You may issue a subrequest (e.g., ngx.location.capture) to your backend within you content handler for example.
Next you may filter a body as you want and use then lua-resty-memcached which use cosocket API.
The drawback of this approach is that you would have full buffered proxy.
AngularJS' $http service doesn't provide for HTTP Options requests, and I very much would like to have them.
I created a Web API using Django REST framework and I'm trying to leverage all it offers me in an AngularJS web application. My Django REST api provides a plethora of data from HTTP Options requests (e.g. required fields, where to obtain connected data from via hyperlinks, etc.) and I want to leverage that in the Angular application.
However, AngularJS' $http service doesn't appear to support native Options requests which makes this a pretty annoying problem...I mean, if it isn't built-in a workaround isn't going to be pretty.
I tried restangular: definitely not what I need because it doesn't allow me to supply my hyperlinks returned in the JSON from the api, and I'll be darned if I'm parsing a URL for an 'id' - absolutely silly if I have the URL already.
I looked at the angular-django-rest library: highly unsupported and couldn't get PUTs to work for the life of me b/c there's only a $save() method, which somehow automagically selects POST/PUT (?).
So, I'm at a loss! Does anyone have any idea how to get an HTTP Options request in AngularJS?
$http supports options. Options doesn't have a function declared like $http.get() or $http.post(), but you can specify any method you want by specifying the method and passing it directly into $http.
$http({method: 'OPTIONS', url: '/foo'})
You can also use $resource:
$resource(address, {}, {options:{method:'OPTIONS'}});
I am using Visual Studio 2010 Ultimate to perform web and load tests. I have a set of web tests that call REST web services that require OAuth credentials and I'm looking for information on how I can access the associated Http Headers and Post body the request. I've created a web test plugin that acts as an Authorization Manager and have overriden the PreWebTest method. When I look at the PreWebTestEventArgs argument, I see the WebTest and its WebTestContext but I don't see any obvious way to access the actual Http Headers or the Post Body where I might be able to insert the OAuth components. Has anyone been able to affect the Http Request with the associated web test? Any insight will be much appreciated. Thanks.
I think you'll need to override PreRequest instead of PreWebTest.
PreRequestEventArgs has the Request property which is the WebTestRequest object that has access to the headers and post body.