I am using a third party library in my code to get access token (ADAL). This library has a lot of calls to requests.get and requests.post. How can I force all the calls to use user provided proxies without having to modify each call to requests.get('http://example.com', proxies=proxies).
I cannot do export HTTP_PROXY. I have to do it from within my script.
You could monkey patch requests.
At the very start of your script:
import requests
import functools
orig_get = requests.get
proxies = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
requests.get = functools.partial(orig_get, proxies=proxies)
Related
This is the first time I'm trying to write test cases.
I've got a simple FastAPI application and I'm trying to create tests with unittest module.
My goal is to test how app behaves for success case
I've got a simple route in my app:
from fastapi import APIRouter,Request
import requests
router = APIRouter()
#router.post("/demo_router")
async def demo_api(request: Request):
# now calling some private APi's
resp = requests.post("https://example.com", json=data)
return {"resp_data": resp.json()}
Now in my unittest module I'm trying to patch above api. I'm using unittest.mock but I'm getting very strange behavior.
import unittest
from fastapi.testclient import TestClient
from unittest.mock import patch
from main import app
class DemoViewTestCase(unittest.TestCase):
def test_demo_api(self):
with patch('src.endpoints.demo_module.demo_api') as mocked_post:
mocked_post.return_value.status_code = 200
mocked_post.return_value.json = {
"message": "request accepted",
"success": True
}
url = router.url_path_for('demo_api') #fetch the api router
client = TestClient(app)
response = client.post(url, json={"id": "BXBksk8920", "name": "Pjp"})
My problem is TestClient is calling the api and executing it. So it is triggering the internal call "https://example.com" which is causing some execution in the pipelines. So how can I overcome this?
Internal Api shouldn't trigger, I should even mock that? Any solution for that?
When testing all the code will be executed. Thus also the calls to the APIs. If you don't want this, you have to provide mock APIs (postman, mockon and many others provide this).
Because you don't want to be bothered to change the URL's etc when you are testing etc, you could look at automating this.
One way of doing this is to provide all URLs for external APIs using pedantic BaseSettings
config.py:
from pydantic import BaseSettings
class Settings(BaseSettings):
external_api_url: str = "https://api.example.com"
And use this in your code:
settings = Settings() # scans environment for any matching env settings!
...
resp = requests.post(setting.external_api_url, json=data)
In your tests you can override these settings:
settings = Settings(external_api_url="https://mockservice")
This is documented further in Pydantic BaseSettings
There are more way do enhance testing and this is found at the FastAPI documentation:
Dependency Override: https://fastapi.tiangolo.com/advanced/testing-dependencies/
Use different databases for testing: https://fastapi.tiangolo.com/advanced/testing-database/
I've been looking for a while through documentation to find a way to accomplish this and haven't been successful yet. The basic idea is, that I have a piece of html that I load through Qt's webview. The same content can be exported to a single html file.
This file uses Libraries such as Bootstrap and jQuery. Currently I load them through CDN which works when online just fine. However, my application also needs to run offline. So I'm looking for a way to intercept loading of the Libraries in Qt and serve a locally saved file instead. I've tried installing a https QWebEngineUrlSchemeHandler, but that never seems to trigger the requestStarted method on it.
(PyQT example follows)
QWebEngineProfile.defaultProfile().installUrlSchemeHandler(b'https', self)
If I use a different text for the scheme and embed that into the page it works, so my assumption is that it doesn't work as Qt has a default handler for it already registered. But that different scheme would fail in the file export.
Anyway, back to the core question; Is there a way to intercept loading of libraries, or to change the url scheme specifically within Qt only?
Got Further with QWebEngineUrlRequestInterceptor, now redirecting https requests to my own uri, which has a uri handler. However, the request never gets through to it, because: Redirect location 'conapp://webresource/bootstrap.min.css' has a disallowed scheme for cross-origin requests.
How do I whitelist my own conapp uri scheme?
Edit: For completeness sake, it turns out back when I originally stated the question, it was impossible to accomplish with PySide 5.11 due to bugs in it. The bug I reported back then is nowadays flagged as fixed (5.12.1 I believe) so it should now be possible to accomplish this again using Qt methods, however for my own project I'll stick to jinja for now which has become a solution for many other problems.
The following example shows how I've done it. It uses the QWebEngineUrlRequestInterceptor to redirect content to a local server.
As an example, I intercept the stacks.css for stackoverflow and make an obvious change.
import requests
import sys
import threading
from PyQt5 import QtWidgets, QtCore
from PyQt5.QtWebEngineWidgets import QWebEngineView, QWebEnginePage, QWebEngineProfile
from PyQt5.QtWebEngineCore import QWebEngineUrlRequestInterceptor, QWebEngineUrlRequestInfo
from http.server import HTTPServer, SimpleHTTPRequestHandler
from socketserver import ThreadingMixIn
# Set these to the address you want your local patch server to run
HOST = '127.0.0.1'
PORT = 1235
class WebEngineUrlRequestInterceptor(QWebEngineUrlRequestInterceptor):
def patch_css(self, url):
print('patching', url)
r = requests.get(url)
new_css = r.text + '#mainbar {background-color: cyan;}' # Example of some css change
with open('local_stacks.css', 'w') as outfile:
outfile.write(new_css)
def interceptRequest(self, info: QWebEngineUrlRequestInfo):
url = info.requestUrl().url()
if url == "https://cdn.sstatic.net/Shared/stacks.css?v=596945d5421b":
self.patch_css(url)
print('Using local file for', url)
info.redirect(QtCore.QUrl('http:{}:{}/local_stacks.css'.format(HOST, PORT)))
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
"""Threaded HTTPServer"""
app = QtWidgets.QApplication(sys.argv)
# Start up thread to server patched content
server = ThreadingHTTPServer((HOST, PORT), SimpleHTTPRequestHandler)
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = True
server_thread.start()
# Install an interceptor to redirect to patched content
interceptor = WebEngineUrlRequestInterceptor()
profile = QWebEngineProfile.defaultProfile()
profile.setRequestInterceptor(interceptor)
w = QWebEngineView()
w.load(QtCore.QUrl('https://stackoverflow.com'))
w.show()
app.exec_()
So, the solution I went with in the end was, first, introduce jinja templates. Then, using those the template would have variables and blocks set based on export or internal use and from there I did not need the interceptor anymore.
I have a problem that I known the web services require authentication,but I can't determine Basic or Digest it need,How can I deal with this situation use python2.7 http library that I have own the username and password？
You should use requests.It has requests_toolbelt lib for auto check authentication type,the following sample code:
import requests
import requests_toolbelt
auth = requests_toolbelt(username,password)
req = requests.get(url,auth=auth)
I was using "angular-in-memory-web-api" to mock my REST web api, however now I am have started writing the actual web api and I want to replace the "angular-in-memory-web-api" step by step.
Example:
I have written following asp.net web api
/api/values
however when I issue http.get() request using above api url angular tries to serve it using "angular-in-memory-web-api", however in this case I want to bypass "angular-in-memory-web-api" so that request is entertained by my actual web api controller.
you can set passThruUnknownUrl to forward to real backend for un-matching endpoints
e.g.,
InMemoryWebApiModule.forRoot(InMemoryDataService, {
passThruUnknownUrl: true
}),
You can try by removing InMemoryWebApiModule.forRoot(InMemoryDataService) from your app.module. If you have done so already, showing some code would be helpful.
Also, consider using Postman to verify your new service
in app.module.ts
import { HttpClientInMemoryWebApiModule } from 'angular-in-memory-web-api';
import { InMemoryDataService } from './my-module/services/in-memory-data.service';
in app.module.ts imports array
HttpClientInMemoryWebApiModule.forRoot(InMemoryDataService, { dataEncapsulation: false, passThruUnknownUrl: true })
passThruUnknownUrl related doc
I need to contact a proprietary http service, not supporting chunks.
I started using as documented here so i create the client this way:
Client client = ClientBuilder.newBuilder().newClient();
WebTarget target = client.target("http://localhost:8080/rs");
The problem is how to configure the client, how to disable chunking. The way documented here doesn't work for me (wrong classes).
Thanks in advance
If you'd like to continue to use the ClientBuilder etc, you can do this instead:
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.WebTarget;
import org.apache.cxf.jaxrs.client.WebClient;
Client client = ClientBuilder.newBuilder().build();
WebTarget target = client.target("https://www.example.com/");
target.request(); /* Must call this first to initialise the client in the target */
WebClient.getConfig(target).getHttpConduit().getClient().setAllowChunking(false);
Rather that using jaxrs standard Client you can use org.apache.cxf.jaxrs.client.WebClient part of cxf-rt-rs-client dependency.
WebClient client = WebClient.create("http://localhost:8080/rs");
WebClient.getConfig(client).getHttpConduit().getClient().setAllowChunking(false);