I am able to successfully send requests to a sandbox via postman, given by a provider following their specs (see images below)
Successful request (see below)
In order to do that, aside from the respective headers and parameters (see image 2) I have to add a ssl/Tls certificate (.pfx) given that the server requires a 2 way handshake so it needs SSl client certificate:
Authorization (see below).
Headers (see below)
Body (see below)
Now, I am trying to do ir programatically using dotnet core 6, but I keep running into the same problem:
And here is my code:
public static string GetAccessToken(IConfiguration _config)
{
string UserName = Environment.GetEnvironmentVariable("USER_NAME");
string Password = Environment.GetEnvironmentVariable("PASSWORD");
var client = new RestClient("https://connect2.xyz.com/auth/token");
var request = new RestRequest();
X509Certificate2 FullChainCertificate = new X509Certificate2("Path/to/Cert/cert.pfx", "test");
client.Options.ClientCertificates = new X509CertificateCollection() { FullChainCertificate };
client.Options.Proxy = new WebProxy("connect2.xyz.com");
var restrequest = new RestRequest();
restrequest.Method = Method.Get;
restrequest.AddHeader("Accept", "*/*");
restrequest.AddHeader("Cache-Control", "no-cache");
restrequest.AddHeader("Content-Type", "application/x-www-form-urlencoded");
restrequest.AddHeader("Authorization", "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes($"{UserName}:{Password}")));
restrequest.AddParameter("grant_type", "client_credentials");
RestResponse response = client.Execute(restrequest);
AccessTokenPointClickCare accessToken = JsonConvert.DeserializeObject<AccessTokenPointClickCare>(response.Content);
string strToken = accessToken.access_token;
return strToken;
}
Now, as the error seems to show, it has to do with the certificates (apparently), but I don't know if something in the code is wrong, or if I'm missing something, etc...
It is worth noting that this code did run in someone else's pc with the same set-up, but of course with that person's own pfx, but for the rest, it is essentially the same, and not to mention that it does work on my postman.
Finally, as the title on this question, the only thing I can think it might also be affecting the request is the host. If I reference the postman, there is a field where I have to place the host name of the server https://connect2.xyz.com/auth/token
So made it work by changing to a new Windows 10. Researching in other Stackoverflow threads found the answer: .NET CORE 5 '''HandshakeFailure'" when making HTTPS request
So I conclude it has to do with the cyphers
This question already has answers here:
Logging Messages from Java Class back to the Karate Report
(3 answers)
Closed 1 year ago.
I want to be able to write log statements, that get added to the karate.log file as well as to the Cucumber Reports that get generated when using standalone karate.jar.
When I use karate.log from a javascript function it only adds the log statement to the karate.log file and not the cucumber report.
I have also tried to do this from a java function as well by using both slf4j logger as well as the com.intuit.karate.Logger class. However both of these only add logs to the karate.log file and not to the cucumber reports.
I need this because I am writing some common code for which I don't want my QA-Engineers to write * print <> statements in the karate feature files.
I also looked at the com.intuit.karate.core.ScriptBridge.log(Object... objects) method which is what I am assuming gets called when you call karate.log(..), it looks like it should work, but it isn't working for me.
I am using karate-0.9.4, and here's what my karate-config.js looks like
function conf() {
var env = karate.env // set the environment that is to be used for executing the test scripts
var host = '<some-host-name>';
var port = '443';
var protocol = 'https';
var basePath = java.lang.System.getenv('GOPATH') + '/src/karate-tests';
// a custom 'intelligent' default
if (!env) {
env = 'dev';
}
var applicationURL = ((!port || port == '') || (port == '80' && protocol == 'http') || (port == '443' && protocol == 'https'))
? protocol + '://' + host
: protocol + '://' + host + ":" + port;
// Fail quickly if there is a problem establishing connection or if server takes too long to respond
karate.configure('connectTimeout', 30000);
karate.configure('readTimeout', 30000);
// pretty print request and response
//karate.configure('logPrettyRequest', true);
//karate.configure('logPrettyResponse', true);
karate.configure('printEnabled', true);
// do not print steps starting with * in the reports
//karate.configure('report',{showLog: true, showAllSteps: true });
// Turn off SSL certificate check
karate.configure('ssl', true);
var config = {
env: env,
appBaseURL: applicationURL,
sharedBasePath: basePath
};
karate.log("config.sharedBasePath = ", config.sharedBasePath)
karate.log('karate.env = ', config.env);
karate.log('config.appBaseURL = ', config.appBaseURL);
return config
}
This is because of a bug in karate-0.9.4 which seems to be partially fixed in karate-0.9.5.RC4 release. I have opened a ticket for it on GitHub - https://github.com/intuit/karate/issues/975
I just tried this in 0.9.5.RC4. If you are looking for something more than this - it needs a change in Karate. You are welcome to contribute. I have to say that I'm surprised (and somewhat annoyed) to see these requests. Why are you so concerned about pretty reports instead of focusing on testing. I'd like you to think about it.
This other discussion may be a related reference: https://github.com/intuit/karate/issues/951 | https://github.com/intuit/karate/issues/965
If you really want to pursue this, you can look at the "hook" interceptor mentioned in this comment: https://github.com/intuit/karate/issues/970#issuecomment-557443551
So in void afterStep(StepResult result, ScenarioContext context); - you can modify the StepResult by calling appendToStepLog(String log).
EDIT: other references:
https://stackoverflow.com/a/57079152/143475
https://stackoverflow.com/a/47366897/143475
I'm attempting to set the request header 'Referer' to spoof a request coming from another site. We need the ability test that a specific referrer is used, which returns a specific form to the user, otherwise an alternative form is given.
I can do this within poltergeist by:
page.driver.headers = {"Referer" => referer_string}
but I can't find the equivalent functionality for the selemium driver.
How can I set request headers in the capybara selenium driver?
Webdriver doesn't contain an API to do it. See issue 141 from Selenium tracker for more info. The title of the issue says that it's about response headers but it was decided that Selenium won't contain API for request headers in scope of this issue. Several issues about adding API to set request headers have been marked as duplicates: first, second, third.
Here are a couple of possibilities that I can propose:
Use another driver/library instead of selenium
Write a browser-specific plugin (or find an existing one) that allows you to add header for request.
Use browsermob-proxy or some other proxy.
I'd go with option 3 in most of cases. It's not hard.
Note that Ghostdriver has an API for it but it's not supported by other drivers.
For those people using Python, you may consider using Selenium Wire which can set request headers as well as provide you with the ability to inspect requests and responses.
from seleniumwire import webdriver # Import from seleniumwire
# Create a new instance of the Chrome driver (or Firefox)
driver = webdriver.Chrome()
# Create a request interceptor
def interceptor(request):
del request.headers['Referer'] # Delete the header first
request.headers['Referer'] = 'some_referer'
# Set the interceptor on the driver
driver.request_interceptor = interceptor
# All requests will now use 'some_referer' for the referer
driver.get('https://mysite')
Install with:
pip install selenium-wire
I had the same issue. I solved it downloading modify-headers firefox add-on and activate it with selenium.
The code in python is the following
fp = webdriver.FirefoxProfile()
path_modify_header = 'C:/xxxxxxx/modify_headers-0.7.1.1-fx.xpi'
fp.add_extension(path_modify_header)
fp.set_preference("modifyheaders.headers.count", 1)
fp.set_preference("modifyheaders.headers.action0", "Add")
fp.set_preference("modifyheaders.headers.name0", "Name_of_header") # Set here the name of the header
fp.set_preference("modifyheaders.headers.value0", "value_of_header") # Set here the value of the header
fp.set_preference("modifyheaders.headers.enabled0", True)
fp.set_preference("modifyheaders.config.active", True)
fp.set_preference("modifyheaders.config.alwaysOn", True)
driver = webdriver.Firefox(firefox_profile=fp)
Had the same issue today, except that I needed to set different referer per test. I ended up using a middleware and a class to pass headers to it. Thought I'd share (or maybe there's a cleaner solution?):
lib/request_headers.rb:
class CustomHeadersHelper
cattr_accessor :headers
end
class RequestHeaders
def initialize(app, helper = nil)
#app, #helper = app, helper
end
def call(env)
if #helper
headers = #helper.headers
if headers.is_a?(Hash)
headers.each do |k,v|
env["HTTP_#{k.upcase.gsub("-", "_")}"] = v
end
end
end
#app.call(env)
end
end
config/initializers/middleware.rb
require 'request_headers'
if %w(test cucumber).include?(Rails.env)
Rails.application.config.middleware.insert_before Rack::Lock, "RequestHeaders", CustomHeadersHelper
end
spec/support/capybara_headers.rb
require 'request_headers'
module CapybaraHeaderHelpers
shared_context "navigating within the site" do
before(:each) { add_headers("Referer" => Capybara.app_host + "/") }
end
def add_headers(custom_headers)
if Capybara.current_driver == :rack_test
custom_headers.each do |name, value|
page.driver.browser.header(name, value)
end
else
CustomHeadersHelper.headers = custom_headers
end
end
end
spec/spec_helper.rb
...
config.include CapybaraHeaderHelpers
Then I can include the shared context wherever I need, or pass different headers in another before block. I haven't tested it with anything other than Selenium and RackTest, but it should be transparent, as header injection is done before the request actually hits the application.
I wanted something a bit slimmer for RSpec/Ruby so that the custom code only had to live in one place. Here's my solution:
/spec/support/selenium.rb
...
RSpec.configure do |config|
config.after(:suite) do
$custom_headers = nil
end
end
module RequestWithExtraHeaders
def headers
$custom_headers.each do |key, value|
self.set_header "HTTP_#{key}", value
end if $custom_headers
super
end
end
class ActionDispatch::Request
prepend RequestWithExtraHeaders
end
Then in my specs:
/specs/features/something_spec.rb
...
$custom_headers = {"Referer" => referer_string}
If you are using javacsript and only want to implement on chrome, Puppeteer is the best option as it has native support to modify headers.
Check this out: https://pptr.dev/#?product=Puppeteer&version=v10.1.0&show=api-pagesetextrahttpheadersheaders
Although for cross-browser usage you might check out #requestly/selenium npm package. It is a wrapper around requestly extension to enable easy integration in selenium-webdriver.The extension can modify headers.
Check out: https://www.npmjs.com/package/#requestly/selenium
Setting request headers in the web driver directly does not work. This is true.
However, you can work around this problem by using the browser devtools (I tested with edge & chrome) and this works perfectly.
According to the documentation, you have the possibility to add custom headers:
https://chromedevtools.github.io/devtools-protocol/tot/Network/
Please find below an example.
[Test]
public async Task AuthenticatedRequest()
{
await LogMessage("=== starting the test ===");
EdgeOptions options = new EdgeOptions {UseChromium = true};
options.AddArgument("no-sandbox");
var driver = new RemoteWebDriver(new Uri(_testsSettings.GridUrl), options.ToCapabilities(), TimeSpan.FromMinutes(3));
//Get DevTools
IDevTools devTools = driver;
//DevTools Session
var session = devTools.GetDevToolsSession();
var devToolsSession = session.GetVersionSpecificDomains<DevToolsSessionDomains>();
await devToolsSession.Network.Enable(new Network.EnableCommandSettings());
var extraHeader = new Network.Headers();
var data = await Base64KerberosTicket();
var headerValue = $"Negotiate {data}";
await LogMessage($"header values is {headerValue}");
extraHeader.Add("Authorization", headerValue);
await devToolsSession.Network.SetExtraHTTPHeaders(new Network.SetExtraHTTPHeadersCommandSettings
{
Headers = extraHeader
});
driver.Url = _testsSettings.TestUrl;
driver.Navigate();
driver.Quit();
await LogMessage("=== ending the test ===");
}
This is an example written in C# but the same shall probably work with java, python as well as the major platforms.
Hope it helps the community.
If you use the HtmlUnitDriver, you can set request headers by modifying the WebClient, like so:
final case class Header(name: String, value: String)
final class HtmlUnitDriverWithHeaders(headers: Seq[Header]) extends HtmlUnitDriver {
super.modifyWebClient {
val client = super.getWebClient
headers.foreach(h => client.addRequestHeader(h.name, h.value))
client
}
}
The headers will then be on all requests made by the web browser.
With the solutions already discussed above the most reliable one is using Browsermob-Proxy
But while working with the remote grid machine, Browsermob-proxy isn't really helpful.
This is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.
Add the ModHeader extension to the chrome browser
How to download the Modheader? Link
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));
// Set the Desired capabilities
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);
Go to the browser extensions and capture the Local Storage context ID of the ModHeader
Navigate to the URL of the ModHeader to set the Local Storage Context
.
// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");
Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2
Now add the headers to the request using Javascript
.
((Javascript)driver).executeScript(
"localStorage.setItem('profiles', JSON.stringify([{ title: 'Selenium', hideComment: true, appendMode: '',
headers: [
{enabled: true, name: 'token-1', value: 'value-1', comment: ''},
{enabled: true, name: 'token-2', value: 'value-2', comment: ''}
],
respHeaders: [],
filters: []
}]));");
Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.
Now navigate to the required web-application.
driver.get("your-desired-website");
You can do it with PhantomJSDriver.
PhantomJSDriver pd = ((PhantomJSDriver) ((WebDriverFacade) getDriver()).getProxiedDriver());
pd.executePhantomJS(
"this.onResourceRequested = function(request, net) {" +
" net.setHeader('header-name', 'header-value')" +
"};");
Using the request object, you can filter also so the header won't be set for every request.
If you just need to set the User-Agent header, there is an option for Chrome:
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"')
Now the browser sends User-Agent.
WebClient fileReader = new WebClient();
fileAddress = listItem;
//error here uri formats are not supported
fileReader.DownloadFile(fileAddress, saveTo);
here in fileaddress address coming
http://pcdev04.pcsolution.net:83\Update32\rts\RTSUpdate.dll
You need to replace \ with / in your URI string in order to have a valid address:
fileReader.DownloadFile(fileAddress.Replace("\\", "/"), saveTo);
You may try using WebClient Class to open File on internet
Take a look at this - example
Opens and download image file.
hope it helps
I'm trying to make a HTTP request from a Adobe Illustrator Script (this question is not really JSX-related, though) via BridgeTalk to a local Rails app using a Socket connection (as you you see — plenty that could go wrong ;) ).
Finally managed to get a request through, but the Rails app throws an error:
ERROR bad Request-Line `GET /test.json HTTP/1.1'.
Currently no special headers are sent — the set–up is pretty simple:
var connection = new Socket;
var response = '';
var host = '127.0.0.1:3000';
var path = '/test.json';
var method = 'GET';
var request = method + " " + path + " HTTP/1.1";
if (connection.open(host)) {
connection.write(request);
response = connection.read(999999);
} else {
$.writeln('Socket connection failed.');
}
connection.close();
I'm assuming that the request is illformed, but I'm not sure what's expected.
The Rails app runs on WEBrick.
Any help or hint on what Rails expects here, would be greatly appreciated.
Update
It seems that WEBrick expects a carriage return in the end of the request line: \r\n .
Unfortunately I’m using BridgeTalk to execute a piece of code within Bridge which requires me to use uneval() or toSource() on the function mentioned above. These seem to escape backslashes, converting
GET /test.json HTTP/1.1\r\n
into
GET /test.json HTTP/1.1\\r\\n
causing the same error.
Any ideas?
Took a while until I got this figured out, but as usual — “too many moving parts are making it hard to spot the problem”:
Defining the request using the encoded version of the carriage return and decoding on destination will help:
var request = decodeURI(method + " " + path + " HTTP/1.1%0d%0a");
However, it turns out that a proper request should end in \n\n not \r\n, so make this:
var request = decodeURI(method + " " + path + " HTTP/1.1%0a%0a");
Now the requests come through and now longer throw errors.