https://groups.google.com/group/caelyf/feed/rss_v2_0_topics.xml in a browser window correctly returns xml stream;
Using groovy in cloudfoundry app, this fails with http 403 permission failure like:
def url = "https://groups.google.com/group/caelyf/feed/rss_v2_0_topics.xml:443".toURL()
def tx = url.getText('UTF-8')
cloudfoundry forum implies only https plus port 443 can read an external url
any ideas ?
not sure why you stuck :443 on the end of the url?
403 means forbidden. I'm guessing Google doesn't let you scrape the groups site with java.
you could try setting the user agent to that of a browser?
def tx = url.openConnection().with {
setRequestProperty("User-Agent", "Firefox/2.0.0.4")
inputStream.with {
def ret = getText( 'UTF-8' )
close()
ret
}
}
or similar...
I don't think this is a cloudfoundry issue. have you tried running the above from your machine to confirm this?
Edit:
Just tried it, and it works (at least on my machine). This shows how to load the XMl into a parser, and print the titles from the feed:
URL url = "https://groups.google.com/group/caelyf/feed/rss_v2_0_topics.xml".toURL()
def tx = new XmlSlurper().with { x ->
url.openConnection().with {
// Pretend to be an old Firefox version
setRequestProperty("User-Agent", "Firefox/2.0.0.4")
// Get a reader
inputStream.withReader( 'UTF-8' ) {
// and parse it with the XmlSlurper
parse( it )
}
}
}
// Print all the titles
tx.channel.item.title.each { println it }
Related
Is there a way with QWebEngineView to intercept an http request, and to serve it server-less from the app ?
I heard about QWebEngineUrlRequestInterceptor and acceptNavigationRequest(), but they provide only inspection on requests, and redirection for get... But I would like to make the http response from the Qt app.
(I added pyqt in the tags because I would use it from python, but a c++ answer is acceptable too)
To intercept http request you will need to use this code:
// on app startup
QWebEngineProfile.defaultProfile().installUrlSchemeHandler(new
QByteArray("https"), new QWebEngineUrlSchemeHandler() {
#Override public void requestStarted(QWebEngineUrlRequestJob job) {
final String url = job.requestUrl().url();
if (**some url not ok condition**) {
job.fail(QWebEngineUrlRequestJob.Error.UrlInvalid);
}
String data = loadSomeData();
if (data != null) {
QBuffer buffer = new QBuffer();
// this is IMPORTANT! or you will have memory leaks
job.destroyed.connect(buffer::disposeLater);
buffer.open(QIODeviceBase.OpenModeFlag.WriteOnly);
buffer.write(data.getBytes(StandardCharsets.UTF_8));
buffer.close();
job.reply(new QByteArray("text/html"), buffer);
}
}
});
My versions is for QTJambi(Java), but it's not hard to convert it to C++/Python
The qt documentation says the redirection is only for GET request. However, when trying it out (PyQt6==6.4.0) we found out that this is actually not true. If you redirect a POST request in the WebEngine to a local webserver listening on localhost, you will actually receive the request including the payload.
Perhaps this is because Webkit doesn't include the payload for redirect and Chromium does? (I couldn't find docs stating the difference.)
from PyQt6.QtCore import QUrl
from PyQt6.QtWebEngineCore import QWebEngineUrlRequestInterceptor
class WebEngineUrlRequestInterceptor(QWebEngineUrlRequestInterceptor):
def interceptRequest(self, info):
method = info.requestMethod()
if method == "POST":
if info.requestUrl().url() == "https://my-url-to-something":
info.redirect(QUrl("http://127.0.0.1:8000"))
I am trying to play Widevine encrypted content on an Android TV application using Exoplayer. I have my video URL which is served from a CDN and acquired with a ticket. I have my widevine license URL, a ticket and a auth token for the license server.
I am creating a drmSessionManager, putting the necessary headers needed by the license server as follows:
UUID drmSchemeUuid = C.WIDEVINE_UUID;
mediaDrm = FrameworkMediaDrm.newInstance(drmSchemeUuid);
static final String USER_AGENT = "user-agent";
HttpMediaDrmCallback drmCallback = new HttpMediaDrmCallback("my-license-server", new DefaultHttpDataSourceFactory(USER_AGENT));
keyRequestProperties.put("ticket-header", ticket);
keyRequestProperties.put("token-header", token);
drmCallback.setKeyRequestProperty("ticket-header", ticket);
drmCallback.setKeyRequestProperty("token-header", token);
new DefaultDrmSessionManager(drmSchemeUuid, mediaDrm, drmCallback, keyRequestProperties)
After this Exoplayer handles most of the stuff, the following breakpoints are hit.
response = callback.executeKeyRequest(uuid, (KeyRequest) request);
in class DefaultDrmSession
return executePost(dataSourceFactory, url, request.getData(), requestProperties) in HttpMediaDrmCallback
I can observe that everything is fine till this point, the URL is correct, the headers are set fine.
in the following piece of code, I can observe that the dataSpec is fine, trying to POST a request to the license server with the correct data, but when making the connection the response code returns 405.
in class : DefaultHttpDataSource
in method : public long open(DataSpec dataSpec)
this.dataSpec = dataSpec;
this.bytesRead = 0;
this.bytesSkipped = 0;
transferInitializing(dataSpec);
try {
connection = makeConnection(dataSpec);
} catch (IOException e) {
throw new HttpDataSourceException("Unable to connect to " + dataSpec.uri.toString(), e,
dataSpec, HttpDataSourceException.TYPE_OPEN);
}
try {
responseCode = connection.getResponseCode();
responseMessage = connection.getResponseMessage();
} catch (IOException e) {
closeConnectionQuietly();
throw new HttpDataSourceException("Unable to connect to " + dataSpec.uri.toString(), e,
dataSpec, HttpDataSourceException.TYPE_OPEN);
}
When using postman to make a request to the URL, a GET request returns the following body with a response code of 405.
{
"Message": "The requested resource does not support http method 'GET'." }
a POST request also returns response code 405 but returns an empty body.
In both cases the following header is also returned, which I suppose the request must be accepting GET and POST requests.
Access-Control-Allow-Methods →GET, POST
I have no access to the configuration of the DRM server, and my contacts which are responsible of the DRM server tells me that POST requests must be working fine since there are clients which have managed to get the content to play from the same DRM server.
I am quite confused at the moment and think maybe I am missing some sort of configuration in exoplayer since I am quite new to the concept of DRMs.
Any help would be greatly appreciated.
We figured out the solution. The ticket supplied for the DRM license server was wrong. This works as it is supposed to now and the content is getting played. Just in case anyone somehow gets the same problem or is in need of a basic Widevine content playing code, this works fine at the moment.
Best regards.
I need to make body filter sleep before response
location /configure/result.php {
body_filter_by_lua_block {
--I am using ngx.arg[1] content return here if content contains somevalue then sleep
--Need to execute sleep code before response =>
ngx.sleep(60) --API disabled in the context of content_by_lua??
}
}
}
But i cant execute sleep function in body filter API disabled in the context of content_by_lua* is there any other method i can use
I rebuild source code to be able to use sleep function in body filters but it did not work the error "no co ctx was found" some suggestions would really help me i found out that i can use the (echo_sleep 10.0;) from nginx but it does so before the content from server has been requested
You can use access_by_lua to make request to "/" but then unfortunately you will have double the data request sent
I need to download a text file using basic authentication (the kind of authentication that prompts the browser to ask you for domain\username and password) using Groovy. I would like to avoid using additional libraries, isn't there anything to do this in in Groovy?
My current code is:
new File("test.txt").withOutputStream { out ->
def url = new URL(myurl).openConnection()
def remoteAuth = "Basic " + "myusername:mypassword".bytes.encodeBase64()
url.setRequestProperty("Authorization", remoteAuth);
out << url.inputStream
}
But the server replies with a 401 error. What should I do?
Groovy uses the java.net.Authenticator API. You can provide a default Authenticator using java.net.Authenticator#setDefault. An Example for an BasicAuth usage can be found in another Answer.
Authenticator.setDefault (new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication ("username", "password".toCharArray());
}
});
I'm using Retrofit (with RxJava) to download a text file.
#Streaming
#GET
Observable<ResponseBody> download(#Url String file);
Everything works fine except when user is using a public wifi that need web authentication. In this case download is still successful and what I receive is a file contains HTML of that authentication web page. How do I check and raise error in this case? I tried with Interceptor, NetworkInterceptor (Okhttp), disable redirect (Retrofit) but no luck because no exception is thrown.
Either checksum your file and check it when you get it, or check the Content-type:
#Streaming
#GET
Observable<Response<ResponseBody>> download(#Url String file);
download("http://...")
.map(response -> {
if(!"application/my-content-type".equals(
response.headers("Content-Type")) {
throw new RuntimeException("Bad download");
}
return resource.body();
})