I use htmlunit 2.36.0 and try to scrape:
https://delightful.dussmann.com/menu/B%C3%BCropark%20Bredeney/B%C3%BCropark%20Bredeney
Somehow no dynamic content fetch is executed via javascript.
Anyone any ideas how to fix it?
#Test
public void testPDFFetch() throws IOException {
String url = "https://delightful.dussmann.com/menu/B%C3%BCropark%20Bredeney/B%C3%BCropark%20Bredeney";
WebClient client = new WebClient(BrowserVersion.CHROME);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(true);
client.getOptions().setCssEnabled(true);
client.getOptions().setUseInsecureSSL(true);
client.setAjaxController(new AjaxController() {
#Override
public boolean processSynchron(HtmlPage page, WebRequest request, boolean async) {
return true;
}
});
try {
HtmlPage page = client.getPage(url);
// page.wait(20000);
client.waitForBackgroundJavaScript(10000);
client.waitForBackgroundJavaScriptStartingBefore(10000);
Thread.sleep(10000);
System.out.println(page.asXml());
} catch (Exception e) {
e.printStackTrace();
} finally {
client.close();
}
}
Looks like this page uses javascript modules - this is currently not supported by HtmlUnit (Rhino)
Related
I am receiving the following runtime-error when attempting to load an image via ImageSource assignment.
Cannot use the specified Stream as a Windows Runtime IRandomAccessStream because this Stream does not support seeking.
Specifically, I am passing an ImageSource object from one page to another page.
Thus, I am surprised that I am receiving this error.
I have even tried extracting the stream object from the ImageSource object.
However, this does not appear to be supported.
Can anyone provide guidance on how I can resolve this issue?
Same problem i was facing,
Please follow following step.
1.In your xaml page add like this a img tag
Image x:Name="productImage" Source="{Binding SelectedSalesItem.Source}"
2. on any grid call ItemSelected="OnSalesItemSelected" event called bellow method.
Public async void OnSalesItemSelected(object sender, EventArgs args) {
try
{
if (SaleVm.SelectedSalesItem != null)
{
OnImages();//Call point no 3 method "OnImages"
}
}
catch (Exception ex)
{
throw ex;
}
}
3.Create a method in your xaml.cs page.
async void OnImages() {
try
{
if (SaleVm.SelectedSalesItem != null)
{
IFolder rootFolder = FileSystem.Current.LocalStorage;
IFolder imageFolder = await rootFolder.GetFolderAsync("ItemsImg");
IFile imageFile = await imageFolder.GetFileAsync(SaleVm.SelectedSalesItem.ItemCode + ".png");
var stream = await imageFile.OpenAsync(FileAccess.Read);
productImage.Source = ImageSource.FromStream(() => stream);
}
}
catch (Exception ex)
{
throw ex;
}
}
4. Call "OnImages()" method from other class like ViewModel.
and change image at run time.
I've set up SignalR Realtime communication for my ASP.NET based website. However, I need to know if it's possible to share the same SignalR Hub with a separate Mobile Backend Project which deals with the same server and database. Basically like accessing the facebook's inbox through web and mobile at the same time?
That would be a ASP.NET Web project and a Mobile Client connected to one hub.
If you want to find SignalR for Android, I suggest the following working link for your to start
SignalR/java-client at GitHub
You can refer to the following sample code (this is from my question on SO about SignalR for Android: SignalR for Android: how can I pass dynamic class to SubscriptionHandler1
public <T> void startSignalR(String transport, String serverUrl, final String userName, final Class<T> tClass) {
Platform.loadPlatformComponent(new AndroidPlatformComponent());
Credentials credentials = new Credentials() {
#Override
public void prepareRequest(Request request) {
request.addHeader(HEADER_KEY_USERNAME, userName);
}
};
mConnection = new HubConnection(serverUrl);
mConnection.setCredentials(credentials);
mHub = mConnection.createHubProxy(SERVER_HUB_CHAT);
if (transport.equals("ServerSentEvents")) {
mTransport = new ServerSentEventsTransport(mConnection.getLogger());
} else if (transport.equals("LongPolling")) {
mTransport = new LongPollingTransport(mConnection.getLogger());
}
mAwaitConnection = mConnection.start(mTransport);
try {
mAwaitConnection.get();
} catch (InterruptedException e) {
e.printStackTrace();
return;
} catch (ExecutionException e) {
e.printStackTrace();
return;
}
mHub.on("broadcastMessage",
new SubscriptionHandler1<Object>() {
#Override
public void run(final Object msg) {
final String finalMsg;
Gson gson = new Gson();
Object object = gson.fromJson(msg.toString(), tClass);
Field[] fields = object.getClass().getDeclaredFields();
for (int i = 0; i < fields.length; i++) {
try {
System.out.println("Value = " + fields[i].get(object));
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
, Object.class);
...
}
Hope this helps!
I have created a dynamic web project in eclipse.
It contains a Servlet ResidentApi.java and two java classes:GeoLocationDemo.java and Geolocation.java. I am calling GeoLocationDemo.java from my servlet and getting result in a ResultSet.But i am not getting any value in ResultSet.
When i ran same GeoLocationDemo.java separatly i am getting right results.I don't know servlet is able to call my java class or not but if it is then why i am not getting results.
I am having hard time debugging it.What i am doing is running .war file of this project every time on tomcat server and checking results there.Please suggest a good method to test it on eclipse.
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
response.setContentType("text/html");
PrintWriter out = response.getWriter();
ResultSet rs = null;
try{
GeoLocationDemo geo = new GeoLocationDemo(); //Here i created a new object
rs = geo.getResults(); //here i called a method of GeoLocation
}catch(Exception e){
// System.out.println(e);
out.write("<head><b You suck</b></head>");
}
out.write("<head><b>Congratulation! connected</b></head>"); //i am getting this output
try{
while(rs.next()){
String s = rs.getString("Details");
out.write("<head><b> "+s+ " </b></head>"); //not able to get this output
}
}catch(Exception e){
// System.out.println(e);
out.write("<head><b>You Built </b></head>");
}
out.close();
}
Don't put everything in a <head> tag! Open a body somewhere. Don't silently swallow any Exception you might be getting. Do remember to close() the ResultSet. Also, you should probably be returning all of your data in a List with a POJO.
out.write("<body>");
out.write("<b>Congratulation! connected</b><br/>");
try {
while (rs.next()) {
String s = rs.getString("Details");
out.write("<b> "+s+ "</b><br/>");
}
} catch (Exception e){
// System.out.println(e);
out.write("<b>" + e.getMessage() + "</b>");
e.printStackTrace(out);
} finally {
try {
out.close();
} catch (Exception ignored) {
}
try {
rs.close();
} catch (Exception ignored) {
}
}
configure remote debugging between your IDE and Tomcat server e.g. take a look on Remote debugging Tomcat with Eclipse
localize the problem - is the problem in GeoLocationDemo or in ResultSet or in output processing
p.s. do not close resources you never open - out.close(); - it is managed by servlet container
I've implemented a custom exception filter to my Web API. It is working as intended, except for one small detail...
In the following code sample, SaveToErrorLog saves exception details and tries to get the request url from context.Request.RawUrl. But context.Request does not contain the url that the API tried to serve when the exception happened. Is there a way to get the url when using an exception filter like this?
public class APIExceptionFilter : ExceptionFilterAttribute
{
private HttpContextBase context;
public APIExceptionFilter()
{
context = new HttpContextWrapper(HttpContext.Current);
}
public override void OnException(HttpActionExecutedContext actionContext)
{
actionContext.Response = new HttpResponseMessage(HttpStatusCode.InternalServerError);
if (actionContext != null && context != null)
{
facade.SaveToErrorLog(actionContext.Exception, context.Request);
}
throw new HttpResponseException(new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
Content = new StringContent(actionContext.Exception.Message),
ReasonPhrase = "APIException"
});
}
}
As per the comment above by #emre_nevayeshirazi, you need to use the HttpActionExecutedContext. This gives you access to the request and then the required Uri.
public override void OnException(HttpActionExecutedContext actionExecutedContext)
{
var requestedUri = actionExecutedContext.Request.RequestUri;
//Do something
}
I was trying to write an interceptor using spring AOP.The interceptor will find if a request URL is a bookmark,if so will redirect to the authentication page.
Code Snippet:
public Object invoke(MethodInvocation invocation) throws Throwable {
logger.entering(this.getClass().getSimpleName(), "invoke", invocation);
Object result = null;
try {
// Logic to exclude the beans as per the list in the configuration.
boolean excluded = false;
for (String excludebean : excludedBeans) {
if (excludebean != null && excludebean.equalsIgnoreCase(invocation.getThis().getClass().getSimpleName())) {
excluded = true;
break;
}
}
// If the Target Method is "toString", then set EXCLUDE to TRUE and process the request
if(excluded == false && invocation.getMethod().getName().equalsIgnoreCase("toString"))
{
excluded = true;
}
// if user session object is available, then process the request or
// else forward to the configured view.
if (excluded || getSessionHolder().getUserVO() != null) {
result = invocation.proceed();
}
else {
logger.logp(Level.INFO, this.getClass().getSimpleName(),
"invoke(MethodInvocation)", "User Object is "+ getSessionHolder().getUserVO()
+ ". So redirecting user to home page");
result = new ModelAndView("redirect:/security/authenticate.do");
}
}
catch (Throwable ex) {
throw ex;
}
logger.exiting(this.getClass().getSimpleName(), "invoke");
return result;
}
When I debug the control comes inside the else block as expected but after I return the result,control goes to the handle method for the bookmarked URl ratehr than the handler for the redirect view.
Pls help me on this..Thanks in advance.
Why Do you need AOP for the interceptor. You can redirect easily using Regular interceptor.
public class RedirectInterceptor extends HandlerInterceptorAdapter{
private String redirectMapping;
public void setRedirectMapping(String redirectMapping) {
this.redirectMapping = **maintenanceMapping**;
}
//before the actual handler will be executed
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response, Object handler)
throws Exception {
if (somethingHappened){
response.sendRedirect(redirectMapping);
return false;
} else
return true;
}
}