I am wanting to use htmlunit to scrape data from a website. I am passing the address as an attribute from a form. I keep getting errors, it says the "java.lang.NoClassDefFoundError: com/gargoylesoftware/htmlunit/WebClient" even though I have the .jar files imported and the javadoc file location set correctly. Am I missing something?
package coreservlets;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlDivision;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
#WebServlet("/WebScrape")
#SuppressWarnings("serial")
public class WebScrape extends HttpServlet {
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
PrintWriter out = response.getWriter();
// Create and initialize WebClient object
final WebClient webClient = new WebClient();
String Address = (String) request.getAttribute("address");
HtmlPage page = webClient.getPage(Address);
final HtmlDivision div = (HtmlDivision) page.getByXPath("//*[#id=\"LDPOffMarketPropertyInfo\"]//div//ul//li[4]//span[1]//text()");
out.println("<!DOCTYPE html>\n" +
"<html>\n" +
"<head>\n" +
"<meta name=" + "\"viewport\" " + "content=" + "\"initial-scale=1.0, user-scalable=no\" " + "/>\n" +
"<style type=" + "\"text/css\">\n" +
" html { height: 100% }\n" +
" body { height: 100%; margin: 0; padding: 0 }\n" +
" #default { height: 800px;\n"+
" width: 400px; }\n" +
" </style>\n" + div);
}
}
Assuming you use Eclipse, the build path is just that: the set of libraries used to build the application.
You also need the libraries to be available at runtime, inside your webapp. The servlet specification explains where the libraries of a webapp must go: in WEB-INF/lib.
Remove the jar files from the build path, and drop them in the folder WEB-INF/lib of your WebContent directory. This will automatically add them back to the build path, and also make them part of the deployed application, and thus available at runtime.
They will also appear under the Web App libraries node in the package explorer of Eclipse, confirming that these libraries are part of the webapp.
Related
Hi guys i am beginner in alfresco.I have done many services such as creating folder,subfolder,uploading document,downloading document,creating permissions using cmis.
But i am not able to create link of one folder to another folder using cmis.
Somebody told me its not possible using cmis.
Somehow i got this link http://basanagowdapatil.blogspot.in/2011/01/code-for-creating-links-in-alfresco.html.
But this code is not in cmis.
I have never done this kind of programming.
Can somebody suggest me how to do this program in maven.
What dependency or jars i should add.
It is better if someone explain me step by step(in sense how to give authentication).
Thanks in advance
I got my answer and we can do the same using CMIS API.
import java.util.HashMap;
import java.util.Map;
import org.apache.chemistry.opencmis.client.api.Folder;
import org.apache.chemistry.opencmis.client.api.Session;
import org.apache.chemistry.opencmis.commons.PropertyIds;
import org.apache.chemistry.opencmis.commons.enums.BaseTypeId;
import org.apache.log4j.BasicConfigurator;
import org.apache.log4j.Logger;
import com.bizruntime.alfresco.session.CreateSession;
import com.bizruntime.alfresco.util.Config;
public class CreateLink {
static Logger log = Logger.getLogger(CreateLink.class);
public static void getLink() {
// creating Session
Session cmiSession = new CreateSession().getSession();
log.debug("Session Created...");
Map<String,Object> properties = new HashMap<>();
properties.put(PropertyIds.BASE_TYPE_ID, BaseTypeId.CMIS_ITEM.value());
// Define a name and description for the link
properties.put(PropertyIds.NAME, Config.getConfig().getProperty("nameOfLink"));
properties.put("cmis:description", Config.getConfig().getProperty("linkDescription"));
properties.put(PropertyIds.OBJECT_TYPE_ID, "I:app:filelink");
// Define the destination node reference
properties.put("cm:destination", Config.getConfig().getProperty("destination-nodRef"));
// Choose the folder where the link to be create
Folder rootFoler = cmiSession.getRootFolder();
Folder targerFolder = (Folder) cmiSession.getObjectByPath(rootFoler.getPath() + Config.getConfig().getProperty("targetFolder"));
cmiSession.createItem(properties, targerFolder);
log.info("Link Created Successfully....");
}
public static void main(String[] args) {
BasicConfigurator.configure();
CreateLink cl = new CreateLink();
cl.getLink();
}
}
Code for creating folder link:
import java.util.HashMap;
import java.util.Map;
import org.apache.chemistry.opencmis.client.api.Folder;
import org.apache.chemistry.opencmis.client.api.Session;
import org.apache.chemistry.opencmis.commons.PropertyIds;
import org.apache.chemistry.opencmis.commons.enums.BaseTypeId;
import org.apache.log4j.BasicConfigurator;
import org.apache.log4j.Logger;
import com.bizruntime.alfresco.session.CreateSession;
import com.bizruntime.alfresco.util.Config;
public class CreateLink {
static Logger log = Logger.getLogger(CreateLink.class);
public static void getLink() {
// creating Session
Session cmiSession = new CreateSession().getSession();
log.debug("Session Created...");
Map<String,Object> properties = new HashMap<>();
properties.put(PropertyIds.BASE_TYPE_ID, BaseTypeId.CMIS_ITEM.value());
// Define a name and description for the link
properties.put(PropertyIds.NAME, Config.getConfig().getProperty("nameOfLink"));
properties.put("cmis:description", Config.getConfig().getProperty("linkDescription"));
properties.put(PropertyIds.OBJECT_TYPE_ID, "I:app:filelink");
// Define the destination node reference
properties.put("cm:destination", Config.getConfig().getProperty("destination-nodRef"));
// Choose the folder where the link to be create
Folder rootFoler = cmiSession.getRootFolder();
Folder targerFolder = (Folder) cmiSession.getObjectByPath(rootFoler.getPath() + Config.getConfig().getProperty("targetFolder"));
cmiSession.createItem(properties, targerFolder);
log.info("Link Created Successfully....");
}
public static void main(String[] args) {
BasicConfigurator.configure();
CreateLink cl = new CreateLink();
cl.getLink();
}
}
i have made a sample servlet program and on compiling it is giving error. Error says that none of my included packages exists. following is both my servlet class and errors.
classpath :
.;C:\Program
Files\Java\jdk1.7.0_45\bin;D:\apache-tomcat-7.0.37\lib\servlet-api.jar;D:\apache-tomcat-7.0.37\lib\jsp-api;
Path
%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program
Files (x86)\Intel\OpenCL SDK\2.0\bin\x86;C:\Program Files
(x86)\Intel\OpenCL SDK\2.0\bin\x64;C:\Program
Files\Java\jdk1.7.0_45\bin;D:\apache-tomcat-7.0.37\bin
Servlet class (Ch1Servlet.java)
import javax.servlet.*;
import javax.servlet.http.*;
import javax.io.*;
public class Ch1Servlet extends HttpServlet
{
public void doGet (HttpServletRequest request , HttpServletResponse response) throws IOException
{
PrintWriter out = response.getWriter();
java.util.Date today = new java.util.Date();
out.prtinln( "<html> " + "<body> " +
"<h1 align = center> HF\'s chapter1 servlet <h1>"
+ "<br>" +today+ "</body> " + "</html>"
) ;
}
}
ERROR
There was one error :
i have written -- >
import javax.io.*;
it should be -- >
import java.io.*;
and rest of the errors got sorted out after i restarted my system.
I have a webapplication for displaying images using imageio on a servlet. The webapp works fine when hosted in tomcat 7 using jre 1.6. But the same webapp when deployed on tomcat 5.5 with servlet 2.4 and Jre 1.5 it doesnt work. To access the image i pass an identifier as parameter and i get the image which is a blob column in DB. Initially i developed the application on tomcat 7 instance using eclipse and it worked fine. Compiler version compatibility is selected as 1.5 and Dynamic web module version is 2.4. In the below code i have checked the jdbc connectivity, it works fine and i can display textual information. Also i have checked separately whether the BufferedImage is having some data by checking for null and it seemed to have data. But the servlet simply fails to showup the image and i just end up getting a blank screen.
works on tomcat 7
[http://localhost:8081/testimage/ReturnImage?code=AUS]
doesnt work on tomcat 5.5
[http://localhost:8080/testimage/ReturnImage?code=AUS]
The servlet just displays a blank screen for the latter.
Below is the code for my servlet.
package flags;
import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintWriter;
import javax.imageio.ImageIO;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.sql.*;
import java.awt.image.BufferedImage;
public class ReturnImage extends HttpServlet {
private static final long serialVersionUID = 1L;
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
//PrintWriter out = response.getWriter();
// OutputStream outimg = response.getOutputStream();
try {
Class.forName("com.mysql.jdbc.Driver");
Connection connection = DriverManager.getConnection(
"jdbc:mysql://192.168.2.2:3306/world", "root", "abcdef");
//out.println("Connecting to database <br>");
Statement statement = connection.createStatement();
String param;
param = request.getParameter("code");
String sql = "Select Name,Flag,Code from world.Country where Code='"+ param + "'";
ResultSet res = statement.executeQuery(sql);
while (res.next()) {
String Name = res.getString("Name");
String Code = res.getString("Code");
BufferedImage image = javax.imageio.ImageIO.read(res.getBlob("Flag").getBinaryStream());
//out.println(System.getProperty("java.runtime.version"));
//out.println(Code + " ");
//out.println(Name + "<br>");
if (image == null) {
//out.println("null image");
}
ImageIO.write(image, "gif", outimg);
outimg.close();
}
res.close();
statement.close();
connection.close();
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
Image shows the servlet output in two different tomcat instances. Tomcat 5.5 uses JDK1.5 and Tomcat 7 uses JDK 1.6
You probably need the setContentType set correctly (as you currently do).
However, have you tried using a different format, like PNG (and of course, use "image/png" as content type)? I'm not sure Java 1.5 has a GIF writer, due to the LZW licensing issues of the past. Note that the ImageIO.write methods have a boolean return type, to check if anything was written. In any case, PNG should always work.
PS: Unless you are planning to modify the image in the servlet, it is of course much faster to store the image in the right format in the blob, and just passing it down to the client without decoding/encoding it.
I am using the below code to upload a file in to tomcat5.5 and it gives me the following exception
java.lang.NoClassDefFoundError: org/apache/commons/io/output/DeferredFileOutputStream
could you please help me to find it out?
import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.List;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileItemFactory;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
/**
* Servlet implementation class FileUploadServlet
*/
public class FileUploadServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
/**
* #see HttpServlet#HttpServlet()
*/
public FileUploadServlet() {
super();
// TODO Auto-generated constructor stub
}
/**
* #see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
#SuppressWarnings("rawtypes")
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
}
/**
* #see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
boolean isMultipart = ServletFileUpload.isMultipartContent(request);
System.out.println("Status : "+isMultipart);
if (isMultipart) {
FileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
try {
List items = upload.parseRequest(request);
Iterator iterator = items.iterator();
while (iterator.hasNext()) {
FileItem item = (FileItem) iterator.next();
if (!item.isFormField()) {
String fileName = item.getName();
String root = getServletContext().getRealPath("/");
File path = new File(root + "/uploads");
if (!path.exists()) {
boolean status = path.mkdirs();
}
File uploadedFile = new File(path + "/" + fileName);
System.out.println(uploadedFile.getAbsolutePath());
item.write(uploadedFile);
}
}
} catch (FileUploadException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
response.sendRedirect("upload.jsp");
}
}
this is the jar i use commons-fileupload-1.2.2.jar
The particular exception message is telling you that the mentioned class is missing in the classpath. As the org.apache.commons.io package name hints, the mentioned class is part of the http://commons.apache.org/io project.
And indeed, Commons FileUpload has Commons IO as a dependency. You need to download and drop commons-io.jar in the /WEB-INF/lib as well.
See also:
How to upload files to server using JSP/Servlet?
How to add JAR libraries to WAR project without facing java.lang.ClassNotFoundException? Classpath vs Build Path vs /WEB-INF/lib
How do I import the javax.servlet API in my Eclipse project?
use maven dependency
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
</dependency>
or download commons-io.1.3.2.jar to your lib folder
Solution
By default, Struts is using Apache “commons-io.jar” for its file upload process. To fix it, you have to include this library into your project dependency library folder.
Get Directly
Get “commons-io.jar” from official website – http://commons.apache.org/io/
Get From Maven
The prefer way is get the “commons-io.jar” from Maven repository
File : pom.xml
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>1.4</version>
</dependency>
just put all apache comons jar and file upload jar in lib folder of tomcat
If you are receiving this error in a WebSphere container, then make sure you set your Apps class loading policy correctly. I had to change mine from the default to 'parent last' and also ‘Single class loader for application’ for the WAR policy. This is because in my case the commons-io*.jar was packaged with in the application, so it had to be loaded first.
You will have to download file from here https://commons.apache.org/proper/commons-io/download_io.cgi and select https://prnt.sc/tk5ewt
Now, Next add this downloaded files into your project:
Right click to your project ->Build path->Configure BuidPath -> https://prnt.sc/tk5d93
I'm trying to process a large text file through a HttpServlet (tomcat).
As this file can be large and the process should be very fast, I don't want to upload the file on the server and I've used the method HttpRequest.getInputStream to process the input on the fly. For example, I want to transform the input to upper-case with the code below:
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
#SuppressWarnings("serial")
public class EchoServlet extends HttpServlet
{
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException
{
OutputStream out=null;
InputStream in=req.getInputStream();
if(in==null) throw new IOException("input is null");
try
{
resp.setContentType("text/plain");
out=resp.getOutputStream();
int c;
while((c=in.read())!=-1)
{
out.write((char)Character.toUpperCase(c));
}
}
catch(IOException err)
{
//ignore
}
finally
{
if(out!=null) out.flush();
if(out!=null) out.close();
in.close();
}
}
}
I invoked my servlet with CURL:
curl -s -d #big.file.txt "http://localhost:8080/test/toupper"
1) processing the input on the fly through a servlet, is it a good/common practice ?
2) my code seems to remove the carriage return ('\n') . Why ?
Thanks
1) processing the input on the fly through a servlet, is it a good/common practice ?
Depends on the functional requirement. I would personally have used a servlet which accepts HTTP multipart/form-data requests instead of raw request bodies. This way it's reuseable on normal HTML forms.
2) my code seems to remove the carriage return ('\n') . Why ?
The given code example doesn't do that. Maybe you've oversimplified it and you was originally using BufferedReader#readLine() while not using PrintWriter#println(). The readLine() indeed eats CRLFs.
There are more issues/inefficiencies in the given code snippet as well, but I can't give a decent answer on that as you seem not to actually be running the code as you posted in the question.