I need to call Crawler4j from a different class. Instead of the main method in the Controller class I used a simple method called setup.
class Controller {
public void setup(String seed) {
try {
String rootFolder = "data/crawler";
int numberOfCrawlers = 1;
CrawlConfig config = new CrawlConfig();
config.setCrawlStorageFolder(rootFolder);
config.setPolitenessDelay(300);
config.setMaxDepthOfCrawling(1);
PageFetcher pageFetcher = new PageFetcher(config);
RobotstxtConfig robotstxtConfig = new RobotstxtConfig();
RobotstxtServer robotstxtServer = new RobotstxtServer(robotstxtConfig, pageFetcher);
CrawlController controller = new CrawlController(config, pageFetcher, robotstxtServer);
controller.addSeed(seed);
controller.setCustomData(seed);
controller.start(MyCrawler.class, numberOfCrawlers);
} catch(Exception e) {
e.printStackTrace();
}
}
}
Tried to call it like this in another class, but props up an error.
Controller c = new Controller();
c.setup(seed);
Is it possible to not have a main method in the Controller class and still run crawler4j. In short, I would like to know how to integrate the crawler in to my application which already has a main method. Help would be appreciated.
There should be no problem running Crawler like you want. The code below is tested and will work like expected:
public class Controller {
public void setup(String seed) {
try {
String rootFolder = "data/crawler";
int numberOfCrawlers = 4;
CrawlConfig config = new CrawlConfig();
config.setCrawlStorageFolder(rootFolder);
config.setPolitenessDelay(300);
config.setMaxDepthOfCrawling(2);
PageFetcher pageFetcher = new PageFetcher(config);
RobotstxtConfig robotstxtConfig = new RobotstxtConfig();
RobotstxtServer robotstxtServer = new RobotstxtServer(robotstxtConfig, pageFetcher);
CrawlController controller = new CrawlController(config, pageFetcher, robotstxtServer);
controller.addSeed(seed);
controller.setCustomData(seed);
controller.start(BasicCrawler.class, numberOfCrawlers);
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws Exception {
Controller crawler = new Controller();
crawler.setup("http://www.ics.uci.edu/");
}
}
Sorry I forgot to place an access modifier "public" before the class name. Hence the error. Thank you for your answer.
Related
I have the following POST servlet that adds new node under certain resource with parameters(name and last nam) from the request:
#Component(
service = Servlet.class,
property = {
"sling.servlet.paths=/bin/createuser",
"sling.servlet.methods=" + HttpConstants.METHOD_POST
})
public class CreateNodeServlet extends SlingAllMethodsServlet {
/**
* Logger
*/
private static final Logger log = LoggerFactory.getLogger(CreateNodeServlet.class);
#Override
protected void doPost(final SlingHttpServletRequest req, final SlingHttpServletResponse resp) throws IOException {
log.info("Inside CreateNodeServlet");
ResourceResolver resourceResolver = req.getResourceResolver();
final Resource resource = resourceResolver.getResource("/content/test/us/en");
String name = req.getParameter("name");
String lastname = req.getParameter("lastname");
log.info("name :{}",name);
log.info("lastname :{}",lastname);
Node node = resource.adaptTo(Node.class);
try {
log.info("Node {}", node.getName() );
Node newNode = node.addNode(name+lastname, "nt:unstructured");
newNode.setProperty("name", name);
newNode.setProperty("lastname", lastname);
resourceResolver.commit();
} catch (RepositoryException e) {
e.printStackTrace();
} catch (PersistenceException e) {
e.printStackTrace();
}
resp.setStatus(200);
resp.getWriter().write("Simple Post Test");
}
}
I tried creating unit test for this I got this so far:
#ExtendWith(AemContextExtension.class)
class CreateNodeServletTest {
private final AemContext context = new AemContext();
private CreateNodeServlet createNodeServlet = new CreateNodeServlet();
#Test
void doPost() throws IOException, JSONException {
context.currentPage(context.pageManager().getPage("/bin/createuser"));
context.currentResource(context.resourceResolver().getResource("/bin/createuser"));
context.requestPathInfo().setResourcePath("/bin/createuser");
MockSlingHttpServletRequest request = context.request();
MockSlingHttpServletResponse response = context.response();
createNodeServlet.doPost(request, response);
JSONArray output = new JSONArray(context.response().getOutputAsString());
assertEquals("Simple Post Test", output);
}
}
however this is not working I am getting null pointer on this line
Node node = resource.adaptTo(Node.class);
can some one help what I am missing and some tips will be of great help as I am new to AEM, and there is not much resources about unit testing sling servlets ?
I think you need to register JCR_MOCK as resource resolver type
new AemContext(ResourceResolverType.JCR_MOCK);
I am a newbie to Windows Application Driver and my project demands automating the desktop application, so I decided to use winappdriver as it is similar to selenium, on which I am pretty confident about using.
speaking of the issue,
Just wondering if there is a way to achieve explicit wait and implicit wait using winappdriver. Following is the code i used as part of my test cases, the test fails with an exception (NoSuchElementException), however, if I put a static wait in place instead of explicit wait, it works as expected.
//Driver Setup
public class OscBase {
public static WindowsDriver<WebElement> applicaitonSession, driver = null;
public static WindowsDriver<RemoteWebElement> desktopSession = null;
public static DesiredCapabilities capabilities, cap1, cap2;
public static ProcessBuilder pBuilder;
public static Process p;
public void startDriver() {
try {
pBuilder = new ProcessBuilder("C:\\Program Files (x86)\\Windows Application Driver\\WinAppDriver.exe");
pBuilder.inheritIO();
p = pBuilder.start();
}
catch (IOException e) {
e.printStackTrace();
}
}
public void stopDriver() {
p.destroy();
}
public void createDesktopSession() throws MalformedURLException {
cap1 = new DesiredCapabilities();
cap1.setCapability("app", "Root");
desktopSession = new WindowsDriver<RemoteWebElement>(new URL("http://localhost:4723"), cap1);
}
public void openApplication() throws InterruptedException, MalformedURLException {
if (driver == null) {
try {
capabilities = new DesiredCapabilities();
capabilities.setCapability("app",
"Appnamewithlocation");
applicaitonSession = new WindowsDriver<WebElement>(new URL("http://localhost:4723"),
capabilities);
} catch (Exception e) {
System.out.println("Application opened!!!");
} finally {
createDesktopSession();
}
Thread.sleep(8000L);
String handle = desktopSession.findElementByAccessibilityId("InstallerView5")
.getAttribute("NativeWindowHandle");
System.out.println(handle);
int inthandle = Integer.parseInt(handle);
String hexHandle = Integer.toHexString(inthandle);
//System.out.println(hexHandle);
cap2 = new DesiredCapabilities();
cap2.setCapability("appTopLevelWindow", hexHandle);
driver = new WindowsDriver<WebElement>(new URL("http://localhost:4723"), cap2);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
}
}
public boolean isDisplayed_SafeLoginNoBtn() {
wait = new WebDriverWait(driver, 40);
return wait.until(ExpectedConditions.visibilityOf(safeLoginNoBtn())).isDisplayed();
}
#Test
public void osc_Get_Data() throws InterruptedException, IOException {
//Thread.sleep(20000);
// Boolean value=oscLogin.safeLoginNoBtn().isDisplayed();
try {
Boolean value = oscLogin.isDisplayed_SafeLoginNoBtn();
System.out.println("IS displayed========>"+value);
if (value) {
oscLogin.click_safeLogin();
}
} catch (Exception e) {
System.out.println("Safe Login!!!!");
}
Of course yes, the WebDriverWait class will work. Here's an example
WebDriverWait waitForMe = new WebDriverWait();
WebDriverWait waitForMe = new WebDriverWait(session, new TimeSpan.Fromseconds(10));
var txtLocation = session.FindElementByName("Enter a location");
waitForMe.Until(pred => txtLocation.Displayed);
I've created a detailed course about UI Automation using WinAppDriver and C# .Net. I'll be publishing it in a few days. Do let me know if you're interested :)
Can someone explain to me how to create an instance of this component in a Moq TestMethod? Here is the definition of the class. I need to test the ProcessAutomaticFillRequest method.
public class AutomaticDispenserComponent : IAutomaticDispenserComponent
{
private readonly Lazy<IMessageQueueComponent> _messageQueueComponent;
protected IMessageQueueComponent MessageQueueComponent { get { return _messageQueueComponent.Value; } }
public AutomaticDispenserComponent(Func<IMessageQueueComponent> messageQueueComponentFactory)
{
_messageQueueComponent = new Lazy<IMessageQueueComponent>(messageQueueComponentFactory);
}
public void ProcessAutomaticFillRequest(FillRequestParamDataContract fillRequestParam)
{
if (fillRequestParam.PrescriptionServiceUniqueId == Guid.Empty)
throw new InvalidOperationException("No prescription service was specified for processing fill request.");
if (fillRequestParam.Dispenser == null)
throw new InvalidOperationException("No dispenser was specified for processing fill request.");
var userContext = GlobalContext.CurrentUserContext;
var channel = string.Format(Channel.FillRequest, userContext.TenantId,
userContext.PharmacyUid, fillRequestParam.Dispenser.DeviceAgentUniqueId);
NotificationServer.Publish(channel, fillRequestParam);
}
Here is how I started my test, but I don't know how to create an instance of the component:
[TestMethod]
[ExpectedException(typeof (InvalidOperationException))]
public void FillRequestFailsWhenPrescriptionServiceUniqueIdIsEmpty()
{
// How do I create an instance of automatiqueDispenserComponent here
// since there is Func as constructor parameter?
var fillRequestParam = new FillRequestParamDataContract
{
PrescriptionServiceUniqueId = Guid.Empty
};
_automaticDispensercomponent.ProcessAutomaticFillRequest(fillRequestParam);
// ...
}
Updated the answer based on the comments below. You need to mock the Func parameter for the test.
[TestMethod]
[ExpectedException(typeof(InvalidOperationException))]
public void FillRequestFailsWhenPrescriptionServiceUniqueIdIsEmpty()
{
var mockMsgQueueComponent = new Mock<Func<IMessageQueueComponent>>();
var _automaticDispensercomponent = new AutomaticDispenserComponent
(mockMsgQueueComponent.Object);
var fillRequestParam = new FillRequestParamDataContract
{
PrescriptionServiceUniqueId = Guid.Empty
};
_automaticDispensercomponent.ProcessAutomaticFillRequest(fillRequestParam);
}
How to moq the OfType to return
class A
{
public void iterate()
{
foreach (var aTool in Handler.Tools.OfType<IDrawTool>())
{
//some code
}
}
}
//tools implementes both interface ITool and IDrawTool
//wanted to get the tool which implements the IDrawTool from the
//Handler.Tools<ITool> list
Mock<List<ITool>> aToolMockList = new Mock<ITool>();
Mock<IDrawTool> adrawToolMock = new Mock<IDrawTool>();
aToolMockList.Setup(list => list.OfType<IDrawTool>()).Returns((IEnumerable<IDrawTool>) adrawToolMock .Object);
But the OfType throws the exception setup on a non-member method
Should I override this? What should I do?
List<ITool> aToolMockList = new List<ITool>();
Mock<IDrawTool> adrawToolMock = new Mock<IDrawTool>();
Mock<ITool> aToolMock = new Mock<ITool>;
//additional setups
aToolMockList.Add(adrawToolMock.object);
aToolMockList.Add(aToolMock.object);
//
A a = new A(aToolMockList);
A.iterate();
//
adrawToolMock.Toolwork.Verify(Times.Once);
aToolMock.Toolwork.Verify(Times.Never);
Something like that :)
I am attempting to write some Tests for a small project of mine but they seem to fail (I am starting with 1 test 'Create_Class')
I use the repository pattern and use Constructor Dependency Injection:
public KlasController() {
db = ObjectContextPerHttpRequest.Context;
KlasRepo = new KlasRepository(db);
LesRepo = new LesRepository(db);
OpdrachtRepo = new OpdrachtRepository(db);
}
//dependency injection constructor
public KlasController(IKlasRepository KlasRepo, ILesRepository LesRepo,
IOpdrachtRepository OpdrachtRepo) {
this.KlasRepo = KlasRepo;
this.LesRepo = LesRepo;
this.OpdrachtRepo = OpdrachtRepo;
}
here is my TestClass with testinitializer (which runs before every test) and the first test
[TestClass()]
public class KlasControllerTest
{
private KlasController Controller;
private IOpdrachtRepository OpdrachtRepo;
//Use TestInitialize to run code before running each test
[TestInitialize()]
public void MyTestInitialize()
{
OpdrachtRepo = new DummyOpdrachtRepository();
Controller = new KlasController(new DummyKlasRepository(),
new DummyLesRepository(), OpdrachtRepo);
Opdracht TestOpdracht = new Opdracht
{
OpdrachtID = 1,
VakID = 1,
StamNummer = "im1"
};
Vak TestVak = new Vak { VakID = 1, VakNaam = "FOOP" };
TestOpdracht.Vak = TestVak;
OpdrachtRepo.addOpdracht(TestOpdracht);
}
/// <summary>
///A test for Index
///</summary>
[TestMethod()]
public void CreateKlasDirectsToToonKlassen()
{
Klas Klas = new Klas { KlasNaam = "2dNet" };
RedirectToRouteResult view = Controller.Create(1) as RedirectToRouteResult;
Assert.IsNotNull(view);
Assert.AreEqual("ToonKlassen", view.RouteValues["action"]);
}
}
at the moment I get a nullreferenceException on the view (assert.isNotNull fails)
and here is one of my DummyRepository's:
class DummyOpdrachtRepository : IOpdrachtRepository
{
List<Opdracht> opdrachten;
public DummyOpdrachtRepository()
{
opdrachten = new List<Opdracht>();
}
public void addOpdracht(Opdracht opdracht)
{
opdrachten.Add(opdracht);
}
public string GetDocentID(int OpdrachtID)
{
var opdracht = opdrachten.Where(o => o.OpdrachtID == OpdrachtID).FirstOrDefault();
return opdracht.StamNummer;
}
public Opdracht Find(int id)
{
return opdrachten.Where(o => o.OpdrachtID == id).FirstOrDefault();
}
}
Normally I should have written the tests Before writting the code, I know (and I am convinced off TDD, as I have used it successfully in my latest Java-project). but it just doesn't seem to work..
here is the code for KlasController.Create action
public ActionResult Create(int id) //id = opdrachtID
{
var Opdracht = OpdrachtRepo.Find(id);
Vak vak;
if(Opdracht != null)
vak = Opdracht.Vak;
else
throw new NullReferenceException("Deze opdracht werd niet gevonden");
return View(new CreateKlasModel(id,vak));
}
I know this is a lot of code, but I really want to make this work.
Thanks for helping me out in advance :)
As vladimir77 already says in his comment, the method public ActionResult Create(int id) is of type ViewResult, so either you change you method to do areturn RedirectToRoute() or you change your test to
ViewResult view = Controller.Create(1);
Assert.IsNotNull(view);
A ViewResult can not be cast as a RedirectToRouteResult.