I'm a rails developer, and I want to do some capybara+(rspec or cucumber) style of integration testing with cakephp 2.0. I was using PHPUnit (the default test framework for cake 2.0) but I don't know how to integrate it with Selenium and Cake at the same time to get the full stack effect that you have on rails
So, for the experience php devs that have used cake 2.0... how do you guys do a proper integration testing?
I'm not surprised there's been no answer on this - the community of testers within CakePHP seems to be extremely small. I've worked on large projects with huge numbers of functional and unit tests (currently a suite of around 1500 tests), but to do that I needed to extend the CakePHP test suite functionality (you can take a look at my now slightly outdated TDD plugin).
When we started we didn't know a whole lot about integration testing, and setting up a decent unit testing environment was enough of a challenge. We now use Rails :)
Basically, this is not a well-worn path. To get something working will require an extension to the framework - it will also require a new test case class, which extends PHPUnit_Extensions_Selenium2TestCase, and integrates any necessary functionality from CakeTestCase and ControllerTestCase.
I hope you get somewhere! If you do, it would be great if you could share the code with the rest of the community.
Related
So, I'm using Junit for Java and I've gotten convinsed of the benefit of TDD. Now I'm working (with a team) on a webapplication for my boss (html/css/php) and proposed to write some tests in PHPunit for the WA. But I can't find any way the code processes data (but the WA does work so far). I started google-ing and found only tutorials who work with functions in the code to test.
My question is: is it even possible to use PHPunit for testing the php-code from between the html-lines?
You're talking about functional testing rather than unit testing. Codeception (built on top of PHPUnit) might be a good place to start for you, since it has more natural functional testing out of the box.
Some example:
https://codeception.com/docs/02-GettingStarted
https://codeception.com/docs/03-AcceptanceTests
I am adding a typescript project to a VS2015 MVC 5 project (that makes it ASP.NET 4, not asp.net 5 or asp.net 6 code: only the MVC is version 5). This is the sole target for all aspects of my question, I cannot use generalized or theoretical guidance on typescript, node.js, module loaders, etc.
The problem is simpler with ASP.NET Core. But that's not what I'm facing. All the usual sources of examples and guidance avoid or provide scraps when it comes to ASP.NET 4 MVC 5 because it is hard. And no one will state exactly how hard or what precisely are the obstacles.
Worst, Typescript documentation is like Open Source Documentation: you can only get one issue, one step deep. This produces a research workflow consisting of endless-issue-tree-recursion.
I understand opinions, I even have one. But I'm looking for the experiential answer, what is the one combination that has proven to work for a production team.
So here are the specific items that need to be addressed and made to work within the confines of a working, medium-sized ASP.NET 4 MVC 5 LOB app:
Visual Studio's version of typescript. This is an installation issue (using node most simply) and the Tools/options - typescript settings have to match.
Browser-style testing (typically manual TDD workflow) or node.js testing (automated). This has to be chosen up-frontly to prevent more issue-tree-recursion. We are going with browser-based... phantomjs using WallabyJs.
NPM #types/library-name: supposed to fill a node_modules folder with both library-name and library-name.d.ts based only on a package.json with #Types references. But actually requires the package.json to hold a reference for both the #types/library-name and the library-name to work in my VS 2015 ENT v3 and asp.net 4 mvc 5 project. And all the versions specified then require manual correction' and even then the version look-up process is a little suspicious. This #types process may not be the way to go with ASP.NET 4 MVC 5, but I can't tell what the correction alternative might be. #Types is currently the only recommended option for typescript.
Which version of ECMAScript: es6 is apparently too far ahead. es2015 is likely, but this (maybe) appears to have relationships to several of the other issues. Supposedly, these designations are the same, but there are two places they can be set. I've chosen es2015 in tools/options/typescript. But getting any of these (now 3) settings wrong could be a problem.
Module system: CommonJs is for node and automated testing, and VS development testing is automated only for server-side, and VS UI tests are a manual process. So AMD, require JS is probably an option for VS, but it adds its own workflow and maintenance and considerations that are really hard to get right in ASP.NET. Using ASP.NET bundling and triple-slash references (dependable) might work, but after you have put the libraries in node-modules, you would want to use the full path into node-modules in the file name slug in an import statement. This is all very clumsy and involves the most guesswork. But solving this whole item might be the 'key' for the overall question.
There are probably a lot of other, smaller issues. But someone who has done this will have solved all the mentioned items and the others as well.
What I'm looking for is all the settings across all these issues in detail based on a working Typescript app in ASP.NET 4 MVC 5 implementation for browser-based unit/behavior tests in VS 2015. Those who have done it will understand.
Thanks very much for your consideration.
What you're missing is separation of concerns, in spite of the initial benefit of such starter templates, they start to cause incidental dependencies and complicate the mental model. It's much easier to have your front end in a separate project.
Regardless:
Visual Studio's version of typescript.
Always use the very latest available. This controls the version of TypeScript which powers the IDE. You will probably end up compiling in a separate process or in the browser during development. Again you will want to use the latest but it will likely be installed with a different package manager.
Browser-style testing (typically manual TDD workflow) or node.js
testing (automated). This has to be chosen up-frontly to prevent more
issue-tree-recursion.
Firstly, I definitely agree with the importance of choosing up front but, if it is still possible, just unpleasant, to add tests to an existing project.
TDD workflows involve automated testing as they rely on rapid feedback. This is orthogonal to whether you run your tests in the browser or using NodeJS.
You should use whichever approach makes the most sense for your application and that may be a mix of both.
Since you are writing a frontend JavaScript application you will likely want to run some tests in the browser. However, as Uncle Bob (Robert C. Martin) has stated, views should be dumb and require little testing. My interpretation of this is that we should not spend too much time testing things like Angular or React components to ensure that they render correctly, and instead focus on testing behavioral elements of the system such as services and plain old functions.
That said, you may well want to run tests of your client side services against an actual browser runtime, as opposed to just Node.js, and that is reasonable.
There are a number of testing libraries to help you with this. I do not have a specific recommendation besides to say that you should find a reliable test runner, and a simple assertion library. Tried and true testing libraries like QUnit and Tape are examples of solid options.
One last note, it is important that one not confuse the concept of integration testing with running tests in a web browser, it's perfectly valid to run TDD style tests, which implies unit tests, in a web browser.
NPM #types/library-name: supposed to fill a node_modules folder with
both library-name and library-name.d.ts, but requires the package.json
to hold a reference for both the #types/library-name and the
library-name to work in my VS 2015 ENT v3 and asp.net 4 mvc 5 project.
Simply put this goes to back to decoupling your front end from your back end. Visual Studio and certainly ASP.NET have nothing to do with versioning your types packages.
If a package comes with its own type declarations then you don't need to install an auxiliary types package otherwise you do.
Either way install JavaScript and TypeScript dependencies using a JavaScript oriented package manager (such as NPM, JSPM, or Yarn).
Do not use NuGet for these!
As you suggest there are versioning issues, it's currently a difficult problem in TypeScript. However once again it has nothing to do with ASP.NET or Visual Studio.
Which version of ECMAScript: es6 is apparently too far ahead. es2015
is likely, but this appears to have (maybe) relationships to several
of the other issues.
ES6 is the same as ES2015, the latter being the name under which the former was ultimately released. ECMAScript now follows a yearly cadence, roughly, with ES2017 just around the corner.
The nice thing about having a transpiler, such as TypeScript, is that you can use the latest features from es2017 and still target es5 for emit and you'll be fine.
Module system: CommonJS is for NodeJS and automated testing, and VS
development testing is automated only for server-side, and VS UI tests
are a manual process. So AMD/UMD require JS is probably the option for
VS, but it adds its own workflow and maintenance and considerations.
Using triple-slash references (dependable) might work, but after you
have put your/their libraries in node-modules you would want to use
the full path into node-modules in the file name slug in an import
statement. (solving this whole item might be the 'key' for the overall
question).
This is a very complex subject and probably the only one of your questions that you really need to spend a lot of time considering. As I said earlier using NodeJS or not is orthogonal to automated testing. But if you're targeting NodeJS natively with your test code then you will need to use CommonJS output.
For the actual application code, the choice has nothing at all remotely to do with whether or not you are using Visual Studio, I'm sorry for reiterating this but it really is important that you separate these ideas.
The question of which module format to use for your front end application code is a very important and contentious one.
Triple /// references are not a module format but rather a way of declaring the dependencies between global variables that are declared and referenced across multiple files.
They do not scale well, working acceptably when you have only a handful of files.
Triple /// references should not be used. They are not a modularity mechanism and their use is completely different from using any of the module systems/module formats you mention, including CommonJS.
Never combine them with a module system, which is what you would have to do in order to run your tests under NodeJS or load your app with RequireJS or anything else.
RequireJS is an excellent option which would imply AMD modules as you say. RequireJS does not require any use of triple slash references. In fact they should be avoided as the plague when using this format or any other module format!
I recommend strongly against using UMD modules. Isomorphic JavaScript is a problematic idea, and it offers you no benefits since you are creating a browser application with a .NET backend.
Many developers actually do use CommonJS modules in a browser. This requires bundling them continuously, using tools such as Webpack. This approach has advantages and disadvantages. The primary advantages are the ability to lean on existing NodeJS JavaScript server-side tools, such as npm, by way of Webpack or Browserify. This may not sound like a big advantage but the amount of rich tooling available for CommonJS modules is nothing to scoff at, making it a strong option.
Consider using the System module format and the SystemJS loader via jspm to both manage your packages and to load your code. With this approach, you gain the advantages of RequireJS, are able to run your tests under NodeJS and the browser using jspm run without needing to switch targets formats or bundle your code just to test it. There's also no need to bundle your code during development, although this is supported. More importantly, you gain the advantage of writing future compatible code, as it offers the only module format and loader which correctly models the semantics that ES Modules will eventually have when implemented natively in browsers. JSPM has first class support for TypeScript, Babel, and Traceur.
For posterity here is the description of the System module format taken from the link above:
System.register can be considered as a new module format designed to support the exact semantics of ES6 modules within ES5. It is a format that was developed out of collaboration and is supported as a module output in Traceur (as instantiate), Babel and TypeScript (as system). All dynamic binding and circular reference behaviors supported by ES6 modules are supported by this format. In this way it acts as a safe and comprehensive target format for the polyfill path into ES6 modules.
Disclaimer:
I am a member of the JSPM GitHub organization, playing a role in maintaining the registry and have made very minor contributions to the jspm cli.
Some years ago I´ve tried to work with BDD and some tools of Acceptance tests, like Selenium/Web-driver, Fitnesse and JBehave.
I´d like to back to work with that in my current project, so I would like to know what the community is using of tools to perform that!
My project is based on Java.
Acceptance tests tools ?
BDD tools?
Should I consider a scripting language ?
The most successful teams are using conversations and not worrying too much about the tools!
Having said that, here are a few Java BDD tools that are in use in the community:
JBehave* (still)
Cucumber for the JVM (I don't think it's been as maven-ized as JBehave)
Fitnesse (though I recommend putting "Slim" behind it instead of "Fit")
Custom DSLs (it's not that hard).
Selenium is still the automation tool of choice for Java and the web.
*I helped write JBehave. One reason we got into it in the first place was because the acceptance tests we saw using scripts were such an astonishing mess. Meh. Also, you can't collaborate with the business or have conversations about scripts. I strongly recommend having conversations first**, then worrying about the tools!
** If you're working on your own, buy a rubber duck.
I strongly recommend spockframework + Geb. You need groovy support though. We have lot of tests running as part of CI every night. The reports are in junit format (being enhanced to be used by business users soon) and hence can be published to servers like Hudson or Sonar.
I am trying to get a simple prototype using ASP.NET S#arp-architecture (Sharp architecture) to run on a shared hosting site. This site runs in medium trust, which appears to give problems with this framework. At first I went off on a wild goose chase with the mySQL.data.dll, but the latest version already supports partially trusted callers.
The only way I found up until now is this blogpost, which basically describes re-compiling everything from the ground up, starting with the various Castle components. Is this the only way to get this to work, and more importantly, does this actually work?
I have had some good experiences with S#arp-architecture, and since I only have a few days to build this prototype, I figured S#arp-architecture would be a good match. So, my options are to stick with SA, or on the other end of the spectrum to hand-roll an MVC/NH/mySQL application. Other suggestions (using NHibernate and ASP.NET MVC) are also welcome.
Two-fold answer: re-compiling everything does work. I got a simple S#arp-architecture app to work at this hosting provider. I did not want to risk going over my prototype deadline however, so for the demo I built a very straightforward, simple app with (gasp!) straight ADO.NET.
What is the best way to test my webforms applications?
Looks like people are loving Watin, and selenium.
Just wondering, why would you call WatiN a unit testing tool? Last time I checked, it ran integration tests.
The best way would be to move all code that doesn't depend on HttpContext to a separate assembly and run unit tests as usual. The rest can be tested with Ivonna. She doesn't test the client behavior, that's where WatiN can be helpful; however, if you want to test your pages or controls in isolation, she's your only choice.
UPDATE: Given WatiN has been stagnant for over a year now, I would direct anyone that needs web ui tests towards selenium, it is in continuous use & development by many contributors, and is actively used by Google.
WatiN is the best that I've found. It integrates into Visual Studio unit testing or nunit & you can do pretty much anything you need in the browser (click links, submit forms, look for text/images, etc.)
See the following questions for similar answers:
What is the best way to do unit testing for ASP web pages (C#)?
Web Application Testing for .Net (watin Test Recorder)
How do you programmatically fill in a form and ‘POST’ a web page?
That's the biggest shortcoming of Webforms -- it's, for all practical reasons, untestable in terms of unit testing of testing controllers, etc.
That is one of the major advantages of the MVC framework.
I tend to favor the approach of separating the buisness logic out of the UI code.
Here's an article that describes a unit test friendly pattern (Model-View-Presenter)
http://www.unit-testing.net/CurrentArticle/How-To-Use-Model-View-Presenter-With-AspNet-WebForms.html
I would use a tool like WaitIn:
" WatiN is Web Application Testing in .NET, and this Test Recorder will generate chunks of source for you by recording your clicks in an embedded IE browser" (from Scott Hanselman's blog - which I found thanks to another post on StackOverflow
WaitIn website
I'd go with WATIR (Web Application Testing in Ruby) - http://wtr.rubyforge.org/. We (Acsys Interactive) have been using for about a year and the tool is great.
I developed a simple wrapper in .NET so that I can execute my WATIR scripts from Unit tests. The framework is incredible and you have entire Ruby power behind you. There's support for Firefox & Safari (FireWatir project).
It's very similar to WATIN (in fact I think WATIN was inspired by WATIR) but I find that WATIR community is much larger than WATIN one.
There're test recorders out there that you can use and tons of tutorials.
It's really your choice. If you feel like the tests need to be in .NET and you don't want to support any other language then your choice is WATIN. On the other hand, if you want to try a fun and quite powerful scripting language (that's what Ruby is) then go for WATIR.
Question to WATIN guys, does it support FireFox/Safari?
Here is a review of Watin,Watir and Selenium
http://adamesterline.com/2007/04/23/watin-watir-and-selenium-reviewed/
Apparently Selenium worked quite slow for the tester but if you'll notice, as one of the comments points out, that this is only the case due to its support of multiple browsers.
However there is a CTP (Community Technology Preview) release of WatiN which offers support for both Internet Explorer and FireFox automation.
I have had a great experience using Selenium. Web tests can be very fragile, but here is an article from my blog where I talk about how to make the tests less fragile.
http://www.unit-testing.net/CurrentArticle/How-To-Make-Web-Tests-Less-Fragile.html