Background I: *.phpt in phpunit
I recently read an article about *.phpt support in PhpUnit:
https://www.moxio.com/blog/32/start-testing-with-phpt-tests-in-phpunit
The big advantages here:
Supported out of the box with phpunit, no need for custom code to discover and deal with those files.
Common format that everybody seems to agree upon.
One can directly specify a specific *.phpt as a cli parameter.
A typical *.phpt file would look like this:
--TEST--
Basic arithmetic - addition
--FILE--
<?php
var_dump(42 + 1);
?>
--EXPECT--
int(43)
Background II: Self-updating fixtures
Earlier than that, I read about unit tests that can update (= overwrite) their own fixture files if a cli variable is present:
https://tomasvotruba.com/blog/2020/07/20/how-to-update-hundreds-of-test-fixtures-with-single-phpunit-run/
The following cli command would then overwrite the "expected" part in fixtures for failing tests:
$ UPDATE_TESTS=1 ./vendor/bin/phpunit
One would then review the git diff, and either accept the changes as desired functionality change, or fix the behavior.
The format of fixture files proposed in this article is very similar to the *.phpt file format, but in the end the format can be completely arbitrary and project-specific.
Benefits:
Start with empty "--EXPECT--" section, and have this part generated automatically.
Automatically update tests after behavior change, e.g. after a bug was fixed, or when BC-breaking changes were introduced.
Frequently update fixtures during early development, when behavior is not really stable yet.
No more copy/paste from cli diff output.
Disadvantages:
No built-in support in phpunit. Requires custom code. This does not scale well, if I want to do it in multiple projects.
One cannot target the *.phpt file as a cli parameter.
Question
Is there a built-in way to auto-update those *.phpt test files with phpunit?
Or should I perhaps use something other than phpunit?
Related
I'm trying to obtain a dependency graph (either as an image or in text-form) from a bazel cquery. According to the documentation, the option --output=graph is currently only supported by bazel query, but not by cquery. Unfortunately, in our project it's not possible to use query since it fetches some external dependencies with restricted access. Only using a config (with cquery) prevents fetching these restricted dependencies.
Is there a work-around to somehow get a graph-like structure from cquery? The default output is just a flattened list which seems to contain no information on the inter-dependencies between the targets.
If the inter-dependencies can somehow be printed, I guess it would be quite easy to reconstruct an image from it.
The following works: Using query instead of cquery and appending the flag --keep_going to ignore errors caused by external dependencies that cannot be fetched by everybody. Then --output=graph can be used.
Note: The result might be different from a configured cquery, but for our purposes, it doesn't matter much.
With symfony console commands you can prefix/group each one by putting in the setName("group:command"), and this is great.
myown
myown:cool
myown:foo
myown:bar
But the problem is that some external dependencies dont use this format. Ex: Phinx Migrations.
Since I'm importing an dependency that has a console command called migrate, I dont want for it to show without prefix/group. Not just because I might have another command called migrate or just for readability. I don't event know if we have 2 with same command name which one will show (need to check).
My question is: Is there any away for me to force a group commands from an external depencency to be inside an prefix/group?
You can achieve that, I wouldn't recommend but that would be the approach:
Create a compiler Pass which removes the definitions of the Commands you don't like
Register again all those commands while setting the names you like
If you need information on compiler passes:
https://symfony.com/doc/current/components/dependency_injection/compilation.html#components-di-compiler-pass
Maybe there's an easier way which I'm not aware, but for now that's my answer to you, I can't post all the code because that would be a lot of code and if you do it maybe you can update with the solution.
Good luck
How do you make unit tests for the HTML output of your PHP functions/scripts, specifically to check that the output is HTML5 valid?
Currently a can test functionality in PHPUnit and presentation with online copy/paste validators. But it would be much nicer if this could be integrated into the PHPUnit testing.
Is there a standard way to go about such things, or is it mainly a matter of using regular unit tests on functions which create the inserted content, and then making sure it looks correct in the browser/W3C Validator?
Similar question for older version of PHPUnit that no longer applies:
Unit tests for HTML Output?
What you looking for is behavior testing. Take a look at Behat
The Twine project (http://twineproject.sourceforge.net/doc/phphtml.html) replaces the copy/paste manual process. It might be useful; it still sends the HTML to the w3C site each time, which is not ideal for unit tests. (The W3C says all their stuff is open source and so you might be able to download it and run it locally... I couldn't find the download link though!)
An alternative approach is to use DomDocument::validate() However it requires the DTD to be referenced inside, and as this answer https://stackoverflow.com/a/15245834/841830 explains, HTML5 has no DTD.
(I'm assuming you meant that you have functions that return HTML5 strings, and you want to unit test those functions: if you want to test the whole output of a web app, e.g. as run through Apache and seen in a browser, I would use CasperJS or Selenium. But this is high-level functional testing, notably slower to run than unit tests, so I recommend to unit test what can be unit tested: and I still cannot find an offline HTML5 validator for Casper/Phantom/Slimer nor for Selenium!)
I have a rather big library with a significant set of APIs that I need to expose. In fact, I'd like to expose the whole thing. There is a lot of namespacing going on, like:
FooLibrary.Bar
FooLibrary.Qux.Rumps
FooLibrary.Qux.Scrooge
..
Basically, what I would like to do is make sure that the user can access that whole namespace. I have had a whole bunch of trouble with this, and I'm totally new to closure, so I thought I'd ask for some input.
First, I need closurebuilder.py to send the full list of files to the closure compiler. This doesn't seem supported: --namespace Foo does not include Foo.Bar. --input only allows a single file, not a directory. Nor can I simply send my list of files to the closure compiler directly, because my code is also requiring things like "goog.assers", so I do need the resolver.
In fact, the only solution I can see is having a FooLibrary.ExposeAPI JS file that #require's everything. Surely that can't be right?
This is my main issue.
However, later the closure compiler, with ADVANCED_OPTIMIZATIONS on, will optimize all these names away. Now I can fix that by adding "#export" all over the place, which I am not happy about, but should work. I suppose it would also be valid to use an extern here. Or I could simply disable advanced optimizations.
What I can't do, apparently, is say "export FooLibrary.*". Wouldn't that make sense?
Finally, for working in source mode, I need to do goog.require() for every namespace I am using. This is merely an inconvenience, though I am mentioning because it sort of related to my trouble above. I would prefer to be able to do:
goog.requireRecursively('FooLibrary')
in order to pull all the child namespaces as well; thus, recreating with a single command the environment that I have when I am using the compiled version of my library.
I feel like I am possibly misunderstanding some things, or how Closure is supposed to be used. I'd be interested in looking at other Closure-based libraries to see how they solve this.
You are discovering that Closure-compiler is built more for the end consumer and not as much for the library author.
If you are exporting basically everything, then you would be better off with SIMPLE_OPTIMIZATIONS. I would still highly encourage you to maintain compatibility of your library with ADVANCED_OPTIMIZATIONS so that users can compile the library source with their project.
First, I need closurebuilder.py to send the full list of files to the closure compiler. ...
In fact, the only solution I can see is having a FooLibrary.ExposeAPI JS file that #require's everything. Surely that can't be right?
You would need to specify an --root of your source folder and specify the namespaces of the leaf nodes of your file dependency tree. You may have better luck with the now deprecated CalcDeps.py script. I still use it for some projects.
What I can't do, apparently, is say "export FooLibrary.*". Wouldn't that make sense?
You can't do that because it only makes sense based on the final usage. You as the library writer wish to export everything, but perhaps a consumer of your library wishes to include the source (uncompiled) version and have more dead code elimination. Library authors are stuck in a kind of middle ground between SIMPLE and ADVANCED optimization levels.
What I have done for this case is maintain a separate exports file for my namespace that exports everything. When compiling a standalone version of my library for distribution, the exports file is included in the compilation. However I can still include the library source (without the exports) into a project and get full dead code elimination. The work/payoff balance of this though must be weighed against just using SIMPLE_OPTIMIZATIONS for the standalone library.
My GeolocationMarker library has an example of this strategy.
We have an autotools project that has a mixture of unit and integration tests, all of which run via 'make check'. This isn't ideal, as some of the integration tests take a while, and have all sorts of dependencies (database, etc.)
I'd like to separate the integration tests and assign them their own make target. That way, unit tests can still be run often (via make check), and the integration tests can be run as needed in a similar fashion.
Is there a straightforward (or otherwise) way to add an additional make target?
NOTE: I should probably also add that this is a large project, so editing/maintaining every makefile by hand is not desirable. I'd like to do it the 'autotools way' if possible.
-- UPDATE 1 --
I've attempted Jon's solution, and it's a step closer, but not quite there. I still have a couple of issues:
1) Recursion - I'm OK with modifying the makefile.am in the root of the build tree, as well as any directory that contains the tests, but it seems like there should be a way to do this where I don't have to change every Makefile.am in the hierarchy. (the check target works this way, after all)
2) .PHONY - I keep getting messages about .PHONY being redefined. Which is understandable, because it's being set by another package (specifically, doxygen). How do I make the two play nice together?
In your am files, all make syntax is passed into the generated Makefile. So if you want a new target just create it like you would in a Makefile and it will appear in the auto generated Makefile. Put the following at the bottom of your am files.
integration-tests: prerequisites....
commands to run test
.PHONY: integration-tests
Since there haven't been any more responses, I'm going to answer with my solution.
I solved the recursion issue by eliminating the recursion altogether. Using this page
as a guide, I switched the entire project from recursive make to non-recursive make. I then cloned the non-recursive check-related targets (check, check-am, check-TESTS, etc.) into a new set of targets for the integration tests. So far, this works extremely well.
Note: you may be wondering why I didn't just clone the recursive targets instead. Quite frankly, I couldn't find them. Either I didn't know where to look (the rules weren't in the generated Makefile) or something is happening implicitly, and I don't understand autotools well enough to follow it.
As for the issue with .PHONY being redefined, I still haven't found a solution, other than to conditionally exclude the other definition when I'm doing the integration tests.