I am new to Gherkin and BDD.
We are doing BDD regression tests with Squish.
Our application is very complex - similar to a flight simulator.
Now I ask myself how to write the tests in Gherkin.
As we have a large number of variables as preconditions to a certain situation I would normally put a lot of
Given some precondition
And some other precondition
into my tests.
My natural feeling is that I should avoid this because it would make things unnecessarily complex.
Is there a rule of thumb for how many preconditions there should be?
Should I try to reduce it to only one?
The general rule is to have as few as possible while still making the tests useful. How many that is will depend on the audience for your scenarios.
If you are using gherkin scenarios for actual BDD (Behaviour Driven Development) then you will have to write the scenarios in such a way that your stakeholders can make sense of them. If this means that you have to write many Given, And, And steps then that is the way it has to be. If it means that you can put several of these steps into one more general set up step then that is better.
If you are using the Gherkin scenarios only as a way of automating tests then do whatever is good for your dev team. The rule about having as few steps as possible comes from trying to make sure everyone, technical and non technical understands what is meant by the scenario (i.e. that they are as clear AND concise as possible. If it is only the technical team that needs to understand it then as above you can use many steps if that is what is necessary for your team to understand it, or you can use fewer steps that contain more code.
For your case of having a very complex system to get into the correct state for testing, I wouldn't be worried about having may set up steps, so long as the scenarios you end up with are clear and as short as possible. There wouldn't be any point in having a smaller neater scenario that no one really understood without looking at the code!
When you are writing scenarios, you can make the code behind the steps do whatever you need to set the application in a state that is needed for testing. Thre is a pretty good article explaining the point here.
I find that if you can be more descriptive in your steps, then your tests will make more sense when you need to go back to reference them. If your steps are just Given, click, click, click, Then .. you can easily lose track of the point of the test. Your tests should be about system behaviors not step by step instructions for using the system.
So as far as preconditions are concerned. You need to do whatever it takes to get the application in the state that you wish to test.
Gherkin is a Business Readable, Domain Specific Language created especially for behavior descriptions.
Gherkin serves two purposes: serving as your project’s documentation and automated tests. Gherkin’s grammar is defined in the Treetop grammar that is part of the Cucumber codebase.
To understand Gherkin in a better way, please take a look at simple scenario mentioned below:
Feature: As a existing facebook user, I am able to post a birthday greeting on any other existing user's facebook page
Scenario: Verify that Joe Joseph can post a birthday greeting on Sam Joseph's facebook page
Given Joe Joseph is an existing facebook user
Given Sam Joseph is an existing facebook user
Given Joe Joseph is on login page of facebook
Given Joe Joseph logs into his facebook account
When Joe Joseph opens Sam Joseph's facebook page
And Joe Joseph writes "Happy Birthday Sam" on Sam Joseph's facebook page
And Joe Joseph clicks on Post button
Then Joe Joseph verifies that "Happy Birthday Sam" is successfully posted on Sam Joseph's facebook page
In the above scenario, all the statements that starts with "Given" are my preconditions.
So, as far as precondictions are concerned, you can use as many preconditions which is required for the test.
Gherkin is the language that Cucumber understands. It is a Business Readable, Domain Specific Language that lets you describe software’s behaviour without detailing how that behaviour is implemented More here
From above statement my understanding is. You don't need to add precondition every time as who is using scenario knows how to write and understand Gherkin language.
Here is example:
When I take login as system admin
And I click on new user
And I enter new user details
Then new user is created successful
Here I don't need to mention where I am and other precondition like URL opened or not ?
First precondition is URL opened or not.
Given I opened URL "http://www.stackoverflow.com"
And I enter system admin details
And I click on new user
When I enter new user details
Then new user is created successfully
Second Condition New User Details form have all the required field. This can be separate scenario.
Goal is tell business and team that we are testing this scenario. It is not test case. You don't need to create tons of document for testing purpose.
Like YAML or Python, Gherkin is a line-oriented language that uses indentation to define structure. Line endings terminate statements (called steps) and either spaces or tabs may be used for indentation. Finally, most lines in Gherkin start with a special keyword:
Sample
Feature: Some terse yet descriptive text of what is desired
In order to realize a named business value
As an explicit system actor
I want to gain some beneficial outcome which furthers the goal
Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
And yet another action
Then some testable outcome is achieved
And something else we can check happens too
Scenario: A different situation
...
The parser divides the input into features, scenarios and steps. Let’s walk through the above example:
Feature: Some terse yet descriptive text of what is desired starts the feature and gives it a title.
Scenario: Some determinable business situation starts the scenario, and contains a description of the scenario.
The next 7 lines are the scenario steps, each of which is matched to a regular expression defined elsewhere.
Scenario: A different situation starts the next scenario, and so on.
When you’re executing the feature, the trailing portion of each step (after keywords like Given, And, When, etc) is matched to a regular expression
Also note, if you have precondition like Given I opened URL "http://www.stackoverflow.com" as mentioned my #Boston. It is recomended to use it as Background.
Example
Feature: This is test feature
Background: background scenario
Given I opened URL "http://www.stackoverflow.com"
Scenario: access first senario
Given: I should see this
...
Scenario: access second scenario
Given: I should see that
...
The background will execute before every scenario.
Reference:
https://github.com/cucumber/cucumber/wiki/Gherkin
Related
I want to have an option on the cucumber report to mute/hide scenarios with a given tag from the results and numbers.
We have a bamboo build that runs our karate repository of features and scenarios. At the end it produces nice cucumber html reports. On the "overview-features.html" I would like to have an option added to the top right, which includes "Features", "Tags", "Steps" and "Failures", that says "Excluded Fails" or something like that. That when clicked provides the same exact information that the overview-features.html does, except that any scenario that's tagged with a special tag, for example #bug=abc-12345, is removed from the report and excluded from the numbers.
Why I need this. We have some existing scenarios that fail. They fail due to defects in our own software, that might not get fixed for 6 months to a year. We've tagged them with a specified tag, "#bug=abc-12345". I want them muted/excluded from the cucumber report that's produced at the end of the bamboo build for karate so I can quickly look at the number of passed features/scenarios and see if it's 100% or not. If it is, great that build is good. If not, I need to look into it further as we appear to have some regression. Without these scenarios that are expected to fail, and continue to fail until they're resolved, it is very tedious and time consuming to go through all the individual feature file reports and look at the failing scenarios and then look into why. I don't want them removed completely as when they start to pass I need to know so I can go back and remove the tag from the scenario.
Any ideas on how to accomplish this?
Karate 1.0 has overhauled the reporting system with the following key changes.
after the Runner completes you can massage the results and even re-try some tests
you can inject a custom HTML report renderer
This will require you to get into the details (some of this is not documented yet) and write some Java code. If that is not an option, you have to consider that what you are asking for is not supported by Karate.
If you are willing to go down that path, here are the links you need to get started.
a) Example of how to "post process" result-data before rendering a report: RetryTest.java and also see https://stackoverflow.com/a/67971681/143475
b) The code responsible for "pluggable" reports, where you can implement a new SuiteReports in theory. And in the Runner, there is a suiteReports() method you can call to provide your implementation.
Also note that there is an experimental "doc" keyword, by which you can inject custom HTML into a test-report: https://twitter.com/getkarate/status/1338892932691070976
Also see: https://twitter.com/KarateDSL/status/1427638609578967047
I want to have an option on the cucumber report to mute/hide scenarios with a given tag from the results and numbers.
We have a bamboo build that runs our karate repository of features and scenarios. At the end it produces nice cucumber html reports. On the "overview-features.html" I would like to have an option added to the top right, which includes "Features", "Tags", "Steps" and "Failures", that says "Excluded Fails" or something like that. That when clicked provides the same exact information that the overview-features.html does, except that any scenario that's tagged with a special tag, for example #bug=abc-12345, is removed from the report and excluded from the numbers.
Why I need this. We have some existing scenarios that fail. They fail due to defects in our own software, that might not get fixed for 6 months to a year. We've tagged them with a specified tag, "#bug=abc-12345". I want them muted/excluded from the cucumber report that's produced at the end of the bamboo build for karate so I can quickly look at the number of passed features/scenarios and see if it's 100% or not. If it is, great that build is good. If not, I need to look into it further as we appear to have some regression. Without these scenarios that are expected to fail, and continue to fail until they're resolved, it is very tedious and time consuming to go through all the individual feature file reports and look at the failing scenarios and then look into why. I don't want them removed completely as when they start to pass I need to know so I can go back and remove the tag from the scenario.
Any ideas on how to accomplish this?
Karate 1.0 has overhauled the reporting system with the following key changes.
after the Runner completes you can massage the results and even re-try some tests
you can inject a custom HTML report renderer
This will require you to get into the details (some of this is not documented yet) and write some Java code. If that is not an option, you have to consider that what you are asking for is not supported by Karate.
If you are willing to go down that path, here are the links you need to get started.
a) Example of how to "post process" result-data before rendering a report: RetryTest.java and also see https://stackoverflow.com/a/67971681/143475
b) The code responsible for "pluggable" reports, where you can implement a new SuiteReports in theory. And in the Runner, there is a suiteReports() method you can call to provide your implementation.
Also note that there is an experimental "doc" keyword, by which you can inject custom HTML into a test-report: https://twitter.com/getkarate/status/1338892932691070976
Also see: https://twitter.com/KarateDSL/status/1427638609578967047
I know litte bit bout blackbox functionality testing. But my supervisor asked me to do the form for my system's evaluations. And i don't know how to started it. I need guidances to build the form. Which topics should i include in the form?
Black box testing document covers mainly
Action on a particular field
Steps to be followed
Input
Expected Output
Result
Please confirm from your Supervisor, whether he/she expects the same from you
Suppose I have a website where I showcase people's artwork, recipes, and journals (and this list is likely to grow). I want people to be able to comment, like, flag, etc. all of them.
I'm not looking for a database schema. I am using ASP.net Web API but the platform should be irrelevant. I am passing an Authorization Bearer header to identify the user who wants to like, comment on, flag, etc. another user's work.
I currently have two "schemes" but I'm not sure what the advantages/disadvantages of each one is, so I can't make a good decision.
The first looks like this:
POST api/<entity>/{id}/<action>
For example, api/journal/21793/like, or api/recipe/1005/comment. Of course, each action would have the appropriate body that includes info to store to the back end.
Implementing it this way, though, would require us to write entityCount x interactionTypeCount functions (actions), which is very tedious, though the API call is friendly.
The second API scheme looks like this:
POST api/<action>
For example, api/like, api/comment - and yes, there would be no way to depict which object type and which record of the object type in the URL. So, if we were to comment on the recipe mentioned, we would call:
POST api/comment
{"id":"1005", "objectType":"recipe", "comment":"This was excellent!"}
This way, we call only 1 API to comment on anything in our system. So, each of the POST api/ functions would require the id and objectType properties in the body, plus whatever other required data as appropriate for the action. (I'm using JSON as an example).
I see that developers would have to know beforehand what our accepted objectType values are in order to post interactions to the right record. I'm not sure if this is option is a good idea.
This question might get flagged for being off-topic or open to debate, but I'm hoping someone can tell me more pros and cons of each approach indicated above so I can make a better decision. Better yet, offer a solution that is either totally different or combines aspects of the two approaches above.
We are in the process of designing a web site were users can fill an application form and submit it. Once an application is created, it goes through different departments for review.After each review, the department persons log on to the website and update the status of the application. Once review is completed, the application is said to be 'approved'. Also, communication(email) is sent to the applicants updating them about the status of their application.
I am thinking of using Windows Workflow for this application, but am new to it.
Can anyone tell me if it would make sense to use WF for such an app..or would it be an overkill..I was thinking of using a State machine workflow, as each application created goes through different states. One of my major concerns is, this process also involves manual work such as reviewing the application(this could take days) and then updating the status(say by clicking a button on the website). Can workflow handle this kind of thing?
I have gone through many websites, but they talk about very basic examples where states change automatically.But in my case there is a lot of human interaction.
Any help is appreciated
Thanks!!!
Here is a sample on MSDN that describes a very similar workflow that you've described - although they use a Windows Forms client rather than a web client - but those details shouldn't matter given the use of the Workflow Runtime.
I would say that your situation would be well suited to Workflow - long-running workflows are supported by having the runtime automatically persist workflow details at certain points while it runs, so steps that take days or weeks to fulfill are perfectly fine.
"my case there is a lot of human interaction".
That's the easiest kind of thing to build. It's just transactions.
Each stage in the process is a simple web application.
Fetch all items that are waiting at this state. Display them in a list. User picks one.
Display that piece of work. User makes changes. Saves it. Some change will move it to the next state. It's still just an update.
That's it. Nothing fancy.
You just need to very, very clearly define each state.
You must have a simple query that finds work in this state.
You must have a simple update that moves work on to the next state.
The states must be very, very clear and simple definitions. One column with a state name, for example.
The state transition rules, however, may be complicated. If the work is highly manual, it may be as simple as a drop-down list of available states.
If the state transitions are complex, then, perhaps you need something more sophisticated to embody the transition rules. But since it's manual, you don't need much.
Do I need a separate workflow instance for each application created?
I don't know what this can possibly mean. Each "application" that's moving through the pipeline of manual processing steps has a "state" -- the step of the pipeline where it's waiting, right now.
Each application also has a complete history of each state change. I don't know which of this is a "workflow" that could have an instance. They're just states of being of an object.
It is a web application. So it will have some other logic such as view an appln., navigation etc. do I need to accommodate this in the workflow as well?
Yes and No. Yes, each processing stage which show the application so the user at that stage can do whatever value-add thing they do. They add information or they approve information.
Each stage in the workflow is a place where some human being makes a decision and takes an action. So each stage displays all the information the person needs to make the decision. There's no additional view required.
Navigation isn't terribly interesting. People go to pages to see their queues of unprocessed work. I guess you'll have to build a page with some URL's, but there isn't much to that.
Would it be ok to have just a part of the application based on workflow.
Sure. Why not? I'm not sure I understand the question.
Your web apps just queries a queue of stuff out of the database and present that queue of stuff to a person. The person can request details and make changes to a specific item in the queue. One change the person can make will be to move the stuff out of their queue and into someone else's queue.
This isn't very complex. It shouldn't be. What's hard is defining the allowed state changes.