Can LUIS reliably differentiate between questions and statements?
We are designing a process flow which will differ depending on whether an utterance contains a question. We can't rely on users using obvious tokens like question marks.
I could train a model with a number of utterances with question type syntax and flag those as Question intent, while those without have a None intent? Seems like this might be a fairly standard requirement and wonder if there's some existing generic solution
LUIS is intended to differentiate between utterances not only based on the question marks. However the tokens that appear iteratively in one intent and not in al others would be used to distinguish the difference based on the learnt models. If you replicate all the questions in one intent in the none intent without the question mark the only identifier that would evolve is the question mark. Which apparently is not what you want. This is also not the function of the none-intent. The question could present itself in multiple forms and what you want is to cover these forms and then add none-question type statements that are not directly related to the none-intent. This would capture tokens that are relevant like "What is", "Where is" "What if" etc... and not just the question mark
Related
I'm attempting to write a "trivia quiz" skill, like many others that already exist, but I would like the user to be able to give actual answers rather than "a/b/c" or "1/2/3". So the conversation will go like this:
Alexa: Which planet is closest to the Sun: Mercury or Venus?
User: Mercury
Alexa: That's correct! You have one point. Which company makes the Corvette, Cadillac or Chevrolet?
User: Chevrolet
Alexa: That's right! You have two points. What were George Washington's false teeth made from? Wood or ivory?
...etc...
Since each question has its own set of answers, I'm happy to create one custom slot type per question, so I'd have a LIST_Q1_ANSWERS slot of ["Mercury", "Venus"] and a LIST_Q2_ANSWERS slot of ["Cadillac", "Chevrolet"], and that's all fine. However, I do not know how to tell my skill that answers should come from this particular custom slot only.
I can of course create a specific Intent for each question, so I create Q1Intent, I start a Dialog, and I Elicit my Q1Intent. However, this doesn't work out, because in my response to their filling in the required LIST_Q1_ANSWERS slot, I have to say "correct" and also ask the next question, which means that I have to Elicit the Q2Intent... and that's not allowed; I'd have to end the Dialog first, and then start a new one. (So the conversation would go "Which planet...?" "Mercury" "Correct! Do you want the next question?" "Yes" "OK. Which company..." and that's no good.)
I may be over-complicating things here. Perhaps there's an easier way to model the voice interface that I want. I can of course define all the different answers in one big custom slot, and then just have one single AnswerIntent, but then people can answer Chevrolet to the planets question and that's silly. (I know that I have to cope with any answers to a question, not just the ones in the slot, but I'd like to bias the question towards choosing answers from the slot; that's what slots are for.)
So, how should I be doing this? Dialogs won't work, I don't think.
That is a very reasonable request, IMO, but it is still not possible with Alexa. With some other bot/AI platforms you can specify a context that causes it to try to match the user response against a subset of your skills intents (usually just 1, I would think).
Here is a popular Alexa feature request that gets at the same point:
Allow temporarily disabling intents/utterances based on application state
https://forums.developer.amazon.com/content/idea/40045/allow-temporarily-disabling-intentsutterances-base.html
Yes, I believe that you can do exactly what you want. I did something similar in my "Who's On First?" baseball skit skill.
There are probably many other ways to accomplish this also, but I took the following approach:
Create an intent for each user response. For example, in the above example you stated, "mercury" and "chevrolet" would each be intents.
Pass information forward in the session object. A database could be used, but the session works well in this case. This information will allow your intent handlers to identify the question that was asked. This isn't really necessary if every answer is unique, but it allows multiple questions to have the same answer. For example, maybe the answer "mercury" is an answer to both questions "name a planet" as well as "name a liquid metal".
Since there will be more possible incorrect answers than correct answers, it might make sense to use slots for the incorrect answers. Alternatively, you might just handle unmatched intents as incorrect answers, and use the question ID passed in the session to identify which question was incorrect.
You can put together the response string programmatically. So if the answer is correct, prepend "Correct, the next question is " to the beginning of the next question your skill says.
I hope this gives you an idea for how to proceed. A lot is possible with Alexa. We just need to get creative sometimes.
From the documentation: "Questionnaire is a separately identifiable Resource, whereas the individual questions are not.". What is the best strategy to identify individual questions?
a. Can I maintain individual questions as single question questionnaires and embed them in a container Questionnaire?
b. Is there any other way to maintain questions separately and refer them in a container Questionnaire?
The DataElement resource would allow you to define questions separately and reference them. There's an extension on Questionnaire that lets you point a question or a group at a data element. Good practice is still to include the question text, data type, etc. in the Questionnaire instance in addition to pointing to the data element to ensure the questionnaire makes sense to systems that might not understand the extension or have access to the data elements, but it's technically legal to have the question only contain a link to the data element and nothing else if you're confident the consumers of the Questionnaire will know what to do with that.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
We using Behaviour Driven Development to develop a SOA system using Scrum and have come across two approaches to producing the stories.
Approach 1
Given Specific Message Type is available
And Specific State exists
When the Message is processed
Then expected resulting state exists
Approach 2
Given a Specific state exists
When Specific Message Type is processed
Then expected resulting state exists
Few if any of the examples available are applied to testing SOA systems. I would appreciate any experiences of these or any insights on the consequences of each approach.
We are aiming for declarative rather than imperative stories. The message arrival in the first has a slightly imperative feel but I'm not confident the second approach covers acceptance criteria adequately, because it doesn't seem to account for the event driven nature of the SUT.
The aim of the story is to communicate with your customer, so whatever style promotes that goal is best - and that will vary from one team to another. I might prefer 'when some business event occurs' rather than your suggestions, but I don't know your team! Beware of trying to find a 'one-size-fits-all' template, use whatever communicates best for each situation. And the heart of agile is the ability to adapt - try one style and feel free to adapt if it doesn't seem to be working.
Whenever I'm producing a library or service, I often find it useful to provide an example of the kind of scenario which a service user might want. So for instance:
Given the server has information about risk limits for Lieumoney Brothers
And we are $2m from those limits
When we process EOD orders that take us to $1m for those limits
Then our status with Lieumoney Brothers should be Amber.
The actual contents of the message can then reflect the interaction with those particular sums and that particular counterparty. You can use this for lots of different domains, and your approach will depend on the domain and whether the availability of a message is unusual, for that domain. In the above example where you're trading millions then having risk information for a particular counterparty might be valuable, and if that's the state, it's worth calling out separately. It's probably less important if you're buying baby rabbits, for example.
Given "Rotweiller Pets Limited" is trading baby rabbits $2 cheaper than anyone else
When we ask the system to order 15 baby rabbits
Then it should place an order with "Rotweiller Pets Limited".
It's hard to discuss this without specific examples. However, you can probably see how providing those scenarios would then act as documentation for how to use your APIs, even if the underlying automation for those scenarios talks directly to the API, and has nothing actually specific for pets or Lieumoney trades.
This question is to learn and understand whether a particular technology exists or not. Following is the scenario.
We are going to provide 200 english words. Software can add additional 40 words, which is 20% of 200. Now, using these, the software should write dialogs, meaningful dialogs with no grammar mistake.
For this, I looked into Spintax and Article Spinning. But you know what they do, taking existing articles and rewrite it. But that is not the best way for this (is it? let me know if it is please). So, is there any technology which is capable of doing this? May be semantic theory that Google uses? Any proved AI method?
Please help.
To begin with, a word of caution: this is quite the forefront of research in natural language generation (NLG), and the state-of-the-art research publications are not nearly good enough to replace human teacher. The problem is especially complicated for students with English as a second language (ESL), because they tend to think in their native tongue before mentally translating the knowledge into English. If we disregard this fearful prelude, the normal way to go about this is as follows:
NLG comprises of three main components:
Content Planning
Sentence Planning
Surface Realization
Content Planning: This stage breaks down the high-level goal of communication into structured atomic goals. These atomic goals are small enough to be reached with a single step of communication (e.g. in a single clause).
Sentence Planning: Here, the actual lexemes (i.e. words or word-parts that bear clear semantics) are chosen to be a part of the atomic communicative goal. The lexemes are connected through predicate-argument structures. The sentence planning stage also decides upon sentence boundaries. (e.g. should the student write "I went there, but she was already gone." or "I went there to see her. She has already left." ... notice the different sentence boundaries and different lexemes, but both answers indicating the same meaning.)
Surface Realization: The semi-formed structure attained in the sentence planning step is morphed into a proper form by incorporating function words (determiners, auxiliaries, etc.) and inflections.
In your particular scenario, most of the words are already provided, so choosing the lexemes is going to be relatively simple. The predicate-argument structures connecting the lexemes needs to be learned by using a suitable probabilistic learning model (e.g. hidden Markov models). The surface realization, which ensures the final correct grammatical structure, should be a combination of grammar rules and statistical language models.
At a high-level, note that content planning is language-agnostic (but it is, quite possibly, culture-dependent), while the last two stages are language-dependent.
As a final note, I would like to add that the choice of the 40 extra words is something I have glossed over, but it is no less important than the other parts of this process. In my opinion, these extra words should be chosen based on their syntagmatic relation to the 200 given words.
For further details, the two following papers provide a good start (complete with process flow architectures, examples, etc.):
Natural Language Generation in Dialog Systems
Stochastic Language Generation for Spoken Dialogue Systems
To better understand the notion of syntagmatic relations, I had found Sahlgren's article on distributional hypothesis extremely helpful. The distributional approach in his work can also be used to learn the predicate-argument structures I mentioned earlier.
Finally, to add a few available tools: take a look at this ACL list of NLG systems. I haven't used any of them, but I've heard good things about SPUD and OpenCCG.
I'm writing some children's Math Education software for a class.
I'm going to try and present problems to students of varying skill level with randomly generated math problems of different types in fun ways.
One of the frustrations of using computer based math software is its rigidity. If anyone has taken an online Math class, you'll know all about the frustration of taking an online quiz and having your correct answer thrown out because your problem isn't exactly formatted in their form or some weird spacing issue.
So, originally I thought, "I know! I'll use an expression parser on the answer box so I'll be able to evaluate anything they enter and even if it isn't in the same form I'll be able to check if it is the same answer." So I fire up my IDE and start implementing the Shunting Yard Algorithm.
This would solve the problem of it not taking fractions in the smallest form and other issues.
However, It then hit me that a tricky student would simply be able to enter most of the problems into the answer box and my expression parser would dutifully parse and evaluate it to the correct answer!
So, should I not be using an expression parser in this instance? Do I really have to generate a single form of the answer and do a string comparison?
One possible solution is to note how many steps your expression evaluator takes to evaluate the problem's original expression, and to compare this to the optimal answer. If there's too much difference, then the problem hasn't been reduced enough and you can suggest that the student keep going.
Don't be surprised if students come up with better answers than your own definition of "optimal", though! I was a TA/grader for several classes, and the brightest students routinely had answers on their problem sets that were superior to the ones provided by the professor.
For simple problems where you're looking for an exact answer, then removing whitespace and doing a string compare is reasonable.
For more advanced problems, you might do the Shunting Yard Algorithm (or similar) but perhaps parametrize it so you could turn on/off reductions to guard against the tricky student. You'll notice that "simple" answers can still use the parser, but you would disable all reductions.
For example, on a division question, you'd disable the "/" reduction.
This is a great question.
If you are writing an expression system and an evaluation/transformation/equivalence engine (isn't there one available somewhere? I am almost 100% sure that there is an open source one somewhere), then it's more of an education/algebra problem: is the student's answer algebraically closer to the original expression or to the expected expression.
I'm not sure how to answer that, but just an idea (not necessarily practical): perhaps your evaluation engine can count transformation steps to equivalence. If the answer takes less steps to the expected than it did to the original, it might be ok. If it's too close to the original, it's not.
You could use an expression parser, but apply restrictions on the complexity of the expressions permitted in the answer.
For example, if the goal is to reduce (4/5)*(1/2) and you want to allow either (2/5) or (4/10), then you could restrict the set of allowable answers to expressions whose trees take the form (x/y) and which also evaluate to the correct number. Perhaps you would also allow "0.4", i.e. expressions of the form (x) which evaluate to the correct number.
This is exactly what you would (implicitly) be doing if you graded the problem manually -- you would be looking for an answer that is correct but which also falls into an acceptable class.
The usual way of doing this in mathematics assessment software is to allow the question setter to specify expressions/strings that are not allowed in a correct answer.
If you happen to be interested in existing software, there's the open-source Stack http://www.stack.bham.ac.uk/ (or various commercial options such as MapleTA). I suspect most of the problems that you'll come across have also been encountered by Stack so even if you don't want to use it, it might be educational to look at how it approaches things.