As i follow the link usingglossary and batchtranslate, and batchTranslateText api say glossaries should define like this.
glossaries:map (key: string, value: object (TranslateTextGlossaryConfig))
i want to use google batch translate with glossary,but always get errorTypeError: 'TranslateTextGlossaryConfig' object is not iterable, my glossary is Equivalent term.Can any one give me a clue? i use python as demo code here:
glossary_config = translate.TranslateTextGlossaryConfig(glossary=glossary,ignore_case=True)
glossaries=map("th",(glossary_config))
operation = client.batch_translate_text(
request={
"parent": parent,
"source_language_code": sourcefrom,
"target_language_codes": [targetto], # Up to 10 language codes here.
"input_configs": input_configs,
"output_config": output_config,
"glossaries": glossaries
}
)
Related
Unable to make speech_contexts phrase lists work with speech.SpeechAsyncClient in Google Speech to Text..
The transcription works, but the phrase list appears to be ignored.
Is there any config that needs to be in-place?
When Using the speech.SpeechAsyncClient (version 2.17.2 in python) I created a phrase list :
speech_contexts {
phrases: "Burrito"
boost: 10.0
}
speech_contexts {
phrases: "burrito"
boost: 5.0
}
I expected the word audio for 'burrito' to be transcribed as 'Burrito' as text. However it continued to be 'burrito'. Also I tried various phrase lists, but the recognition seems to ignore the phrase lists (same result with/without phrase list).
I verified that the proper speech_context is being sent in the 'streaming_config/Recogntionconfig like this:
Recognitionconfig = speech.RecognitionConfig(
encoding=speech.RecognitionConfig.AudioEncoding.LINEAR16,
#encoding = cloud_speech.ExplicitDecodingConfig.AudioEncoding.LINEAR16,
sample_rate_hertz=16000,
language_code="en-US",
model="latest_long",
#enable_word_confidence=True,
speech_contexts=speech_contexts #this contains the phrase list
)
#The first message is the following streaming_config and is then followed by audio
streaming_config = speech.StreamingRecognitionConfig(
config=Recognitionconfig, interim_results=True
)
Try using model adaptation to strengthen the accuracy of your transcription results. It also uses RecognitionConfig for the request body. Also follow this format when using SpeechContext.
{
"phrases": [
string
],
"boost": number
}
attribute itemJson stored as follow
"itemJson": {
"S": "{\"sold\":\"3\",\"listingTime\":\"20210107211621\",\"listCountry\":\"US\",\"sellerCountry\":\"US\",\"currentPrice\":\"44.86\",\"updateTime\":\"20210302092220\",\"itemLocation\":\"Miami,FL,USA\",\"listType\":\"FixedPrice\",\"categoryName\":\"Machines\",\"itemID\":\"293945109477\",\"sellerID\":\"holiday_for_you\",\"s3Key\":\"US/2021/2/FixedPrice/293945109477.json\",\"visitCount\":\"171\",\"createTime\":\"20210201233158\",\"listingStatus\":\"Completed\",\"endTime\":\"2021-02-28T20:22:57\",\"currencyID\":\"USD\"}"
},
i want to query with filter:contains(itemJson, "sold":"0") with java sdk,i tried those syntax,all fail
expressionValues.put(":v2", AttributeValue.builder().s("\\\"sold\\\":\\\"0\\\"").build());
expressionValues.put(":v2", AttributeValue.builder().s("sold:0"").build());
what is the right way to my filter syntax?
I try #Balu Vyamajala's syntax on the dynamodb web console as follow,did not get the solution yet
contains (itemJson, :subValue) with value of "sold\":\"3\"" seems to be working.
Working example on a Query Api and worked as expected:
QuerySpec querySpec = new QuerySpec()
.withKeyConditionExpression("pk = :v_pk")
.withFilterExpression("contains (itemJson, :subValue)")
.withValueMap(new ValueMap().withString(":v_pk", "6").withString(":subValue", "sold\":\"3\""));
and to test from Aws console we just need to enter "sold":"2"
I'm following this tutorial that codes a sentiment analysis classifier using BERT with the huggingface library and I'm having a very odd behavior. When trying the BERT model with a sample text I get a string instead of the hidden state. This is the code I'm using:
import transformers
from transformers import BertModel, BertTokenizer
print(transformers.__version__)
PRE_TRAINED_MODEL_NAME = 'bert-base-cased'
PATH_OF_CACHE = "/home/mwon/data-mwon/paperChega/src_classificador/data/hugingface"
tokenizer = BertTokenizer.from_pretrained(PRE_TRAINED_MODEL_NAME,cache_dir = PATH_OF_CACHE)
sample_txt = 'When was I last outside? I am stuck at home for 2 weeks.'
encoding_sample = tokenizer.encode_plus(
sample_txt,
max_length=32,
add_special_tokens=True, # Add '[CLS]' and '[SEP]'
return_token_type_ids=False,
padding=True,
truncation = True,
return_attention_mask=True,
return_tensors='pt', # Return PyTorch tensors
)
bert_model = BertModel.from_pretrained(PRE_TRAINED_MODEL_NAME,cache_dir = PATH_OF_CACHE)
last_hidden_state, pooled_output = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask']
)
print([last_hidden_state,pooled_output])
that outputs:
4.0.0
['last_hidden_state', 'pooler_output']
While the answer from Aakash provides a solution to the problem, it does not explain the issue. Since one of the 3.X releases of the transformers library, the models do not return tuples anymore but specific output objects:
o = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask']
)
print(type(o))
print(o.keys())
Output:
transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions
odict_keys(['last_hidden_state', 'pooler_output'])
You can return to the previous behavior by adding return_dict=False to get a tuple:
o = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask'],
return_dict=False
)
print(type(o))
Output:
<class 'tuple'>
I do not recommend that, because it is now unambiguous to select a specific part of the output without turning to the documentation as shown in the example below:
o = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'], return_dict=False, output_attentions=True, output_hidden_states=True)
print('I am a tuple with {} elements. You do not know what each element presents without checking the documentation'.format(len(o)))
o = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'], output_attentions=True, output_hidden_states=True)
print('I am a cool object and you can acces my elements with o.last_hidden_state, o["last_hidden_state"] or even o[0]. My keys are; {} '.format(o.keys()))
Output:
I am a tuple with 4 elements. You do not know what each element presents without checking the documentation
I am a cool object and you can acces my elements with o.last_hidden_state, o["last_hidden_state"] or even o[0]. My keys are; odict_keys(['last_hidden_state', 'pooler_output', 'hidden_states', 'attentions'])
I faced the same issue while learning how to implement Bert. I noticed that using
last_hidden_state, pooled_output = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'])
is the issue. Use:
outputs = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'])
and extract the last_hidden state using
output[0]
You can refer to the documentation here which tells you what is returned by the BertModel
I have an Elastic Search cluster with a lot of nice data, that I have created some nice Kibana dashboards for.
For the next level I decided to take a look at scripted fields to make some of the dashboards even nicer.
I want to translate some of the numeric fields into more easily understandable text values. As an example of what I want to do and what I have tried I will use the http response status code field, that most will understand quite easily but also illustrates the problem.
We log the numeric status code (200, 201, 302, 400, 404, 500 etc.) I can create a data table visualization that tells me the count for each of these status codes. But I would like to display the text reason in my dashboard.
I can create a painless script with a lot of IF statements like this:
if (doc['statuscode'].value == 200) {return "OK";}
if (doc['statuscode'].value == 201) {return "Created";}
if (doc['statuscode'].value == 400) {return "Bad Request";}
return doc['statuscode'].value;
But that isn't very nice I think.
But since I will most likely have about 150 different values and that list won't change very often, so I can live with maintaining a static map. But I haven't found any examples of implementing a map or dictionary in painless scripting.
I was thinking of implementing something like this:
Map reasonMap;
reasonMap[200] = 'OK';
reasonMap[201] = 'Created';
def reason = reasonMap[doc['statuscode'].value];
if (reason != null)
{
return reason;
}
return doc['statuscode'].value;
I haven't been able to make this code work though. The question is also if this will perform well enough for a map with up to 150 values.
Thanks
EDIT
After some trial and error... and a lot of googling, this is what I came up with that works (notice that the key needs to start with a character and not a number):
def reasonMap =
[
's200': 'OK',
's201': 'Created'
];
def key = 's' + doc['statuscode'].value
def reason = reasonMap[key];
if (reason != null)
{
return reason;
}
return doc['statuscode'].value;
Should it be
def reason = reasonMap[doc['statuscode']value];
It will perform well with a Map of 150 values.
Upon compiling, I am getting the following error:
C:\UDK\UDK-2010-03\Development\Src\FixIt\Classes\ZInteraction.uc(41) : Error, Unrecognized member 'OpenMenu' in class 'GameUISceneClient'
Line 41 is the following:
GetSceneClient().OpenMenu("ZInterface.ZNLGWindow");
But when I search for OpenMenu, I find that it is indeed defined in GameUISceneClient.uc of the UDK:
Line 1507: exec function OpenMenu( string MenuPath, optional int PlayerIndex=INDEX_NONE )
It looks like I have everything correct. So what's wrong? Why can't it find the OpenMenu function?
From the wiki page on Legacy:Exec Function:
Exec Functions are functions that a player or user can execute by typing its name in the console. Basically, they provide a way to define new console commands in UnrealScript code.
Okay, so OpenMenu has been converted to a console command. Great. But still, how do I execute it in code? The page doesn't say!
More searching revealed this odd documentation page, which contains the answer:
Now then, there is also a function
within class Console called 'bool
ConsoleCommand(coerce string s)'. to
call your exec'd function,
'myFunction' from code, you type:
* bool isFunctionThere; //optional
isFunctionThere = ConsoleCommand("myFunction myArgument");
So, I replaced my line with the following:
GetSceneClient().ConsoleCommand("OpenMenu ZInterface.ZNLGWindow");
Now this causes another error which I covered in my other question+answer a few minutes ago. But that's it!
Not sure if this is your intent, but if you are trying to create a UIScene based on an Archetype that has been created in the UI Editor, you want to do something like this:
UIScene openedScene;
UIScene mySceneArchetype;
mySceneArchetype = UIScene'Package.Scene';
GameSceneClient = class'UIRoot'.static.GetSceneClient();
//Open the Scene
if( GameSceneClient != none && MySceneArchetype != none )
{
GameSceneClient.OpenScene(mySceneArchetype,LocalPlayer(PlayerOwner.Player), openedScene);
}