Is there a way to generate a list of the names of variables/functions for the current OpenCL version?
For example, I want to create arrays like these:
constants = ['M_E_F', 'M_LOG2E_F', 'M_LOG10E_F', 'M_LN2_F', 'M_LN10_F', 'M_PI_F', 'M_PI_2_F', 'M_PI_4_F', 'M_1_PI_F', 'M_2_PI_F', 'M_2_SQRTPI_F', 'M_SQRT2_F', 'M_SQRT1_2_F']
common_functions = ['clamp', 'degrees', 'max', 'min', 'mix', 'radians', 'sign', 'smoothstep', 'step']
Would I have to use a script to parse the man pages? Or copy/paste from documentation? Or is there a better way?
Related
firstly apologies: I am a beginner in Earth Engine, but googling my question hasn't yielded any results. I have a reasonable amount of experience with other languages/platforms.
I have a table of data with headers 'name' and 'value' with N entries, I also have a multiband images in which the bands are named the same as the 'name' column in my table.
I want to apply a functions to each band in the image, based on its corresponding value in the table.
I'm struggling to find a way to do this without using loops and getInfo(), both of which I understand are not efficient and generally frowned upon.
I think perhaps I'm missing something fundamental here regarding the interaction between local variables and things occuring serverside - help would be greatly appreciated!
You could perhaps iterate over the band names in the image:
var updatedImage = ee.Image(
image.bandNames().iterate(
function (bandName, acc) {
bandName = ee.String(bandName) // Must cast from ee.Element to actual type
var feature = features
.filter(ee.Filter.eq('name', bandName))
.first()
var value = feature.getNumber('value')
var bandWithFunctionApplied = image.select(bandName)
.add(value) // Apply some function
return ee.Image(acc)
.addBands(bandWithFunctionApplied)
},
ee.Image([])
)
)
https://code.earthengine.google.com/082411908a7525a4c8a87916b5ea88fc
I'm following this tutorial that codes a sentiment analysis classifier using BERT with the huggingface library and I'm having a very odd behavior. When trying the BERT model with a sample text I get a string instead of the hidden state. This is the code I'm using:
import transformers
from transformers import BertModel, BertTokenizer
print(transformers.__version__)
PRE_TRAINED_MODEL_NAME = 'bert-base-cased'
PATH_OF_CACHE = "/home/mwon/data-mwon/paperChega/src_classificador/data/hugingface"
tokenizer = BertTokenizer.from_pretrained(PRE_TRAINED_MODEL_NAME,cache_dir = PATH_OF_CACHE)
sample_txt = 'When was I last outside? I am stuck at home for 2 weeks.'
encoding_sample = tokenizer.encode_plus(
sample_txt,
max_length=32,
add_special_tokens=True, # Add '[CLS]' and '[SEP]'
return_token_type_ids=False,
padding=True,
truncation = True,
return_attention_mask=True,
return_tensors='pt', # Return PyTorch tensors
)
bert_model = BertModel.from_pretrained(PRE_TRAINED_MODEL_NAME,cache_dir = PATH_OF_CACHE)
last_hidden_state, pooled_output = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask']
)
print([last_hidden_state,pooled_output])
that outputs:
4.0.0
['last_hidden_state', 'pooler_output']
While the answer from Aakash provides a solution to the problem, it does not explain the issue. Since one of the 3.X releases of the transformers library, the models do not return tuples anymore but specific output objects:
o = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask']
)
print(type(o))
print(o.keys())
Output:
transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions
odict_keys(['last_hidden_state', 'pooler_output'])
You can return to the previous behavior by adding return_dict=False to get a tuple:
o = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask'],
return_dict=False
)
print(type(o))
Output:
<class 'tuple'>
I do not recommend that, because it is now unambiguous to select a specific part of the output without turning to the documentation as shown in the example below:
o = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'], return_dict=False, output_attentions=True, output_hidden_states=True)
print('I am a tuple with {} elements. You do not know what each element presents without checking the documentation'.format(len(o)))
o = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'], output_attentions=True, output_hidden_states=True)
print('I am a cool object and you can acces my elements with o.last_hidden_state, o["last_hidden_state"] or even o[0]. My keys are; {} '.format(o.keys()))
Output:
I am a tuple with 4 elements. You do not know what each element presents without checking the documentation
I am a cool object and you can acces my elements with o.last_hidden_state, o["last_hidden_state"] or even o[0]. My keys are; odict_keys(['last_hidden_state', 'pooler_output', 'hidden_states', 'attentions'])
I faced the same issue while learning how to implement Bert. I noticed that using
last_hidden_state, pooled_output = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'])
is the issue. Use:
outputs = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'])
and extract the last_hidden state using
output[0]
You can refer to the documentation here which tells you what is returned by the BertModel
I have 2 Dictionaries:
StatePops={'AL':4887871, 'AK':737438, 'AZ':7278717, 'AR':3013825}
StateNames={'AL':'Alabama', 'AK':'Alaska', 'AZ':'Arizona', 'AR':'Arkansas'}
I am trying to merge so the Value of StateNames is the Key for StatePops.
Ex.
{'Alabama': 4887871, 'Alaska': 737438, ...
I also have to display the name of states w/ population over 4million.
Any help is appreciated!!!
You have not specified in what programming language you want this problem to be solved.
Nonetheless, here is a solution in Python.
state_pops = {
'AL': 4887871,
'AK': 737438,
'AZ':7278717,
'AR':3013825
}
state_names = {
'AL':'Alabama',
'AK':'Alaska',
'AZ':'Arizona',
'AR':'Arkansas'
}
states = dict([([state_names[k],state_pops[k]]) for k in state_pops])
final = {k:v for k, v in states.items() if v > 4000000}
print(states)
print(final)
First, you can merge two dictionaries with the predefined dict python function in the states variable as such. Here, k is an iterator and it is used as index for state_names and state_pops.
Then, store the filtered dictionary in final where the states.items() is used to access the keys and values in states and type-cast it as a string with the str function.
There may be more simpler solutions but this is as far as I can optimize the problem.
Hope this helps.
Dictionary Keys cannot be changed in Python. You need to either add the modified key-value pair to the dictionary and then delete the old key, or you can create a new dictionary. I'd opt for the second option, i.e., creating a new dictionary.
myDict = {}
for i in StatePops:
myDict.update({StateNames[i] : StatePops[i]})
This outputs myDict as
{'Alabama': 4887871, 'Alaska': 737438, 'Arizona': 7278717, 'Arkansas': 3013825}
I am a new user of Julia and I want to work on graphs. I found the Graphs.jl library but not very documented. I tried to create a GenericGraph based on ExVertex and ExEdge but I need more information.
The code I'm using :
using Graphs
CompGraph = GenericGraph{ExVertex, ExEdge{ExVertex}}
temp = ExVertex(1, "VertexName")
temp.attributes["Att"] = "Test"
add_vertex!(CompGraph, temp)
Now I still need the ExVertex list and ExEdge list. Is there any defined parameters? or how can I create such lists?
The solution was too simple. a list is juste a simple array and not a new type. Besides, there is a simple defined function which creates graphs based on different types of edges and vertecies.
I changed my code to :
using Graphs
CG_VertexList = ExVertex[]
CG_EdgeList = ExEdge{ExVertex}[]
CompGraph = graph(CG_VertexList, CG_EdgeList)
temp = ExVertex(1, "VertexName")
temp.attributes["Att"] = "Test"
add_vertex!(CompGraph, temp)
I am a beginner with OOP in R and am stuck at a problem for which I can find no solution.
I defined a class "node" in R using setClass that contains information about a "node" in a network -
setClass(Class = "node",
representation = representation(nID = "integer", links = "integer",
capacity = "numeric"),
prototype = prototype(nID = integer(1), links = integer(20),
capacity = numeric(20)))
What I really want to do is create an array/list that holds several "nodes", each of which is of class "node". Something like
nodeID[100] <- new("node")
But that clearly doesnt work. I have tried creating arrays and converting their class to "node" but that didnt do it either.
This will help me do things like loop over all nodes in my system-
for(i in 1:dim(nodeID))
{
nodeID[i]#capacity <- 1000
blah blah....
}
Note that the problem isnt initializing/defaulting the value of slots (e.g. capacity in this case). I can do that. Any help would be greatly appreciate.
Thanks,
Sumit
Answer ----
Thanks #Ricardo and #dickoa. This created the list of nodeID just like I wanted .
Want to add for anyone else facing the same problem that in order to access the elements/slots of the list of class "node" you have to use the following:
nodeID[[1]]#capacity[1]
Also, I will use lapply instead of for.
Sumit
Try using replicate
nodeID <- replicate(100, new("node"), simplify="list")
is(nodeID)
# [1] "list" "vector"
is(nodeID[[1]])
# [1] "node"
Using something like nodeID[100] <- new("node") as you found, does not work. What that (is attempting) to do is look for an object called nodeID and if found, try to set its 100th element to new("node").
It does not, say, create an object nodeID and populate it with 100 elements.
Also, notice that you can avoid your for loop by instead using, say, lapply:
eg, instead of:
for(i in 1:dim(nodeID))
{
nodeID[i]#capacity <- 1000
blah blah....
}
use:
lapply(nodeID, function(n) {blah balh...} )