I am currently trying to get the type of such declarations:
const x = new BehaviourSubject<string>(); // Should be of type BehaviourSubject<string>
Using ts-morph i tried:
const declX = sourceFile.getVariableDeclarationOrThrow("x");
declX.getType().getText();
declX.getApparentType().getText();
but in both cases i just get any as type. I suspect that i need to evaluate the NewExpression in some form using the ts-typechecker but can not find any references on how to do this.
Any input would be appreciated.
Related
I'm following this tutorial that codes a sentiment analysis classifier using BERT with the huggingface library and I'm having a very odd behavior. When trying the BERT model with a sample text I get a string instead of the hidden state. This is the code I'm using:
import transformers
from transformers import BertModel, BertTokenizer
print(transformers.__version__)
PRE_TRAINED_MODEL_NAME = 'bert-base-cased'
PATH_OF_CACHE = "/home/mwon/data-mwon/paperChega/src_classificador/data/hugingface"
tokenizer = BertTokenizer.from_pretrained(PRE_TRAINED_MODEL_NAME,cache_dir = PATH_OF_CACHE)
sample_txt = 'When was I last outside? I am stuck at home for 2 weeks.'
encoding_sample = tokenizer.encode_plus(
sample_txt,
max_length=32,
add_special_tokens=True, # Add '[CLS]' and '[SEP]'
return_token_type_ids=False,
padding=True,
truncation = True,
return_attention_mask=True,
return_tensors='pt', # Return PyTorch tensors
)
bert_model = BertModel.from_pretrained(PRE_TRAINED_MODEL_NAME,cache_dir = PATH_OF_CACHE)
last_hidden_state, pooled_output = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask']
)
print([last_hidden_state,pooled_output])
that outputs:
4.0.0
['last_hidden_state', 'pooler_output']
While the answer from Aakash provides a solution to the problem, it does not explain the issue. Since one of the 3.X releases of the transformers library, the models do not return tuples anymore but specific output objects:
o = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask']
)
print(type(o))
print(o.keys())
Output:
transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions
odict_keys(['last_hidden_state', 'pooler_output'])
You can return to the previous behavior by adding return_dict=False to get a tuple:
o = bert_model(
encoding_sample['input_ids'],
encoding_sample['attention_mask'],
return_dict=False
)
print(type(o))
Output:
<class 'tuple'>
I do not recommend that, because it is now unambiguous to select a specific part of the output without turning to the documentation as shown in the example below:
o = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'], return_dict=False, output_attentions=True, output_hidden_states=True)
print('I am a tuple with {} elements. You do not know what each element presents without checking the documentation'.format(len(o)))
o = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'], output_attentions=True, output_hidden_states=True)
print('I am a cool object and you can acces my elements with o.last_hidden_state, o["last_hidden_state"] or even o[0]. My keys are; {} '.format(o.keys()))
Output:
I am a tuple with 4 elements. You do not know what each element presents without checking the documentation
I am a cool object and you can acces my elements with o.last_hidden_state, o["last_hidden_state"] or even o[0]. My keys are; odict_keys(['last_hidden_state', 'pooler_output', 'hidden_states', 'attentions'])
I faced the same issue while learning how to implement Bert. I noticed that using
last_hidden_state, pooled_output = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'])
is the issue. Use:
outputs = bert_model(encoding_sample['input_ids'], encoding_sample['attention_mask'])
and extract the last_hidden state using
output[0]
You can refer to the documentation here which tells you what is returned by the BertModel
What is the best way to get the symbol for an export by name?
Bellow is functional code. However, it does seems a bit fragile as I can't get the 'symbol' from 'sourceFile' without ignoring the type system.
const sourceFile = tsprogram.getSourceFile('foo_file.ts');
const fileSymbol = (sourceFile as any).symbol as ts.Symbol; // anything better her?
const export = fileSymbol.exports.get('FooComponent');
Use the type checker:
const fileSymbol = tsprogram.getTypeChecker().getSymbolAtLocation(sourceFile);
const fooComponentSymbol = fileSymbol?.exports.get('FooComponent');
Note that fileSymbol will be undefined when there are no file exports.
I would like to speed up figure generation in Bokeh by multiprocessing:
jobs = []
for label in list(peakLabels):
args = {'data': rt_proj_data[label],
'label': label,
'tools': tools,
'colors': itertools.cycle(palette),
'files': files,
'highlight': highlight}
jobs.append(args)
pool = Pool(processes=cpu_count())
m = Manager()
q = m.Queue()
plots = pool.map_async(plot_peaks_parallel, jobs)
pool.close()
pool.join()
def plot_peaks_parallel(args):
data = args['data']
label = args['label']
colors = args['colors']
tools = args['tools']
files = args['files']
highlight = args['highlight']
p = figure(title=f'Peak: {label}',
x_axis_label='Retention Time',
y_axis_label='Intensity',
tools=tools)
...
return p
Though I ran into this error:
MaybeEncodingError: Error sending result: '[Figure(id='1078', ...)]'. Reason: 'PicklingError("Can't pickle at 0x7fc7df0c0ea0>: attribute lookup ColumnDataSource. on bokeh.models.sources failed")'
Can I do something to the object p, so that it becomes pickleable?
Individual Bokeh objects are not serializable in isolation, including with pickle. The smallest meaningful unit of serialization in Bokeh is the Document, which is a specific collection of Bokeh objects guaranteed to be complete with respect to following references. However, I would be surprised if pickle works with Document either (AFAIK you are the first person to ask about it since the project started, it's never been a priority, or even looked into that I know of). Instead, I would suggest if you want to do something like this, to use Bokeh's own JSON serialization functions, such as json_item:
# python code
p_serialized = json.dumps(json_item(p))
This will properly serialize p in the context of the Document it is a part of. Then you can pass this to your page templates to display with the Bokeh JS embed API:
# javascript code
p = JSON.parse(p_serialized);
Bokeh.embed.embed_item(p, "mydiv")
I'm having trouble retrieving a filtered list from google app engine datastore (using python for server side). My data entity is defined as the following
class Course_Table(db.Model):
course_name = db.StringProperty(required=True, indexed=True)
....
head_tags_1=db.ListProperty(db.Key)
So the head_tags_1 property is a list of keys (which are the keys to a different entity called Headings_1).
I'm in the Handler below to spin through my Course_Table entity to filter the courses that have a particular Headings_1 key as a member of the head_tags_1 property. However, it doesn't seem like it is retrieving anything when I know there is data there to fulfill the request since it never displays the logs below when I go back to iterate through the results of my query (below). Any ideas of what I'm doing wrong?
def get(self,level_num,h_key):
path = []
if level_num == "1":
q = Course_Table.all().filter("head_tags_1 =", h_key)
for each in q:
logging.info('going through courses with this heading name')
logging.info("course name filtered is %s ", each.course_name)
MANY MANY THANK YOUS
I assume h_key is key of headings_1, since head_tags_1 is a list, I believe what you need is IN operator. https://developers.google.com/appengine/docs/python/datastore/queries
Note: your indentation inside the for loop does not seem correct.
My bad apparently '=' for list is already check membership. Using = to check membership is working for me, can you make sure h_key is really a datastore key class?
Here is my example, the first get produces result, where the 2nd one is not
import webapp2 from google.appengine.ext import db
class Greeting(db.Model):
author = db.StringProperty()
x = db.ListProperty(db.Key)
class C(db.Model): name = db.StringProperty()
class MainPage(webapp2.RequestHandler):
def get(self):
ckey = db.Key.from_path('C', 'abc')
dkey = db.Key.from_path('C', 'def')
ekey = db.Key.from_path('C', 'ghi')
Greeting(author='xxx', x=[ckey, dkey]).put()
x = Greeting.all().filter('x =',ckey).get()
self.response.write(x and x.author or 'None')
x = Greeting.all().filter('x =',ekey).get()
self.response.write(x and x.author or 'None')
app = webapp2.WSGIApplication([('/', MainPage)],
debug=True)
I'm trying to write my first type provider and am wondering if someone could point where I am going wrong.
I've used the freebase sample to work from. I've tried to sift thru the essential bits to get something very basic instantiated but obviously missed something or not got it quite right (perhaps in relation to the namespace). I'm simply trying to get this line working
type tp = MyFirstTypeProvider.DataProvider<username="username",password="password",product=prodId>
Intellisense is seeing the DataProvider, but I'm getting squiggly saying that a reference to the type can be found the type could not be found in the assembly.
namespace MyFirstProvider
type internal MyFirstRuntimeInfo(config: TypeProviderConfig) =
let runtimeAssembly = Assembly.LoadFrom(config.RuntimeAssembly)
member val DataContextType = runtimeAssembly.GetType("MyFirstProvider.Runtime.DataContext")
member this.RuntimeAssembly = runtimeAssembly
// This type defines the type provider. When compiled to a DLL, it can be added
// as a reference to an F# command-line compilation, script, or project.
[<TypeProvider>]
type public Types(config: TypeProviderConfig) as this =
inherit TypeProviderForNamespaces()
let bfRuntimeInfo = MyFirstRuntimeInfo(config)
let rootNamespace = "MyFirstProvider"
let defaultUsername = "xxxxxxxxxxx"
let defaultPassword = "yyyyyyyyyy"
let defaultProductId = -1
let defaultToken = "none"
let createDataContext = bfRuntimeInfo.DataContextType.GetMethod("_Create")
let createTypes(username, password, productId, rootTypeName) = let bf = new MyFirstProvider.Requests.Queries(defaultToken)
let schema = new MyFirstProvider.Schema.SchemaConnection(bf)
let rootType = ProvidedTypeDefinition(bfRuntimeInfo.RuntimeAssembly,rootNamespace,rootTypeName,baseType=Some typeof<obj>, HideObjectMethods=true)
let theServiceType = ProvidedTypeDefinition("Service",baseType=Some bfRuntimeInfo.DataContextType, HideObjectMethods=true)
let theServiceTypesClass = ProvidedTypeDefinition("ServiceTypes",baseType=Some typeof<obj>,HideObjectMethods=true)
theServiceTypesClass.AddMembers [ theServiceType ]
rootType.AddMembers [ theServiceTypesClass ]
rootType.AddMembersDelayed (fun () ->
[ yield ProvidedMethod ("GetDataContext", [], theServiceType, IsStaticMethod=true,
InvokeCode = (fun _args -> Expr.Call(createDataContext, [ Expr.Value defaultUsername; Expr.Value defaultPassword; Expr.Value defaultProductId ])))
])
rootType
let MyFirstType = createTypes(defaultUsername, defaultPassword, defaultProductId, "Data")
let paramMyFirstType = ProvidedTypeDefinition(bfRuntimeInfo.RuntimeAssembly, rootNamespace, "DataProvider", Some(typeof<obj>), HideObjectMethods = true)
let usernameParam = ProvidedStaticParameter("username", typeof<string>, defaultUsername)
let passwordParam = ProvidedStaticParameter("password", typeof<string>, defaultPassword)
let productIdParam = ProvidedStaticParameter("productId", typeof<int>, defaultProductId)
do paramMyFirstType.DefineStaticParameters([usernameParam;passwordParam;productIdParam], fun typeName providerArgs ->
let username = (providerArgs.[0] :?> string)
let password = (providerArgs.[1] :?> string)
let productId = (providerArgs.[2] :?> int)
createTypes(username, password, productId, typeName))
do
this.AddNamespace(rootNamespace, [MyFirstType ] )
this.AddNamespace(rootNamespace, [paramMyFirstType ] )
[<assembly:TypeProviderAssembly>]
do()
Many thx in advance.
When I try compiling your type provider and reference it in a script file, the reference #r "provider.dll" is underlined with a red squiggly that says:
The type provider 'MyFirstProvider.Types' reproted an error: The type provider constructor has thrown an exception: Object reference not set to an instance of an object.
You can debug such errors by starting a second instance of Visual Studio, and attaching the debugger to another instance (Debug -> Attach to Process) and opening the script file in the instance being debugged.
In the sample you posted here, the null reference exception occurs on this line:
let createDataContext = bfRuntimeInfo.DataContextType.GetMethod("_Create")
This is trying to access method that does not exist. I think the Freebase sample is relatively complicated starting point (here, it is using an object to represent the "data context" - you might want to follow that pattern, but I think it is easier to build it from scratch rather than adapting an existing code). I think a good starting point might be the Hello World type provider which is a lot simpler and just shows how to generate new types, properties and methods.
I would suggest, if you are looking for a simple example of type provider, to look at the File System type provider This is a good place to start as it has no state and no dependency management to take care of.
As to why this one has a dll it can not find, I think it is because this dll has to be seen from where your type provider's dll is. That is, for instance, in the same directory. (You can try to set "copy local" in the build process?)