I have trained and saved some NER models using
torch.save(model)
I need to load these model files (extension .pt) for evaluation using
torch.load('PATH_TO_MODEL.pt')
And I get the following error: 'BertConfig' object has no attribute 'return_dict'
For the same, I updated my transformer package to the latest one, but the error persists.
This is the stack trace:
Traceback (most recent call last):
File "/home/systematicReviews/train_mtl_3.py", line 523, in <module>
test_loss, test_cr, test_cr_fine = evaluate_i(test_model, optimizer, scheduler, validation_dataloader, args, device)
File "/home/systematicReviews/train_mtl_3.py", line 180, in evaluate_i
e_loss_coarse, e_output, e_labels, e_loss_fine, e_f_output, e_f_labels, mask, e_cumulative_loss = defModel(args, e_input_ids, attention_mask=e_input_mask, P_labels=e_labels, P_f_labels=e_f_labels)
File "/home/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/systematicReviews/models/mtl/model.py", line 122, in forward
attention_mask = attention_mask
File "/home/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/transformers/modeling_bert.py", line 784, in forward
return_dict = return_dict if return_dict is not None else self.config.use_return_dict
File "/home/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/transformers/configuration_utils.py", line 219, in use_return_dict
return self.return_dict and not self.torchscript
AttributeError: 'BertConfig' object has no attribute 'return_dict'
Here is some more information about my system:
- `transformers` version: 3.1.0
- Platform: Linux-4.4.0-186-generic-x86_64-with-debian-stretch-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.3.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
It worked pretty fine until now, but suddenly this bug appears. Any help or hint is appreciated.
Try to save your model with model.save_pretrained(output_dir). Then you can load your model with model = *.from_pretrained(output_dir) where * is the model class (e.g. BertForTokenClassification).
To save model dictionary rather than an entire model is slightly different. Instead of torch.save(model) use torch.save('path_to_the_model/model.pth') and load using torch.load('path_to_the_model/model.pth').
Related
I am working on the code below, and I got SSHException('No existing session') error
from ncclient import manager
with manager.connect(host="192.168.1.40", port=830, username="cisco", password="cisco", hostkey_verify=False, device_params={'name':'iosxr'}, ssh_config=True, allow_agent=False,look_for_keys=False, timeout=10) as nc_conn:
nc_config = nc_conn.get_config(source='running').data_xml
print (nc_config)
yapi#MacBook-Pro-3 iTel % python test01.py
Traceback (most recent call last):
File "/Users/yapi/Documents/Scripts/Non-GIT/Python/2023/iTel/test01.py", line 4, in
with manager.connect(host="192.168.1.40", port=830, username="abdul", password="cisco", hostkey_verify=False, device_params={'name':'iosxr'}, ssh_config=True, allow_agent=False,look_for_keys=False, timeout=10) as nc_conn:
File "/usr/local/lib/python3.9/site-packages/ncclient/manager.py", line 176, in connect
return connect_ssh(*args, **kwds)
File "/usr/local/lib/python3.9/site-packages/ncclient/manager.py", line 143, in connect_ssh
session.connect(*args, **kwds)
File "/usr/local/lib/python3.9/site-packages/ncclient/transport/ssh.py", line 364, in connect
self._auth(username, password, key_filenames, allow_agent, look_for_keys)
File "/usr/local/lib/python3.9/site-packages/ncclient/transport/ssh.py", line 480, in _auth
raise AuthenticationError(repr(saved_exception))
ncclient.transport.errors.AuthenticationError: SSHException('No existing session')
yapi#MacBook-Pro-3 iTel %
I have tried using different flags in connect function, but no luck
I have a GCP Cloud Composer environment with airflow version composer-1.10.0-airflow-1.10.6 and python 3, 3.6 to be precise. I am calling an apache-beam pipeline on Dataflow using a python_operator.PythonOperator, operator. Here is the code snippet
Calling the pipeline function
test_greeting = python_operator.PythonOperator(
task_id='python_pipeline',
python_callable=run_pipeline
)
The pipeline function is as follows
def run_pipeline():
print("Test Pipeline")
pipeline_args=[
"--runner","DataflowRunner",
"--project","*****",
"--temp_location","gs://******/temp",
"--region","us-east1",
"--job_name","job1199",
"--zone","us-east1-b"
]
pipeline_options=PipelineOptions(pipeline_args)
pipe=beam.Pipeline(options=pipeline_options)
small_sum = (
pipe
| beam.Create([18,5,7,7,9,23,13,5])
| "Combine Globally" >> beam.CombineGlobally(AverageFn())
| 'Write results' >> beam.io.WriteToText('gs://******/ouptut_from_pipline/combine')
)
run_result=pipe.run()
run_result.wait_until_finish()
return "True"
When I run this the pipeline execution runs in dataflow but fails with the following error
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/dataflow_worker/batchworker.py", line 648, in do_work
work_executor.execute()
File "/usr/local/lib/python3.6/site-packages/dataflow_worker/executor.py", line 150, in execute
test_shuffle_sink=self._test_shuffle_sink)
File "/usr/local/lib/python3.6/site-packages/dataflow_worker/executor.py", line 116, in create_operation
is_streaming=False)
File "apache_beam/runners/worker/operations.py", line 1032, in apache_beam.runners.worker.operations.create_operation
File "apache_beam/runners/worker/operations.py", line 845, in apache_beam.runners.worker.operations.create_pgbk_op
File "apache_beam/runners/worker/operations.py", line 903, in apache_beam.runners.worker.operations.PGBKCVOperation.__init__
File "/usr/local/lib/python3.6/site-packages/apache_beam/internal/pickler.py", line 290, in loads
return dill.loads(s)
File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 275, in loads
return load(file, ignore, **kwds)
File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 270, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 472, in load
obj = StockUnpickler.load(self)
File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 462, in find_class
return StockUnpickler.find_class(self, module, name)
ModuleNotFoundError: No module named 'unusual_prefix_162ac8b7030d5bd1ff5f128a26483932d3968a4d_python_bash'
The beam version is Apache Beam Python 3.6 SDK 2.19.0.
I suspect the version of Python 3.6 may be the issue as calling the pipeline directly (as a runner) from my local system works fine and my local system is running python 3.7.
I cant find a way to test this theory though.
It would be helpful to get tips of how to resolve this issue.
I've already seen this answer:
Gremlin, How to add edge to existing vertex in gremlin-python
and it wasn't really helpful. As suggested in one of the comments I did try to update gremlinpython 3.3.0 but then I get key error.
Stack:
JanusGraph 0.2.0, gremlinpython3.2.3
This is my code
from gremlin_python import statics
from gremlin_python.structure.graph import Graph
from gremlin_python.process.graph_traversal import __
from gremlin_python.process.strategies import *
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
graph = Graph()
g = graph.traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
martha = g.V().has('name','martha').next()
jack = g.V().has('name','jack').next()
#e_id = g.addE(jack,'likes',martha).next()
e_id = g.V(martha).as_('to').V(jack).addE("Likes").to('to').toList()
print e_id.toList()
StackTrace with gremlinpython 3.3.0
Traceback (most recent call last):
File "gremlin-py.py", line 9, in <module>
martha = g.V().has('name','martha').next()
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 70,in next
return self.__next__()
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 43,in __next__
self.traversal_strategies.apply_strategies(self)
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 352, in apply_strategies
traversal_strategy.apply(traversal)
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/driver/remote_connection.py", line 143, in apply
remote_traversal = self.remote_connection.submit(traversal.bytecode)
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/driver/driver_remote_connection.py", line 54, in submit
results = result_set.all().result()
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/concurrent/futures/_base.py", line 429, in result
return self.__get_result()
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/concurrent/futures/_base.py", line 381, in __get_result
raise exception_type, self._exception, self._traceback
KeyError: None
In my case, 3.3.0 is throwing error for all queries including g.V().next(). Now going back to 3.2.3, addvertex and other queries are working absolutely fine, but I couldn't figure out how to add edges. The same code when run with 3.2.3 produces,
StackTrace with gremlinpython 3.2.3
Traceback (most recent call last): File "gremlin-py.py", line 12, in <module>
e_id = g.V(martha).as_('to').V(jack).addE("Likes").to('to').toList()
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 52, in toList return list(iter(self))
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 70, in next
return self.__next__() File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 43, in __next__
self.traversal_strategies.apply_strategies(self) File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/process/traversal.py", line 284, in apply_strategies
traversal_strategy.apply(traversal)
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/driver/remote_connection.py", line 95, in apply remote_traversal = self.remote_connection.submit(traversal.bytecode) File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/driver/driver_remote_connection.py", line 53, in submit traversers = self._loop.run_sync(lambda: self.submit_traversal_bytecode(request_id, bytecode))
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/tornado/ioloop.py", line 457, in run_sync
return future_cell[0].result() File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/tornado/concurrent.py", line 237, in result
raise_exc_info(self._exc_info)
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/tornado/gen.py", line 285, in wrapper
yielded = next(result)
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/driver/driver_remote_connection.py", line 69, in submit_traversal_bytecode
"gremlin": self._graphson_writer.writeObject(bytecode),
File "/Users/arvindn/.virtualenvs/gremlinenv/lib/python2.7/site-packages/gremlin_python/structure/io/graphson.py", line 72, in writeObject
return json.dumps(self.toDict(objectData), separators=(',', ':'))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 250, in dumps
sort_keys=sort_keys, **kw).encode(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: v[4184] is not JSON serializable
It says v[x] is not JSON serializable. I'm not sure what causes this error. It'll be awesome if someone can help. If any more info is needed, I shall update the question accordingly.
JanusGraph 0.2.0 uses Apache TinkerPop 3.2.6. You should use the 3.2.6 version of the gremlinpython driver.
pip uninstall gremlinpython
pip install gremlinpython==3.2.6
I'm running R (3.2.2) on a Debian Jessie VPS thanks to rpy2 (2.6.1) through Python (2.7.10).
I have a Flask (0.10.1) application launching R via rpy2 and then loading R package devEMF (2.0.0) to plot various graphics. The aim is to save vector images created thanks to devEMF.
My sample code is:
robjects.r(u'''
suppressPackageStartupMessages(library(qgraph))
suppressPackageStartupMessages(library(devEMF))
emf(file = "''' + pathToSaveGraphEmf + '''", bg = "white", width = 6.25, height = 5.21)
attach(mtcars)
plot(wt,mpg)
dev.off()
''')
And I get the following error message:
/usr/local/lib/python2.7/dist-packages/rpy2/robjects/functions.py:106: UserWarning: Error in axis(side = side, at = at, labels = labels, ...) :
Can't open connection to X server to read font metric information.
res = super(Function, self).__call__(*new_args, **new_kwargs)
/usr/local/lib/python2.7/dist-packages/rpy2/robjects/functions.py:106: UserWarning: In addition:
res = super(Function, self).__call__(*new_args, **new_kwargs)
/usr/local/lib/python2.7/dist-packages/rpy2/robjects/functions.py:106: UserWarning: Warning message:
res = super(Function, self).__call__(*new_args, **new_kwargs)
/usr/local/lib/python2.7/dist-packages/rpy2/robjects/functions.py:106: UserWarning: no DISPLAY variable so Tk is not available
res = super(Function, self).__call__(*new_args, **new_kwargs)
[2016-01-21 20:03:18 +0000] [21430] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 130, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/sync.py", line 171, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask_cors/extension.py", line 110, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask_cors/extension.py", line 110, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/dist-packages/flask_classy.py", line 200, in proxy
response = view(**request.view_args)
File "/routines/routine2.py", line 124, in keyDriverAnalysis
myCorrelationMatrix, rowName, colName = r2.getCorrelationGraph(myCorrelationMethod, myPreparedData, myGroupUserInput, myDependentVariable, myIndependentVariable, myPathToSaveGraph)
File "../graph.py", line 194, in getCorrelationGraph
''')
File "/usr/local/lib/python2.7/dist-packages/rpy2/robjects/__init__.py", line 317, in __call__
res = self.eval(p)
File "/usr/local/lib/python2.7/dist-packages/rpy2/robjects/functions.py", line 178, in __call__
return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/rpy2/robjects/functions.py", line 106, in __call__
res = super(Function, self).__call__(*new_args, **new_kwargs)
RRuntimeError: Error in axis(side = side, at = at, labels = labels, ...) :
Can't open connection to X server to read font metric information.
devEMF needs X11 and Xft (line 35), when not running on OSX or Windows, so I installed it. But now it is unable to connect to the X server. Certainly because the VPS has no GUI feature.
Hopefully I found this very helping SO question, and I understood that I should use virtual framebuffer X11 server.
I tried the advice from the SO question in a command line R session, and it worked ! (Now it outputs an empty file, but it was fully working two minutes ago. (I had downloaded the file, inserted is in an .xls file and my graph was openable and ungroupable.))
But now I have to make it work within rpy2.
I tried the second answer in the previously cited SO question, but it doesn't work because it gives a trick when X11 is problematic when loading but not needed then.
I think where I am now is: how do I launch an R session thanks to rpy2 that will use virtual framebuffer X11 server ?
EDIT0:
I think I found something useful here. It explains how to configure a Virtual Frame Buffer on headless unix servers.
EDIT1:
Eventually, what I have done:
Launch xvfb thanks to the command from the main environment, and bind it to the desired screen (1) (I don't reminder where I found this precise line, but well it can be analyzed thanks to the xvfb documentation):
Xvfb :1 -screen 0 1024x768x16 &
Attach the tmux session, and export the DISPLAY environment variable linked to the correct screen (1):
export DISPLAY=:1
And it works !
Important note: the environment variable should be set inside the session.
Also: I don't have the "launch at boot time" problem mentioned in the link from EDIT0, probably because I am on a VPS (the xvfb-run folder is still in tmp and it seems to run because I can plot and save in .emf ;-) )
if in my diazo controlpanel > 'Parameter expressions' I put
have_left_portlets = python:context and context.restrictedTraverse('##plone').have_portlets('plone.leftcolumn',context)
I obtain an error only when I'm on the portal homepage:
2012-06-26 16:51:42 ERROR plone.transformchain Unexpected error whilst trying to apply transform chain
Traceback (most recent call last):
File "/Users/vito/.buildout/eggs/plone.transformchain-1.0.2-py2.6.egg/plone/transformchain/transformer.py", line 48, in __call__
newResult = handler.transformIterable(result, encoding)
File "/Users/vito/.buildout/eggs/plone.app.theming-1.0-py2.6.egg/plone/app/theming/transform.py", line 257, in transformIterable
params[name] = quote_param(expression(expressionContext))
File "/Users/vito/.buildout/eggs/Zope2-2.13.13-py2.6.egg/Products/PageTemplates/ZRPythonExpr.py", line 48, in __call__
return eval(self._code, vars, {})
File "PythonExpr", line 1, in <expression>
File "/Users/vito/.buildout/eggs/AccessControl-2.13.7-py2.6-macosx-10.6-x86_64.egg/AccessControl/ImplPython.py", line 675, in guarded_getattr
v = getattr(inst, name)
AttributeError: 'FilesystemResourceDirectory' object has no attribute 'restrictedTraverse'
How I can solve this?
I suspect this is a bug in plone.app.theming: the context isn't set correctly. Strange, though.
Just confirming that the issue exits:
I get about the same traceback, the site itself looks fine, but for every click inside the site I get a the following traceback in my instance fg:
2012-08-10 15:05:05 ERROR plone.transformchain Unexpected error whilst trying to apply transform chain
Traceback (most recent call last):
File "/opt/etc/buildout/eggs/plone.transformchain-1.0.2-py2.6.egg/plone/transformchain/transformer.py", line 48, in __call__
newResult = handler.transformIterable(result, encoding)
File "/opt/etc/buildout/eggs/plone.app.theming-1.0-py2.6.egg/plone/app/theming/transform.py", line 257, in transformIterable
params[name] = quote_param(expression(expressionContext))
File "/opt/etc/buildout/eggs/Zope2-2.13.10-py2.6.egg/Products/PageTemplates/ZRPythonExpr.py", line 48, in __call__
return eval(self._code, vars, {})
File "PythonExpr", line 1, in <expression>
AttributeError: 'FilesystemResourceDirectory' object has no attribute 'Language'
This is because I have the following line in my manifest.cfg (which is about the same as the parameter line in the plone_control_panel:
lang = python: context.Language()
In a way in my case this is sort of logical, since not all content objects have an index called Language().
But the 'context' in this case is apparently refering to the 'FileSystemResourceDirectory' and not to the piece of content you are on?
I'll try with pdb if I can find some more info...