I am recently using google collab juypter notebook.After Uploading text file, unable to open the file using open function in python 3.
from google.colab import files
import io
uploaded = files.upload()
for fn in uploaded.keys():
print('User uploaded file "{name}" with length {length} bytes'.format(
name=fn, length=len(uploaded[fn])))
data_path = io.StringIO(uploaded['fra.txt'].decode('utf-8'))
with open(data_path, 'rb') as f:
lines = f.read().split('\n')
but it gives this error : TypeError: expected str, bytes or os.PathLike object, not _io.StringIO
how to open text file in google collab juypter notebook ?
Change to just
data_path = 'fra.txt'
Should work.
The _io.StringIO refers to the StringIO object (in-memory file stream). "For strings StringIO can be used like a file opened in text mode."
The issue is that the file is already open and you have it available to you as a StringIO buffer. I think you want to do readlines() on the StringIO object (data_path).
You can also call getvalue() on the object and get the str of the entire buffer.
https://docs.python.org/3/library/io.html#io.StringIO
See my example here; which I started with your code...
https://colab.research.google.com/drive/1Vbh13FVm02HMXeHXx-Zko1pFpqyp7bwI
do like this
with open('anna.txt', 'r') as f:
text=f.read()
vocab = sorted(set(text))
vocab_to_int = {c: i for i, c in enumerate(vocab)}
int_to_vocab = dict(enumerate(vocab))
encoded = np.array([vocab_to_int[c] for c in text], dtype=np.int32)
Related
#bot.on(events.CallbackQuery)
async def handler(event):
global fotomandate
global i
if event.data == b"1":
await event.respond("how many photos do i send?")
numerofoto = int(input("how many photos do i send?")) ##ignore this line i'll fix later
print (numerofoto)
while i < numerofoto:
path = (r"C:\Users\x\Desktop\Nuova cartella (2)")
fotorandom = random.choice([
x for x in os.listdir(r"C:\Users\x\Desktop\Nuova cartella (2)")
if os.path.isfile(os.path.join(path, x))
])
i += 1
await event.reply(file=fotorandom)
i need to send n(in input on telegram) random photos from a directory but it says
ValueError: Failed to convert bonni media-jpg to media. Not an existing file, an HTTP URL or a valid bot-API-like file ID
As the error states:
ValueError: Failed to convert bonni media-jpg to media. Not an existing file, an HTTP URL or a valid bot-API-like file ID
Inside the loop guard, you're doing os.path.join(path, x). However, your x does not contain the full path. Then the file is searched in the working directory, but it is not found. You need to specify the correct path there too:
fotorandom = random.choice([
os.path.join(path, x) # <- new
for x in os.listdir(path) # <- better to avoid repeating dir
if os.path.isfile(os.path.join(path, x))
])
I'm wondering if jupyter_client is able to return code sent in to the execute function as HTML somehow?
I'm also wondering if I can do the same with stdout and stderr, as well as markdown?
If jupyter_client cannot do this, is there a jupyter library that does?
Adapting the solution from here might help. This adaptation takes the request of 1+1 or msg_id=c.execute('1+1') and returns the result formatted in html as bold red text with this display(HTML('<div style="color:Red;"><b>' + res + '</b></div>')) using IPython's display module. The kernel info status has been commented out but left for reference.
from subprocess import PIPE
from jupyter_client import KernelManager
from IPython.display import display, HTML
from queue import Empty
km = KernelManager(kernel_name='python3')
km.start_kernel()
# print(km.is_alive())
try:
c = km.client()
msg_id=c.execute('1+1')
state='busy'
data={}
while state!='idle' and c.is_alive():
try:
msg=c.get_iopub_msg(timeout=1)
if not 'content' in msg: continue
content = msg['content']
if 'data' in content:
data=content['data']
if 'execution_state' in content:
state=content['execution_state']
except Empty:
pass
res = data['text/plain']
# print(data)
display(HTML('<div style="color:Red;"><b>' + res + '</b></div>'))
except KeyboardInterrupt:
pass
finally:
km.shutdown_kernel()
# print(km.is_alive())
Also see here for more info.
Question : Here is python code for oci (oracle cloud) . It is able to create Bucket or download and upload fine in bucket in root compartment . But i am not able to do the same on sub compartment. Sub compartment is " My_Sub_Compartment"
Please advise how to fix it .
import os
import oci
import io
from oci.config import from_file
data_dir = "D:\\DataScienceAndStats\\artificialintelligence\\CS223A"
files_to_process = [file for file in os.listdir(data_dir) if file.endswith('txt')]
bucket_name = "Sales_Data"
# this is to configure the oci configuration file
my_config = from_file(file_location="C:\\Users\\amits\\Desktop\\Oracle_Cloud\\config_file_oci.txt")
print(my_config)
# Test Configuration file of oci
# print(validate_config(my_config))
"""
Create object storage client and get its namespace
"""
object_storage_client = oci.object_storage.ObjectStorageClient(my_config)
namespace = object_storage_client.get_namespace().data
"""
Create a bucket if it does not exist
"""
try:
create_bucket_response = object_storage_client.create_bucket(namespace,
oci.object_storage.models.CreateBucketDetails(name=bucket_name, compartment_id=my_config['tenancy']))
except Exception as e:
print("Please read below messages")
print(e.message)
print(e.status)
"""
Uploading the files
"""
print("uploading files to bucket")
for upload_file in files_to_process:
print('Uploading file {}'.format(upload_file))
object_storage_client.put_object(namespace, bucket_name, upload_file, io.open(os.path.join(data_dir, upload_file),
'rb'))
"""
Listing a files in the Bucket
"""
object_list = object_storage_client.list_objects(namespace, bucket_name)
for o in object_list.data.objects:
print(o.name)
"""
Downloading files from Bucket
"""
object_name = "1.txt"
destination_dir = 'D:\\DataScienceAndStats\\artificialintelligence\\CS223A\\moved_files'.format(object_name)
get_obj = object_storage_client.get_object(namespace, bucket_name, object_name)
with open(os.path.join(destination_dir, object_name), 'wb') as f:
for chunk in get_obj.data.raw.stream(1024 * 1024, decode_content=False):
f.write(chunk)
Within create_bucket_response function you should provide the sub compartment OCID, instead of my_config['tenancy']. Have you tried this way? my_config['tenancy'] will always return the root compartment.
If you add compartment=ocid1.compartment.oc1..[...] (whatever OCID the target compartment has)
to your config_file_oci.txt and replace my_config['tenancy'] with my_config['compartment'] it should work.
You cannot have the same name for two buckets in the same namespace/tenancy, have you deleted the one you already created?
I'm trying to use the below mail function for python3 which is throwing error NameError: name 'file' is not defined which its works perfectly for python2.
I got to know file() is not supported in Python 3 what will be substitute of file.
#!/usr/bin/env python3
from subprocess import Popen, PIPE
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import os
############ File comparison & sendmail part starts here ########
def ps_Mail():
filename = "/tmp/ps_msg"
f = file(filename)
if os.path.exists(filename) and os.path.getsize(filename) > 0:
mailp = Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=PIPE)
msg = MIMEMultipart('alternative')
msg['To'] = "sam#seemac.com"
msg['Subject'] = "Uhh!! Unsafe process seen"
msg['From'] = "psCheck#seemac.com"
msg1 = MIMEText(filename.read(), 'text')
msg.attach(msg1)
mailp.communicate(msg.as_string())
ps_Mail()
I have edited your code and this should work, please try this...
There are two key things to change universal_newlines=True and use open() instead of file().
#!/usr/bin/env python3
from subprocess import Popen, PIPE
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import os
############ File comparison & sendmail part starts here ########
def ps_Mail():
filename = "/tmp/ps_msg"
f = open(filename)
if os.path.exists(filename) and os.path.getsize(filename) > 0:
mailp = Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=PIPE, universal_newlines=True)
msg = MIMEMultipart('alternative')
msg['To'] = "sam#seemac.com"
msg['Subject'] = "Uhh!! Unsafe process seen"
msg['From'] = "psCheck#seemac.com"
msg1 = MIMEText(filename.read(), 'text')
msg.attach(msg1)
mailp.communicate(msg.as_string())
ps_Mail()
For more details....
What is the difference between using universal_newlines=True (with bufsize=1) and using default arguments with Popen
The default values are: universal_newlines=False (meaning input/output is accepted as bytes, not Unicode strings plus the universal newlines mode handling (hence the name of the parameter though text_mode might have been a better name here) is disabled -- you get binary data as is (unless POSIX layer on Windows messes it up) and bufsize=-1 (meaning the streams are fully buffered -- the default buffer size is used).
universal_newlines=True uses locale.getpreferredencoding(False) character encoding to decode bytes (that may be different from ascii encoding used in your code).
If universal_newlines=False then for line in Robocopy.stdout: iterates over b'\n'-separated lines. If the process uses non-ascii encoding e.g., UTF-16 for its output then even if os.linesep == '\n' on your system; you may get a wrong result. If you want to consume text lines, use the text mode: pass universal_newlines=True or use io.TextIOWrapper(process.stdout) explicitly.
The second version does include universal_newlines and therefore I specify a bufsize.
In general, It is not necessary to specify bufsize if you use universal_newlines (you may but it is not required). And you don't need to specify bufsize in your case. bufsize=1 enables line-bufferred mode (the input buffer is flushed automatically on newlines if you would write to process.stdin) otherwise it is equivalent to the default bufsize=-1.
I'm creating a fork of my Plone site (which has not been forked for a long time). This site has a special catalog object for user profiles (a special Archetypes-based object type) which is called portal_user_catalog:
$ bin/instance debug
>>> portal = app.Plone
>>> print [d for d in portal.objectMap() if d['meta_type'] == 'Plone Catalog Tool']
[{'meta_type': 'Plone Catalog Tool', 'id': 'portal_catalog'},
{'meta_type': 'Plone Catalog Tool', 'id': 'portal_user_catalog'}]
This looks reasonable because the user profiles don't have most of the indexes of the "normal" objects, but have a small set of own indexes.
Since I found no way how to create this object from scratch, I exported it from the old site (as portal_user_catalog.zexp) and imported it in the new site. This seemed to work, but I can't add objects to the imported catalog, not even by explicitly calling the catalog_object method. Instead, the user profiles are added to the standard portal_catalog.
Now I found a module in my product which seems to serve the purpose (Products/myproduct/exportimport/catalog.py):
"""Catalog tool setup handlers.
$Id: catalog.py 77004 2007-06-24 08:57:54Z yuppie $
"""
from Products.GenericSetup.utils import exportObjects
from Products.GenericSetup.utils import importObjects
from Products.CMFCore.utils import getToolByName
from zope.component import queryMultiAdapter
from Products.GenericSetup.interfaces import IBody
def importCatalogTool(context):
"""Import catalog tool.
"""
site = context.getSite()
obj = getToolByName(site, 'portal_user_catalog')
parent_path=''
if obj and not obj():
importer = queryMultiAdapter((obj, context), IBody)
path = '%s%s' % (parent_path, obj.getId().replace(' ', '_'))
__traceback_info__ = path
print [importer]
if importer:
print importer.name
if importer.name:
path = '%s%s' % (parent_path, 'usercatalog')
print path
filename = '%s%s' % (path, importer.suffix)
print filename
body = context.readDataFile(filename)
if body is not None:
importer.filename = filename # for error reporting
importer.body = body
if getattr(obj, 'objectValues', False):
for sub in obj.objectValues():
importObjects(sub, path+'/', context)
def exportCatalogTool(context):
"""Export catalog tool.
"""
site = context.getSite()
obj = getToolByName(site, 'portal_user_catalog', None)
if tool is None:
logger = context.getLogger('catalog')
logger.info('Nothing to export.')
return
parent_path=''
exporter = queryMultiAdapter((obj, context), IBody)
path = '%s%s' % (parent_path, obj.getId().replace(' ', '_'))
if exporter:
if exporter.name:
path = '%s%s' % (parent_path, 'usercatalog')
filename = '%s%s' % (path, exporter.suffix)
body = exporter.body
if body is not None:
context.writeDataFile(filename, body, exporter.mime_type)
if getattr(obj, 'objectValues', False):
for sub in obj.objectValues():
exportObjects(sub, path+'/', context)
I tried to use it, but I have no idea how it is supposed to be done;
I can't call it TTW (should I try to publish the methods?!).
I tried it in a debug session:
$ bin/instance debug
>>> portal = app.Plone
>>> from Products.myproduct.exportimport.catalog import exportCatalogTool
>>> exportCatalogTool(portal)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File ".../Products/myproduct/exportimport/catalog.py", line 58, in exportCatalogTool
site = context.getSite()
AttributeError: getSite
So, if this is the way to go, it looks like I need a "real" context.
Update: To get this context, I tried an External Method:
# -*- coding: utf-8 -*-
from Products.myproduct.exportimport.catalog import exportCatalogTool
from pdb import set_trace
def p(dt, dd):
print '%-16s%s' % (dt+':', dd)
def main(self):
"""
Export the portal_user_catalog
"""
g = globals()
print '#' * 79
for a in ('__package__', '__module__'):
if a in g:
p(a, g[a])
p('self', self)
set_trace()
exportCatalogTool(self)
However, wenn I called it, I got the same <PloneSite at /Plone> object as the argument to the main function, which didn't have the getSite attribute. Perhaps my site doesn't call such External Methods correctly?
Or would I need to mention this module somehow in my configure.zcml, but how? I searched my directory tree (especially below Products/myproduct/profiles) for exportimport, the module name, and several other strings, but I couldn't find anything; perhaps there has been an integration once but was broken ...
So how do I make this portal_user_catalog work?
Thank you!
Update: Another debug session suggests the source of the problem to be some transaction matter:
>>> portal = app.Plone
>>> puc = portal.portal_user_catalog
>>> puc._catalog()
[]
>>> profiles_folder = portal.some_folder_with_profiles
>>> for o in profiles_folder.objectValues():
... puc.catalog_object(o)
...
>>> puc._catalog()
[<Products.ZCatalog.Catalog.mybrains object at 0x69ff8d8>, ...]
This population of the portal_user_catalog doesn't persist; after termination of the debug session and starting fg, the brains are gone.
It looks like the problem was indeed related with transactions.
I had
import transaction
...
class Browser(BrowserView):
...
def processNewUser(self):
....
transaction.commit()
before, but apparently this was not good enough (and/or perhaps not done correctly).
Now I start the transaction explicitly with transaction.begin(), save intermediate results with transaction.savepoint(), abort the transaction explicitly with transaction.abort() in case of errors (try / except), and have exactly one transaction.commit() at the end, in the case of success. Everything seems to work.
Of course, Plone still doesn't take this non-standard catalog into account; when I "clear and rebuild" it, it is empty afterwards. But for my application it works well enough.