How do I turn a file's contents into a dictionary? - dictionary

I have a function that I want to open .dat files with, to extract data from them, but the problem is I don't know how to turn that data back into a dictionary to store in a variable. Currently, the data in the files are stored like this: "{"x":0,"y":1}" (it uses up only one line of the file, which is just the normal structure of a dictionary).
Below is just the function where I open the .dat file and try to extract stuff from it.
def openData():
file = fd.askopenfile(filetypes=[("Data",".dat"),("All Files",".*")])
filepath = file.name
if file is None:
return
with open(filepath,"r") as f:
contents = dict(f.read())
print(contents["x"]) #let's say there is a key called "x" in that dictionary
This is the error that I get from it: (not because the key "x" is not in dict, trust me)
Exception in Tkinter callback
Traceback (most recent call last):
File "...\AppData\Local\Programs\Python\Python39\lib\tkinter\__init__.py", line 1892, in __call__
return self.func(*args)
File "...\PycharmProjects\[this project]\main.py", line 204, in openData
contents = dict(f.read())
ValueError: dictionary update sequence element #0 has length 1; 2 is required
Process finished with exit code 0
Update: I tried using json and it worked, thanks to #match
def openData():
file = fd.askopenfile(filetypes=[("Data",".dat"),("All Files",".*")])
filepath = file.name
if file is None:
return
with open(filepath,"r") as f:
contents = dict(json.load(f))
print(contents["x"])

You need to parse the data to get a data structure from a string, fortunately, Python provides a function for safely parsing Python data structures: ast.literal_eval(). E.g:
import ast
...
with open("/path/to/file", "r") as data:
dictionary = ast.literal_eval(data.read())
Reference stackoverflow

Related

Running multiple files from with Scilab program

I'm new to Scilab. I have to run the same program with a dozen different input files. Currently I simply uncomment the line and then rerun the program, and change the output file to a new name
// Input data file
data_file = 'data1.txt';
//data_file = 'data2.txt';
//data_file = 'data3.txt';
//data_file = 'data4.txt';
//data_file = 'data5.txt';
//data_file = 'data6.txt';
etc. another 6 lines
// Output data file name
output_data = '/output_files/data1.csv';
Is there a way to read in each file (data1.txt, data2.txt...) execute the body of the program and then output a new output file (data1.csv, data2.csv ...) instead of what I'm doing now, which is running the program and then editing it to use the next file and run again?
Just do something like:
for i=1:6
// Input data file
data_file=msprintf("data%d.txt",i);
// Output data file name
output_data=msprintf("/output_files/data%d.csv",i);
// exec the body of your script
end

Rename Dexterity object (id) after copy

It's simple to choose the object ID at creation time with INameChooser.
But we also want to be able to choose the object ID after a clone (and avoid copy_of in object ID).
We tried several different solutions :
subscribers on events :
OFS.interfaces.IObjectClonedEvent
zope.lifecycleevent.interfaces.IObjectAddedEvent
...
manage_afterClone method on content class
Every time, we get a traceback because we changed the ID "too soon". For example when using Plone API :
File "/Users/laurent/.buildout/eggs/plone.api-2.0.0a1-py3.7.egg/plone/api/content.py", line 256, in copy
return target[new_id]
File "/Users/laurent/.buildout/eggs/plone.folder-3.0.3-py3.7.egg/plone/folder/ordered.py", line 241, in __getitem__
raise KeyError(key)
KeyError: 'copy_of_87c7f9b7e7924d039b832d3796e7b5a3'
Or, with a Copy / Paste in the Plone instance :
Module plone.app.content.browser.contents.paste, line 42, in __call__
Module OFS.CopySupport, line 317, in manage_pasteObjects
Module OFS.CopySupport, line 229, in _pasteObjects
Module plone.folder.ordered, line 73, in _getOb
AttributeError: 'copy_of_a7ed3d678f2643bc990309cde61a6bc5'
It's logical, because the ID is stored before events notification / manage_afterClone call for later use.
Even defining a _get_id on containers cannot work to define the ID, because we don't have the object to get attributes from (and generate the ID).
But then, how could we achieve this in a clean way ?
Please tell me there is a better solution than redefining _pasteObjects (OFS.CopySupport) !
Thank you for your inputs !
So unfortunately not...
But you can access the original object from within _get_id.
For example:
from OFS.CopySupport import _cb_decode
from plone import api
...
def _get_id(self, id_):
# copy_or_move => 0 means copy, 1 means move
copy_or_move, path_segments = _cb_decode(self.REQUEST['__cp']
source_path = '/'.join(path_segments[0]) # Imlement for loop for more than one copied obj.
app = self.getPhysicalRoot()
# access to source obj
source_obj = app.restrictedTraverse(source_path)
# Do whatever you need - probably using INameChooser
....
To have a canonical way of patching this I use collective.monkeypatcher
Once installed add the following in ZCML:
<include package="collective.monkeypatcher" />
<monkey:patch
class="OFS.CopySupport.CopyContainer"
original="_get_id"
replacement="patches.patched_get_id"
/>
Where patches.py is your module containing the new method patched_get_id, which replaces _get_id.
I'm sorry I don't have better news for you, but this is how I solved a similar requirement.
This code (patched _get_id) adds a counter at the end of a id if already there.
def patched_get_id(self, id_)
match = re.match('^(.*)-([\d]+)$', id_)
if match:
id_ = match.group(1)
number = int(match.group(2))
else:
number = 1
new_id = id_
while new_id in self.objectIds():
new_id = '{}-{}'.format(id_, number)
number += 1
return new_id

Workaround for case-sensitive input to dir

I am using Octave 5.1.0 on Windows 10 (x64). I am parsing a series of directories looking for an Excel spreadsheet in each directory with "logbook" in its filename. The problem is these files are created by hand and the filenaming isn't consistent: sometimes it's "LogBook", other times it's "logbook", etc...
It looks like the string passed as input to the dir function is case-sensitive so if I don't have the correct case, dir returns an empty struct. Currently, I am using the following workaround, but I wondered if there was a better way of doing this (for a start I haven't captured all possible upper/lower case combinations):
logbook = dir('*LogBook.xls*');
if isempty(logbook)
logbook = dir('*logbook.xls*');
if isempty(logbook)
logbook = dir('*Logbook.xls*');
if isempty(logbook)
logbook = dir('*logBook.xls*');
if isempty(logbook)
error(['Could not find logbook spreadsheet in ' dir_name '.'])
end
end
end
end
You need to get the list of filenames (either via readdir, dir, ls), and then search for the string in that list. If you use readdir, it can be done like this:
[files, err, msg] = readdir ('.'); # read current directory
if (err != 0)
error ("failed to readdir (error code %d): %s", msg);
endif
logbook_indices = find (cellfun (#any, regexpi (files, 'logbook'));
logbook_filenames = files(logbook_indices);
A much less standard approach could be:
glob ('*[lL][oO][gG][bB][oO][kK]*')

Stuck with string.translate function in python 3

import os
def rename_files():
#(1) get file names from a folder
file_list = os.listdir("my_directory")
#print(file_list)
os.chdir("my_directory")
saved_path = os.getcwd()
print("Current work directory is " + saved_path)
os.getcwd()
#(2) for each file, rename filename
for file_name in file_list:
os.rename(file_name, file_name.translate(None, "0123456789"))
os.chdir("my_directory")
rename_files()
And after this I've got an error:
TypeError: translate() takes exactly one argument (2 given)
str.translate in python 3.x accepts just one argument i.e. translation table.
From docs:
str.translate(table)
Return a copy of the string in which each character has been mapped
through the given translation table
You can create required translation table using str.maketrans
table = str.maketrans(dict.fromkeys('0123456789'))
file_name.translate(table)

incremental select using id from SQLite in Twisted

I am trying to select data from a table in SQLite one row ONLY at a time for each call to the function, and I want the row to increment on each call (self.count is initialized elsewhere and 'line' is irrelevant here) I am using an adbapi connection pool in Twisted to connect to the DB. Here is the code I have tried:
def queryBTData4(self,line):
self.count=self.count+1
uuId=self.count
query="SELECT co2_data, patient_Id FROM btdata4 WHERE uid=:uid",{"uid": uuId}
d = self.dbpool.runQuery(query)
return d
This code works if I just set uid=1 or any other number in the DB (I used autoincrement for uid when I created the DB) but when I try to assign a value to uid (i.e. self.count via uuId) it reports that the operator has to be string or unicode.(I have tried both but it does not seem to help) However, I know that the query statement above works just fine in a previous program when I use a cursor and the execute command but I cannot see why it does not work here. I have tried all sorts of combinations and searched for a solution but have not found anything yet that works.(I have also tried the statement with brackets and other forms)
Thanks for any help or advice.
Here is the entire code:
from twisted.internet import protocol, reactor
from twisted.protocols import basic
from twisted.enterprise import adbapi
import sqlite3, time
class ServerProtocol(basic.LineReceiver):
def __init__(self):
self.conn = sqlite3.connect('biomed2.db',check_same_thread=False)
self.dbpool = adbapi.ConnectionPool("sqlite3" , 'biomed2.db', check_same_thread=False)
def connectionMade(self):
self.sendLine("conn made")
factory = protocol.ClientFactory()
factory.protocol = ClientProtocol
factory.originator = self
reactor.connectTCP('localhost', 1234, factory)
def lineReceived(self, line):
self._received = line
self.insertBTData4(self._received)
self.sendLine("line recvd")
def forwardLine(self, recipient):
recipient.sendLine(self._received)
def insertBTData4(self,data):
print "data in insert is",data
chx=data
PID=2
device_ID=5
query="INSERT INTO btdata4(co2_data,patient_Id, sensor_Id) VALUES ('%s','%s','%s')" % (chx, PID, device_ID)
dF = self.dbpool.runQuery(query)
return dF
class ClientProtocol(basic.LineReceiver):
def __init__(self):
self.conn = sqlite3.connect('biomed2.db',check_same_thread=False)
self.dbpool = adbapi.ConnectionPool("sqlite3" , 'biomed2.db', check_same_thread=False)
self.count=0
def connectionMade(self):
print "server-client made connection with client"
self.factory.originator.forwardLine(self)
#self.transport.loseConnection()
def lineReceived(self, line):
d=self.queryBTData4(self)
d.addCallbacks(self.sendData,self.printError )
def queryBTData4(self,line):
self.count=self.count+1
query=("SELECT co2_data, patient_Id FROM btdata4 WHERE uid=:uid",{"uid": uuId})
dF = self.dbpool.runQuery(query)
return dF
def sendData(self,line):
data=str(line)
self.sendLine(data)
def printError(self,error):
print "Got Error: %r" % error
error.printTraceback()
def main():
factory = protocol.ServerFactory()
factory.protocol = ServerProtocol
reactor.listenTCP(4321, factory)
reactor.run()
if __name__ == '__main__':
main()
The DB is created in another program, thus:
import sqlite3, time, string
conn = sqlite3.connect('biomed2.db')
c = conn.cursor()
c.execute('''CREATE TABLE btdata4
(uid INTEGER PRIMARY KEY, co2_data integer, patient_Id integer, sensor_Id integer)''')
The main program takes data into the server socket and inserts into DB. On the client socket side, data is removed from the DB one line at a time and sent to an external server. The program also has the ability to send data from the server side to the client side if required but I am not doing so here at the moment.
In queryBTData(), every time the function is called the count increments and I assign that value to uuId, which I then pass to the query. I have had this query statement working in a program where I do not use the adbapi but it does not seem to work here. I hope this is clear enough but if not please let me know and I will try again.
EDIT:
I have modified the program to take one row from the DB at a time (see queryBTData() below) but have come across another problem.
def queryBTData4(self,line):
self.count=self.count+1
xuId= self.count
#xuId=10
return self.dbpool.runQuery("SELECT co2_data FROM btdata4 WHERE uid = ?",xuId)
#return self.dbpool.runQuery("SELECT co2_data FROM btdata4 WHERE uid = 10")
When the count gets to 10 I get an error (which I will post below) which states that: "Incorrect number of bindings supplied. The current statement uses 1, and there are 2 supplied"
I have tried setting xuId to 10 (see commented out line xuId=10) but I still get the same error. However, if I switch the return statements (to commented out return) I do indeed get correct row with no error. I have tried converting xuId to unicode but that makes no difference, I still get the same error. Basically, if I I set uid in the return statement to 10 or more (commented out return) it works, but if I set uid to xuId (i.e. uid=?,xuId) in the first return, it only works when xuId is below 10. The API documentation, as far as I can tell, gives no clue as to why this occurs.(I have also disabled the insert into DB to eliminate this and checked the SQLite3_ limit, which is 999)
Here are the errors I am getting when using the first return statement.
Got Error: <twisted.python.failure.Failure <class 'sqlite3.ProgrammingError'>>
Traceback (most recent call last):
File "c:\python26\lib\threading.py", line 504, in __bootstrap
self.__bootstrap_inner()
File "c:\python26\lib\threading.py", line 532, in __bootstrap_inner
self.run()
File "c:\python26\lib\threading.py", line 484, in run
self.__target(*self.__args, **self.__kwargs)
--- <exception caught here> ---
File "c:\python26\lib\site-packages\twisted\python\threadpool.py", line 207, i
n _worker
result = context.call(ctx, function, *args, **kwargs)
File "c:\python26\lib\site-packages\twisted\python\context.py", line 118, in c
allWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "c:\python26\lib\site-packages\twisted\python\context.py", line 81, in ca
llWithContext
return func(*args,**kw)
File "c:\python26\lib\site-packages\twisted\enterprise\adbapi.py", line 448, i
n _runInteraction
result = interaction(trans, *args, **kw)
File "c:\python26\lib\site-packages\twisted\enterprise\adbapi.py", line 462, i
n _runQuery
trans.execute(*args, **kw)
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current sta
tement uses 1, and there are 2 supplied.
Thanks.
Consider the API documentation for runQuery. Next, consider the difference between these three function calls:
c = a, b
f(a, b)
f((a, b))
f(c)
Finally, don't paraphrase error messages. Always quote them verbatim. Copy/paste whenever possible; make a note when you've manually transcribed them.

Resources