I have a python ‘s byte array
b=b'hello'
and I want to use it as the parameter of QtextStream to create a QtextStream object ,but QtextStream accept Qt’s byte array rather than Python’s .I wonder is there anyway to to convert python ‘s byte array to QByteArray ?
I’ve tested the following code ,it seems I cannot do it in this way .
>>> b=b'hello'
>>> from PyQt4.QtCore import *
>>> c=QTextStream(QByteArray(b)).readAll()
>>> c
''
>>>
I guess the issue is that QByteArray object is constructed, passed to the QTextStream and then immediately destroyed. QTextStream doesn't make a copy of data, it only keeps a pointer to the data source. Since the source is destroyed, no data can be longer read. I even managed to get segmentation fault error when trying to call readAll() using some variations of your code. So, the solution is to store your QByteArray object in a variable as long as you need it.
>>> buf = QByteArray(b)
>>> c = QTextStream(buf).readAll()
>>> c
PyQt4.QtCore.QString(u'hello')
The original question 'how to convert python ‘s byte array to QByteArray?' do not refer to your issue. It easy to see that it can be done using the obvious way:
>>> QByteArray(b)
PyQt4.QtCore.QByteArray('hello')
Related
So I have a struct:
typedef struct {
int x = 0;
} Command;
and global vectors:
vector<Command> cmdList = {}; vector<Event*> eventList = {};
I push_back, erase and clear the vector in another .cpp file. This gets pushed back into:
vector<Command> cmdsToExec = {}; inside per Event struct created. I use this to push_back:
eventList.push_back( new Event() ); eventList[int( eventList.size() ) - 1]->cmdsToExec = cmdList;
My problem A) these Event*s can't be erased with delete and B) is that Valgrind gives this error while trying to determine the size of the cmdsToExec:
==25096== Invalid read of size 8
==25096== at 0x113372: std::vector<Command, std::allocator<Command> >::size() const (stl_vector.h:919)
==25096== by 0x11C1C7: eventHandler::processEvent() (eventHandler.cpp:131)
==25096== by 0x124590: main (main.cpp:88)
==25096== Address 0x630a9e0 is 32 bytes inside a block of size 56 free'd
==25096== at 0x484BB6F: operator delete(void*, unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==25096== by 0x11C116: eventHandler::processEvent() (eventHandler.cpp:222)
==25096== by 0x124590: main (main.cpp:88)
==25096== Block was alloc'd at
==25096== at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==25096== by 0x11B4A5: eventHandler::createEvent() (eventHandler.cpp:58)
==25096== by 0x11B412: eventHandler::doState() (eventHandler.cpp:41)
==25096== by 0x124575: main (main.cpp:83)
Ive tracked it to the line:
while( int( eventList[0]->cmdsToExec.size() ) > 0 ) {
Im not trying to solve this specific problem, its more about how to properly delete and unallocate a dynamic pointer from a global vector of dynamic pointers. That being said there are no objects (and I want to keep it that way). Will I need a struct deconstructor (no pun intended)? Also I dont believe cmdList vector ever has a memory leak according to this error message, also as Im clearing it all at once.
My thoughts on fixing this are to place both global vectors into my main() function and pass them into the program from there. I thought it would be unnecessary to do this and would slow the program down. Thinking now, I guess it wouldn't.
My guess is that this is a problem related to the order of destruction of static/global objects.
C++ guarantees that for a given translation unit (i.e., a cpp source file) then statics/global objects get created in the order that they are defined, and they are destroyed in the reverse order.
C++ gives no guarantee between different translation units.
My recommendations are:
Avoid statics/globals. Move them to be class members if possible.
If you have any dependencies between statics/globals then put them all in the same source file so that you have control over the order of their creation and destruction.
I have a data structure in Cython that uses a char * member.
What is happening is that the member value seems to lose its scope outside of a function that assigns a value to the member. See this example (using IPython):
[nav] In [26]: %%cython -f
...: ctypedef struct A:
...: char *s
...:
...: cdef char *_s
...:
...: cdef void fn(A *a, msg):
...: s = msg.encode()
...: a[0].s = s
...:
...: cdef A _a
...: _a.s = _s
...: fn(&_a, 'hello')
...: print(_a.s)
...: print(b'hola')
...: print(_a.s)
b'hello'
b'hola'
b"b'hola'"
It looks like _a.s is deallocated outside of fn and is being assigned any junk that is in memory that fits the slot.
This happens only under certain circumstances. For example, if I assign b'hello' to s instead of the encoded string inside fn(), the correct string is printed outside of the function.
As you can see, I also added an extra declaration for the char variable and assigned it to the struct before executing fn, to make sure that the _a.s pointer does not get out of scope. However, my suspect is that the problem is assigning the member to a variable that is in the function scope.
What is really happening here, and how do I resolve this issue?
Thanks.
Your problem is, that the pointer a.s becomes dangling in the fn-function as soon as it is created.
When calling msg.encode() the temporary byte-object s is created and the address of its buffer is saved to a.s. However, directly afterwards (i.e. at the exit from the function) the temporary bytes-object gets destroyed and the pointer becomes dangling.
Because the bytes object was small, Python's memory manager manages its memory in the arena - which guaranties that there is no segfault when you access the address (lucky you).
While the temporary object is destroyed, the memory isn't overwritten/sanatized and thus it looks as if the temporary object where still alive from A.s's point of view.
Whenever you create a new bytes-object similar in size to the temporary object, the old memory from the arena might get reused, so that your pointer a.s could point to the buffer of the newly allocated bytes-object.
Btw, would you use a[0].s = msg.encode() directly (and I guess you did), the Cython would not build and tell you, that you try to say reference to a temporary Python object. Adding an explicit reference fooled the Cython, but didn't help your case.
So what to do about it? Which solution is appropriate depends on the bigger picture, but the available strategies are:
Manage the memory of A.s. I.e. manually reserve memory, copy from the temporary object, free memory as soon as done.
Manage reference counting: Add a PyObject * to the A-struct. Assign the temporary object ot it (don't forget to increase the reference counter manually), decrease reference counter as soon as done.
Collect references of temporary objects into a pool (e.g. a list), which would keep them alive. Clear the pool as soon as objects aren't needed.
Not always the best, but easiest is the option 3 - you neither have to manage the memory not the reference counting:
%%cython
...
pool=[]
cdef void fn(A *a, msg):
s = msg.encode()
pool.append(s)
a[0].s = s
While this doesn't solve the principal problem, using PyUnicode_AsUTF8 (inspired by this answer) might be a satisfactory solution in this case:
%%cython
# it is not wrapped by `cpython.pxd`:
cdef extern from "Python.h":
const char* PyUnicode_AsUTF8(object unicode)
...
cdef void fn(A *a, msg):
a[0].s = PyUnicode_AsUTF8(msg) # msg.encode() uses utf-8 as default.
This has at least two advantages:
the pointer a[0].s is valid as long as msg is alive
calling PyUnicode_AsUTF8(msg) is faster than msg.encode(), because it reuses cached buffer, so it basically O(1) after first call, while msg.encode() needs at least copy the memory and is O(n) with n-number of characters.
I want to get the data pointer of a string variable(like string::c_str() in c++) to pass to a c function and I found this doesn't work:
package main
/*
#include <stdio.h>
void Println(const char* str) {printf("%s\n", str);}
*/
import "C"
import (
"unsafe"
)
func main() {
s := "hello"
C.Println((*C.char)(unsafe.Pointer(&(s[0]))))
}
Compile error info is: 'cannot take the address of s[0]'.
This will be OK I but I doubt it will cause unneccesary memory apllying. Is there a better way to get the data pointer?
C.Println((*C.char)(unsafe.Pointer(&([]byte(s)[0]))))
There are ways to get the underlying data from a Go string to C without copying it. It will not work as a C string because it is not a C string. Your printf will not work even if you manage to extract the pointer even if it happens to work sometimes. Go strings are not C strings. They used to be for compatibility when Go used more libc, they aren't anymore.
Just follow the cgo manual and use C.CString. If you're fighting for efficiency you'll win much more by just not using cgo because the overhead of calling into C is much bigger than allocating some memory and copying a string.
(*reflect.StringHeader)(unsafe.Pointer(&sourceTail)).Data
Strings in go are not null terminated, therefore you should always pass the Data and the Len parameter to the corresponding C functions. There is a family of functions in the C standard library to deal with this type of strings, for example if you want to format them with printf, the format specifier is %.*s instead of %s and you have to pass both, the length and the pointer in the arguments list.
I'm writing an OpenCL program that applies a convolution matrix on an image. Everything works fine if I store all pixel on an array image[height*width][4] (line 65,commented) (sorry, I speak Spanish, and I code mostly in Spanish). But, since the images I'm working with are really large, I need to allocate the memory dynamically. I execute the code, and I get a Segmentation fault error.
After some poor man's debugging, I found out the problem arises after executing the kernel and reading the output image back into the host, storing the data into the dynamically allocated array. I just can't access the data of the array without getting the error.
I think the problem is the way the clEnqueueReadImage function (line 316) writes the image data into the image array. This array was allocated dynamically, so it has no predefined "structure".
But I need a solution, and I can't find it, nor on my own or on Internet.
The C program and the OpenCL kernel are here:
https://gist.github.com/MigAGH/6dd0fddfa09f5aabe7eb0c2934e58cbe
Don't use pointers to pointers (unsigned char**). Use a regular pointer instead:
unsigned char* image = (unsigned char*)malloc(sizeof(unsigned char)*ancho*alto*4);
Then in the for loop:
for(i=0; i<ancho*alto; i++){
unsigned char* pixel = (unsigned char*)malloc(sizeof(unsigned char)*4);
fread (pixel, 4, 1, bmp_entrada);
image[i*4] = pixel[0];
image[i*4+1] = pixel[1];
image[i*4+2] = pixel[2];
image[i*4+3] = pixel[3];
free(pixel);
}
I'm trying to read an array of unsigned shorts using the Qt API. Unfortunately, I'm not getting the desired results.
The following code
QFile in(fileName);
int len = in.size();
QDataStream d(&in);
quint16 *data = new quint16[len];
qDebug() << data[0];
qDebug() << data[1];
d >> data[0];
qDebug() << data[0];
qDebug() << data[1];
outputs
52685
52685
13109
52685
Implying that the data is only changed at the first array position. Also, I always thought that arrays are zero initialized? Using a QByteArray doesn't seem to work here, that's why I'm trying to use a array of quint16 (= unsigned shorts). Using a loop may be an option, but I'm trying to avoid a costly loop where ever possible.
So, how do fill said array (data) with the desired data from the file? Is it possible to carry the data using a QByteArray?
First of all, in.size() returns the size of the file in bytes, and since you are using unsigned shorts (which are 2 bytes each), the size of your data array should be len/2.
Also, QDataStream is provided for serialization purposes. This means that it is mainly useful for extracting single objects at a time. See the documentation for QDataStrean for more information.
You can extract the whole array without loops or copying with this code:
QFile in(fileName);
in.open(QFile::ReadOnly);
QByteArray byteArray = in.readAll();
quint16 *data = (quint16*) byteArray.data();
If you only wish to read data but never modify it, you can make your program a lot faster by changing the last line to:
const quint16* data = (const quint16*) byteArray.constData();
Keep in mind however that with this (somewhat ugly) code the pointer will only be valid for the lifetime of the QByteArray object. This usually means that you can only use data until the end of the function.
If you wish your data to persist longer than that, you must allocate the array and read directly into it:
QFile in(fileName);
in.open(QFile::ReadOnly);
int len = in.size();
quint16 *data = new quint16[len/2];
in.read((char*)data, len);
This way, you can access the data until you delete[] data.
Finally, to answer your subquestion, arrays are zero-initialized only if they are globally defined (and you shouldn't rely even on this). Arrays allocated with new or malloc are never zero-initialized for performance reasons.
If you look at the documentation for QDataStream, you'll see there are two other methods at your disposal worth considering, if you don't want to use a loop:
QDataStream::readBytes - This will allocate a char[] buffer for you. Or,
QDataStream::readRawData - This will read data into a buffer you provide.
The problem with these is they work with char (bytes), not quint16 as you desire.
I would recommend using a loop to read quint16s. It will be the most clear code. And any respectable underlying stream implementation will be reading-ahead from the hardware into a buffer, so that your many successive >> calls won't be as expensive.