I am using a an object of QScintilla and I am reading the file in QScintilla Object incrementally.
Header myEditor.h
class myScintilla: public QScintilla {
public readFile();
};
#include "myEditor.h"
void myEditor::readFile() {
if (FILE* fp = fopen(ofilename.toLatin1(), "r")) {
QTextStream ts(fp, QIODevice::ReadOnly);
int bufferSize =(1024* 1024)/2;
do {
QString s = ts.read(bufferSize);
append(s);
} while(!ts.atEnd());
}
Even after this change there will be still performance issue while reading large files. It took around
1) 25 seconds to read a file of size 1.5 GB. (Machine cores 4 , 16 GB RAM)
2 10 seconds of file of size 512MB (on same machine)
Is there any way we can load the file in QScintilla object incrementally based on movement of scrollbar?
I found your question interesting so did a little bit of Googling on your behalf. It seems to me that while Scintilla exposes this functionality via the Loader interface, in fact the QScintilla class does not. To make this work, it seems that what you would have to do is use the QScintillaBase class to send the SCI_CREATELOADER message to the Scintilla control.
Edit: Also, you do not want to use append in a loop. That will cause all sorts of terrible things to happen. It will likely force rendering, some sort of indexing, etc. Before using my suggestion above, I would suggest that you instead build up a gigantic QString in memory and then set that at the end. Better to pre-allocate. That might be a little faster.
FINAL ANSWER
Edit #2: OK, it was bothering me that such an industrial strength editor component like Scintilla did not support this natively but it seems that the right way to do this is by using a combination of features:
You start with a document allocated using SCI_ALLOCATE where the number of bytes is the size of your file
You listen for the SCN_UPDATEUI event
Then, based on where the user is scrolling to, you load that data
It should be straightforward to map the above to QScintillaBase as a test.
Related
I want to use QAudioRecorder to record audio from the user and then use the audio output file for speech to text. I could successfully run and record audio from this example, http://doc.qt.io/qt-5/qtmultimedia-multimedia-audiorecorder-example.html.
But my problem is, I need to detect if user has stopped speaking while QAudioRecorder is actively recording audio. So QAudioRecorder should only stop when user is not speaking.
I could stop QAudioRecorder for fixed seconds using QTimer as below:
void AudioRecorder::toggleRecord()
{
if (audioRecorder->state() == QMediaRecorder::StoppedState) {
audioRecorder->setAudioInput(boxValue(ui->audioDeviceBox).toString());
QAudioEncoderSettings settings;
settings.setCodec(boxValue(ui->audioCodecBox).toString());
settings.setSampleRate(boxValue(ui->sampleRateBox).toInt());
settings.setBitRate(boxValue(ui->bitrateBox).toInt());
settings.setChannelCount(boxValue(ui->channelsBox).toInt());
settings.setQuality(QMultimedia::EncodingQuality(ui->qualitySlider->value()));
settings.setEncodingMode(ui->constantQualityRadioButton->isChecked() ?
QMultimedia::ConstantQualityEncoding :
QMultimedia::ConstantBitRateEncoding);
QString container = boxValue(ui->containerBox).toString();
audioRecorder->setEncodingSettings(settings, QVideoEncoderSettings(), container);
audioRecorder->record();
this->recordTimeout();
}
else {
this->stopRecording();
}
}
void AudioRecorder::recordTimeout()
{
QTimer* mTimer = new QTimer(this);
mTimer->setSingleShot(true);
connect(mTimer, SIGNAL(timeout()), SLOT(stopRecording()));
mTimer->start(6000);
}
void AudioRecorder::stopRecording()
{
audioRecorder->stop();
}
But instead of this it should stop recording when user is not speaking. The QAudioProbe class has this signal audioBufferProbed(QAudioBuffer) which may be helpful to check level of audio but I don't know how to use it and what level can be used to detect if user is not speaking.
I've been trying to do more or less the same thing for a while now. There is an example - https://doc.qt.io/qt-5/qtdatavisualization-audiolevels-example.html that shows you how implement an audio level meter which should be helpful. The example uses QAudioInput. Specifically, it uses QAudioInput::start(QIODevice * device) and passes a custom QIODevice to implement the audio level meter. The problem with this approach using QAudioInput is once you've got the data, it's not easy to encode it and write it out to file where as with QAudioRecorder it's simple.
Anyway ... your right QAudioProbe is your best bet if you want to record the easy way with QAudioRecorder. I adapted the Qt audio level meter example to work with QAudioProbe instead of QAudioInput/QIODevice. See - https://gist.github.com/sam-at-github/bf66e84105cc3e23e7113cca5e3b1772.
One minor issue the level meter needs QAudioFormat but QAudioRecorder only provides you with a QEncoderSettings (Should probably fix code to use the latter. I don't know why both QEncoderSettings and QAudioFormat need to exist ...). You just gotta get a QAudioDeviceInfo for the device your using then use QAudioDeviceInfo::preferredFormat().
Relaed Post: Qt: API to write raw QAudioInput data to file just like QAudioRecorder
What is an optimal and an appropriate way to save the state of a Qt GUI so I get the same state I had back when I closed the application ?
By state I mean : current indexes (for combo box ...), color palette, widgets positions... right before closing the application
You can use the QSettings class.
Simple use of QSettings class (code inspired from Qt's documentation):
In the main-window of your application code member functions that saves and restore the settings:
void MainWindow::writeSettings()
{
QSettings settings("reaffer Soft", "reafferApp");
settings.beginGroup("MainWindow");
settings.setValue("size", size());
settings.setValue("pos", pos());
settings.endGroup();
}
void MainWindow::readSettings()
{
QSettings settings("reaffer Soft", "reafferApp");
settings.beginGroup("MainWindow");
resize(settings.value("size", QSize(400, 400)).toSize());
move(settings.value("pos", QPoint(200, 200)).toPoint());
settings.endGroup();
}
Call those 2 functions from the MainWindow constructor and from the closeEvent override, like this:
MainWindow::MainWindow()
{
// code from constructor
//...
readSettings();
}
void MainWindow::closeEvent(QCloseEvent *event)
{
//optional check if the user really want to quit
// and/or if the user want to save settings
writeSettings();
event->accept();
}
The direct answer requires specific elaborated design for your code and not really a short Qt question or even the question specific to Qt. That is about C++ which is not the VM-based language that assists with serializing the state of program code to data. Having all objects serializable we can then attempt to apply certain C++/Qt classes/techniques.
This task is much easier to accomplish with languages like Java, though. And with C++/Qt you have to routinely make serialize-able / serialize / restore everything that is running in your code and still no guarantee that works as long as the context is not fully captured. This task is not easy for sure and makes sense only in specific application.
The most you can get directly from Qt is to save/restore QMainWindow and other independent widgets geometry (position/size):
saveGeometry
restoreGeometry
... and that solution is still somewhat incomplete or you may/not use QSettings for the storage.
I use QSettings for this. With routines similar to Zlatomir's.
For each window I have in the project I use a different section in QSettings and have readSettings() and writeSettings() in the source for each window.
Anything on the form that I want to persist I have to explicitly save and recall. In the case of a QComboBox it would be something like:
QSettings settings("Organisation", "MySoftware");
settings.beginGroup("WindowNumberTwo");
settings.setValue("ComboIndex", combobox->currentIndex());
// save more values here
// ...
settings.endGroup();
I don't know of a built in way to persist your window states - it has to be don't value by value.
Lets assume I am a game and I have a global int* that contains my health. A game trainer's job is to modify this value to whatever in order to achieve god mode. I've looked up tutorials on game trainers to understand how they work, and the general idea is to use a memory scanner to try and find the address of a certain value. Then modify this address by injecting a dll or whatever.
But I made a simple program with a global int* and its address changes every time I run the app, so I don't get how game trainers can hard code these addresses? Or is my example wrong?
What am I missing?
The way this is usually done is by tracing the pointer chain from a static variable up to the heap address containing the variable in question. For example:
struct CharacterStats
{
int health;
// ...
}
class Character
{
public:
CharacterStats* stats;
// ...
void hit(int damage)
{
stats->health -= damage;
if (stats->health <= 0)
die();
}
}
class Game
{
public:
Character* main_character;
vector<Character*> enemies;
// ...
}
Game* game;
void main()
{
game = new Game();
game->main_character = new Character();
game->main_character->stats = new CharacterStats;
// ...
}
In this case, if you follow mikek3332002's advice and set a breakpoint inside the Character::hit() function and nop out the subtraction, it would cause all characters, including enemies, to be invulnerable. The solution is to find the address of the "game" variable (which should reside in the data segment or a function's stack), and follow all the pointers until you find the address of the health variable.
Some tools, e.g. Cheat Engine, have functionality to automate this, and attempt to find the pointer chain by themselves. You will probably have to resort to reverse-engineering for more complicated cases, though.
Discovery of the access pointers is quite cumbersome and static memory values are difficult to adapt to different compilers or game versions.
With API hooking of malloc(), free(), etc. there is a different method than following pointers. Discovery starts with recording all dynamic memory allocations and doing memory search in parallel. The found heap memory address is then reverse matched against the recorded memory allocations. You get to know the size of the object and the offset of your value within the object. You repeat this with backtracing and get the jump-back code address of a malloc() call or a C++ constructor. With that information you can track and modify all objects which get allocated from there. You dump the objects and compare them and find a lot more interesting values. E.g. the universal elite game trainer "ugtrain" does it like this on Linux. It uses LD_PRELOAD.
Adaption works by "objdump -D"-based disassembly and just searching for the library function call with the known memory size in it.
See: http://en.wikipedia.org/wiki/Trainer_%28games%29
Ugtrain source: https://github.com/sriemer/ugtrain
The malloc() hook looks like this:
static __thread bool no_hook = false;
void *malloc (size_t size)
{
void *mem_addr;
static void *(*orig_malloc)(size_t size) = NULL;
/* handle malloc() recursion correctly */
if (no_hook)
return orig_malloc(size);
/* get the libc malloc function */
no_hook = true;
if (!orig_malloc)
*(void **) (&orig_malloc) = dlsym(RTLD_NEXT, "malloc");
mem_addr = orig_malloc(size);
/* real magic -> backtrace and send out spied information */
postprocess_malloc(size, mem_addr);
no_hook = false;
return mem_addr;
}
But if the found memory address is located within the executable or a library in memory, then ASLR is likely the cause for the dynamic. On Linux, libraries are PIC (position-independent code) and with latest distributions all executables are PIE (position-independent executables).
EDIT: never mind it seems it was just good luck, however the last 3 numbers of the pointer seem to stay the same. Perhaps this is ASLR kicking in and changing the base image address or something?
aaahhhh my bad, i was using %d for printf to print the address and not %p. After using %p the address stayed the same
#include <stdio.h>
int *something = NULL;
int main()
{
something = new int;
*something = 5;
fprintf(stdout, "Address of something: %p\nValue of something: %d\nPointer Address of something: %p", &something, *something, something);
getchar();
return 0;
}
Example for a dynamicaly allocated varible
The value I want to find is the number of lives to stop my lives from being reduced to 0 and getting game over.
Play the Game and search for the location of the lifes variable this instance.
Once found use a disassembler/debugger to watch that location for changes.
Lose a life.
The debugger should have reported the address that the decrement occurred.
Replace that instruction with no-ops
Got this pattern from the program called tsearch
A few related websites found from researching this topic:
http://deviatedhacking.com/index.php?/topic/75-dynamic-memory-allocation/
http://www.edgeofnowhere.cc/viewforum.php?f=183
http://www.oldschoolhack.de/tutorials/Theories%20and%20methods%20of%20code-caves.htm
http://webcache.googleusercontent.com/search?q=cache:4wzMzFIZx54J:gamehacking.com/forums/tutorials-beginners/11597-c-making-game-trainer.html+reading+a+dynamic+memory+address+game+trainer&cd=2&hl=en&ct=clnk&gl=au&client=firefox-a (A google cache version)
http://www.codeproject.com/KB/cpp/codecave.aspx
The way things like Gameshark codes were figured out were by dumping the memory image of the application, then doing one thing, then looking to see what changed. There might be a few things changing, but there should be patterns to look for. E.g. dump memory, shoot, dump memory, shoot again, dump memory, reload. Then look for changes and get an idea for where/how ammo is stored. For health it'll be similar, but a lot more things will be changing (since you'll be moving at the very least). It'll be easiest though to do it when minimizing the "external effects," e.g. don't try to diff memory dumps during a firefight because a lot is happening, do your diffs while standing in lava, or falling off a building, or something of that nature.
in my project, I let users pick pictures using the FileReference class. I then load these pictures into their .data properties, using the load() function. After this I perform some local manipulation and send them to the server.
What I would like to do, is to be able to iterate over the picked FileReferences again, load them into .data properties, perform different manipulation and send them to the server once again. I know that I should be able to do this from user-invoked event, that is not an issue here.
Problem is, once the FileReference is loaded for the first time, I can not unload it in any way, and I can not keep the data for all the pictures in the memory because these are huge.
So I guess there is only one thing I can do, which is performing a DeepCopy on the FileReference... Then I could load the first version, scrap it and use the copy for the second 'run'.
I tried to use ObjectUtil.copy, but when I access e.g. .name property of the copy, it fails with:
Error #2037: Functions called in incorrect sequence, or earlier call was unsuccessful.
at flash.net::FileReference/get name()
the relevant snippet:
registerClassAlias("FileReference",FileReference);
masterFileList.addItem(FileReference(ObjectUtil.copy(fr_load.fileList[i])));
trace(masterFileList[i].name)
Is it true that there are some protected properties of FileReference class that prevent it from being copied? If it is so, can I sidestep this somehow? Or is there any other solution to my overall problem?
I appreciate any hints/ideas!
I was trying to do almost exactly what you were doing, and I almost gave up after reading some of the answers, but I think I found a way to do it. I've found that if you have a FileReference object and call load() multiple times, it will work, but the main problem is that you're keeping the high-res bytes in memory after the first load. As you've mentioned, for people who don't know image processing, this is a big no-no.
The way to get around this is that after your first load(), you need to call the cancel() method on FileReference. From my testing so far, it looks like that will clear out the bytes in the FileReference, and load() will still work if you call it a second time later. Just a word of caution, this isn't explicitly-defined behavior in the API, so it is definitely subject to change, but it may help get you where you need to go in the mean time.
Hope that helps.
you cant use a ObjectUtil.copy. This method is designed for copying only data objects (VO classes).
you should create a new FileReference and copy the porperties, one by one. Create a function to do this..
Would copying it to a temporary file and then uploading the temporary file work? For example
var fileRef:FileReference = new FileReference();
fileRef.browse();
......................
var tmpFile:File = File.createTempFile();
try {
var tmpFileStream:FileStream = new FileStream();
tmpFileStream.open(tmpFile, FileMode.WRITE);
trace("Opened file: " + tmpFile.nativePath);
tmpFileStream.writeBytes(fileRef.data);
trace("copied file");
} catch ( error:Error ) {
trace("Unable to open file " + tmpFile.nativePath + "\n");
throw error;
}
I'm thinking that the operation is completely disallowed, for good reasons. If you can duplicate a new FileReference through ActionScript code, then you'd also be able to manufacture a FileReference object through ActionScript code. Of course, that'd be a pretty bad security hole if you could force the upload of an arbitrary file.
Keeping a copy of the data in memory really isn't that bad of a solution. After all, it's temporary. The typical client computer should be able to manage a few hundred extra MB of data with no problem. It's certainly a better option than having their browser do two separate uploads, which is what your attempted solution would end up doing.
A completely different potential solution to this problem is to avoid image manipulation by Flex altogether. Flex could post the uploaded file directly to the server, and the server could do the image manipulation itself. Of course, if the manipulation is driven through user interactions, then that wouldn't work at all.
I've encountered a memory problem using FileReference.save(). My Flash application generates of a lot of data in real-time and needs to save this data to a local file. As I understand, Flash 10 (as opposed to AIR) does not support streaming to a file. But, what's even worse is that FileReference.save() duplicates all the data before saving it. I was looking for a workaround to this doubled memory usage and thought about the following approach:
What if I pass a custom subclass of ByteArray as an argument to FileReference.save(), where this ByteArray subclass would override all read*() methods. The overridden read*() methods would wait for a piece of data to be generated by my application, return this piece of data and immediately remove it from the memory. I know how much data will be generated, so I could also override length/bytesAvailable methods.
Would it be possible? Could you give me some hint how to do it? I've created a subclass of ByteArray, registered an alias for it, passed an instance of this subclass to FileReference.save(), but somehow FileReference.save() seems to treat it just as it was a ByteArray instance and doesn't call any of my overridden methods...
Thanks a lot for any help!
It's not something I've tried before, but can you try sending the data out to a php application that would handle saving the ByteArray to the server, much like saving an image to the server, so then you'd use URLLoader.data instead, using something like this:
http://www.zedia.net/2008/sending-bytearray-and-variables-to-server-side-script-at-the-same-time/
It's an interesting idea. Perhaps to start you should just add traces in your extended ByteArray to see how the FileReference#save() functions internally.
If it has some kind of
while( originalByteArray.bytesAvailable )
writeToSaveBuffer( originalByteArray.readByte() );
functionality the overrides could just truncate the original buffer on every read like you say, something like:
override function readByte() : uint {
var b : uint = super.readByte();
// Truncate the bytes (assuming bytesAvailable = length - removedBytes)
length = length - bytesAvailable;
return b;
}
On the other hand, if this now works I guess the original byte array would not be available afterwards in the application anymore.
(i havn't tested this myself, truncating might require more work than the example)