Redis plus plus set is very delayed in loop - qt

I'm trying to set data to Redis Cluster in this code: (I'm using this https://github.com/sewenew/redis-plus-plus#redis-cluster)
QByteArray request = m_client.readAll();
qDebug() << "get data from" << bk[static_cast<int>(id) - 1];
QString mess = QTextCodec::codecForMib(106)->toUnicode(request);
// messss.fromUtf8(request);
QStringList l_mess = mess.split("|");
QRegularExpression key("\"key\": \"(.*?)\"");
QRegularExpressionMatch m_key;
qDebug() << "Count " << l_mess.count();
for (int i = 0; i < l_mess.count(); i++)
{
m_key = key.match(l_mess[i]);
cluster1->set(m_key.captured(1).toStdString(), l_mess[i].toStdString());
cluster1->rpush("fonbet", "i"/*m_key.captured(1).toStdString()*/);
}
And when l_mess.count() just over 100 or 200, delay time in "for" loop is over 20 seconds!!
I tried to set a static key-value, but the delay is the same. What am I doing wrong?

You are sending 200-400 commands, you are seeing 200-400 ms per command, most of that time is network latency, RTT (Round Trip Time).
To improve your performance, use a pipeline to reduce RTT.
Be sure to check the section on Pipeline and Transaction of redis-plus-plus when it comes to Cluster.

Related

Unexplained time offset in QApplication event loop

I'm currently working with a program for predicting the locations of satellites in real-time. Something similar to this. The underlying library uses system time as input.
time_t now(time(0));
This program accurately predicts the real-time position of satellites when I run it on a C++ console application using Qt Creator.
The problem is when I use it in a fully-fledged Qt Gui application with a QApplication object in the main function. In the program, the prediction function is periodically by the timer event function. That way I update the positions every 2 seconds. Unfortunately, The output doesn't match (either on the GUI or when I print it). It is like the orbital propagator is using a different time when calculating the satellite positions.
void TrackingManager::timerEvent(QTimerEvent *event)
{
int nNumSats = m_Satellites.size() ;
//std::cout << __func__ << " - Number of satellites = " << nNumSats << std::endl;
std::vector<SatPosition> vSatPositions;
if (nNumSats >= 0)
{
time_t now(time(0));
std::cout << __func__ << "time(0) = " << asctime(gmtime(&now)) << std::endl;
for(int i = 0; i < nNumSats; i++)
{
// Get satellite names and calculate position, altitude etc
SatPosition spPos;
GetInstantPredict(m_Satellites[i], now, spPos);
vSatPositions.push_back(spPos);
}
emit UpdateSatPosition(vSatPositions);
}
}
Even more confusing, the program works fine when I run the debugger (GDB on Ubuntu). It is as if GDB somehow manages to "fix" the problem. Does this make any sense?

Is this code send hex data in correct way in Qt C++?

I am new to Qt.I am working on finger print madoule with this document. I want to send my data to serial port in this format:
I wrote my code in this format, but I think my data has mistake, because this code turn on the LED in some device:
QByteArray ba;
ba.resize(24);
ba[0]=0x55;
ba[1]=0xAA;
ba[2]=0x24;
ba[3]=0x01;
ba[4]=0x01;
ba[5]=0x00;
ba[6]=0x00;
ba[7]=0x00;
ba[8]=0x00;
ba[9]=0x00;
ba[10]=0x00;
ba[11]=0x00;
ba[12]=0x00;
ba[13]=0x00;
ba[14]=0x00;
ba[15]=0x00;
ba[16]=0x00;
ba[17]=0x00;
ba[18]=0x00;
ba[19]=0x00;
ba[20]=0x00;
ba[21]=0x00;
ba[22]=0x27;
ba[23]=0x01;
p->writedata(ba);
Is this data correct?
You're just copying a drawing into code. It won't work without understanding what the drawing means. You seem to miss that:
The LEN field seems to be a little-endian integer that gives the number of bytes in the DATA field - perhaps it's the number of bytes that carry useful information if the packet has a fixed size.
The CKS field seems to be a checksum of some sort. You need to calculate it based on the contents of the packet. The protocol documentation should indicate whether it's across the entire packet or not, and how to compute the value.
It seems like you are talking to a fingerprint identification module like FPM-1502, SM-12, ADST11SD300/310 or similar. If so, then you could obtain a valid command packet as follows:
QByteArray cmdPacket(quint16 cmd, const char *data, int size) {
Q_ASSERT(size <= 16);
QByteArray result(24, '\0');
QDataStream s(&result, QIODevice::WriteOnly);
s.setByteOrder(QDataStream::LittleEndian);
s << quint16(0xAA55) << cmd << quint16(size);
s.writeRawData(data, size);
s.skipRawData(22 - s.device()->pos());
quint16 sum = 0;
for (int i = 0; i < 22; ++i)
sum += result[i];
s << sum;
qDebug() << result.toHex();
return result;
}
QByteArray cmdPacket(quint16 cmd, const QByteArray& data) {
return cmdPacket(cmd, data.data(), data.size());
}
The command packet to turn the sensor led on/off can be obtained as follows:
QByteArray cmdSensorLed(bool on) {
char data[2] = {'\0', '\0'};
if (on) data[0] = 1;
return cmdPacket(0x124, data, sizeof(data));
}

QAudioBuffer providing different data of the same song in different computers

I'm creating a music player for PC. I want to visualize the FFT of the song. I've crated an entire class that buffers 1024 points of data does the FFT and displays it (this is handled by another class). My program was developed in my laptop which uses Debian Testing x64. My work pc uses Centos 7 x64. When I compiled my program (both use Qt 5.7.0) on my work PC the FFT visualization was garbage. Snooping into my code I found that the sample type provided by QAudioBuffer from QAudioProbe was Signed (in my work PC) while it was float in my Laptop. Here is the code that is called whenever QAudioProbe emits that data has been buffered:
void SpectrumController::setAudioBuffer(QAudioBuffer buffer){
// Used to momentarily stop the process.
if (!enableBuffering) return;
// Only process stereo frames
if (buffer.format().channelCount() != 2) return;
if (buffer.format().sampleType() == QAudioFormat::SignedInt){
//qWarning() << "Signed";
QAudioBuffer::S16S *data = buffer.data<QAudioBuffer::S16S>();
bufferData(data,buffer.frameCount());
}
else if (buffer.format().sampleType() == QAudioFormat::UnSignedInt){
//qWarning() << "Unsigned";
QAudioBuffer::S16U *data = buffer.data<QAudioBuffer::S16U>();
bufferData(data,buffer.frameCount());
}
else if(buffer.format().sampleType() == QAudioFormat::Float){
//qWarning() << "Float";
QAudioBuffer::S32F *data = buffer.data<QAudioBuffer::S32F>();
bufferData(data,buffer.frameCount());
}
}
template<typename T>
void SpectrumController::bufferData(T *data, qint32 N){
for (qint32 i = 0; i < N; i++){
//if (qAbs(data[i].left) > largest){largest = qAbs(data[i].left); qDebug() << "Largest" << largest;}
//currentBuffer << ((qreal)data[i].left/(largest));
//qWarning() << "Added data" << currentBuffer.last();
currentBuffer << data[i].left;
if (datcounter < 100000){
*writer << data[i].left;
*writer << "\n";
datcounter++;
}
else if (writeFile->isOpen()){
qWarning() << "Closed file";
writeFile->close();
}
if (currentBuffer.size() == FFT_SIZE){
dataBuffer << currentBuffer;
currentBuffer.clear();
if (!isRunning) run();
}
What I ended up doing is writing, to a file, the first 100.000 points of data gathered by both my laptop and my work PC in order to plot them.
This is what I've got
What I think is that difference is in the base system's handling of the the mp3, which, in turn, is what Qt uses. I think is gstreamer. Centos uses a much older version. The plot on the right corresponds to my laptop while the plot on the left corresponds to my work pc.
Any ideas on how to fix this? Or am I just stuck with no way of accessising the raw audio data correctly?
UPDATE:
Even though this is not a Fix or anything like that, the data in the other channel (data[i].right) did have more appropiate data. I'm using the right channnel, for now.

Combine Thread Results with openmp

I have some problems combining the processing results I recieve from several Threads. And I'm not sure, if I use openmp correctly. The below code extract shows the openmp portion of my code.
Parameters:
thread private:
it: map iterator (timestamp, userkey)
ite: map iterator ((timestamp,userkey)/int amount)
thread_result_map: typedef map < userkey(str),timestamp(str) >
when, who: matching regex (timestamp, userkey)
shared among threads:
log: char array
size: log.size()
identifier, timestamp, userkey: boost::regex patterns
combined_result_map: typedef map < thread_result_map, hits(int) >
#pragma omp parallel shared(log, size, identifier, timestamp, userkey) private(it, ite, str_time, str_key, vec_str_result, i, id, str_current, when, who, thread_result_map)
{
#pragma omp for
for (i = 0 ; i < size ; i++){
str_current.push_back(log[i]);
if (log[i] == '\n') {
if (boost::regex_search(str_current, identifier)){
boost::regex_search(str_current, when, timestamp);
str_time = when[0];
boost::regex_search(str_current, who, userkey);
str_key = who[0];
thread_result_map.insert(make_pair(str_time, str_key));
}
str_current = ""; //reset temp string
}
}
#pragma omp critical
{
for (it=thread_result_map.begin(); it!=thread_result_map.end(); it++) {
id = omp_get_thread_num();
cout << thread_result_map[it->first] <<
thread_result_map[it->second];
cout << "tID_" << id << " reducing" << endl;
}
}
}
As you can see every thread has his own partition of the char array, it parses line by line from the array and if the current string is identified by "identifier", the timestamp and userkey are added to the thread's private result map (string/string).
Now after the loop I have several thread's private result maps. The combined_result_map is a map inside a map. The key is the combination of key/value of the threads result and the value is the amount of occurences of this combination.
I'm parsing only a portion of the timestamp so when in 1 hour the same userkey appears multiple times the hit counter will be increased.
The result should look something like this:
TIME(MMM/DD/HH/);USERKEY;HITS
May/25/13;SOMEKEY124345;3
So I have no problems combining hit amounts in the critical section (which I removed) by specifying combined+=results.
But how can I combine my result maps the same way? I know I have to iterate through threads maps, but when I put a "cout" inside the loop for testing every thread calls it only once.
A test run on my local syslog gives me the following output when I set all the regex to "error" (to make sure every identified line will have a userkey and a timestamp with the same name):
Pattern for parsing Access String:
error Pattern for parsing Timestamp:
error Pattern for parsing Userkey:
error
*** Parsing File /var/log/syslog
errortID_0 reducing errortID_1
reducing errortID_2 reducing
errortID_3 reducing
*** Ok! ________________ hits :
418 worktime: 0.0253871s
(The calculated hits come from thread private counters, that I removed in the code above)
So every of my 4 threads does a single cout and leaves the loop, although all together should have 418 hits. So what do I do wrong? How do I iterate through my results from inside my openmp area?
Found the problem myself, sorry for asking stupid questions.
I was trying to add the same key multiple times, that's why map size didn't increase and every thread looped only once.
Edit:
If anybody is interested in the solution how to combine thread results, this is how I did it. perhaps you see anything that could be improved.
I just changed the local threads result map to a vector of pairs(str,str).
This is the full working openmp code section. Pehaps it's useful for anyone:
#pragma omp parallel shared(log, size, identifier, timestamp, userkey) private(it, ite, str_time, str_key, i, id, str_current, when, who, local_res)
{
#pragma omp for
for (i = 0 ; i < size ; i++){
str_current.push_back(log[i]);
if (log[i] == '\n') { // if char is newline character
if (boost::regex_search(str_current, identifier)){ // if current line is access string
boost::regex_search(str_current, when, timestamp); // get timestamp from string
str_time = when[0];
boost::regex_search(str_current, who, userkey); // get userkey from string
str_key = who[0];
local_res.push_back((make_pair(str_time, str_key))); // append key-value-pair(timestamp/userkey)
id = omp_get_thread_num();
//cout << "tID_" << id << " - adding pair - my local result map size is now: " << local_res.size() << endl;
}
str_current = "";
}
}
#pragma omp critical
{
id = omp_get_thread_num();
hits += local_res.size();
cout << "tID_" << id << " had HITS: " << local_res.size() << endl;
for (i = 0; i < local_res.size(); i++) {
acc_key = local_res[i].second;
acc_time = local_res[i].first;
if(m_KeyDatesHits.count(acc_key) == 0) { // if there are no items for this key yet, make a new entry
m_KeyDatesHits.insert(make_pair(acc_key, str_int_MapType()));
}
if (m_KeyDatesHits[acc_key].count(acc_time) == 0) { // "acc_time" is a key value, if it doesn't exist yet, add it and set "1" as value
m_KeyDatesHits[acc_key].insert(make_pair(acc_time, 1 ));
it = m_KeyDatesHits.begin(); // iterator for userkeys/maps
ite = m_KeyDatesHits[acc_key].begin(); // iterator for times/clicks
} else m_KeyDatesHits[acc_key][acc_time]++; // if userkey already exist and timestamp already exists, count hits +1 for it
}
}
}
I did some tests and it's really running fast.
Using 4 Threads this searches a 150MB LogFile for access events, parses a custom user key and date from every event and combines the results in under 4 seconds.
At the End it creates a export list. This is the program output:
HELLO, welcome to LogMap 0.1!
C++/OpenMP Memory Map Parsing Engine
__________________ Number of processors available = 4
Number of threads = 4
Pattern for parsing Access String:
GET /_openbooknow/key/ Pattern for
parsing Timestamp: \d{2}/\w{3}/\d{4}
Pattern for parsing Userkey:
[a-zA-Z0-9]{20,32}
* Parsing File
/home/c0d31n/Desktop/access_log-test.txt
HITS: 169147 HITS: 169146 HITS: 169146
HITS: 169147
* Ok! ________ hits :
676586 worktime: 4.03816s
* new export file created: "./test.csv"
root#c0d3b0x:~/workspace/OpenBookMap/Release#
cat test.csv
"1nDh0gV6eE3MzK0517aE6VIU0";"28/Mar/2011";"18813"
"215VIU1wBN2O2Fmd63MVmv6QTZy";"28/Mar/2011";"6272"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"18/Mar/2011";"18816"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"21/Mar/2011";"12544"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"22/Mar/2011";"12544"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"23/Mar/2011";"18816"
"9E1608JFGk2GZQ4ppe1Grtv2";"28/Mar/2011";"12544"
"pachCsiog05bpK0kDA3K2lhEY";"17/Mar/2011";"18029"
"pachCsiog05bpK0kDA3K2lhEY";"18/Mar/2011";"12544"
"pachCsiog05bpK0kDA3K2lhEY";"21/Mar/2011";"18816"
"pachCsiog05bpK0kDA3K2lhEY";"22/Mar/2011";"6272"
"pachCsiog05bpK0kDA3K2lhEY";"23/Mar/2011";"18816"
"pachCsiog05bpK0kDA3K2lhEY";"28/Mar/2011";"501760"
"1nDh0gV6eE3MzK0517aE6VIU0";"28/Mar/2011";"18813"
"215VIU1wBN2O2Fmd63MVmv6QTZy";"28/Mar/2011";"6272"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"18/Mar/2011";"18816"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"21/Mar/2011";"12544"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"22/Mar/2011";"12544"
"36Pu0A2Wly3uYeIPZ4YPAuBy";"23/Mar/2011";"18816"
"9E1608JFGk2GZQ4ppe1Grtv2";"28/Mar/2011";"12544"
"pachCsiog05bpK0kDA3K2lhEY";"17/Mar/2011";"18029"
"pachCsiog05bpK0kDA3K2lhEY";"18/Mar/2011";"12544"
"pachCsiog05bpK0kDA3K2lhEY";"21/Mar/2011";"18816"
"pachCsiog05bpK0kDA3K2lhEY";"22/Mar/2011";"6272"
"pachCsiog05bpK0kDA3K2lhEY";"23/Mar/2011";"18816"
"pachCsiog05bpK0kDA3K2lhEY";"28/Mar/2011";"501760"

qt - qprocess start in a loop

I am calling a process in a loop. Need to ensure that one process ends before it starts again. how is it possible.
void MainWindow::setNewProjectInQueueList()
{
// this is already gotten in queueList now loop thru the list and add project
QStringList arguments;
projNm = ui->lineEditCreateProject->text();
qDebug() << " projNm " << projNm;
for (int j= 0; j < queueList.length(); j++)
{ if (! QString(queueList[j]).isEmpty())
{
// call process
// QString queueName = queueList[j];
arguments << "-sq" << queueList[j];
qDebug() << " arguments sq " << queueList[j];
procQueueList.start("qconf",arguments);
}
}
// and append for each queue with new project name
// and store into the system
}
Brgds,
kNish
Call QProcess::waitForFinished() to wait until the process terminates.
Using the waitForFinished approach from within a loop in the main thread will freeze the application. Instead, putting the loop in a separate thread, or making a queue of processes to start and then launch upon the finished signal from the previous one is good alternatives.

Resources