I've just started switching to QRegularExpression, and I'm using it to tokenize a string with multiple delimiter possibilities. I've encountered a surprising behavior, which seems to me to be a bug. I'm using Qt 5.5.1 on Windows.
Here's sample code:
#include <QRegularExpression>
#include <QString>
#include <QtDebug>
int main(int argc, char *argv[])
{
Q_UNUSED (argc);
Q_UNUSED (argv);
QRegularExpression regex ("^ ");
qDebug () << "Expected: " << QString ("M 100").indexOf(regex);
qDebug () << "NOT expected:" << QString ("M 100").indexOf(regex, 1);
qDebug () << "Expected: " << QString (" 100").indexOf(regex);
QRegularExpression regex1 (" ");
qDebug () << "Expected: " << QString ("M 100").indexOf(regex1);
}
And the output:
Expected: -1
NOT expected: -1
Expected: 0
Expected: 1
The use of the caret (^) when used with a starting position other than 0 in the "indexOf" call is preventing the expression from matching. Intuitively, I expected that the caret matches the string at the position that I specified. Instead, it simply never matches.
I'm going to switch my tokenizing to use splitRref to avoid this problem. While that's probably slightly cleaner anyway, I need to understand whether this is correct behavior or if I should be reporting a bug to Qt.
UPDATE: Using splitRef doesn't entirely solve my problem because I need to use a regular expression to detect if some tokens are floating point numbers, and I can't use a QRegularExpression with QStringRef. For that possibility, I have to convert my QStringRef token into an actual QString, which was what I was trying to avoid in the first place.
^ matches at the beginning of the subject string, or after a newline when in multiline mode. The offset does not alter these semantics. Hence, matching /^ / (in regex notation) against M 100 at offset 1 correctly results in no match.
Perhaps you want \G? From pcrepattern(3):
\G matches at the first matching position in the subject
The \G assertion is true only when the current matching position is at the start point of the match, as specified by the startoffset argument of pcre_exec(). It differs from \A when the value of startoffset is non-zero.
With that, this code:
QRegularExpression regex ("\\G ");
qDebug () << "Expected: " << QString ("M 100").indexOf(regex);
qDebug () << "NOT expected:" << QString ("M 100").indexOf(regex, 1);
qDebug () << "Expected: " << QString (" 100").indexOf(regex);
prints
Expected: -1
NOT expected: 1
Expected: 0
I'm writing a unix cp program, but I'm unclear about checking for EOF. The code I have is:
int main(int argc, const char * argv[]) {
int in, out;
char buf[BUFFER_SIZE];
if (argc != 3)
cout << "Error: incorrect number of params" << endl;
if ((in = open(argv[1], O_RDONLY, 0666)) == -1)
cout << "Error: cannot open input file" << endl;
if ((out = open(argv[2], O_WRONLY | O_CREAT, 0666)) == -1)
cout << "Cannot create output file" << endl;
else
while ((read(in, buf, BUFFER_SIZE)) != -1)
write(out, buf, BUFFER_SIZE);
return 0;
}
It reads and writes fine, but writes past EOF when writing the output file. So I get a couple lines of gibberish past the end of the file. Am I just not checking for EOF correctly? I appreciate the input.
You should read the man page for the read function.
On end-of-file, read returns 0. It returns -1 only if there's an error.
read can read fewer bytes than you asked to (and it must do so if there aren't that many bytes remaining to be read). Your write call assumes that read actually read BUFFER_SIZE bytes.
You need to save the result returned by read and write only that many bytes -- and you need to terminate the loop when read returns 0 (indicating end-of-file) or -1 (indicating an error). In the latter case, you should probably do something to handle the error, or at least inform the user.
Incidentally, you don't need the 0666 mode argument when calling open to open the file for reading; that applies only with O_CREAT. Since open is actually a variadic function (like printf), you don't have to supply all the arguments.
The man page is not clear on this point; it pretends that there are two different forms of the open function:
int open(const char *pathname, int flags);
int open(const char *pathname, int flags, mode_t mode);
but in fact that's not legal in C. The POSIX description correctly shows the declaration as:
int open(const char *path, int oflag, ...);
I've looked around a little bit but couldn't find an answer to this.
I have a function returning a pair of pointers to objects, the situation can be simplified to:
#include <iostream>
#include <utility>
#include <memory>
std::pair<int *, int *> shallow_copy()
{
int *i = new int;
int *j = new int;
*i = 5;
*j = 7;
return std::make_pair(i, j);
}
int main(int argc, char *argv[])
{
std::pair<int *, int *> my_pair = shallow_copy();
std::cout << "a = " << my_pair.first << " b = " << *my_pair.second << std::endl;
// This is just creating a newpointer:
std::unique_ptr<int> up(my_pair.first);
std::cout << "a = " << &up << std::endl;
delete my_pair.first;
delete my_pair.second;
return 0;
}
I cannot change the return value of the function. From std::cout << "a = " << &up << std::endl; I can see that the address of the smart pointer is different from the address of the raw pointer.
Is there a way to capture tha std::pair returned by the function in a std::unique_ptr and prevent memory leaks without calling delete explicitly?
NB: The question have been edited to better state the problem and make me look smarter!
You're doing it the right way, but testing it the wrong one. You're comparing the address in first with the address of up. If you print up.get() instead (the address stored in up), you'll find they're equal.
In addition, your code has a double-delete problem. You do delete my_pair.first;, which deallocates the memory block pointed to by my_pair.first and also by up. Then, the destructor of up will deallocate it again when up goes out of scope, resulting in a double delete.
You also asked how to capture both pointers in smart pointers. Since the constructor of std::unique_ptr taking a raw pointer is explicit, you cannot directly do this with a simple std::pair<std::unique_ptr<int>, std::unique_ptr<int>>. You can use a helper function, though:
std::pair<std::unique_ptr<int>, std::unique_ptr<int>> wrapped_shallow_copy()
{
auto orig = shallow_copy();
std::pair<std::unique_ptr<int>, std::unique_ptr<int>> result;
result.first.reset(orig.first);
result.second.reset(orig.second);
return result;
}
Now, use wrapped_shallow_copy() instead of shallow_copy() and you will never leak memory from the call.
I started to learn coding backwards: high level first. This has the obvious liability of missing some basic concepts that I should definitely know, and when I try to learn a low level language, it throws me.
I have tried many times to understand pointers, however the explanations rapidly go over my head, usually because all of the example code uses languages that use pointers, which I don't understand other things about, and then I spin.
I am the most (and very at that) fluent in Javascript.
How would you explain pointers to a sad Javascript developer like me? Could someone provide me a practical, real life example?
Maybe even showing how, if Javascript had pointers, you could do x, and a pointer is different than a raw variable because of y.
Here's an attempt at a self-contained answer from first principles.
Pointers are part of a type system that permit the implementation of reference semantics. Here's how. We suppose that our language has a type system, by which every variable is of a certain type. C is a good example, but many languages work like this. So we can have a bunch of variables:
int a = 10;
int b = 25;
Further, we assume that function arguments are always copied from the caller scope into the function scope. (This is also true for many real languages, though the details can quickly become subtle when the type system gets 'hidden' from the user (e.g. such as in Java)). So let's have a function:
void foo(int x, int y);
When calling foo(a, b), the variables a and b are copied into local variables x and y corresponding to the formal parameters, and those copies are visible within the function scope. Whatever the function does with x and y has no effect on the variables a and b at the call site. The entire function call is opaque to the caller.
Now let's move on to pointers. A language with pointers contains, for every object type T, a related type T *, which is the type "pointer to T". Values of type T * are produced by taking the address of an existing object of type T. So a language that has pointers also needs to have a way to produce pointers, which is "taking the address of something". The purpose of a pointer is to store the address of an object.
But that's only one half of the picture. The other half is what to do with the address of an object. The main reason for caring about the address of an object is to be able to refer to the object whose address is being stored. This object is obtained by a second operation, suitably called dereferencing, which when applied to a pointer produces the object which is being "pointed to". Importantly, we do not a copy of the object, but we get the actual object.
In C, the address-of operator is spelled &, and the dereference operator is spelled *.
int * p = &a; // p stores the address of 'a'
*p = 12; // now a == 12
The first operand of the final assignment, *p, is the object a itself. Both a and *p are the same object.
Now why is this useful? Because we can pass pointers to functions to allow functions to change things outside the function's own scope. Pointers allow for indirection, and thus for referencing. You can tell the function about "something else". Here's the standard example:
void swap(int * p, int * q)
{
int tmp = *p;
*p = *q;
*q = tmp;
}
We can tell the function swap about our variables a and b by giving it the addresses of those variables:
swap(&a, &b);
In this way, we are using pointers to implement reference semantics for the function swap. The function gets to refer to variables elsewhere and can modify them.
The fundamental mechanism of reference semantics can thus be summarized thus:
The caller takes the address of the object to be refered to:
T a;
mangle_me(&a);
The callee takes a pointer parameter and dereferneces the pointer to access the refered value.
void mangle_me(T * p)
{
// use *p
}
Reference semantics are important for may aspects of programming, and many programming languages supply them in some way or another. For example, C++ adds native reference support to the language, largely removing the needs for pointers. Go uses explicit pointers, but offers some notational "convenience" by sometimes automagically dereferencing a pointer. Java and Python "hide" pointer-ness inside their type system, e.g. the type of a variable is in some sense a pointer to the type of the object. In some languages, some types like ints are naked value types, and others (like lists and dictionaries) are "hidden-pointer-included" reference types. Your milage may vary.
C++ rules are fairly simple and consistent. I actually find how Javascript handles object references and prototypes way more unintuitive.
Preface A: Why is Javascript A Bad Place To Start?
The first thing you need to fundamentally understand before you can tackle pointers is variables. You need to know what they are and how the computer keeps track of them.
Coming from a Javascript background you are used to every variable assigned to an object being a reference. That is, two variables can reference the same object. This is essentially pointers without any syntax to allow for more intricate use. You are also used to implicit copies of "basic" types like numbers. That is to say:
var a = MyObject;
var b = a;
Now if you change b you also change a. You would need to explicitly copy MyObject in order to have two variables pointing to different instances of it!
var a = 5;
var b = a;
Now if you change b, a is not actually changed. This is because assigning a to b when a is a simple type will copy it automatically for you. You cannot get the same behavior as objects with simple numbers and vise versa, so when you want two variables to refer to the same number you have to wrap it in an object. There is no explicit way to indicate how you want to handle references vs copies for primitive types.
You can see this inconsistent behavior with no variation on syntax (but an extreme variation on behavior) can make the relationship between variables and what they contain muddy. For this reason I highly suggest banishing this mental model for a moment as we continue on our journey to understand explicit pointers.
Preface B: YOLO: Variable Lifetime On The Stack
So, let's talk from here on out in C++ terms. C++ is one of the most explicit languages in terms of what a variable is vs a pointer. C++ is a good entry point because it is low level enough to talk in terms of memory and lifespan, but high level enough to understand things at a decent level of abstraction.
So, in C++ when you create any variable it exists in a certain scope. There are two ways to create a variable, on the stack, and on the heap.
The stack refers to the call stack of your application. Every brace pair pushes a new context onto the stack (and pops it when it runs out). When you create a local variable, it exists in that particular stack frame, when that stack frame is popped the variable is destroyed.
A simple example of scope:
#include <iostream>
#include <string>
struct ScopeTest{
ScopeTest(std::string a_name):
name(a_name){
std::cout << "Create " << name << std::endl;
}
~ScopeTest(){
std::cout << "Destroy " << name << std::endl;
}
ScopeTest(ScopeTest &a_copied){
std::cout << "Copy " << a_copied.name << std::endl;
name = a_copied.name + "(copy)";
a_copied.name += "(original)";
}
std::string name;
};
ScopeTest getVariable(){ //Stack frame push
ScopeTest c("c"); //Create c
return c; //Copy c + Destroy c(original)
}
int main(){
ScopeTest a("a"); //Create a
{
ScopeTest b("b"); //Create b
ScopeTest d = getVariable();
} //Destroy c(copy) + Destroy b
} //Destroy a
Output:
Create a
Create b
Create c
Copy c
Destroy c(original)
Destroy c(copy)
Destroy b
Destroy a
This should illustrate explicitly how a variable ties its life to the stack, how it is copied around, and when it dies.
Preface C: YOLO Variable Lifetime on the Heap
So, that's interesting conceptually, but variables can also be allocated outside of the stack, this is called "heap" memory because it is largely structure-less. The issue with heap memory is that you don't really have automatic cleanup based on scope. So you need a way to tie it to some kind of "handle" to keep track of it.
I'll illustrate here:
{
new ScopeTest("a"); //Create a
} //Whoa, we haven't destroyed it! Now we are leaking memory!
So, clearly we can't just say "new X" without keeping track of it. The memory gets allocated, but doesn't tie itself to a lifespan so it lives forever (like a memory vampire!)
In Javascript you can just tie it to a variable and the object dies when the last reference to it dies. Later I'll talk about a more advanced topic in C++ which allows for that, but for now let's look at simple pointers.
In C++ when you allocate a variable with new, the best way to track it is to assign it to a pointer.
Preface D: Pointers and The Heap
As I suggested, we can track allocated memory on the heap with a pointer. Our previous leaky program can be fixed like so:
{
ScopeTest *a = new ScopeTest("a"); //Create a
delete a; //Destroy a
}
ScopeTest *a; creates a pointer, and assigning it to a new ScopeTest("a") gives us a handle we can actually use to clean up and refer to the variable which exists in heap memory. I know heap memory sounds kinda confusing, but it's basically a jumble of memory that you can point to and say "hey you, I want a variable with no lifespan, make one and let me point at it".
Any variable created with the new keyword must be followed by exactly 1 (and no more than 1) delete or it will live forever, using up memory. If you try to delete any memory address other than 0 (which is a no-op) more than one time, you could be deleting memory not under your program's control which results in undefined behavior.
ScopeTest *a; declares a pointer. From here on out, any time you say "a" you are referring to a specific memory address. *a will refer to the actual object at that memory address, and you can access properties of it (*a).name. a-> in C++ is a special operator that does the same thing as (*a).
{
ScopeTest *a = new ScopeTest("a"); //Create a
std::cout << a << ": " << (*a).name << ", " << a->name << std::endl;
delete a; //Destroy a
}
Output for the above will look something like:
007FB430: a, a
Where 007FB430 is a hex representation of a memory address.
So in the purest sense, a pointer is literally a memory address and the ability to treat that address as a variable.
The Relationship Between Pointers and Variables
We don't just have to use pointers with heap allocated memory though! We can assign a pointer to any memory, even memory living on the stack. Just be careful your pointer doesn't out-live the memory it points to or you'll have a dangling pointer which could do bad things if you continue to try and use it.
It is always the programmer's job to make sure a pointer is valid, there are literally 0 checks in place in C++ to help you out when dealing with bare memory.
int a = 5; //variable named a has a value of 5.
int *pA = &a; //pointer named pA is now referencing the memory address of a (we reference "a" with & to get the address).
Now pA refers to the same value as &a, that is to say, it is the address of a.
*pA refers to the same value as a.
You can treat *pA = 6; the same as a = 6. Observe (continuing from the above two lines of code):
std::cout << *pA << ", " << a << std::endl; //output 5, 5
a = 6;
std::cout << *pA << ", " << a << std::endl; //output 6, 6
*pA = 7;
std::cout << *pA << ", " << a << std::endl; //output 7, 7
You can see why *pA is called a "pointer". It is literally pointing to the same address in memory as a. So far we have been using *pA to de-reference the pointer and access the value at the address it points to.
Pointers have a few interesting properties. One of those properties is that it can change the object it is pointing at.
int b = 20;
pA = &b;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 20, 7, 20
*pA = 25;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 25, 7, 25
pA = &a;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 7, 7, 25
*pA = 8;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 8, 8, 25
b = 30;
pA = &b;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 30, 8, 30
So you can see that a pointer is really just a handle to a point in memory. This can be exceptionally useful in many cases, do not write it off just because this sample is simplistic.
Now, the next thing you need to know about pointers is that you can increment them as long as the memory you are incrementing to belongs to your program. The most common example is C strings. In modern C++ strings are stored in a container called std::string, use that, but I will use an old C style string to demonstrate array access with a pointer.
Pay close attention to ++letter. What this does is increment the memory address the pointer is looking at by the size of the type it is pointing to.
Let's break this down a bit more, re-read the above sentence a few times then continue on.
If I have a type that is sizeof(T) == 4, every ++myPointerValue will shift 4 spaces in memory to point to the next "value" of that type. This is part of why the pointer "type" matters.
char text[] { 'H', 'e', 'l', 'l', 'o', '\0' }; //could be char text[] = "Hello"; but I want to show the \0 explicitly
char* letter = text;
for (char* letter = &text[0]; *letter != '\0';++letter){
std::cout << "[" << *letter << "]";
}
std::cout << std::endl;
The above will loop over the string as long as there is no '\0' (null) character. Keep in mind this can be dangerous and is a common source of insecurity in programs. Assuming your array is terminated by some value, but then getting an array that overflows allowing you to read arbitrary memory. That's a high level description anyway.
For that reason it is much better to be explicit with string length and use safer methods such as std::string in regular use.
Alright, and as a final example to put things into context. Let's say I have several discreet "cells" that I want to link together into one coherent "list". The most natural implementation of this with non-contiguous memory is to use pointers to direct each node to the next one in the sequence.
With pointers you can create all sorts of complex data structures, trees, lists, and more!
struct Node {
int value = 0;
Node* previous = nullptr;
Node* next = nullptr;
};
struct List {
List(){
head = new Node();
tail = head;
}
~List(){
std::cout << "Destructor: " << std::endl;
Node* current = head;
while (current != nullptr){
Node* next = current->next;
std::cout << "Deleting: " << current->value << std::endl;
delete current;
current = next;
}
}
void Append(int value){
Node* previous = tail;
tail = new Node();
tail->value = value;
tail->previous = previous;
previous->next = tail;
}
void Print(){
std::cout << "Printing the List:" << std::endl;
Node* current = head;
for (Node* current = head; current != nullptr;current = current->next){
std::cout << current->value << std::endl;
}
}
Node* tail;
Node* head;
};
And putting it to use:
List sampleList;
sampleList.Append(5);
sampleList.Append(6);
sampleList.Append(7);
sampleList.Append(8);
sampleList.Print();
List may seem complicated at a glance, but I am not introducing any new concepts here. This is exactly the same things I covered above, just implemented with a purpose.
Homework for you to completely understand pointers would be to provide two methods in List:
Node* NodeForIndex(int index)
void InsertNodeAtIndex(int index, int value)
This list implementation is exceptionally poor. std::list is a much better example, but it most cases due to data locality you really want to stick with std::vector. Pointers are exceptionally powerful tools, and fundamental in computer science. You need to understand them to appreciate how the common data types you rely on every day are composed, and in time you will come to appreciate the explicit separation of value from pointer in C++.
Beyond simple pointers: std::shared_ptr
std::shared_ptr gives C++ the ability to deal with reference counted pointers. That is to say, it gives a similar behavior to Javascript object assignment (where an object is destroyed when the last reference to that object is set to null or destroyed).
std::shared_ptr is just like any other stack based variable. It ties its lifetime to the stack, and then holds a pointer to memory allocated on the heap. In this regard, it encapsulates the concept of a pointer in a safer manner than having to remember to delete.
Let's re-visit our earlier example that did leak memory:
{
new ScopeTest("a"); //Create a
} //Whoa, we haven't destroyed it! Now we are leaking memory!
With a shared_ptr we can do the following:
{
std::shared_ptr<ScopeTest> a(new ScopeTest("a")); //Create a
}//Destroy a
And, a little more complex:
{
std::shared_ptr<ScopeTest> showingSharedOwnership;
{
std::shared_ptr<ScopeTest> a(new ScopeTest("a")); //"Create a" (ref count 1)
showingSharedOwnership = a; //increments a's ref count by 1. (now 2)
} //the shared_ptr named a is destroyed, decrements ref count by 1. (now 1)
} //"Destroy a" showingSharedOwnership dies and decrements the ref count by 1. (now 0)
I won't go too much further here, but this should open your mind to pointers.
hi a have a function that reads from a text file line by line each line I do some operations on it substitute a string..etc
then I push_back that line into a vector
this is my class in Parser.h
class Parser
{// start class
public:
vector<const char*> patterns;
Parser();
~Parser();
void RuleParser(const char *TextFileName); // this is the function that takes the file name
private:
};// end class
segment from function RuleParser
std::ifstream ifs(TextFileName);
while (!ifs.eof())
{
.
.modification code
.
patterns.push_back((buildString).c_str()); //buildString is the modified line
cout << buildString << endl;
}
but when I try to check out if the data in the vector is correct it output totally different data.
I even put a cout after the push_back to check it's integrty but I found buildString is correct... thats the data each time being pushed ... what I am doing wrong.
here is the loop I use to see if my data correct.
for (int i = 0;i < patterns.size() ;i++)
{
cout << patterns.at(i) << endl;
}
Well patterns is the collection of pointers so you end up push_back'ing a pointer to the same buildString in each iteration of the loop, instead of push_back'ing the string contents. Then when buildString changes in next iteration of the loop, the pointer becomes invalid but it still remains in patterns - not good
I suggest you declare patterns as:
vector<std::string> patterns;
This way when you do:
patterns.push_back(buildString.c_str())
the contents of the string will be copied instead of the pointer, and remain valid througout.