If I want to share something like a char **keys array between fork()'d processes using shm_open and mmap can I just stick a pointer to keys into a shared memory segment or do I have to copy all the data in keys into the shared memory segment?
All data you want to share has to be in the shared segment. This means that both the pointers and the strings have to be in the shared memory.
Sharing something which includes pointers can be cumbersome. This is because mmap doesn't guarantee that a given mapping will end up in the required address.
You can still do this, by two methods. First, you can try your luck with mmap and hope that the dynamic linker doesn't load something at your preferred address.
Second method is to use relative pointers. Inside a pointer, instead of storing a pointer to a string, you store the difference between the address of the pointer and the address of the string. Like so:
char **keys= mmap(NULL, ...);
char *keydata= (char*) keys + npointers * sizeof(char*);
strcpy(keydata, firstring);
keys[0]= (char*) (keydata - (char*) &keys[0]);
keydata+= strlen(firststring)+1;
When you want to access the string from the other process, you do the reverse:
char **keys= mmap(NULL, ...);
char *str= (char*) (&keys[0]) + (ptrdiff_t) keys[0];
It's a little cumbersome but it works regardless of what mmap returns.
Related
I've taken this picture and code from The Rust Book.
Why does s point to s1 rather than just the data on the heap itself?
If so this is how it works? How does the s point to s1. Is it allocated memory with a ptr field that contains the memory address of s1. Then, does s1, in turn point to the data.
In s1, I appear to be looking at a variable with a pointer, length, and capacity. Is only the ptr field the actual pointer here?
This is my first systems level language, so I don't think comparisons to C/C++ will help me grok this. I think part of the problem is that I don't quite understand what exactly pointers are and how the OS allocates/deallocates memory.
fn main() {
let s1 = String::from("hello");
let len = calculate_length(&s1);
println!("The length of '{}' is {}.", s1, len);
}
fn calculate_length(s: &String) -> usize {
s.len()
}
The memory is just a huge array, which can be indexed by any offset (e.g. u64).
This offset is called address,
and a variable that stores an address called a pointer.
However, usually only some small part of memory is allocated, so not every address is meaningful (or valid).
Allocation is a request to make a (sequential) range of addresses meaningful to the program (so it can access/modify).
Every object (and by object I mean any type) is located in allocated memory (because non-allocated memory is meaningless to the program).
Reference is actually a pointer that is guaranteed (by a compiler) to be valid (i.e. derived from address of some object known to a compiler). Take a look at std doc also.
Here an example of these concepts (playground):
// This is, in real program, implicitly defined,
// but for the sake of example made explicit.
// If you want to play around with the example,
// don't forget to replace `usize::max_value()`
// with a smaller value.
let memory = [uninitialized::<u8>(); usize::max_value()];
// Every value of `usize` type is valid address.
const SOME_ADDR: usize = 1234usize;
// Any address can be safely binded to a pointer,
// which *may* point to both valid and invalid memory.
let ptr: *const u8 = transmute(SOME_ADDR);
// You find an offset in our memory knowing an address
let other_ptr: *const u8 = memory.as_ptr().add(SOME_ADDR);
// Oversimplified allocation, in real-life OS gives a block of memory.
unsafe { *other_ptr = 15; }
// Now it's *meaningful* (i.e. there's no undefined behavior) to make a reference.
let refr: &u8 = unsafe { &*other_ptr };
I hope that clarify most things out, but let's cover the questions explicitly though.
Why does s point to s1 rather than just the data on the heap itself?
s is a reference (i.e. valid pointer), so it points to the address of s1. It might (and probably would) be optimized by a compiler for being the same piece of memory as s1, logically it still remains a different object that points to s1.
How does the s point to s1. Is it allocated memory with a ptr field that contains the memory address of s1.
The chain of "pointing" still persists, so calling s.len() internally converted to s.deref().len, and accessing some byte of the string array converted to s.deref().ptr.add(index).deref().
There are 3 blocks of memory that are displayed on the picture: &s, &s1, s1.ptr are different (unless optimized) memory addresses. And all of them are stored in the allocated memory. The first two are actually stored at pre-allocated (i.e. before calling main function) memory called stack and usually it is not called an allocated memory (the practice I ignored in this answer though). The s1.ptr pointer, in contrast, points to the memory that was allocated explicitly by a user program (i.e. after entering main).
In s1, I appear to be looking at a variable with a pointer, length, and capacity. Is only the ptr field the actual pointer here?
Yes, exactly. Length and capacity are just common unsigned integers.
I have defined (globally) a large array of strings thus:
const String opCodes[256]PROGMEM = {""...""}; // all 256 defined
However, building this now consumes 20% more RAM than it did before I added the array.
This was unexpected. Why did it happen? Thanks
The Arduino String object is a dynamic string much like std::string. And as such stores its data in dynamically allocated memory in RAM.
If you want to store the actual string data itself in PROGMEM then the Arduino PROGMEM reference will tell you how do do it using actual arrays of characters instead. In short, create arrays of characters stored in PROGMEM, and then make an array of const char * (also in PROGMEM) pointing to the strings.
In the end I decided not to use PROGMEM as its use seems a bit suspect.
A usable workaround was to use the F() function instead. This works.
How do you declare a pointer on a 16 bit Renesas RL78 microcontroller using IAR's EWB RL78 compiler to a register which has a 20 bit address?
Ex:
static int *ptr = (int *)0xF1000;
The above does not work because pointers are 16 bit addresses.
If the register in question is an on-chip peripheral, then it is likely that your toolchain already includes a processor header with all registers declared, in which case you should use that. If for some reason you cannot or do not wish to do that, then you could at least look at that to see how it declares such registers.
In any event you should at least declare the address volatile since it is not a regular memory location and may change beyond the control and knowledge of your code as part of the normal peripheral behaviour. Moreover you should use explicit sized data types and it is unlikely that this register is signed.
#include <stdint.h>
...
static volatile uint16_t* ptr = (uint16_t*)0xF1000u ;
Added following clarification of target architecture:
The IAR RL78 compiler supports two data models - near and far. From the IAR compiler manual:
● The Near data model can access data in the highest 64 Kbytes of data
memory
● The Far data model can address data in the entire 1 Mbytes of
data memory.
The near model is the default. The far model may be set using the compiler option: --data_model=far; this will globally change the pointer type to allow 20 bit addressing (pointers are 3 bytes long in this case).
Even without specifying the data model globally it is possible to override the default pointer type by explicitly specifying the pointer type using the keywords __near and __far. So in the example in the question the correct declaration would be:
static volatile uint16_t __far* ptr = (uint16_t*)0xF1000u ;
Note the position of the __far keyword is critical. Its position can be used to declare a pointer to far memory, or a pointer in far memory (or you can even declare both to and in far memory).
On an RL78, 0xF1000 in fact refers to the start of data flash rather then a register as stated in the question. Typically a pointer to a register would not be subject to alteration (which would mean it referred to a different register), so might reasonably be declared const:
static volatile uint16_t __far* const ptr = (uint16_t*)0xF1000u ;
Similarly to __far the position of const is critical to the semantics. The above prevents ptr from being modified but allows what ptr refers to to be modified. Being flash memory, this may not always be desirable or possible, so it is possible that it could reasonably be declared a const pointer to a const value.
Note that for RL78 Special Function Registers (SFR) the IAR compiler has a keyword __sfr specifically for addressing registers in the area 0xFFF00-0xFFFFF:
Example:
#pragma location=0xFFF20
__no_init volatile uint8_t __sfr PORT1; // PORT1 is located at address 0xFFF20
Alternative syntax using IAR specfic compiler extension:
__no_init volatile uint8_t __sfr PORT1 # 0xFFF20 ;
I have some code that uses nested Structs to store device parameters see below:
This is using an Ardunio Zero ( Atmel SAMD21)
The declares Storeage with up to 3 networks each network with 64 devices.
I would like to use 5 networks however when I increase the networks to 4 the code will not compile.
I get region RAM overflowed with stack / RAM overflowed by 4432 bytes.
I understand that this is taking more ram then I have? I am looking to see if there is a solution using a different method to achieve the same thing but get it to fit?
struct device {
int stat;
bool changed;
char data[51];
char state[51];
char atime[14];
char btime[14];
};
struct outputs {
device fitting[64];
};
struct storage {
int deviceid =0;
int addstore =0;
bool set;
bool run_events = false;
char authkey[10];
outputs network[3];
} ;
storage data_store;
Well, the usual approches are:
Consider if all or any of the data is actually read-only, and thus can be made const (which should move it to read-only memory, if that fails you can usually force it by adding compiler-specific magic).
Figure out means of representing the data using fewer bits. For instance using 14 bytes for each of three timestamps might seem excessive; switching these to 32-bit timestamps and generating the strings when needed would save around 70%.
If there are duplicates, then perhaps each storage doesn't need three unique outputs, but can instead store pointers into a shared "pool" of unique configurations.
If not all 64 fittings are used, that array could also be refactored into having non-constant length.
It's hard to be more specific since I don't know your data or application well enough.
Your struct is taking too much place. That's all. Assuming chars, ints and bools are internally 1 byte each, your device struct takes 132 bytes. Then, your outputs struct takes 8448 bytes or 8.25Kb. Your unit has 32Kb of RAM...
I'm having some trouble copying pointers' contents. I'm simply trying this:
char* vigia1;
char* vigia2;
And..
char* aux = (char*) malloc (strlen (vigia1)+1);
aux=vigia1;
vigia1=vigia2;
vigia2=aux;
free (aux);
vigia1, vigia2 are pointers to a char pointer. They both have a malloc greater than their maximum possible size, that's OK.
Since I'm trying to make an order for a list, I need to make this change to order the nodes' content. But I'm getting confused: after the free(aux) , vigia2 doesn't have any value. I think I must be pointing vigia2 to the memory region where aux is, region that 'disappear' after the free. So what should I do?
Thanks!
Pointers, pointers, bad with them, worse without them
A pointer is a number that stores where in memory sth is stored, with that in mind, let's delve into what you've done there:
char* aux = (char*) malloc (strlen (vigia1)+1);
Good, you've created space somewhere in a part of the memory called heap, and stored the address of the newly created memory space at aux.
aux=vigia1;
Ops, now you've overwritten the address of the memory space you've "created" with the number stored at vigia1, that happens to be an address to another memory space.
vigia1=vigia2;
Now you're assinging to vigia1 the value of vigia2, another address of some memory space out there.
vigia2=aux;
And, by the end of it, you make vigia2 point to the memory region previously pointed by vigia1.
free (aux);
Now, you're freeing the memory pointed by aux. Wait a second, on the line above this one you've just made vigia2 point to this same address. No wonder it holds nothing useful :)
Trying to help you with what you want to do:
So long you don't have any constraint that obliges you to mantain your list nodes ordered in memory, you don't need to copy the content of the node, just make the pointer of the first node point to the memory region of the second node.
A perfect swap would be:
char *aux; // you'll need an aux to make the swap, the normal stuff
aux = vigia1; // now aux points to the same address as vigia1
vigia1 = vigia2; // vigia1 now points to the contents of vigia2
vigia2 = aux; // and now vigia2 points to the content pointed previously by vigia1
/* and tada! the swap is done :D */
Assigning one pointer to another simply copies the value of one pointer to another, i.e., an address. It doesn't copy what the pointer refers to into another location at a different address. So yes, you can have N pointers all pointing to the same chunk of memory, but once free() is called one of them they are all invalid.
So that means that this:
char* aux = (char*) malloc (strlen (vigia1)+1);
aux=vigia1;
Is a memory leak. You malloc'd some memory for aux and then immediately discarded the address. There is no way to get it back, no way to free() it any longer.
What you are making is just pointer assignments. Malloc'ed memory is just getting wasted causing a leak.
aux=vigia1; // Makes aux point to the location where vigia1 is pointing to
// Doesn't copy the contents of vigia1 to malloc'ed memory for aux
You need to make deep copy using strcpy.
strcpy(aux, vigia1);
Hope this gives you the hint.