I'm doing research about xen migration. And I see the operation xc_shadow_control in code to get dirty map, as the following code:
xc_shadow_control(xch, dom, XEN_DOMCTL_SHADOW_OP_PEEK, HYPERCALL_BUFFER(to_skip),
dinfo->p2m_size, NULL, 0, NULL)
the declaration of this method as following:
int xc_shadow_control(xc_interface *xch,
uint32_t domid,
unsigned int sop,
xc_hypercall_buffer_t *dirty_bitmap,
unsigned long pages,
unsigned long *mb,
uint32_t mode,
xc_shadow_op_stats_t *stats)
{
int rc;
DECLARE_DOMCTL;
DECLARE_HYPERCALL_BUFFER_ARGUMENT(dirty_bitmap);
memset(&domctl, 0, sizeof(domctl));
domctl.cmd = XEN_DOMCTL_shadow_op;
domctl.domain = (domid_t)domid;
domctl.u.shadow_op.op = sop;
domctl.u.shadow_op.pages = pages;
domctl.u.shadow_op.mb = mb ? *mb : 0;
domctl.u.shadow_op.mode = mode;
if (dirty_bitmap != NULL)
set_xen_guest_handle(domctl.u.shadow_op.dirty_bitmap,
dirty_bitmap);
rc = do_domctl(xch, &domctl);
if ( stats )
memcpy(stats, &domctl.u.shadow_op.stats,
sizeof(xc_shadow_op_stats_t));
if ( mb )
*mb = domctl.u.shadow_op.mb;
return (rc == 0) ? domctl.u.shadow_op.pages : rc;
}
Does anyone know the mapping relationship between guest line address and the dirty map this method return?
Thanks,
Yunbin Wang
Related
I'm struggling with an issue where an ESP32 is running as a AP with AsyncTCP connecting multiple ESP32 clients. The AP receives some JSON data and replies with some JSON data. Without the handleData() function, the code runs 100% fine with no issues. Heap is static when no clients connect and issues only occur when clients start connecting.
Can anyone see anything with my code that could be causing heap corruption or other memory weirdness?
static void handleData(void* arg, AsyncClient* client, void *data, size_t len) {
int i = 0, j = 0;
char clientData[CLIENT_DATA_MAX];
char packetData[len];
char *packetBuf;
packetBuf = (char *)data;
clientData[0] = '\0';
for (i=0;i <= len;i++) {
packetData[j] = packetBuf[i]; //packetBuf[i];
if ((packetData[j] == '\n') || (i == len)) {
packetData[j] = '\0';
if ((j > 0) && (packetData[0] != '\n') && (packetData[0] != '\r')) {
// See sensorData() below...
parseData.function(packetData, clientData);
if (clientData != NULL) {
// TCP reply to client
if (client->space() > 32 && client->canSend()) {
client->write(clientData);
}
}
}
j = 0;
} else
j++;
}
}
void sensorData(void *data, void *retData) {
StaticJsonDocument<CLIENT_DATA_MAX> fields;
StaticJsonDocument<CLIENT_DATA_MAX> output;
char sensor[15] = "\0";
char MAC[18] = "\0";
char value[20] = "\0";
bool sendOK = false;
memcpy((char *)retData, "\0", 1);
DeserializationError error = deserializeJson(fields, (char *)data, CLIENT_DATA_MAX);
if (error) {
DEBUG_PRINTLN(F("deserializeJson() failed"));
return;
}
if (fields["type"])
strcpy(sensor, fields["type"]);
switch (sensor[0]) {
case 'C':
if (fields["value"])
strcpy(value, fields["value"]);
sendOK = true;
break;
case 'T': //DEBUG_PRINT(F("Temp "));
setExtTempSensor(fields["value"]);
sendOK = true;
break;
case 'N':
output["IT"] = intTempC; //Internal temp
output["B1"] = battLevels[0];
serializeJson(output, (char *)retData, CLIENT_DATA_MAX-1);
break;
}
if (sendOK) {
output["Resp"] = "Ok";
serializeJson(output, (char *)retData, CLIENT_DATA_MAX-1);
}
strcat((char *)retData, "\n");
}
static void handleNewClient(void* arg, AsyncClient* client) {
client->setRxTimeout(1000);
client->setAckTimeout(500);
client->onData(&handleData, NULL);
client->onError(&handleError, NULL);
client->onDisconnect(&handleDisconnect, NULL);
client->onTimeout(&handleTimeOut, NULL);
}
void startServer() {
server = new AsyncServer(WIFI_SERVER_PORT);
server->onClient(&handleNewClient, &server)
}
Using AsyncTCP on the ESP32 was having multiple issues. Heap issues, socket issues, assert issues, ACK timeouts, connection timeouts, etc. Swapping to AsyncUDP using the exact same code as shown above with romkey's changes, resolved all of my issues. (Just using romkey's fixes did not fix the errors I was having with AsyncTCP.) I don't believe the issue is with AsyncTCP but with ESP32 libraries.
Either you should declare packetData to be of length len + 1 or your for loop should iterate until i < len. Because the index starts at 0, packetData[len] is actually byte len + 1, so you'll overwrite something random when you store something in packetData[len] if the array is only len chars long.That something random may be the pointer stored in packetBuf, which could easily cause heap corruption.
You should always use strncpy() and never strcpy(). Likewise use strncat() rather than strcat(). Don't depend on having done the math correctly or on sizes not changing as your code evolves. strncpy() and strncat() will guard against overflows. You'll need to pass a length into sensorData() to do that, but sensorData() shouldn't be making assumptions about the available length of retData.
Your test
if (clientData != NULL) {
will never fail because clientData is the address of array and cannot change. I'm not sure what you're trying to test for here but this if will always succeed.
You can just write:
char sensor[15] = "";
you don't need to explicitly assign a string with a null byte in it.
And
memcpy((char *)retData, "\0", 1);
is equivalent to
((char *)retData)[0] = '\0';
What's the point of declaring retData to be void * in the arguments to sensorData()? Your code starts out with it being a char* before calling sensorData() and uses it as a char* inside sensorData(). void * is meant to be an escape hatch for passing around pointers without worrying about their type. You don't need that here and end up needing to extra casts back to char* because of it. Just declare the argument to be char* and don't worry about casting it again.
You didn't share the code that calls handleData() so there may well be issues outside of these functions.
I am doing an timerfd hello world in ubuntu 14.04, but got a strange situation: the int count is reset after read timerfd but uint64_int not.
int main(int agrc, char **argv) {
unsigned int heartbeat_interval = 1;
struct itimerspec next_timer;
struct timespec now;
if (clock_gettime(CLOCK_REALTIME, &now) == -1)
err_sys((WHERE + std::string("timer error")).c_str());
next_timer.it_value.tv_sec = now.tv_sec;
next_timer.it_value.tv_nsec = 0;
next_timer.it_interval.tv_sec = heartbeat_interval;
next_timer.it_interval.tv_nsec = 0;
int timefd = timerfd_create(CLOCK_REALTIME, 0);
if (timerfd_settime(timefd, TFD_TIMER_ABSTIME, &next_timer, NULL) == -1) {
err_sys((WHERE).c_str());
}
uint64_t s;
int exp;
int count = 1;
uint64_t count1=0;
while (1) {
s = read(timefd, &exp, sizeof(uint64_t));
if (s != sizeof(uint64_t)) {
err_sys((WHERE).c_str());
}
}
}
int exp;
^^^
s = read(timefd, &exp, sizeof(uint64_t));
^^^ ^^^^^^^^
Unless your int and uint64_t types are the same size, this is a very bad idea. What's most likely happening is that the 64 bits you're reading are overwriting exp and whatever else happens to be next to it on the stack.
Actually, even if they are the same size, it's a bad idea. What you should have is something like:
s = read(timefd, &exp, sizeof(exp));
That way, you're guaranteed to never overwrite the data and your next line would catch the problem for you:
if (s != sizeof(uint64_t)) {
It won't solve the problem that an unsigned integral type and an integral type will be treated differently but you can fix that just by using the right type for exp.
I've created the following network topology in Mininet to run an algorithm I've implemented using the Linux kernel eXpress Data Path.
The objective is to sample packets on the incoming link s1-eth1 on Switch 1 using XDP and store metadata in a shared BPF map. The execution is successful when run on multiple VMs (instead of using Mininet to create an emulation).
However, when using XDP on Mininet (to listen on the emulated network interface), packets aren't recorded.
To further diagnose the cause, I ran Wireshark to listen on the s1-eth1 interface, which does record packets hitting the interface, but for some reason these same packets aren't being registered through the XDP pipeline.
#define KBUILD_MODNAME "foo"
#include <linux/bpf.h>
#include <linux/in.h>
#include <linux/if_ether.h>
#include <linux/if_packet.h>
#include <linux/if_vlan.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
//BPF_TABLE("percpu_array", uint32_t, long, dropcnt, 256);
BPF_HASH(proto_map, uint32_t, uint32_t, 256);
//Packet Counter to keep track of number of packets flowing through XDP
BPF_ARRAY(pkt_count, uint64_t, 1);
//Map to keep track of the current EPOCH SIZE
BPF_ARRAY(epoch_size_map, uint64_t, 1);
static inline int parse_ipv4(void *data, u64 nh_off, void *data_end,
__be32 *src, __be32 *dest)
{
struct iphdr *iph = data + nh_off;
if (iph + 1 > data_end)
return 0;
*src = iph->saddr;
*dest = iph->daddr;
return iph->protocol;
}
static inline int bitXor(int* x, int* y)
{
int a = *x & *y;
int b = ~*x & ~*y;
int z = ~a & ~b;
return z;
}
int xdp_dsa(struct CTXTYPE *ctx) {
void* data_end = (void*)(long)ctx->data_end;
void* data = (void*)(long)ctx->data;
struct ethhdr *eth = data;
// drop packets
int rc = RETURNCODE; // let pass XDP_PASS or redirect to tx via XDP_TX
uint32_t *value;
uint32_t *counter_value;
uint32_t *epoch_size;
uint16_t h_proto;
uint64_t nh_off = 0;
uint32_t ipproto;
uint64_t magic_value = 12345678;
uint32_t packet = 0;
__be32 src_ip = 0, dest_ip = 0;
nh_off = sizeof(*eth);
if (data + nh_off > data_end)
pkt_count.increment(packet);
return rc;
h_proto = eth->h_proto;
if (h_proto == htons(ETH_P_IP))
ipproto = parse_ipv4(data, nh_off, data_end, &src_ip, &dest_ip);
/*
else if (h_proto == htons(ETH_P_IPV6))
index = parse_ipv6(data, nh_off, data_end);
*/
else
ipproto = 0; //i.e. unknown protocol
/*XOR the srcIP, destIP, and ipproto to encode, then hash*/
int xor_src_dest = bitXor(&src_ip, &dest_ip);
int xor_srcdst_ipproto = bitXor(&xor_src_dest, &ipproto);
uint32_t zero = 0;
//Predecided initial epoch size
uint32_t init_epoch_size = 10;
//Variable to store the current epoch size (to check end of epoch)
uint32_t cur_epoch_size;
//Lookup epoch size from shared map (to check whether intialized else read)
epoch_size = epoch_size_map.lookup(&zero);
// Start condition (epoch size map is initialized with zero), then set to initial epoch size
// Else read the current epoch size into a variable
if(epoch_size)
{
if(*epoch_size == 0)
{
*epoch_size = init_epoch_size;
}
else
{
cur_epoch_size = *epoch_size;
}
}
counter_value = pkt_count.lookup(&packet);
if (counter_value)
{
if (*counter_value < cur_epoch_size)
{
value = proto_map.lookup_or_init(&xor_srcdst_ipproto, &zero);
if (value)
{
pkt_count.increment(packet);
*value += 1;
}
}
else if (*counter_value == cur_epoch_size)
{
pkt_count.update(&packet, &magic_value);
}
else if(*counter_value == magic_value)
{
return rc;
}
}
return rc;
}
Any ideas?
My chip just stop doing anything. sometimes it prints good results, sometimes its not, i just cant understand whats wrong with this code( and generally any time you using Strings it happens )
void ParseGetRequest(char* data)
{
String parseGET=data;
String from="GET /";
String to="HTTP";
int ind1 = parseGET.indexOf(from);
int ind2 = parseGET.indexOf(to);
parseGET=parseGET.substring(ind1+from.length(), ind2-1);
strcpy(data, parseGET.c_str () );
}
And calling it with :
void readWifDataAsSever(char* reqData)
{
uint8_t buffer[128] = {0};
uint8_t mux_id;
uint32_t len = wifi.recv(&mux_id, buffer, sizeof(buffer), 100);
char serverData[100]={0};
if (len > 0)
{
for(uint32_t i = 0; i < len; i++)
serverData[i]=(char)buffer[i];
ParseGetRequest( serverData ); ///****** the call
Serial.println(serverData); // prints only part of the values
//here the chip just freeze and stop the main loop
NULL termination !!!!
serverData[len ] = '\0';
I am not getting ref_count to decrease properly for my GMainContext. The example program here is a small version of a large program (which uses threads, hence the need to create a context and push it on the thread).
GMainLoop *loop;
GMainContext *ctx;
struct conn
{
GSocketClient *client;
GSocketConnection *conn;
GInputStream *in;
GOutputStream *out;
gchar data[8192];
unsigned int count;
};
static void
read_done_cb(GObject *source_object, GAsyncResult *res, gpointer user_data)
{
struct conn *c = (struct conn *)user_data;
gssize len = g_input_stream_read_finish(c->in, res, NULL);
g_input_stream_read_async(c->in, c->data, sizeof c->data / sizeof *c->data, G_PRIORITY_DEFAULT, NULL, read_done_cb, c);
if (c->count++ == 1) {
printf("End of life as I know it...\n");
g_main_loop_quit(loop);
}
}
static void
write_done_cb(GObject *source_object, GAsyncResult *res, gpointer user_data)
{
}
static void
connect_done_cb(GObject *source_object, GAsyncResult *res, gpointer user_data)
{
printf("## %s\n", __FUNCTION__);
struct conn *c = (struct conn *)user_data;
c->conn = g_socket_client_connect_to_host_finish(c->client, res, NULL);
c->in = g_io_stream_get_input_stream(G_IO_STREAM(c->conn));
c->out = g_io_stream_get_output_stream(G_IO_STREAM(c->conn));
char *data = "GET /axis-cgi/mjpg/video.cgi HTTP/1.0\r\n\r\n";
g_output_stream_write_async(c->out, data, strlen(data), G_PRIORITY_DEFAULT, NULL, write_done_cb, c);
g_input_stream_read_async(c->in, c->data, sizeof c->data / sizeof *c->data, G_PRIORITY_DEFAULT, NULL, read_done_cb, c);
}
int
main(int argc, char **argv)
{
g_type_init();
struct conn *c = g_malloc0(sizeof *c);
ctx = g_main_context_new();
loop = g_main_loop_new(ctx, FALSE);
g_main_context_push_thread_default(ctx);
c->client = g_socket_client_new();
g_socket_client_connect_to_host_async(c->client, "10.85.25.20", 80, NULL, connect_done_cb, c);
g_main_loop_run(loop);
g_io_stream_close(G_IO_STREAM(c->conn), NULL, NULL);
g_object_unref(c->client);
g_object_unref(c->conn);
g_main_context_pop_thread_default(ctx);
g_main_loop_unref(loop);
g_main_context_unref(ctx);
return 0;
}
Using gdb, inserting breakpoint just before return I can see that ctx still have one ref count:
(gdb) p ctx->ref_count
$2 = 1
If I do another g_main_context_unref(ctx); everything shuts down as expected. I do not understand where I get this ownership though.
Thanks in advance for your help
I found the error. I read_done_cb I issued another g_input_stream_read_async and immediately after quitting the main loop. g_input_stream_read_async upped the ref_count but GMainLoop never got a chance to return to my callback (and decreasing the ref_count on my GMainContext).
Moving the call to g_input_stream_read_async in my callback to below the if statement
static void
read_done_cb(GObject *source_object, GAsyncResult *res, gpointer user_data)
{
struct conn *c = (struct conn *)user_data;
gssize len = g_input_stream_read_finish(c->in, res, NULL);
if (c->count++ == 1) {
printf("End of life as I know it...\n");
g_main_loop_quit(loop);
}
g_input_stream_read_async(c->in, c->data, sizeof c->data / sizeof *c->data, G_PRIORITY_DEFAULT, NULL, read_done_cb, c);
}
correctly resolved the number of ref counts on my main context.
Silly mistake. Hopefully someone will find some use of my post at least.
g_main_context_new(), g_main_loop_new(), and g_main_context_push_thread_default() all ref the context. g_main_context_pop_thread_default(), g_main_loop_unref(), and g_main_context_unref() all unref it. So your intuition is sound.
I would use a watchpoint in gdb: watch ctx->ref_count to find out where the extra reference is being added.