I red about netmap which allows user programmers to access packets in the user space, that means user applications can read / send network packets very quickly using this netamp.
netmap :
http://info.iet.unipi.it/~luigi/netmap/
Can any one who are very familiar with netamp, tell me should we create entire packet that we want to send out, or we using the stack features to send it out.
Edit : here is example on how to use this api
https://www.freebsd.org/cgi/man.cgi?query=netmap&sektion=4
#include <net/netmap_user.h>
void sender(void)
{
struct netmap_if *nifp;
struct netmap_ring *ring;
struct nmreq nmr;
struct pollfd fds;
fd = open("/dev/netmap", O_RDWR);
bzero(&nmr, sizeof(nmr));
strcpy(nmr.nr_name, "ix0");
nmr.nm_version = NETMAP_API;
ioctl(fd, NIOCREGIF, &nmr);
p = mmap(0, nmr.nr_memsize, fd);
nifp = NETMAP_IF(p, nmr.nr_offset);
ring = NETMAP_TXRING(nifp, 0);
fds.fd = fd;
fds.events = POLLOUT;
for (;;) {
poll(&fds, 1, -1);
while (!nm_ring_empty(ring)) {
i = ring->cur;
buf = NETMAP_BUF(ring, ring->slot[i].buf_index);
// here they are saying to construct the packet
... prepare packet in buf ...
ring->slot[i].len = ... packet length ...
ring->head = ring->cur = nm_ring_next(ring, i);
}
}
}
You need to create entire packed, including ethernet, ip and tcp headers. Netmap completely bypasses kernel network stack, so you need to do all work yourself.
Related
I need to filter priority into VLAN header to ensure Voice priority value.
Using BPF filtering is possible to evaluate which packets contains priority bits value into VLAN header are equals to five ?
Regards
Vincenzo
Yes you can, the exact way to do it depends on the type of eBPF program.
For programs with __sk_buff contexts(TC, Socket filter, cGroup SKB)
eBPF program types which get a __sk_buff as context can just access the vlan_tci field. This fields should already be in host byte order so you can just mask and bit shift the value to get the PCP field.
For XDP programs
In XDP programs we need to manually parse all network layers before we can access the TCI field.
XDP tutorial has a few parsing functions which are a good base, including parse_ethhdr_vlan:
/* Notice, parse_ethhdr() will skip VLAN tags, by advancing nh->pos and returns
* next header EtherType, BUT the ethhdr pointer supplied still points to the
* Ethernet header. Thus, caller can look at eth->h_proto to see if this was a
* VLAN tagged packet.
*/
static __always_inline int parse_ethhdr_vlan(struct hdr_cursor *nh,
void *data_end,
struct ethhdr **ethhdr,
struct collect_vlans *vlans)
{
struct ethhdr *eth = nh->pos;
int hdrsize = sizeof(*eth);
struct vlan_hdr *vlh;
__u16 h_proto;
int i;
/* Byte-count bounds check; check if current pointer + size of header
* is after data_end.
*/
if (nh->pos + hdrsize > data_end)
return -1;
nh->pos += hdrsize;
*ethhdr = eth;
vlh = nh->pos;
h_proto = eth->h_proto;
/* Use loop unrolling to avoid the verifier restriction on loops;
* support up to VLAN_MAX_DEPTH layers of VLAN encapsulation.
*/
#pragma unroll
for (i = 0; i < VLAN_MAX_DEPTH; i++) {
if (!proto_is_vlan(h_proto))
break;
if (vlh + 1 > data_end)
break;
h_proto = vlh->h_vlan_encapsulated_proto;
if (vlans) /* collect VLAN ids */
vlans->id[i] =
(bpf_ntohs(vlh->h_vlan_TCI) & VLAN_VID_MASK);
vlh++;
}
nh->pos = vlh;
return h_proto; /* network-byte-order */
}
You will have to modify this function for your purposes since it currently discards the PCP field you are after vlans->id[i] = (bpf_ntohs(vlh->h_vlan_TCI) & VLAN_VID_MASK);
I'm just learning how to handle sockets and TCP connections in C. I've got an application (a long one) which basically sends and receives char arrays with the system call write from server to client and vice versa (two separate C applications of course). As long as I use it with a local connection, on the same PC, running the server on a terminal and the client on an another, everything just works fine and the data arrives at the destination. But if I try it with the server on one computer and the client on another but on the same internet line, passing to the client an address like 192.168.1.X (took from the machine on which the server is running), after the connection is established, I've got an error that tells me that the number of expected bytes (which I pass before sending the real char[]) isn't arrived. Same thing if I try the server on my PC, and the client on another one with a different line on a different provider.
There's something I'm missing, are there any limitations in sending a bunch of bytes in sequence?
The code where the error pops up.
SERVER SIDE:
r=htonl(lghstr);
w=write(myFd,&r,sizeof(int));//writes the number of incoming bytes
if(w<0) perror("writeServer4"),exit(-1);
w=write(myFd,tmp->string,lghstr);
if(w<0) perror("writeServer5"),exit(-1);
if(w!=lghstr) perror("ERROR");
CLIENT SIDE
rC=read(fdc,&cod,sizeof(int));//read incoming number of bytes
lghstr=ntohl(cod);
if(rC<0) perror("readClient3"),exit(-1);
rC=read(fdc,dest,lghstr);
if(rC<0) perror("readClient4"),exit(-1);
if(rC!=lghstr) perror("error : "), printf("didn't read the right number of bytes"),exit(-1);
Now this is basically repeated a lot of times, let's even say 300 times, and it's with big numbers that the program doesn't work.
This is the problem:
rC=read(fdc,dest,lghstr);
...
if(rC!=lghstr) perror("error : ")
The #1 fallacy with socket programming is expecting that recv() and read() will return exactly the same number of bytes corresponding to the write/send call made by the other side.
In reality, partial data is extremely likely and expected. The simple workaround is to loop on read/recv until you get the exact number of bytes expected:
size_t count = 0;
while (count < lghstr)
{
ssize_t readresult = read(fdc, dest+count, lghstr-count);
if (readresult == -1)
{
// socket error - handle appropriately (typically, just close the connection)
}
else if (readresult == 0)
{
// The other side closed the connection - handle appropriately (close the connection)
}
else
{
count += readresult;
}
}
The other alternative to looping is to the use the MSG_WAITALL flag with the socket. This means, using recv() instead of read(). You'll still need to handle the error cases.
rc = recv(fdc, dest, lghstr, MSG_WAITALL);
if (rc == -1)
{
// socket error
}
else if (rc == 0)
{
// socket closed by remote
}
else if (rc < lghstr)
{
// the other side likely closed the connection and this is residual data (next recv will return 0)
}
You do ntohl() on one side and not the other. That might be interpreting the bytes with the wrong value.
You should printf() the bytes on both sides and see what the int is being evaluated to.
Edit: I'm convinced this is a programming bug for the record.
If I had to guess, I'd say that you are not synchronous with the other side for some reason. You say this runs 'about 300 times'.
Try adding a magic integer to the protocol.
Heres an example of a client that sends in this order.
A magic integer which is always constant.
A lengh of bytes about to be sent.
The bytes to be sent.
This uses scatter gather mechanics (its nicer for serialization) but other than that it effectively is doing the same thing yours is doing, as a client, just adding a magic value.
When the receiver receives the data, it can validate that the data is coming in the right order, by checking what the magic number was that came in. If the magic is wrong it means the client or server has lost themselves positionally in the stream.
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/uio.h>
#include <err.h>
#include <time.h>
#define MAGIC 0xDEADBEEFLU
#define GARBAGE_MAX 65536
const int iterations = 3000;
char * create_garbage_buf(
void)
{
int rc = -1;
int fd = -1;
char *buf = NULL;
buf = malloc(GARBAGE_MAX);
if (!buf)
err(1, "Cannot allocate buf");
fd = open("/dev/urandom", O_RDONLY);
if (fd < 0)
err(1, "Cannot open urandom");
rc = read(fd, buf, GARBAGE_MAX);
if (rc < 0)
err(1, "Cannot read from urandom");
else if (rc != GARBAGE_MAX)
errx(1, "Expected %d bytes, but got %d reading from urandom",
GARBAGE_MAX, rc);
close(fd);
return buf;
}
int main() {
int fd, offset, i, rc;
uint32_t magic = MAGIC;
uint32_t blen = 0;
char *buf = NULL;
struct iovec vecs[3];
/* Seed poor random number generator */
srand(time(NULL));
/* Use a file for demonstration, but a socket will do just fine */
fd = open("/dev/null", O_WRONLY);
/* Create some garbage to send */
buf = create_garbage_buf();
if (fd < 0)
err(1, "Cannot open file");
/* The first vector, is always the magic */
vecs[0].iov_len = sizeof(uint32_t);
vecs[0].iov_base = &magic;
for (i=0; i < iterations; i++) {
/* The second vector represents lengh of what we send
* in this demonstration it is a number between 0 and
* GARBAGE_MAX/2.
*/
blen = rand() % (GARBAGE_MAX / 2);
vecs[1].iov_len = sizeof(uint32_t);
vecs[1].iov_base = &blen;
/* The last record is the data to send. Its another random
* number between 0 and GARBAGE_MAX which represents the offset
* in our garbage data to send */
offset = rand() % (GARBAGE_MAX / 2);
vecs[2].iov_len = blen;
vecs[2].iov_base = &buf[offset];
rc = writev(fd, vecs, 3);
if (rc < 0)
err(1, "Could not write data");
if (rc != (sizeof(uint32_t)*2 + blen))
errx(1, "Did not write proper number of bytes to handle");
printf("Wrote %u bytes from offset %u in garbage\n", blen, offset);
}
free(buf);
printf("Done!\n");
return 0;
}
Closely read the documentation for read()/write() and learn that those two functions do not necessarily read()/write() as much bytes as they were told to, but few. So looping around such calls counting until all data expected had been read/written is a good idea, not to say an essential necessity.
For examples how this could be done for writing you might like to have look at this answer: https://stackoverflow.com/a/24260280/694576 and for reading on this answer: https://stackoverflow.com/a/20149925/694576
I am tring to send a screenshot of a desktop over winsock.
As such, there are four tasks:
Save bitmap to buffer
Write data across wire using a socket
Read data from wire using a socket
Load a bitmap from a buffer
I have saved the bitmap to a char array using GetDIBits.
Writing the data to the server, I have done but I have questions.
For writing data over from server to the client, do I need to use only 1 recv() call (I am using TCP), or do i need to split it up into multiple parts? Ive read that TCP is stream concept and that I wouldnt have to worry about packets because that is abstracted for me?
How would I go about loading the information from GetDIBits into a bitmap and displaying it on the main window?
I am guessing I have to use SetDIBits, but into which device contexts do i use?
The Server screenshot capturer is here:
HDC handle_ScreenDC = GetDC(NULL);
HDC handle_MemoryDC = CreateCompatibleDC(handle_ScreenDC);
BITMAP bitmap;
int x = GetDeviceCaps(handle_ScreenDC, HORZRES);
int y = GetDeviceCaps(handle_ScreenDC, VERTRES);
HBITMAP handle_Bitmap = CreateCompatibleBitmap(handle_ScreenDC, x, y);
SelectObject(handle_MemoryDC, handle_Bitmap);
BitBlt(handle_MemoryDC, 0, 0, x, y, handle_ScreenDC, 0, 0, SRCCOPY);
GetObject(handle_Bitmap, sizeof(BITMAP), &bitmap);
BITMAPFILEHEADER bmfHeader;
BITMAPINFOHEADER bi;
bi.biSize = sizeof(BITMAPINFOHEADER);
bi.biWidth = bitmap.bmWidth;
bi.biHeight = bitmap.bmHeight;
bi.biPlanes = 1;
bi.biBitCount = 16;
bi.biCompression = BI_RGB;
bi.biSizeImage = 0;
bi.biXPelsPerMeter = 0;
bi.biYPelsPerMeter = 0;
bi.biClrUsed = 0;
bi.biClrImportant = 0;
//std::cout<< bitmap.bmWidth;
DWORD dwBmpSize =((bitmap.bmWidth * bi.biBitCount + 5) / 32) * 4 * bitmap.bmHeight;
//int i = bitmap.bmWidth;
//DWORD dwBmpSize = 99;
HANDLE hDIB = GlobalAlloc(GHND, dwBmpSize);
char* bufptr = (char *)GlobalLock(hDIB);
GetDIBits(handle_ScreenDC, handle_Bitmap, 0, (UINT)bitmap.bmHeight, bufptr, (BITMAPINFO *)&bi, DIB_RGB_COLORS);
send(clientsock, bufptr , GlobalSize((char *)GlobalLock(hDIB)), 0);
/*Do i need to packetize/split it up? Or 1 send() is good for the matching Recv on the client?*/
/*I am assuming i must send bi structure over winsock also correct?*/
And The receiveing client code:
case WM_PAINT:{
//Im a Gdi beginner so I dont have a clue what im doing here as far as blitting the recved bits, this is just some stuff i tried myself before asking for help
PAINTSTRUCT paintstruct;
HDC handle_WindowDC = BeginPaint(hwnd, &paintstruct);
handle_MemoryDC = CreateCompatibleDC(handle_WindowDC);
handle_Bitmap = CreateCompatibleBitmap(handle_WindowDC, 640, 360);
std::cout << SetDIBits(handle_MemoryDC, handle_Bitmap, 0, bi.biHeight, buffer, (BITMAPINFO *)&bi, DIB_RGB_COLORS);
SelectObject(handle_MemoryDC, handle_Bitmap);
StretchBlt(handle_WindowDC, 50, 50, 640, 360, handle_MemoryDC, 0, 0, x, y, SRCCOPY);
EndPaint(hwnd, &paintstruct);
}
Sockets do have limited buffer sizes at both ends, typically around 4000 bytes. So if you dump a large block of data (like a full screendump) in one call to a non-blocking send, you will likely get errors, and you will need to manage your own buffers, calling multiple sends. However, if you are using non-blocking socket, you should be OK, as send() will simply block until all the data is sent.
On the receiving side, it is a similar case - a blocking receive can just keep waiting until it has the full data size that you asked for, but a non-blocking receive will return with whatever data is available at that time, which will result in the data filtering through bit by bit, and you will need to reassemble the data from multiple recv() calls.
I have heard of issues with sending really large blocks of data in one hit, so if you are sending 5 megabytes of data in one hit, be aware there might be other issues coming into play as well.
I use the following function to read from a file descriptor...
int cread(int fd, char *buf, int n){
int nread;
if((nread=read(fd, buf, n))<0){
perror("Reading data");
exit(1);
}
return nread;
}
Following is the function that uses the above function
if(FD_ISSET(tap_fd, &rd_set)){
/* data from tun/tap: just read it and write it to the network */
nread = cread(tap_fd, buffer, BUFSIZE);
tap2net++;
do_debug("TAP2NET %lu: Read %d bytes from the tap interface\n", tap2net, nread);
/* write length + packet */
plength = htons(nread);
nwrite = cwrite(net_fd, (char *)&plength, sizeof(plength));
nwrite = cwrite(net_fd, buffer, nread);
do_debug("TAP2NET %lu: Written %d bytes to the network\n", tap2net, nwrite);
}
They both work fine with TCP siocket but not with udp socket.. Any help would be appreciated
It's not clear exactly what your problem is, but if net_fd is a UDP socket, then the two cwrite() calls will create two UDP datagrams.
There isn't a great deal of point in prepending the size with UDP - UDP maintains the message boundaries for you. So in the UDP case, just remove the plength part entirely.
I've setup a simple TCP file transfer. Everything appears to work OK, except for the received file size is sporadically a smaller size than the file that was sent. There doesn't appear to be any pattern to the size of the received file.
(in the code below, note that the typical client/server rolls are reversed)
My client code is like:
#define kMaxBacklog (5)
// fill out the sockadd_in for the server
struct sockaddr_in servAdddress;
//memcpy() to fill in the sockaddr
//setup the socket
int sockd, returnStatus;
sockd = socket(AF_INET, SOCK_STREAM, 0);
if (sockd == -1)
NSLog(#"could not create client socket");
else
NSLog(#"created client socket");
returnStatus = connect(sockd, (struct sockaddr*)&servAdddress, sizeof(servAdddress));
if (returnStatus == -1)
NSLog(#"could not connect to server - errno:%i", errno);
else
NSLog(#"connected to server");
NSData *dataWithHeader = [self getDataToSend];
returnStatus = send(sockd, [dataWithHeader bytes], [dataWithHeader length], 0);
if (returnStatus == -1)
NSLog(#"could not send file to server");
else if( returnStatus < [dataWithHeader length])
NSLog(#"ONLY PARTIAL FILE SENT");
else
NSLog(#"file sent of size: %i", returnStatus);
shutdown(sockd, SHUT_WR);
close(sockd);
The client method ALWAYS reports that it sent the entire file.
For the server:
#define MAXBUF (10000)
int _socket;
_socket = socket(AF_INET, SOCK_STREAM, 0); // set up the socket
struct sockaddr_in addr;
bzero(&addr, sizeof(addr));
addr.sin_len = sizeof(addr);
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_ANY;
addr.sin_port = htons(0);
int retval = bind(_socket, (struct sockaddr *)&addr, sizeof(addr));
if (retval == -1)
NSLog(#"server could not bind to socket");
else
NSLog(#"server socket bound");
socklen_t len = sizeof(addr);
retval = getsockname(_socket, (struct sockaddr *)&addr, &len);
if (retval == -1)
NSLog(#"server could not get sock name");
else
NSLog(#"server socket name got");
int socket1, socket2, clientAddrLen, returnStatus;
struct sockaddr_in servAdddress, clientAddress;
clientAddrLen = sizeof(servAdddress);
socket1 = _socket;
returnStatus = listen(socket1, kMaxBacklog);
if (returnStatus == -1)
NSLog(#"server could not listen on socket");
else
NSLog(#"server socket listening");
while(1){
FILE *fd;
int i, readCounter;
char file[MAXBUF];
NSLog(#"server blocking on accept()");
socket2 = accept(socket1, (struct sockaddr*)&clientAddress, (socklen_t*)&clientAddrLen);
if (socket2 == -1)
NSLog(#"server could not accpet the connection");
else
NSLog(#"server connection accepted");
i = 0;
readCounter = recv(socket2, file, MAXBUF, 0);
if(!readCounter)
NSLog(#"server connection cancelled, readCount = 0");
else if (readCounter == -1){
NSLog(#"server could not read filename from socket");
close(socket2);
continue;
}
else
NSLog(#"server reading file of size: %i", readCounter);
fd = fopen([myfilePathObject cStringUsingEncoding:NSASCIIStringEncoding], "wb");
if(!fd){
NSLog(#"server could not open the file for creating");
close(socket2);
continue;
}
else
NSLog(#"server file open for creating");
returnStatus = fwrite([myData bytes], 1, [myData length], fd);
if (returnStatus == -1)
NSLog(#"Error writing data to server side file: %i", errno);
else
NSLog(#"file written to disk);
readCounter = 0;
//close (fd);
returnStatus = fclose(fd);
if(returnStatus)
NSLog(#"server error closing file");
So sporadically, the readCounter variable will not contain the same size as the file that was sent, but some times it does.
If it matters the file transfer is occurring between an iPhone and an iPhone simulator, both over WIFI. This happens regardless of if the phone is the server or if the simulator is the server.
If anyone can help me understand why this is occurring I'd appreciate it. I thought the whole purpose of TCP was to avoid this kind of problem.
(to give credit where it's due, for my server and client code I borrowed heavily from the book: The Definitive Guide to Linux Network Programming, by Davis, Turner and Yocom from Apress)
The recv function can receive as little as 1 byte, you may have to call it multiple times to get your entire payload. Because of this, you need to know how much data you're expecting. Although you can signal completion by closing the connection, that's not really a good idea.
Update:
I should also mention that the send function has the same conventions as recv: you have to call it in a loop because you cannot assume that it will send all your data. While it might always work in your development environment, that's the kind of assumption that will bite you later.
Tim Sylvester and gnibbler both have very good answers, but I think the most clear and complete is a combination of the two.
The recv() function return immediately with whatever is the in the buffer. This will be somewhere between 1 byte and MAXBUF. If the buffer is being written to while recv returns, you wont have the entire data that was sent in the buffer yet.
So you need to call recv() multiple times, and concatenate the data, to get everything that was sent.
A convenient way to do this (since we are working in cocoa) is to use NSMutableData like:
NSMutableData *fileData = [[NSMutableData alloc] init]; //Don't forget to release
while ((readCounter = recv(socket2, file, MAXBUF, 0)) > 0){
if (readCounter == -1){
NSLog(#"server could not read filename from socket");
close(socket2);
continue;
}
else{
NSLog(#"server reading file of size: %i", readCounter);
[fileData appendData:[NSData dataWithBytes:file length:readCounter]];
}
bzero(file, MAXBUF);
readCounter = 0;
}
You should probably have some kind of sequence of characters to signal termination of the file transfer, and only when you read those at the end of a block do you break out of your recv loop.
Of course, you will have to find a sequence that won't occur in your files, or that can be easily escaped. If you're working with text files this is pretty easy, but if not you'll have to be clever.
Alternatively, the client could first send the file size (in a separate send call), so the server knows how many bytes to expect in the file transfer.
recv returns right away with whatever is in the buffer (upto MAXBUF). If the buffer is being written to at the same time you might not get all the data
What TCP ensures is that your message will get to the remote peer correctly. As long as it fits in the send buffer, it will be automatically split into smaller chunks and sent by the local peer, and reordered and reassembled by the remote peer. It is not uncommon for a route to dynamically change while you are sending a message, which you would have to reorder manually (the smaller chunks) before delivering to your application.
As for your actual data transfer, your application needs to agree on a custom protocol. For instance, if you are only sending one message (the file), the sender could signal the receiver that it does not intend to write anymore to the socket (with shutdown(sock, SHUT_WR)), this way recv() returns with 0 and you know the transfer is complete (this is how a HTTP/1.0 server signals the client the transfer is complete). If you intend to send more data, then this alternative is not appropriate.
Another way would be to let the receiver know how much data the sender is going to transmit by including a header, for instance. It does not need to be overly elaborate, you could simply reserve the first 8 bytes to send the length as a 64-bit unsigned integer. In this case, you still need to be careful about byte ordering (big-endian / little-endian).
There is a very useful tutorial on network programming for UNIX environments:
Beej's Guide to Network Programming
You could refer to it to get a quick start, then refer back to the book for completeness, if you need. Even though you did not ask for additional references, TCP/IP Illustrated Vol. 1 and UNIX Network Programming Vol. 1 (both by W. Richard Stevens, the latter with a recent third edition) are excellent references.