I have written the host code in OpenCL. But I need first to read data from a .csv file. I need to make sure that what I did in reading the file is correct. (I am not sure if this is the way of reding a file in opencl)
1- I put the read file function which is written in c++ before the main .
2 - then, I put function to mix the data. Also before the main
3- In the main function, I call the above two function to read the data and then mix it.
4- then I write the part of host code which include(platform, device, context, queue, buffers....etc)
This is my code:
bool read_data_set(string filename, array<array<int, 20>, 5430>& array_X_dataset, array<int, 5430>& array_Y_dataset) {
int field0, field1, field2, field3, field4, field5, field6, field7, field8, field9, field10, field11,
field12, field13, field14, field15, field16, field17, field18, field19, field20, field21;
char comma;
int line = 0;
ifstream myfile(filename);
if (myfile.is_open())
{
while (myfile
>> field0 >> comma
>> field1 >> comma
>> field2 >> comma
>> field3 >> comma
>> field4 >> comma
>> field5 >> comma
>> field6 >> comma
>> field7 >> comma
>> field8 >> comma
>> field9 >> comma
>> field10 >> comma
>> field11 >> comma
>> field12 >> comma
>> field13 >> comma
>> field14 >> comma
>> field15 >> comma
>> field16 >> comma
>> field17 >> comma
>> field18 >> comma
>> field19 >> comma
>> field20 >> comma
>> field21)
{
array<int, 20> inner_array{ field1, field2, field3, field4, field5, field6, field7, field8, field9, field10, field11,
field12, field13, field14, field15, field16, field17, field18, field19, field20 };
array_X_dataset[line] = inner_array;
array_Y_dataset[line] = field21;
line++;
}
myfile.close();
}
else {
cout << "Unable to open file";
return true;
}
return false;
}
//functoin to randomly mix the dataset.
void mix_dataset(array<array<int, 20>, 5430>& array_X_dataset, array<int, 5430>& array_Y_dataset) {
size_t len = array_X_dataset.size();
for (size_t i = 0; i < len; ++i) {
size_t swap_index = rand() % len; // Random number between 0 and len-1.
if (i == swap_index)
continue;
array<int, 20> data_point{ };
data_point = array_X_dataset[i];
array_X_dataset[i] = array_X_dataset[swap_index];
array_X_dataset[swap_index] = data_point;
int Y = array_Y_dataset[i];
array_Y_dataset[i] = array_Y_dataset[swap_index];
array_Y_dataset[swap_index] = Y;
}
}
int main()
{
// Read dataset from file.
string filename = ".//Dataset.csv";
static array<array<int, 20>, 5430> array_X_dataset{};
static array<int, 5430> array_Y_dataset{};
size_t rows = sizeof(array_X_dataset) / sizeof(array_X_dataset[0]);
size_t cols = sizeof(array_X_dataset[0]) / sizeof(int);
bool error = read_data_set(filename, array_X_dataset, array_Y_dataset);
if (error) {
cout << "Exiting with error while reading dataset file " << filename << endl;
exit(-1);
}
// Randomly mix the dataset and printout.
// Initialize the seed.
srand(3);
mix_dataset(array_X_dataset, array_Y_dataset);
int array_X_set[5430][20];
int array_Y_set[5430];
// copy contents of the mixed std::arrays into plain arrays
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++)
array_X_set[i][j] = array_X_dataset[i][j];
array_Y_set[i] = array_Y_dataset[i];
}
int X_train[4344][20] = {};
int Y_train[4344] = {};
int X_test[1086][20] = {};
int Y_test[1086] = {};
//split the dataset using 5 - fold cross validation
float sum_accurecy = 0.0;
int fold = 1;
// cout << "inseret fold num " << endl;
//cin >> fold;
split_dataset(fold, array_X_set, array_Y_set, X_train, Y_train, X_test, Y_test);
//--------------------------host code--------------------------------------------------------------//
// Search for an openCL platform
cl_platform_id fpga_paltform = NULL;
if (clGetPlatformIDs(1, &fpga_paltform, NULL) != CL_SUCCESS) {
printf("Unable to get platform_id\n");
return 1;
}
// Search for an openCL device
cl_device_id fpga_device = NULL;
if (clGetDeviceIDs(fpga_paltform, CL_DEVICE_TYPE_ALL, 1, &fpga_device, NULL) != CL_SUCCESS) {
..............
.................
In short, the OpenCL programming model contains two codes, host code(.c/.cpp..) which runs on host(CPU) and kernel code(.cl) which runs on device(eg:GPU..).
Host Side :
you'll initialize the data(like you do in any C program)
Create a buffer object using clCreateBuffer() (think of it as reserving memory on the device) (similarly allocate for output)
Send the initialized data to the device using clEnqueueWriteBuffer()(to the earlier reserved space)
Invoke the kernel using clEnqueueNDRangeKernel()(now the device has kernel code and data)
Device Side:
Execute the kernel code
Write the output data to reserved space by host
Host Side:
Afer device completes its execution host reads the data from the Device using clEnqueueReadBuffer().
With this flow, you've offloaded the computation to the device and read the output to host.
NOTE:
This explanation is not 100% accurate, I tried to explain it in a simpler manner. I would suggest you read chapter-3 from (https://www.khronos.org/registry/OpenCL/specs/opencl-1.2.pdf)
Related
I was trying to implement pwd command in xv6 system. But i am getting a error in sysfile.c execution. The function is as follows showing the error:-
int sys_getcwd(void) {
char *p;
int n;
if(argint(1, &n) < 0 || argptr(0, &p, n) < 0)
return -1;
return name_for_inode(p, n, proc->cwd);
}
I get error as follows:
error: ‘proc’ undeclared (first use in this function)
return name_for_inode(p, n, proc->cwd);
But I also included proc.h in this file.
proc is not defined, and warning, it's a struct name.
You have to query the current process, you can do it with myproc() function
int sys_getcwd(void) {
char *p;
int n;
struct proc *curproc = myproc();
if(argint(1, &n) < 0 || argptr(0, &p, n) < 0)
return -1;
return name_for_inode(p, n, curproc->cwd);
}
Yes proc is not global variable so initialize it with current process on which cpu is working.
struct proc *proc = myproc();
return name_for_inode(p, n, proc->cwd);
I'm trying to decrypt a file in memory and this file was encrypted with openssl.
For doing that, i use mmap for loading my encrypted file in memory like that:
void* src = mmap(0, statbuf.st_size,PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
and duplicate it because i don't want to modify my original file
void* dst = mmap(0, statbuf.st_size,PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
memcpy (dst, src, statbuf.st_size);
At this step all is ok but i don't know what to do next.
Initialy for testing purpose, i encrypt my file with the openssl command :
system("openssl enc -aes-256-cbc -salt -in my_encryptedfile -out my_encryptedfile.enc -pass")
and decrypt it with this command:
system("openssl enc -d -aes-256-cbc -in my_encryptedfile.enc -out my_encryptedfile -pass pass:")
But i can't use dst in this case, so i search and discovered EVP Symmetric Encryption and Decryption.
link here
Then i encrypted and decrypted my file with that code github code
I try using Key and IV and the decryption in memory and it seem working but i have a problem that i don't understand. When i dump the buffer of my decripted file, i saw "SPACES/NULS" at the end of the file and i don't figure out why it display that. When i try to execute my binany in memory by calling this function :
func()
i got a segmentation fault
Any clues?
typedef void (*JittedFunc)(void);
void* alloc_writable_memory(void *ptr, size_t size) {
ptr = mmap(0, size,
PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (ptr == (void*)-1) {
perror("mmap");
return NULL;
}
return ptr;
}
int make_memory_executable(void* m, size_t size) {
if (mprotect(m, size, PROT_READ |PROT_WRITE | PROT_EXEC) == -1) {
perror("mprotect");
return -1;
}
return 0;
}
int do_crypt(char *in, char *out, int do_encrypt, int inlen)
{
/* Allow enough space in output buffer for additional block */
unsigned char outbuf[inlen + EVP_MAX_BLOCK_LENGTH];
int outlen;
EVP_CIPHER_CTX *ctx;
/* Bogus key and IV: we'd normally set these from
* another source.
*/
unsigned char key[] = "0123456789abcdeF";
unsigned char iv[] = "1234567887654321";
//int n;
printf("step1\n");
/* Don't set key or IV right away; we want to check lengths */
ctx = EVP_CIPHER_CTX_new();
printf("step2\n");
EVP_CipherInit_ex(ctx, EVP_aes_128_cbc(), NULL, NULL, NULL,do_encrypt);
printf("step3\n");
OPENSSL_assert(EVP_CIPHER_CTX_key_length(ctx) == 16);
OPENSSL_assert(EVP_CIPHER_CTX_iv_length(ctx) == 16);
/* Now we can set key and IV */
EVP_CipherInit_ex(ctx, NULL, NULL, key, iv, do_encrypt);
printf("step4\n");
if(!EVP_CipherUpdate(ctx, outbuf,&outlen, in, inlen))
{
printf("test 2.1: %d %d\n", inlen, outlen);
printf("step8\n");
/* Error */
EVP_CIPHER_CTX_free(ctx);
return 0;
}
//BIO_dump_fp (stdout, (const char *)outbuf, outlen);
printf(" test 2: %d %d\n", inlen, outlen);
if(!EVP_CipherFinal_ex(ctx, outbuf, &outlen))
{
printf("step11\n");
EVP_CIPHER_CTX_free(ctx);
return 0;
}
//copy the decryted buffer in another memory space
memcpy(out, outbuf, outlen);
printf(" test 3: %d %d\n", inlen, outlen);
//BIO_dump_fp (stdout, (const char *)outbuf, outlen);
printf("step12\n");
//fwrite(outbuf, 1, outlen, out);
printf("step13\n");
EVP_CIPHER_CTX_free(ctx);
return 1;
}
int main()
{
FILE *src, *dst;
char *src_mem, *dst_mem, *dst2_mem = NULL;
struct stat statbuf;
int fd;
src = fopen("hello_encrypted", "rb");
if (!src) {
/* Unable to open file for reading */
fprintf(stderr, "ERROR: fopen error: %s\n", strerror(errno));
return errno;
}
/*get the file des from a file*/
fd = fileno(src);
/* find size of input file */
if (fstat (fd,&statbuf) < 0)
{printf ("fstat error");
return 0;
}
/* go to the location corresponding to the last byte */
if (lseek (fd, statbuf.st_size - 1, SEEK_SET) == -1)
{printf ("lseek error");
return 0;
}
if ((src_mem = mmap (0, statbuf.st_size, PROT_READ, MAP_SHARED, fd, 0)) == (caddr_t) -1)
{
printf ("mmap error for input");
return 0;
}
if ((dst_mem = mmap (0, statbuf.st_size, PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS , -1, 0)) == (caddr_t) -1)
{
printf ("mmap error for output");
return 0;
}
if ((dst2_mem = mmap (0, statbuf.st_size , PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS , -1, 0)) == (caddr_t) -1)
{
printf ("mmap error for output");
return 0;
}
memcpy(dst_mem, src_mem, statbuf.st_size);
int n;
/* 0 for decrypting or 1 for encrypting*/
n = do_crypt(dst_mem,dst2_mem, 0, statbuf.st_size);
printf("%d\n", n);
make_memory_executable(dst2_mem, statbuf.st_size);
//dump of the decrypt binary
BIO_dump_fp (stdout, (const char *)dst2_mem, statbuf.st_size);
//try to launch the decrypted binary ==> segmentation fault
JittedFunc func = dst2_mem;
func();
fclose(src);
return 0;
}
I would remove from the first mmap the flag PROT_WRITE, because you dont want to modify the original file, I think you can also use PROT_PRIV, but have a look to what says the man of mmap.
On the other hand for decrypt your buffer you can use many libraries (a lot of them based on openssl), I specially like CryptoPP but there are many others. Here is an example of how your code will look like on CryptoPP
try {
// CryptoPP::ECB_Mode< CryptoPP::AES >::Decryption d;
// CrypoPP::CBC_Mode< CryptoPP::AES >::Decryption d;
CrypoPP::CBC_CTS_Mode< CryptoPP::AES >::Decryption d;
// What ever mode is best for you
d.SetKey(key.data(), key.size());
// The StreamTransformationFilter removes
// padding as required.
CryptoPP::StringSource s((const uint8_t*)dst, length_dst, true,
new CryptoPP::StreamTransformationFilter(d,
new CryptoPP::StringSink(recovered)
) // StreamTransformationFilter
); // StringSource
} catch(const CryptoPP::Exception& e) {
std::cout << "ERROR decrypting:" << e.what() << std::endl;
return;
}
I am storing some data in QDataStream and immediately taking the data, but the count is showing zero while retriving. code looks fine but unexpected behaviour
//Overloading
QDataStream& operator<< (QDataStream& writeTO, const CascadeJobInfo& data)
{
writeTO << data.m_infoJobType << data.m_connectionName << data.m_submitJobId << data.m_submitJobStat;
return writeTO;
}
QDataStream& operator>> (QDataStream& readIn, CascadeJobInfo& data)
{
readIn >> data.m_infoJobType >> data.m_connectionName >> data.m_submitJobId >> data.m_submitJobStat;
return readIn;
}
void Fun()
{
// Code Starts here
projectFileName = /*Path to folder*/
QFile file(projectFileName);
file.open(QFile::ReadWrite);
file.close();
QDataStream dStream(&file);
int jobLstCount = /*Get the Count, assume 4*/
dStream << jobLstCount;
for(int i = 0; i < jobLstCount; i++)
{
JobInfo.m_infoJobType = jobFlowItem->getJobType();
JobInfo.m_connectionName = submitItem->connectionName();
JobInfo.m_submitJobId = submitItem->jobID();
JobInfo.m_submitJobStat = submitItem->jobState();
// All valid data stored here
}
file.close();
QDataStream dStreamOut(&file);
dStreamOut >> jobLstCount; /*Count returns zero here insted of 4*/
CascadeJobInfo jobInfo;
// Why jobLstCount is getting zero here
for(int i = 0 ; i < jobLstCount ; i++)
{
dStreamOut >> jobInfo;
}
}
file.open(QFile::ReadWrite);
file.close(); <--- HERE
QDataStream dStream(&file);
You are closing the file as soon as you open it, so basically you are working with an invalid file descriptor which won't work. Put file.close() at the end of the code when you are done.
Pointer related question. I'm going through some example code that currently reads in data from a file called dataFile into a buffer. The reading is done inside a loop as follows:
unsigned char* buffer = (unsigned char*)malloc(1024*768*);
fread(buffer,1,1024*768,dataFile);
redPointer = buffer;
bluePointer = buffer+1024;
greenPointer = buffer+768;
Now, I want to try and write the entire contents of the array buffer to a file, so that I can save just those discrete images (and not have a large file). However, I am not entirely sure how to go about doing this.
I was trying to cout statements, however I get a print-out of garbage characters on the console and also a beep from the PC. So then I end my program.
Is there an alternative method other than this:
for (int i=0; i < (1024*768); i++) {
fprintf(myFile, "%6.4f , ", buffer[i]);
}
By declaring your buffer as a char*, any pointer arithmatic or array indexes will use sizeof(char) to calculate the offset. A char is 1 byte (8 bits).
I'm not sure what you are trying to do with the data in your buffer. Here are some ideas:
Print the value of each byte in decimal, encoded as ASCII text:
for (int i=0; i < (1024*768); i++) {
fprintf(myFile, "%d , ", buffer[i]);
}
Print the value of each byte in hexadecimal, encoded in ASCII text:
for (int i=0; i < (1024*768); i++) {
fprintf(myFile, "%x , ", buffer[i]);
}
Print the value of each floating point number, in decimal, encoded in ASCII text (I think my calculation of the array index is correct to process adjacent non-overlapping memory locations for each float):
for (int i=0; i < (1024*768); i += sizeof(float)) {
fprintf(myFile, "%6.4f , ", buffer[i]);
}
Split the buffer into three files, each one from a non-overlapping section of the buffer:
fwrite(redPointer, sizeof(char), 768, file1);
fwrite(greenPointer, sizeof(char), 1024-768, file2);
fwrite(bluePointer, sizeof(char), (1024*768)-1024, file3);
Reference for fwrite. Note that for the count parameter I simply hard-coded the offsets that you had hard-coded in your question. One could also subtract certain of the pointers to calculate the number of bytes in each region. Note also that the contents of these three files will only be sensible if those are sensibly independent sections of the original data.
Maybe this gives you some ideas.
Updated: so I created a complete program to compile and test the formatting behavior. This only prints the first 20 items from the buffer. It compiles (with gcc -std=c99) and runs. I created the file /tmp/data using ghex and simply filled in some random data.
#include <stdlib.h>
#include <stdio.h>
int main()
{
FILE* dataFile = fopen("/tmp/data", "rb");
if (dataFile == NULL)
{
printf("fopen() failed");
return -2;
}
unsigned char* buffer = (unsigned char*)malloc(1024*768);
if (buffer == NULL)
{
printf("malloc failed");
return -1;
}
const int bytesRead = fread(buffer,1,1024*768,dataFile);
printf("fread() read %d bytes\n", bytesRead);
// release file handle
fclose(dataFile); dataFile = NULL;
printf("\nDecimal:\n");
for (int i=0; i < (1024*768); i++) {
printf("%hd , ", buffer[i]);
if (i > 20) { break; }
}
printf("\n");
printf("\nHexadecimal:\n");
for (int i=0; i < (1024*768); i++) {
printf("%#0hx , ", buffer[i]);
if (i > 20) { break; }
}
printf("\n");
printf("\nFloat:\n");
for (int i=0; i < (1024*768); i += sizeof(float)) {
printf("%6.4f , ", (float)buffer[i]);
if (i > 20) { break; }
}
printf("\n");
return 0;
}
I am currently making a simple client and server but I have run into an issue. Part of the system is for the client to query about a local file on the server. The contents of that file must be then sent to the client. I am able to send all the text within a file to the client however it seems to be stuck in the read loop on the client. Below are the code spits for both the client and server that are meant to deal with this:
Client Code That Reads The Loop
else if(strcmp(commandCopy, get) == 0)
{
char *ptr;
int total = 0;
char *arguments[1024];
char copy[2000];
char * temp;
int rc;
strcpy(copy, command);
ptr = strtok(copy," ");
while (ptr != NULL)
{
temp = (char *)malloc(sizeof(ptr));
temp = ptr;
arguments[total] = temp;
total++;
ptr = strtok (NULL, " ");
}
if(total == 4)
{
if (strcmp(arguments[2], "-f") == 0)
{
printf("1111111111111");
send(sockfd, command, sizeof(command), 0 );
printf("sent %s\n", command);
memset(&command, '\0', sizeof(command));
cc = recv(sockfd, command, 2000, 0);
if (cc == 0)
{
exit(0);
}
}
else
{
printf("Here");
strcpy(command, "a");
send(sockfd, command, sizeof(command), 0 );
printf("sent %s\n", command);
memset(&command, '\0', sizeof(command));
cc = recv(sockfd, command, 2000, 0);
}
}
else
{
send(sockfd, command, sizeof(command), 0 );
printf("sent %s\n", command);
memset(&command, '\0', sizeof(command));
while ((rc = read(sockfd, command, 1000)) > 0)
{
printf("%s", command);
}
if (rc)
perror("read");
}
}
Server Code That Reads the File
char* getRequest(char buf[], int fd)
{
char * ptr;
char results[1000];
int total = 0;
char *arguments[1024];
char data[100];
FILE * pFile;
pFile = fopen("test.txt", "r");
ptr = strtok(buf," ");
while (ptr != NULL)
{
char * temp;
temp = (char *)malloc(sizeof(ptr));
temp = ptr;
arguments[total] = temp;
total++;
ptr = strtok (NULL, " ");
}
if(total < 2)
{
strcpy(results, "Invaild Arguments \n");
return results;
}
if(pFile != NULL)
{
while(fgets(results, sizeof(results), pFile) != NULL)
{
//fputs(mystring, fd);
write(fd,results,strlen(results));
}
}
else
{
printf("Invalid File or Address \n");
}
fclose(pFile);
return "End of File \0";
}
Server Code to execute the command
else if(strcmp(command, "get") == 0)
{
int pid = fork();
if (pid ==-1)
{
printf("Failed To Fork...\n");
return-1;
}
if (pid !=0)
{
wait(NULL);
}
else
{
char* temp;
temp = getRequest(buf, newsockfd);
strcpy(buf, temp);
send(newsockfd, buf, sizeof(buf), 0 );
exit(1);
}
}
The whole else if clause in the client code is a bit large for a function, let alone a part of a function as it presumably is. The logic in the code is ... interesting. Let us dissect the first section:
else if (strcmp(commandCopy, get) == 0)
{
char *ptr;
int total = 0;
char *arguments[1024];
char *temp;
ptr = strtok(copy, " ");
while (ptr != NULL)
{
temp = (char *)malloc(sizeof(ptr));
temp = ptr;
arguments[total] = temp;
total++;
ptr = strtok(NULL, " ");
}
I've removed immaterial declarations and some code. The use of strtok() is fine in context, but the memory allocation is leaky. You allocate enough space for a character pointer, and then copy the pointer from strtok() over the only pointer to the allocated space (thus leaking it). Then the pointer is copied to arguments[total]. The code could, therefore, be simplified to:
else if (strcmp(commandCopy, get) == 0)
{
char *ptr;
int total = 0;
char *arguments[1024];
ptr = strtok(copy, " ");
while (ptr != NULL)
{
arguments[total++] = ptr;
ptr = strtok(NULL, " ");
}
Nominally, there should be a check that you don't overflow the arguments list, but since the original limits the string to 2000 characters, you can't have more than 1000 arguments (all single characters separated by single spaces).
What you have works - it achieves the same assignment the long way around, but it leaks memory prodigiously.
The main problem seems to be that the server sends all the contents, but it doesn't close the socket, so the client has no way of knowing the server's done. If you close the socket after you finish sending the data (or just call shutdown()), then the client's read() will return 0 when it finishes reading the data.
FWIW, there are lots of other problems with this code:
getRequest: you call malloc but never free. In fact, the return value is thrown away.
Why bother forking if you're just going to wait() on the child?
You probably want to use strlcpy instead of strpcy to avoid buffer overruns.