I'm trying to accelerate a legacy MPI Fortran program with GPU and OpenACC. And after I compile and profile the program with Nsight system, I found some unexpected data copy from HtoD and DtoH. I want to mitigate those data transfers by copy them in advance, but I could not find out what exactly was copied in and out. The program is quite large with many many source files, is there any ways to quickly identify the variable name corresponding to memcpy HtoD and DtoH shown on the profiler.
I also tried with PGI_ACC_DEBUG mode, but the information is overwhelming and I could not identify which is actually the data transfer I'm looking for. Is there some good practices to do that?
Unfortunately, I don't know a way to get this association from Nsight-systems. You might try using the environment variable "PGI_ACC_NOTIFY=2". This displays a more concise debug output, showing the file line, variable name, and size in bytes. Note that PGI_ACC_NOTIFY uses a bit mask to control what's being displayed with "1" being the kernel launches, "2" being the data transfers, and "3" being both.
Related
I am writing in VHDL (at Quartus II Web Edition [Software v9.1 Service Pack 2]) for a project of mine and I encountered a problem.
I have created a large decoder which I would like to test but not via creating a .bdf file and then adding a vector waveform file on it.
For the needs of this testing I have made a Python script to create .txt files with the proper 32bit vectors on it and then I am inserting them (c/p) to the .vwf file. Finally I do a timing simulation and then observe the outputs if they are the correct ones or not...but this is really time consuming and I think there is probably a better way to do it.
Or should I add the pain of converting by hand the 32bit values to integers to see that they are the ones expected after testing.
So, is there a faster/easier way to do it?
Thanks in advance!
Assume I have sensitive information (passwords, private keys,...) that I saved to a file which I encrypted.
Is there an easy to use tool to convert back and forth between a small file (say 0.5kB) and an image (QR code?) that I can print out to have a safe backup?
You can use LaTeX with the ps-tricks and pst-barcode modules, it produces nice QR codes, and yesm we used it exactly for this purpose: Paper backup of SSH private keys.
Denso Wave of QR code developer distributes software on their site.
Membership registration on the site is required to obtain it.
Even if you can not print with this, there are various tools regardless of free/commercial, so please search.
The maximum amount of data that can be stored in the QR code is 2953 bytes in binary mode.
However, it depends on the ability of the scanner to use.
QR code FAQ #6 Can an image or sound be stored in a QR Code?
I wrote a linux program to do this, called qr-backup.
In researching similar programs as part of it, I discovered a number of alternative projects as well. All of these are also linux-only.
asc2qr.sh
paperbackup. Focused on GPG/SSH key backup. See also the paperkey preprocessor, to reduce the size of keys.
qrdump (incomplete)
qrpdf
If your file is very small (0.5KB is a good cutoff), you can generate one single QR code. An example command-line program to generate it is qrencode. Several web converters are also available.
I can't find anything on Google for some tool that encrypts/obfuscates my lua files, so I decided to ask here. Maybe some professional knows how to do it? (For free).
I have made a simple game in lua now and I don't want people to see the code, otherwise they can easily cheat. How can I make the whole text inside the .lua file to just random letters and stuff?
I used to program in C# and I had this .NET obfuscator called SmartAssembly which works pretty good. When someone would try check the code of my applications it would just be a bunch of letters and numbers together with chinese characters and stuff.
Anyone knows any program that can do this for lua aswell? Just load what file to encrypt, click Encrypt or soemthing, and bam! It works!?
For example this:
print('Hello world!')
would turn into something like
sdf9sd###&/sdfsdd9fd0f0fsf/&
Just precompile your files (chunks) and load binary chunks. luacallows you to strip debugging info. If that is not enough, define your own transformations on the compiled lua, stripping names where possible. There's not really so much demand for lua obfuscators though...
Also, you loose one of the main advantages of using an embedded scripting language: Extensibility.
The simplest obfuscation option is to compile your Lua code as others suggested, however it has two major issues: (1) the strings are still likely to be easily visible in your compiled code, and (2) the compiled code for Lua interpreter is not portable, so if you target different architectures, you need to have different compiled chunks for them.
The first issue can be addressed by using a pre-processor that (for example) converts your strings to a sequence of numbers and then concatenates them back at run-time.
The second issue is not easily addressed without changes to the interpreter, but if you have a choice of interpreters, then LuaJIT generates portable bytecode that will run across all its platforms (running the same version of LuaJIT); note that LuaJIT bytecode is different from Lua bytecode, so it can't be run by a Lua interpreter.
A more complex option would be to encrypt the code (possibly before compiling it), but you need to weight any additional mechanisms (and work on your part) against any possible inconvenience for your users and any loss you have from someone cracking the protection. I'd personally use something sufficiently simple to deter the majority of curious users as you likely stand no chance against a dedicated hacker anyway.
You could use loadstring to get a chunk then string.dump and then apply some transformations like cycling the bytes, swapping segments, etc. Transformations must be reversible. Then save to a file.
Note that anyone having access to your "encryptor" Lua module will know how to decrypt your file. If you make your encrypted module in C/C++, anyone with access to source will too, or to binary of Lua encryption module they could require the module too and unofuscate the code. With interpreted language it is quite difficult to do: you can raise the bar a bit via the above the techniques but raising it to require a significant amount of work (the onlybreal deterent) is very difficult AFAIK.
If you embed the Lua interpreter than you can do this from C, this makes it significantly harder (assuming a Release build with all symbols stripped), person would have to be comfortable with stepping through assembly but it only takes one capable person to crack the algorithm then they can make the code available to others.
Yo still interested in doing this? :)
I thought I'd add some example code, since the answers here were helpful, but didn't get us all the way there. We wanted to save some lua table information, and just not make it super easy for someone to inject their own code. serialize your table, and then use load(str) to make it into a loadable lua chunk, and save with string.dump. With the 'true' parameter, debug information is stripped, and there's really not much there. Yes you can see string keys, but it's much better than just saving the naked serialized lua table.
function tftp.SaveToMSI( tbl, msiPath )
assert(type(tbl) == "table")
assert(type(msiPath) == "string")
local localName = _GetFileNameFromPath( msiPath )
local file,err = io.open(localName, "wb")
assert(file, err)
-- convert the table into a string
local str = serializer.Serialize( tbl )
-- create a lua chunk from the string. this allows some amount of
-- obfuscation, because it looks like gobblygook in a text editor
local chunk = string.dump(load(str), true)
file:write(chunk)
file:close()
-- send from /usr to the MSI folder
local sendResult = tftp.SendFile( localName, msiPath )
-- remove from the /usr folder
os.remove(localName)
return sendResult
end
The output from one small table looks like this in Notepad++ :
LuaS У
Vx#w( # АKА└АJБ┴ JА #
& А & name
Coulombmetervalue? С╘ ажў
I'm using KDE, and I'm toying with the idea of hacking the code for Dolphin File Manager (and potentially Konqueror if necessary) to get context-sensitive drag and drop behaviour (i.e. files are moved within the same partition, or copied if they're moved across partitions or the source is read only).
To do this, I think I'd need to find out the containing partition of the source and destination (easy enough on Windows using the drive letter, but on Linux, as mount points can be almost anywhere, it can't be reliably derived from the file path), and compare them.
Does anyone know how I can find out the partition that contains a given file?
It must be possible - I know Nautilus provides this sort of behaviour, but I'm not familiar enough with GTK to track down the appropriate section in the source code to see how its done...
Qt doesn't provide API for this. For POSIX, have a look at stat.
For KDE, you can use KIO::stat() to get mostly the same info as POSIX' stat function but asynchronously.
The device id should be in the field UDS_DEVICE_ID of the result.
I will be creating a structure more or less of the form:
type FileState struct {
LastModified int64
Hash string
Path string
}
I want to write these values to a file and read them in on subsequent calls. My initial plan is to read them into a map and lookup values (Hash and LastModified) using the key (Path). Is there a slick way of doing this in Go?
If not, what file format can you recommend? I have read about and experimented with with some key/value file stores in previous projects, but not using Go. Right now, my requirements are probably fairly simple so a big database server system would be overkill. I just want something I can write to and read from quickly, easily, and portably (Windows, Mac, Linux). Because I have to deploy on multiple platforms I am trying to keep my non-go dependencies to a minimum.
I've considered XML, CSV, JSON. I've briefly looked at the gob package in Go and noticed a BSON package on the Go package dashboard, but I'm not sure if those apply.
My primary goal here is to get up and running quickly, which means the least amount of code I need to write along with ease of deployment.
As long as your entiere data fits in memory, you should't have a problem. Using an in-memory map and writing snapshots to disk regularly (e.g. by using the gob package) is a good idea. The Practical Go Programming talk by Andrew Gerrand uses this technique.
If you need to access those files with different programs, using a popular encoding like json or csv is probably a good idea. If you just have to access those file from within Go, I would use the excellent gob package, which has a lot of nice features.
As soon as your data becomes bigger, it's not a good idea to always write the whole database to disk on every change. Also, your data might not fit into the RAM anymore. In that case, you might want to take a look at the leveldb key-value database package by Nigel Tao, another Go developer. It's currently under active development (but not yet usable), but it will also offer some advanced features like transactions and automatic compression. Also, the read/write throughput should be quite good because of the leveldb design.
There's an ordered, key-value persistence library for the go that I wrote called gkvlite -
https://github.com/steveyen/gkvlite
JSON is very simple but makes bigger files because of the repeated variable names. XML has no advantage. You should go with CSV, which is really simple too. Your program will make less than one page.
But it depends, in fact, upon your modifications. If you make a lot of modifications and must have them stored synchronously on disk, you may need something a little more complex that a single file. If your map is mainly read-only or if you can afford to dump it on file rarely (not every second) a single csv file along an in-memory map will keep things simple and efficient.
BTW, use the csv package of go to do this.