I'm working on an embedded project that has no file system, and our kludge of a database has ballooned in functionality. It's now so inefficient, we can't stand it any longer.
I'm trying to figure out if there's built-in support for a flat file system in SQLite. I've dug around http://sqlite.org for a while now, but haven't found anything specifically covering it. Has anyone found a supported implementation written in C? It seems like such a straightforward request, but I can't find anything written about it.
We have plenty of memory available. In a perfect world, I'd like to just set aside a large portion of it, pass the FS a pointer, then let the FS/SQLite library just use what it wants.
You could implement an SQLite Virtual File System for your non-volatile memory.
Related
I'd like to know whether it's possible to encrypt the compiled code in a Unity3D build in a way that de-compiling dlls won't give access to the legible source code to a potential hacker.
Cosmore pointed out in this post that it's possible to
'encrypt the assembly dlls and modify the mono loader (libmono.so)'
My question is, has anybody done this? If so, how? And can it be done on Windows, Mac and Linux builds?
If this is not possible, is there another way?
But the hacker should be capable to run the build (ie. play the game)?
Then in principle it's not possible. While you can encrypt the code, you have to also provide the key, so people can play it (decrypt).
So you are just adding one more layer of obfuscation for hacker. A seasoned hacker will simply disassemble the dll loader and put a breakpoint to a place where the content is decrypted already, and dump that for further tinkering.
That said, not every hacker is a seasoned hacker, and few layers of (unexpected) obfuscation may turn them away. So practically some encryption/obfuscation is done by some people, even if in principle it's hopeless.
I have a bunch of R scripts which I am running on a Windows machine and want to ensure that the code remains unread by those not intended to see it. On a Linux box, I could wrap the R code in a bash script #! and make an encrypted (and perhaps even a limited-life) executable shell script. What are my options to do something on similar lines under Windows?
My answer is a bit late, but I believe this is a good question. Unfortunately, I don't believe that there is a solution, or at least an easy one, at the present time.
The difficulty is common because, for most interpreted languages, including R, it is often possible to turn on logging and inspection of all commands being run. This can negate many tricks to obfuscate the code.
For those who prefer to think of code being open == good, one should know that a common reason to obfuscate the code is if one is consulting with a client that hires multiple vendors. It is not uncommon for a client to take scripts from vendor A and ask vendor B why it doesn't work with their system. (This may be done by a low-level IT flunkie, rather than someone responsible for the NDA contracts.) If A & B are competitors, A's code has just been handed to B. When scripts == serious programs, then serious code has been given away.
The ways I've seen this addressed are:
Make a call to a compiled language, and use standard protections available there.
Host the executable on a different server, and use calls to the server to execute the calculations. (In R, there are multiple server-side options.)
Use compiled (preprocessed / bytecode) code within the language.
Option 2 is actually easier and better when the code may be widely distributed, not just for IP reasons. A major advantage is that it lets you upgrade the code without having to go through the pain of a site-wide release process. If new libraries are needed, no problem - update the server.
Option 3 is done in Matlab with .p files, and can be done with py2exe for Python on Windows. In R, the new bytecode compilation may be analogous, but I am not familiar enough with it to address any differences between .Rc files in the R context and .p files in the Matlab context. For more info on the compiler, see: http://www.inside-r.org/r-doc/compiler/compile
Hosting computations on the server is great for working with unsophisticated users, because it is easier to iterate quickly in response to bugs or feature requests. The IP protection is simply a benefit.
This is not a specifically R-oriented strategy. (And it's a bit unclear what your constraints or goals really are anyway.) If you want a cross-platform encryption method, you should look into the open-source program TrueCrypt. It supports creating encrypted files that can be mounted as volumes on any machine that supports the volume formatting method. I have tested this across the Mac PC divide , since the Mac can read FAT files, but have no experience with how it might work across the Linux-PC chasm.
(Their TODO list for Windows includes;"Command line options for volume creation (already implemented in Linux and Mac OS X versions)". So I don't see any clear way to use this from within R without you running the program from the OS.)
I don't think this is possible because the R interpreter has to be able to decrypt and read the code in order to execute it which means that whoever is using that interpreter will also be able to decrypt and read the code.
I am by no means an expert, so I reserve the right to be 100% wrong about that statement.
I believe the best solution is to ensure value comes from the expertise and services provided by your company and it's employers---not from keeping secrets.
Failing that, you could try separating the code into a client/server model. That way the client just sends data and receives results---they never have access to the code that runs on the server.
However, the scientist in me just said "that solution sucks and I would never trust results provided under such conditions".
I'm working on a closed-source game that uses a scripting language for automation. Almost all of the game logic is handled by scripts. Scripts can be compiled to a bytecode format, but due to the nature of the language, identifiers must be preserved. Compiled scripts can be embedded in other text-based resource formats using a binary-to-text encoding.
I want to encrypt the compiled scripts to protect the source during distribution, but because the language, bytecode format, and binary-to-text encoding scheme are all proprietary, do I need to worry about encryption at all? If so, should I simply perturb some bytes and call it a day, or should I make use of a fully featured encryption solution? Encryption should not increase the size of the executable unduly, because scripts can be large and load times are important.
On Windows, the size of the executable has no impact on load times because the exe is just mapped into memory and then paged in as needed. I can't imagine why that would not be true for *nix as well.
So, if the scripts don't need to change separately from your .exe, you could imbed them into the .exe, that would make them difficult for users to change even if they could find them. I wrote a little tool once that turned data files into .obj files that made it really easy to imbed data into my exe - it turned out to be pretty easy to write an object file that contains only data.
Of course, if you really care about protecting this data than full encryption in your only choice, but if you are just trying to discourage casual hacking, making the files hard to get at might be good enough.
You shouldn't assume that people can't read binary proprietary formats. There are many people who are very good at reverse-engineering protocols without any documentation.
So if you want to keep your source safe, you need some real security. The only problem is that if you encrypt the files, you'll need to give your users the decryption key in order to play the game, and when you do that then its only a matter of time before someone works out how to get the key and use it to decrypt all the files.
So basically, there's not much you can do unfortunately. You could try obfuscating your code, but even that's not going to stop everyone.
What you're talking about isn't going to be encryption, because you're going to have to ship the decryption key with it. It's just obfuscation. No matter how much you try to hide the decryption key, if your program can find it, so can the user.
So once you understand that we're just talking about various obfuscation schemes, the question is how much obfuscation you need. Likely the proprietary byte-compiling is a higher barrier than the encryption would be, and I'd call it a day. Anyone who wants to trace the logic can just put a debugger on it whether you encrypt or not. If they've already reverse-engineered your run-time engine to work out the byte-codes, then they're already in the portion of the code that has the unencrypted data.
That said, if you find the identifiers in the file to be problematic you can mechanically converts them to random strings prior to byte-compiling.
Encryption would not buy you much here.
Basically, whatever encryption layer you add, the executable itself must be able to perform the decryption in order to run the scripts. You lock the door, but you leave the key in the lock. This is unavoidable.
What encryption does is that it somewhat raises the bar on who may access the data. It requires some disassembly skills. Simply embedding the files in the executable will already filter out the casual not-very-good hacker. Those who are not deterred by such embedding are also those who will be able to follow the data processing path, find the decryption logic, and siphon out the decrypted code at will. A layer of encryption may also increase the feeling of importance: that which was encrypted is certainly worthwhile. Hence, trying too funky things may just make your situation worse, not better.
On the other hand, embedding the files in the executable binary is probably a good idea. It would remove the need to locate them at runtime on the filesystem (locating things at runtime is known to be somewhat more difficult on Unix systems than on Windows, because of hard links: on Windows, an executable can easily obtained its own path, but on Unix the presence of hard links means that "the" executable path is ill-defined).
I have seen a few blog entries on this and have had a discussion or two with my team mates but I would like to see what the stack overflow community thinks.
So why does the Adobe Alchemy Tool create so much faster running flash byte code than the flex compiler?
Also, when will the flex compiler be able to make similar performance gains?
Will it require programmer specific use of special Array's or something of that nature to get the same performance?
Alchemy is an implementation of LLVM in ActionScript. Simply put, it's an virtual machine that uses a ByteArray as it's memory store.
The C code compiled by Alchemy has direct access to "memory" (via some opcodes introduced in Flash 10), allowing it to chunk memory around at it's leisure (including pointers to objects). This results in some, but by no means all, code running faster. Some types of code will actually run slower in Alchemy due to it being a VM running on top of the AVM (another VM).
Additionally, Alchemy does not have native access to ActionScript classes and must access them through interop classes.
The alchemy tool creates code that uses instructions in the flash player that aren't available to the regular compiler (and the talk is that these instructions were exposed especially for alchemy).
Whether the regular compiler will eventually make similar gains, hopefully. It's been proven a few times that the compiler creates substandard code, and there are a couple of projects which optimise the generated code. These may shame Adobe into improving.
Chances are, no, there won't be anything special a programmer needs to do to get these performance gains (though check out the optimising blogs, writing loops in a particular way means they can be optimised better).
On most UNIX systems passing an open file between processes can be easily done for child/parent processes by fork(); however I need to share a fd "after" the child was already forked.
I've found some webpages telling me that sendmsg() may work for arbitary processes; but that seems very OS dependent and complex. The portlisten seems like the best example I can find, but I'd prefer a good wrapper library like libevent that hides all the magic of kqueue, pool, ....
Does anyone know if there's some library (and portable way) to do this?
Your best bet is to try sending the file descriptor over a Unix domain socket. This is described in Stephens, and in a few places on the web, but I can dig up code for you if you ask nicely.
This will be pretty portable these days; a lot of the things considered "non-portable" way back when (such as mmap!) are extremely common now. If you need to be more portable than "most systems these days," you've got a lot of interesting issues ahead of you, but possibly if you tell us more about what you're doing and what platforms you're working on (perhaps non-Unix POSIX platforms?) we might be able to help out.
There is a Unix domain socket-based mechanism for transferring file descriptors (such as sockets - which cannot be memory mapped, of course) between processes - using the sendmsg() system call.
You can find more in Stevens (as mentioned by Curt Sampson), and also at Wikipedia.
You can find a much more recent question with working code at Sending file descriptor by Linux socket.