So I'll be honest right up front. I know what the end product I need is, but I'm not 100% sure how to get there. Please read on so I can fully explain my situation and also some ideas I tried. I was torn between whether I should send this to a gaming form or to here. Considering that this is more "backend" related tho I opted to send it here.
So here's my dilemma. I'm not gonna bother explaining why because it isn't necessary, but essentially, I'm a sys admin for a minecraft server. We work on extremely high quality productions generally including custom coded systems and resourcepacks. One of the biggest problems we have is other individuals coming on to our networks and looking through our resourcepacks (which, if you aren't familiar, resourcepacks are sent from the server to the client as a .zip)
So essentially a individual from, say, a rivaling network comes onto ours and will snoop around our resourcepacks and uncover cool techniques or technologies that we develop. Only for us to later see those technologies implemented elsewhere!
A number of days ago I stumbled across a network which actually have managed to encrypt their resourcepacks so that it's not possible to snoop around them (via extracting them. Remember, these are archive files). When you try to extract or make any other changes to the archive, it will return a 80004005 error which I've come to learn is related to operating system permissions.
So here's what I'm looking for:
How can I encrypt a ResourcePack, or rather, an archive (specifically a .zip) but still have it be readable by Minecraft?
It's that second half which has stumped me.
I've already been able to encrypt a resourcepack myself (using WinZip) which recreated the 80004005 error as well as expected results (like not being able to copy files from inside of the archive for example.) However, obviously because the file is encrypted, there isn't any way for minecraft to read it!
And since the original network's file is encrypted (which I did reach out to them to see if they'll talk to me about how they implemented this feature) there's no way that I know of to reverse engineer the encrypted archive to figure out what method they used.
To reiterate, I have no interest in obtaining the contents of this other network's resourcepack. Rather I'm only interested in figuring out how to be able to encrypt/lock my own network's resourcepacks so that other's can't access it, but Minecraft can.
I'm already well aware that this question is outside of the realm of what we normally see on this website. I'm seriously taking shots in the dark to figure this cool technology out since file encryption and system permissions stuff is not exactly down my alley.
Like I said, 0 confidence on this topic, hopefully I gave enough details for you to help me out!
Even if you may not know much about Minecraft as a game. But have insight into how programs might be able to access an encrypted file while users can't (specific protocols to look into maybe?) I'm hoping I can get at least enough information to piece together the answer I need! Thank you again for your help!
This is completely impossible. In this case, not only do you have the standard reasons that effective DRM is impossible, but also that Minecraft only knows how to open completely normal .zip files. As for this:
A number of days ago I stumbled across a network which actually have managed to encrypt their resourcepacks so that it's not possible to snoop around them (via extracting them. Remember, these are archive files). When you try to extract or make any other changes to the archive, it will return a 80004005 error which I've come to learn is related to operating system permissions.
You've misunderstood what's going on here somehow. They didn't encrypt anything. If you post a new question (probably on a sister site and not this one) with the details of exactly how to get that .zip and the steps you followed, someone will be able to explain the real reason why you couldn't extract it at first.
Related
I have created an R code script that:
Reads some data from a database
Makes some transformations and..
exports into a csv the modified table.
This code needs to run in a client's machine, but we need to "hide" the actual code from the user.
Is there any useful suggestions on how we can achieve that?
Up front
... it will be nearly impossible to deploy an R <something> to another computer in a way that prevents curious users from accessing the source code.
From a mailing list conversation in 2011, in response to "I would not like anyone to be able to read the code.",
R is an open source project, so providing ways for you to do this is not
one of our goals.
Duncan Murdoch https://stat.ethz.ch/pipermail/r-help/2011-July/282755.html
(Prof Murdoch was on the R Core Team and R Foundation for many years.)
Background
Several (many?) programming languages provide the ability to compile a script or program into an executable, the .exe you reference. For example, python has tools like py2exe and PyInstaller. The tools range from merely compactifying the script into a zip-ball, perhaps obfuscating the script; ... to actually creating a exe with the script either tightly embedded or such. (This part could use some more citations/research.)
This is usually good enough for many people, by keeping the honest out. I say it that way because all you need to do is google phrases like decompile py2exe and you'll find tools, howtos, tutorials, etc, whose intent might be honestly trying to help somebody recover lost code. Regardless of the intentions, they will only slow curious users.
Unfortunately, there are no tools that do this easily for R.
There are tools with the intent of making it easy for non-R-users to use R-based tools. For instance, RInno and DesktopDeployR are two tools with the intent of creating Windows (no mac/linux) installers that support R or R/shiny tools. But the intent of tools like this is to facilitate the IT tasks involved with getting a user/client to install and maintain R on their computer, not with protecting the code that it runs.
Constrain R.exe?
There have been questions (elsewhere?) that ask if they can modify the R interpreter itself so that it does not do everything it is intended to do. For instance, one could redefine base::print in such a way that functions' contents cannot be dumped, and debug doesn't show the code it's about to execute, and perhaps several other protective steps.
There are a few problems with this approach:
There is always another way to get at a function's contents. Even if you stop print.default and the debugger from doing this, there are others ways to get to the functions (body(.), for one). How many of these rabbit holes do you feel you will accurately traverse, get them all ... with no adverse effect on normal R code?
Even if you feel you can get to them all, are you encrypting the source .R files that contain your proprietary content? Okay, encrypting is good, except you need to decrypt the contents somehow. Many tools that have encrypted contents do so to thwart reverse-engineering, so they also embed (obfuscatedly, of course) the decryption key in the application itself. Just give it time, somebody will find and extract it.
You might think that you can download the key on start-up (not stored within the app), so that the code is decrypted in real-time. Sorry, network sniffers will get the key. Even if you retrieve it over https://, tools such as https://mitmproxy.org/ will render this step much less effective.
Let's say you have recompiled R to mask print and such, have a way to distribute source code encrypted, and are able to decrypt it in a way that does not easily reveal the key (for full decryption of the source code files). While it takes a dedicated user to wade through everything above to get to the source code, none of the above steps are required: they may legally compel you to release your changes to the R interpreter itself (that you put in place to prevent printing function contents). This doesn't reveal your source code, but it will reveal many of your methods, which might be sufficient. (Or just the risk of legal costs.)
R is GPL, and that means that anything that links to it is also "tainted" with the GPL. This means that anything compiled with Rcpp, for instance, will also be constrained/liberated (your choice) by the GPL. This includes thoughts of using RInside: it is also GPL (>= 2).
To do it without touching the GPL, you'd need to write your interpreter (relatively from scratch, likely) without code from the R project.
Alternatives
Ultimately, if you want to release R-based utilities/apps/functionality to clients, the only sure-fire way to allow them to use your code without seeing it is to ... control the computers on which R will run (and source code will reside). I'll add more links supporting this claim as I find them, but a small start:
https://stat.ethz.ch/pipermail/r-help/2011-July/282717.html
https://www.researchgate.net/post/How_to_make_invisible_the_R_code
Options include anything that keeps the R code and R interpreter completely under your control. Simple examples:
Shiny apps, self-hosted (or on shinyapps.io if you trust their security); servers include Shiny Server (both free and commercial versions), RStudio Connect (commercial only), and ShinyProxy. (The list is not known to be exclusive.)
Rplumber is an API server, not a shiny server. The intent is for single HTTP(s) endpoint calls, possibly authenticated, supporting whatever HTTP supports (post, get, etc). This can be served in various ways, see its hosting page for options.
Rserve. I know less about this, but from what I've experienced with it, I've not had as much luck integrating with enterprise systems (where, e.g., authentication and fine-control over authorization is important). This does allow near-raw access to R, so it might not be what you want (especially when the intent is to give to clients who may not be strong R users themselves).
OpenCPU should be discussed, but not as a viable candidate for "protect your code". It is very similar to rplumber in that it provides HTTP endpoints, but it supports endpoints for every exported function in every package installed in its R library. This includes the base package, so it is not at all difficult to get the source code of any function that you could get on the R console. I believe this is a design feature, even if it is perfectly at odds with your intent to protect your code.
Anything that can call R or Rscript. This might be PHP or mod_python or similar. Any web-page serving language that can exec("/usr/bin/Rscript",...) can take its output and turn it around to the calling agent. (It might also be possible, for example, for a PHP front-end to call an opencpu endpoint that only permits connections from the PHP-serving host.)
penetration testing is a small hobby of mine, so I don't have a lot of experienced doing it. Keep that in mind when answering please.
I recently came across a network, where access to the C drive was blocked, so you couldn't access it by typing a path into Windows file manager, but there are some shortcuts on the desktop (which are the same and unchangeable for every standard user like me on the network), eg Photoshop, which if you do view file location on, you get into the C drive, and can navigate wherever you want. This made me think that that file viewer, which has access to the C drive has something special about it, and that I might be able to use it for some sort of privilege escalation, but being an amateur, I don't know if I'm correct, and I wouldn't be able to capitalise on it even if I was correct.
If someone could explain this to me, I would highly appreciate that. P.S.: the programming language I know best is python, and I have experience in using Kali Linux (I have a live bootable USB)
I'm using VS2010 and TFS to build a complex medium sized website.
Which protocol is most efficient and secure? I think I can enable any protocol I need since I own the IIS server and control all aspects of it.
My choices are:
Webdeploy
FTP
FileSystem
FPSE
There is also a hint at something called "one click"... not sure what that is, or if it relates to any of the above.
OK.. I'm sorry, but I'm not sure where to even start, and I'm not sure the question is answerable as-is. I'd probably put this as a note if there weren't a limit on the number of characters.
So much depends on the type of data in this app, your financial resources, etc. This is one of those subjects that seems like a simple question, but the more you learn, the more you realize you don't know. What you're talking about it Release management, which is just one piece of the puzzle in an overall Application Life-cycle Management strategy.
(hint, start at the link posted, and be prepared to spend months learning).
Some of the factors you may need to be aware of are regulatory factors that you many not even have thought of. Certain data is protected, and different standards require you to have formalized risk and release management built into your processes. For example, credits card data, medical records, etc, all have different regulations (some actual laws, some imposed by the Payment Card Industry) that you need to be aware of.
If your site contains ANY sensitive data, you need to first find out whether any of these rules apply to you, and if so, which ones? Do any of them require audit trails for how code goes from development to deployment? (PCI does, for example. That's because we take credit card payments, and in order to do that, you need to be PCI Certified or face heavy fines.)
If your site contains NO sensitive information at all, then your question could be answered as-is, and the question becomes a matter of what you're comfortable with.
If your application DOES contain sensitive info that makes it subject to rules that mandate a documented, secure ALM process, then the question becomes more complex, because doing deployments manually in such a situation is a PAIN IN THE BUTT. It' doesn't take too long before you start looking at tools to help automate some of the processes. (Build servers, tools such as Aldon for deployment, etc. There is a whole host of commercial and open source software to choose from.)
(we're using Atlassian for most of our ALM, but Team Foundation Server is also excellent, and there are a TON of other options.)
Our company is using some software that ONLY accepts input from an "Imaging Device" i.e. a TWAIN device (e.g. scanner).
The problem is that we are receiving our files digitally, so using an actual scanner would require us to print, scan, and shred documents that we already have on the computer, but not in the software.
I was curious if anybody has any idea of how we might be able to work around this problem in the meantime. My first thought was to find some way to trick the program into thinking we're using a scanner, via some new 'imaging device' that would just read in the file, and spit it out to the software, but I don't even know where to begin with that.
We put in a feature request, seeing as how this problem should obviously be addressed in the software itself, but the company is notorious for lagging pretty hard when it comes to updates.
The system used by scanners is called TWAIN, so you'd be looking for some sort of virtual twain driver.
A quick google search will produce several hits, I don't have any experience with the software myself so can't advise any further.
Two such providers I found via experts exchange:
http://www.twaintools.de
http://www.scanpoint-usa.com
OK, months late... but in case you are interested, I have a TWAIN driver framework/toolkit that might let you build this fairly easily, depending on just what your scanning app expects, and how hard it is to read images from your digital documents. It's a Microsoft Visual C++ project. No charge but you'd need our permission to redistribute a driver based on it: GenDS
The TWAIN Working Group also has a sample/skeleton driver, I think it's straight C - and used to have some rather bad bugs (Why I wrote mine ;-) but, it might have got better.
Look for the "sample data source and application" on their download page.
And of course I have a 'commercial' version of GenDS that I use to write TWAIN drivers on contract.
I am curious as to what others are using in this situation. I know a couple of the options that are out there like a memcached port or ScaleOutSoftware. The memcached ports don't seem to be actively worked on (correct me if I'm wrong). ScaleOutSoftware is too expensive for me (I don't doubt it is worth it). This is not to say that I don't want to hear about people using memcached or ScaleOutSoftware. I'm just stating what I "know" at this point.
So my question is basically this: for those of you ACTIVELY using distributed caching, what are you using, are you happy with it, and what should I look out for?
I am moving to two servers very soon...both will be at the same location. I use caching fairly heavily (but carefully) to reduce the load on my database server.
Edit: I downloaded Scaleout Software's solution. I've coded for it and it seems to work real well. I just have to decide if my wallet will part with the cash for it. :) Anyone have experiences good or bad with ScaleoutSoftware?
Edit Again: It's been a little while since I asked this? Any more thoughts on it? We ended up buying the solution from ScaleOutSoftware and have been happy with it, but I'm curious what others are doing.
Microsoft has a product pending code-named Velocity. It's still in CTP, and is moving slowly, but looks like it will be pretty good. We'll be beating it up in the near future to see how it handles what we want it to do (> 2 million read/writes per hour). Will post back with results.
There is a 100% native .NET, well documented open source (LGPL) project called Shared Cache. Looks like it is not yet mentioned on SO, but it's promising and should be able to do what most people expect from a distributed cache. It even supports different strategies like distributed or replicated caching etc.
I will update this post with more details as soon as I had a chance to try it on a real project.
We're currently using an incredibly simple cache that I wrote in a couple of hours, based on re-hosting the ASP.NET cache in a Windows Service (more info and source code here). I won't pretend it's anywhere near as optimised as something like Memcached but we were just looking for something simple and free until Velocity came along, and it's held up extremely well even under fairly heavy load.
It comes down to our personal preference for core components - i.e. ones that affect whether the site is available or not - that they are either (a) supported by a vendor with a history of rapid and high quality support, or (b) written by us so that if something goes wrong we can fix it quickly. Open source is all well and good, and indeed we do use some OSS, but if your site is offline then unfortunately newsgroups et al don't have a 1 hour SLA, and just because it's OSS doesn't mean you have the necessary understanding or ability to fix it yourself.
We are using the memcached port for Windows and we are very pleased with it. The enyim.com memcached client API is great and easy to work with. It's also open source, which is a big advantage, if you ask me.
We are now using this setup in a production web-app and it has helped a lot in improving its performance.
There's a great .NET wrapper/port found here on Codeplex. Awesomesauce!
We use memcached with the enyim library in a production environment (www.funda.nl). Works fine, very pleased with it, but we did notice a substantial raise in CPU use on the clients. Presumably due to the serializing/deserializing going on. We do around 1000 reads per second.
One tried and tested product by 100's of customers worldwide is NCache. Its
a feature rich product that lets you store session state in a redundant and highly available manner, lets you share data
within the enterprise as well as bridging for WAN communication essentially acting as a data fabric and lastly it lets you build an elastic caching tier so that when
your application scales, you can add servers to the cache and actually boost performance further.