I am pretty new to HBase and have been assigned a task to move our infrastructure to cloud. Our HBase data contains some customer information and hence needs to be encrypted while at-rest. I am already reading this: Transparent Encryption of Data At Rest (http://hbase.apache.org/book/ch08s03.html#hbase.encryption.server)
It looks like a good solution except the fact that we have to store the password as plain text on each node. Is there a way to avoid this? Like store the password at just one location and source it to each node or something similar?
Thanks
Related
I have a node that contains lots of data like 20MB and I need to search for a specific string in that node.
So I just wrote a Cloud Function to achieve this but it's consuming too much memory. So I asked a friend of mine and he suggested this:
download the file to your local server daily and then use binary search on requests.
This is doable and makes sense to me but if you guys have a better idea I really would like to hear.
How can I search for a specific string in a 20MB file with NodeJS?
I have been unable to find any documentation that discusses best practices when it comes to managing images using Firebase.
I am considering whether or not to add the filenames of uploaded images into the database or not.
If they're in the database then I can make one db get request to get the list of existing images. These would be put into the database on fileUploaded events and deleted from the database on delete.
Is it better for me to just do a few file exists requests directly on the Storage and not have the db overhead, for instance (pseudo code);
if ( gcs.exists('/storage_path/favicon.png') ) {
...
}
if ( gcs.exists('/storage_path/favicon-32x32.png') ) {
...
}
Or is it better to store those images filenames in the database, assume they exist, and pull them out in a document .get?
I would like this to be as lightweight as possible, I know in advance the list of filenames we want to know about (they're favicons) so looping and doing .exists is less code, but perhaps slower than putting and pulling from the database.
Any information you have on the efficiency of db document requests and storage exists requests (which I would assume are doing something similar behind the scenes anyway).
Please ask for more information if I'm not clear.
Based on the fact that Firestore clients try to maintain an open socket connection to the database, I'd give the edge to a database get being faster than checking for file existence in Cloud Storage. With Firestore, you're less likely to pay the cost of establishing an SSL connection to the cloud service.
From architectural point of view, I would save the file names in Firestore and retrieve them from the client directly based on the file name, its much simpler and you can add different types of images for different purposes without much thought down the road under same bucket
I'm using an Instant Messaging software, and I suspect that the software is retaing a lot of information about my machine (such as my MAC address) and possibly leaks it. I decided I want to check the local DBs of the software and see what it saves locally.
I have been able to locate, using the software's own log dump and Procmon, the interesting DBs. However, they are SQLite DBs that are key-protected.
Do I have any way to know what will be the format and size of the key? Will it be hex?
How can I efficiantly continue my research? I looked, using procmon, and been able to detect the first time that the software uses a key-protected DB from the first time it is being opened. However, I couldn't detect any 'interesting' local file that the software uses and could hint about the key's location - apart from several Windows Registries values that are being used - but I'm not so sure on how to approach that.
Sorry if I have mistakes in English, and thank in advance.
Do I have any way to know what will be the format and size of the key? Will it be hex?
The key is just in plaintext (just like normal passwords) and the size is (also like passwords) defined by the creator of the database.
How can I efficiantly continue my research?
I would recommend reverse engineering the application and look for the part, where the connection to the database gets initiated. For that, you can use dynamic analysis (with a debugger) or static analysis (analyse the binary with a disassembler).
I am using a AES encryption/decryption class that needs a key value and vector value encrypt and decrypt data in an MVC3 application.
On saving the record I am encrypting the data then storing in a database. When i retrieve the record i am decrypting in the controller and passing the unencrypted value to the view.
The concern is not protecting data as it traverses the network but to protect the database should it be compromised.
I have read many posts that say dont put the keys for encryption in your code.
Ok so where should they be kept? File system? Another Database?
Looking for some direction.
Common sense says, if an intruder gets access to your database, they will most likely also have access to your file system. It really comes down to you. For one, you can try to hide it. In configuration files, in plain files somewhere in file system, encrypt it with another key that is within the application ... and so on and so forth.
Configuration files are a logical answer, but why take a chance - mix it. Feel free to mix keys with multi-level encryptions - one requiring something from the record itself and being unique to every record, other one requiring a configuration value, third one requiring an application-specific value, and perhaps a fourth one from a library hidden well within your application's references? This way, even if one layer somehow gets compromised, you will have several others protecting it.
Yes, it adds overhead. Yes, it is relatively expensive. But is it worth it if you have sensitive data like user credit card details? You bet it is.
I'm using similar encryption and hashing techniques in one of my personal pet projects that is highly security focused and carefully controlled. It depends how much data you need to display at any one time - for example, mine will ever fetch only 10 records at a time, most likely even less.
... To specify what I mean by mixing: Encrypt once. Then encrypt that data again with different key and suggestedly different algorithm.
I would use Registry Keys protected by ACL, so only the account under which your app pool is running can read them.
I'm just trying to determine if the files on the filesystem used by Raven DB are encrypted or not? Can someone just open the files on the filesystem and convert them from binary to ASCII directly, or are they encrypted?
I am trying to convince our management to give RavenDB a shot, but they have concerns about security. They gave the example that you can't just open up an MS SQL db file, convert it from binary to ASCII, and read it. So I am trying to verify if RavenDB prevented that kind of thing as well?
Well, personally I think that your management sucks if they come up with such straw-man arguments.
To answer your question: No, you can't just open any file inside ravens data folder with Notepad and expect to see something meaningful. So, for the ones that don't know how to program, yes they are encrypted.
To convice your management you can tell them that raven uses the same encryption algorithm as Microsofts Exchange Server does. If they want to dig deeper - it's called Esent.
RavenDb storage is not encrypted. You can open it with notepad and see some pieces of data. At the same time I do not think that MS SQL encrypts files by default either.
RavenDB added encryption in mid-2012. Get RavenDB's “bundle:encryption” and then make sure your key is properly encrypted in the .NET config file or whatever.
http://ravendb.net/docs/article-page/3.0/csharp/server/bundles/encryption
http://ayende.com/blog/157473/awesome-ravendb-feature-of-the-day-encryption
SQL Server 2008 does have encryption, but you need to prepare the DB instance beforehand to enable it, then create the DB with encryption enabled and then store data.
If you haven't, you could just copy the DB off the machine and open it in a tool that does have access to it.
With RavenDB, you can tick the box and off you go! (although I do not know the intricacies of moving backups to another machine and restoring them).
In relation to the point your management made, this is a relatively pointless argument.
If you had access directly to the file of a DB, it's game over. Encryption is your very last line of defence.
[I don't think hackers are going to be opening a 40GB file in Notepad .. thats just silly :-)]
So instead of ending up at the worst case, you have to look at the controls you can implement to even get to that level of concern.
You need to work out how would someone even get to that file (and the costs associated with all of the mitigation techniques):
What if they steal the server, or the disk inside it?
What if they can get to the DB via a file share?
What if they can log onto the DB server?
What if an legitimate employee syphons off the data?
Physical Access
Restricting direct access to a server mitigates stealing it. You have to think about all of the preventative controls (door locks, ID cards, iris scanners), detective controls (alarm systems, CCTV) and how much you want to spend on that.
Hence why cloud computing is so attractive!
Access Controls
You then have to get onto the machine via RDP or connect remotely to its file system via Active Directory, so that only a select few could access it - probably IT support and database administrators. Being administrators, they should be vetted and trusted within the organisation (through an Information Security Governance Framework).
If you also wanted to reduce the risk even further, maybe implement 2 Factor Authentication like banks do, so that even knowing the username and password doesn't get you to the server!
Then there's the risk of employees of your company accessing it - legitimately and illegitimately. I mean why go to all of the trouble of buying security guards, dogs and a giant fence when users can query it anyway! You would only allow certain operations on certain parts of the data.
In summary ... 'defence in depth' is how you respond to it. There is always a risk that can be identified, but you need to consider the number of controls in place, add more if the risk is too high. But adding more controls to your organisation in general makes the system less user friendly.