I'm writing a small EFI application that oversees deployment of new computers, and among other things prepares the harddisk with a minimal partition table and a rescue system.
Now I'd like to generate random UUIDs for the partition table and partition IDs, but to my astonishment there doesn't seem to be a readymade UUID generator in the standard set of protocols available.
Have I overlooked something here? Which algorithm has the best chance of giving me unique IDs if I don't have a reliable system clock yet, but I have the EFI random generator protocol available?
A GUID is only statistically guaranteed to be unique if generated to a specific algorithm. For UEFI, that means following RFC 4122 to generate a time-based UUID but with TimeLow, TimeMid and TimeHighAndVersion in little-endian byte order. This is also the format used by Microsoft.
There are tools that will generate conformant GUIDs for you.
The Online GUID Generator is one. Another is the not-very-maintained-but-sort-of-useful EDK2 UEFI Driver Wizard, a Python-based GUI application that auto-generates the boilerplate for new edk2 modules.
The linux command line utility uuidgen command does not generate conformant GUIDs, since the timestamp is encoded big-endian.
Related
TL;DR: Are the IDs that are auto-generated by DocumentDB supposed to be GUIDs or UUIDs, and is there actually a difference? If they are UUIDs, then which variant/version of UUID?
Background: Some of the DocumentDB client libraries will auto generate an ID for you if you do not provide one. I have seen it mentioned in the Azure blog and in several related questions that the generated IDs are GUIDs. I know there is some discussion over whether GUIDs are UUIDs, with many people saying that they are.
The problem: However, I have noticed that some of the IDs that DocumentDB auto-generates do not follow the UUID RFC, which allows only the digits 1-5 in the "version" nibble (V in xxxxxxxx-xxxx-Vxxx-xxxx-xxxxxxxxxxxx). DocumentDB generates IDs with any hex digit in that nibble, for example d981befd-d19b-ee48-35bd-c1b507d3ec4f, whose version nibble is the first e of ee48.
It is possible that this depends on which client is used to create the documents. In our DocumentDB database, we have documents with the third grouping dde5, 627a, fe95, and so on. These documents were stored from within a stored procedure by calling Collection.createDocument() with the options {'disableAutomaticIdGeneration': false}. Other documents that I create through the third party DocumentDB Studio application always have 4xxx in the third grouping, which is a valid UUID version. However, documents that I create through the Azure portal have non-standard third groupings like b359.
Question: Are the auto-generated DocumentDB IDs supposed to be GUIDs or UUIDs, and is there actually a difference? If UUIDs, then which variant?
Poking around in the source code on GitHub, I found that the various client and server side libraries use several different methods for creating what they're calling a GUID (in some libraries) or a UUID (in other libraries).
The nodejs client, Javascript client, and server-side library manufacture what they call a GUID by concatenating series of hex digits and hyphens. Note that these are random, but do not comply with the rules for creating RFC4122 version 4 UUIDs.
The Python client and Java client call their respective standard library methods to generate a random (version 4) UUID.
The .NET client is available via NuGet, but the source code is not yet published.
Summary:
Microsoft is not making a distinction between GUID and UUID in their client libraries. They are using the terms interchangeably.
What you get for a GUID/UUID depends on which client library you're using to call DocumentDB when you create your documents.
I'm currently developing an Orchard CMS platform for one of our customers, a small club of about 400 members. After another customer recently had their website hacked and user data leaked (for the record: we didn't develop their website), we want to provide optimal security to our current and new clients. This does not mean the best security, we just want to make the site secure enough that whatever data can be found in the website isn't worth the effort the attacker has to do to get it. Since we only have a small member pool for this website, I doubt it's attractive to begin with, but considering the recent events, we can't be too sure.
One of the subjects we're studying is password storage. We don't want to skimp on this because this is a vital part of security. I've read up on SHA512, BCrypt and other hashing algorithms, but it's hard to form a clear overview of all the different algorithms and methods found online. The information is spread out across different sources, some of which can be found on MSDN, but some of which can only be found hidden in blog posts and comments.
What algorithms for hashing and salthashing passwords and other sensitive data are currently viewed as the best? And which ones are to be avoided like the plague?
PBKDF2/RFC2898/PKCS#5v2, BCrypt, and SCrypt are the current "good" ways to store passwords. All other ways are bad as of early 2014.
Looking at Orchard SetPassword, it appears to be a single round of salted SHA-1. This is pitiful! See How to securely hash passwords? for the general treatise.
In your case, if you want to stick with pure .NET, I would say use the Rfc2898DeriveBytes Class, which is a PBKDF2-HMAC-SHA-1 implementation, feed it a random per-user 16 byte salt generated by the RNGCryptoServiceProvider Class (the same class Orchard uses to generate its salt), and as many iterations as you can use expect to be just below a CPU bound server at maximum expected load - that number should be in the tens of thousands.
Store the password hash, the plaintext (binary or hex string or Base64 string) salt, and the number of iterations in the database. The number of itersations should be stored so you can easily increase security later. If possible, also store a "Version" so that you can easily upgrade to a new algorithm later.
If an encryption function requires a key, how do you obfuscate the key in your source so that decompilation will not reveal the key and thereby enable decryption?
The answer to large extent depends on the platform and development tool, but in general there's no reliable solution. Encryption function is the point at which the key must be present in it's "natural" form. So all the hacker needs to do is to put the breakpoint there and dump the key. There's no need to even decompile anything. Consequently any obfuscation is only good for newbies or when debugging is not possible for whatever reason. Using the text string that exists in the application as the key is one of variants.
But the best approach is not to have the key inside, of course. Depending on your usage scenario you sometimes can use some system information (eg. smartphone's IMEI) as the key. In other cases you can generate the key when the application is installed and store that key as an integral part of your application data (eg. use column names of your DB as the key, or something similar).
Still, as said, all of this is tracked relatively easy when one can run the debugger.
There's one thing to counteract debugging -- offload decryption to third-party. This can be done by employing external cryptodevice (USB cryptotoken or smartcard) or by calling a web service to decrypt certain parts of information. Of course, there methods are also suitable only for a limited set of scenarios.
Encryption is built into the .NET configuration system. You can encrypt chunks of your app/web.config file, including where you store your private key.
http://www.dotnetprofessional.com/blog/post/2008/03/03/Encrypt-sections-of-WebConfig-or-AppConfig.aspx
source
I am going to write my own encryption, but would like to discuss some internals. Should be employed on several mobile platforms - iOS, Android, WP7 with desktop serving more or less as a test platform.
Let's start first with brief characteristics of existing solutions:
SQLite standard (commercial) SEE extension - I have no idea how it works internally and how it co-operates with mentioned mobile platforms.
System.data.sqlite (Windows only): RC4 encyption of the complete DB, ECB mode. They encrypt also DB header, which occasionally (0.01% chance) leads to DB corruption.*) Additional advantage: They use SQLite amalgamation distribution.
SqlCipher (openssl, i.e. several platforms): Selectable encryption scheme. They encrypt whole DB. CBC mode (I think), random IV vector. Because of this, they must modify page parameters (size + reserved space to store IV). They realized the problems related to unencrypted reading of the DB header and tried to introduce workarounds, yet the solution is unsatisfactory. Additional disadvantage: They use SQLite3 source tree. (Which - on the other hand - enables additional features, i.e. fine tuning of the encryption parameters using special pragmas.)
Based on my own analysis I think the following could be a good solution that would not suffer above mentioned problems:
Encrypting whole DB except the DB header.
ECB mode: Sounds risky, but after briefly looking at the DB format I cannot imagine how this could be exploited for an attack.
AES128?
Implementation on top of the SQLite amalgamation (similarly as system.data.sqlite)
I'd like to discuss possible problems of this encryption scheme.
*) Due to SQLite reading DB header without decryption. Due to RC4 (a stream cipher) this problem will manifest at the very first use only. AES would be a lot more dangerous as every "live" DB would sooner or later face this problem.
EDITED - case of VFS-based encryption
Above mentioned methods use codec-based methodology endorsed by sqlite.org. It is a set of 3 callbacks, the most important being this one:
void *(*xCodec)(void *iCtx, void *data, Pgno pgno, int mode)
This callback is used at SQLite discretion for encrypting/decrypting data read from/written to the disk. The data is exchanged page by page. (Page is a multiple of 512 By.)
Alternative option is to use VFS. VFS is a set of callbacks used for low-level OS-services. Among them there are several file-related services, e.g. xOpen/xSeek/xRead/xWrite/xClose. In particular, here are the methods used for data exchange
int (*xRead)(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
int (*xWrite)(sqlite3_file*, const void*, int iAmt, sqlite3_int64 iOfst);
Data size in these calls ranges from 4 By (frequent case) to the DB page size. If you want to use a block cipher (what else to use?), then you need to organize underlying block cache. I cannot imagine an implementation that would be as safe and as efficient as SQLite built-in transactions.
Second problem: VFS implementation is platform-dependent. Android/iOS/WP7/desktop all use different sources, i.e. VFS-based encryption would have to be implemented platform-by-platform.
Next problem is a more subtle: Platform may use VFS calls to realize file locks. These uses must not be encrypted. More over, shared locks must not be buffered. In other words, encryption at the VFS level might compromise locking functionality.
EDITED - plaintext attack on VFS-based encryption
I realized this later: DB header starts with fixed string "SQLite format 3" and the header contains a lot of other fixed byte values. This opens the door for known plaintext attacks (KPA).
This is mainly the problem of VFS-based encryption as it does not have the info that the DB header is being encrypted.
System.data.sqlite has also this problem as it encrypts (RC4) also the DB header.
SqlCipher overwrites hdr string with salt used to convert password to the key. Moreover, it uses by default AES, hence KPA attack presents no danger.
You don't need to hack db format or sqlite source code. SQLite exposes virtual file-system (vfs) API, which can be used to wrap file system (or another vfs) with encryption layer which encrypts/decrypts pages on the fly. When I did that it turned out to be very simple task, just hundred lines of code or so. This way whole DB will be encrypted, including journal file, and it is completely transparent to any client code. With typical page size of 1024, almost any known block cipher can be used. From what I can conclude from their docs, this is exactly what SQLCipher does.
Regarding the 'problems' you see:
You don't need to reimplement file system support, you can wrap around the default VFS. So no problems with locks or platform-dependence.
SQLite's default OS backend is also VFS, there is no overhead for using VFS except that you add.
You don't need block cache. Of course you will have to read whole block when it asks for just 4 bytes, but don't cache it, it will never be read again. SQLite has its own cache to prevent that (Pager module).
Didn't get much response, so here is my decision:
Own encryption (AES128), CBC mode
Codec interface (same as used by SqlCipher or system.data.sqlite)
DB header unencrypted
Page headers unencrypted as well and used for IV generation
Using amalgamation SQLite distribution
AFAIK this solution should be better than either SqlCipher or system.data.sqlite.
I am looking for a programmatic way to generate consumer key/secrets for HMAC-SHA1 to be used by clients invoking our API over OAuth. Any pointers to existing implementations would be highly helpful.
Secrets are best when generated from random data. That way there is no external data which could help an attacker deduce or guess part or the entire key. Of course, it depends on how much protection your secret key needs. Java includes some random number generators in java.util.Random (since JDK1.0). If you don't have backward compatibility issues, Java 6 has java.security.SecureRandom which meets FIPS 140-2 requirements. The Java libraries are not truly random, but it is probably good enough for most applications. If you need better random data, you should go for a hardware-based random generator.