I'm writing a little program that stores passwords in Python. I access to the DB with Mysqldb module.
I want to encrypt the password with AES 256 with CBC. I'm trying to do this with SQL AES_ENCRYPT function.
I set the cbc mode with:
mysql> SET block_encryption_mode = 'aes-256-cbc';
So in python, I have:
def add_password(self, table, site, password, key, iv):
try:
query = "INSERT INTO " +table+ " (sites, password) VALUES (AES_ENCRYPT(%s, %s, %s), AES_ENCRYPT(%s,%s,%s) )"
self.cursor.execute(query, ( site, key, iv, password, key, iv))
self.conn.commit()
...
But, if a run the program I get:
Warning: <IV> option ignored, as if the function uses ECB mode (so without iv).
Where I wrong?
Possible way:
Or is it better encrypt all with python modules and the insert into the DB the data already encrypted?
EDIT
Seems that if I add every time SET block_encryption_mode = 'aes-256-cbc', all woks.
def add_password(self, table, site, password, key,iv):
try:
query = "SET block_encryption_mode = 'aes-256-cbc'"
self.cursor.execute(query)
query = "INSERT INTO " +table+ " (sites, password) VALUES (AES_ENCRYPT(%s, %s, %s), AES_ENCRYPT(%s,%s,%s) )"
self.cursor.execute(query, ( site, key,iv, password, key, iv))
self.conn.commit()
...
Related
Problem Statement : I would like write a BigQuery UDF to de·crypt table columns
Setup :
PII information is encrypted in table columns with tink package
KEK (Key Encryption Key ) is sitting in KMS
DEK (data encryption Key) is sitting in Cloud storage
I have written BigQuery EXTERNAL Table to access DEK json i.e. select encryptedKeyset from my_project.my_dataset.external_table_for_decrypted_keys will give me required dek key
This is my sample code
CREATE OR REPLACE FUNCTION my_proj.my_dataset.udf_decrypt_column(table_name string, column_name string)
BEGIN
DECLARE KMS_RESOURCE_NAME STRING;
DECLARE FIRST_LEVEL_KEYSET STRING;
SET KMS_RESOURCE_NAME= "gcp-kms://projects/dev/locations/us/keyRings/dev/cryptoKeys/dev-kek";
SET FIRST_LEVEL_KEYSET = (select encryptedKeyset from my_project.my_dataset.external_table_for_decrypted_keys`);
SELECT
AEAD.DECRYPT_STRING(KEYS.KEYSET_CHAIN(KMS_RESOURCE_NAME,
from_base64(FIRST_LEVEL_KEYSET)),
from_base64(column_name),
"") as decrypted_name
FROM table_name
Issues/Question :
Declare variables do not work in Functions (while they work in procedures). So my question is how to assign values to variables in UDF
How to run SQL & assign value to variable in UDF. In my case I want to fetch column encryptedKeyset from external_table_for_decrypted_keys & assign to FIRST_LEVEL_KEYSET (in declare section)
Any idea/pointer how to achieve this ? Thanks in advance for your reply.
Our team is currently exploring the ways to encrypt PII data on the field level within BigQuery and we found out the following way to encrypt/decrypt using Crypto-JS:
#standardSQL
CREATE TEMPORARY FUNCTION encrypt(_text STRING) RETURNS STRING LANGUAGE js AS
"""
let key = CryptoJS.enc.Utf8.parse("<key>");
let options = { iv: CryptoJS.enc.Utf8.parse("<iv>"), mode: CryptoJS.mode.CBC };
let _encrypt = CryptoJS.AES.encrypt(_text, key, options);
return _encrypt;
""";
CREATE TEMPORARY FUNCTION decrypt(_text STRING) RETURNS STRING LANGUAGE js AS
"""
let key = CryptoJS.enc.Utf8.parse("<key>");
let options = { iv: CryptoJS.enc.Utf8.parse("<iv>"), mode: CryptoJS.mode.CBC };
let _decrypt = CryptoJS.AES.decrypt(_text, key, options).toString(CryptoJS.enc.Utf8);
return _decrypt;
""" OPTIONS (library="gs://path/to/Crypto-JS/crypto-js.js");
-- query to encrypt fields
SELECT
<fields>, encrypt(<pii-fields>)
FROM
`<project>.<dataset>.<table>`
-- query to decrypt fields
SELECT
<fields>, decrypt(<pii-fields>)
FROM
`<project>.<dataset>.<table>`
I am trying to benchmark the performance of AES CBC encryption & decryption using Crypto JS library in the big query before deploying it into our production. We found out the rate of data to encrypt & decrypt is growing exponential per records with increasing number of data compared to the usual query. However with the increasing number of data to process, the progress of processing per record & record processing time is improving.
As there are no available documentation regarding this, could someone from the community help provide better ways, optimize query, best practices to use field level encryption & decryption within the big query?
BigQuery now supports encryption functions. From the documentation, here is a self-contained example that creates some keysets and uses them to encrypt data. In practice, you would want to store the keysets in a real table so that you can later use them to decrypt the ciphertext.
WITH CustomerKeysets AS (
SELECT 1 AS customer_id, KEYS.NEW_KEYSET('AEAD_AES_GCM_256') AS keyset UNION ALL
SELECT 2, KEYS.NEW_KEYSET('AEAD_AES_GCM_256') UNION ALL
SELECT 3, KEYS.NEW_KEYSET('AEAD_AES_GCM_256')
), PlaintextCustomerData AS (
SELECT 1 AS customer_id, 'elephant' AS favorite_animal UNION ALL
SELECT 2, 'walrus' UNION ALL
SELECT 3, 'leopard'
)
SELECT
pcd.customer_id,
AEAD.ENCRYPT(
(SELECT keyset
FROM CustomerKeysets AS ck
WHERE ck.customer_id = pcd.customer_id),
pcd.favorite_animal,
CAST(pcd.customer_id AS STRING)
) AS encrypted_animal
FROM PlaintextCustomerData AS pcd;
Edit: if you want to decrypt using AES-CBC with PKCS padding (it's not clear what kind of padding you are using in your example) you can use the KEYS.ADD_KEY_FROM_RAW_BYTES function to create a keyset, then call AEAD.DECRYPT_STRING or AEAD.DECRYPT_BYTES. For example:
SELECT
AEAD.DECRYPT_STRING(
KEYS.ADD_KEY_FROM_RAW_BYTES(b'', 'AES_CBC_PKCS', b'1234567890123456'),
FROM_HEX('deed2a88e73dccaa30a9e6e296f62be27db30db16f76d3f42c85d31db3f46376'),
'')
This returns abcdef. The IV is expected to be the first 16 bytes of the ciphertext.
I've tried to write it but it didn't work.
I have a table called files and a column called Username and a string called str.
I want a simple thing - if the str equals to anything in the Username column, then delete it (there can not be double values, so don't worry about that).
I wrote it like that:
IF EXISTS (SELECT * FROM files WHERE Username = #username)
BEGIN
DELETE FROM files
WHERE Username = #username
#username = str
It doesn't work, I hope you can assist.
Thanks.
You don't need to check whether any rows exist:
DELETE FROM [files] WHERE Username = #username
If there are no rows, they won't be deleted!
Read this:
https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
On the dynamic sql part it has various such as this:
So, if you had an existing Dynamic query being generated in your code that was going to Oracle that looked like this:
String query = "SELECT user_id FROM user_data WHERE user_name = '" + req.getParameter("userID")
+ "' and user_password = '" + req.getParameter("pwd") +"'";
try {
Statement statement = connection.createStatement( … );
ResultSet results = statement.executeQuery( query );
}
You would rewrite the first line to look like this:
Codec ORACLE_CODEC = new OracleCodec();
String query = "SELECT user_id FROM user_data WHERE user_name = '" +
ESAPI.encoder().encodeForSQL( ORACLE_CODEC, req.getParameter("userID")) + "' and user_password = '"
+ ESAPI.encoder().encodeForSQL( ORACLE_CODEC, req.getParameter("pwd")) +"'";
And it would now be safe from SQL injection, regardless of the input supplied.
But the later is says:
Oracle 10g escaping
An alternative for Oracle 10g and later is to place { and } around the string to escape the entire string. However, you have to be careful that there isn't a } character already in the string. You must search for these and if there is one, then you must replace it with }}. Otherwise that character will end the escaping early, and may introduce a vulnerability.
I did not see an example, but does this mean I can use braces instead of the Codec ORACLE_CODEC....etc.? Does anyone have an example? Thanks.
No, this is not an injection prevention technique. The only way to be 100% sure that you're not vulnerable to injection is to use prepared statements and bind parameters for all user input that needs to be inserted into the query. Anything less than that, and you're pretty much just rolling the dice.
In SQLite I can run the following query to get a list of columns in a table:
PRAGMA table_info(myTable)
This gives me the columns but no information about what the primary keys may be. Additionally, I can run the following two queries for finding indexes and foreign keys:
PRAGMA index_list(myTable)
PRAGMA foreign_key_list(myTable)
But I cannot seem to figure out how to view the primary keys. Does anyone know how I can go about doing this?
Note: I also know that I can do:
select * from sqlite_master where type = 'table' and name ='myTable';
And it will give the the create table statement which shows the primary keys. But I am looking for a way to do this without parsing the create statement.
The table_info DOES give you a column named pk (last one) indicating if it is a primary key (if so the index of it in the key) or not (zero).
To clarify, from the documentation:
The "pk" column in the result set is zero for columns that are not
part of the primary key, and is the index of the column in the primary
key for columns that are part of the primary key.
Hopefully this helps someone:
After some research and pain the command that worked for me to find the primary key column name was:
SELECT l.name FROM pragma_table_info("Table_Name") as l WHERE l.pk = 1;
For the ones trying to retrieve a pk name in android, and while using the ROOM library.
#Oogway101's answer was throwing an error: "no such column [your_table_name] ... etc.. etc...
my way of query submition was:
String pkSearch = "SELECT l.name FROM pragma_table_info(" + tableName + ") as l WHERE l.pk = 1;";
database.query(new SimpleSQLiteQuery(pkSearch)
I tried using the (") quotations and still error.
String pkSearch = "SELECT l.name FROM pragma_table_info(\"" + tableName + "\") as l WHERE l.pk = 1;";
So my solution was this:
String pragmaInfo = "PRAGMA table_info(" + tableName + ");";
Cursor c = database.query(new SimpleSQLiteQuery(pragmaInfo));
String id = null;
c.moveToFirst();
do {
if (c.getInt(5) == 1) {
id = c.getString(1);
}
} while (c.moveToNext() && id == null);
Log.println(Log.ASSERT, TAG, "AbstractDao: pk is: " + id);
The explanation is that:
A) PRAGMA table_info returns a cursor with various indices, the response is atleast of length 6... didnt check more...
B) index 1 has the column name.
C) index 5 has the "pk" value, either 0 if it is not a primary key, or 1 if its a pk.
You can define more than one pk so this will not bring an accurate result if your table has more than one (IMHO more than one is bad design and balloons the complexity of the database beyond human comprehension).
So how will this fit into the #Dao? (you may ask...)
When making the Dao "abstract" you have access to a default constructor which has the database in it:
from the docummentation:
An abstract #Dao class can optionally have a constructor that takes a Database as its only parameter.
this is the constructor that will grant you access to the query.
There is a catch though...
You may use the Dao during a database creation with the .addCallback() method:
instance = Room.databaseBuilder(context.getApplicationContext(),
AppDatabase2.class, "database")
.addCallback(
//You may use the Daos here.
)
.build();
If you run a query in the constructor of the Dao, the database will enter a feedback loop of infinite instantiation.
This means that the query MUST be used LAZILY (just at the moment the user needs something), and because the value will never change, it can be stored. and never re-queried.