Related
On web, I'm using CryptoJS for decrypto JS:
CryptoJS.AES.decrypt(inputBase64, key).toString(CryptoJS.enc.Utf8);
Example:
input: "tzfwnxVwE/qNoaWRRfqLp11ZyhB4UtKO+0/Lvv5B7eE="
key: "20190225165436_15230006321670000_15510884759030000"
On flutter I can't find any library to decrypt with a key of any length.
I know "For AES, NIST selected three members of the Rijndael family, each with a block size of 128 bits, but three different key lengths: 128, 192 and 256 bits.
"
But I don't know how to convert any length key to 128 bit format?
import 'dart:convert';
import 'dart:math';
import 'dart:typed_data';
import 'package:crypto/crypto.dart';
import 'package:tuple/tuple.dart';
import 'package:encrypt/encrypt.dart' as encrypt;
String encryptAESCryptoJS(String plainText, String passphrase) {
try {
final salt = genRandomWithNonZero(8);
var keyndIV = deriveKeyAndIV(passphrase, salt);
final key = encrypt.Key(keyndIV.item1);
final iv = encrypt.IV(keyndIV.item2);
final encrypter = encrypt.Encrypter(
encrypt.AES(key, mode: encrypt.AESMode.cbc, padding: "PKCS7"));
final encrypted = encrypter.encrypt(plainText, iv: iv);
Uint8List encryptedBytesWithSalt = Uint8List.fromList(
createUint8ListFromString("Salted__") + salt + encrypted.bytes);
return base64.encode(encryptedBytesWithSalt);
} catch (error) {
throw error;
}
}
String decryptAESCryptoJS(String encrypted, String passphrase) {
try {
Uint8List encryptedBytesWithSalt = base64.decode(encrypted);
Uint8List encryptedBytes =
encryptedBytesWithSalt.sublist(16, encryptedBytesWithSalt.length);
final salt = encryptedBytesWithSalt.sublist(8, 16);
var keyndIV = deriveKeyAndIV(passphrase, salt);
final key = encrypt.Key(keyndIV.item1);
final iv = encrypt.IV(keyndIV.item2);
final encrypter = encrypt.Encrypter(
encrypt.AES(key, mode: encrypt.AESMode.cbc, padding: "PKCS7"));
final decrypted =
encrypter.decrypt64(base64.encode(encryptedBytes), iv: iv);
return decrypted;
} catch (error) {
throw error;
}
}
Tuple2<Uint8List, Uint8List> deriveKeyAndIV(String passphrase, Uint8List salt) {
var password = createUint8ListFromString(passphrase);
Uint8List concatenatedHashes = Uint8List(0);
Uint8List currentHash = Uint8List(0);
bool enoughBytesForKey = false;
Uint8List preHash = Uint8List(0);
while (!enoughBytesForKey) {
int preHashLength = currentHash.length + password.length + salt.length;
if (currentHash.length > 0)
preHash = Uint8List.fromList(
currentHash + password + salt);
else
preHash = Uint8List.fromList(
password + salt);
currentHash = md5.convert(preHash).bytes;
concatenatedHashes = Uint8List.fromList(concatenatedHashes + currentHash);
if (concatenatedHashes.length >= 48) enoughBytesForKey = true;
}
var keyBtyes = concatenatedHashes.sublist(0, 32);
var ivBtyes = concatenatedHashes.sublist(32, 48);
return new Tuple2(keyBtyes, ivBtyes);
}
Uint8List createUint8ListFromString(String s) {
var ret = new Uint8List(s.length);
for (var i = 0; i < s.length; i++) {
ret[i] = s.codeUnitAt(i);
}
return ret;
}
Uint8List genRandomWithNonZero(int seedLength) {
final random = Random.secure();
const int randomMax = 245;
final Uint8List uint8list = Uint8List(seedLength);
for (int i=0; i < seedLength; i++) {
uint8list[i] = random.nextInt(randomMax)+1;
}
return uint8list;
}
Usage
import 'package:app/utils/cryptojs_aes_encryption_helper.dart';
String plainText = 'PlainText is Me';
var encrypted = encryptAESCryptoJS(plainText, "password");
var decrypted = decryptAESCryptoJS(encrypted, "password");
When you pass CryptoJS a string as the key it treats it as a passphrase and generates the key from it using a key derivation function - in this case PBKDF2. It generates a 256 bit key and a 128 bit initialization vector (IV). It then uses those for the encryption/decryption. You also need to find out what chaining method CryptoJS uses (probably cipher block chaining (CBC)) and what padding method it uses (to make sure that the plain text is a round number of 128 bit blocks - probably PKCS#7).
CryptoJS has this "works out of the box" mode, but it isn't particularly clear what it's doing under the hood - you'd need to read the source code or scour the documentation.
When trying to inter-operate between two systems/languages it's best if you remain in charge of things, rather than letting one end make arbitrary decisions. That way you can make sure that you have the settings the same at each end.
So, you might choose to:
Use PBKDF2 to generate a 128 bit key and 128 bit IV from the string
passphrase - with, say, 9999 rounds
Use PKCS#7 padding
Use AES in CBC mode
The pointycastle package supports all the above in Dart. It looks like CryptoJS supports all of those too.
Start with a passphrase and make sure you can generate the same key and IV in JS and Dart. Then move onto creating the ciphers.
Remember, too, never to encrypt two messages with the same key/IV pair. Use a message sequence number, for example, to slightly change the IV for each message.
Sample encrypt using nodejs script:
var key = CryptoJS.PBKDF2("123456", "123456", {
keySize: 256 / 32
});
var iv = CryptoJS.PBKDF2("123456", "123456", {
keySize: 128 / 32
});
var encrypted = CryptoJS.AES.encrypt('my message', key, { iv: iv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
}).toString();
decript using dart
import 'package:encrypt/encrypt.dart' as aes;
import 'package:crypto/crypto.dart';
import 'package:hex/hex.dart';
import 'package:password_hash/pbkdf2.dart';
void main(List<String> arguments) {
String encrypted = 'HbsmGAigiIWmU3MNZAf8+w==';
final generator = PBKDF2(hashAlgorithm: sha1);
final key = aes.Key.fromBase16(HEX.encode(generator.generateKey("123456", "123456", 1, 32)));
final iv = aes.IV.fromBase16(HEX.encode(generator.generateKey("123456", "123456", 1, 16)));
final encrypter = aes.Encrypter(aes.AES(key, mode: aes.AESMode.cbc, padding: 'PKCS7'));
final decrypted = encrypter.decrypt64(encrypted, iv:iv);
print(decrypted);
}
Long Story:
There is an H264/MPEG-4 Source
I can able to connect this source with RTSP protocol.
I can able to get raw UDP packets with RTP protocol.
Then send those raw UDP packets to a Decoder[h264/mpeg-4] [DS Source Filter]
But those "raw" UDP packets can not be decoded by the Decoder[h264/mpeg-4] filter
Shortly:
How do I process those raw UDP data in order to be decodable by H264/ MPEG-4 decoder filter? Can any one clearly identify steps I have to do with H264/MPEG stream?
Extra Info:
I am able to do this with FFmpeg... But I can not really figure out how FFmpeg processes the raw data so that is decodable by a decoder.
Peace of cake!
1. Get the data
As I can see, you already know how to do that (start RTSP session, SETUP a RTP/AVP/UDP;unicast; transport, and get user datagrams)... but if you are in doubt, ask.
No matter the transport (UDP or TCP) the data format is mainly the same:
RTP data: [RTP Header - 12bytes][Video data]
UDP: [RTP Data]
TCP: [$ - 1byte][Transport Channel - 1byte][RTP data length - 2bytes][RTP
data]
So to get data from UDP, you only have to strip off first 12 bytes which represent RTP header. But beware, you need it to get video timing information, and for MPEG4 the packetization information!
For TCP you need to read first byte until you get byte $. Then read next byte, that will be transport channel that the following data belongs (when server responds on SETUP request it says: Transport: RTP/AVP/TCP;unicast;interleaved=0-1 this means that VIDEO DATA will have TRANSPORT_CHANNEL=0 and VIDEO RTCP DATA will have TRANSPORT_CHANNEL=1). You want to get VIDEO DATA, so we expect 0... then read one short (2 byte) that represents the length of the RTP data that follows, so read that much bytes, and now do the same as for UDP.
2. Depacketize data
H264 and MPEG4 data are usually packetized (in SDP there is packetization-mode parameter that can have values 0, 1 and 2 what each of them means, and how to depacketize it, you can see HERE) because there is a certain network limit that one endpoint can send through TCP or UDP that is called MTU. It is usually 1500 bytes or less. So if the video frame is larger than that (and it usually is), it needs to be fragmented (packetized) into MTU sized fragments. This can be done by encoder/streamer on TCP and UDP transport, or you can relay on IP to fragment and reassemble video frame on the other side... the first is much better if you want to have a smooth error prone video over UDP and TCP.
H264: To check does the RTP data (which arrived over UDP, or interleaved over TCP) hold fragment of one larger H264 video frame, you must know how the fragment looks when it is packetized:
H264 FRAGMENT
First byte: [ 3 NAL UNIT BITS | 5 FRAGMENT TYPE BITS]
Second byte: [ START BIT | END BIT | RESERVED BIT | 5 NAL UNIT BITS]
Other bytes: [... VIDEO FRAGMENT DATA...]
Now, get the first VIDEO DATA in byte array called Data and get the following info:
int fragment_type = Data[0] & 0x1F;
int nal_type = Data[1] & 0x1F;
int start_bit = Data[1] & 0x80;
int end_bit = Data[1] & 0x40;
If fragment_type == 28 then video data following it represents the video frame fragment. Next check is start_bit set, if it is, then that fragment is the first one in a sequence. You use it to reconstruct IDR's NAL byte by taking the first 3 bits from first payload byte (3 NAL UNIT BITS) and combine them with last 5 bits from second payload byte (5 NAL UNIT BITS) so you would get a byte like this [3 NAL UNIT BITS | 5 NAL UNIT BITS]. Then write that NAL byte first into a clear buffer with VIDEO FRAGMENT DATA from that fragment.
If start_bit and end_bit are 0 then just write the VIDEO FRAGMENT DATA (skipping first two payload bytes that identify the fragment) to the buffer.
If start_bit is 0 and end_bit is 1, that means that it is the last fragment, and you just write its VIDEO FRAGMENT DATA (skipping the first two bytes that identify the fragment) to the buffer, and now you have your video frame reconstructed!
Bare in mind that the RTP data holds RTP header in first 12 bytes, and that if the frame is fragmented, you never write first two bytes in the defragmentation buffer, and that you need to reconstruct NAL byte and write it first. If you mess something up here, the picture will be partial (half of it will be gray or black or you will see artifacts).
MPEG4:
This is an easy one. You need to check the MARKER_BIT in RTP Header. That byte is set (1) if the video data represents the whole video frame, and it is 0 of the video data is one video frame fragment. So to depacketize that, you need to see what the MARKER_BIT is. If it is 1 thats it, just read the video data bytes.
WHOLE FRAME:
[MARKER = 1]
PACKETIZED FRAME:
[MARKER = 0], [MARKER = 0], [MARKER = 0], [MARKER = 1]
First packet that has MARKER_BIT=0 is the first video frame fragment, all others that follow including the first one with MARKER_BIT=1 are fragments of the same video frame. So what you need to do is:
Until MARKER_BIT=0 place VIDEO DATA in depacketization buffer
Place next VIDEO DATA where MARKER_BIT=1 into the same buffer
Depacketization buffer now holds one whole MPEG4 frame
3. Process data for decoder (NAL byte stream)
When you have depacketized video frames, you need to make NAL byte stream. It has the following format:
H264: 0x000001[SPS], 0x000001[PPS], 0x000001[VIDEO FRAME], 0x000001...
MPEG4: 0x000001[Visual Object Sequence Start], 0x000001[VIDEO FRAME]
RULES:
Every frame MUST be prepended with 0x000001 3 byte code no matter the codec
Every stream MUST start with CONFIGURATION INFO, for H264 that are SPS and PPS frames in that order (sprop-parameter-sets in SDP), and for MPEG4 the VOS frame (config parameter in SDP)
So you need to build a config buffer for H264 and MPEG4 prepended with 3 bytes 0x000001, send it first, and then prepend each depacketized video frame with the same 3 bytes and send that to the decoder.
If you need any clarifying just comment... :)
I have an implementation of this # https://net7mma.codeplex.com/
Here is the relevant code
/// <summary>
/// Implements Packetization and Depacketization of packets defined in <see href="https://tools.ietf.org/html/rfc6184">RFC6184</see>.
/// </summary>
public class RFC6184Frame : Rtp.RtpFrame
{
/// <summary>
/// Emulation Prevention
/// </summary>
static byte[] NalStart = { 0x00, 0x00, 0x01 };
public RFC6184Frame(byte payloadType) : base(payloadType) { }
public RFC6184Frame(Rtp.RtpFrame existing) : base(existing) { }
public RFC6184Frame(RFC6184Frame f) : this((Rtp.RtpFrame)f) { Buffer = f.Buffer; }
public System.IO.MemoryStream Buffer { get; set; }
/// <summary>
/// Creates any <see cref="Rtp.RtpPacket"/>'s required for the given nal
/// </summary>
/// <param name="nal">The nal</param>
/// <param name="mtu">The mtu</param>
public virtual void Packetize(byte[] nal, int mtu = 1500)
{
if (nal == null) return;
int nalLength = nal.Length;
int offset = 0;
if (nalLength >= mtu)
{
//Make a Fragment Indicator with start bit
byte[] FUI = new byte[] { (byte)(1 << 7), 0x00 };
bool marker = false;
while (offset < nalLength)
{
//Set the end bit if no more data remains
if (offset + mtu > nalLength)
{
FUI[0] |= (byte)(1 << 6);
marker = true;
}
else if (offset > 0) //For packets other than the start
{
//No Start, No End
FUI[0] = 0;
}
//Add the packet
Add(new Rtp.RtpPacket(2, false, false, marker, PayloadTypeByte, 0, SynchronizationSourceIdentifier, HighestSequenceNumber + 1, 0, FUI.Concat(nal.Skip(offset).Take(mtu)).ToArray()));
//Move the offset
offset += mtu;
}
} //Should check for first byte to be 1 - 23?
else Add(new Rtp.RtpPacket(2, false, false, true, PayloadTypeByte, 0, SynchronizationSourceIdentifier, HighestSequenceNumber + 1, 0, nal));
}
/// <summary>
/// Creates <see cref="Buffer"/> with a H.264 RBSP from the contained packets
/// </summary>
public virtual void Depacketize() { bool sps, pps, sei, slice, idr; Depacketize(out sps, out pps, out sei, out slice, out idr); }
/// <summary>
/// Parses all contained packets and writes any contained Nal Units in the RBSP to <see cref="Buffer"/>.
/// </summary>
/// <param name="containsSps">Indicates if a Sequence Parameter Set was found</param>
/// <param name="containsPps">Indicates if a Picture Parameter Set was found</param>
/// <param name="containsSei">Indicates if Supplementatal Encoder Information was found</param>
/// <param name="containsSlice">Indicates if a Slice was found</param>
/// <param name="isIdr">Indicates if a IDR Slice was found</param>
public virtual void Depacketize(out bool containsSps, out bool containsPps, out bool containsSei, out bool containsSlice, out bool isIdr)
{
containsSps = containsPps = containsSei = containsSlice = isIdr = false;
DisposeBuffer();
this.Buffer = new MemoryStream();
//Get all packets in the frame
foreach (Rtp.RtpPacket packet in m_Packets.Values.Distinct())
ProcessPacket(packet, out containsSps, out containsPps, out containsSei, out containsSlice, out isIdr);
//Order by DON?
this.Buffer.Position = 0;
}
/// <summary>
/// Depacketizes a single packet.
/// </summary>
/// <param name="packet"></param>
/// <param name="containsSps"></param>
/// <param name="containsPps"></param>
/// <param name="containsSei"></param>
/// <param name="containsSlice"></param>
/// <param name="isIdr"></param>
internal protected virtual void ProcessPacket(Rtp.RtpPacket packet, out bool containsSps, out bool containsPps, out bool containsSei, out bool containsSlice, out bool isIdr)
{
containsSps = containsPps = containsSei = containsSlice = isIdr = false;
//Starting at offset 0
int offset = 0;
//Obtain the data of the packet (without source list or padding)
byte[] packetData = packet.Coefficients.ToArray();
//Cache the length
int count = packetData.Length;
//Must have at least 2 bytes
if (count <= 2) return;
//Determine if the forbidden bit is set and the type of nal from the first byte
byte firstByte = packetData[offset];
//bool forbiddenZeroBit = ((firstByte & 0x80) >> 7) != 0;
byte nalUnitType = (byte)(firstByte & Common.Binary.FiveBitMaxValue);
//o The F bit MUST be cleared if all F bits of the aggregated NAL units are zero; otherwise, it MUST be set.
//if (forbiddenZeroBit && nalUnitType <= 23 && nalUnitType > 29) throw new InvalidOperationException("Forbidden Zero Bit is Set.");
//Determine what to do
switch (nalUnitType)
{
//Reserved - Ignore
case 0:
case 30:
case 31:
{
return;
}
case 24: //STAP - A
case 25: //STAP - B
case 26: //MTAP - 16
case 27: //MTAP - 24
{
//Move to Nal Data
++offset;
//Todo Determine if need to Order by DON first.
//EAT DON for ALL BUT STAP - A
if (nalUnitType != 24) offset += 2;
//Consume the rest of the data from the packet
while (offset < count)
{
//Determine the nal unit size which does not include the nal header
int tmp_nal_size = Common.Binary.Read16(packetData, offset, BitConverter.IsLittleEndian);
offset += 2;
//If the nal had data then write it
if (tmp_nal_size > 0)
{
//For DOND and TSOFFSET
switch (nalUnitType)
{
case 25:// MTAP - 16
{
//SKIP DOND and TSOFFSET
offset += 3;
goto default;
}
case 26:// MTAP - 24
{
//SKIP DOND and TSOFFSET
offset += 4;
goto default;
}
default:
{
//Read the nal header but don't move the offset
byte nalHeader = (byte)(packetData[offset] & Common.Binary.FiveBitMaxValue);
if (nalHeader > 5)
{
if (nalHeader == 6)
{
Buffer.WriteByte(0);
containsSei = true;
}
else if (nalHeader == 7)
{
Buffer.WriteByte(0);
containsPps = true;
}
else if (nalHeader == 8)
{
Buffer.WriteByte(0);
containsSps = true;
}
}
if (nalHeader == 1) containsSlice = true;
if (nalHeader == 5) isIdr = true;
//Done reading
break;
}
}
//Write the start code
Buffer.Write(NalStart, 0, 3);
//Write the nal header and data
Buffer.Write(packetData, offset, tmp_nal_size);
//Move the offset past the nal
offset += tmp_nal_size;
}
}
return;
}
case 28: //FU - A
case 29: //FU - B
{
/*
Informative note: When an FU-A occurs in interleaved mode, it
always follows an FU-B, which sets its DON.
* Informative note: If a transmitter wants to encapsulate a single
NAL unit per packet and transmit packets out of their decoding
order, STAP-B packet type can be used.
*/
//Need 2 bytes
if (count > 2)
{
//Read the Header
byte FUHeader = packetData[++offset];
bool Start = ((FUHeader & 0x80) >> 7) > 0;
//bool End = ((FUHeader & 0x40) >> 6) > 0;
//bool Receiver = (FUHeader & 0x20) != 0;
//if (Receiver) throw new InvalidOperationException("Receiver Bit Set");
//Move to data
++offset;
//Todo Determine if need to Order by DON first.
//DON Present in FU - B
if (nalUnitType == 29) offset += 2;
//Determine the fragment size
int fragment_size = count - offset;
//If the size was valid
if (fragment_size > 0)
{
//If the start bit was set
if (Start)
{
//Reconstruct the nal header
//Use the first 3 bits of the first byte and last 5 bites of the FU Header
byte nalHeader = (byte)((firstByte & 0xE0) | (FUHeader & Common.Binary.FiveBitMaxValue));
//Could have been SPS / PPS / SEI
if (nalHeader > 5)
{
if (nalHeader == 6)
{
Buffer.WriteByte(0);
containsSei = true;
}
else if (nalHeader == 7)
{
Buffer.WriteByte(0);
containsPps = true;
}
else if (nalHeader == 8)
{
Buffer.WriteByte(0);
containsSps = true;
}
}
if (nalHeader == 1) containsSlice = true;
if (nalHeader == 5) isIdr = true;
//Write the start code
Buffer.Write(NalStart, 0, 3);
//Write the re-construced header
Buffer.WriteByte(nalHeader);
}
//Write the data of the fragment.
Buffer.Write(packetData, offset, fragment_size);
}
}
return;
}
default:
{
// 6 SEI, 7 and 8 are SPS and PPS
if (nalUnitType > 5)
{
if (nalUnitType == 6)
{
Buffer.WriteByte(0);
containsSei = true;
}
else if (nalUnitType == 7)
{
Buffer.WriteByte(0);
containsPps = true;
}
else if (nalUnitType == 8)
{
Buffer.WriteByte(0);
containsSps = true;
}
}
if (nalUnitType == 1) containsSlice = true;
if (nalUnitType == 5) isIdr = true;
//Write the start code
Buffer.Write(NalStart, 0, 3);
//Write the nal heaer and data data
Buffer.Write(packetData, offset, count - offset);
return;
}
}
}
internal void DisposeBuffer()
{
if (Buffer != null)
{
Buffer.Dispose();
Buffer = null;
}
}
public override void Dispose()
{
if (Disposed) return;
base.Dispose();
DisposeBuffer();
}
//To go to an Image...
//Look for a SliceHeader in the Buffer
//Decode Macroblocks in Slice
//Convert Yuv to Rgb
}
There are also implementations for various other RFC's which help getting the media to play in a MediaElement or in other software or just saving it to disk.
Writing to a container format is underway.
With UDP packets you receive bits of H.264 stream which you are expected to depacketize into H.264 NAL Units, which, in their turn, you are typically pushing into DirectShow pipeline from your filter.
The NAL Units will be formatted as DirectShow media samples, and possibly also, as a part of media type (SPS/PPS NAL Units).
Depacketization steps are described in RFC 6184 - RTP Payload Format for H.264 Video. This is payload part of RTP traffic, defined by RFC 3550 - RTP: A Transport Protocol for Real-Time Applications.
Clear, but not quite short though.
I have recently streamed h264 and encountered similar issues. Here is my depacketizer class. I wrote a long blog post to save other time in understanding this process http://cagneymoreau.com/stream-video-android/
Package networking;
import org.apache.commons.logging.Log;
import utility.Debug;
import java.io.Console;
import java.io.IOException;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.util.*;
/**
* This class is used to re-assemble udp packets filled with rtp packets into network abstraction layer units
*
*/
public class VideoDecoder {
private static final String TAG = "VideoDecoder";
private PipedOutputStream pipedOutputStream; //this is where we pass the nalus we extract
private Map<Integer, NaluBuffer> assemblyLine = new HashMap<>(); // This holds nalus we are building. Ideally only 1 and if it exceeds 3 there might be a problem
private final int thresh = 30;
private int assemblyThresh = thresh;
private final int trashDelay = 3000;
//unpacking
private final static int HEADER_SIZE = 12;
private final static int rtpByteHeader1 = 128; //rtp header byte 1 should always equal
private final static int typeSPSPPS = 24;
private final static byte typeFUA = 0b01111100;
private final static byte[] startcode = new byte[] { 0x00, 0x00, 0x00, 0x01};
//experimental bools that can mix piped data
private boolean annexB = true; //remove lengths and dd aprefix
private boolean mixed = false; //keep lengths and add pefix dont use with annexb
private boolean prelStyle = false; //include avcc 6 byte data
private boolean directPipe = false; //send in the data with no editing
public VideoDecoder(PipedOutputStream pipedOutputStream)
{
this.pipedOutputStream = pipedOutputStream;
}
// raw udp rtp packets come in here from the the udp.packet.getdata filled at socket
public void addPacket(byte[] incoming)
{
if (directPipe){
transferTOFFmpeg(incoming);
return;
}
if (incoming[0] != (byte) rtpByteHeader1){
System.out.println(TAG + " rtpHeaderError " + Byte.toString(incoming[0]));
}
if (incoming[1] == typeSPSPPS){
System.out.println(TAG + "addPacket type: 24" );
unpackType24(incoming);
}
else if (incoming[1] == typeFUA){
//System.out.println(TAG + "addPacket type: 28" );
unpackType28(incoming);
}
else if (incoming[1] == 1){
System.out.println(TAG + "addPacket type: 1" );
unpackType1(incoming);
}else if (incoming[1] == 5){
System.out.println(TAG + "addPacket type: 5" );
unpackType5(incoming);
}else{
System.out.println(TAG + "addPacket unknown type - ERROR " + String.valueOf(incoming[1]) );
}
}
//SPS & PPS this will get hit before every type 5
//im not rtp compliant.
// length sps length pps prel = 6length
// LL SPSPSPSPSP LL PPSPPSPPSPPS 123456
private void unpackType24(byte[] twentyFour)
{
if (annexB){
int sp = (twentyFour[13] << 8 | twentyFour[14] & 0XFF);
int pp = (twentyFour[sp + 15] << 8 | twentyFour[sp + 16] & 0XFF);
byte[] sps = new byte[sp];
byte[] pps = new byte[pp];
System.arraycopy(twentyFour,15, sps,0,sp);
System.arraycopy(twentyFour,sp + 17, pps,0,pps.length);
transferTOFFmpeg(sps);
transferTOFFmpeg(pps);
}else if (prelStyle)
{
//Debug.debugHex("unpack24 " , twentyFour, twentyFour.length);
int spsl = (twentyFour[14] & 0xff) + 2;
int ppsl = (twentyFour[14+ spsl] & 0xff) +2;
int prel = 6;
byte[] buf = new byte[spsl + ppsl + prel]; //rtp header length - type + experimental data
System.arraycopy(twentyFour, 13, buf, 6,spsl + ppsl);
System.arraycopy(twentyFour, spsl + ppsl + 13, buf,0, 6);
transferTOFFmpeg(buf);
}else{
int spsl = (twentyFour[14] & 0xff) + 2;
int ppsl = (twentyFour[14+ spsl] & 0xff) +2;
byte[] buf = new byte[spsl + ppsl ]; //rtp header length - type + experimental data
System.arraycopy(twentyFour, 13, buf, 0,spsl + ppsl);
//System.arraycopy(twentyFour, spsl + ppsl + 13, buf,0, 6);
transferTOFFmpeg(buf);
}
}
//Single NON IDR Nal - This seems liekly to never occur
private void unpackType1(byte[] one)
{
byte[] buf = new byte[one.length-12];
System.arraycopy(one, 12, buf, 0,buf.length);
transferTOFFmpeg(buf);
}
//Single IDR Nal - This seems likely to never occur
private void unpackType5(byte[] five)
{
byte[] buf = new byte[five.length-12];
System.arraycopy(five, 12, buf, 0,buf.length);
transferTOFFmpeg(buf);
}
// Unpack either any split up nalu - This will get 99.999999 of nalus
synchronized private void unpackType28(byte[] twentyEight)
{
//Debug.deBugHexTrailing("unpack 28 ", twentyEight, 20 );
int ts = (twentyEight[4] << 24 | twentyEight[5] << 16 | twentyEight[6] << 8 | twentyEight[7] & 0XFF); //each nalu has a unique timestamp
//int seqN = (twentyEight[2] << 8 | twentyEight[3] & 0xFF); //each part of that nalu is numbered in order.
// numbers are from every packet ever. not this nalu. no zero or 1 start
//check if already building this nalu
if (assemblyLine.containsKey(ts)){
assemblyLine.get(ts).addPiece(twentyEight);
}
//add a new nalu
else
{
assemblyLine.put(ts, new NaluBuffer(ts, twentyEight));
}
}
//this will transfer the assembled nal units to the media codec/trans-coder/decoder/whatever?!?
private void transferTOFFmpeg(byte[] nalu)
{
Debug.debugHex("VideoDecoder transferTOFFmpg -> ", nalu, 30);
try{
if (annexB || mixed){
pipedOutputStream.write(startcode);
}
pipedOutputStream.write(nalu,0,nalu.length);
}catch (IOException ioe){
System.out.println(TAG + " transferTOFFmpeg - unable to lay pipe ;)");
}
if (assemblyLine.size() > assemblyThresh){
System.err.println(TAG + "transferToFFmpeg -> assemblyLine grows to a count of " + String.valueOf(assemblyLine.size()));
assemblyThresh += thresh;
}
}
private void clearList()
{
String n = "\n";
List<Integer> toremove = new ArrayList<>();
StringBuilder description = new StringBuilder();
for(Map.Entry<Integer, NaluBuffer> entry : assemblyLine.entrySet()) {
Integer key = entry.getKey();
NaluBuffer value = entry.getValue();
if (value.age < System.currentTimeMillis() - trashDelay){
toremove.add(key);
description
.append(String.valueOf(value.timeStamp)).append(" timestamp").append(n)
.append(String.valueOf(value.payloadType)).append(" type").append(n)
.append(String.valueOf(value.count)).append(" count").append(n)
.append(String.valueOf(value.start)).append(" ").append(String.valueOf(value.finish)).append(n)
.append(n);
}
}
for (Integer i :
toremove) {
assemblyLine.remove(i);
}
if (toremove.size() > 0){
System.out.println(TAG + " cleaList current size : " + String.valueOf(assemblyLine.size()) + n + "deleting: " + toremove.size() + n + description);
assemblyThresh = thresh;
}
}
private void deletMe(int key)
{
assemblyLine.remove(key);
if (assemblyLine.size() > 3){
clearList();
}
}
/*
Once a multipart FU-A rtp packet is found it is added to a hashset containing this class
Here we do everything needed to either complete assembly and send or destroy if not completed due to presumable packet loss
** Example Packet From First FU-A with SER = 100 **
description-> |-------RTP--HEADER------| |FU-A--HEADER| |-NAL--HEADER|
byte index-> 0|1|2|3|4|5|6|7|8|9|10|11| 12|13 14|15|16|17|18
| | | | | | | | |S S R C| | |__header | | | | |__type
| | | | |TIMESTM| |__indicator | | | |__length
| | | |__sequence number | | |__length
| | |____sequence number | |___length
| |__payload |__length
|___version padding extension
*/
private class NaluBuffer
{
private final static String TAG = "NaluBuffer";
//private static final int BUFF_SIZE = 200005; // this is the max nalu size + 5 byte header we searched for in our androids nalu search
long age;
//List<String> sizes = new ArrayList<>();
NaluePiece[] buffer = new NaluePiece[167];
int count = 0;
int start;
int finish;
int timeStamp; //from rtp packets.
int completedSize; //this is number of nalu
int payloadType; //nalu type 5 or 1
int byteLength;
int naluByteArrayLength = 0;
//if it doesnt exist
NaluBuffer(int timeStamp, byte[] piece)
{
//System.out.println(TAG + " constructor " + String.valueOf(timeStamp) );
this.timeStamp = timeStamp;
age = System.currentTimeMillis();
addPieceToBuffer(piece);
count++;
}
//adding another piece
synchronized public void addPiece(byte[] piece)
{
//System.out.println(TAG + " addPiece " + String.valueOf(timeStamp));
addPieceToBuffer(piece);
count++;
}
//add to buffer. incoming data is still raw rtp packet
private void addPieceToBuffer(byte[] piece)
{
//System.out.println(TAG + " addPiecetobuffer " + String.valueOf(piece[13]));
int seqN = (piece[2] << 8 | piece[3] & 0xFF);
//add to buffer
buffer[count] = new NaluePiece(seqN, Arrays.copyOfRange(piece, 14,piece.length)); // 14 because we skip rtp header of 12 and fu-a header of 2
int in = ( piece.length - 14); //we save each byte[] copied size so we can easily construct a completed array later
//sizes.add(String.valueOf(in));
naluByteArrayLength += in;
//check if first or last, completed size type etc
if ((start == 0) && (piece[13] & 0b11000000) == 0b10000000){
//start of nalu
start = (piece[2] << 8 | piece[3] & 0xFF);
//type
payloadType = (piece[13] & 0b00011111); //could have used [18] //get type
byteLength = (piece[17]&0xFF | (piece[16]&0xFF)<<8 | (piece[15]&0xFF)<<16 | (piece[14]&0xFF)<<24); //get the h264 encoded length
byteLength += 4; //Now add 4 bytes for the length encoding itself
if (payloadType == 1 || payloadType == 5 && byteLength < 200000){
}else{
System.err.println(TAG + " addpiecetobuffer type: " + String.valueOf(payloadType) + "length: " + String.valueOf(byteLength) );
}
//System.out.println(TAG + " addpiecetobuffer start " + String.valueOf(start) + " type " + String.valueOf(payloadType));
}else if ((finish == 0) && (piece[13] & 0b11000000) == 0b01000000){
//end of nalu
finish = (piece[2] << 8 | piece[3] & 0xFF);
//System.out.println(TAG + " addpiecetobuffer finish " + String.valueOf(finish));
}
if (finish != 0 && start != 0 && completedSize == 0){
//completed size in packet sequnce number NOT in byte length
completedSize = finish - start;
//System.out.println(TAG + " addpiecetobuffer completedsize " + String.valueOf(completedSize));
//originally put in bytes but thats not what I was counting ...duh!
// (piece[14] <<24 | piece[15] << 16 | piece[16] << 8 | piece[17] & 0xFF);
}
//check if complete
if (completedSize != 0 && count == completedSize){
assembleDeliver();
}
}
// we have every sequence number accounted for.
// reconstruct the nalu and send it to the decoder
private void assembleDeliver()
{
count++; //make up for the ount that didn't get called following addpiecetobuffer method
// System.out.println(TAG + " assembleDeliver " + String.valueOf(timeStamp));
//create a new array the exact length needed and sort each nalu by sequence number
NaluePiece[] newbuf = new NaluePiece[count];
System.arraycopy(buffer,0,newbuf,0, count);
Arrays.sort(newbuf);
// TODO: 9/28/2018 we have no gaps in data here checking newbuff !!!!!
//this will be an array we feed/pipe to our videoprocessor
byte[] out;
if (annexB){
out = new byte[naluByteArrayLength-4]; //remove the 4 bytes of length
int tally = 0;
int destPos = 0;
int src = 4;
for (int i = 0; i < count; i++) {
if (i == 1){
src = 0;
}
tally += newbuf[i].piece.length;
System.arraycopy(newbuf[i].piece, src, out, destPos, newbuf[i].piece.length - src);
//Debug.fillCompleteNalData(out, destPos, newbuf[i].piece.length);
destPos += newbuf[i].piece.length - src;
}
/*
StringBuilder sb = new StringBuilder();
sb.append("VideoDecoder assembleDeliver out.length ").append(String.valueOf(out.length))
.append(" destPos ").append(String.valueOf(destPos)).append(" tally ").append(String.valueOf(tally))
.append(" count ").append(String.valueOf(count)).append(" obuf ").append(String.valueOf(completedSize));
for (String s :
sizes) {
sb.append(s).append(" ");
}
System.out.println(sb.toString());
*/
}else{
out = new byte[naluByteArrayLength];
int destPos = 0;
for (int i = 0; i < count; i++) {
System.arraycopy(newbuf[i].piece, 0, out, destPos, newbuf[i].piece.length);
destPos += newbuf[i].piece.length;
}
}
if (naluByteArrayLength != byteLength){
System.err.println(TAG + " assembleDeliver -> ERROR - h264 encoded length: " + String.valueOf(byteLength) + " and byte length found: " + String.valueOf(naluByteArrayLength) + " do not match");
}
// TODO: 9/28/2018 we have gaps in data here
//Debug.checkNaluData(out);
transferTOFFmpeg(out);
deletMe(timeStamp);
}
}
//This class stores the payload and ordering info
private class NaluePiece implements Comparable<NaluePiece>
{
int sequenceNumber; //here is the number we can access to order them
byte[] piece; //here we store the raw payload data to be aggregated
public NaluePiece(int sequenceNumber, byte[] piece)
{
this.sequenceNumber = sequenceNumber;
this.piece = piece;
//Debug.checkNaluPieceData(piece);
}
#Override
public int compareTo(NaluePiece o) {
return Integer.compare(this.sequenceNumber, o.sequenceNumber);
}
}
}
Hey guys I'm having a huge problem when Encrypting a message in an older phone in comparison with the newer ones.
I've compiled the code to run on older hardware (CLDC1.0, MIDP2.0), and for some reason, when I do a TEA Encryption in a Nokia N70 I end up having one ruined character when it goes from plain-text to TEA. (yes I know, from a lot of chars only that one little char gets ruined...)
When I run exactly the same app on the N8 and other more recent phones however I get it encrypting correctly.
before I post the code however here's a small explanation on what it does:
basically it receives a String and a boolean inputs, the boolean states if it's for encryption or decryption purposes, whilst the string is what I want to encode or decode.
from there, I basically strip the String into a byte array, treat it accordingly (if for encrypt or decrypt) and later turn it into a String, which I then return (decrypt) or I encode in Base64 (encrypt).
The reason to encapsulate in Base64 is so it can be sent by sms, since this encoding uses non-special characters it keeps the sms limit up to 160 characters, which is desirable for the app.
now for the code:
private String HandleTEA(String input, boolean aIsEncryption) throws UnsupportedEncodingException
{
System.out.println(input);
String returnable = "";
try
{
TEAEngine e = new TEAEngine();
if (aIsEncryption)
{
e.init(true, TEAkey);
}
else
{
if(getDebug())
{
input = input.substring(1);
}
input = base64.decodeString(input);
e.init(false, TEAkey);
}
byte[] aData = input.getBytes("ISO-8859-1");
byte[] textToUse = aData;
int len = ((textToUse.length + 16 - 1) / 16) * 16;
byte[] secondUse = new byte[len];
for(int i = 0; i < textToUse.length; i++)
{
secondUse[i] = textToUse[i];
}
for(int i = textToUse.length; i < secondUse.length; i++)
{
secondUse[i] = 0;
}
int blockSize = e.getBlockSize();
byte[] outBytes = new byte[secondUse.length];
for (int chunkPosition = 0; chunkPosition < secondUse.length; chunkPosition += blockSize)
{
int chunkSize = Math.min(blockSize, (textToUse.length - (chunkPosition * blockSize)));
e.processBlock(secondUse, chunkPosition, outBytes, chunkPosition);
}
if(aIsEncryption)
{
Baseless = new String(outBytes, "ISO-8859-1");
String encodedString = base64.encodeString(Baseless);
char[] theChars = new char[encodedString.length()+1];
for(int i = 0; i < theChars.length; i++)
{
if(i == 0)
{
theChars[i] = '1';
}
else
{
theChars[i] = encodedString.charAt(i-1);
}
}
byte[] treating = new byte[theChars.length];
for(int i = 0; i < theChars.length; i++)
{
treating[i] = (byte)theChars[i];
}
returnable = new String(treating, "ISO-8859-1");
}
else
{
char[] theChars = new String(outBytes, "ISO-8859-1").toCharArray();
String fixed ="";
for(int i = 0; i < theChars.length; i++)
{
char c = theChars[i];
if (c > 0) fixed = fixed + c;
}
returnable = fixed;
}
}
catch(Exception e)
{
e.printStackTrace();
}
return returnable;
}
Anyone have any idea on what might be happening?
for comparison this is what I'm getting from the N70:
e+TgV/fU5RUOYocMRfG7vqpQT+jKlujU6eIzZfEjGhXdFwNB46wYNSiUj5H/tWbta26No6wjQylgTexhS6uqyw==
and from the N8:
e+TgV/fU5RUOYocMRfG7vqpQT+jKlujU6eIzZfEjgBXdFwNB46wYNSiUj5H/tWbta26No6wjQylgTexhS6uqyw==
as you can see everything looks similar, but in the middle of the code what gets encoded as Gh on the N70 shows up as gB on the N8...
when decrypting the data encrypted by the N70 we get some really weird chars:
will add this here tomorrow since I don't have the saved output with me
both are using the same key (in real-life tho they'll be using a key that's randomly generated on startup)
here's the key used:
0b1b5e0167aaee06
Hope you can help me out with this and Thanks for your interest and assistance!
your code is hard to understand, but Baseless = new String(outBytes, "ISO-8859-1"); and any similar constructs are almost certainly incorrect. Why do you want to make a String out of cipher? Just base64 encode outBytes directly.
I have identified a random series of bytes inserted into any blob field when the database is manipulated via Adobe AIR. (from my results it appear to always start with bytes[12, ...] but I'm not sure of that)
I think it's a sizeinfo of the bytes, let me explain how I came to this conclusion.
First my context : I manipulate sqlite databases through Adobe AIR (client-side) and System.data.sqlite (C# server-side)
With System.data.sqlite if I read a Sqlite db filled with BLOB by Adobe AIR I have to get ride of those bytes appended in the beginning by AIR and then I have the binary data all well shaped. PERFECT!
With Adobe AIR if I tried to read a sqlite db filled with BLOB by System.data.Sqlite the data are corrupted I get an error! Obviously because I don't have the missing bytes researched by AIR.
Of course I tried to add those bytes by just copying a series of 3 bytes that I removed in my first case but then it returned the data partially and in the cases of images the last rows of pixels go all gray and in some images I have more or less grays lines. Because the data were corresponding to a series of images of the same ~4k size and I added 3 bytes from one of them and I had this result.
And Air will also sometimes throw this error :
Error: Error #2030: End of file was
encountered.
So obviously those bytes give an info on the size but I don't really get how it does!?!
I tried to add a bytearray with 4k length it tends to add 3 bytes but I tried to add 4M and it goes up to 5 bytes.
I found this Question How do you convert 3 bytes into a 24 bit number in C#? and I thought that could be how the size info is stored.
But I still don't get it...
Thanks to FluorineFX (AMF for .NET) opensource project here is the answer.
Because in my Adobe AIR project I have to pass my air.ByteArray object as a parameter to store everything in the sqlite Blob field; AIR will serialize everything into AMF, a compact binary actionscript message format.
Page 11 Secion 3.14 ByteArray type
http://opensource.adobe.com/wiki/download/attachments/1114283/amf3_spec_05_05_08.pdf
The doc stipulate :
AMF 3 serializes this type using a
variable length encoding 29-bit
integer for the byte-length prefix
followed by the raw bytes of the
ByteArray.
But thats not all, I searched for an AMF .NET opensource project and founded FluorineFX. Looking into the code I identified that every AMF binaries is prefixed with a byte TypeCode which is 12 for a ByteArray:
/// <summary>
/// AMF ByteArray data type.
/// </summary>
public const byte ByteArray = 12;
Further search and again I found in the FluorineFX sources AMFReader.ReadAMF3ByteArray() and AMFWriter.WriteByteArray()
Which help me to quickly build what I need :
private static byte[] RemoveAMF3ByteArrayPrefixBytes(byte[] ar)
{
var ms = new MemoryStream(ar);
var br = new BinaryReader(ms);
// if first byte is AMF TypeCode for ByteArray
if (br.Read() != 12)
return ar;
int handle = ReadAMF3IntegerData(br);
bool inline = ((handle & 1) != 0);
handle = handle >> 1;
if (inline)
{
int length = handle;
byte[] buffer = br.ReadBytes(length);
return buffer;
}
return ar;
}
private static byte[] AddAMF3ByteArrayPrefixBytes(byte[] ar)
{
var ms = new MemoryStream();
var bw = new BinaryWriter(ms);
bw.Write((byte)12); // AMF TypeCode for ByteArray
var handle = (int)ar.Length;
handle = handle << 1;
handle = handle | 1;
WriteAMF3IntegerData(bw, handle);
bw.Write(ar);
return ms.ToArray();
}
/// <summary>
/// Handle decoding of the variable-length representation which gives seven bits of value per serialized byte by using the high-order bit
/// of each byte as a continuation flag.
/// </summary>
/// <returns></returns>
private static int ReadAMF3IntegerData(BinaryReader br)
{
int acc = br.ReadByte();
if(acc < 128)
return acc;
else
{
acc = (acc & 0x7f) << 7;
int tmp = br.ReadByte();
if(tmp < 128)
acc = acc | tmp;
else
{
acc = (acc | tmp & 0x7f) << 7;
tmp = br.ReadByte();
if(tmp < 128)
acc = acc | tmp;
else
{
acc = (acc | tmp & 0x7f) << 8;
tmp = br.ReadByte();
acc = acc | tmp;
}
}
}
//To sign extend a value from some number of bits to a greater number of bits just copy the sign bit into all the additional bits in the new format.
//convert/sign extend the 29bit two's complement number to 32 bit
int mask = 1 << 28; // mask
int r = -(acc & mask) | acc;
return r;
//The following variation is not portable, but on architectures that employ an
//arithmetic right-shift, maintaining the sign, it should be fast.
//s = 32 - 29;
//r = (x << s) >> s;
}
private static void WriteAMF3IntegerData(BinaryWriter bw, int value)
{
//Sign contraction - the high order bit of the resulting value must match every bit removed from the number
//Clear 3 bits
value &= 0x1fffffff;
if (value < 0x80)
bw.Write((byte)value);
else
if (value < 0x4000)
{
bw.Write((byte)(value >> 7 & 0x7f | 0x80));
bw.Write((byte)(value & 0x7f));
}
else
if (value < 0x200000)
{
bw.Write((byte)(value >> 14 & 0x7f | 0x80));
bw.Write((byte)(value >> 7 & 0x7f | 0x80));
bw.Write((byte)(value & 0x7f));
}
else
{
bw.Write((byte)(value >> 22 & 0x7f | 0x80));
bw.Write((byte)(value >> 15 & 0x7f | 0x80));
bw.Write((byte)(value >> 8 & 0x7f | 0x80));
bw.Write((byte)(value & 0xff));
}
}
I hope that will help someone else.
Thanks a lot, that helped me solve the problem. Here is how to do it with Java code:
Add org.granite granite-core dependency to your project
Init GDS with boilerplate code
GraniteConfig graniteConfig = new GraniteConfig(null, null, null, null);
ServicesConfig servicesConfig = new ServicesConfig(null, null, false);
Map<String, Object> applicationMap = new HashMap<String, Object>();
SimpleGraniteContext.createThreadIntance(graniteConfig, servicesConfig, applicationMap);
In my example, i read an image file that i will insert into a BLOB:
fis = new FileInputStream(file);
ps = connection.prepareStatement(INSERT_PICTURE);
ps.setString(1, key);
byte[] fileBytes = FileUtils.readFileToByteArray(file);
ByteArrayOutputStream out = new ByteArrayOutputStream();
AMF3Serializer ser = new AMF3Serializer(out);
ser.writeObject(fileBytes);
ser.flush();
ps.setBytes(2, out.toByteArray());
Works like a charm, thanks for the hint :)
Fabien
What is a good link or article on encrypting a URL link with AES to pass username to another web site in ASP.NET using VB.NET 2005?
FYI: The receiving web site will have access to the private KEY to decrypt.
First
Don't do it! Writing your own crypto system can easily lead to making mistakes. It's best to use an existing system, or if not, get someone who knows cryptography to do it. If you have to do it yourself, read Practical Cryptography.
And please, remember: "We already have enough fast, insecure systems." (Bruce Schneier) -- Do things correct and worry about performance later.
That said, if you are stuck on using AES to roll your own, here are a few pointers.
Initialization Vector
AES is a block cipher. Given a key and a block of plaintext, it converts it to a specific ciphertext. The problem with this is that the same blocks of data will generate the same ciphertext with the same key, every time. So suppose you send data like this:
user=Encrypt(Username)&roles=Encrypt(UserRoles)
They're two separate blocks, and the UserRoles encryption will have the same ciphertext each time, regardless of the name. All I need is the ciphertext for an admin, and I can drop it right in with my cipher'd username. Oops.
So, there are cipher operation modes. The main idea is that you'll take the ciphertext of one block, and XOR it into the ciphertext of the next block. That way we'll do Encrypt(UserRoles, Username), and the Username ciphertext is affected by the UserRoles.
The problem is that the first block is still vulnerable - just by seeing someone's ciphertext, I might know their roles. Enter the initialization vector. The IV "starts up" the cipher and ensures it has random data to encrypt the rest of the stream. So now the UserRoles ciphertext has the ciphertext of the random IV XOR'd in. Problem solved.
So, make sure you generate a random IV for each message. The IV is not sensitive and can be sent plaintext with the ciphertext. Use an IV large enough -- the size of the block should be fine for many cases.
Integrity
AES doesn't provide integrity features. Anyone can modify your ciphertext, and the decrypt will still work. It's unlikely it'll be valid data in general, but it might be hard to know what valid data is. For instance, if you're transmitting a GUID encrypted, it'd be easy to modify some bits and generate a completely different one. That could lead to application errors and so on.
The fix there is to run a hash algorithm (use SHA256 or SHA512) on the plaintext, and include that in the data you transmit. So if my message is (UserName, Roles), you'll send (UserName, Roles, Hash(UserName, Roles)). Now if someone tampers with the ciphertext by flipping a bit, the hash will no longer compute and you can reject the message.
Key derivation
If you need to generate a key from a password, use the built-in class: System.Security.Cryptography.PasswordDeriveBytes. This provides salting and iterations, which can improve the strength of derived keys and reduce the chance of discovering the password if the key is compromised.
Timing/replay
Edit: Sorry for not mentioning this earlier :P. You also need to make sure you have an anti-replay system. If you simply encrypt the message and pass it around, anyone who gets the message can just resend it. To avoid this, you should add a timestamp to the message. If the timestamp is different by a certain threshold, reject the message. You may also want to include a one-time ID with it (this could be the IV) and reject time-valid messages that come from other IPs using the same ID.
It's important to make sure you do the hash verification when you include the timing information. Otherwise, someone could tamper with a bit of the ciphertext and potentially generate a valid timestamp if you don't detect such brute force attempts.
Sample code
Since apparently using an IV correctly is controversial for some folks, here's some code that'll generate random IVs and add them to your output for you. It'll also perform the authentication step, making sure the encrypted data wasn't modified.
using System;
using System.Security.Cryptography;
using System.Text;
class AesDemo {
const int HASH_SIZE = 32; //SHA256
/// <summary>Performs encryption with random IV (prepended to output), and includes hash of plaintext for verification.</summary>
public static byte[] Encrypt(string password, byte[] passwordSalt, byte[] plainText) {
// Construct message with hash
var msg = new byte[HASH_SIZE + plainText.Length];
var hash = computeHash(plainText, 0, plainText.Length);
Buffer.BlockCopy(hash, 0, msg, 0, HASH_SIZE);
Buffer.BlockCopy(plainText, 0, msg, HASH_SIZE, plainText.Length);
// Encrypt
using (var aes = createAes(password, passwordSalt)) {
aes.GenerateIV();
using (var enc = aes.CreateEncryptor()) {
var encBytes = enc.TransformFinalBlock(msg, 0, msg.Length);
// Prepend IV to result
var res = new byte[aes.IV.Length + encBytes.Length];
Buffer.BlockCopy(aes.IV, 0, res, 0, aes.IV.Length);
Buffer.BlockCopy(encBytes, 0, res, aes.IV.Length, encBytes.Length);
return res;
}
}
}
public static byte[] Decrypt(string password, byte[] passwordSalt, byte[] cipherText) {
using (var aes = createAes(password, passwordSalt)) {
var iv = new byte[aes.IV.Length];
Buffer.BlockCopy(cipherText, 0, iv, 0, iv.Length);
aes.IV = iv; // Probably could copy right to the byte array, but that's not guaranteed
using (var dec = aes.CreateDecryptor()) {
var decBytes = dec.TransformFinalBlock(cipherText, iv.Length, cipherText.Length - iv.Length);
// Verify hash
var hash = computeHash(decBytes, HASH_SIZE, decBytes.Length - HASH_SIZE);
var existingHash = new byte[HASH_SIZE];
Buffer.BlockCopy(decBytes, 0, existingHash, 0, HASH_SIZE);
if (!compareBytes(existingHash, hash)){
throw new CryptographicException("Message hash incorrect.");
}
// Hash is valid, we're done
var res = new byte[decBytes.Length - HASH_SIZE];
Buffer.BlockCopy(decBytes, HASH_SIZE, res, 0, res.Length);
return res;
}
}
}
static bool compareBytes(byte[] a1, byte[] a2) {
if (a1.Length != a2.Length) return false;
for (int i = 0; i < a1.Length; i++) {
if (a1[i] != a2[i]) return false;
}
return true;
}
static Aes createAes(string password, byte[] salt) {
// Salt may not be needed if password is safe
if (password.Length < 8) throw new ArgumentException("Password must be at least 8 characters.", "password");
if (salt.Length < 8) throw new ArgumentException("Salt must be at least 8 bytes.", "salt");
var pdb = new PasswordDeriveBytes(password, salt, "SHA512", 129);
var key = pdb.GetBytes(16);
var aes = Aes.Create();
aes.Mode = CipherMode.CBC;
aes.Key = pdb.GetBytes(aes.KeySize / 8);
return aes;
}
static byte[] computeHash(byte[] data, int offset, int count) {
using (var sha = SHA256.Create()) {
return sha.ComputeHash(data, offset, count);
}
}
public static void Main() {
var password = "1234567890!";
var salt = new byte[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 };
var ct1 = Encrypt(password, salt, Encoding.UTF8.GetBytes("Alice; Bob; Eve;: PerformAct1"));
Console.WriteLine(Convert.ToBase64String(ct1));
var ct2 = Encrypt(password, salt, Encoding.UTF8.GetBytes("Alice; Bob; Eve;: PerformAct2"));
Console.WriteLine(Convert.ToBase64String(ct2));
var pt1 = Decrypt(password, salt, ct1);
Console.WriteLine(Encoding.UTF8.GetString(pt1));
var pt2 = Decrypt(password, salt, ct2);
Console.WriteLine(Encoding.UTF8.GetString(pt2));
// Now check tampering
try {
ct1[30]++;
Decrypt(password, salt, ct1);
Console.WriteLine("Error: tamper detection failed.");
} catch (Exception ex) {
Console.WriteLine("Success: tampering detected.");
Console.WriteLine(ex.ToString());
}
}
}
Output:
JZVaD327sDmCmdzY0PsysnRgHbbC3eHb7YXALb0qxFVlr7Lkj8WaOZWc1ayWCvfhTUz/y0QMz+uv0PwmuG8VBVEQThaNTD02JlhIs1DjJtg=
QQvDujNJ31qTu/foDFUiVMeWTU0jKL/UJJfFAvmFtz361o3KSUlk/zH+4701mlFEU4Ce6VuAAuaiP1EENBJ74Wc8mE/QTofkUMHoa65/5e4=
Alice; Bob; Eve;: PerformAct1 Alice;
Bob; Eve;: PerformAct2 Success:
tampering detected.
System.Security.Cryptography.CryptographicException:
Message hash incorrect. at
AesDemo.Decrypt(String password,
Byte[] passwordSalt, Byte[]
cipherText) in
C:\Program.cs:line
46 at AesDemo.Main() in
C:\Program.cs:line
100
After removing the random IV and the hash, here's the type of output:
tZfHJSFTXYX8V38AqEfYVXU5Dl/meUVAond70yIKGHY=
tZfHJSFTXYX8V38AqEfYVcf9a3U8vIEk1LuqGEyRZXM=
Notice how the first block, corresponding to "Alice; Bob; Eve;" is the same. "Corner case" indeed.
Example without hashing
Here's a simple example of passing a 64-bit integer. Just encrypt and you're open to attack. In fact, the attack is easily done, even with CBC padding.
public static void Main() {
var buff = new byte[8];
new Random().NextBytes(buff);
var v = BitConverter.ToUInt64(buff, 0);
Console.WriteLine("Value: " + v.ToString());
Console.WriteLine("Value (bytes): " + BitConverter.ToString(BitConverter.GetBytes(v)));
var aes = Aes.Create();
aes.GenerateIV();
aes.GenerateKey();
var encBytes = aes.CreateEncryptor().TransformFinalBlock(BitConverter.GetBytes(v), 0, 8);
Console.WriteLine("Encrypted: " + BitConverter.ToString(encBytes));
var dec = aes.CreateDecryptor();
Console.WriteLine("Decrypted: " + BitConverter.ToUInt64(dec.TransformFinalBlock(encBytes, 0, encBytes.Length), 0));
for (int i = 0; i < 8; i++) {
for (int x = 0; x < 250; x++) {
encBytes[i]++;
try {
Console.WriteLine("Attacked: " + BitConverter.ToUInt64(dec.TransformFinalBlock(encBytes, 0, encBytes.Length), 0));
return;
} catch { }
}
}
}
Output:
Value: 6598637501946607785 Value
(bytes): A9-38-19-D1-D8-11-93-5B
Encrypted:
31-59-B0-25-FD-C5-13-D7-81-D8-F5-8A-33-2A-57-DD
Decrypted: 6598637501946607785
Attacked: 14174658352338201502
So, if that's the kind of ID you're sending, it could quite easily be changed to another value. You need to authenticate outside of your message. Sometimes, the message structure is unlikely to fall into place and can sorta act as a safeguard, but why rely on something that could possibly change? You need to be able to rely on your crypto working correctly regardless of the application.
I wrote a blog post which has a sample project that you can download here (C# though):
http://www.codestrider.com/blog/read/AESFileEncryptorWithRSAEncryptedKeys.aspx
The code basically uses AES for encryption of binary data and then RSA encrypts the Key and the IV using an X509Certificate. So, as long as the private key certificate is available, the Key and IV can be decrypted, and then in turn the AES encrypted data can be decrypted ..
You could set up your certificate stores so that the 'encryptor' only has access to the public key certificate, while the 'decryptor' has access to the private key.
This allows you to encrypt using different Key and IV each time and avoid hardcoding anything.. which I believe is more secure. There should be nothing in your source code that would easily allow someone to decrypt your data - and if your system was ever compromised, you would only need to swap out the certificates with new ones. No need to recompile the application with new hardcoded values.. :)
The sample code may be slightly different from your intended use, but I think the technique and some of the code might be useful to you.
Below you'll find a class that provides AES Encryption/Decryption methods that explicitly provide URL-friendly strings for use in applications like yours. It also has the methods that work with byte arrays.
NOTE: you should use different values in the Key and Vector arrays! You wouldn't want someone to figure out your keys by just assuming that you used this code as-is! All you have to do is change some of the numbers (must be <= 255) in the Key and Vector arrays.
Using it is easy: just instantiate the class and then call (usually) EncryptToString(string StringToEncrypt) and DecryptString(string StringToDecrypt) as methods. It couldn't be any easier (or more secure) once you have this class in place.
using System;
using System.Data;
using System.Security.Cryptography;
using System.IO;
public class SimpleAES
{
// Change these keys
private byte[] Key = { 123, 217, 19, 11, 24, 26, 85, 45, 114, 184, 27, 162, 37, 112, 222, 209, 241, 24, 175, 144, 173, 53, 196, 29, 24, 26, 17, 218, 131, 236, 53, 209 };
private byte[] Vector = { 146, 64, 191, 111, 23, 3, 113, 119, 231, 121, 2521, 112, 79, 32, 114, 156 };
private ICryptoTransform EncryptorTransform, DecryptorTransform;
private System.Text.UTF8Encoding UTFEncoder;
public SimpleAES()
{
//This is our encryption method
RijndaelManaged rm = new RijndaelManaged();
//Create an encryptor and a decryptor using our encryption method, key, and vector.
EncryptorTransform = rm.CreateEncryptor(this.Key, this.Vector);
DecryptorTransform = rm.CreateDecryptor(this.Key, this.Vector);
//Used to translate bytes to text and vice versa
UTFEncoder = new System.Text.UTF8Encoding();
}
/// -------------- Two Utility Methods (not used but may be useful) -----------
/// Generates an encryption key.
static public byte[] GenerateEncryptionKey()
{
//Generate a Key.
RijndaelManaged rm = new RijndaelManaged();
rm.GenerateKey();
return rm.Key;
}
/// Generates a unique encryption vector
static public byte[] GenerateEncryptionVector()
{
//Generate a Vector
RijndaelManaged rm = new RijndaelManaged();
rm.GenerateIV();
return rm.IV;
}
/// ----------- The commonly used methods ------------------------------
/// Encrypt some text and return a string suitable for passing in a URL.
public string EncryptToString(string TextValue)
{
return ByteArrToString(Encrypt(TextValue));
}
/// Encrypt some text and return an encrypted byte array.
public byte[] Encrypt(string TextValue)
{
//Translates our text value into a byte array.
Byte[] bytes = UTFEncoder.GetBytes(TextValue);
//Used to stream the data in and out of the CryptoStream.
MemoryStream memoryStream = new MemoryStream();
/*
* We will have to write the unencrypted bytes to the stream,
* then read the encrypted result back from the stream.
*/
#region Write the decrypted value to the encryption stream
CryptoStream cs = new CryptoStream(memoryStream, EncryptorTransform, CryptoStreamMode.Write);
cs.Write(bytes, 0, bytes.Length);
cs.FlushFinalBlock();
#endregion
#region Read encrypted value back out of the stream
memoryStream.Position = 0;
byte[] encrypted = new byte[memoryStream.Length];
memoryStream.Read(encrypted, 0, encrypted.Length);
#endregion
//Clean up.
cs.Close();
memoryStream.Close();
return encrypted;
}
/// The other side: Decryption methods
public string DecryptString(string EncryptedString)
{
return Decrypt(StrToByteArray(EncryptedString));
}
/// Decryption when working with byte arrays.
public string Decrypt(byte[] EncryptedValue)
{
#region Write the encrypted value to the decryption stream
MemoryStream encryptedStream = new MemoryStream();
CryptoStream decryptStream = new CryptoStream(encryptedStream, DecryptorTransform, CryptoStreamMode.Write);
decryptStream.Write(EncryptedValue, 0, EncryptedValue.Length);
decryptStream.FlushFinalBlock();
#endregion
#region Read the decrypted value from the stream.
encryptedStream.Position = 0;
Byte[] decryptedBytes = new Byte[encryptedStream.Length];
encryptedStream.Read(decryptedBytes, 0, decryptedBytes.Length);
encryptedStream.Close();
#endregion
return UTFEncoder.GetString(decryptedBytes);
}
/// Convert a string to a byte array. NOTE: Normally we'd create a Byte Array from a string using an ASCII encoding (like so).
// System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
// return encoding.GetBytes(str);
// However, this results in character values that cannot be passed in a URL. So, instead, I just
// lay out all of the byte values in a long string of numbers (three per - must pad numbers less than 100).
public byte[] StrToByteArray(string str)
{
if (str.Length == 0)
throw new Exception("Invalid string value in StrToByteArray");
byte val;
byte[] byteArr = new byte[str.Length / 3];
int i = 0;
int j = 0;
do
{
val = byte.Parse(str.Substring(i, 3));
byteArr[j++] = val;
i += 3;
}
while (i < str.Length);
return byteArr;
}
// Same comment as above. Normally the conversion would use an ASCII encoding in the other direction:
// System.Text.ASCIIEncoding enc = new System.Text.ASCIIEncoding();
// return enc.GetString(byteArr);
public string ByteArrToString(byte[] byteArr)
{
byte val;
string tempStr = "";
for (int i = 0; i <= byteArr.GetUpperBound(0); i++)
{
val = byteArr[i];
if (val < (byte)10)
tempStr += "00" + val.ToString();
else if (val < (byte)100)
tempStr += "0" + val.ToString();
else
tempStr += val.ToString();
}
return tempStr;
}
}
Markt pointed out that Rijndael uses the AES encryption algorithm. Since a managed implementation ships with the .net framework (and has since at least 1.1), using it should satisfy the OP.
The API docs have a pretty straightforward example of using Rijndael as an encryption and decryption stream.
If you've got a way to get the shared secret (e.g., the private key) to the other website then you might be able to get away with using plain old symmetric encryption (no public key, both sides know the IV and private key). This is especially the case if your brain is the "insecure channel" across which the key is shared (e.g., you administer both websites). :)
Have a look at "Keep Your Data Secure
with the New Advanced Encryption
Standard". An AES implementation
doesn't ship with the .NET framework
but it links to a custom
implementation (AES.exe).
1:
http://msdn.microsoft.com/en-us/magazine/cc164055.aspx