Decrypting And Combining .ts Audio Files with .m3u8 - encryption

I have a few thousand .ts AES-128 encrypted audio files with a .key and .m3u8 file.
The key file just contains a key comprised of 44 characters. The .m3ud files appears to be some type of playlist.
#EXTM3U
#EXT-X-ALLOW-CACHE:NO
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-KEY:METHOD=AES-128,URI="http://localhost:[port]/hls/keys/nax_9781843794066.key"
#EXTINF:10,
http://localhost:[port]/filesequence0000000.ts
#EXTINF:10,
etc...
Note that both the key URI and path to the .ts files is now wrong.
Looking around, it appears ffmpeg might work with this format. But I am unsure of the syntax.
How can I decrypt and combine these files?
I have been playing around with fixing the playlist syntax and figuring out how to use ffmpeg and got to.
ffmpeg -i nax_9781843794066.m3u8 -c copy output.ts
ffmpeg version N-77197-gdf2ce13 Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 5.2.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 10.100 / 55. 10.100
libavcodec 57. 17.100 / 57. 17.100
libavformat 57. 19.100 / 57. 19.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 20.100 / 6. 20.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
[hls,applehttp # 0000003e6348a660] Error when loading first segment 'filesequence0000000.ts'
nax_9781843794066.m3u8: Invalid data found when processing input
filesequence0000000.ts is the first file, in the same folder. I cannot tell if this is some problem with the encryption or something else.
This is the key file, if it matters: MoOoNvcKlThWBm2T+VzYq9QKZLw7MFUqSyLYjiwquTQ=

Recent versions of ffmpeg should be able to decrypt the AES-128 HLS streams. You don't need a webserver. If the m3u8 URIs/paths are wrong you can:
create a directory
copy the key to a key file, ie my.key, and place it in the dir. Note that keys can be rotated, if the playlist has multiple keys copy all of them to different files.
copy all .ts segments to the same dir
copy and edit the playlist.m3u8 and use just the filename(s) for the key(s) URI(s) and segments.
to obtain a single .ts file do:
ffmpeg -i playlist.m3u8 -c copy output.ts
if you want just the audio stream without the .ts container you can extract it. Eg: assuming you have a single audio stream using the AAC codec run:
ffmpeg -i playlist.m3u8 -map 0:a -c copy output.aac
This will extract the AAC stream to a file without re-encoding. If you want a codec different than your source you will have to re-encode.
If for some reason you have to use openssl to decrypt the segments keep in mind that if no IV is specified then the IV is equal to the segment's media sequence, ie. the first segment has IV=0, the second has IV=1 and so on. After decryption update the playlist to point the decrypted segments and remove the EXT-X-KEY line. If you go this route you don't even need ffmpeg to obtain a single .ts file as MPEG-TS is directly concatenable, ie. you can just use cat on the decrypted segments.

I've had few free hours today and toyed with this. Long story short - that base64 key is AES encrypted. This additional encryption is done with key which is dynamically generated from device data... meaning that even if I have whole data folder from your device I wouldn't be able to decrypt it.
Now, when you posses rooted device with offline data that's another matter - you can obviously inject your code to intercept key when it's decrypted so content can start playing... which is how I got it.
When you have proper key, decryption and joining of *.ts files is trivial. I recommend that you use FFMPEG for this task, my C# code that I'm leaving for illustration works well works only in some cases (depending on how files are encoded):
var folder = "path_to_folder";
byte[] encryptionKey = File.ReadAllBytes(folder + "path_to_key.key");
var outputFile = "c:\\i_love_you_guys.ts";
using (FileStream outputFileStream = new FileStream(outputFile, FileMode.Create))
{
var files = Directory.GetFiles(folder, "*.ts");
for (int i = 0; i < files.Length; i++)
{
byte[] encryptionIV = new byte[16];
using (FileStream inputFileStream = new FileStream(files[i], FileMode.Open))
{
using (var aes = new AesManaged { Key = encryptionKey, IV = encryptionIV, Mode = CipherMode.CBC })
using (var encryptor = aes.CreateDecryptor())
using (var cryptoStream = new CryptoStream(inputFileStream, encryptor, CryptoStreamMode.Read))
{
cryptoStream.CopyTo(outputFileStream);
}
}
}
}
So, this turned out to be wild goose chase. What #aergistal says in his answer is completely valid as long as you have proper my.key. Thus focus on obtaining key in plain format and decryption will then be super easy.

I have a little question about this codes:
var folder = "path_to_folder";
byte[] encryptionKey = File.ReadAllBytes(folder + "path_to_key.key");
var outputFile = "c:\\i_love_you_guys.ts";
using (FileStream outputFileStream = new FileStream(outputFile, FileMode.Create))
{
var files = Directory.GetFiles(folder, "*.ts");
for (int i = 0; i < files.Length; i++)
{
byte[] encryptionIV = new byte[16];
using (FileStream inputFileStream = new FileStream(files[i], FileMode.Open))
{
using (var aes = new AesManaged { Key = encryptionKey, IV = encryptionIV, Mode = CipherMode.CBC })
using (var encryptor = aes.CreateDecryptor())
using (var cryptoStream = new CryptoStream(inputFileStream, encryptor, CryptoStreamMode.Read))
{
cryptoStream.CopyTo(outputFileStream);
}
}
}
}
How to decrypt individually not a single file output?

Related

PGP encryption with multiple keys in mule

Given that PGP supports encryption using multiple public keys, how it can be achieved in mulesoft?
adding two keys in Public Key Ring File Name field on encryption config window for pgp encrypter tab results in error as org.mule.module.pgp.exception.MissingPGPKeyException: No key file found in: abc.gpg,test.pgp
Is it possible to add multiple public keys from this encrypt module or else how it can be done? mule runtime : 3.8.5
much appreciate any help. Thank you!
I solved this by using java component- invoking terminal and running gpg encrypt command from java runtime. I am just checking for OS first to build a command string to run its respective terminal
boolean isWindows = System.getProperty("os.name")
.toLowerCase().startsWith("windows");
/*gpg command options may vary as per your requirement. multiple --recipient option here is the way to encrypt with multiple public keys.
Using StringBuilder helps to build this string from input/dynamic values.
*/
String command = "gpg --pgp6 --armor --batch --output encryptedHelloWorld.pgp --trust-model always --recipient "<part of UserID1 (either name or emailId)>" --recipient "<part of UserID2>" --encrypt helloWorld.txt"
/*in case you need to change directory to where your file is to encrypt it from one command, you could append this
`"cd"+ <your path to file> "&" + command` ----> for windows
`"cd"+ <your path to file> ";" + command` ----> for linux
*/
public int executeCommand(String command) throws IOException, InterruptedException {
Process pr;
if (isWindows) {
String[] cmd = { "cmd.exe", "/c", command };
pr = Runtime.getRuntime().exec(cmd);
}
else {
String[] cmd = { "/bin/sh", "-c", command };
pr = Runtime.getRuntime().exec(cmd);
}
int exitStatus = pr.waitFor(); // this gives you value 0 if success or other than 0 which ties to error message
errorInputStream = pr.getErrorStream(); //streaming error message
return exitStatus;
}

Ffmpeg/Fluent-Ffmpeg: Output File Does Not Reflect Input Parameters

Use Case
When a user records an audio file and uploads to firebase storage, run a trigger that crops a clip of that file and saves it to a "preview" directory.
Problem
When the file arrives on the server, it is just dead air before the start time (10 seconds) and then doesn't stop after the duration of 10 seconds is exceeded. Whatever I put in the "-t" and "-ss" arguments seems to have no impact on the output file - not cropping correctly.
Environment
mac client / firebase storage
node v8.1.0
ffmpeg v3.2.2
fluent-ffmpeg v2.1.2
Node Code
var command = new ffmpeg({ source: tempFilePath, timeout: 0 })
.setFfmpegPath(ffmpegPath)
.setFfprobePath(ffprobePath)
.inputOption('-t', '10')
.inputOption('-ss', '10')
.outputOption('-acodec', 'copy')
.on('start', function(commandLine) {
console.log('Spawned Ffmpeg with command: ', commandLine);
})
.on('end', function() {
console.log('Preview file cropping done successfully');
})
.on('error', function(err, stdout, stderr) {
var build = err;
if (!stdout === '') { build = build + '\n' + stdout; }
build = build + '\n' + stderr;
console.log(build);
}).save(tempFilePath);
Ffmpeg Command
Spawned Ffmpeg with command: ffmpeg -t 10 -i
/tmp/2EB100B0-6C1E-4D51-9087-764CA653AFC720170711232034.aac -y -acodec
aac /tmp/2EB100B0-6C1E-4D51-9087-764CA653AFC720170711232034.aac
Console Output
$ ffmpeg -t 10 -i E506FE2E-DC81-4DD8-B751-211F7FD9A25820170816160502.aac -y -acodec aac output.aac -loglevel debug
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
built with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --enable-libvidstab --enable-libx265 --disable-doc --arch=x86_64 --enable-runtime-cpudetect
libavutil 55. 34.100 / 55. 34.100
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.100 / 57. 56.100
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100
Splitting the commandline.
Reading option '-t' ... matched as option 't' (record or transcode "duration" seconds of audio/video) with argument '10'.
Reading option '-i' ... matched as input url with argument 'E506FE2E-DC81-4DD8-B751-211F7FD9A25820170816160502.aac'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'aac'.
Reading option 'output.aac' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url E506FE2E-DC81-4DD8-B751-211F7FD9A25820170816160502.aac.
Applying option t (record or transcode "duration" seconds of audio/video) with argument 10.
Successfully parsed a group of options.
Opening an input file: E506FE2E-DC81-4DD8-B751-211F7FD9A25820170816160502.aac.
[file # 0x7fe42f700420] Setting default whitelist 'file,crypto'
[aac # 0x7fe431000000] Format aac probed with size=2048 and score=51
[aac # 0x7fe431000000] Before avformat_find_stream_info() pos: 0 bytes read:32928 seeks:4 nb_streams:1
[aac # 0x7fe431000000] All info found
[aac # 0x7fe431000000] Estimating duration from bitrate, this may be inaccurate
[aac # 0x7fe431000000] After avformat_find_stream_info() pos: 37888 bytes read:98464 seeks:4 frames:50
Input #0, aac, from 'E506FE2E-DC81-4DD8-B751-211F7FD9A25820170816160502.aac':
Duration: 00:01:24.28, bitrate: 335 kb/s
Stream #0:0, 50, 1/28224000: Audio: aac (LC), 44100 Hz, stereo, fltp, 335 kb/s
Successfully opened the file.
Parsing a group of options: output url output.aac.
Applying option acodec (force audio codec ('copy' to copy stream)) with argument aac.
Successfully parsed a group of options.
Opening an output file: output.aac.
[file # 0x7fe42f7016e0] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 4 logical cores
[graph 0 input from stream 0:0 # 0x7fe42f701880] Setting 'time_base' to value '1/44100'
[graph 0 input from stream 0:0 # 0x7fe42f701880] Setting 'sample_rate' to value '44100'
[graph 0 input from stream 0:0 # 0x7fe42f701880] Setting 'sample_fmt' to value 'fltp'
[graph 0 input from stream 0:0 # 0x7fe42f701880] Setting 'channel_layout' to value '0x3'
[graph 0 input from stream 0:0 # 0x7fe42f701880] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3
[audio format for output stream 0:0 # 0x7fe42f702060] Setting 'sample_fmts' to value 'fltp'
[audio format for output stream 0:0 # 0x7fe42f702060] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[AVFilterGraph # 0x7fe42f6101a0] query_formats: 5 queried, 12 merged, 0 already done, 0 delayed
Output #0, adts, to 'output.aac':
Metadata:
encoder : Lavf57.56.100
Stream #0:0, 0, 1/44100: Audio: aac (LC), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
Metadata:
encoder : Lavc57.64.101 aac
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[output stream 0:0 # 0x7fe42f701ee0] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
[aac # 0x7fe431001200] Trying to remove 344 more samples than there are in the queue
size= 160kB time=00:00:10.00 bitrate= 130.9kbits/s speed=20.2x
video:0kB audio:157kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.881580%
Input file #0 (E506FE2E-DC81-4DD8-B751-211F7FD9A25820170816160502.aac):
Input stream #0:0 (audio): 432 packets read (398208 bytes); 432 frames decoded (442368 samples);
Total: 432 packets (398208 bytes) demuxed
Output file #0 (output.aac):
Output stream #0:0 (audio): 431 frames encoded (441000 samples); 432 packets muxed (160716 bytes);
Total: 432 packets (160716 bytes) muxed
432 frames successfully decoded, 0 decoding errors
[AVIOContext # 0x7fe42f611040] Statistics: 0 seeks, 432 writeouts
[aac # 0x7fe431001200] Qavg: 4508.253
[AVIOContext # 0x7fe42f7004c0] Statistics: 458912 bytes read, 4 seeks

qmake and generated qm files

What is the best (proper) way to organize compiled translations (*.qm) into resources?
*.qm files referred in qrc file and generated by two (three) extra targets this way:
trans_update.commands = lupdate $$_PRO_FILE_
trans_update.depends = $$_PRO_FILE_
trans_release.commands = lrelease $$_PRO_FILE_
trans_release.depends = trans_update $$TRANSLATIONS
translate.depends = trans_release
QMAKE_EXTRA_TARGETS += trans_update trans_release translate deploy
CONFIG(release, debug|release) {
DESTDIR=release
PRE_TARGETDEPS += translate
}
but the problem is at the moment qmake runs first time, there're no qm files generated yet and make prints errors like:
RCC: Error in 'qml.qrc': Cannot find file ...
I don't like an idea of saving compiled qm files into VSC.
Is there a way to organize it nicely?
I like to point out a solution which I use in some projects. It might be far from perfect, but it works out nicely.
CONFIG(release, debug|release) {
TRANSLATION_TARGET_DIR = $${OUT_PWD}/release/translations
LANGUPD_OPTIONS = -locations relative -no-ui-lines
LANGREL_OPTIONS = -compress -nounfinished -removeidentical
} else {
TRANSLATION_TARGET_DIR = $${OUT_PWD}/debug/translations
LANGUPD_OPTIONS =
LANGREL_OPTIONS = -markuntranslated "MISS_TR "
}
isEmpty(QMAKE_LUPDATE) {
win32:LANGUPD = $$[QT_INSTALL_BINS]\lupdate.exe
else:LANGUPD = $$[QT_INSTALL_BINS]/lupdate
}
isEmpty(QMAKE_LRELEASE) {
win32:LANGREL = $$[QT_INSTALL_BINS]\lrelease.exe
else:LANGREL = $$[QT_INSTALL_BINS]/lrelease
}
langupd.command = \
$$LANGUPD $$LANGUPD_OPTIONS $$shell_path($$_PRO_FILE_) -ts $$_PRO_FILE_PWD_/$$TRANSLATIONS
langrel.depends = langupd
langrel.input = TRANSLATIONS
langrel.output = $$TRANSLATION_TARGET_DIR/${QMAKE_FILE_BASE}.qm
langrel.commands = \
$$LANGREL $$LANGREL_OPTIONS ${QMAKE_FILE_IN} -qm $$TRANSLATION_TARGET_DIR/${QMAKE_FILE_BASE}.qm
langrel.CONFIG += no_link
QMAKE_EXTRA_TARGETS += langupd
QMAKE_EXTRA_COMPILERS += langrel
PRE_TARGETDEPS += langupd compiler_langrel_make_all
There might be a sensful tweak to lupdate options because the various builds (release and debug) generate different *.ts files which then trigger a change in the used VCS.
I also like to guide the tended reader to an example where experts use it.
The recommended way -- which may not have been available at the time this question was originally asked would be to use
TRANSLATIONS += <your *.ts files>
CONFIG += lrelease embed_translations
If you really need/want to build the qm files separately, I'd point to what qmake does with the above config and adapt it according to your needs. See https://github.com/qt/qtbase/blob/5.15.2/mkspecs/features/lrelease.prf
(Basically, it creates and adds a list of resources to RESOURCES).

redefine \nomencl_command in LyX

I am trying to redefine the \nomencl_command in LyX, in order to be able to use the glossaries package in stead of the obsolete nomencl.
LyX allows you to specify the Nomenclature command, which by default is set to:
makeindex -s nomencl.ist
For glossaries the command is thus changed to:
makeglossaries
However, the LyX implementation of nomencl uses the more recent .nlo as an input file, and the .nls as an output file. Whereas glossaries uses the 'older' .glo and .gls Unfortunately, the extensions cannot be specified.
I found that the preferences file only says:
\nomencl_command "makeglossaries"
but the log output says:
makeglossaries "[filename].nlo" -o [filename].nls
So my question is where is \nomencl_command defined further?
The relevant code is in src/LaTeX.cpp.
Note below that some diagnostic information is written to the latex debug flag. You can see this info on the terminal if you run LyX with lyx -dbg latex.
The following are excerpts from the file src/LaTeX.cpp from the soon-to-be-released (a matter of days) LyX 2.1.
FileName const nlofile(changeExtension(file.absFileName(), ".nlo"));
// If all nomencl entries are removed, nomencl writes an empty nlo file.
// DepTable::hasChanged() returns false in this case, since it does not
// distinguish empty files from non-existing files. This is why we need
// the extra checks here (to trigger a rerun). Cf. discussions in #8905.
// FIXME: Sort out the real problem in DepTable.
if (head.haschanged(nlofile) || (nlofile.exists() && nlofile.isFileEmpty()))
rerun |= runMakeIndexNomencl(file, ".nlo", ".nls");
FileName const glofile(changeExtension(file.absFileName(), ".glo"));
if (head.haschanged(glofile))
rerun |= runMakeIndexNomencl(file, ".glo", ".gls");
and
bool LaTeX::runMakeIndexNomencl(FileName const & file,
string const & nlo, string const & nls)
{
LYXERR(Debug::LATEX, "Running MakeIndex for nomencl.");
message(_("Running MakeIndex for nomencl."));
string tmp = lyxrc.nomencl_command + ' ';
// onlyFileName() is needed for cygwin
tmp += quoteName(onlyFileName(changeExtension(file.absFileName(), nlo)));
tmp += " -o "
+ onlyFileName(changeExtension(file.toFilesystemEncoding(), nls));
Systemcall one;
one.startscript(Systemcall::Wait, tmp, path);
return true;
}
and
// nomencl file
FileName const nls(changeExtension(file.absFileName(), ".nls"));
nls.removeFile();
// nomencl file (old version of the package)
FileName const gls(changeExtension(file.absFileName(), ".gls"));
gls.removeFile();

How can I specify server-side encryption of Amazon S3 objects with PowerShell?

Would someone explain how to enable Amazon S3 server-side encryption in a PowerShell script? I'm using the sample code below but when I check encryption in the AWS Console or Cloudberry S3 Explorer Pro the encryption type is still set to 'none'. Using AWS / Cloudberry to do this manually after files are uploaded isn't feasible because the script is to be deployed to 200+ servers, each with it's own bucket in S3. Here's a snippet of code from the script:
$TestFile="testfile.7z"
$S3ObjectKey = "mytestfile.7z"
#Create Amazon PutObjectRequest.
$AmazonS3 = [Amazon.AWSClientFactory]::CreateAmazonS3Client($S3AccessKeyID,$S3SecretKeyID)
$S3PutRequest = New-Object Amazon.S3.Model.PutObjectRequest
$S3PutRequest.BucketName = $S3BucketName
$S3PutRequest.Key = $S3ObjectKey
$S3PutRequest.FilePath = $TestFile
$S3Response = $AmazonS3.PutObject($S3PutRequest)
I've tried inserting the following without success (before the $S3Response line):
$S3PutRequest.ServerSideEncryption
When the above is added I get this message in the output but the file is still not tagged as encrypted on S3:
MemberType : Method
OverloadDefinitions : {Amazon.S3.Model.PutObjectRequest WithServerSideEncryptionMethod(Amazon.S3.Model.ServerSideEncryptionMethod encryption)}
TypeNameOfValue : System.Management.Automation.PSMethod
Value : Amazon.S3.Model.PutObjectRequest WithServerSideEncryptionMethod(Amazon.S3.Model.ServerSideEncryptionMethod encryption)
Name : WithServerSideEncryptionMethod
IsInstance : True
Can anyone tell me what I'm doing wrong? Many thanks in advance.
You should add:
$S3PutRequest.WithServerSideEncryptionMethod([Amazon.S3.Model.ServerSideEncryptionMethod]::AES256)
Or:
$S3PutRequest.ServerSideEncryptionMethod = [Amazon.S3.Model.ServerSideEncryptionMethod]::AES256
If you are using CloudBerry, it has its own PowerShell snapin
Add-PSSnapin CloudBerryLab.Explorer.PSSnapin
$s3 = Get-CloudS3Connection -Key XXXXXXX -Secret YYYYYYY
$destFolder = $s3 | Select-CloudFolder -path "mybucket"
$local = Get-CloudFilesystemConnection
$srcFolder = $local | Select-CloudFolder -path "c:\myzips"
$srcFolder | Copy-CloudItem $destFolder -filter "testfile.7z" -SSE
Notice -SSE parameter in the Copy-CloudItem command.
Some helpful examples can be found on their blog: http://blog.cloudberrylab.com/search?q=powershell

Resources