Record audio on one device and play on another device using UDP - avaudioplayer

I want to record audio from one iOS device and send and play it to another device at the same time.
I have tried to use AVCaptureAudioDataOutput and send/received CMSampleBufferRef properly on both end but when I play it on receiver device, I am getting : Error Domain=NSOSStatusError Domain Code=1954115647.
I am using following code to convert received CMSampleBufferRef to NSData on Sender Device :
if (captureOutput == audioOutput) {
AudioBufferList audioBufferList;
NSMutableData *data=[[NSMutableData alloc] init];
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, NULL, &audioBufferList,
sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
for( int y=0; y<audioBufferList.mNumberBuffers; y++ )
{
AudioBuffer audioBuffer = audioBufferList.mBuffers[y];
Float32 *frame = (Float32*)audioBuffer.mData;
[data appendBytes:frame length:audioBuffer.mDataByteSize];
}
[self performSelectorOnMainThread:#selector(sendAudio:) withObject:data waitUntilDone:NO];
CFRelease(blockBuffer);
blockBuffer=NULL;
}
and following code to play audio on Receiver Device :
audioPlayer = [[AVAudioPlayer alloc] initWithData:audioData error:&error];
audioPlayer.numberOfLoops = -1;
audioPlayer.volume = 1.0;
if (audioPlayer == nil)
NSLog(#"%#", [error description]);
else
[audioPlayer play];
It seems that it does not recognize the bytes received as proper audio format.
Please help me to receive this issue.
Thanks.

Related

How to use sync skcipher in linux kernel?

I want to use sync encryption in linux kernel (since the code is run in interrupt context, which cannot sleep). Under /proc/crypto, there are several candidates which is marked sync, like __gcm(aes), __ctr(aes), __xts(aes). I tried to use this code example in kernel crypto API documentation, but get error when trying to allocate tfm using crypto_alloc_req. The code is as below:
static int test_skcipher(void)
{
struct crypto_skcipher *tfm = NULL;
struct skcipher_request *req = NULL;
u8 *data = NULL;
const size_t datasize = 512; /* data size in bytes */
struct scatterlist sg;
DECLARE_CRYPTO_WAIT(wait);
u8 iv[16]; /* AES-256-XTS takes a 16-byte IV */
u8 key[64]; /* AES-256-XTS takes a 64-byte key */
int err;
/*
* Allocate a tfm (a transformation object) and set the key.
*
* In real-world use, a tfm and key are typically used for many
* encryption/decryption operations. But in this example, we'll just do a
* single encryption operation with it (which is not very efficient).
*/
tfm = crypto_alloc_skcipher("__xts(aes)", 0, 0);
if (IS_ERR(tfm)) {
pr_err("Error allocating xts(aes) handle: %ld\n", PTR_ERR(tfm));
return PTR_ERR(tfm);
}
get_random_bytes(key, sizeof(key));
err = crypto_skcipher_setkey(tfm, key, sizeof(key));
if (err) {
pr_err("Error setting key: %d\n", err);
goto out;
}
/* Allocate a request object */
req = skcipher_request_alloc(tfm, GFP_KERNEL);
if (!req) {
err = -ENOMEM;
goto out;
}
/* Prepare the input data */
data = kmalloc(datasize, GFP_KERNEL);
if (!data) {
err = -ENOMEM;
goto out;
}
get_random_bytes(data, datasize);
/* Initialize the IV */
get_random_bytes(iv, sizeof(iv));
/*
* Encrypt the data in-place.
*
* For simplicity, in this example we wait for the request to complete
* before proceeding, even if the underlying implementation is asynchronous.
*
* To decrypt instead of encrypt, just change crypto_skcipher_encrypt() to
* crypto_skcipher_decrypt().
*/
sg_init_one(&sg, data, datasize);
skcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP,
crypto_req_done, &wait);
skcipher_request_set_crypt(req, &sg, &sg, datasize, iv);
err = crypto_wait_req(crypto_skcipher_encrypt(req), &wait);
if (err) {
pr_err("Error encrypting data: %d\n", err);
goto out;
}
pr_debug("Encryption was successful\n");
out:
crypto_free_skcipher(tfm);
skcipher_request_free(req);
kfree(data);
return err;
}
This code is basically the same as the document, only replacing xts(aes) with __xts(aes). I have already checked that __xts(aes) appears in /proc/crypto, but it seems kernel cannot find the corresponding algorithm. Is there any other documentation about how to use sync skcipher inside linux kernel? A working code example will be appreciated.
Today I have spent a lot of time on the same issue and during my "No-Straightforward-Documentation-Blues". I found your almost-the-same story here and without any reply (omg....). Thus, if you are still facing this problem, I hope it will be helpful.
GCM is about an AEAD cipher operation mode and this operation mode also reflects into linux/crypto. The transformations done by using skcipher is not suitable for GCM. AEAD does much more than just encrypt/decrypt data. It is capable of authenticating. So skcipher cannot "see" ciphers working under GCM mode, because it does not know how to handle them. So if you try to get a transformation handle for GCM by using skcipher, it will say "gc what?! gc who?!!". It is clearer now, because I also was considering that skcipher was about a meta data transformation stuff, but it is not. There is another "tfm" (poorly documented, btw) that can attend us.
Linux crypto api offers struct crypto_aead. The crypto_aead has all its correspondent convenience functions that skcipher has. In general it is almost the "same". However, it is necessary taking into consideration the particularities of GCM and how it is organized into crypto_aead stuff. Anyway, it is not so arcane (if you already know some aspects of the concepts behind of AEAD/GCM and I am taking into consideration that you know). Take a look:
int do_aes_gcm_min_sample(void) {
struct crypto_aead *tfm = NULL;
struct aead_request *req = NULL;
u8 *buffer = NULL;
size_t buffer_size = TEST_DATA_SIZE;
u8 *bp = NULL, *bp_end = NULL;
struct scatterlist sg = { 0 };
DECLARE_CRYPTO_WAIT(wait);
// INFO(Rafael): The majority of AES/GCM implementation uses 12 bytes iv (crypto_aead_ivsize()
// returned this, so for this reason I am using this "magic" value here.)
u8 iv[12] = { 0 };
u8 key[32] = { 0 }; // INFO(Rafael): The version of AES is defined by the size (in bytes) of the
// informed key. So, here we are using AES-256.
int err = -1;
tfm = crypto_alloc_aead("gcm(aes)", 0, 0);
if (IS_ERR(tfm)) {
err = PTR_ERR(tfm);
pr_err("AES/GCM min sample: crypto_alloc_aead() has failed: %d.\n", err);
goto do_aes_gcm_min_sample_epilogue;
}
// INFO(Rafael): Telling to api how many bytes will compound our MAC (a.k.a tag etc [etc no!...]).
err = crypto_aead_setauthsize(tfm, AES_GCM_TAG_SIZE);
if (err != 0) {
pr_err("AES/GCM min sample: crypto_aead_setauthsize() has failed: %d.\n", err);
goto do_aes_gcm_min_sample_epilogue;
}
// WARN(Rafael): In practice it could come from a `KDF(weak_usr_password)` stuff.
// Never ever use directly the "key" informed by the user.
// It demolishes the good will of any crypto algorithm on
// doing good encryption. Let's value hard work of cryptographers on
// seeking to create state of the art ciphers ;), please!
// So, I am only getting the key from a csprng that is a thing
// near to what an alleged good KDF is able to do with a weak password
// or at least must do.
get_random_bytes(key, sizeof(key));
err = crypto_aead_setkey(tfm, key, sizeof(key));
if (err != 0) {
pr_err("AES/GCM min sample: crypto_aead_setkey() has failed: %d.\n", err);
goto do_aes_gcm_min_sample_epilogue;
}
req = aead_request_alloc(tfm, GFP_KERNEL);
if (req == NULL) {
err = -ENOMEM;
pr_err("AES/GCM min sample: aead_request_alloc() has failed.\n");
goto do_aes_gcm_min_sample_epilogue;
}
req->assoclen = 0; // INFO(Rafael): No associated data, just reinforcing it.
// Anyway, when you want to also authenticated
// plain data (a.k.a AAD, associated data) you
// must indicate the size in bytes of the
// aad here and prepend your plaintext with
// aad.
get_random_bytes(iv, sizeof(iv));
// INFO(Rafael): The AES/GCM encryption primitive will also spit at the end of
// the encrypted buffer the n bytes asked tags generated by GHASH.
// Since we are using the same buffer for output stuff, this buffer
// must be able to fit the input and ***also*** the result.
buffer_size = TEST_DATA_SIZE + AES_GCM_TAG_SIZE;
buffer = kmalloc(buffer_size, GFP_KERNEL);
if (buffer == NULL) {
err = -ENOMEM;
pr_err("AES/GCM min sample: kmalloc() has failed.\n");
goto do_aes_gcm_min_sample_epilogue;
}
// INFO(Rafael): Copying the one block input...
memcpy(buffer, TEST_DATA, TEST_DATA_SIZE);
bp = buffer;
bp_end = bp + TEST_DATA_SIZE;
// INFO(Rafael): Currently buffer contains only the one dummy test block. Right?...
pr_info("Original data: ");
while (bp != bp_end) {
pr_info("%c\n", isprint(*bp) ? *bp : '.');
bp++;
}
// INFO(Rafael): ...however our scattterlist must be initialised
// by indicating the whole allocated buffer segment (including room
// for the tag). Because it will also output data, got it?
sg_init_one(&sg, buffer, buffer_size);
aead_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG |
CRYPTO_TFM_REQ_MAY_SLEEP, crypto_req_done, &wait);
// INFO(Rafael): Thus, for ***encrypting*** our input buffer is
// `TEST_DATA_SIZE == buffer_size - AES_GCM_TAG_SIZE`,
// since
// `buffer_size == TEST_DATA_SIZE + AES_GCM_TAG_SIZE`.
aead_request_set_crypt(req, &sg, &sg, buffer_size - AES_GCM_TAG_SIZE, iv);
err = crypto_wait_req(crypto_aead_encrypt(req), &wait);
if (err != 0) {
pr_err("AES/GCM min sample: error when encrypting data: %d.\n", err);
goto do_aes_gcm_min_sample_epilogue;
}
// INFO(Rafael): If aad would be also passed it would prepend the cryptogram.
// req-assoclen give you the clue of traverse or even skip it.
pr_info("Cryptogram: ");
// INFO(Rafael): Now buffer contains the authenticated cryptogram. I meant <cryptogram><MAC>.
// Here the intention is only print the cryptogram.
bp = buffer;
bp_end = bp + buffer_size - AES_GCM_TAG_SIZE;
while (bp != bp_end) {
pr_info("%c\n", isprint(*bp) ? *bp : '.');
bp++;
}
pr_info("Authentication tag: ");
// INFO(Rafael): Since bp is already pointing to the first byte of what should be the tag, let's only moving
// AES_GCM_TAG_SIZE bytes ahead the end marker of the output buffer.
bp_end += AES_GCM_TAG_SIZE;
while (bp != bp_end) {
pr_info("%c\n", isprint(*bp) ? *bp : '.');
bp++;
}
// INFO(Rafael): I hate incomplete samples, so let's decrypt, too.
// Decrypting with GCM involves checking if the tag informed at the end of cryptogram,
// is really the same of the on-the-fly calculated by GHASH. Thus, when decrypting the
// is necesary to indicate the cryptogram and ***also*** the tag, so here its size is
// expressed by buffer_size.
aead_request_set_crypt(req, &sg, &sg, buffer_size, iv);
// INFO(Rafael): What about testing if GCM is really detecting tampered data?
// Give it a try by uncomment all or even one of the following three lines.
//key[sizeof(key) >> 1] += 1;
//buffer[buffer_size >> 1] += 1;
//buffer[buffer_size - AES_GCM_TAG_SIZE + 1] += 1; // INFO(Rafael): Bit flipping MAC.
// INFO(Rafael): For the context of this sample, it would not be necessary. Anyway, we want to test
// corrupted key cases.
err = crypto_aead_setkey(tfm, key, sizeof(key));
if (err != 0) {
pr_err("AES/GCM min sample: crypto_aead_setkey() has failed: %d.\n", err);
goto do_aes_gcm_min_sample_epilogue;
}
err = crypto_wait_req(crypto_aead_decrypt(req), &wait);
if (err != 0) {
pr_err("AES/GCM min sample: Error when decrypting data, it seems tampered. "
"Ask for a retransmission or verify your key.\n");
goto do_aes_gcm_min_sample_epilogue;
}
// INFO(Rafael): If aad would be also passed it would prepend the plaintext.
// req->assoclen give you the clues of how to traverse or even
// skipping it. But even skipped it must be passed by the
// decryption routine. Because it also authenticates the whole
// buffer, got it?
pr_info("Authenticated plaintext: ");
bp = buffer;
bp_end = bp + buffer_size - AES_GCM_TAG_SIZE; // INFO(Rafael): It will not reallocate the buffer so, let's exclude the MAC.
// Due to it maybe should be good to ensure a buffer_size multiple of four.
// It would keep this simpler. Anyway you can apply a more sophisticated
// padding technique, but for this sample I think it express the main idea.
while (bp != bp_end) {
pr_info("%c\n", isprint(*bp) ? *bp : '.');
bp++;
}
do_aes_gcm_min_sample_epilogue:
if (req != NULL) {
aead_request_free(req);
}
if (tfm != NULL) {
crypto_free_aead(tfm);
}
if (buffer != NULL) {
kfree(buffer);
}
return err;
}
If you want to test quickly, you can clone this sample repo here.
I also have tried to run this code on kernel 4.4.14 (it has needed some minor workarounds, it seems fine, but I did not tested it a lot). For the sake of brevity I will not comment what I done but if you are interested feel free on cloning the repo and grasp into those "hacks". But the code is only a sample, much more must be done to deliver a good crypto there, avoid copying and pasting, please.

Watchkit radio streaming

I am trying to playback a streamed media (radio).
I have failed in the attempt and also I have received the error message: App Transport Security (ATS).
How I can play a streaming radio?
My first try:
NSURL *url = [NSURL URLWithString:#"http://ccma-radioa-int-abertis-live.hls.adaptive.level3.net/int/mp4:catradio/chunklist.m3u8"];
WKAudioFileAsset *fileAsset = [WKAudioFileAsset assetWithURL:url];
WKAudioFilePlayerItem *playerItem = [WKAudioFilePlayerItem playerItemWithAsset:fileAsset];
self.audioPlayer = [WKAudioFilePlayer playerWithPlayerItem:playerItem];
if (self.audioPlayer.status == WKAudioFilePlayerStatusReadyToPlay) {
[self.audioPlayer play];
}
My second try:
NSURL *url = [NSURL URLWithString:#"http://ccma-radioa-int-abertis-live.hls.adaptive.level3.net/int/mp4:catradio/chunklist.m3u8"];
NSDictionary *options = #{WKMediaPlayerControllerOptionsAutoplayKey:#YES};
[self presentMediaPlayerControllerWithURL:url
options:options
completion:^(BOOL didPlayToEnd, NSTimeInterval endTime, NSError * __nullable error) {
if (!didPlayToEnd) {
NSLog(#"The player did not play all the way to the end. The player only played until time - %.2f.", endTime);
}
if (error) {NSLog(#"There was an error with playback: %#.", error);}
}];
Does anyone know how to get it?

Nested bridge transfer call with ARC

I'm trying to get the email address of a contact and the type of the email address (work/home). This is the code I've written
//Assume that 'personRef' of type ABRecordRef is available
....
ABMultiValueRef emailRef = ABRecordCopyValue(personRef, kABPersonEmailProperty);
NSMutableArray *emailAddresses = nil, *emailAddressLabels = nil;
int ctr = ABMultiValueGetCount(emailRef);
if(ctr!=0) {
emailAddresses = [[NSMutableArray alloc]init];
emailAddressLabels = [[NSMutableArray alloc]init];
for(int i=0; i<ctr; i++) {
NSString *eId = (__bridge_transfer NSString*)ABMultiValueCopyValueAtIndex(emailRef, i);
[emailAddresses addObject:eId];
CFStringRef label = ABMultiValueCopyLabelAtIndex (emailRef, i);
if(label!=NULL) {
NSString *eType = (__bridge_transfer NSString*)ABAddressBookCopyLocalizedLabel(label);
if([eType isEqualToString:#""]) {
[emailAddressLabels addObject:#"Email"];
} else {
[emailAddressLabels addObject:eType];
}
CFRelease(label);
}
}
}
The code crashes at CFRelease(label), but to prevent memory leak, I should be doing it. When I try the following
NSString *eType = (__bridge_transfer NSString*) ABAddressBookCopyLocalizedLabel(ABMultiValueCopyLabelAtIndex (emailRef, i));
I get the following warning from ARC
1. Call to function 'ABMultiValueCopyLabelAtIndex' returns a Core Foundation object with a +1 retain count
2. Object leaked: allocated object is not referenced later in this execution path and has a retain count of +1
Now the question I have is, How to do a nesting __bridge_transfer call?
NSString *eType = (__bridge_transfer NSString*)
ABAddressBookCopyLocalizedLabel(
ABMultiValueCopyLabelAtIndex (emailRef, i) /* <-- this object is leaked */
);
This code is invalid because you leak the Label here (which I realise is your point I guess?).
You should run that code under the Instruments NSZombie instrument, it will trace all the retain/releases and you'll have a clue what's going on, because frankly, looking at the code, I don't see why it's wrong.

How to store images and keys to NSMutableDictionary

I'm receiving an error when writing to my NSMutableDictionary "dictionary".
When implementing (below) from my item list class:
- (void)setImage:(UIImage *)i forKey:(NSString *)s
{
[dictionary setObject:i forKey:s];
// Create full path for image
NSString *imagePath = [self imagePathForKey:s];
// Turn image into JPEG data,
NSData *d = UIImageJPEGRepresentation(i, 0.5);
// Write it to full path
[d writeToFile:imagePath atomically:YES];
}
From DetailViewController:imagePickerController:
- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info
{
NSString *oldKey = [_detailItem imageKey];
//did the item already have an image
if (oldKey) {
//delete old image
[[RoomList sharedStore] deleteImageForKey:oldKey];
}
//get picked image from info dictionary
UIImage *image = [info objectForKey:UIImagePickerControllerOriginalImage];
//create CFUUID object; it knows how to create uique identifier strings
CFUUIDRef newUniqueID = CFUUIDCreate(kCFAllocatorDefault);
//create string from unique identifier
CFStringRef newUniqueIDString = CFUUIDCreateString(kCFAllocatorDefault, newUniqueID);
//use unique ID to set our item's imageKey
NSString *key = (__bridge NSString *)newUniqueIDString;
[_detailItem setImageKey:key];
NSLog(#"image key: %#", key);
[_detailItem setImage:image forKey:key];
//tell objects ointed to by newUniqueIDString and newUniqueID to lose an owner to avoid
//memory leak
CFRelease(newUniqueIDString);
CFRelease(newUniqueID);
//remove image picker (dismiss method)
[self dismissViewControllerAnimated:YES completion:nil];
[self configureView];
}
I am being met with the following error, that is tied to the [_detailItem setImage:image forKey:key]; operation in imagePickerController: -[NSManagedObject setImage:forKey:]: unrecognized selector sent to instance 0x8195770
My guess is that either the UIImage *image, NSString *key, or both are not able to be passed to the objectForKey method "setImage:forKey". Any way of fixing this, or getting around it?
Thanks!

AVAudioPlayer sounds garbled on iPhone without headphones

I have 23 music files (10-20 seconds in length) that I want to play when the user selects them and presses the play button. It works find on my iPad and on the simulators, with or without earphones. But on the iPhone device without earphones, around 80% of the files sound garbled, the volume is low, and some are barely legible. I can plug the earphones in, and they sound fine. As soon as I unplug the earphones, the sounds are garbled again.
Since some of the files always play fine, I converted all the original mp3's that I was using into the AAC format with a sample rate of 44100. But the same files play fine, while most are distorted when playing on the iphone device.
I've tried several settings for AVAudioSession category: playback, playAndRecord, ambient. I've tried storing the files in core data and using initWithData instead of initWithContentsOfURL. Can anyone suggest what else I might try? Or how I might get clues into why I'm having this problem?
Here is my code. There are 3 relevant files. The ViewController that has a button to play a selected music file, an AVPlaybackSoundController that has all the audio logic, and a Singleton that has an array with the filenames.
My ViewController creates an AVPlaybackSoundController through the Nib.
IBOutlet AVPlaybackSoundController *avPlaybackSoundController;
#property (nonatomic, retain) AVPlaybackSoundController *avPlaybackSoundController;
- (IBAction) playButtonPressed:(id)the_sender
{
self.currentRhythmSampleIndex = arc4random() % [[CommonData sharedInstance].rhythmSampleArray count];
[self.avPlaybackSoundController playMusic:self.currentRhythmSampleIndex];
self.playBtn.hidden=YES;
self.pauseBtn.hidden=NO;
}
AVPlaybackSoundController.m
- (void) awakeFromNib
{
[self initAudioSession:AVAudioSessionCategoryPlayback];
[self initMusicPlayer];
}
- (void) initAudioSession:(NSString *const)audioSessionCategory
{
NSError* audio_session_error = nil;
BOOL is_success = YES;
is_success = [[AVAudioSession sharedInstance] setCategory:audioSessionCategory error:&audio_session_error];
if(!is_success || audio_session_error){
NSLog(#"Error setting Audio Session category: %#", [audio_session_error localizedDescription]);
}
[[AVAudioSession sharedInstance] setDelegate:self];
audio_session_error = nil;
is_success = [[AVAudioSession sharedInstance] setActive:YES error:&audio_session_error];
if(!is_success || audio_session_error){
NSLog(#"Error setting Audio Session active: %#", [audio_session_error
localizedDescription]);
}
}
- (void) initMusicPlayer
{
NSError* file_error = nil;
NSURL* file_url = [[NSURL alloc] initFileURLWithPath:[[NSBundle mainBundle]
pathForResource:#"musicFile1" ofType:#"m4a"]
isDirectory:NO];
avMusicPlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:file_url
error:&file_error];
if(!file_url || file_error){
NSLog(#"Error loading music file: %#", [file_error
localizedDescription]);
}
self.avMusicPlayer.delegate = self;
self.avMusicPlayer.numberOfLoops = 0xFF;
[file_url release];
}
-(void) playMusic:(NSUInteger)rhythmSampleIdx
{
CommonData *glob = [CommonData sharedInstance];
if (rhythmSampleIdx > [glob.rhythmSampleArray count])
return; // bug if we hit here
// if we were already playing it, just resume
if ((self.avMusicPlayer) && (rhythmSampleIdx == self.currentRhythmSampleIndex)){
[self.avMusicPlayer play];
}
else{ // re-init player with new rhythm
if (self.avMusicPlayer){
[self.avMusicPlayer stop];
[self.avMusicPlayer setCurrentTime:0.0];
}
NSError* error = nil;
RhythmSample *rhythmSample = [glob.rhythmSampleArray objectAtIndex:rhythmSampleIdx];
NSString* rhythm_filename = [self getRhythmFileName:rhythmSampleIdx];
NSURL* file_url = [[[NSURL alloc] initFileURLWithPath:[[NSBundle mainBundle]
pathForResource:rhythm_filename ofType:#"m4a"] isDirectory:NO]autorelease];
self.avMusicPlayer = [[[AVAudioPlayer alloc] initWithContentsOfURL:file_url error:&error]autorelease];
[self.avMusicPlayer prepareToPlay]; // shouldn't be needed since we are playing immediately
if(error){
NSLog(#"Error loading music file: %#", [error localizedDescription]);
}else{
self.avMusicPlayer.delegate = self;
self.avMusicPlayer.numberOfLoops = 0xFF; // repeat infinitely
self.currentRhythmSampleIndex = rhythmSampleIdx;
[self.avMusicPlayer play];
}
}
}
I finally decided to obtain the sound files from a different source. The new sound files play fine. I still have no theory as to why the old sound files would play with earphones, but not without them. I really dislike finding solutions without understanding why they work.

Resources