I am trying to put together some code in Python 3.6 to help test the computer hardware that passes through my hands as an IT tech.
I'd like to have a script that plays a simple sine wave tone on the left speaker, then the right and then the both speakers together.
I have found a potentially helpful script over at Pyaudio How to get sound on only one speaker but some of the code to actually run it is missing - chiefly the code for making sin wave tones. I have looked around online and have tried reverse-engineering this back into the code on that page but the maths is a little high-level for me! Sorry.
Thanks,
Will
Update:
I think I have found a partial (albeit long-winded solution) with 'sounddevice' for python 3
#!/usr/bin/env python3
import argparse
import logging
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("filename", help="audio file to be played back")
parser.add_argument("-d", "--device", type=int, help="device ID")
args = parser.parse_args()
try:
import sounddevice as sd
import soundfile as sf
data, fs = sf.read(args.filename, dtype='float32')
sd.play(data, fs, device=args.device, blocking=True, mapping=[1])
status = sd.get_status()
if status:
logging.warning(str(status))
except BaseException as e:
# This avoids printing the traceback, especially if Ctrl-C is used.
raise SystemExit(str(e))
The main chunk of code is repeat twice more but with "mapping = [1]" changed to "mapping = [2]" to test the right speaker and finally with "mapping = [?]" removed in the final block to test both speakers.
I found this over at https://python-sounddevice.readthedocs.io/en/0.2.1/examples.html.
Of course, if anyone knows a quicker and graceful way of getting this done, please share!
You could generate the sine tone directly in Python instead of loading it from a file. I've written some tutorials about creating simple sine tones:
https://nbviewer.jupyter.org/github/mgeier/python-audio/blob/master/simple-signals.ipynb
http://nbviewer.jupyter.org/github/spatialaudio/communication-acoustics-exercises/blob/master/intro.ipynb
Those tutorials use NumPy, because it makes manipulating the audio buffers very easy. But you can of course also do it in pure Python, if you prefer.
Here's an example:
#!/usr/bin/env python3
import math
import sounddevice as sd
sd.default.device = None
sd.default.samplerate = samplerate = 48000
duration = 1.5
volume = 0.3
frequency = 440
# fade time in seconds:
fade_in = 0.01
fade_out = 0.3
buffer = memoryview(bytearray(int(duration * samplerate) * 4)).cast('f')
for i in range(len(buffer)):
buffer[i] = volume * math.cos(2 * math.pi * frequency * i / samplerate)
fade_in_samples = int(fade_in * samplerate)
for i in range(fade_in_samples):
buffer[i] *= i / fade_in_samples
fade_out_samples = int(fade_out * samplerate)
for i in range(fade_out_samples):
buffer[-(i + 1)] *= i / fade_out_samples
for mapping in ([1], [2], [1, 2]):
sd.play(buffer, blocking=True, mapping=mapping)
sd.sleep(500)
Note that this code is using 32-bit floating point numbers (each one using 4 bytes), that's why we reserve 4 times more bytes in our bytearray than the required number of samples.
Related
I am trying to get a stream of 24-bit audio samples from some USB audio device using Python.
I already searched for some solutions and found this thread that is using PyAudio stream with a format of pyaudio.paInt24, unfortunately I still got 16-bit samples. I tried to record with PyAudio stream and save with WAVE, sample code:
def initialize(self):
self.p = pyaudio.PyAudio()
self.stream = self.p.open(format=self.format, # pyaudio.paInt24
channels=self.channels, # 1
rate=self.fs, # 16000
input=True,
input_device_index=self.input_device_index,
frames_per_buffer=self.chunk) # 1024
def run(self):
frames = []
for _ in range(0, int(self.fs / self.chunk * self.duration)):
data = self.stream.read(self.chunk)
frames.append(data)
self.stream.stop_stream()
self.stream.close()
self.p.terminate()
# write recording to a WAV file
wf = wave.open(self.recorded_file, 'wb')
wf.setnchannels(self.channels)
wf.setsampwidth(self.p.get_sample_size(self.format))
wf.setframerate(self.fs)
wf.writeframes(b''.join(frames))
wf.close()
I also tried the same the SoundDevice, sample code:
def initialize(self):
sd.default.samplerate = self.fs # 16000
sd.default.channels = self.channels # 1
sd.default.device = self.input_device_index
def run(self):
data = sd.rec(int(self.duration * self.fs), dtype=np.int32)
sd.wait()
# Save file
wf = wave.open(self.recorded_file, 'wb')
wf.setnchannels(self.channels)
wf.setsampwidth(3)
wf.setframerate(self.fs)
wf.writeframes(b''.join(data))
In order to make sure that the problem occurs when receiving the samples and not while saving them, I made sure that a 24-bit audio file can indeed be saved with WAVE, using this sample code:
import wave
with wave.open(path_to_file_to_read, mode='rb') as rb:
white_noise = rb.readframes(rb.getframerate())
with wave.open(path_to_file_to_save+'_save_with_wave.wav', mode='wb') as wb:
wb.setnchannels(rb.getnchannels())
wb.setsampwidth(rb.getsampwidth())
wb.setnframes(rb.getnframes())
wb.setframerate(rb.getframerate())
wb.writeframesraw(white_noise)
Also, I put a break point right after the recording was finished to take a look on the recorded samples and I see that it looks like 16-bit:
I will note that in order to make sure that the USB audio device does send me the 24-bit samples I did a recording using Audacity and indeed I got 24-bit, for some reason I can not do anything similar with Python.
Is there a simple way to record 24-bit samples using Python code?
Both PyAudio and SoundDevice use portaudio bindings between some windows sound architecture and python.
PortAudio does not suppport full depth 24 bit, and the result is 16 bit of audio with additional 8 zero bits.
SoundCard supports 24 bit - https://soundcard.readthedocs.io/en/latest/
In order to get real 24 bit samples, you can use the following:
import soundcard as sc
import soundfile as sf
FS = 16000
TIME_IN_SEC = 5
FRAMES = FS * TIME_IN_SEC
DEVICE_NAME = 'default'
FILE_PATH = 'abc.wav'
# get device
audio_device = sc.get_microphone(DEVICE_NAME)
# receive samples
data = audio_device.record(samplerate=FS, numframes=FRAMES)
# write recording to a WAV file
sf.write(FILE_PATH, data, FS, subtype='PCM_24')
I am trying to learn how to work with nls.lm in the R library minpack.lm by using the Rosenbrock function to see if the algorithm converges to the global minimum at f(x,y) = (1,1). I do so both with and without the analytic Jacobian. In both instances, I get a warning telling me that the algorithm has decided to revert the maximum number of iterations specified in the call to nls.lm to 1024:
Warning messages:
1: In nls.lm(par = initpar, fn = objective_rosenbrock, jac = gradient_rosenbrock, :
resetting `maxiter' to 1024!
2: In nls.lm(par = initpar, fn = objective_rosenbrock, jac = gradient_rosenbrock, :
lmder: info = -1. Number of iterations has reached `maxiter' == 1024.
The algorithm never quite reaches (1,1) as a result given my initial guess of (-1.2, 1.0). I found the source code for the library on GitHub and the following lines of code are pertinent here:
https://github.com/cran/minpack.lm/blob/master/src/nls_lm.c
OS->maxiter = INTEGER_VALUE(getListElement(control, "maxiter"));
if(OS->maxiter > 1024) {
OS->maxiter = 1024;
warning("resetting `maxiter' to 1024!");
}
Is there any logic to why the maximum number of iterations is capped to 1024? Something with bits and 2^10? I would like to use the library for a different application, but this cap on iterations might prevent that. Any insight would be appreciated.
Git blame says that this code limiting the max iterations was introduced in version 1.1-0, in 2008. The NEWS file for the package only goes back as far as version 1.1-6. I can't find the code in any public repo other than the one you point to (which is only a CRAN mirror; it doesn't contain any comments/commit messages/etc. from developers that might give us clues.)
Other than contacting the maintainer I think it's going to be hard to figure out what the rationale is for this limit.
I do have some guesses though.
The only places that maxiter is actually used in the code are here and here - in R code, not Fortran or C code, so it seems extremely unlikely that we are dealing with something like a 10-bit unsigned integer type (which seems an unlikely choice in any case). I think the limitation is there because we also have a buffer defined for holding trace information here:
double rsstrace[1024];
which, as you can see, is hard-coded to a length of 1024. Presumably bad things would happen if we tried to stuff 1025 iterations'-worth of tracing information into this array ...
My suggestions:
change all instances of '1024' in the code to something larger and see what happens. There are only four:
$ find . -type f -exec grep -Hn 1024 {} \;
./src/nls_lm.c:141: if(OS->maxiter > 1024) {
./src/nls_lm.c:142: OS->maxiter = 1024;
./src/nls_lm.c:143: warning("resetting `maxiter' to 1024!");
./src/minpack_lm.h:20: double rsstrace[1024];
it would be best to #define MAXITER 2048 (or whatever) in src/minpack_lm.h and use that instead of the numerical value.
Contact the maintainer (maintainer("minpack.lm")) and ask them about this issue.
I like the simplicity of dask and would love to use it for scraping a local supermarket. My multiprocessing.cpu_count() is 4, but this code only achieves a 2x speedup. Why?
from bs4 import BeautifulSoup
import dask, requests, time
import pandas as pd
base_url = 'https://www.lider.cl/supermercado/category/Despensa/?No={}&isNavRequest=Yes&Nrpp=40&page={}'
def scrape(id):
page = id+1; start = 40*page
bs = BeautifulSoup(requests.get(base_url.format(start,page)).text,'lxml')
prods = [prod.text for prod in bs.find_all('span',attrs={'class':'product-description js-ellipsis'})]
prods = [prod.text for prod in prods]
brands = [b.text for b in bs.find_all('span',attrs={'class':'product-name'})]
sdf = pd.DataFrame({'product': prods, 'brand': brands})
return sdf
data = [dask.delayed(scrape)(id) for id in range(10)]
df = dask.delayed(pd.concat)(data)
df = df.compute()
Firstly, a 2x speedup - hurray!
You will want to start by reading http://dask.pydata.org/en/latest/setup/single-machine.html
In short, the following three things may be important here:
you only have one network, and all the data has to come through it, so that may be a bottleneck
by default, you are using threads to parallelise, but the python GIL limits concurrent execution (see the link above)
the concat operation is happening in a single task, so this cannot be parallelised, and with some data types may be a substantial part of the total time. You are also drawing all the final data into your client's process with the .compute().
There are meaningful differences between multiprocessing and multithreading. See my answer here for a brief commentary on the differences. In your case that results in only getting a 2x speedup instead of, say, a 10x - 50x plus speedup.
Basically your problem doesn't scale as well by adding more cores as it would by adding threads (since it's I/O bound... not processor bound).
Configure Dask to run in multithreaded mode instead of multiprocessing mode. I'm not sure how to do this in dask but this documentation may help
For learning purposes, I wrote a small C Python module that is supposed to perform an IPC cuda memcopy to transfer data between processes. For testing, I wrote equivalent programs: one using theano's CudaNdarray, and the other using pycuda. The problem is, even though the test programs are nearly identical, the pycuda version works while the theano version does not. It doesn't crash: it just produces incorrect results.
Below is the relevant function in the C module. Here is what it does: every process has two buffers: a source and a destination. Calling _sillycopy(source, dest, n) copies n elements from each process's source buffer to the neighboring process's dest array. So, if I have two processes, 0 and 1, processes 0 will end up with process 1's source buffer and processes 1 will end up with process 0's source buffer.
Note that to transfer cudaIpcMemHandle_t values between processes, I use MPI (this is a small part of a larger project which uses MPI). _sillycopy is called by another function, "sillycopy" which is exposed in Python by the standard Python C API methods.
void _sillycopy(float *source, float* dest, int n, MPI_Comm comm) {
int localRank;
int localSize;
MPI_Comm_rank(comm, &localRank);
MPI_Comm_size(comm, &localSize);
// Figure out which process is to the "left".
// m() performs a mod and treats negative numbers
// appropriately
int neighbor = m(localRank - 1, localSize);
// Create a memory handle for *source and do a
// wasteful Allgather to distribute to other processes
// (could just use an MPI_Sendrecv, but irrelevant right now)
cudaIpcMemHandle_t *memHandles = new cudaIpcMemHandle_t[localSize];
cudaIpcGetMemHandle(memHandles + localRank, source);
MPI_Allgather(
memHandles + localRank, sizeof(cudaIpcMemHandle_t), MPI_BYTE,
memHandles, sizeof(cudaIpcMemHandle_t), MPI_BYTE,
comm);
// Open the neighbor's mem handle so we can do a cudaMemcpy
float *sourcePtr;
cudaIpcOpenMemHandle((void**)&sourcePtr, memHandles[neighbor], cudaIpcMemLazyEnablePeerAccess);
// Copy!
cudaMemcpy(dest, sourcePtr, n * sizeof(float), cudaMemcpyDefault);
cudaIpcCloseMemHandle(sourcePtr);
delete [] memHandles;
}
Now here is the pycuda example. For reference, using int() on a_gpu and b_gpu returns the pointer to the underlying buffer's memory address on the device.
import sillymodule # sillycopy lives in here
import simplempi as mpi
import pycuda.driver as drv
import numpy as np
import atexit
import time
mpi.init()
drv.init()
# Make sure each process uses a different GPU
dev = drv.Device(mpi.rank())
ctx = dev.make_context()
atexit.register(ctx.pop)
shape = (2**26,)
# allocate host memory
a = np.ones(shape, np.float32)
b = np.zeros(shape, np.float32)
# allocate device memory
a_gpu = drv.mem_alloc(a.nbytes)
b_gpu = drv.mem_alloc(b.nbytes)
# copy host to device
drv.memcpy_htod(a_gpu, a)
drv.memcpy_htod(b_gpu, b)
# A few more host buffers
a_p = np.zeros(shape, np.float32)
b_p = np.zeros(shape, np.float32)
# Sanity check: this should fill a_p with 1's
drv.memcpy_dtoh(a_p, a_gpu)
# Verify that
print(a_p[0:10])
sillymodule.sillycopy(
int(a_gpu),
int(b_gpu),
shape[0])
# After this, b_p should have all one's
drv.memcpy_dtoh(b_p, b_gpu)
print(c_p[0:10])
And now the theano version of the above code. Rather than using int() to get the buffers' address, the CudaNdarray way of accessing this is via the gpudata attribute.
import os
import simplempi as mpi
mpi.init()
# select's one gpu per process
os.environ['THEANO_FLAGS'] = "device=gpu{}".format(mpi.rank())
import theano.sandbox.cuda as cuda
import time
import numpy as np
import time
import sillymodule
shape = (2 ** 24, )
# Allocate host data
a = np.ones(shape, np.float32)
b = np.zeros(shape, np.float32)
# Allocate device data
a_gpu = cuda.CudaNdarray.zeros(shape)
b_gpu = cuda.CudaNdarray.zeros(shape)
# Copy from host to device
a_gpu[:] = a[:]
b_gpu[:] = b[:]
# Should print 1's as a sanity check
print(np.asarray(a_gpu[0:10]))
sillymodule.sillycopy(
a_gpu.gpudata,
b_gpu.gpudata,
shape[0])
# Should print 1's
print(np.asarray(b_gpu[0:10]))
Again, the pycuda code works perfectly and the theano version runs, but gives the wrong result. To be precise, at the end of the theano code, b_gpu is filled with garbage: neither 1's nor 0's, just random numbers as though it were copying from a wrong place in memory.
My original theory regarding why this was failing had to do with CUDA contexts. I wondered if it was possible theano was doing something with them that meant that the cuda calls made in sillycopy were run under a different CUDA context than had been used to create the gpu arrays. I don't think this is the case because:
I spent a lot of time digging deep in theano's code and saw no funny business being played with contexts
I would expect such a problem to result in a bad crash, not an incorrect result, which is not what happens.
A secondary thought is whether this has to do the fact that theano spawns several threads, even when using a cuda backend, which can be verified this by running "ps huH p ". I don't know how threads might affect anything, but I have run out of obvious things to consider.
Any thoughts on this would be greatly appreciated!
For reference: the processes are launched in the normal OpenMPI way:
mpirun --np 2 python test_pycuda.py
Note: marked as community wiki.
Is there a good analysis of why visual programming languages still haven't taken off? We're still coding these days 'linearly' in a 80x25 text window; while the concepts we represent (data structures, algorithms) seem like they can be more intuitively represented visually.
Two approaches to programming that aren't just simple text come to mind:
Structured Editing (which I know from Kirill Osenkov's blog).
LabView.
I think Structured Editing is pretty interesting, because it takes the 'braces with idententation' convention, which has proven really useful for keeping code organized, to its logical extreme. I think it could really be something, if someone were to make a brilliant (from a usability perspective) implementation of it.
The LabView approach, on the other hand, doesn't excite me so much. The visual idioms don't seem powerful and obvious enough, compared to text. I haven't used LabView much though, so it's probably better than I think.
Dont forget with VS 2010 (.NET 4), its now multi monitor supported, which mean you can now allow editors, designers and tool-windows to be moved outside the top-level window and positioned anywhere you want across to any monitor on your system.
An 80x25 text window? Really? Not to compare sizes, but my text window is quite a bit bigger than that. But regardless of that, I personally can't imagine a visual programming language that would satisfy me. For technical information, text is far more information-dense than video. I would much rather skim an article about a technical subject than watch a video about that subject in five times as much time (seriously guys, knock it off with the videos already).
In a similar way, I would much rather spend a few seconds typing a few lines of code, than a few minutes dragging and dropping things around to accomplish the same thing. It's about conciseness, and expressiveness. Visual programming languages just don't have it, in my experience. Good for teaching fundamentals of programming? Sure. Alice is pretty neat. But not for day-to-day work.
On a somewhat-related note, Code Bubbles is an interesting take on improving the "80x25 text window".
There is quite some mixing and matching.
For instance, people do use GUI editors like NetBeans Matisse or VS.Net because some things are easier to draw than to code. Some people use GUI data model editors: it's much easier, faster and (I would argue) produces a better result than writing DDL. Even when you write code, you have all sorts of graphical tools to help you understand what you're doing (the eclipse hierarchy view, for example).
On the other hand, we do still use similar text editors to the ones people used 30 years ago for a lot of our work. :) It's obvious that there is value to be had from both.
simulink is part of matlab and works great for engineering problems
Visual programming languages have never taken off because no one has done it right yet. In the same way that C++ / visual studio were the right technology for people at the time of their advent.
However, the age of talking to our machines (Alex voice service), and programming with nicer tools than text editors is upon us.
Here's the start of one I'm working on. I'm trying to bootstrap my project since if you're making a cool programming tool, why wouldn't the tool itself eventually be written in the tool's input language. I did start out at first by rendering my own graphs with PyQt5 / QGraphicsScene but debugging a 2D scene is actually extremely hard - that is unless you have a visual graph to program with instead! So rendering my own graph and writing the graph editor comes after I can get basic graphs to run. My favorite general purpose graph editor is yEd. It outputs .graphml which is good because the networkx library for python can already read in .graphml (Only problem is loading in the graph colors + other properties than position; so feature will wait til we are doing our own graph drawings).
Here is an example input graph:
Here's some basic code to run it:
import networkx as nx
from PyQt5.QtCore import QThread, QObject, pyqtSignal
from PyQt5.QtGui import QColor
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton
import re
import sys
EDAT = 2
NDAT = 1
class CodeGraphThread(QThread):
ifRgx = r'^if\s+(.+)\s*$'
elseRgx = r'\s+|^$'
def __init__(self, graph, parent=None):
super(CodeGraphThread, self).__init__(parent)
self._nodes = {}
self.setGraph(graph)
self._retVal = None
self._locals = []
def setGraph(self, graph):
self._graph = graph
G = graph.G()
nodes = [x for x in G.nodes(data=True) if x[NDAT]['label'] == 'start']
if nodes: self._setStart(nodes[0][0])
def _setStart(self, nstr):
self._nodes['start'] = nstr
def start(self):
self._running = True
self._nodes['current'] = self._nodes['start']
QThread.start(self)
def _exec(self, codeText):
try:
exec('self._retVal=' + codeText)
except:
try:
exec(codeText)
except:
self.codeGraph().errorMessage.emit('Coudln\'t execute code: "' + codeText + '"')
def returnVal(self):
return self._retVal
def run(self):
while self._running:
cg = self.codeGraph()
G = cg.G()
current = self._nodes['current']
#TODO transfer over to regex system
data = [d for x,d in G.nodes(data=True) if x == current and 'label' in d and d['label'] not in ['start']]
if data:
codeText = data[0]['label']
self._exec(codeText)
rgx = self.ifRgx
edges = cg.edgesFr(current, rgx)
if edges:
e= edges[0]
ifArg = cg.matches(rgx).group(1)
self._exec(ifArg)
if self.returnVal():
self._nodes['current'] = e[1]
continue
rgx = self.elseRgx
edges = cg.edgesFr(current, rgx)
edges += cg.edgesFr(current, None)
if edges:
e = edges[0]
self._nodes['current'] = e[1]
continue
break
def codeGraph(self):
return self._graph
class CodeGraph(QObject):
errorMessage = pyqtSignal(str)
statusMessage = pyqtSignal(str)
_rgxMemo = {}
def __init__(self, gmlpath=None):
QObject.__init__(self)
if gmlpath != None:
self.loadGraphML(gmlpath)
else:
self._gmlpath = None
self._G = nx.MultiDiGraph()
self._thread = CodeGraphThread(self)
def G(self):
return self._G
def loadGraphML(self, gmlpath):
self._gmlpath = gmlpath
self._G = nx.read_graphml(gmlpath)
def saveGraphML(self, gmlpath):
self._gmlpath = gmlpath
nx.write_graphml(self._G, gmlpath)
def debugPrintNodes(self):
print(self._G.nodes(data=True))
def debugPrintEdges(self):
print(self._G.edges(data=True))
def matches(self, rgx):
if rgx in self._rgxMemo:
return self._rgxMemo[rgx][1]
return None
def rgx(self, rgx):
if rgx not in self._rgxMemo:
self._rgxMemo[rgx] = [re.compile(rgx), None]
return self._rgxMemo[rgx][0]
def rgxMatch(self, rgx, string):
if rgx not in self._rgxMemo:
rgx_ = self.rgx(rgx)
else:
rgx_ = self._rgxMemo[rgx][0]
match = self._rgxMemo[rgx][1] = rgx_.match(string)
return match
def edgesFr(self, n0, rgx):
if rgx != None:
return [(u,v,d) for u,v,d in self.G().edges(data=True) if u == n0 and 'label' in d and self.rgxMatch(rgx, d['label'])]
else:
return [(u,v,d) for u,v,d in self.G().edges(data=True) if u == n0 and 'label' not in d]
if __name__ == '__main__':
cg = CodeGraph('unnamed0.graphml')
cgthread = CodeGraphThread(cg)
def printError(errorMsg):
print(errorMsg)
cg.errorMessage.connect(printError)
# Qt application reqd for QThread testing
app = QApplication(sys.argv)
win = QMainWindow()
win.setWindowTitle('PyGraphML Practice 0')
button0 = QPushButton('Start thread running')
button0.clicked.connect(cgthread.start)
win.setCentralWidget(button0)
win.show()
sys.exit(app.exec_())
Current issues: Python 3 doesn't handle exec / locals() well (hence use of self.x instead of just x), so thinking of using a 3rd-party python interpreter or just statically modifying the code.
Not saying that my tool does anything right yet. For things to be done right, there also must be auto-refactoring tools, debugging, etc.