Is it possible to extract the song info from a live HLS M3U8 radio stream using for example python or any other program?
I can currently view the song info by opening the response in Google DevTools, but I'd like to automatically scrape/extract the song title and save it to a file each time the song changes.
You should try using the Python library (https://github.com/globocom/m3u8) to parse and manipulate m3u8 manifests instead of writing your own script. You have to check if it supports the tags you are trying to get metadata for. If not you should be able to easily extend the library to add the functionality you need.
import m3u8
def get_movie(line, data, lineno):
if line.startswith('#MOVIE-NAME:'):
custom_tag = line.split(':')
data['movie'] = custom_tag[1].strip()
m3u8_obj = m3u8.load('http://videoserver.com/playlist.m3u8',
custom_tags_parser=get_movie)
print(m3u8_obj.data['movie']) # million dollar baby
Related
https://www.nyse.com/quote/XNYS:A
After I access the above URL, I open Developer Tools in Firefox. Then change the date in HISTORIC PRICES, then click 'GO'. The table is updated. But I don't see relevant HTTP requests sent in devtools.
So this means that the data has already been downloaded in the first request. But I can not figure out how to extract the raw data of the table. Could anybody take a look at how to extract the raw data from the table? (Note that I don't want to use methods like selenium, I want to stay with raw HTTP requests to get the raw data.)
EDIT: websocket is mentioned in the comment. But I can't see it in Developer Tools. I add websocket tag anyway in case somebody knows more about websocket can chime in.
I am afraid you cannot extract javascript rendered content without selenium. You can always make use of a headless browser(you don't see any instance on your screen, the only pitfall is that you have to wait until the page fully loads) and it won't bother you anymore.
In other words, all the other scraping libs are based on urls and forms. Scrapy can post forms but not run javascripts.
Selenium will save the day, all you lose is a couple of seconds for each attempt(will be milliseconds if it is run in frontend). You can share page source with driver.page_source and it can be directly used for parsing(as a html text) with BeautifulSoup or whatever.
You can do it with requests-html, for example let's grab the first row of the table:
from requests_html import HTMLSession
session = HTMLSession()
url = 'https://www.nyse.com/quote/XNYS:A'
r = session.get(url)
r.html.render(sleep=7)
first_row = r.html.find('.flex_tr', first=True)
print(first_row.text)
Output:
06/18/2021
146.31
146.83
144.94
145.01
3,220,680
As #Nikita said you will have to wait the page loading (here 7sec but maybe less), but if you want to do multiple requests you can do it asynchronously !
I am streaming real time tweets using R script and sending it to Power BI using (Streaming data and REST API) to create a live streaming dashboard.
I have to create a tile which shows tweets over time. But the date/time field does not get reflected in the tile, it is just empty.
I'm not sure if I have to change the datatype or format of the field? Below is how the date/time field looks in R.
The streaming dataset is as below:
Below is the tile I created (Shows blank)
If you don't see anything in the tile, probably your dataset doesn't received anything. Try to post some data manually, e.g. from a PowerShell script (place the correct URL):
$endpoint = "https://api.powerbi.com.com/beta/........."
$Time_Stamp = ($Time_Stamp = Get-Date).Datetime
$payload = #{ "date" = $Time_Stamp; "count" = 98.6; "score" = 98.6 }
Invoke-RestMethod -Method Post -Uri "$endpoint" -Body (ConvertTo-Json #($payload))
Does anything shows up in the tile? As we see from your screenshot, you enabled Historic data analysis. This means that you can create reports with this dataset. Try to make one to show the contents in a table. Is there any rows in this dataset?
UPDATE: Actually, to be sure, copy the sample PowerShell code to post to your dataset and try it. Go to the list of datasets in your workspace, find your one in the list and click on the (i) icon. Then from API Info panel go to PowerShell and copy/paste the code in PowerShell ISE to execute it:
Also, keep in mind that the tile will auto update itself only in a dashboard. You may need to refresh the report to get the up to date data, if you are looking at the tile in a report.
So I'm trying to create a GUI that searches a dictionary for a word, returns the entries and suggestions and downloads the audio, returning "related words" (sometimes more than one) that have audio. Then, the user clicks on one of three buttons to play the audio, based on the dialect. I can get the words to play, but I can't get the last played one to close so I can download another.
Here is the relevant code:
#staticmethod
def munster():
""" This method plays the Munster recording, if it exists"""
url = QtCore.QUrl.fromLocalFile("./CanM.mp3")
content = QtMultimedia.QMediaContent(url)
player = QtMultimedia.QMediaPlayer()
player.setMedia(content)
player.play()
player.stateChanged(app.quit)
When it gets to the last line it gives me this error:
TypeError: native Qt signal is not callable
Is there any way to close the player to allow downloading of another audio file with the same name?
For the record, I'm using the latest release of PyQt5, and Qt5.4 on Python 3.4 with Windows 81. However, I'm also working on this project (with the same Qt and PyQt) on an Archlinux system as well, and would like it to be portable easily.
Edit: Quite easy, once I started looking at what all was contained within the Player class. Just needed to use the disconnect method.
Edit2: Edited to show complete working code
#staticmethod
def play_audio(dialect):
file_names = {'Munster': './CanM.mp3', 'Connacht': './CanC.mp3', 'Ulster': './CanU.mp3'}
url = QtCore.QUrl.fromLocalFile(os.path.abspath(file_names[dialect]))
content = QtMultimedia.QMediaContent(url)
player = QtMultimedia.QMediaPlayer()
player.setMedia(content)
player.play()
player.stateChanged.connect(lambda: player.disconnect())
User making the call is asked to dial an extension. This is done by 1#playing a prompt with Background and then 2#wait_for_digit. Based on the extension that has been dialed, the destination number is determined and the call is forwarded to that number.
If the called person doesn't not answer, then Playback is used to play a prompt that asks the user to record the voice message; recording the voice message is done with the Record application.
This Record application is always generating empty wav files, size 44 bytes. If I remove the 1#playing a prompt with Background the Record application is generating proper files. If the Background is included, all recordings are empty.
I am using Perl Asterisk::AGI module.
$agi->exec('Answer');
....
.....
$agi->exec('Background', 'en/extra/please-enter-the-extension,n'); # this is the troubling part
my $my_extension = $agi->wait_for_digit(5000);
....
.....
$agi->exec('Playback', 'en/extra/the-party-you-are-calling&en/extra/is-curntly-busy,noanswer');
$agi->exec('Playback', 'en/vm-intro,noanswer');
my $file = 'xyz.wav';
$agi->exec('Record', "$file,0,10,k");
...
...
What should I do to make it work as I want it to?
Thank you.
UPDATE 1:
The same script is working without glitches now. Not sure if something unrelated to the script has changed.
Most likly you have check your codecs. IF you use g729 or g723 and no transcoder,it just can't write in wav format.
I just have simple filter graph whick takes media streams from rtsp source[ generally h264 and mp4 ] and save them using an mp4 muxer to a file...
RtspSourceFilter ---> MP4 Muxer ---> File Writer.
It works OK. But i have constraint[ new requirment now]. I have to write file based on two criteria: their size and duration...Suppose that user can define rules such as:
if duration > 1 hour or size > 1 gb then write stream to new file
In my graph in order to this,
I have to stop my graph based on conditions and create and start new
one with new file name...
That is bad since at ever file i have to re-connect my source and possibly lost some data...
What is the best way to deal with it ?
My Solution: [ But not satisfied with it ]
I have the source code of RtspSourceFilter and MP4Muxer[open source] so that forgot FileWriter...MP4 Muxer became a writer with Muxer...So stop it internally and write when necessary and then cretae new file...Do some buffering for not looosing data...
RTSP Source Filter ---> New MP4 Writer [ a writer with mp4 muxer in it]
But this introduce unnecessary complexity...Now i became maintainer of MUX operation via New MP4 Writer...Since i have no time to really understand what Mux do, i have to modify-hack it to behave what i want... Analogy: I have car and i will make a helicopter from it...It will be very ugly and un-trusted helicopter...Probably my New MP4 Filter [code] will be so...[ Big Ball of Mud]
It sounds like GMFBridge may be of use to you. It allows you to create one source graph, and multiple sink graphs. Then when your constraint is met, bridge the source graph to a new sink graph.
If you put the bridge in buffer (non-discard) mode, you should not loose any samples.
However, you will have to investigate if this solution works for you. Have a look at the sample applications for a quick overview.