How SoundCloud stream their mp3 files? - nginx

Im trying to implement same method to play .mp3 files with JwPlayer on my webiste like SoundCloud has.
I have a large mp3 files which sometime have 4 hours length. So I don't want to load it as one source file in JwPlayer.
I see soundcloud stream thier mp3 files in partial (160k pices). How they do that?
I try configure my nginx to
location /uploads/ {
mp4;
mp4_buffer_size 160k;
mp4_max_buffer_size 200k;
}
But it does not working. Mp3 still load in a one big request.
I try option 2 read file manualy by php script like
if (isset($_SERVER['HTTP_RANGE'])) {
$c_start = $start;
$c_end = $end;
list(, $range) = explode('=', $_SERVER['HTTP_RANGE'], 2);
if (strpos($range, ',') !== false) {
header('HTTP/1.1 416 Requested Range Not Satisfiable');
header("Content-Range: bytes $start-$end/$size");
exit;
}
if ($range == '0-') {
$c_start = 0;
}
else {
$range = explode('-', $range);
$c_start = (int)$range[0];
if($c_start > 2407200) {
exit;
}
$c_end = (isset($range[1]) && is_numeric($range[1])) ? $range[1] : $c_start + $end;
}
$c_end = ($c_end > $size) ? $size : $c_end;
if ($c_start > $c_end || $c_start > $size - 1 || $c_end >= $size) {
header('HTTP/1.1 416 Requested Range Not Satisfiable');
header("Content-Range: bytes $start-$end/$size");
exit;
}
$start = $c_start;
$end = $c_end;
$length = $end - $start + 1; // Calculate new content length
fseek($fp, $start);
header('HTTP/1.1 206 Partial Content');
}
header("Content-Range: bytes $start-$end/$size");
header("Content-Length: $length");
$buffer = 1024 * 8;
while(!feof($fp) && ($p = ftell($fp)) <= $end) {
if ($p + $buffer > $end) {
$buffer = $end - $p + 1;
}
set_time_limit(0); // Reset time limit for big files
echo fread($fp, $buffer);
flush(); // Free up memory. Otherwise large files will trigger PHP's memory limit.
}
fclose($fp);
It's not bad, but when I STOP playing mp3 the script load other pieces of file.
So how SoundCloud do this? When play music they load music in small pieces and when stop player the request stop too.

Related

Write to text file but escape special characters

I'm trying to use R to write some perl script to a text file. I just cannot figure out how to escape certain characters?
I've used backslash (single and double), square brackets, "\\Q...\\E", etc. but still can't make it work.
Any assistance would be appreciated. Thanks in advance!
taskFilename = "example.txt"
cat("
#!/usr/bin/perl
use strict;
use warnings;
use File::Find;
use File::Temp qw(tempfile);
my #imagedir_roots = ("/Users/Ross/Desktop/images");
my $parallel = 8;
my $exiftool_command = 'exiftool -all= -tagsfromfile # -all:all --gps:all --xmp:geotag -unsafe -icc_profile -overwrite_original';
# Create the (temporary) -# files
my #atfiles;
my #atfilenames;
for (my $i = 0; $i < $parallel; ++$i) {
my ($fh, $filename) = tempfile(UNLINK => 1);
push #atfiles, $fh;
push #atfilenames, $filename;
}
# Gather all JPG image files and distribute them over the -# files
my $nr = 0;
find(sub { print { $atfiles[$nr++ % $parallel] } "$File::Find::name\n" if (-f && /\.(?:jpg|jpeg)/i); }, #imagedir_roots);
# Process all images in parallel
printf("Processing %d JPG files...\n", $nr);
for (my $i = 0; $i < $parallel; ++$i) {
close($atfiles[$i]);
my $pid = fork();
if (!$pid) {
# Run exiftool in the background
system qq{$exiftool_command -# \"$atfilenames[$i]\"};
last;
}
}
# Wait for processes to finish
while (wait() != -1) {}
", fill = TRUE, file = taskFilename
)
I also played around with this once. If I remember correctly:
the double quotes you need to escape with \
if you want to write a \ you also need to escape it with \
if you want to write a \" you need to \\"
taskFilename = "example.txt"
cat("
#!/usr/bin/perl
use strict;
use warnings;
use File::Find;
use File::Temp qw(tempfile);
my #imagedir_roots = (\"/Users/Ross/Desktop/images\");
my $parallel = 8;
my $exiftool_command = 'exiftool -all= -tagsfromfile # -all:all --gps:all --xmp:geotag -unsafe -icc_profile -overwrite_original';
# Create the (temporary) -# files
my #atfiles;
my #atfilenames;
for (my $i = 0; $i < $parallel; ++$i) {
my ($fh, $filename) = tempfile(UNLINK => 1);
push #atfiles, $fh;
push #atfilenames, $filename;
}
# Gather all JPG image files and distribute them over the -# files
my $nr = 0;
find(sub { print { $atfiles[$nr++ % $parallel] } \"$File::Find::name\n\" if (-f && /\\.(?:jpg|jpeg)/i); }, #imagedir_roots);
# Process all images in parallel
printf(\"Processing %d JPG files...\n\", $nr);
for (my $i = 0; $i < $parallel; ++$i) {
close($atfiles[$i]);
my $pid = fork();
if (!$pid) {
# Run exiftool in the background
system qq{$exiftool_command -# \\\"$atfilenames[$i]\\\"};
last;
}
}
# Wait for processes to finish
while (wait() != -1) {}
", fill = TRUE, file = taskFilename
)

Downloading files from the Internet using Powershell with progress

I have been working on a powershell script that uses a .txt file to download multiple files from tinyurls. I have been successful in using Jobs to make this happen simultaneously, thanks to those on this forum.
The project requires some pretty large files to be downloaded, and using the current method has no progress indicator. I figured some users might think the program died. Looking for a way give a status of where it is in the download. Here is what I came up with, but I'm lost in how to pipe this information back out to the console. Any suggestions?
#Checks to see if NT-Download folder is on the Desktop, if not found, creates it
$DOCDIR = [Environment]::GetFolderPath("Desktop")
$TARGETDIR = "$DOCDIR\NT-Download"
if(!(Test-Path -Path $TARGETDIR )){
New-Item -ItemType directory -Path $TARGETDIR
}
$filepaths = Resolve-Path "files.txt"
Get-Content "$filepaths" | Foreach {
Start-Job {
function Save-TinyUrlFile
{
PARAM (
$TinyUrl,
$DestinationFolder
)
$response = Invoke-WebRequest -Uri $TinyUrl
$filename = [System.IO.Path]::GetFileName($response.BaseResponse.ResponseUri.OriginalString)
$filepath = [System.IO.Path]::Combine($DestinationFolder, $filename)
$totalLength = [System.Math]::Floor($response.get_ContentLength()/1024)
$responseStream = $response.GetResponseStream()
$buffer = new-object byte[] 10KB
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $count
try
{
$filestream = [System.IO.File]::Create($filepath)
$response.RawContentStream.WriteTo($filestream)
$filestream.Close()
while ($count -gt 0)
{
[System.Console]::CursorLeft = 0
[System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
$targetStream.Write($buffer, 0, $count)
$count = $responseStream.Read($buffer,0,$buffer.length)
$downloadedBytes = $downloadedBytes + $count
}
"`nFinished Download"
$targetStream.Flush()
$targetStream.Close()
$targetStream.Dispose()
$responseStream.Dispose()
}
finally
{
if ($filestream)
{
$filestream.Dispose();
}
}
}
Save-TinyUrlFile -TinyUrl $args[0] -DestinationFolder $args[1]
} -ArgumentList $_, "$TARGETDIR"
}
Have a look at Write-Progress
PS C:> for ($i = 1; $i -le 100; $i++ )
{write-progress -activity "Search in Progress" -status "$i% Complete:" -percentcomplete $i;}
Far more simple way : rely on Bits:
Start-BitsTransfer -Source $tinyUrl

sending arduino data to my database using EtherCard.h library

I am having a problem in sending Arduino data into my database. I am currently using WAMPSERVER. I have an arduino mini w/ ATMEGA 328 and I am using ENC28J60. The problem is I have tried different libraries and examples to send data into the server but I failed. I just notice that some libraries are not compatible with my enc28J60. I have tried UIPEthernet.h, Ethernet.h, etherShield.h and EtherCard.h. The etherShield.h and EtherCard.h seemed to work just fine. But I prefer to use EtherCard.h because I heard etherShield is the older lib. I know a little php.
I think the things that might guide me is to see a working example of using the EtherCard.h library demonstrating the sending of sensor data from arduino into my database. The network setup I am currently working on is that, the ENC28J60 is connected in my home network with an ip address of 192.168.10.10. The server to which I placed the database is my laptop with an IP address of 192.168.10.2. I am placing the php files in this directory C:\wamp\www. I hope I have explained it well. I'm sorry for my English.
I am currently working of a port of my homewatch server from Arduino Uno w Ethernet lib to AVR-NETIO with Ethercard lib (or UIPEthernet).
On the server side there is a Apache/PHP/MySQL server.
To update data on a webserver connected database, you can use POST or GET requests to a php web page. These data can then be added easily to a database and the same or another web page can also display the data of the database.
<?php
require 'dbhelp.php';
//header('Content-type: text/plain');
echo "<html>";
echo "<head>";
echo "<meta name='viewport' content='width=device-width, user-scalable=false, initial-scale=1;'>";
echo "</head>";
echo "<body>" . "<h2>Temperatur und Luftfeuchte</h2>";
// GET requests are sent as a query string on the URL:
// GET index.html?name1=value&name2=value HTTP/1.1
// Host: about.com
// http://192.168.0.40/homewatch/index.php?channel=1&temp=165&humidity=80&datetime=010120131234
if($DEBUG){
print("<pre>");
print_r($_GET);
print("</pre>");
}
openDB();
if( isset($_GET['channel']) && isset($_GET['temp']) && isset($_GET['humidity']) )
{
if($DEBUG)
echo "<p>addData()";
$c=$_GET['channel'];
$t=$_GET['temp'];
$h=$_GET['humidity'];
addData($c, $t, $h);
if($DEBUG)
echo "<p>all done";
//listData();
echo "<p>OK updated data for channel:" . $c . "</p>";
}
else
{
if($DEBUG)
echo "listData()";
//listData();
//echo "<p>missing arg";
}
echo "<p><a href='linechart_hour.php'>Stunden-Übersicht</a></p>";
echo "<p><a href='barchart_days.php'>Tages-Übersicht</a></p>";
echo "<p><a href='http://www.unwetterzentrale.de/uwz/getwarning_de.php?plz=41363&uwz=UWZ-DE&lang=de'>Unwetterwarnungen Jüchen</a></p>";
echo showAllCharts();
//#################################################################################
// see http://code.google.com/p/googlechartphplib/wiki/GettingStarted
//#################################################################################
// don't forget to update the path here
require './lib/GoogleChart.php';
$chart = new GoogleChart('lc', 500, 200);
// manually forcing the scale to [0,100]
$chart->setScale(0,100);
// add one line
$data = new GoogleChartData(array(49,74,78,71,40,39,35,20,50,61,45));
$chart->addData($data);
// customize y axis
$y_axis = new GoogleChartAxis('y');
$y_axis->setDrawTickMarks(false)->setLabels(array(0,50,100));
$chart->addAxis($y_axis);
// customize x axis
$x_axis = new GoogleChartAxis('x');
$x_axis->setTickMarks(5);
$chart->addAxis($x_axis);
echo $chart->toHtml();
//#################################################################################
// END
//#################################################################################
echo "<p>v0.9" . "</body>" . "</html>";
?>
the dbhelp.php file creates the DB automatically if not already exists:
<?php
//global dbhelp.php file
$server="localhost";
$user="root";
$passwd="password";
$names=array(
1 => "Aussen",
2 => "Schlaf",
3 => "Andreas");
$DEBUG=false;
function openDB(){
global $DEBUG;
global $user;
global $passwd;
$link = mysql_connect("localhost", $user, $passwd);
if(!$link){
echo "SqlConnect failed";
echo mysql_error();
}
else
if($DEBUG)
echo "SqlConnect OK";
$query="CREATE database IF NOT EXISTS avrdb;";
$result = mysql_query($query, $link);
if(!$result){
echo "CreateDB failed";
echo mysql_error();
}
else
if($DEBUG)
echo "CreateDB OK";
$result = mysql_select_db("avrdb", $link);
if(!$result){
echo "SelectDB failed";
echo mysql_error();
}
else
if($DEBUG)
echo "SelectDB OK";
$query="CREATE TABLE IF NOT EXISTS `avrtemp` (" .
"`id` int(11) NOT NULL AUTO_INCREMENT, " .
"`channel` int(11), " .
"`temp` int(11), " .
"`humidity` int(11), " .
"`date_time` TIMESTAMP, " .
"PRIMARY KEY (`id`) );";
$result = mysql_query($query, $link);
if(!$result){
echo "CreateTable failed";
echo mysql_error();
}
else
if($DEBUG)
echo "CreateTable OK";
return true;
}
...
function addData($c,$t,$h){
global $DEBUG;
$lastStoredDateTime=getLastStoredDateTime();
if($DEBUG){
echo "<p>" . $c . "</p>\n";
echo "<p>LastDateTime=".$lastStoredDateTime."</p>\n";
}
// add data but do not save seconds (use 00)
$query="INSERT INTO avrtemp (`channel`,`temp`,`humidity`,`date_time`)".
" VALUES ( $c,$t,$h,".
// " DATE_FORMAT(NOW()),".
" DATE_FORMAT('".$lastStoredDateTime."', ".
" '%Y-%c-%d %H:%i') )";
if($DEBUG)
echo "<p>" . $query;
$result = mysql_query($query);
if(!$result){
echo "addData failed";
echo mysql_error();
}
else
if($DEBUG)
echo "addData OK";
}
The Arduino posts (using GET) updated data every 5 minutes. The data arrives at the Arduino via 433MHz wireless sensors every minute or so.
Using the Ethernet Lib the data update is done via
char cBuf[maxBuf+1];
const char getHttpString[] =
"GET /homewatch/index.php?channel=%i&temp=%i&humidity=%i&time=%i\0";
...
// and send data: /index.php?channel=1&temp=165&humidity=80&datetime=010120131234
if (client.connect(server, 80)) {
Serial.println("connected. Send GET...");
snprintf(cBuf, maxBuf,
getHttpString,
sensorData[idxChannel].channel,
sensorData[idxChannel].temp,
sensorData[idxChannel].humidity,
sensorData[idxChannel].time_long
);
Serial.println(F("####"));
Serial.println(cBuf);
Serial.println(F("####"));
client.println(cBuf);
...
https://github.com/hjgode/homewatch/blob/master/arduino/SketchBook/WebClientTemperature/WebClientTemperature.ino:
The string "GET /homewatch/index.php?channel=%i&temp=%i&humidity=%i&time=%i\0" is simply filled with actual sensor data and the server saves that to its database.
The Ethercard lib comes with similar examples: see the examples noipClient, pachube and webclient. Unfortunately I was not yet able to adopt these for my AVR-NETIO.
The code to post (GET) some data to the server I am currently working is
void sendData(int idxChannel){
Serial.print(F("in sendData for idx=")); Serial.println(idxChannel);
byte sd;
int channel=sensorData[idxChannel].channel;
int temp=sensorData[idxChannel].temp;
int humidity=sensorData[idxChannel].humidity;
char* stime="201301011122";
stime=printDateTime((char*)&stime, 13, sensorData[idxChannel].time_long);
if(sensorData[idxChannel].bUpdated==0){
//nothing new
if (MYDEBUG==0){
Serial.println(F("leaving for not updated channel data"));
goto exit_sendData;
}
else
{
sensorData[idxChannel].channel=idxChannel;
sensorData[idxChannel].temp=222;
sensorData[idxChannel].humidity=55;
sensorData[idxChannel].time_long=now();
}
}
// generate two fake values as payload - by using a separate stash,
// we can determine the size of the generated message ahead of time
sd = stash.create();
stash.print("0,");
stash.println((word) millis() / 123);
stash.print("1,");
stash.println((word) micros() / 456);
stash.save();
// generate the header with payload - note that the stash size is used,
// and that a "stash descriptor" is passed in as argument using "$H"
Stash::prepare(PSTR("GET http://$F/homewatch/index.php?channel=$D&temp=$D&humidity=$D&time=$S" "\r\n"
"Host: $F" "\r\n"
"Content-Length: $D" "\r\n"
"\r\n"
"$H"),
website,
channel,
temp,
humidity,
stime,
website, stash.size(), sd);
// send the packet - this also releases all stash buffers once done
ether.tcpSend();
exit_sendData:
lastConnectionTime = now();//millis();
Serial.println(F("...end of sendData()"));
}
As you see, there is a formatted string
PSTR("GET http://$F/homewatch/index.php?channel=$D&temp=$D&humidity=$D&time=$S"
which is filled with variables. $F fills from Flash Memory, $D a number var, $S from a RAM memory string. Using this without my SensorCode (which uses Interrupts) is working.

Powershell Scheduled Task Open Web Page to Execute Script

In an effort to setup a "cron job" like scheduled task on Windows I've setup a Powershell script using code recommended via previous stackoverflow question.
I have some backups that I need to cleanup daily and delete old backups so I created a asp.net script to perform this task - the file name is BackupCleanup.aspx and I have confirmed that the ASP.net script does work when executed on its own by visiting the above url - I however cannot get it to execute using the Powershell script below.
The Powershell Script code I'm using is:
$request = [System.Net.WebRequest]::Create("http://127.0.0.1/BackupCleanup.aspx")
$response = $request.GetResponse()
$response.Close()
I have created this file with a PS1 extension, it shows properly in my os (Windows 2008) - I have tried both manually executing this task by right clicking and choosing "Run with Powershell" and also have scheduled this as a task - both to no avail.
I cannot figure out why the script does not work - any help would be GREATLY appreciated.
Here is the Powershell script I use to call up web pages using IE. Hopefully this will work for you as well.
Function NavigateTo([string] $url, [int] $delayTime = 100)
{
Write-Verbose "Navigating to $url"
$global:ie.Navigate($url)
WaitForPage $delayTime
}
Function WaitForPage([int] $delayTime = 100)
{
$loaded = $false
while ($loaded -eq $false) {
[System.Threading.Thread]::Sleep($delayTime)
#If the browser is not busy, the page is loaded
if (-not $global:ie.Busy)
{
$loaded = $true
}
}
$global:doc = $global:ie.Document
}
Function SetElementValueByName($name, $value, [int] $position = 0) {
if ($global:doc -eq $null) {
Write-Error "Document is null"
break
}
$elements = #($global:doc.getElementsByName($name))
if ($elements.Count -ne 0) {
$elements[$position].Value = $value
}
else {
Write-Warning "Couldn't find any element with name ""$name"""
}
}
Function ClickElementById($id)
{
$element = $global:doc.getElementById($id)
if ($element -ne $null) {
$element.Click()
WaitForPage
}
else {
Write-Error "Couldn't find element with id ""$id"""
break
}
}
Function ClickElementByName($name, [int] $position = 0)
{
if ($global:doc -eq $null) {
Write-Error "Document is null"
break
}
$elements = #($global:doc.getElementsByName($name))
if ($elements.Count -ne 0) {
$elements[$position].Click()
WaitForPage
}
else {
Write-Error "Couldn't find element with name ""$name"" at position ""$position"""
break
}
}
Function ClickElementByTagName($name, [int] $position = 0)
{
if ($global:doc -eq $null) {
Write-Error "Document is null"
break
}
$elements = #($global:doc.getElementsByTagName($name))
if ($elements.Count -ne 0) {
$elements[$position].Click()
WaitForPage
}
else {
Write-Error "Couldn't find element with tag name ""$name"" at position ""$position"""
break
}
}
#Entry point
# Setup references to IE
$global:ie = New-Object -com "InternetExplorer.Application"
$global:ie.Navigate("about:blank")
$global:ie.visible = $true
# Call the page
NavigateTo "http://127.0.0.1/BackupCleanup.aspx"
# Release resources
$global:ie.Quit()
$global:ie = $null
I had the same issue. I manually opened powershell and executed my script and I received "WebPage.ps1 cannot be loaded because running scripts is disabled on this system.".
You have to allow scripts to run
Execute the below in PowerShell
Set-ExecutionPolicy RemoteSigned -Scope LocalMachine

How can I perform HTTP PUT uploads to a VMware ESX Server in PowerShell?

VMware ESX, ESXi, and VirtualCenter are supposed to be able to support HTTP PUT uploads since version 3.5. I know how to do downloads, that's easy. I've never done PUT before.
Background information on the topic is here: http://communities.vmware.com/thread/117504
You should have a look at the Send-PoshCode function in the PoshCode cmdlets script module ... it uses a POST, not a PUT, but the technique is practically identical. I don't have PUT server I can think of to test against, but basically, set your $url and your $data, and do something like:
param($url,$data,$filename,[switch]$quiet)
$request = [System.Net.WebRequest]::Create($url)
$data = [Text.Encoding]::UTF8.GetBytes( $data )
## Be careful to set your content type appropriately...
## This is what you're going to SEND THEM
$request.ContentType = 'text/xml;charset="utf-8"' # "application/json"; # "application/x-www-form-urlencoded";
## This is what you expect back
$request.Accept = "text/xml" # "application/json";
$request.ContentLength = $data.Length
$request.Method = "PUT"
## If you need Credentials ...
# $request.Credentials = (Get-Credential).GetNetworkCredential()
$put = new-object IO.StreamWriter $request.GetRequestStream()
$put.Write($data,0,$data.Length)
$put.Flush()
$put.Close()
## This is the "simple" way ...
# $reader = new-object IO.StreamReader $request.GetResponse().GetResponseStream() ##,[Text.Encoding]::UTF8
# write-output $reader.ReadToEnd()
# $reader.Close()
## But there's code in PoshCode.psm1 for doing a progress bar, something like ....
$res = $request.GetResponse();
if($res.StatusCode -eq 200) {
[int]$goal = $res.ContentLength
$reader = $res.GetResponseStream()
if($fileName) {
$writer = new-object System.IO.FileStream $fileName, "Create"
}
[byte[]]$buffer = new-object byte[] 4096
[int]$total = [int]$count = 0
do
{
$count = $reader.Read($buffer, 0, $buffer.Length);
if($fileName) {
$writer.Write($buffer, 0, $count);
} else {
$output += $encoding.GetString($buffer,0,$count)
}
if(!$quiet) {
$total += $count
if($goal -gt 0) {
Write-Progress "Downloading $url" "Saving $total of $goal" -id 0 -percentComplete (($total/$goal)*100)
} else {
Write-Progress "Downloading $url" "Saving $total bytes..." -id 0
}
}
} while ($count -gt 0)
$reader.Close()
if($fileName) {
$writer.Flush()
$writer.Close()
} else {
$output
}
}
$res.Close();
In the VI Toolkit Extensions use Copy-TkeDatastoreFile. It will work with binaries.

Resources