I have a relatively old Android phone that I use only for podcast listening. I'm subscribed to a few podcasts, but since the wireless around the house is not so great, the wifi connection of the phone gets interrupted from time to time while downloading new episodes (10 min per episode). Somehow this causes the episodes to be sometimes corrupted, so I have to re-download when at home again or sometimes even error check the SD card.
Since I have a Debian server running locally, I am looking for a way to mirror the podcast rss feeds locally. So that my podcatcher on the phone is subscribed to local feeds, which will be much faster and the chances are smaller that the connection is interrupted during download.
The closest I got is this perl library: MP3::PodcastFetch
Is there anything for Debian in the repositories that can do this?
Check subscribed feed regularly (ie. cron)
Download new episodes
Host a local rss feed pointing to the local copies of the podcast
As the podcast is delivered via HTTP you can use a caching HTTP proxy on the local debian box. You might have a look at squid3 and this tutorial
Related
We have a daily need to ship about 500 MB of compressed image files (about 280K each).
Currently we do this the fast easy way. A web server, and downloads via http.
We are now looking at the putting a better client (nw.js) on the client side. So we have the opportunity to improve the transport protocol.
Data flows only one way.
We have a couple of thoughts but I would love to hear better ideas.
Using a HTTP2 (SPDY) compliant server, and the Chromium hooks in
nw.js for HTTP2 receiving
Using a TCP connection (custom node.js server -> node code in nw.js)
Perhaps we should look at QUIC: https://www.chromium.org/quic
Would bundling this into a zip file (which would not decrease bytecount, since it is already compressed) help?
What does oneDrive, GoogleDrive, and dropbox do in these cases?
Any thoughts?
Has anyone tried ASPERA: http://asperasoft.com/software/transfer-clients/
Windows 10 systems can take advanatge of "TCP Fast Open" technology which you can read about here:
https://en.wikipedia.org/wiki/TCP_Fast_Open
To enable this technology, Chromium accepts this parameter:
--enable-tcp-fastopen
From what I've read this only works in Windows 10 but I've no idea about other platforms. Good luck.
My company just purchased identical Unix servers to the ones we are currently using. The new servers are going to be hosted about 1000 miles away in Langhorn, PA while the current servers are hosted locally. Once the hosted servers are finished being set up, the local servers will be going away. My boss asked me to try and find a tool that could help us determine how much longer it will take to move files from the server to our local PC for the local servers vs. the remote servers.
Does anyone know of a tool that could accomplish this? Obviously I could just move a large file and use wall clock time, but depending on what is running on the server, I doubt this would be very accurate.
Thanks in advance!
Aaron
Most FTP programs i've used show transfer times, FileZilla for example:
Status: File transfer successful, transferred 118,782,740 bytes in 11 seconds
I've moved from constant contact to Drupal Simplenews/Mimemail/SMTP. Previously the open rate was around 50% for constant contact, but now it's 4-5% for the same list via the mentioned setup. Mail is getting out from the server, but it's having an issue anyway.
Here's the setup:
-The e-mail list consists of approximately 80,000 addresses which is queued at 10,000 e-mails per cron run (which runs hourly).
-The server is a Dual Core2Quad machine with 2GB of RAM.
-When mail is being sent, the mail queue will usually go up to ~1000 at the beginning of the hour before reducing to ~250 by the time the next cron occurs.
-Newsletter is themed to display custom style for newsletter on send
-Newsletter is received by some, but appears to be bounced by many (based on low open rate_
-I've added SPF, domain keys, and a PTR record to the DNS
-Server hostname (listed in ptr) is different from hosted domain
-Very low spam number via Spamassassin
-IP and domain are not blacklisted
-Mail goes out via SMTP module on delivery.
Any ideas?
The best way to satisfy your needs is by making the download of a module. I recommend this one: http://drupal.org/sandbox/E-goi/1110712
I've never had a good experience trying to rely on Simplenews. The issue with Drupal cron here is that you're also competing with everything else in Drupal during cron trying to process.
I moved everything to Campaign Monitor.
Imagine you have a web site that you want to send a lot of data. Say 40 files totaling the equivalence of 2 hours of upload bandwidth. You expect to have 3 connection losses along the way (think: mobile data connection, WLAN vs. microwave). You can't be bothered to retry again and again. This should be automated. Interruptions should not cause more data loss than neccessary. Retrying a complete file is a waste of time and bandwidth.
So here is the question: Is there a software package or framework that
synchronizes a local directory (contents) to the server via HTTP,
is multi-platform (Win XP/Vista/7, MacOS X, Linux),
can be delivered as one self-contained executable,
recovers partially uploades files after interrupted network connections or client restarts,
can be generated on a server to include authentication tokens and upload target,
can be made super simple to use
or what would be a good way to build one?
Options I have found until now:
Neat packaging of rsync. This requires an rsync (server) instance on the server side that is aware of a privilege system.
A custom Flash program. As I understand, Flash 10 is able to read a local file as a bytearray (indicated here) and is obviously able to speak HTTP to the originating server. Seen in question 1344131 ("Upload image thumbnail to server, without uploading whole image").
A custom native application for each platform.
Thanks for any hints!
Related work:
HTML5 will allow multiple files to be uploaded or at least selected for upload "at once". See here for example. This is agnostic to the local files and does not feature recovery of a failed upload.
Efficient way to implement a client multiple file upload service basically asks for SWFUpload or YUIUpload (Flash-based multi-file uploaders, otherwise "stupid")
A comment in question 997253 suggests JUpload - I think using a Java applet will at least require the user to grant additional rights so it can access local files
GearsUploader seems great but requires Google Gears - that is going away soon
What application do you use to monitor HTTP communication on OS X?
Charles Proxy
Charles is an HTTP proxy / HTTP
monitor / Reverse Proxy that enables a
developer to view all of the HTTP
traffic between their machine and the
Internet. This includes requests,
responses and the HTTP headers (which
contain the cookies and caching
information).
Runs on JAVA. Available on OSX, Linux and Windows.
I like TcpCatcher. It is free and 100% java based so it works fine on Mac OS X.
Not only, you will be able to monitor HTTP communication but you will also be able to change requests / responses on the fly which opens very interesting possibilities..
There is a dedicated tutorial on capturing iPhone's HTTP communication.
If you're looking to trace application traffic, Wireshark is the best tool I've found - it can log and decode HTTP and many other protocols, and the GUI's search tools make finding the messages you're interesting in pretty quick and painless.
Other reasons I recommend this:
It's quick to install
It captures traffic straight from the network card, there is no need to change the application or set up proxies etc. It'll even read dumps captured from tcpdump and similar tools offline
It's multi-platform (works on Windows/Mac/Linux and others)
It's open source
HTTPTracer
http://simile.mit.edu/wiki/HTTPTracer
You could also use dTrace to monitor in even more detail, if that's what you need.
I second using Charles, it's a really excellent tool for HTTP examination. When used with the iPhone simulator (or any other OS X application) Charles automatically sets up the system settings to use itself as a proxy so you only have to launch and run. It also is very easy to examine the traffic in a few different ways, and has a very lenient free trial version that is fully featured (time limited to an hour with a few nag screens) so you can give it a good try.
Depends on what you mean by monitor...
If you simply want to know/stop when an installed application (or the OS) tries to "phone home", then I recommend LittleSnitch.
The peace of mind you gain is well worth the loss of weight from your bank account.