I am starting to work on a project where I need to stream Twitter data using PowerTrack/GNIP and I have to be honest when I say I am very very inexperienced when it comes to networks and I have absolutely no knowledge when it comes to Data Stream (HTTP), how they work etc.
Are there any resources out there that go through all of this in simple terms? I would love to be able to map Data streaming process in my head before I start looking at APIs etc.
Thanks
Take a look at the following two resources which give a good overview of video streaming. Video streaming has probably more background available and should help you understand the concepts:
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html
http://www.jwplayer.com/blog/what-is-video-streaming/
In very simple terms, streaming breaks a large file or live stream into chunks, and sends those chunks one after another to a client (e.g. browser). The client can generally request a start point for content which is not a live stream. In the background this generally works by the client sending requests for each individual chunk (rather than just one request with multiple responses).
The advantage of the multiple request approach is that you know the client is actually still interested (e.g. the user has not browser to another page etc) and for video and audio etc the client can dynamically request different bandwidth files depending on the current network connection - see: http://en.wikipedia.org/wiki/Adaptive_bitrate_streaming.
Twitter do have a streaming page also, but you have probably already seen this:
https://dev.twitter.com/streaming/overview
Related
I'm trying to understand the section on Mobile Apps on https://developers.google.com/tag-platform/tag-manager/server-side/send-data . I'm at a loss when it comes to following these instructions for mobile. I'm trying to capture data that I can send to my own server (potentially as well as to GA4) so in the server side container I have a client which sends to my API.
The main sentence I need help with is "Set the value of Image URL to a pixel image on the server container at the path in the Measurement Protocol client that you specified earlier".
Can anyone point me to a dummy's type guide about how to create that pixel image on the server container?
Also, "Measurement Protocol client that you specified earlier" doesn't seem to correspond to anything in the previous instructions, and the link to Measurement Protocol implies that GA4 should be used instead. How would this work with the scenario I'm setting up?
Has anyone actually got sGTM working with mobile apps?
Mobile apps are irrelevant here, really. As long as you're capable of generating a generic http request, your platform doesn't matter. You can send hits from an excel sheet via VBA.
Yes, clients in sgtm are confusingly closely tailored to GA. Well, it doesn't really matter. You will still have a custom request tag type, so just go with MP.
Now the path thing. The requests coming to the sGTM endpoint are supposed to have different paths, based of which sGTM will make a decision about which client to use. Here:
Good, now to the image url and pixel image talk. That's just sGTM making it easier to use for marketeers who have a vague idea of a network request, but they know how pixel images work. In case you don't know, by loading a pixel from a third party server on our first-party page, we force the clients to send a request to that server. In the url of the pixel, marketeers typically indicate a lot of information that they want to pass to the server via query params. Basically, it's just a somewhat user-friendly abstraction over http requests. Much easier to deploy, too.
Finally, this is how your tag should look like, approximately:
I recommend reading this article from Simo since it has the exact technical incline that would be more useful to developers.
My requirement is to encode data and send it across network via HTTP, but I am stuck trying to choose the best encoding technique. Which of the three above is best? Is there anything better?
The criteria for "best" should be small size and fast encoding/decoding.
yEnc has less overhead but has other problems mentioned here http://en.wikipedia.org/wiki/YEnc#Criticisms.
What is "best" depends on the criteria you might have and how you plan to send data over the network. Are you using a web service, email or other means. You need to provide more details.
Edit:
Since you are uploading data via HTTP, you do not need to use any of Base64, yEnc or Uuencode. You just use the standard http file upload built in facility in both browser and web server. See this question as a reference:
How does HTTP file upload work?
Also this reference:
http://www.hanselman.com/blog/ABackToBasicsCaseStudyImplementingHTTPFileUploadWithASPNETMVCIncludingTestsAndMocks.aspx
I do understand the basic idea of it. But most of the implementations I've seen have done nothing but confused me - I find myself incapable of fully understanding the concept of Comet and long polling... Simply put, I ask for simple explanation of these ideas. I am especially interested in an explanation of the hidden iframe polling technique. What gets executed, what gets requested etc.
Also, what are the advantages of it over the classic ajax approach? (besides the reduced traffic and more real-time feeling).
Thanks.
The technique is very nicely explained in the following article. The core idea resides on the chunked transfer encoding HTTP technique. A hidden iframe is included in the page which points to a server side script which uses chunked encoding. In chunked encoding the response is not sent entirely in one go and the stream closed. The server doesn't say in advance how much data is going to send so the browser keeps the channel open. Then when the server wants to push some data to the client it simply sends a chunk of response which represents a javascript function. The browser receives and executes this function on the client. This way the server can successfully PUSH information when some events occur like for example some data changes on the server, ...
Also, what are the real advantages of it over the classic ajax
approach? (besides the reduced traffic and more real-time feeling).
Aren't those advantages sufficient? Reduced traffic means more responsive application. Did you know that large sites like Google and Amazon conducted studies and explicitly throttled down their servers in order to increase the response time with a couple of milliseconds. I can't remember the exact but they were flagrant: they lost like 70% of their customers after doing that. Remember: the most important feature of a web application (and not only by the way) is its responsiveness.
So it's basically PULL (Ajax) vs PUSH (Comet). PUSH techniques scale better when the number of clients starts to increase.
It is quite easy to update the interface by sending jQuery ajax request and updating with new content. But I need something more specific.
I want to send the response to client without their having requested it and update the content when they have found something new on the server. No need to send an ajax request every time. When the server has new data it sends a response to every client.
Is there any way to do this using HTTP or some specific functionality inside the browser?
Websockets, Comet, HTTP long polling.
It has name server push (you can also find it under name Comet technology). Do search using these keywords and you will find bunch examples, tools and so on. No special protocol is required for that.
Aaah! You are trying to break the principles of the web :) You see if the web was pure MVC (model-view-controller) the 'server' could actually send messages to the client(s) and ask them to update. The issue is that the server could be load balanced and the same request could be sent to different servers. Now if you were to send a message back to the client you'll have to know who all are connected to the server. Let's say the site is quite popular and you have about 100,000 people connecting to it every day. You'll actually have to store the IPs of each of them to know where on the internet they are located and to be able to "push" them a message.
Caveats:
What if they are no longer browsing your website? You see currently there is no way to log out automatically if you close your browser. The server needs to check after a fixed timeout if you have logged out (or you send a new nonce with every response to prevent the server from doing that check)
What about a system restart/crash etc? You'd lose all the IPs that you were keeping track of and you are back to square one - you have people connected to you but until you receive new requests you can't really "send" them data when they may be expecting it as per your model.
Let's take an example of facebook's news feeds or "Most recent" link close to the top right - sometimes while you are browsing your wall you see the number next to most recent has gone up or a new 'feed' has come to the top of your wall post! It's the client sending periodic requests to the server to find out what was updated rather than the other way round
You see, it keeps it simple and restful. You may feel it's inefficient for the client to "poll" the server to pull the data and you'd prefer push, but the design of the server gets simplified :)
I suggest ajax-pulling is the best way to go - you are distributing computation to the client and keeping it simple (KIS principle :)
Of course you can get around it, the question is, is it worth it?
Hope this helps :)
RFC 6202 might be a good read.
Can anyone tell me why the Range, header is restricted in the Flash player?
I want to be able to pause and resume downloads in my flex application, but I get a RTE when trying to set the Range header.
Error #2096: The HTTP request header Range cannot be set via ActionScript.
I imagine there isn't going to be a work around client side, but expect there is a way you can get a server to change the name for the range header to something else...
Would like to know Adobe's reason for this though, hopefully it's not just to sell more copies of FMS :p
I just discovered exactly the same issue with the Range header while attempting to add ranged GET requests to our REST layer in Flex. Range is on the "blacklist" and the Flash Player simply won't send it.
Flash/Flex headers ate my brain a year or so back (verveguy.blogspot.com) but this is the last straw.
The solution I am now going to finally embrace is to use the open source as3httpclientlib and just abandon the Flash HTTP stack. We've used it successfully for some minor parts of our app (specifically, for talking to the JIRA API) so it's time to beat it into submission for all HTTP traffic.
For your specific problem, you could certainly switch to a custom header, say X-Range. This assumes you have control of the server side code and that you also have a crossdomain.xml policy file that allows headers. (Blacklisted headers are the first set to be culled. After that, the Flash player checks the crossdomain.xml advertised by the server you're talking to see whether it allows specific (or all other) headers)
Hope this helps
Here are a couple of Adobe Tech Notes that explain their reasoning:
Arbitrary headers are not sent from Flash Player to a remote domain
ActionScript error when an HTTP send action contains certain headers (Flash Player)