Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 4 months ago.
Improve this question
What is currently the most seeded torrent in the world?
Is it even possible to accurately determine the most connected torrent ?
I suppose if you had access to every ISP's network and could filter out the protocol bittorrent uses. Maybe that's a little unrealistic and too low-level.
Every tracker could be queried, counting the seeds for each torrent, eventually resulting in the highest count unique to a specific torrent.
Any tracker that has a significant amount of users that would sway the count weighting would be included in the query.
Oddly enough a program BitChe allows for multiple tracker / search engines for a specific textual inquiry. However there is no "popular torrents" section.
"Popular Torrents this Week", like at (https://1337x.to/home/) is not the same or a complete metric.
Every tracker / search engine with a significant volume of monthly traffic would need to be included, and the count metric would have to be a difference accumulated monthly. Accuracy would improve over time.
A good list is at (https://blokt.com/guides/best-torrent-sites) showing the traffic volume (in millions) of multiple popular torrent trackers / search engines.
Has anyone developed this already by any chance?
What is currently the most seeded torrent in the world? Should we know this? A definitive list of the most popular torrents to date.
Is it even possible to accurately determine the most connected torrent ?
Accurately? No. Bittorrent is a protocol, it can be deployed in private networks. E.g. if large cloud providers happened to use bittorrent internally and we don't know about it then they might happen to be operating the largest swarm in the world and any outside observer would miss that.
If we limit ourselves to torrents traversing the public internet then perhaps if you had access to several internet exchanges and could passively sniff traffic...
Since most of bittorrent is unencrypted it would be possible to gather statistics that way. There is no other way to globally observe torrents marked as private since they only communicate with their specific trackers which in turn may not publish their statistics to unauthenticated users.
If you restrict yourself to the subset of non-private torrents on the open internet seeded through clients with DHT peer discovery enabled then it gets easier. You can first build a database of torrents via active infohash sampling and passively observing get_peers lookups coming your way. Since popular torrents should create a lot of traffic you're likely to learn about those infohashes fairly soon this way. Then perform DHT scrapes to get the seed count for each torrent and perhaps connect to the Top-100 torrents gathered that way and perform peer exchanges to validate the DHT scrapes.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 7 years ago.
Improve this question
Unsure if this is the correct place for this kind of question (Server Fault is for administrators) but I'm looking for examples of (networking) protocols and standards that leverage the huge IPv6 address space towards some specific benefits.
Reason is I'm writing a paper about something that does this and I'm searching for stuff to compare it to, but I'm coming up empty even though I'm sure some have to do so.
Probably not completely on topic here, but I'm going to answer it anyway because it might help software developers to learn what possibilities there are when developing for IPv6.
Because most protocols and applications still want to be compatible with IPv4 there aren't that many protocols that use this wonderful feature of IPv6. Most of the ones that do are IPv6-only by nature.
The first one is of course IPv6 Stateless Address Autoconfiguration often abbreviated to SLAAC. Having so many addresses available makes self-assignment of addresses so much easier.
Building on that is SEcure Neighbor Discovery (SEND) with Cryptographically Generated Addresses (CGA) which uses public key cryptography and derives the last 64 bits of the IPv6 address from the fingerprint of that public key. Because the node can create a signature with the private key that matches the public key it can prove that it is the legitimate holder of that IPv6 address. Unfortunately there are no implementations for the most common operating systems so this feature is largely unused.
And another one is the IPv6 Battleships game. It uses IPv6 addresses to encode the coordinates in the game and by sending a ping the opponent can determine of there is a ship at a certain location (address) or not. The game was written in a competition on World IPv6 Day at the IPv6 event in The Netherlands.
It has significant advantages for large Cloud deployments. For instance, Faceebook has reported 90% of its internal traffic is IPv6 (https://t.co/PPHBkUPTdt) because this way is similar to private IPv4 but serves better to v6 native customers (in the same report they claim v6 mobiles surf 30%-40% quicker than v4 ones). Also Openstack has finally added almost-full v6 support for Juno version.
Additionally, the IoT field (when it is really IoT, I mean IP devices) is an excellent field of application. Normally, devices will use either IPv6 (when powerful enough) or 6LowPAN (an reduced IPv6 for sensor networks and supported well in OS like TinyOS, ContikiOS, mbedOS, etc).
It might be also useful for WebRTC-like applications as long as IPv6 peers would normally connect better (firewalls vs NATs seem to behave or be more easily configured).
However, with the massive deployment that has started in 2015 (in USA 17% of all traffic to Google is v6, Germany traffic to Google is 14%, all Internet traffic to Google is 7% IPv6 today) any Internet product of service has to migrate or will lose competitiveness soon.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
If Chris and Pat want to exchange a text message, they send and receive via their network providers, which charge them for a connection.
If Chris and Pat are both located in New York City, and there are enough wireless devices between Chris and Pat all close enough to each other to form a continuous chain, is it possible for all those devices to be programmed to cooperatively forward packets amongst each other, bypassing the need for network providers?
It would seem the "address" of each device would have to include current geographic coordinates, and devices would have to report their movements frequently enough so routing attempts could still find them, but the speed and capacity of devices nowadays could handle that, right?
Would such a network be viable? Does it already exist or has it been attempted? Is there some kind of inherent programming problem that is difficult to overcome?
There are a few interesting things here:
Reachability. At least you need to use a technology that can do ad-hoc and peer-to-peer networking. Of those technologies only bluetooth, NFC and WiFi are more or less often implemented. Of those again only wifi currently may have the strength to connect to devices in other houses or to the street, but even there typical ranges are 30-60m (and that's for APs, it might be lower for UEs).
Mobility. ANY short-range wireless communication protocol has difficulties with fast-moving devices. It's simple math, suppose your coverage is 50m in diameter, if you move at about 20km/h or 5.5m/s, you have less than 10s to actually detect, connect and send data while passing this link. Oh, but then we did not consider receiving traffic, you actually have to let all devices know that for the next 10s you want to receive data now via this access network. To give an example, wifi connectivity times with decent authentication (which you need for something like this) alone takes a few seconds. 10s might be doable, but as soon we talk about cars, trains, ... it's becoming almost impossible with current technology. But then again, if you can't connect to those, what are the odds you will cross some huge boulevards with your limited reachability?
Hop to hop delays. You need a lot of those. We can fairly assume that you need at least a hop each 20-30m, let's average at 40 hops/km. So to send a packet over lets say 5km you'd need 200 hops. Each hop needs to take in a packet (L2 processing), route it (L3 processing) and send it out again (L2 processing). While mobile devices are relatively powerful these days I wouldn't assume they can handle that in the microseconds routers do. Next to that in a wireless network you have to wait for a transmission slot, which can actually take in the order of ms (each hop!). So all in all, odds are huge this would be a terribly slow network.
Loss. Well, this depends a bit on the wireless protocol, either it has its own reliable delivery protocol (which will make the previous point worse) or it doesn't. In that last case, suppose your wireless link has about .1% loss, or 99.9% no-loss, this would actually end up with an 18.1% loss rate for the 200 hops considered previously ( (1-0.999**200)*100) This is nearly impossible to work with in day-to-day communications.
Routing. lets say you need a few millions of devices and thus routes. For traditional routing this usually takes some very heavy multicore routers with loads of processing power. Let's just say mobile devices (today) can't cut that yet. A purely geographically based routing mechanism might work, but I can't personally think of any (even theoretical) system for this that works today. You still have to distribute those routes, deal with (VERY) frequent route updates, avoid routing loops, and so on. So even with that I'd guess you'd hit the same scale issues as with for example OSPF. But all-in-all I think this is something that mobile devices will be able to handle somewhere in the not-so-far future, we're just talking about computing capacity here.
There are some other points why such a network is very hard today, but these are the major ones I know of. Is it impossible? No, of course not, but I just wanted to show why I think it is almost impossible with the current technologies and would require some very significant improvements, not just building the network.
If everyone has a device with sufficient receive/process/send capabilities, then backbones (ISP's) aren't really necessary. Start at mesh networking to find the huge web of implementations, devices, projects, etc., that have already been in development. The early arpanet was essentially true peer-to-peer, but the number of net nodes grew faster than the nodes' individual capabilities, hence the growth of backbones and those damn fees everyone's paying to phone and cable companies.
Eventually someone will realize there are a million teenagers in NYC that would be happy to text and email each other for free. They'll create a 99-cent download to let everyone turn their phones and laptops and discarded devices into routers and repeaters, and it'll go viral.
Someday household rooftop repeaters might become as common as TV antennas used to be.
Please check: Wireless sensor network
A wireless sensor network (WSN) of spatially distributed autonomous sensors to monitor physical or environmental conditions, such as temperature, sound, pressure, etc. and to cooperatively pass their data through the network to a main location
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
I am assigned on project to live broadcast an event on the internet which may have 50,000 user s. This will be a broadcast so all users will see same live video. My query is what would be my bandwidth requirement in terms of no of users viewing that stream.
I am little confused & the reason is does every users need a separate stream of bandwidth in broadcast ?
If i would install a server with streaming capability having Bandwidth of 50Mbps would it be enough considering it is a broadcast.
Do i necessarily need to have Class C IP to run streaming server on rtmp protocol.
Is is possible to achieve this through RTSP. How do services like Google Hangout air work? What protocol is used in Hangout and Youtube when live broadcasts are done.
Kindly suggest me solution especially mention if you have such practical experiance.
Thanks in Advance
You need a ton of bandwidth and resources.
To calculate how much bandwidth you need, you will need to know your average bitrate for the video. Let's just say that your live video's bitrate is 1 megabit (ignoring overhead, retransmissions, sequences that require more bandwidth, etc.). Your 50Mbps only covers 50 users. That's 0.1% of what you require. You need 1,000 of those connections, to barely handle the load.
If you actually have a live event that 50,000 people will see, you no doubt have sponsors and should be able to afford a proper CDN. This isn't something you host yourself. You pay for a CDN so that capacity is available as you need it, and servers are close to your audience.
The best thing to do would be to get a YouTube account with live streaming, and let YouTube pay for the bandwidth.
Now, the protocol you use has nothing to do with what size of IP block you have. Those are unrelated, separate issues.
RTMP, RTSP, HTTP progressive, etc... if you use a CDN, you get to use all of them with little effort. You pick a streaming protocol based on device compatibility and capability.
Google Hangout works using WebRTC, which is primarily peer-to-peer. When you stream them to YouTube, there is a massive CDN that handles distribution in multiple codecs, multiple protocols, and multiple points of presence.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
When I review the hardware requirements of many database backed enterprise solutions, I find requirements for the application server (OS, processor, RAM, disk space, etc), for the database server (versions, RAM, etc) and requirements for the client.
Here is an example of Oracle Fusion Middleware.
I cannot find requirements on network speed or architecture (switch speed, SAN IOPS,RIOPS, etc). I do not want a bad user experience from my application but caused by network latency in the clients environment.
When sending clients required hardware requirements specifications, how do you note the requirements in these areas? What are the relevant measures of network performance? (Or is it simply requiring IOPS=x )
Generally, there is more than one level of detail for requirements. You'd typically differentiate the levels of detail into a range from 0 (rough mission statement) to 4 (technical details), for example.
So if you specify that your SAN shall be operating with at least a bandwidth capacity of x, that would be a high number on that scale. Make sure to break down your main ideas (The system shall be responsive, in order to prevent clients from becoming impatient and leaving for competitors....) into more measurable aims (as the one above).
Stephen Withall has written down good examples in his book "Software Requirement Patterns". See chapter 9, page 191 ff., it is not that expensive.
He breaks it down into recommendations on, and I quote, Response Time, Throughput, Dynamic Capacity, Static Capacity and Availability.
Of course, that's software! Because basically, you'd probably be well advised to begin with defining what the whole system asserts under specified circumstances: When do we start to measure? (e.g. when the client request comes in at the network gateway); what average network delay do we assume that is beyond our influence? from how many different clients do we measure and from how many different autonomous systems do these make contact? Exactly what kind of task(s) do they execute and for which kind of resource will that be exceptionally demanding? When do we stop to measure? Do we really make a complete system test with all hardware involved? Which kinds of network monitoring will we provide at runtime? etc.
That should help you more than if you just assign a value to a unit like transfer rate/ IOPS which might not even solve your problem. If you find the network hardware to perform below your expectations later, it's rather easy to exchange. Especially if you give your hosting to an external partner. The software, however, is not easy to exchange.
Be sure to differentiate between what is a requirement or constraint that you have to meet, and what is actually a part of the technical solution you offer. There might be more solutions. Speed is a requirement (a vague one, though). Architecture for hardware is a solution.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
And what would you recommend for an ASP Net web application, with a not so large SQL server database (around 10Gb)?
I was just wondering, is that a good idea to have an Amazon EC2 instance configured ready to host your app in an emergency?
In this scenario, what would be the best approach to keep the database updated (log shipping? manual backup restore?) and the easiest and fastest way to change the dns settings?
Edit: the acceptable downtime would be something between 4 to 6 hours, thats why i considered using the Amazon ec2 option for its lower cost if compared to renting a secondary server.
Update - Just saw your comment. Amazon EC2 with log shipping is definitely the way to go. Don't use mirroring because that normally assumes the other standby database is available. Changing your DNS should not take more than 1/2 hour if you set your TTL to that. That would give you time to integrate any logs that are pending. Might turn on the server once a week or so just to integrate logs that are pending (or less to avoid racking up hourly costs.)
Your primary hosting location should have redundancy at all levels:
Multiple internet connections,
Multiple firewalls set to failover,
Multiple clustered web servers,
Multiple clustered database servers,
If you store files, use a SAN or Amazon S3,
Every server should have some form of RAID depending on the server's purpose,
Every server can have multiple PSUs connected to separate power sources/breakers,
External and internal server monitoring software,
Power generator that automatically turns on when the power goes out, and a backup generator for good measure.
That'll keep you running at your primary location in the event of most failure scenarios.
Then have a single server set up at a remote location that is kept updated using log shipping and include it in your deployment script (after your normal production servers are updated...) A colocated server on the other side of the country does nicely for these purposes. To minimize downtime of having to switch to the secondary location keep your TTL on the DNS records as low as you are comfortable.
Of course, so much hardware is going to be steep so you'll need to determine what is worth being down for 1 second, 1 minute, 10 minutes, etc. and adjust accordingly.
It all depends on what your downtime requirements are. If you've got to be back up in seconds in order to not lose your multi-billion dollar business, then you'll do things a lot differently to if you've got a site that makes you maybe $1000/month and whose revenue won't be noticeably affected if it's down for a day.
I know that's not a particularly helpful answer, but this is a big area, with a lot of variables, and without more information it's almost impossible to recommend something that's actually going to work for your situation (since we don't really know what your situation is).
The starting point for a rock solid DR Strategy is to first work out what the true cost is to the business of your server/platform downtime.
The following article will get you started along the right lines.
https://web.archive.org/web/1/http://articles.techrepublic%2ecom%2ecom/5100-10878_11-1038783.html
If you require further guidelines good old Google can provide plenty more reading.
A project of this nature requires you to collaborate with your key business decision makers and you will need to communicate to them what the associated costs of downtime are and what the business impact would be. You will likely need to collaborate with several business units in order to gather the required information. Collectively you then need to come to a decision as to what is considered acceptable downtime for your business. Only then can you devise a DR strategy to accommodate these requirements.
You will also find that conducting this exercise may highlight shortcomings in your platforms current configuration with regard to high availability and this may also need to be reviewed as an aside project.
The key point to take away from all of this is that the decision as to what is an acceptable period of downtime is not for the DBA alone to decide but rather to provide the information and expert knowledge necessary so that a realistic decision can be reached. Your task is to implement a strategy that can meet the business requirements.
Don’t forget to test your DR strategy by conducting a test scenario in order to validate your recovery times and to practice the process. Should the time come when you need to implement your DR strategy you will likely be under pressure, your phone will be ringing frequently and people will be hovering around you like mosquitoes. Having already honed and practiced your DR response, you can be confident in taking control of the situation and implementing the recovery will be a smooth process.
Good luck with your project.
I haven't worked with different third party tools but I've experienced cloudendure, and as for the replica you get I can tell it is a really high end product. Replication is done in really tiny time intervals which makes your replica very reliable, but I can see you're not in need of having your site back up within seconds so maybe asking for a price offer or getting away with a different vendor might help.