OpenSource tools/services for server monitoring - tcp

I'm looking for an OpenSource server monitoring tool that allows the following:
Monitor a variety of services on TCP, UDP.
Supports custom port numbers.
Supports some basic response checking (beyond initial connection checking).
Basic dashboard overview of health status.
Alert notifications via e-mail.
I could roll my own, but want to make sure I don't reinvent the wheel if there's a complete suite out there that does what I want (and maybe more!).
EDIT: Preferably software that runs on Windows.

I recommend the "mon" package (http://www.kernel.org/software/mon). It is pre-packages in most Linux distributions, and fairly customizable.

Nagios is the way to go. Very powerful and customizable. Has dashboards, history, alerts, graphs, and much more.
I you want something a bit fancier, there is Groundwork which is built on nagios and various other bits and pieces, but I didn't really like that after playing with it for a little bit. It also costs for anything but the community edition.

Related

Using SNMP to obtain server information

I need to obtain information about my servers running on Linux Redhat, Windows Server 2003 and Windows Server 2008. I have found the SNMP protocol to be useful for my purpose, but I am not clear how to implement it in my application. I don't even know how to get started.
Also, is it possible to monitor the applications/programs running on the servers? It is my first time working with SNMP, so I am finding it extremely difficult to understand how it has to be implemented. I have understood the architecture of SNMP(SNMPManager, SNMPAgent, Trap messages, MIB), but finding it difficult to implement them.
Please guide me in as detailed way as possible and mention links to any tutorials.
SNMP is not so simple as should be... if you want to provide control to your system/resources the best choice is to use an external native tool like:
PRTG:You can use free evaluation sensor
CACTI: open-source and web-based tool
SolarWinds:expensive but powerful choice
After that you can monitor your system using WMI protocol (for Win OS), direct agent installed on it or script.

Looking for a good method to transfer critical real time data over internet

I am searching for a good method to transfer data over internet, and I work in C++/windows environment. The data is binary, a compressed blob of an extracted image. Input and requirements are as follows:
6kB/packet # 10 packets/sec (60kBytes per second)
Reliable data transfer
I am new to network programming and so far I could figure out that one of the following methods will be suitable.
Sockets
MSMQ (MS Message Queuing)
The Client runs on a browser (Shows realtime images on browser). While server runs native C++ code. Please let me know if there are any other methods for achieving the same? Which one should I go for and why?
If the server determines the pace at which images are sent, which is what it looks like, a server push style solution would make sense. What most browsers (and even non-browsers) are settling for these days are WebSockets.
The main advantage WebSockets have over most proprietary protocols, apart from becoming a widely adopted standard, is that they run on top of HTTP and can thus permeate (most) proxies and firewalls etc.
On the server side, you could potentially integrate node.js, which allows you to easily implement WebSockets, and comes with a lot of other libraries. It's written in C++, and extensible via C++ and JavaScript, which node.js hosts a VM for. node.js's main feature is being asynchronous at every level, making that style of programming the default.
But of course there are other ways to implement WebSockets on the server side, maybe node.js is more than you need. I have implemented a C++ extension for node.js on Windows and use socket.io to do WebSockets and non-WebSocket transports for older browsers, and that has worked out fine for me.
But that was textual data. In your binary data case, socket.io wouldn't do it, so you could check out other libraries that do binary over WebSockets.
Is there any specific reason why you cannot run a server on your windows machine? 60kb/seconds, looks like some kind of an embedded device?
Based on our description, you ned to show image information, in realtime on a browser. You can possibly use HTTP. but its stateless, meaning once the information is transferred, you lose the connection. You client needs to poll the C++/Windows machine. If you are prety confident the information generated is periodic, you can use this approach. This requires a server, so only if a yes to my first question
A chat protocol. Something like a Jabber client running on your client, and a Jabber server on your C++/Windows machine. Chat protocols allow almost realtime
While it may seem to make sense, I wouldn't use MSMQ in this scenario. You may not run into a problem now, but MSMQ messages are limited in size and you may eventually hit a wall because of this.
I would use TCP for this application, TCP is built with reliability in mind and you can simply feed data through a socket. You may have to figure out a very simple protocol yourself but it should be the best choice.
Unless you are using an embedded device that understands MSMQ out of the box, your best bet to use MSMQ would be to use a proxy and you are then still forced to play with TCP and possibly HTTP.
I do home automation that includes security cameras on my personal time and I use the .net micro framework and even if it did have MSMQ capabilities I still wouldn't use it.
I recommend that you look into MJPEG (Motion JPEG) which sounds exactly like what you would like to do.
http://www.codeproject.com/Articles/371955/Motion-JPEG-Streaming-Server

UDP or HTTP over TCP/IP for a simple multiplayer game

I am building a multiplayer game where players are simultaneously asked a question and they have to pick an answer. They have a few seconds to answer and they are asked the next question when the time is out or everyone has answered.
I am planning on using Java (or nodeJS if I feel like learning a new framework) on the server side and JS (jQuery and jQuery Mobile) on the client side. I might build native apps later too so I want to keep the option opened.
My question is: what server communication protocol would you suggest?
I was thinking of regular TCP (HTTP with Ajax) calls since latency is not really the issue but I am ready to review this choice if you suggest otherwise.
Thanks a lot for your help
When using UDP the burden to implement retry and congestion control will fall on you. Therefore, I recommend you stick with the reliable and easy TCP until there is a specific reason to switch. If latency turns out not to be acceptable, which I doubt in your case, you can switch to UDP.
Another thing to remember, if you are creating the game on mobile device and you don't want to lose the players connected over 3G, you will need to make sure you are using standard ports to connect to the server side, otherwise operators will disconnect you.
HTTP Streaming and WebSockets over TCP/IP is the best way to go. Lightstreamer, which leverages HTTP and WebSockets, uses TCP but tries to overcome some of its limits with some smart algorithms for latency reduction (which seems to be crucial for the game you described) and bandwidth optimization. Thus, it can be used with great benefits for any multiplayer games of any complexity (including MMORPGs, for example). At a first glance it seems it can fulfill your requirements, and it could be a good solution if you are also planning to develop native apps in a second step.
Lightstreamer is based on a publish/subscribe model, in which every item is defined by a set of fields whose values can change over time (in order of milliseconds). These could include "general items", subscribed by any users (e.g the game countdown; who is the first responder) and delivered in real-time. This way, the data delivery mechanism underlying your game logic seems quite easy to implement with Lightstreamer.
Have a look at this recent article (Optimizing Multiplayer 3D Game Synchronization Over the Web). You can walk through an online demo of a simple multiplayer 3D world in which Lightstreamer has been integrated for real-time synchronization.
The demo allows you to tweak each parameter of the scenario and simulate any flavor of data delivery, while checking the actual bandwidth consumption. Of course, full source code is available for free on GitHub. You could give it a try. Let me know if you need some help :)
[full disclosure: I work for Lightstreamer]

I want to build a decentralized, reddit-like system using P2P. What existing p2p library should I base it on?

I want to build a decentralized, reddit-like system using P2P. Basically, I want to retain the basic capabilities of reddit, but make it decentralized, to make it more robust and immune to censorship. This will also allow people to develop different clients to match the way they want to browse it.
Could you recommend good p2p libraries to base my work on? They should be open-source, cross-platform, robust and easy to use. I don't care much about the language, I can adapt.
Disclaimer: warning, self-promotion here !!!
Have you considered JXTA's latest release? It is probably sufficient for what you want to do. Else, we are working on a new P2P framework called Chaupal, but it is not operational yet.
EDIT
There is also what I call the quick-and-dirty UDP solution (which is not so dirty after all, I should call it minimal).
Just implement one server with a public address and start listening for UPD.
Peers located behind NATs contact the server which can read how their private IP address has been translated into a public IP address from the received datagrams.
You send that information back to the peer who can forward it to other peers. The server can also help exchanging this information between peers.
Then peers can communicate directly (one-to-one) by sending datagrams to these translated addresses.
Simple, easy to implement, but does not cover for lost datagrams, replays, out-of-order etc... (i.e., the typical stuff that TCP solves for you at the IP stack level).
I haven't had a chance to use it, but Telehash seems to have been made for this kind of application. Peer2Peer apps have a particular challenge dealing with the restrictions of firewalls... since Telehash is based on UDP, it's well suited for hole-punching through firewalls.
EDIT for static_rtti's comment:
If code velocity is a requirement libjingle has a lot of effort going into it, but is primarily geared towards XMPP. You can port off parts of the ICE code and at least get hole-punching. See the libjingle architecture overview for details about their implementation.
Check out CouchDB. It's a decentralized web app platform that uses an HTTP API. People have used it to create "CouchApps" which are decentralized CouchDB-based applications that can spread in a viral nature to other CouchDB servers. All you need to know to write CouchApps is Javascript and learn the CouchDB API. You can read this free online book to learn more: http://guide.couchdb.org
The secret sauce to CouchDB is a Master-to-Master replication protocol that lets information spread like a virus. When I attended the first CouchConf, they demonstrated how efficient this is by throwing a "Couch Party" (which is where you have a room full of people replicating to the person next to them simulating an ad hoc network).
Also, all the code that makes a CouchApp work is public by default in special entities known as Design Documents.
P.S. I've been thinking of doing a similar project, but I don't have a lot of time to devote to it at the moment. GOD SPEED MY BOY!

How do modern implementations of Comet/Reverse AJAX work? Any stable C# WCF or ASP.NET implementations?

What is the correct way (or best) way to implement Comet, HTTP Push, or Reverse AJAX?
What .NET implementations would you recommend?
I have hear about, WebSync and PokeIn, both are paid implementations, I have used PokeIn and its pretty straight forward. If you are looking forward to code your own COMET implementation, I just can say that its a complex task, because you need to modify the natural behaviour if IIS. Its a hacky way to get around the limitations of the HTTP protocol and you need to know really well what you doing so don't end up breaking things around =).
It's also known as long-lived
requests. This is also by far the most
complex method to implement.
Basically, a request is made by the
client, and the server very slowly
responds, which causes the connection
to be maintained. Periodically, when
the server has something to push,
it'll "burst" send the information, so
to speak. This approach gives you
real-time push, which is great. But,
it has a serious down-side: holding
connections open like that isn't how
the underlying protocols are meant to
work, and most servers aren't terribly
happy about it. If your traffic gets
too great, you'll chew up threads on
the server and wind up bringing your
site down.
ref: http://www.coderanch.com/t/121668/HTML-JavaScript/does-Reverse-Ajax-Works
JOBG is correct re: the complexities; it's probably not a task you want to undertake lightly. I'm one of the authors of WebSync, and I can attest that it's a difficult task.
There are a ton of examples in the download, and the community edition is free.
Microsoft is developing HTTP push in SignalR
https://github.com/SignalR/SignalR

Resources