As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I understand the basics of networking such as Lan and stuff. I know what many of the protocols are and how to build a client/server socket program in C. But what I really want is a very good understanding of how networks actually work. Not only from a programming aspect but also from a application aspect. I am looking for some material(preferably a book) which will give me a very good foundation to go off of. I am in the middle of wanting to be a programmer or a UNIX admin, so I really should learn and know how to apply networking fundamentals.
Does any such a concise resource exist? Would it be better going the more academic route by buying a networking book(such as those from Tanenbaum or Kurose), or is it better to go the It route possibly looking into network admin text or certification books.
Thank you all so much.
Here is the way I would recommend:
Learn how Internet Evolved, this would give you the reason why they needed it
Learn the different protocols - HTTP, telnet, ssh esp the secure ones SFTP, HTTPS etc
Learn what are sockets and types of sockets
Learn how you can do socket programming. I suggest you to use Python sockets to do the programming
Learn the TCP/IP network stack. That would be beneficial
Learn how routing works, this is important for learning
Try to have a sound knowledge of topics like DNS, it is very important
Get VirtualBox and install various OS and try to internetwork them. Play around with the networking stack of the OS.
The late Richard Stevens' book is a masterpiece -- much more practical and immediately applicable than Tanenbaum's (haven't studied Kurose). Btw, by the same author, I just as warmly recommend the books in the "TCP/IP Illustrated" series, and Advanced Programming in the Unix Environment -- few books are more crucial to the wanna-be "programmer or Unix admin", save perhaps ESR's!
Its kind of unclear to me what exactly you're looking for, so I'm just going to throw this out there:
Start networking your own stuff together. Create a LAN. Go figure out how to create and manage a Linux firewall instead of a consumer one. Install Active Directory just for grins. Run your own DHCP and DNS servers on that Active Directory server.
Once you get that far, if you're still interested, start thinking about how you would plan your LAN if you had 500 computers. Learn about Virtual LANs (VLANs).
I think networking in particular is a great place to start tinkering because A) no one gets hurt, B) its mostly free.
Whoa .. networking is a seriously big field. To truly understand everything will require a PHD, or several PHD's.
Here are some of the aspects I think you need to learn.
1) You need to learn the history of networking. Many of the policies built into protocols were made due to limitations of the time. Learn the history of protocols to learn the "Why" of how it works.
2) Programming is an excellent source of knowledge on how a network works on the lowest level. Learn to write some socket code in C. BSD sockets is a good place to start. You can find alot of references for BSD sockets on the Internet.
3) *nix commands offer a wealth of knowledge on configuring and managing networks. Good network admins know a lot of tricks on how to build complicated networking operations using just the most basic network tools. The GNU networking tools is a good place to start.
4) If your up to it, there are several certificates like MCSE and CCNA which have modules on networking. These papers can be useful to gain knowledge on a particular type of network. I learnt alot about windows NT domain models from sitting for the MCSE paper even though i never really played around with domains much.
There are more aspects. Ask yourself, which do you like more ?
a bit of personal experience.
I have worked as a software developer for 10 years. I am also the "unpaid" network guru in my office. Somehow , i have to wear more than 1 hat as a developer because networking is part of the software i work on.
For fundamentals, you may want to get the W. Richard Stevens Classic, TCP/IP Illustrated, and possibly his other books as well. There will not be any more of them, either.
It sounds like the kind of understanding you're looking for is the kind that can only really be reached through experience. Each and every person will have a different way of looking at things, depending on what makes sense to them -- explanations can help, but there's no substitute for learning by actually solving problems.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
How should an experienced .NET & SQL developer go about becoming a BizTalk expert for a project starting in 1 month? How should I spend my limited time to gain some practical skill & knowledge in BizTalk so I can "walk the talk"?
I am self employed, and would not be willing to spend more than USD300. I have the book "Professional BizTalk Server 2006" by Wrox, but have not found it to be a particularly good learning resource (very dry, needs more real world examples).
The BizTalk Virtual Labs in MSDN are a pretty good place to start with. Pluralsight also has several good BizTalk courses, and their online subscription isn't too expensive; would likely be a good option.
I agree with everything written this far. All solid info.
I have a few addons, coming from a fellow freelancer working with BizTalk since 2002:
Unit testing.
It's not easy to do, but check out BizUnit. A Codeplex based toolset written and maintained by Kevin Smith. One of the early BizTalk heroes :-) http://bizunit.codeplex.com/
Deployment / getting things into production
But also keep in mind that none of the day to day development stuff will prepare you for the part of the project where you have to deploy the app and make sure that it is "manageable" by operations. This can be quite complex, and is a topic in it's own right.
Check out Apress Pro BizTalk 2009, it's got a decent (IMO) chapter on this.
The entire development process around BizTalk.
The first two chapters of the same book will give you a good impression on what a BizTalk project is about. Where to use it, and where to not use it, how to organize projects, and name your stuff. Really a good collection of info that you would only get by reading 5-6 years of blogs back in time :-)
And one last thing. Depending on the roles on the project, you might be asked to optimize and tune BizTalk. And if they don't ask you. Make sure that you ask if others have done that, because you have to do it. BizTalk should always be tuned towards what it is supposed to do. Low latency vs high throughput, tuned according to hardware, correct setup and config of network around the SQL boxes, etc etc etc. This can be hairy stuff, and you should be careful not to jump into it before reading up on it all. But it's a subject we as freelancers are often expected to be able to deal with ... so thought I might bring it up.
Example ... BizTalk x64 processes on an x64 box runs really bad out of the box, actually worse than on the x86 processes. The 64 bit processes need to be tuned to really use all the MEM that are availble to them.
Anyways ... a bag of mixed tips and I hope you can use some of them! And good luck! It can be a tough start, but if used right, BizTalk can be a great product/toolset.
And remember .... if it is ugly, or hard, or both. You are doing it wrong. And don't be afraid to dive into .net code, and bolt it onto the BizTalk box. We all do it ... some just won't admit to it :-D
Start with the advice of tomasr.
Then, try and build something as real as possible. Biztalk is the kind of product where everything seems fine when you read the book and follow the examples, then you sit down to do something and you are thinking "what do I do now".
As per Thomas and Shiraz - set up an environment and get your hands dirty. If you haven't done so already, download and install BizTalk Server 2010 Developer Edition
But just to temper your expectation, IMHO expertise in BizTalk (or any other EAI / BPM / ESB product) can take years to accumulate.
It isn't clear whether you are developing for a client with an established BizTalk installation, or if this is the client's first BizTalk deployment. If so, one thing not to be underestimated is that the operational considerations of running a production BizTalk environment (performance, redundancy, reliability, auditing, tracking, monitoring with SCOM etc) are as complex as the development and testing - but understanding of this will be important to 'walk the talk'.
W.r.t. dev, start with some a simple EAI type mapping project, and then work your way through the SDK samples progress to some common messaging patterns (e.g. batching with aggregator), and then move into the BPM type orchestrations. You can probably leave BAM and the BRE for later.
Good luck!
+1 to tomasr for mentioning the virtual labs. Getting hands-on is definitely the way to go, as Shiraz Bhaiji also mentions. Hopefully you're not starting with BizTalk 2006, and can go with the latest: 2010. If that's the case, you can get the Developer Ed. of BizTalk 2010 for free now (see link from nonnb).
I'd also recommend Richard Seroter's book: 'SOA Patterns with BizTalk Server 2009' (available on Amazon.com). There are many ways to do the "wrong" things with BizTalk, and this book does an excellent job of walking through both the how and the why of building BizTalk solutions (with the code samples available from the publisher's site). And yes, it pretty much takes a whole book to go through it all. It's a good (more readable) companion to the Pro BizTalk 20xx series (which is generally better for very specific questions/tasks).
I would like to develop a Network Inventory application that works on any operating system.
Reports on every possible resource attacehd to a network.
Reports all pertinent details of hardware and software.
Thats (and i hate to use the phrase) my "End Game".
However I am running before i can crawl here.
I have no experience of this type of development, e.g. discovering a computers hardware and software settings.
I've spent almost two weeks googling and come up short! :-(.
So I am turning to you to ask these questions:-
My first step is to find an existing open source project i can incorporate into my own code that extracts the fine grained details i am after, e.g. EVERYTHING there is to know about the hardaware and software on a single machine.
Does this project exist? or do i have to develop that first?
Have i got to write all this in C?
I am guessing getting this information about a computer is going to be easier than for printers, scanners, routers etc... e.g. everything else you would find attached to a network.
Once i have access to a single computers details i then need to investigate how i can traverse an entire newtork of printers, scanners, routers, load balancers, switches, firewalls, workstations, servers, storeage devices, laptops, monitors, the list goes on and on
One problem i have is i dont have a 1000 machine newtork to play on!
Is there any such resource available on theinternet? (is that a silly question?)
Anywho, if you dont ask you wont find out!
One aspect iam really looking forward to finding out how to travers the entire network,
should i be using TCP/IP for this?
Whats a good site, blog, usergorup, book for TCP/IP development?
How do i go about getting through firewalls?
How many questions can i ask in one go? :-)
My previous question on this topic ended up with PYTHON being championed as the language/script to go with to develop this application in.
Having looked at a few PYTHON examples they all seemed to be related to WINDOWS networks
and interrogating Windows Management Instrumentation (WMI). I had the feeling you cant rely on whats in WMI, and even if you can that s no good for UNIX netwrks.
Surely there exist common code for extracting hardware and software details from a computer? Why cant i find it on the internet?
Pease help?
Theres no prizes though :-(
Thanks in advance
I would like to appologise if i have broken forum rules or not tried hard enough on my own before asking for assistance.
I just would like to start moving forward with this as its one of the best projects i have been involved with.
I am inspired by the many differnt number of challenges involved and that if i manage to produce a useful application at the end of it it would hopefully be extremely helpful to many people.
That sit
Thanks in advance
DD
as a software vendor of a discovery solution, I can just say: Respect, that you want to start a new one :-). Just in case you are interested in what it could look like: http://www.jdisc.com
Now to some of our experience:
Programming Language:
I wouldn't write it in C. Use Java or .NET. Those languages have great advantages when it comes to tracking down errors or problems. For instance, in Java (and I guess also in .NET), you can see the stack trace when something is failing. For some pieces of code (e.g. WMI access), you might need to use C++ or C (e.g. access to native APIs from Microsoft). Use a native interface or a COM bridge from Java. In .NET, it should even be easier to access the Windows APIs).
Devices:
well, network printers, router, and switches are actually easier to discover. They usually expose their information via SNMP. SNMP is pretty easy to use and pretty robust. Getting information from Windows (or even Unix) systems is a bit trickier. Protocols can be blocked, misconfigured, messed up... We had cases, where WMI was simply hanging when requesting data from a remote device.
Test Devices:
Since we are also a smaller company, we also do not have 1000 different devices to test with. But, there are some things that might help:
a) For SNMP devices use a SNMP simulator. We use MIMIC 9.0 from Gambit Solutions and we are pretty happy with it. You can import SNMP walks from network devices and simulate the device as if it would be in your network.
b) Secondly, use virtualization whenever possible. With VMware, you can install Windows, Linux, or even Solaris. We also use a project called GNS3 to emulate Cisco Routers, Firewalls or Juniper routers.
c)You can test the rest of the devices only, if you have a customer that helps you with testing and implementing new devices.
This are just some ideas to start with. But I have to tell you, that it is not trivial and it takes a lot of time....
Hope that you got some ideas to start with...
I don't know that it's open source, but we use Spiceworks (http://www.spiceworks.com) here as an IT management platform. You may get some use out of exploring that.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
What could be the optimum programming languages (Perl, PHP, Java, Python or something else) to implement server (multi-threaded?) using tcp/ip socket serving like 1000's of clients with the streaming data?
Using C/C++ with libevent, we were streaming 800mbps sustained to 30,000 active connections (two four core processors, 7 threads each running one event loop). Erlang is a reasonable choice too. It is far safer against programmer errors. But it cannot keep pace to event driven c/c++ ... been there and had to rewrite (hint, erlang is written in c).
Python with Twisted Framework
www.twistedmatrix.com
Java with XSocket or Apache Mina Frameworks (which Red5 Flash/video streaming media sever based on)
mina.apache.org
xsocket.sourceforge.net
They all are multithreaded , easy and very powerful.
Erlang of course :-) But then again, your requirements are not clear ;-)
It was designed from ground up to handle multi-threaded networking applications. It's origin comes from Ericsson: they use Erlang in (some of) their networking products.
This doesn't precisely answer this question, but it will help answer future questions. The problem of connecting thousands of clients to the same server is known as the c10k problem. There you will find lots of answers and helpful information about setting up that kind of server.
based on the sparse information given I would say either c or erlang
What language are you most familiar with? What kind problem set do you have? A lot depends on these questions. Most popular programming languages have good documentation for doing socket programmimng. It depends on tastes. I prefer the C programming language. I'm sure some people will also chime to offer Erlang as a good language to use. Again, it depends.
Apple already sells an optimum multi-threaded streaming media server.
http://www.apple.com/quicktime/streamingserver/
You might be able to buy it and save yourself a lot of work.
I can't tell from your question what you're trying to do, but buying a solution is usually optimal.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am currently looking at a distributed cache solution.
If money was not an issue, which would you recommend?
www.scaleoutsoftware.com
ncache
memcacheddotnet
MS Velocity
Out of your selection I've only ever attempted to use memcached, and even then it wasn't the C#/.NET libraries.
However memcached technology is fairly well proven, just look at the sites that use it:
...The system is used by several very large, well-known sites including YouTube, LiveJournal, Slashdot, Wikipedia, SourceForge, ShowClix, GameFAQs, Facebook, Digg, Twitter, Fotolog, BoardGameGeek, NYTimes.com, deviantART, Jamendo, Kayak, VxV, ThePirateBay and Netlog.
I don't really see a reason to look at the other solution's.
Good Luck,
Brian G.
One thing that people typically forget when evaluating solutions is dedicated support.
If you go with memcached then you'll get none, because you're using completely open source software that is not backed by any vendor. Yes, the core platform is well tested by virtue of age, but the C# client libraries are probably much less so. And yes, you'll probably get some help on forums and the like, but there is no guarantee responses will be fast, and no guarantee you'll get any responses at all.
I don't know what the support for NCache or the ScaleOut cache is like, but it's something that's worth finding out before choosing them. I've dealt with many companies for support over the last few years and the support is often outsourced to people who don't even work at the company (with no chance of getting to the people who do) and this means no chance of getting quality of timely support. On the other hand I've also dealt with companies who'll escalate serious issues to the right people, fix important issues very fast, and ship you a personal patch.
One of those companies is Microsoft, which is one of the reasons that we use their software as our platform. If you have a production issue, then you can rely on their support. So my inclination would be to go with Velocity largely on this basis.
Possible the most important thing though, whichever cache you choose, is to abstract it behind your own interface (e.g. ICache) which will allow you to evaluate a number of them without holding up the rest of the development process. This means that even if your initial decision turns out not to work for you, you can switch it without breaking much of the application.
(Note: I'm assuming here that all caches have sufficient features to support what you need from them, and that all caches have sufficient and broadly similar performance. This may not be a valid assumption, in which case you'll need to provide more detail in your question as to why it isn't).
You could also add Oracle Coherence to your list. It has both .NET and Java APIs.
From microsoft : App fabric
Commerical : NCache
Open source : RIAK
We tried a couple in the end we use the SQL session provider for asp.net/mvc yes there is the overhead of the connection to the DB but our DB server is very fast and the web farm has loads of capacity so not an issue.
Very interested in RIAK has .net client and used by Yahoo - can be scaled to many manu server
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 12 years ago.
Improve this question
Recently a friend of mine had gone from a high level NOC position to a developer. Before that he was just doing the help desk stuff. He has no degree, only the usual MIS/networking certifications and as far as I know only tinkers with code on the weekends. I can see where in some scenarios having a good understanding of configurations, packets, users, OU's, etc would be extremely beneficial to a developer.
My question is this, how many full time developers started off this way? Even how many people dual wield the responsibility of developer/systems administrator/network administration?
I'm sure that this is a fairly common scenario. I've spent 12 years in I.T. and I find that as time goes on, the real income comes from being a specialist (DBA, coder, etc.) as opposed to a generalist (network admin, helpdesk).
It's actually the path that my career is taking. I'm not quite a full-time DBA or developer but that's where I'm heading.
I'm also willing to bet that the people skills I've picked up along the way (helpdesk support, network admin, systems analyst) will help me in my DBA/Developer career. Skills I don't feel I would have gotten had I jumped right in to a coding career.
Indeed. I think developers should know the platform they are building software for. If a dev has worked as sysadmin before, he will know how to integreate his software well. Some Windows-Desktop-App related "integration smells" that come to my mind:
App does not run unter normal-user privileges (run on properly secured enterprise desktops? oops!)
App requires write permissions to all kind of system folders (security? oops!)
App stores user settings in 'nonstandard' locations like %programfiles% (backup? permissions? oops!)
App does not provide silent-installable setups (deployment? oops!)
Etc..
A real sysadmin would never write software that has one of the above integration smells. Really.
It's quite common in small companies. I did that for some time - developing the software we sold to customers, keeping the network going, and adding features to the database as needed for a manufacturing company of fewer than 20 people.
You wear many hats in a small business.
But I started off programming microcontrollers in high school, so I can't claim this is where I started.
It is very helpful to have a working knowledge of all these systems as a developer.
-Adam
The overlap of developers and admins happens quite a bit. Our last admin developed on the side just so he'd have a better understanding of what he was helping support. When he left I became the admin just because I tinkered with admin stuff on the side to know how my software was being supported.
A broad understanding with a few focuses is what I'd say is best for any technical professional. Then with a bit of study you can change to meet whatever need may arise.
I've seen it more the other way where a programmer also "admins" the servers and sometimes network. I've definitely been in that position.
I would think it can easily go the other way as well where an admin can start programming systems, but from my experience it's not as common. Whenever I ask a server admin or network person "do you program too?" most of the time the answer is "no".
I think it might be easier for programmers to cross the line because when you are programming a system unless you always have an admin available you need to be able to set up your own environment and that usually includes setting up a server.
I started off as a NOC operator, eventually working my way up to a senior network engineer position. During the last 2-3 years of my tenure at my previous company, I picked up a fondness for programming and started teaching myself everything I could on my own time. Around 2005, I left said company for a small startup and still work there today as as the admin and primary developer.
The one challenge I impose upon myself is to not make admin changes at the drop of a hat to satisfy programming challenges. I must force myself to code in a way that any application I make can be redeployed elsewhere with minimal privileges, despite the fact that I can do pretty much anything I want with our own servers. It's a fine line between performing both duties well and performing one duty badly due to the needs of the other.
I'm here.
Although I've been tinkering with code since I was a child, my first full-time job was being a system administrator, a DBA and other related roles.
Afterwards I worked full time job as a developer, and now I'm both a developer and a security researcher.
Also, I managed to complete M.Sc in CS.
I believe that such transitions are possible, and very beneficial, as you get a wider view on your field of work.