I'm beginning to learn to code.
Someone said to me: "cURL is the best http client".
To help me understand this sentence, I have two questions:
what is an HTTP CLIENT; and
what is cURL?
I understand you are asking two things:
What is an HTTP CLIENT?
This is any program/application used make communications on the web using the Hypertext Transfer Protocol (HTTP). A common example is a browser.
What is cURL?
This a particular HTTP CLIENT designed to make HTTP communications on the web but built to be used via the command line of a terminal (command prompt).
If you perform a search for these topics, you will easily be able to find more in depth explanations about each.
Related
This question is similar to
Starting to use OpenSSL
but more specific and detailed so I think it's fair to ask.
Suppose I have an simple http server that does the following in a successful GET scenario
creates a listening socket
a client connects
reads the data through recv
parses the GET request, now it knows which resource to return
writes the response through send
close the socket
This server is written in c++ on linux.
My question is, What does it take to convert this server into an minimal https server? (in particular using OpenSSL, but answers in a general sense are welcome.)
Here's my understanding (question marks mean I'm have no idea)
initialize the library
read the server certificate and private key and other configurations
create a normal listening socket(?)
a client connects
do the handshaking through a library function(?)
handshaking done
do I need a special step before I start receiving and sending data?
read data through library function(?)
does the data look exactly like an HTTP GET at this point?
if it does, parse the GET and get the resource
write return data through library function(?)
close the connection through a library function(?)
In summary, I'm hoping that it only requires adding some extra steps to the current code and does not affect the HTTP parsing. Is this assumption correct?
Many thanks to anybody who could fill in the blanks.
Look through "Network Security with OpenSSL", as it covers this. Even if you don't have the book, you can look through the code.
I'm trying to write a small program that I can talk to Omegle strangers via command line for school. However I'm having some issues, I'm sure I could solve the problem if I could view the headers sent however if you talk to a stranger on Omegle while Live HTTP Headers (or a similar plug-in or program) is running the headers don't show. Why is this? Are they not sending HTTP headers and using a different protocol instead?
I'm really lost with this, any ideas?
I had success in writing a command line Omegle chat client. However it is hardcoded in C for POSIX and curses.
I'm not sure what exactly your problem is, maybe it's just something with your method of reverse engineering Omegle's protocol. If you want to make a chat client, use a network packet analyzer such as Wireshark (or if you're on a POSIX system I recommend tcpdump), study exactly what data is sent and received during a chat session and have your program emulate what the default web client is doing. Another option is to de-compile/reverse engineer the default web client itself, which would be a more thorough method but more complicated.
when I type wget http://yahoo.com:80 on unix shell. Can some one explain me what exactly happens from entering the command to reaching the yahoo server. Thank you very much in advance.
RFC provide you with all the details you need and are not tied to a tool or OS.
Wget uses in your case HTTP, which bases on TCP, which in turn uses IP, then it depends on what you use, most of the time you will encounter Ethernet frames.
In order to understand what happens, I urge you to install Wireshark and have a look at the dissected frames, you will get an overview of what data belongs to which network layer. That is the most easy way to visualize and learn what happens. Beside this if you really like (irony) funny documents (/irony) have a look at the corresponding RFCs HTTP: 2616 for example, for the others have a look at the external links at the bottom of the wikipedia articles.
The program uses DNS to resolve the host name to an IP. The classic API call is gethostbyname although newer programs should use getaddrinfo to be IPv6 compatible.
Since you specify the port, the program can skip looking up the default port for http. But if you hadn't, it would try a getservbyname to look up the default port (then again, wget may just embed port 80).
The program uses the network API to connect to the remote host. This is done with socket and connect
The program writes an http request to the connection with a call to write
The program reads the http response with one or more calls to read.
I have some software which runs as a black box, I have no access to it. This software makes HTTP requests. What I want to do is intercept these requests, forward them on, catch the response, do something with it, before passing the response back to the software.
Can this be done? What's the best method?
Thanks
Edit: Requests are to the public internet from a local intranet via a gateway/router. I have root access to my machine. Another machine could be used as intermediate gateway.
Edit 2: Requests are not encrypted. What I am actually trying to do is save down any images that are requested.
Try yellosoft-alchemy.
If the communication isn't encrypted, use Ethereal (or any other similar program) to sniff the communication on the wire.
edit: since the communication isn't encrypted, you can do that easily with Ethereal. You can save each TCP stream independently from there.
Edit2: Ok, you want to do this automatically. In this case, I would suggest you look at two tools available on Linux called tcpflow and tcpreen.
tcpreen creates a proxy similar to what you want between a local port and a remote one. It's a TCP proxy, not an HTTP proxy so this means you'll have to write some parsing tool to isolate the HTTP streams that contain the images you want (probably based on the MIME type of the response). it's not too complex a task, though, if you understand how HTTP works.
tcpflow is similar to tcpreen except that it's a sniffer instead of a proxy. Use whatever tool you think its more adapted to your environment.
I have heard that http is a nice way to design my own protocol. although i can design a binary protocol, i would prefer to follow the HTTP standard to design my protocol.
basically the flow of the application is that with the request the client sends some parameter strings to the server, the server sends the response string to the application. this procedure continues several times, before the connection thread terminates.
i am using java servlets for the above.
how should the client send the HTTP parameters so that parsing is easy at the server.
Get /a HTTP/1.1
Host: localhost
??? what comes here
??? what comes here
Since that is a GET request, nothing.
I'd suggest using querystring parameters, then you can access them using ServletRequest.getParameterNames(), getParameterValues(), getParameterMap() etc.
So, your request line would take the form:
GET /a?x=1&y=1 HTTP/1.1
since this is the standard way of passing parameter data, other clients, such as web browsers, will be able to use your service easily.
This assumes that the operation does not cause side-effects on the server. If it does then you should be using a POST, PUT or DELETE request depending on the exact nature of the operation.
HTTP Made Really Easy is a useful document since, at least initially, the HTTP Spec can be a bit daunting.
Why not base your protocol on something that already exists for example SOAP?
What you're designing is a data exchange format, not a protocol really.
So the question is, really, what sort of data do you want to send? To answer that, you need to consider who is receiving it. If it's yourself, then just keep it simple.