Remap only the scheme part of URIs - tcp

The mosquitto MQTT client builds URIs of the form mqtt://host:port or mqtts://host:port without an option to override the scheme part. The broker, however, accepts MQTT connections only under tcp://host:port or ssl://host:port. Is there a way to proxy mqtt:// to tcp:// and mqtts:// to ssl://?

There is no need to make any change.
The client library interprets the scheme section internally, it is not presented to the broker in any form.
You just need to pick the right one for the version of the client you are using.

Related

Most important format features: Redesign the format of the client to server messages to a HTTP message

I need to redesign the format of the messages that send from a simple client to server java application to a format that utilizes and is supported by HTTP.
I dont need to actually change the program but rather just come up with the key design changes that would need to be implemented.
I understand that HTTP still uses the TCP transport protocol and then some reformatting of CRUD messages into GET POST DELETE PUT is needed.
Is there any other important design / format requirements that i need to consider?

Python ZeroMQ : connecting two different clients together in a ROUTER and a REP configuration

I have a configuration with the following server/clients :
One server with two bound sockets, a REP and a ROUTER
A client (we will call it a worker) that stays connected to the ROUTER socket
Another (real) client that connects on the REP socket.
I want the server to be able to tell the real client to connect (directly or somehow through the server) to a websocket, opened on the worker client. But it seems, I cannot retrieve the worker's IP-address from a ZeroMQ socket.
How could I achieve this, without some dirty IP-address retrieve hacks?
How could I achieve this, without some dirty IP-address retrieve hacks?
The best would be to use an explicitly communicated IP-address dialogue / handshaking between the server and the worker which would take place upon their setup / initialisation, in which the worker adviced these configuration details to server, upon having been asked to provide a such answer.
Given that, the "new"-real-client .connect()-s it's REQ onto the server's REP, and asks the server about where to go next, the server thus can answer this and the "new"-real-client will get received this way a legitimate IP-address:port# and any additionally needed details for any additional TCP/IP-L3 service establishment and use.
That simple :o) distributed-system
Design-side Epilogue:Because there are some further, design-side implications, hardwired inside of each type of the ZeroMQ sockets' Access-Point, it might be found more appropriate to serve a separate REP-AccessPoint on the server side, so as not to subordinate each "new"-real-client to become dependent upon a presence of events outside of the domains of control of both the server and such "new"-real-client, but to rather allow both such REQ/REP-endpoints to enjoy the independence of anything but their temporally (semi-)private details (re-)negotiation(s).

non-http in mochiweb

I am using mochiweb for a server that may also get a TCP connction to which the client sends a simple string (without a newline, the string is not http). Mochiweb uses HTTP sockets and therefore fails to detect this (i dont even get http_error that i can easily get in mochiweb). How can I solve this? Ideally I wish to change mochiweb code to do setopt({packet, http_or_raw}) but this kind of thing does not exist. How would you recommend handling this? my current idea was to modify mochiweb and use erlang:decode_packet, is there a better approach?
EDIT:
More info.
Our server is a websocket service. We wish to allow
people without a ws supporting browser to use it so we use a
flash object to do websocket when the browser can't. The flash object needs to get a flash policy file. Flash
forces the file to be in one of two places:
- port 843 (flash hard coded)
- the port of the ws service
The flash protocol is NOT HTTP based.
Amazon ELB does not allow port forwarding for
most ports below 1024, so we
implemented the flash server in the same port via a patch to
mochiweb (https://github.com/nivertech/mochiweb/tree/ori_flash_170811).
Any advice?
mochiweb isn't designed to handle this use case, if it doesn't look
like HTTP then the connection is closed and it gets discarded. You
would have to go around mochiweb_http for this purpose. I'd suggest
using an alternate port, or making it look like HTTP.
If I really wanted to do what you say you want to do, I would copy
mochiweb_http.erl to some other name (e.g. sometimes_not_http.erl) and
make the appropriate changes to loop/2 and request/2… then instead of
adding mochiweb_http to your supervisor you'd add sometimes_not_http.
It is not necessary or recommended to make modifications to mochiweb
in-place.

how to intercept and modify HTTP responses on server side?

I am working with a client/server application which uses HTTP, and my goal is to add new features to it. I can extend the client by hooking my own code to some specific events, but unfortunately the server is not customizable. Both client and server are in a Windows environment.
My current problem is that performance is awful when a lot of data are received from the server: it takes time to transmit it and time to process it. The solution could be to have an application on server side to do the processing and send only the result (which is much smaller). The problem is there is not built-in functions to manipulate responses from the server before sending them.
I was thinking to listen to all traffic on port 80, identifying relevant HTTP responses and send them to my application while blocking the response (to avoid sending huge data volume which won't be processed by the client). As I am lacking a lot of network knowledge, I am a bit lost when thinking about how to do it.
I had a look at some low-level packet intercepting methods like WinPCap, but it seems to require a lot of work to do what I need. Moreover I think it is not possible to block or modify responses with this API.
A reverse proxy which allows user scripts to be triggered by specific requests or responses would be perfect, but I am wondering if there is no simpler way to do this interception/send elsewhere work.
What would be the simplest and cleanest method to enable this behavior?
Thanks!
I ended making a simple reverse proxy to access the HTTP server. The reverse proxy then extracts relevant information from the server response and sends it to the server-side processing component, and replaces information extracted from the response by an ID the client uses to request the other component to get the processing results.
The article at http://www.codeproject.com/KB/web-security/HTTPReverseProxy.aspx was very helpful to make the first draft of the reverse proxy.
Hmm.... too much choices.
2 ideas:
configure on all clients a Http Proxy. there are some out there, that let you manipulate what goes through in both directions (with scripts, plugins).
or
make a pass through project, that listens to port 80, and forewards the needed stuff to port 8080 (where your original server app runs)
question is, what software is the server app running at,
and what knowledge (dev) do you have?
ah. and what is "huge data"? kilobyte? megabyte? gigabyte?

"Proxying" HTTP requests

I have some software which runs as a black box, I have no access to it. This software makes HTTP requests. What I want to do is intercept these requests, forward them on, catch the response, do something with it, before passing the response back to the software.
Can this be done? What's the best method?
Thanks
Edit: Requests are to the public internet from a local intranet via a gateway/router. I have root access to my machine. Another machine could be used as intermediate gateway.
Edit 2: Requests are not encrypted. What I am actually trying to do is save down any images that are requested.
Try yellosoft-alchemy.
If the communication isn't encrypted, use Ethereal (or any other similar program) to sniff the communication on the wire.
edit: since the communication isn't encrypted, you can do that easily with Ethereal. You can save each TCP stream independently from there.
Edit2: Ok, you want to do this automatically. In this case, I would suggest you look at two tools available on Linux called tcpflow and tcpreen.
tcpreen creates a proxy similar to what you want between a local port and a remote one. It's a TCP proxy, not an HTTP proxy so this means you'll have to write some parsing tool to isolate the HTTP streams that contain the images you want (probably based on the MIME type of the response). it's not too complex a task, though, if you understand how HTTP works.
tcpflow is similar to tcpreen except that it's a sniffer instead of a proxy. Use whatever tool you think its more adapted to your environment.

Resources