simple and quick tool on linux to watch a http request coming - http

A HTTP request is coming your way, hitting your machine on a port( assumed that you know the request url/route and the port)
what is a simple and quick tool which helps you watch its headers, parameters being passed, using any language, or any tiny web server proxy, any tool.
Being a ruby/rails guy, I quickly created a rails app, set its routes, ran it, and simply followed its logs.
Curious to know how other web app developers deal with this.

I use WireShark and it is very easy and configurable.
If you use debian or derivates:
sudo apt-get install wireshark
else:
http://www.wireshark.org/docs/wsug_html_chunked/ChapterBuildInstall.html

I'd do this with tcpdump. See the man page. There is also a version for windows.
edit
here's the requested example: tcpdump port 80 (yes, it's that simple). Run it with sudo or as root, otherwise you'll get no suitable device found.
If you need more verbose output, add -v or -vv.

Here is how to use wireshark on a Debian-derived GNU/Linux distro:
$ sudo apt-get install wireshark wireshark-doc
$ sudo dumpcap -i eth0
Ctrl+C
$ sudo chmod 644 /tmp/wireshark_pcapng_eth0...
$ wireshark /tmp/wireshark_pcapng_eth0...

Related

systemd network service does not work?

Today, I install ubuntu 16.04 in my machine.
When I boot the system, I can not get the IP address from DHCP server.
The network interface is renamed to enp2s0 from eth0.
I only get the IP address after run command:
$ sudo ethtool -s enp2s0 autoneg off speed 100
I try to add this command in init script (/etc/rc.local) but it does not work after I reboot system.
So, from now, when my machine is booted, I always need to add the above command mannually to get IP address.
Could anyone help me to solve this problem?
Thanks in advance.
Best Regards,
Since you're using ubuntu, use command line and execute the following:
$ sudo crontab -e
and add your script that you want to execute upon restarts by adding this at the end:
#reboot ethtool -s enp2s0 autoneg off speed 100
Hope this helps.
systemd considers both "rc.local" and "crontabs" to be "legacy". It has replacements for both. However, Ubuntu 16.04 supports both concepts.
I think your issue is that now rc.local is run before the network is fully online, causing your command to fail.
Look at /lib/systemd/system/rc.local.service to see how systemd is running your rc.local file. In particular note that the line After=network.target.
Try creating this file:
/etc/systemd/systemd/rc.local.service.d/10-wait-for-network.conf
Add these lines to it:
[Unit]
After=network-online.target
Then reboot and see if that works for you.
References:
Force services to wait for the network to be configured
Using systemd Drop-In units
Also, in the future SO is not the best place for systemd questions. Consider http://askubuntu.com for Ubuntu question instead.
I'll also add that the behavior change you found might be considered a bug in how Ubuntu emulates rc.local support with systemd. If the proposed fix works, perhaps Ubuntu shoudl change the rc.local replacement to load after network-online instead of network.

QUIC traffic generation

I wanted to know if anyone in this community know/found a way to simulate QUIC traffic profile. For analyzing/simulating youtube over WLAN (WiFi), i wanted to know if there is a convenient package like iperf out there.
Thanks
Bharat C P
At the moment there are not any simulation frameworks that can be used to test QUIC traffic.
However there is a remarkable effort in order to provide stand alone implementations with the aim of testing and analyzing QUIC.
I cite just some references you might want to use to this purpose:
Official Google Guide which contains a sample server and client implementation in Chromium.
libquic a library extracted from Chromium's QUIC Implementation
quic-go is an implementation of the QUIC protocol in Go.
I was able to generate QUIC traffic using aioquic. I'm using KDE Neon, which is an Ubuntu derivative.
git clone https://github.com/aiortc/aioquic.git
sudo apt install libssl-dev python3-dev
virtualenv venv1
source venv1/bin/activate
cd aioquic/
pip install -e .
pip install asgiref dnslib httpbin starlette "werkzeug<2.1" wsproto
Then, in one terminal, run the server:
python examples/http3_server.py --certificate tests/ssl_cert.pem --private-key tests/ssl_key.pem
And in another, make an http3 request with the client:
python examples/http3_client.py --ca-certs tests/pycacert.pem https://localhost:4433/
Done.
You can see the traffic in Wireshark. The version that worked for me was 3.6.5. Note that you would need to sniff the loopback interface (Loopback: lo).
Here's how to install Wireshark 3.6.5:
sudo add-apt-repository ppa:wireshark-dev/stable
(need to press enter)
sudo apt update
sudo apt install wireshark

curl can't connect to only certain HTTPS hosts

I am trying to install Meteor.js on a VM (Ubuntu 12.04) created with Vagrant.
The install should be as simple as:
curl https://install.meteor.com | /bin/sh
However this fails with curl: (7) couldn't connect to host
I have isolated the failure to a request within that shell script to this URL:
https://warehouse.meteor.com/bootstrap/0.7.0.1/meteor-bootstrap-Linux_i686.tar.gz
When I changed it to use HTTP instead of HTTPS it works. However I am running into problems elsewhere where it needs to pull things from httpS://warehouse.meteor.com/...
I thought the problem was with https, but if I do:
curl https://google.com
I get the page no problem, so what could be the issue?
Per another Ubuntu/Meteor question, it appears that there's some kind of certificate error (Meteor's SSL CA may not be installed by default in Ubuntu?) that goes away when you:
sudo apt-get update && sudo apt-get upgrade
For me upgrade didn't solve the problem.
My solution was to download the script from install.meteor.com and replace TARBALL_URL from https to http and I ran the script manually.

Why doesn't wireshark detect my interface?

I just installed Wireshark, but when I click capture > interfaces, the dialog box appears, but it does not contain my network interface.
When click on capture > interfaces it appears as in the screenshot below. What can cause this?
This is usually caused by incorrectly setting up permissions related to running Wireshark correctly. While you can avoid this issue by running Wireshark with elevated privileges (e.g. with sudo), it should generally be avoided (see here, specifically here). This sometimes results from an incomplete or partially successful installation of Wireshark. Since you are running Ubuntu, this can be resolved by following the instructions given in this answer on the Wireshark Q&A site. In summary, after installing Wireshark, execute the following commands:
sudo dpkg-reconfigure wireshark-common
sudo usermod -a -G wireshark $USER
Then log out and log back in (or reboot), and Wireshark should work correctly without needing additional privileges. Finally, if the problem is still not resolved, it may be that dumpcap was not correctly configured, or there is something else preventing it from operating correctly. In this case, you can set the setuid bit for dumpcap so that it always runs as root.
sudo chmod 4711 `which dumpcap`
One some distros you might get the following error when you execute the command above:
chmod: missing operand after ‘4711’
Try 'chmod --help' for more information.
In this case try running
sudo chmod 4711 `sudo which dumpcap`
In Windows, with Wireshark 2.0.4, running as Administrator did not solve this for me. What did was restarting the NetGroup Packet Filter Driver (npf) service:
Open a Command Prompt with administrative privileges.
Execute the command sc query npf and verify if the service is running.
Execute the command sc stop npf followed by the command sc start npf.
Open WireShark and press F5.
Source: http://dynamic-datacenter.be/?p=1279
For *nix OSes, run wireshark with sudo privileges. You need to be superuser in order to be able to view interfaces. Just like running tcpdump -D vs sudo tcpdump -D, the first one won't show any of the interfaces, won't compalain/prompt for sudo privileges either.
So, from terminal, run:
$ sudo wireshark
As described in other answer, it's usually caused by incorrectly setting up permissions related to running Wireshark correctly.
Windows machines:
Run Wireshark as administrator.
By Restarting NPF, I can see the interfaces with wireshark 1.6.5
Open a Command Prompt with administrative privileges.
Execute the command "sc stop npf".
Then start npf by command "sc start npf".
Open WireShark.
That's it.
On Fedora 29 with Wireshark 3.0.0 only adding a user to the wireshark group is required:
sudo usermod -a -G wireshark $USER
Then log out and log back in (or reboot), and Wireshark should work correctly.
I hit the same problem on my laptop(win 10) with Wireshark(version 3.2.0), and I tried all the above solutions but unfortunately don't help.
So,
I uninstall the Wireshark bluntly and reinstall it.
After that, this problem solved.
Putting the solution here, and wish it may help someone......
Just uninstall NPCAP and install wpcap. This will fix the issue.

What does cifs_mount failed w/return code = -22 indicate

I am trying
sudo mount -t cifs //<server>/<share> -o username=user#domain,password=**** /mnt/<mountpoint>
error message:
mount: wrong fs type, bad option, bad superblock on //server/share,
missing codepage or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
The syslog has
CIFS VFS: cifs_mount failed w/return code = -22
I am able to mount the same share on another centos system. I can ping the server, mount point directory has been created.
I ran into this problem when using a host name and solved it by using an IP address. E.g.:
use
mount -t cifs //192.168.1.15/share
rather than
mount -t cifs //servername/share
Another possible solution is to install
cifs-utils
.
Ah, the dreaded -22. Basically this seems to be used as a catchall for "something didn't work", although technically it's referred to as an invalid argument.
The client does IMHO a very poor job of telling you the actual problem. (This may not be its fault - it doesn't always have access to that information).
However -- have you checked the logs on the server/machine you are connecting to?
I was connecting to an OS X samba server, and learned from what I found in the logs there that it was necessary to specify additional options under -o as follows:
nounix,sec=ntlmssp
Among the things these settings enable are "allow long names", and "ignore UNIX filename endings"...sec is to specify security flags.
Another possibility is that you're trying to access a filesystem of a type that mount.cifs can't actually handle.
For RHEL/Centos install package - "cifs-utils"
Maybe move the target?
sudo mount -t cifs -o username=user#domain,password=**** //<server>/<share> /mnt/<mountpoint>
Or maybe this solution? (Ubuntu, Debian methods)
sudo apt-get install smbfs
Or for CentOS, RedHat, Fedora try:
sudo yum install samba-client
I had a similar issue on Ubuntu 12.04 with the "mount" package (version 2.20.1-1ubuntu3).
It happened when I was trying to mount the server share using its hostname rahter than its IP.
Another way to solve the issue on Ubuntu was to install the cifs-utils package. That way I could also mount the samba share using the exact same command line (or fstab) but with hostname.
sudo mount -t cifs //hostname/share -o username=user,password=pwd /mnt/share
Just did a clean install of Ubuntu 12.04 LTS and got this trying to hook up my Linux HTPC.
Solved it by running: sudo apt-get install cifs-utils then remounting it.
CIFS returns code "-22" in many cases (not only invalid arguments).
For me installing keyutils did the trick:
apt-get install keyutils
My distribution is "Ubuntu 14.04.2 LTS".
I figured this out by increasing the logging verbosity of CIFS:
echo 7 > /proc/fs/cifs/cifsFYI
# disable again via:
#echo 0 > /proc/fs/cifs/cifsFYI
Documentation on the bitmask ("7") for cifsFYI can be found here: https://www.kernel.org/doc/readme/Documentation-filesystems-cifs-README
After trying to mount once more dmesg included more helpful information:
Dec 7 12:34:20 pc1471 kernel: [ 5442.667417] CIFS VFS: dns_resolve_server_name_to_ip: unable to resolve:
Another maybe helpful link:
http://vlkan.com/blog/post/2015/01/08/smb-mount-troubleshoot/
I have Ubuntu Server 12.10 x64 installed as a VMware VM, running on OS X 10.8 (Mountain Lion).
On the Mac, in SYSTEM PREFERENCES > SHARING > FILE SHARING (on), I added a folder to share. For my tests, I created a new folder within my Public folder called "ubuntu".
In Ubuntu, I issued the following commands:
sudo mkdir /media/target
sudo mount.cifs //10.0.20.3/ubuntu /media/target -o username=davidallie,nounix,sec=ntlmssp,rw
Ubuntu prompted me for the password and, once entered, mounted the folder. I then ran:
df -H
which allowed me to verify the mounts and mount-points.
This has recently manifested thanks to a kernel bug in v5.18.8+, I was able to reproduce on v5.18.9 and v5.18.11.
Here is the relevant ticket on kernel.org, quote:
it appears that kernel 5.18.8 breaks cifs mounts on my machine. With
5.18.7, everything works fine. With 5.18.8, I am getting:
$ sudo mount /mnt/openmediavault/
mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel
log messages (dmesg)
The relevant /etc/fstab line is:
//odroidxu4.local/julian /mnt/openmediavault cifs
credentials=/home/julas/.credentials,uid=julas,gid=julas,vers=3.1.1,nobrl,_netdev,auto
0 0
Here is the offending commit, and here is the fix, which applies cleanly to v5.18.11. The cause is, from what I understand, a bug in old versions of the samba server in the negotiation protocol.
If this is your issue, you can:
patch your kernel yourself;
downgrade to v5.18.7;
switch to an LTS kernel;
use the userspace (and also really slow and awful) gvfs-smb;
upgrade the samba version on your server; or
add vers=2.0 to the mount.cifs options in /etc/fstab.
Note that while I haven't tried the last one personally, the venerable #SEBiGEM has confirmed in the comments that it works for v5.18.10.
Note also that I didn't try upgrading samba on the server at all because I hate touching the box it's running on - every time I upgrade anything everything breaks. Doing so might also not be an option for those with NAS appliances.
As a personal sidenote, it's a little sad that so many different things can cause -22. My answer is correct, but very very niche and specific to this point in time. I imagine in a month it will simply be useless noise.
Just experience the problem on RHEL 5. You don't need to install the samba suite, just the samba-client and any dependencies.
Maybe it's too late, but simplest solution described in kernel bug 50631:
in the latest code, unc mount parameter in mandatory. Modified command works for me:
sudo mount -t cifs //<server>/<share> -o username=user#domain,password=****,unc=\\\\<server>\\<share> /mnt/<mountpoint>
Try run the comamnd:
$modinfo cifs
filename: /lib/modules/3.2.0-60-virtual/kernel/fs/cifs/cifs.ko
version: 1.76
description: VFS to access servers complying with the SNIA CIFS Specification e.g. Samba and Windows
license: GPL
author: Steve French <sfrench#us.ibm.com>
srcversion: 9435BBC2F61D29F06643803
depends:
intree: Y
vermagic: 3.2.0-60-virtual SMP mod_unload modversions 686
parm: CIFSMaxBufSize:Network buffer size (not including header). Default: 16384 Range: 8192 to 130048 (int)
parm: cifs_min_rcv:Network buffers in pool. Default: 4 Range: 1 to 64 (int)
parm: cifs_min_small:Small network buffers in pool. Default: 30 Range: 2 to 256 (int)
parm: cifs_max_pending:Simultaneous requests to server. Default: 32767 Range: 2 to 32767. (int)
parm: echo_retries:Number of echo attempts before giving up and reconnecting server. Default: 5. 0 means never reconnect. (ushort)
parm: enable_oplocks:Enable or disable oplocks (bool). Default:y/Y/1 (bool)
If your getting any error then cifs is not installed. Just check with your admin. I thought it helps out.
Adding the option vers=3.0 to the mount command worked for me: sudo mount -t cifs -v <src> <dst> -o ...,vers=3.0,...
You need to install cifs-utils first , just as follows:
sudo yum install cifs-utils
I know this is old, but on older cifs-utils versions, you may have to add the following two lines to /etc/request-key.conf
create cifs.spnego * * /usr/sbin/cifs.upcall -c %k
create dns_resolver * * /usr/sbin/cifs.upcall %k
Workaround without installing additional packages (cifs-utils adds another 81mb in Debian Stretch):
$ FILESERVER_IP=$(getent hosts myfileserver.com | awk '{ print $1 ; exit }')
$ sudo mount -t cifs //${FILESERVER_IP}/<share> -o username=user#domain,password=**** /mnt/<mountpoint>
Many answers, but wasn't work for me.
Solution:
My NAS didn't support Samba 3.0, on which my mount switch automatically.
So I downgraded smb version:
mount -t cifs //192.168.0.2/Share -o rw,vers=1.0,username=*****,password=******* /media/1
It's work.

Resources