Multiple servers in Jupyter Lab? - jupyter-notebook

I want to run concurrent Jupyter Lab sessions, where the sessions are served by different servers (for example, a local machine and a remote (cloud) server, or two cloud servers).
If a Jupyter Lab instance is running on my local machine and I type jupyter lab on a remote machine and paste the URL in my browser, it asks for a new workspace name or sometimes offers a localhost address that ends in :8889 instead of :8888, but I haven't been able to figure out how to add the remotely-hosted notebook to the existing Jupyter Lab instance as a new tab in Jupyter lab, or to run it side-by-side as a new browser tab.

I finally figured out how to get two cloud VMs running side-by-side Jupyter Lab sessions in the tabs of one browser.
I had configured SSH so that my cloud VM was forwarded to port 8888. The answer was to add a second SSH configuration for the second VM so that it forwarded to port 8889, and then make matching entries in the jupyter_notebook_config files on the two machines, like this:
Edit .ssh/config on your local machine (replace the contents of <...> and the identity file to adapt this to your situation)
##Override for Azure machine 1 ##
Host <IP address of VM 1>
User <your username on VM1>
IdentityFile ~/.ssh/vm1_rsa #SSH private key
LocalForward 8888 localhost:8888
##Override for Azure machine 2 ##
Host <IP address of VM 2>
User <your username on VM2>
IdentityFile ~/.ssh/vm2_rsa #SSH private key (could be the same as for VM1)
LocalForward 8889 localhost:8889
Edit the .jupyter/jupyter_notebook_config.py file on each machine so that VM1's file includes the line c.NotebookApp.port = 8888 and VM2's file includes the line c.NotebookApp.port = 8889.
I haven't tested it yet for the case where you want to run side-by-side Jupyter Lab sessions from a local machine and a remote machine, but I assume the mechanics would be the same.

Related

Mosquitto: Starting in local only mode

I have a virtual machine that is supposed to be the host, which can receive and send data. The first picture is the error that I'm getting on my main machine (from which I'm trying to send data from). The second picture is the mosquitto log on my virtual machine. Also I'm using the default config, which as far as I know can't cause these problems, at least from what I have seen from other examples. I have very little understanding on how all of this works, so any help is appreciated.
What I have tried on the host machine:
Disabling Windows defender
Adding firewall rules for "mosquitto.exe"
Installing mosquitto on a linux machine
Starting with the release of Mosquitto version 2.0.0 (you are running v2.0.2) the default config will only bind to localhost as a move to a more secure default posture.
If you want to be able to access the broker from other machines you will need to explicitly edit the config files to either add a new listener that binds to the external IP address (or 0.0.0.0) or add a bind entry for the default listener.
By default it will also only allow anonymous connections (without username/password) from localhost, to allow anonymous from remote add:
allow_anonymous true
More details can be found in the 2.0 release notes here
You have to run with
mosquitto -c mosquitto.conf
mosquitto.conf, which exists in the folder same with execution file exists (C:\Program Files\mosquitto etc.), have to include following line.
listener 1883 ip_address_of_the_machine(192.168.1.1 etc.)
By default, the Mosquitto broker will only accept connections from clients on the local machine (the server hosting the broker).
Therefore, a custom configuration needs to be used with your instance of Mosquitto in order to accept connections from remote clients.
On your Windows machine, run a text editor as administrator and paste the following text:
listener 1883
allow_anonymous true
This creates a listener on port 1883 and allows anonymous connections. By default the number of connections is infinite. Save the file to "C:\Program Files\Mosquitto" using a file name with the ".conf" extension such as "your_conf_file.conf".
Open a terminal window and navigate to the mosquitto directory. Run the following command:
mosquitto -v -c your_conf_file.conf
where
-c : specify the broker config file.
-v : verbose mode - enable all logging types. This overrides
any logging options given in the config file.
I found I had to add, not only bind_address ip_address but also had to set allow_anonymous true before devices could connect successfully to MQTT. Of course I understand that a better option would be to set user and password on each device. But that's a next step after everything actually works in the minimum configuration.
For those who use mosquitto with homebrew on Mac.
Adding these two lines to /opt/homebrew/Cellar/mosquitto/2.0.15/etc/mosquitto/mosquitto.conf fixed my issue.
allow_anonymous true
listener 1883
you can run it with the included 'no-auth' config file like so:
mosquitto -c /mosquitto-no-auth.conf
I had the same problem while running it inside docker container (generated with docker-compose).
In docker-compose.yml file this is done with:
command: mosquitto -c /mosquitto-no-auth.conf

SSH within R script to get to MySQL Database

I am trying to connect to a MySQL server, which is restricted by being connected to a given server. I am trying to connect through this restricting server while not physically connected.
Through the command line this is doable by creating a SSH connection, after which I can run MySQL commands from the command line. For example:
ssh myUsername#Hostname
myUsername#Hostname's password:
[myUsername#Host ~]$ mysql -h mySQLHost -u mySQLUsername -p mySQLPassword
However, I wish to connect to the MySQL database from within R, so I can send queries to read in tables into my current R session. Usually I would run a R session inside of the commandline, but the server does not have R installed on it.
For example, I have this snippet of code that work when I am physically connected to the server (filled in information changed):
myDB <- dbConnect(MySQL(), user="mySQLUsername", password="mySQLPassword", dbname="myDbname", host="mySQLHost")
In essence, I want to run this same command through a pipe, so that the myDB object is a working mySQL connection.
I have been trying to pipe my way into the restricting server from within R, and have been able to read in a csv file. For example:
dat <- read.table(pipe('ssh myUsername#Hostname "cat /path/to/your/file"'))
This prompts me for my password, and the table is read (as is suggested it would here). However, I am unsure how to translate this to a MySQL connection. For example, should I make the pipe part of the host argument? That was my first thought, but have been unable to make that work.
Any help would be appreciated.
I accomplish a similar task with Postgres using SSH tunneling. Effectively, what you're doing with an SSH tunnel is saying "establish a connection to the remote server, and make a port from that server available as a port on my local machine."
You can set up a SSH tunnel using the following command on your local machine:
ssh -L local_port:lochalhost:remote_port username#remote_host
Specifically, what you're doing with this command is creating a Local Port Forwarding SSH tunnel, which is taking the port you'd connect to directly on the machine with your database installed (remote_port), and securely sending it to the machine you have R installed on as local_port.
For example, for a database server with the following options:
hostname: 192.168.1.3
username: mysql
server mysql port: 3306
You could use the following command (at the command line, or in R using system2) to create a tunnel to port 9000 on your machine:
ssh -L 9000:localhost:3306 mysql#192.168.1.3
Depending on what your exact DBI connection looks like in R, you may have to edit the connection configuration slightly to make it connect to your newly created tunneling port. The reason why I use a different localhost port is that it prevents conflicts with a local version of the database, if you've got one.

how to write a file to a remote server using rhdfs, R and hadoop dfs [duplicate]

I have small example script (script_p.r) like the following, which in intend run in terminal.
#!/usr/bin/Rscript
sink("output_capture.txt")
mn <- mean(1:10)
# and so on, much longer list of tasks
I want to run this script remotely with other iMac host computer (ip address e.g.. not real : 111.111.111.111) which allows me to log in and work (e.g., not real. username user101, password p12334)
Is this way to run this script remotely (say using ssh), say from other computer with ip address: 222.222.222.222 and user name user102 ?
First, put script_p.r on the remote machine.
Then either just do:
ssh user102#222.222.222.222
user102:-$ ./script_p.r
or ssh user102#222.222.222.222 'script_p.r'
or put it in a script:
runremote.sh :
#!/bin/bash
ssh user102#222.222.222.222 'script_p.r'
and run locally
user101:-$ ./runremote.sh

Accessing remote Fuse/Karaf console using SSH

I have a Fuse ESB standalone server running in a RHEL box. I want to connect to the Karaf console remotely to manage the bundles.
If I close my current session, How I go back to my karaf console again ?
I have my Fuse ESB configured to 8101 port for SSH. Will I be able to connect it directly through my SSH client(Putty)
Or Do I need another fuse esb instance locally to access the remote Fuse instance ?
Either ways I am not able to connect, It says access denied. Is there any other easier way to connect to remote fuse/karaf instance ?
Even I tried using Client.sh from bin directory, it says authentication failure. But I have created a JAAS user with Admin role.
By the way, Is just a user is enough to do this ? Or does it need Public/Private key configuration also ?
What is the usual approach for managing the remote Fuse/Karaf instance ?
You can find many details in the JBoss Fuse documentation (eg successor to Fuse ESB) at
https://access.redhat.com/site/documentation/en-US/JBoss_Fuse/
And there is a chapter on remote connecting to containers here
https://access.redhat.com/site/documentation/en-US/JBoss_Fuse/6.0/html-single/Configuring_and_Running_JBoss_Fuse/index.html#ESBRuntimeRemote
You need to pass in credentials for a user on the container that is valid and is in the admin role.
The karaf shell also has a jaas command, which allows you to list the users and their roles etc. And as well add new users, etc. You can also do some user management form the FMC web console that is part of Fuse ESB.
You might also want to check your IPtables
http://ask.xmodulo.com/open-port-firewall-centos-rhel.html.
- $ sudo iptables -I INPUT -p tcp -m tcp --dport 8101 -j ACCEPT
- $ sudo service iptables save
- $ service iptables restart
From another karaf instance you can run this command
JBossFuse:karaf#root> ssh -l username -P password -p port hostname
e.g
- JBossFuse:karaf#root> ssh -l smx-P smx -p 8101 10.234.12.12
You have to make sure that the ssh role name that is defined in etc/org.apache.karaf.shell.cfg
# shRole defines the role required to access the console through ssh
#
sshRole = ssh
matches the one in etc/user.properties
#
# This file contains the users, groups, and roles.
# Each line has to be of the format:
#
# USER=PASSWORD,ROLE1,ROLE2,...
# USER=PASSWORD,_g_:GROUP,...
# _g_\:GROUP=ROLE1,ROLE2,...
#
# All users, grousp, and roles entered in this file are available after Karaf startup
# and modifiable via the JAAS command group. These users reside in a JAAS domain
# with the name "karaf".
#
karaf = karaf,_g_:admingroup
_g_\:admingroup = group,admin,manager,viewer,ssh

client failing to connect to ldap server

I am new to using ldap and slapd and I am having some trouble getting my client machine to connect to the server that is hosting slapd.
here is the run down:
on a ubuntu box I have an instance of virtualbox running a vm with CentOS. I have installed and configured slapd on the CentOS vm and as long as I am on the vm I can use the ldapsearch, ldapadd, etc. once I move to the client machine (the ubuntu distro housing the vm) I run the following:
ldapsearch -x -LLL -b 'dc=example,dc=com' 'uid=Al' -d 255 -H ldap://192.168.1.73:389/
and the following is what I get
ldap_url_parse_ext(ldap://192.168.1.73:389/)
ldap_create
ldap_url_parse_ext(ldap://192.168.1.73:389/??base)
ldap_pvt_sasl_getmech
ldap_search
put_filter: "(objectclass=*)"
put_filter: simple
put_simple_filter: "objectclass=*"
ldap_build_search_req ATTRS: supportedSASLMechanisms
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 192.168.1.73:389
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 192.168.1.73:389
ldap_pvt_connect: fd: 3 tm: -1 async: 0
ldap_close_socket: 3
ldap_msgfree
ldap_err2string
ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
I can connect to the vm via ssh and run the ldapsearch, so connecting shouldn't be an issue. I have configured the router to make the machines ip's static (both the vm and the physical)
any help I could get would be very appreciated.
Thanks,
Al
Firewall? It wouldn't be inconceivable that out of the box the firewall would allow ssh through but not ldap. You also need to verify that your ldap server is configured to listen on the outside interface and not just the loop back. Openldap logging can also be setup to be very verbose about the connections it is receiving. You should do that and monitor your syslog while attempting to connect. That should give you enough information to figure out where the connection is being blocked.

Resources