I have several Jupyter notebooks which perform analysis on datasets. Right now, a dataset is specified by its filename. Every time the user wants to perform analysis on a new dataset, she/he has to edit the appropriate line in the notebook and modify dataset path string. The datasets can be located in different directories. The notebooks can also be located in different directories. In each notebook I would like to provide a widget that allows the user to browse the remote file system and pick the dataset he/she wants to analyse.
Are there any open source projects that support the above functionality? I am looking for something that is still active/supported and has some basic documentation. I did quick search on Google and surprisingly I didn't find anything.
Then I realised that JupyterLab, the evolution of Jupyter, has something very similar to what I want. It already has a very capable file browser but it is a bit "isolated" from everything else.
Is it possible somehow to get the relative (to the currently opened notebook) path of the selected file in the JupyterLab file browser?
Thank you.
Here's code for a server-side file browsing widget. Only tested in regular Jypter notebook - not Jupyter Lab. Also, must use a fairly recent version. Hope this helps.
import sys
import os
import ipywidgets as ui
from IPython.display import display
class PathSelector():
def __init__(self,start_dir,select_file=True):
self.file = None
self.select_file = select_file
self.cwd = start_dir
self.select = ui.SelectMultiple(options=['init'],value=(),rows=10,description='')
self.accord = ui.Accordion(children=[self.select])
self.accord.selected_index = None # Start closed (showing path only)
self.refresh(self.cwd)
self.select.observe(self.on_update,'value')
def on_update(self,change):
if len(change['new']) > 0:
self.refresh(change['new'][0])
def refresh(self,item):
path = os.path.abspath(os.path.join(self.cwd,item))
if os.path.isfile(path):
if self.select_file:
self.accord.set_title(0,path)
self.file = path
self.accord.selected_index = None
else:
self.select.value = ()
else: # os.path.isdir(path)
self.file = None
self.cwd = path
# Build list of files and dirs
keys = ['[..]'];
for item in os.listdir(path):
if item[0] == '.':
continue
elif os.path.isdir(os.path.join(path,item)):
keys.append('['+item+']');
else:
keys.append(item);
# Sort and create list of output values
keys.sort(key=str.lower)
vals = []
for k in keys:
if k[0] == '[':
vals.append(k[1:-1]) # strip off brackets
else:
vals.append(k)
# Update widget
self.accord.set_title(0,path)
self.select.options = list(zip(keys,vals))
with self.select.hold_trait_notifications():
self.select.value = ()
f = PathSelector('/some/data')
display(f.accord)
Related
I have a file selection window from an windows application I'm trying to automate.
I can change the File name box at the bottom by using
app2.window(title_re='Select a batch file') \
.File_name_Edit.set_text(selected_filename)
I have previously entered the full filepath and filename in to the lower box, however, it will no longer accept slashes in the File name box after a windows update. Hence the problem.
So the problem is the top (directory/folder name) box.
Manually I can select the box, highlight it then overtype the directory.
I cannot find a way to do this using pywinauto.
Methods attempted are:
toolbar2 = app2.window(title_re='Select a batch file') # \
# .child_window(title_re='Address', auto_id='1001')
# .child_window(title_re='Address', control_type="ToolBar", auto_id='1001')
file_path_address = toolbar2['Address band toolbarToolbar'].click_input()
file_path_address.set_text(directory_path)
# ToolbarWrapper(toolbar2).set_text(directory_path)
# ToolbarWrapper(toolbar2).click(button='All locationsSplitButton').set_text(directory_path)
# ToolbarWrapper(toolbar2)['Address:'].set_text(directory_path)
I'm using set_text as the automation must run behind a locked screen, type_keys does not work with a locked screen as type_keys is a keyboard method and Windows blocks keyboard entry when the screen is locked.
I have no access to the internals of the program being automated in order to change the original directory path and need to change it to read in from a different location.
Any assistance would be appreciated.
See below code, It runs fine with Notepad++ on Win 10
import time
from pywinauto.application import Application
app = Application().Connect(title=u'Open', class_name='#32770')
window = app.Dialog
toolbarwindow = window.Toolbar3
toolbarwindow.Click()
time.sleep(0.30)
static = window.Edit2
directory_path = "C:\Users\Desktop"
raw_directory_path = r'{}'.format(directory_path)
static.set_edit_text(text = raw_directory_path)
static.type_keys("{ENTER}")
you can use right click then edit the field
toolbar2['Address band toolbarToolbar'].click_input(button="right")
app2.PopupMenu.child_window(title="Edit address", control_type="MenuItem").click_input()
app2.child_window(title="Address", control_type="Edit").set_edit_text(directory_path)
I would like to know if it is possible to add a custom command to the GNAT Programming Studio (GPS)?
If the custom command is invoked (by a button in the menu bar or a keyboard shortcut) an external Python script should be called with the full/absolute path to the file that is opened and selected in the editor.
This is a quick-and-dirty script that might give some direction. I tested it on Linux, but it should also work on Windows. Change the action near the end to invoke the script you like. To actually use it, you must put it in the (hidden) .gps/plug-ins directory which can be found in your home directory. The actual action can be invoked from the context menu in the source code window.
run_my_script.py
"""Run Python Script
This plug-in executes a python script.
"""
###########################################################################
# No user customization below this line
###########################################################################
import os, sys
import GPS
from gps_utils import interactive
def __contextualMenuFilter(context):
# Check if the context is generated by the source editor
if not (context.module_name == "Source_Editor"):
return False
# If all OK, show the menu item in the context menu.
return True
def __contextualMenuLabel(context):
# Get current buffer
name = context.file().name()
basename = os.path.basename(name)
# Name of the menu item.
return "Run Python script for <b>{}</b>".format(basename)
#interactive(
name ="Run Python script",
contextual = __contextualMenuLabel,
filter = __contextualMenuFilter)
def on_activate():
# If modified, then save before proceeding.
eb = GPS.EditorBuffer.get()
if eb.is_modified:
eb.save()
# Run the action (defined below).
GPS.execute_action("my_script")
GPS.parse_xml ("""
<action name="my_script">
<external output="Output of my_script">python3 /home/deedee/my_script.py %F</external>
</action>""")
my_script.py (some test script)
import sys
print ("Running script {0} for {1}".format(sys.argv[0], sys.argv[1]));
output (shown in GPS on a new tab named "Output of my_script")
python3 /home/deedee/my_script.py /home/deedee/example/src/main.adb
Running script /home/deedee/my_script.py for /home/deedee/example/src/main.adb
Some relevant info from the GNAT Studio (formerly GPS) documentation:
15.5.2. Defining Actions
15.5.3. Macro arguments
15.8.7.3. Redirecting the output of spawned processes
17. Scripting API reference for GPS
I would like to save a copy of a notebook (or rename it) from within a cell of the notebook.
Preferably without too much JavaScript. Actually I guess something of this form should work
from IPython.display import display_html
display_html("script>Jupyter....???...()</script>")
Here is a solution only in Python. The notebook_path function comes from P.Toccaceli's solution on How do I get the current IPython Notebook name.
from notebook import notebookapp
import urllib
import json
import os
import ipykernel
from shutil import copy2
def notebook_path():
"""Returns the absolute path of the Notebook or None if it cannot be determined
NOTE: works only when the security is token-based or there is also no password
"""
connection_file = os.path.basename(ipykernel.get_connection_file())
kernel_id = connection_file.split('-', 1)[1].split('.')[0]
for srv in notebookapp.list_running_servers():
try:
if srv['token']=='' and not srv['password']: # No token and no password, ahem...
req = urllib.request.urlopen(srv['url']+'api/sessions')
else:
req = urllib.request.urlopen(srv['url']+'api/sessions?token='+srv['token'])
sessions = json.load(req)
for sess in sessions:
if sess['kernel']['id'] == kernel_id:
return os.path.join(srv['notebook_dir'],sess['notebook']['path'])
except:
pass # There may be stale entries in the runtime directory
return None
def copy_current_nb(new_name):
nb = notebook_path()
if nb:
new_path = os.path.join(os.path.dirname(nb), new_name+'.ipynb')
copy2(nb, new_path)
else:
print("Current notebook path cannot be determined.")
Then, simply use copy_current_nb('Save1') to create a copy named Save1.ipynb in the same directory.
I want to put couple of cells with commands I need in almost every new notebook in every new notebook I create.
For example when I create a new notebook it should put a
%matplotlib inline
import matplotlib.pyplot as plt
in a cell by default but not execute it.
How could I set something like that up?
This will work for both terminal based IPython shell and Browser based Notebook:
Navigate to ~/.ipython/profile_default
Create a folder called startup if it’s not already there
Add a new Python file called start.py
Put your favorite imports (and custom functions may be) in this file
Launch IPython or a Jupyter Notebook and your favorite libraries will be automatically loaded every time!
Here is my sample for start.py:
Another Source
To define set of commands on default startup, you need to add the commands in the templete ipy_user_conf.py file in your ~/.ipython directory.
This module is imported during IPython startup. So you can easily do : import modules, configure extensions, change options, define magic commands, put variables and functions in the IPython namespace etc.
Here is the sample ipy_user_conf.py :
# Most of your config files and extensions will probably start
# with this import
import IPython.ipapi
ip = IPython.ipapi.get()
# You probably want to uncomment this if you did %upgrade -nolegacy
# import ipy_defaults
import os
def main():
#ip.dbg.debugmode = True
ip.dbg.debug_stack()
# uncomment if you want to get ipython -p sh behaviour
# without having to use command line switches
import ipy_profile_sh
import jobctrl
# Configure your favourite editor?
# Good idea e.g. for %edit os.path.isfile
#import ipy_editors
# Choose one of these:
#ipy_editors.scite()
#ipy_editors.scite('c:/opt/scite/scite.exe')
#ipy_editors.komodo()
#ipy_editors.idle()
# ... or many others, try 'ipy_editors??' after import to see them
# Or roll your own:
#ipy_editors.install_editor("c:/opt/jed +$line $file")
o = ip.options
# An example on how to set options
#o.autocall = 1
o.system_verbose = 0
#import_all("os sys")
#execf('~/_ipython/ns.py')
# -- prompt
# A different, more compact set of prompts from the default ones, that
# always show your current location in the filesystem:
#o.prompt_in1 = r'\C_LightBlue[\C_LightCyan\Y2\C_LightBlue]\C_Normal\n\C_Green|\#>'
#o.prompt_in2 = r'.\D: '
#o.prompt_out = r'[\#] '
# Try one of these color settings if you can't read the text easily
# autoexec is a list of IPython commands to execute on startup
#o.autoexec.append('%colors LightBG')
#o.autoexec.append('%colors NoColor')
o.autoexec.append('%colors Linux')
# some config helper functions you can use
def import_all(modules):
""" Usage: import_all("os sys") """
for m in modules.split():
ip.ex("from %s import *" % m)
def execf(fname):
""" Execute a file in user namespace """
ip.ex('execfile("%s")' % os.path.expanduser(fname))
main()
For more details, please refer the link : Customization of IPython.
I hope this is what you wanted to know.
JupyterLab
In a comment to one of the other answers, the OP pointed out the need to insert the actual code instead of having it load in the background. One way is to create a text keyboard shortcut by going to Settings -> Advanced settings editor -> JSON settings Editor and adding the following under User Preferences:
{
"shortcuts": [
{
"command": "apputils:run-first-enabled",
"selector": "body",
"keys": ["Alt I"],
"args": {
"commands": [
"console:replace-selection",
"fileeditor:replace-selection",
"notebook:replace-selection",
],
"args": {"text": "import pandas as pd\nimport altair as alt\n\n"}
}
}
]
}
This will insert the following snippet each time you press Alt + i in the notebook:
import pandas as pd
import altair as alt
# <-- Cursor placed here
More on text shortcuts in jupyterlab
IPython console
If you are interested in automatically importing commonly used libraries in the IPython console only so that they are there for interactive use, but not in the notebook to avoid issues with sharing notebooks lacking some imports, you can launch IPython like so (and set up an alias to not have to type this each time):
ipython -c "import pandas as pd; import numpy as np" -i
(This was what I was looking for when I originally found this question)
I am enjoying developing inside the ipython notebook, but I am having a problem when I want to write a main() function that reads the command line args (with OptionParser, for example). I want to be able to export the code to a .py file and run it from the command line, but I haven't found a way to have a main() that runs both in the notebook with predefined arguments or from the command line with python and command line args. What is the secret?
In case that is not clear, I would like to do something like this:
if __name__ == '__main__':
# if in the notebook
vals = {'debug':True, 'tag_file': 't.tags'}
options = Object()
for k,v in vals.items():
options.setattr(k,v)
args = 'fname1.txt'
# if running as a command line python script
from optparse import OptionParser
parser = OptionParser()
parser.add_option('-d','--debug',action='store_true',dest='debug')
parser.add_option('-t','--tags',action='store',dest='tag_file')
options,args = parser.parse_args()
You cannot determine that you are in an IPython notebook or a qtconsole, or a simple IPython shell, for the simple reason the 3 can be connected to the same kernel at the same time.
It would be like asking, what color is the current key the user is typing. You could get it by looking the plugged usb devices and look for images on the internet and guess the keyboard color, but nothing guarantees you it will be accurate, nor that it won't change, and user can have multiple keyboard plugged, or even painted keyboard.
It is really the same with the notebook, Even if you determine you are in ZMQKernel, are you speeking to qtconsole or webserver ? Again, you found that you were talking to the webserver, are you talking to JS or Emacs ? And so on and so forth.
The only thing you can do, you can ask the user.
What is reliable, is test wether you are in IPython or not.
If you really but reeaaalllyy want a way, as until now, the notebook is the only thing that can display Javascript. And javascript can execute code in pyton. So you might be able to create something that display JS that send back info to the kernel. And using thread and timer you can say that you were not in a notebook (but you will have a race condition).
Don't worry about the distinction. Just set default values, and unless they are overridden from the command line, use those.
if __name__ == '__main__':
parser = OptionParser()
parser.add_option('-d', '--debug', action='store_true', dest='debug',
default=True)
parser.add_option('-t','--tags',action='store',dest='tag_file',
default='t.tags')
options, args = parser.parse_args()
if not args:
args = ['fname1.txt']