I am using IPython Notebook through django_extensions:
python manage.py shell_plus --notebook
This saves the Notebook files to the current folder (Django project folder). How can I change the save location for .ipynb files?
You can change the directories where files are stored and read from using these parameters in ~/.ipython/profile_projectname/ipython_notebook_config.py, where projectname is your Django project.
c.NotebookManager.notebook_dir = u'/path/to/files'
c.FileNotebookManager.notebook_dir = u'/path/to/files'
This seems to ruin the import path to django however. I've been trying to play around with syspath to get this right via adding startup scripts in the startup directory, but have not found a solution that works yet. If you find a solution, let me know, because I'd like to have my notebook files outside of my project root directory as well.
A little late to the game, but I managed to save the notebooks in a different location + auto importing Django settings.
I start my notebooks with:
PYTHONPATH=/path/to/project/root DJANGO_SETTINGS_MODULE=settings python manage.py --notebook --no-browser
The PYTHONPATH enables finding the correct modules of my project and all shell_plus imports work automatically like a charm.
P.S.
As I am running this command executed from my host in my vagrant box --no-browser prevents opening w3m ^^
Related
Question: How can one set the working directory of all notebooks opened in Jupyter Lab with a double-click, to be the project's folder, /myproject/, regardless of the notebook's subfolder within that parent folder? The working directory is identified by !pwd on Linux/Mac or !cd on Windows.
Context:
The Jupyter Lab sessions is initiated from the project's folder, by: /myproject/jupyter lab.
I am not looking for changing the working directory within the notebook with code (e.g. with !cd.. or using os), but change the settings of Jupyter Lab, such that the all kernels will start with the folder from where Jupyter Lab was initiated.
This is useful for being able to use consistent relative folders, regardless of the subfolder they are referred from. For example, for loading data in a specific subfolder.
I agree that this is often a preferred approach! I always have my notebooks configured to work like that because it:
makes it easy to specify paths to data and for outputs and
allows moving a notebook between directories without the need to change the paths
makes jupyterlab-lsp code intelligence work more reliably
I have a python module called make_paths_absolute.py with the following contents:
from os import chdir
from pathlib import Path
def make_paths_relative_to_root():
"""Always use the same, absolute (relative to root) paths
which makes moving the notebooks around easier.
"""
top_level = Path(__file__).parent
chdir(top_level)
make_paths_relative_to_root()
And in the first cell of every notebook, I add a line import make_paths_absolute. I like it this way because it makes it:
reproducible: anyone who copies/clones my project will be able to run the code without the need to customize anything in their Jupyter environment
work with all notebook interfaces (whether JupyterLab/RetroLab/classic Notebook)
is quite obvious for anyone reading the notebook that they should expect the paths to be absolute (=relative to the root).
To make that work you first need to set PYTHONPATH pointing to the root of the repository when starting JupyterLab. To do so on Linux/Mac you can prepend the jupyter lab command with:
PYTHONPATH=/path/to/your/lab/root:$PYTHONPATH jupyter lab
Alternatively, you can specify PYTHONPATH in kernel.json (either by modifying the default kernel specification or creating a copy of it first, see https://stackoverflow.com/a/53595397).
PS. The above is a simplification of my setup. In reality, I store make_paths_absolute.py in helpers package (so there is also an empty helpers/__init__.py and there is extra .parent) together with other initialization-time code, including a notebook called notebook_setup.ipynb which contains things like %matplotlib inline, pandas customizations (like making sure it uses stable sort), warning filters etc. So my every notebook starts with:
import helpers.make_paths_absolute
%run helpers/notebook_setup.ipynb
and I am really liking this setup working like that for two years now without any problems.
There is a feature request for making this easier at: https://github.com/jupyterlab/jupyterlab/issues/11619.
The most easiest solution in my opinion
Open Notepad
Paste the command "jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10"
Save the notepad file with an extension of ".bat" instead of ".txt"
Paste the file in which directory you want to initialize your jupyter
Double click and open the ".bat" file
Jupyter opens with desired directory as base
This way you control the jupyter root directory as and when required and don't
really have to perform any manual settings
Hope this helps
".bat" file created on Desktop
enter image description here
".bat" file double clicked and executed
enter image description here
jupyter opens with Desktop as the intended base directory
enter image description here
So, the title basically covers my question. I've created a project using virtualenv, e.g. I have to
source ./env/bin/activate
to run my script.
When I try creating an executable using:
pyinstaller --onefile <myscript.py>
None of the virtualenv packages are included; just the ones that are installed globally. I have a requirements.txt file that contains all of the modules I need. Is there a way to have pyinstaller point to that for the needed modules, or is there another way?
As Valentino pointed out by looking at How can I create the minimum size executable with pyinstaller?
You have to run PyIntaller from inside the virtual environment:
(venv_test) D:\testenv>pyinstaller
How to solve the not importing modules from the virtual environment
The virtual environment saves modules in a different directory than the global module directory. If you are using Windows like me, you can find the modules directory here:
C:\Users\<your username>\.virtualenvs\<your project name>\Lib\site-packages
When you find your virtualenv directory, run this command instead of this simple command(pyinstaller <script>.py):
pyinstaller --paths "C:\Users\<your username>\.virtualenvs\<your project name>\Lib\site-packages" --hidden-import <module name that should be import> <your script name>.py
To export just one file you can add this: -F or --onefile
As many modules as you can add to be imported by adding more --hidden-import flags and module name
Flag description
--paths: The pyinstaller will search for imports here
--hidden-import: Which modules should be imported by pyinstaller from the path
I would like to set a custom stylesheet for my jupyter notebook that I run wtih sage -n jupyter.
This command loads the configuration stored in the .jupyter-Folder, but I can't find the ipython profile directory to put my css files in there.
Where is this folder located with the default sage installation?
I was looking for the iPython configuration before, but sage now uses jupyter now, so the custom css files should be located here.
~/.jupyter/custom/custom.css
and the js in here:
~/.jupyter/custom/custom.js
i got the answer from How do I set custom CSS for my IPython/IHaskell/Jupyter Notebook?
Find the path using sage -n jupyter --show-config.
sage#3ea4b9627bb1:~$ sage -n jupyter --show-config
Please wait while the Sage Jupyter Notebook server starts...
Loaded config files:
/home/sage/sage/local/var/lib/sage/venv-python3.9.9/etc/jupyter/jupyter_notebook_config.py
...
How to change working directory in Julia Jupyter?
I tried and read everything, still have no idea how to do that. It only allows me to select directories under my home ~/ dir. I can't find any button to go up to /.
I'm pretty sure once you have started the server you cannot then go up directories, I may be wrong though.
So best thing to do is start the jupyter notebook server somewhere that contains all of the folders you might need -i.e. the root dir if you want to make sure you have access to everything.
You can use the --notebook-dir flag for this. Or you can set defaults in the config.
you need to create the config file, using cmd :
jupyter notebook --generate -config
Then, search for C:\Users\your_username\.jupyter folder (Search for that folder), and right click edit the jupyter_notebook_config.py.
Then, Ctrl+F: #c.NotebookApp.notebook_dir ='' . Note that the quotes are single quotes. Select your directory you want to have as home for your jupyter, and copy it with Ctrl+C, for example: C:\Users\username\JuliaProjects.
Then on that line, paste it like this : c.NotebookApp.notebook_dir = 'C:\\Users\\username\\JuliaProjects'
Make sure to remove #, as it is as comment.
Make sure to double slash \\ on each name of your path.
Ctrl+S to save the config.py file !!!
Go back to your cmd and run jupyter notebook. It should be in your directory of choice. Test it by making a folder and watch your directory from your computer.
I use Jupyter Lab and start it from the Julia REPL (1.4) like this:
using IJulia
jupyterlab(dir=pwd(), detached=true)
Hi I'm using pyspark interactively. I think I'm failing loading a LOCAL file correctly.
how do I check current directory, so that I can go to browser to take a look at that actual file?
Or is the default directory where pyspark is? Thanks
You can't load local file unless you have same file in all workers under same path. For example if you want to read data.csv file in spark, copy this file to all workers under same path(say /tmp/data.csv). Now you can use sc.textFile("file:///tmp/data.csv") to create RDD.
Current working directory is the folder from where you have started pyspark. You can start pyspark using ipython and run pwd command to check working directory.
[Set PYSPARK_DRIVER_PYTHON=/path/to/ipython in spark-env.sh to use ipython]
import os
cwd = os.getcwd()
print(cwd)