I am aiming to create live documentation for a project and I decided to use notebooks.
So far I have created a number of notebooks (one per chapter) and now I want to merge them into one and update it every time I change the individual ones. I also want to create a table of contents.
Is that possible?
I managed to work arround this by creating individual html and pdf versions of my notebooks and then use a python prost-processor(for the html) and pandoc(for the pdf) to merge them. The extra steps allowed me to remove any cells where not necessary in the final html and pdf documents.
If anyone needs it I can post the code too.
Look at NBmerge. You can merge notebooks with nbmerge.
https://github.com/jbn/nbmerge
You must install it with pip. I just tried a conda install nbmerge but it failed because there's no conda package available at Continuum IO or conda-forge. However it is a small utility so installing using pip install should not hurt your conda environment.
pip install nbmerge
and merge your files using the command.
nbmerge file_1.ipynb file_2.ipynb file_3.ipynb > merged.ipynb
Related
I'm trying to use the SemiMarkov package and I want to change one small line of code in there. I've done some digging via:
getAnywhere("semiMarkov")
& I've identified that I want to change this line:
hessian <- diag(ginv(hessian(V, solution)))
to try something like:
hessian <- diag(ginv(pracma::hessian(V, solution)))
How do I go about this? Do I need to rebuild the package from scratch, and if so do I need rTools etc for this, or is there a simple-ish workaround (I'm a relevant R novice)? I've done some searching online and can't find anything obvious. Any ideas/pointers gratefully appreciated.
If you'd like to simply test out the effect of that change in an interactive R session, you can do so using trace(). Here's how:
Type trace("semiMarkov", edit=TRUE)
In the text editor that that launches, edit the line of interest.
Save the modified file.
Close the text editor
Back in R, use the modified function.
Linux environment
Starting with downloading the package source from CRAN.
This is the landing page: https://cran.r-project.org/web/packages/SemiMarkov/index.html
This is the package source: https://cran.r-project.org/src/contrib/SemiMarkov_1.4.2.tar.gz
Download and extract the source:
wget https://cran.r-project.org/src/contrib/SemiMarkov_1.4.2.tar.gz
tar -xvzf SemiMarkov_1.4.2.tar.gz
This should result in a directory named SemiMarkov. Open up the source (cd SemiMarkov), and modify as necessary.
Next, build the changes:
cd ..
R CMD build SemiMarkov/
This will result in a new archive file named SemiMarkov_1.4.2.tar.gz.
Lastly, install your modified archive:
R CMD INSTALL SemiMarkov_1.4.2.tar.gz
Windows environment
I'm less familiar with the Windows platform. *nix tooling is available in Cygwin, but it's painful. Instead, as Josh O'Brien points out, you should follow the Windows-specific instructions in the R Installation and Administration manual.
I would like to look at a list of all possible packages at the command line, from which I can choose which ones I would like to install.
This used to be possible using the meteor list command, but now that command only lists installed packages.
I've found several posts about best practice, reproducibility and workflow in R, for example:
How to increase longer term reproducibility of research (particularly using R and Sweave)
Complete substantive examples of reproducible research using R
One of the major preoccupations is ensuring portability of code, in the sense that moving it to a new machine (possibly running a different OS) is relatively straightforward and gives the same results.
Coming from a Python background, I'm used to the concept of a virtual environment. When coupled with a simple list of required packages, this goes some way to ensuring that the installed packages and libraries are available on any machine without too much fuss. Sure, it's no guarantee - different OSes have their own foibles and peculiarities - but it gets you 95% of the way there.
Does such a thing exist within R? Even if it's not as sophisticated. For example simply maintaining a plain text list of required packages and a script that will install any that are missing?
I'm about to start using R in earnest for the first time, probably in conjunction with Sweave, and would ideally like to start in the best way possible! Thanks for your thoughts.
I'm going to use the comment posted by #cboettig in order to resolve this question.
Packrat
Packrat is a dependency management system for R. Gives you three important advantages (all of them focused in your portability needs)
Isolated : Installing a new or updated package for one project won’t break your other projects, and vice versa. That’s because packrat gives each project its own private package library.
Portable: Easily transport your projects from one computer to another, even across different platforms. Packrat makes it easy to install the packages your project depends on.
Reproducible: Packrat records the exact package versions you depend on, and ensures those exact versions are the ones that get installed wherever you go.
What's next?
Walkthrough guide: http://rstudio.github.io/packrat/walkthrough.html
Most common commands: http://rstudio.github.io/packrat/commands.html
Using Packrat with RStudio: http://rstudio.github.io/packrat/rstudio.html
Limitations and caveats: http://rstudio.github.io/packrat/limitations.html
Update: Packrat has been soft-deprecated and is now superseded by renv, so you might want to check this package instead.
The Anaconda package manager conda supports creating R environments.
conda create -n r-environment r-essentials r-base
conda activate r-environment
I have had a great experience using conda to maintain different Python installations, both user specific and several versions for the same user. I have tested R with conda and the jupyter-notebook and it works great. At least for my needs, which includes RNA-sequencing analyses using the DEseq2 and related packages, as well as data.table and dplyr. There are many bioconductor packages available in conda via bioconda and according to the comments on this SO question, it seems like install.packages() might work as well.
It looks like there is another option from RStudio devs, renv. It's available on CRAN and supersedes Packrat.
In short, you use renv::init() to initialize your project library, and use renv::snapshot() / renv::restore() to save and load the state of your library.
I prefer this option to conda r-enviroments because here everything is stored in the file renv.lock, which can be committed to a Git repo and distributed to the team.
To add to this:
Note:
1. Have Anaconda installed already
2. Assumed your working directory is "C:"
To create desired environment -> "r_environment_name"
C:\>conda create -n "r_environment_name" r-essentials r-base
To see available environments
C:\>conda info --envs
.
..
...
To activate environment
C:\>conda activate "r_environment_name"
(r_environment_name) C:\>
Launch Jupyter Notebook and let the party begins
(r_environment_name) C:\> jupyter notebook
For a similar "requirements.txt", perhaps this link will help -> Is there something like requirements.txt for R?
Check out roveR, the R container management solution. For details, see https://www.slideshare.net/DavidKunFF/ownr-technical-introduction, in particular slide 12.
To install roveR, execute the following command in R:
install.packages("rover", repos = c("https://lair.functionalfinances.com/repos/shared", "https://lair.functionalfinances.com/repos/cran"))
To make full use of the power of roveR (including installing specific versions of packages for reproducibility), you will need access to a laiR - for CRAN, you can use our laiR instance at https://lair.ownr.io, for uploading your own packages and sharing them with your organization you will need a laiR license. You can contact us on the email address in the presentation linked above.
I'm trying to install the great table of contents extension in a new computer. but I cant find it anymore. the only page I do find does not explain how to install the extension on windows.
So.. How can I install it, and why is it not a part of the official Ipython notebook? I simply can't understand how people are getting along without it.
I've installed toc nbextension successfully with Jupyter 4 (ie. ipython notebook 4) recently.
In fact installing extension is easier than before :)
I post my solution here, may it help.
## download
mkdir toc
cd toc
wget https://raw.githubusercontent.com/minrk/ipython_extensions/master/nbextensions/toc.js
wget https://raw.githubusercontent.com/minrk/ipython_extensions/master/nbextensions/toc.css
## install and enable
cd ..
jupyter-nbextension install --user toc
jupyter-nbextension enable toc/toc
A bit more explain:
install will copy toc to ~/.local/share/jupyter/nbextensions/
enable will modify ~/.jupyter/nbconfig/notebook.json.
You can check these two place to see what happened.
Note: we use enable toc/toc here is because toc.js is in ~/.local/share/jupyter/nbextensions/toc/.
If you put toc.js and toc.css directly in ~/.local/share/jupyter/nbextensions/ then you should use enable toc here.
Edit
Sorry, I didn't notice the orginal problem is on windows. I'm not sure if it's same for windows jupyter, any report is welcome.
Update
Now the toc nbextension has been added into this project which provide a collection of kinds of nbextensions. It's very easy to install and manage, worth to try!
I cannot tell you specific Windows advice, but think the key points should be platform independent:
Create a profile (either a default profile or a named one - you'll probably want default to start).
Locate where the profile is.
Add the custom.js file into the profile.
Edit the custom.js file to point to the notebook extension code.
In a bit more detail, setting up a profile is covered in detail here but for a default profile just go to the command line and enter
ipython profile
Next, locate where your profile is stored by typing at the command line
ipython locate
Call that <profile_dir>.
The rest follows the (Windows equivalent of!) the instructions on the link you have: underneath <profile_dir> navigate to (creating any directories that do not already exist)
<profile_dir>/static/custom/
and add the custom.js file as shown. Then edit the first line, where it has "nbextensions/toc" to point to the location where you have placed the toc.js file you have downloaded. This location is relative to the <profile_dir>; for me I have
<profile_dir>/static/custom/custom.js
<profile_dir>/static/custom/nbextensions/toc.js
<profile_dir>/static/custom/nbextensions/toc.css
and the first line of custom.js reads
require(["/static/custom/nbextensions/toc.js"], function (toc) {
Finally, note that this is with version 1.1.0 of the notebook - if you're using an earlier version I strongly suggest you upgrade before trying this.
You'll also find the official installation instructions at:
https://github.com/minrk/ipython_extensions
These instructions include curl commands for retrieving the toc.js and toc.css files from GitHub, which worked fine for me in a bash shell on linux Mint.
For Windows 7, I used a Git Shell (see http://msysgit.github.io/) to execute the curl commands
This IPython Notebook semi-automatically generates the files for minrk's table of contents in Windows. It does not use the 'curl'-commands or links, but writes the *.js and *.css files directly into your IPython Notebook-profile-directory.
There is a section in the notebook called 'What you need to do' - follow it and have a nice floating table of contents : )
Here is an html version which already shows it:
http://htmlpreview.github.io/?https://github.com/ahambi/140824-TOC/blob/master/A%20floating%20table%20of%20contents.htm
I have an IPython notebook and I would like to use one of my variables inside a markup cell. Is this even possible? If so, how do you do it?
If you don't mind a code cell that does the job, there is a possibility without adding any extensions.
from IPython.display import Markdown as md
fr=2 #GHz
md("$f_r = %i$ GHz"%(fr))
This will show a markdown cell in a nicely LaTeX formatted output
Currently, this is not possible, however there is a large discussion on this topic here https://github.com/ipython/ipython/pull/2592. The PR is currently closed, but a corresponding issue is opened https://github.com/ipython/ipython/issues/2958 and marked as wishlist.
Update
In the meantime an IPython extension has appeared which allows to render python variables in markdown cells. This extension is part of the IPython notebook extensions and works with IPython 2.x and 3.x. For a detailed description see the wiki page.
It is not officially supported, but installing the python markdown extension will allow you to do so. It is part of the nbextensions, for which you will find installation instructions on their github page. Make sure you'll enable the python markdown extension using a jupyter command or the extension configurator.
Calling python variables then should work with the {{var-name}} syntax, which is described in the readme of the corresponding github page (linked in the wiki):
For example: If you set variable a in Python
a = 1.23
and write the following line in a markdown cell:
a is {{a}}
It will be displayed as:
a is 1.23
Further info on this functionality being integrated into ipython/jupyter is discussed in the issue trackers for ipython and jupyter.
The link: installing notebook extention
gives a clear description of what is necessary to enable the use of variables in markdown cells. Following it, performed the following actions to realize it:
conda install -c conda-forge jupyter_contrib_nbextensions
jupyter contrib nbextension install --user
after a successful completion of the above command I enabled the python markup extension, from jupyter dashboard, as per the following illustration:
Last but not least!!! The NOTEBOOK HAS TO BE TRUSTED to make the markup extension works with python variables
and it worked for me!