Background
I have preprocessed over 4.5K jupyter notebooks.
I would like to use jupyter-book to build a book from the processed notebooks.
I have execute_notebooks: "off" in my jupyter-book configs.
However, when I run jupyter-book build on the notebooks, I run into an error with the RAM required: the RAM required keeps on increasing.
Questions
QS1. Is there something that I should be doing to avoid this RAM increase? Maybe something in my config/toc ymls?
QS2. Alternatively, is there any way I can estimate how much time/mem will be needed to build the book before running `jupyter-book build?
Extra
Below is a graph of the mem usage in 1hr of processing (when logs are at writing output... [ 3%] content/toi_notebooks/toi_2175)
_toc.yml:
format: jb-book
root: index
parts:
- chapters:
- file: content/example_loader
- file: citation
- file: content/toi_fits
sections:
- glob: content/toi_notebooks/toi_*
_config.yml
# in _config.yml
title: "TESS Atlas"
author: the TESS Atlas community
logo: _static/atlas_logo.png
execute:
execute_notebooks: "off"
# Define the name of the latex output file for PDF builds
latex:
latex_documents:
targetname: book.tex
html:
use_edit_page_button: false
use_repository_button: true
use_issues_button: true
# Information about where the book exists on the web
repository:
url: https://github.com/dfm/tess-atlas
branch: main
gitissue: https://github.com/executablebooks/jupyter-book/issues/1571
Related
I'm running a command to (re)create a remote file. I'd like to see a difference between the old and the new versions of the file -- and for the task to set changed: false, if there are no differences.
Is there a way to do this without doing it all by hand -- with multiple tasks (creating a backup, running the command, diff-ing the two, etc.)?
Ok, apparently, this is not currently possible. The best I could come up with was:
Run the command with output sent simply to stdout, rather than directly to a file. This is possible with many commands (such as with -o /dev/stdout), allows to declare it with changed_when: False and check_mode: ansible_check_mode. Save the results:
- name: Run the command to obtain content
command: foo -
register: foo
changed_when: False
check_mode: ansible_check_mode
...
Use the copy module to (re)write the file with the captured output:
- name: Write the content into file
copy:
dest: /where/ever/it/was
content: '{{ foo.stdout }}'
...
This is Ok for when your files aren't too large -- requiring "only" one additional task... One per file.
For everything else, I created this enhancement-request.
I'm trying to create snap package of a Qt/QML application, the application is packaged well, when I try to run it I get /snap/swipe-app/x2/bin/qt5-launch: 74: exec: application: not found error.
here's my snapcraft.yaml file
name: swipe-app # you probably want to 'snapcraft register <name>'
version: '0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: description
grade: devel # must be 'stable' to release into candidate/stable channels
confinement: strict # use 'strict' once you have the right plugs and slots
apps:
swipe-app:
command: qt5-launch application
plugs:
- unity7
- home
parts:
application:
# See 'snapcraft plugins'
plugin: qmake
project-files: ["snap.pro"]
source: .
build-packages:
- qtbase5-dev
stage-packages:
# Here for the plugins-- they're not linked in automatically.
- libqt5gui5
after: [qt5conf] # A wiki part
As you have told the launch script that your program is called application then it will try to execute application from the current working directory when you run your snap. There are two things to note here:
The working directory is preserved from the terminal outside the snap context. For example if you are in your home directory /home/your-user then the working directory for swipe-app will also be /home/your-user.
As the working directory above is your home directory then commands without any anchor, such as application, will try to execute in your home directory. So in your example the launch script will attempt to run the command equivalent of /home/your-user/application.
You can fix this by either ensuring that the launch script executes a cd to change the working directory, e.g. cd $SNAP; or anchor your command by adding an achor, e.g. command: qt5-launch $SNAP/application.
Another thing you might need to check is that your qmake build actually outputs a binary called application. If you have not set TARGET= in your snap.pro project file then the binary will default to being called snap, not application. The line should read TARGET=application to make a binary called application: (ref: https://doc.qt.io/qt-5/qmake-variable-reference.html#target).
Dev Environment: R 3.3.2 / R Studio 0.99.1266 / Windows 7
Web server: Kubuntu ; Apache/2.14.18
Browser: Firefox 52.0.1
Context :
I have a simple set of web pages I'm creating in R Studio for a short course series. I develop this on a Windows machine, then copy the _site folder to the server. It is the first time I've done web pages in R Studio, but also my server was recently replaced.
Consequently, I don't know if this is an RStudio issue or something to my Apache configuration. And I don't know how to diagnose where the problem described below lies.
Problem
It looks fine on my Windows machine with Firefox as a file:// URL
also looks fine when I open the site on the server,
but as a file:// URL (I have a samba share to the server that lets me see this as another drive.)
similarly, looks OK if I open the site using the command line tool, xdg-open index.html on the server machine
BUT, when I open this in Firefox with the server http:// URL on either the Windows machine or on the server machine, none of the style or other attributes are rendered correctly.
To illustrate: this is what I see opening the site from my Windows machine, with a file:// URL
The server URL is Introduction to R Graphics
which looks like this:
The _site.yml file is straight-forward:
name: "RGraphics"
exclude: ["notes/", "sources/", "*.pptx", "*.bak"]
navbar:
title: "An Introduction to R Graphics"
type: inverse
left:
- text: "Home"
icon: fa-home fa-2x # handy way to use FontAwesome icons in nav!
href: index.html
- text: "Abstract"
icon: fa-file fa-2x
href: abstract.html
- text: "Resources"
icon: fa-book fa-2x
href: resources.html
And so are the various .Rmd files, e.g., index.Rmd:
---
title: "An Introduction to R Graphics"
author: "Michael Friendly, SCS Short Course"
date: "March, 2017"
---
## Session 1: Overview
- Lecture notes: [1up PDF](R-Graphics1.pdf); [4up PDF](R-Graphics1-2x2.pdf)
#### Topics:
+ Getting started: R, R Studio
+ The roles of graphics in data analysis (exploration, analysis, presentation)
+ What can I do with R graphics: Anything!
...
The diagnosis of the source of the problem came from the 403 (forbidden) errors noted above. But the cause of the problem was that folder permissions for site_libs/ weren’t set correctly. That folder, and all the subfolders containing the javascript libraries needed to be set to 755 (drwxr-xr-x), i.e., allowing read and execute permissions to folders for all users.
Why this happened: the properties of the site_libs/ files generated by R Studio on Windows don't map well to the Unix permissions required for the web server.
Cure: On the server, cd to the top level of this web tree. Then, use something like
find . -type d -exec chmod 755 {} \;
I'm trying to use Salt to deploy an online tool to a new VPS. The process involves cloning a git repo and then various set-up commands - however there seems to be an issue with including other .sls files from within sub directories.
Here's a simplified version:
Master config file:
file_roots:
base:
- /srv/salt/saltstates
I have a a file in /srv/salt/saltstates/test/test.sls containing:
base:
'*':
- basic
The file /srv/salt/saltstates/test/basic.sls contains:
Europe/London:
timezone.system
However, when I run salt 'Minion1' state.sls test.test, an error is returned:
Minion1:
----------
ID: base
Function: *.basic
Result: False
Comment: State *.basic found in sls test.test is unavailable
Started:
Duration:
Changes:
OK, so you've confused several things here.
First of all the contents you've put in /srv/salt/saltstates/test/test.sls really is what is called a top file and should probably be moved to /srv/salt/saltstates/top.sls
The top.sls is only needed if you want to do a highstate, but since you're trying to run salt 'Minion1' state.sls test.test you don't really need the top.sls.
Now since you have your sls file here: /srv/salt/saltstates/test/basic.sls, then the command you want to run is the following:
salt 'Minion1' state.sls test.basic
The "dot" traverses down directories.
Can someone point me towards a working example where packrat is used with AppVeyor to build an R package? Searching through Google and GitHub, I can't find any packrat-enable package that uses AppVeyor.
Does the appveyor.yml file need to change? Are there some settings I need to add through the AppVeyor website?
I have a very minimal package (testthat is the only dependency) that broke AppVeyor builds. Here is the code frozen for that commit. Here is the AppVeyor log.
(If this SO question sounds familiar, I'm about to ask a similar question for Travis-CI.)
Yes, the solution here is similar to the same question for Travis-CI.
Here's an example of an appveyor.yml file that will enable you to use packrat packages in your package:
# DO NOT CHANGE the "init" and "install" sections below
# Download script file from GitHub
init:
ps: |
$ErrorActionPreference = "Stop"
Invoke-WebRequest http://raw.github.com/krlmlr/r-appveyor/master/scripts/appveyor-tool.ps1 -OutFile "..\appveyor-tool.ps1"
Import-Module '..\appveyor-tool.ps1'
install:
ps: Bootstrap
# Adapt as necessary starting from here
environment:
global:
WARNINGS_ARE_ERRORS: 0
USE_RTOOLS: true
build_script:
- R -e "0" --args --bootstrap-packrat
test_script:
- travis-tool.sh run_tests
on_failure:
- 7z a failure.zip *.Rcheck\*
- appveyor PushArtifact failure.zip
artifacts:
- path: '*.Rcheck\**\*.log'
name: Logs
- path: '*.Rcheck\**\*.out'
name: Logs
- path: '*.Rcheck\**\*.fail'
name: Logs
- path: '*.Rcheck\**\*.Rout'
name: Logs
- path: '\*_*.tar.gz'
name: Bits
- path: '\*_*.zip'
name: Bits
The important parts that differ from the template are:
environment:
global:
WARNINGS_ARE_ERRORS: 0
USE_RTOOLS: true
build_script:
- R -e "0" --args --bootstrap-packrat
This enables Rtools for the build, and loads the packrat packages.
It's also important to note that we are excluding - travis-tool.sh install_deps because that would cause the packages you depend on to be downloaded from CRAN, rather than built from your packrat directory.
Here's an example of an appveyor build for a simple R package where this is working: https://ci.appveyor.com/project/benmarwick/javaonappveyortest/build/1.0.21