When I am removing files from Jupyter notebook environment, the disk space does not free up. I removed for about 40GB files and files disappeared from list, even from ls -a, however df -h shows that nothing happened. Also I killed all the processes using these files and even rebooted the system.
When I remove files using rm everything is fine. How can I free up space, or restore thos files to delete them using rm?
I also encountered the same issue, but later found out that files deleted in jupyter notebook are automatically moved to the trash, which is not permanently deleted instantly.
This function was added very early ago: https://github.com/jupyter/notebook/pull/1968
Thus, to free up space, you need to go to your computer's trash folder to completely delete those files. In Linux, you can do rm -rf $HOME/.local/share/Trash/files to empty trash.
In Windows or MacOS, you just need to "Empty Trash" in the desktop folder
To restore those files, try to access them in your Trash folder, which is located in the .local folder in your home directory (in my system.).
This worked for me. I'm using jupyter lab with Amazon Linux 2 AMI.
Ref.
recycle bin in linux:
https://www.pcquest.com/recycle-bin-linux/#:~:text=Henceforth%20whenever%20you%20delete%20a,SYSTEM_ROOT%20of%20the%20Trash%20directory.
I freed up the space and solved the issue when I was working with workbench(Google Cloud Vertex AI). Applicable to AI platform too. Open terminal from workbench File>New>Terminal
df -h -T /home/jupyter
Above code helps to know the free size
Then this code below will delete the trash to free up the size
sudo rm -rf /home/jupyter/.local/share/Trash/*
After deleting the trash
Related
I have almost 20 GiB on the trash in my external disk and I want to remove it, I've tried manually and with the sudo privileges with sudo rm -rf /path/to/folder/.Trash-1000 but gets the error cannot remove '/run/media/d7/PC-User/.Trash-1000/files/Emilia.md': Input/output error.
I'm on Arch Linux and this happened I think after using FreeFileSync because I have some problems with other files too.
Any command that removes or clear the trash directory?
No problem in other directories. Is there an environmental variable or something else I need to erase?
Deleted cache file...
OK, I think I need be much clearer here.
First software:
MacOS Catalina 10.15.6
jupyter notebook 6.0.3
Python 3.8.3
IPython 7.16.1
jupyter notebook is installed and runs fine.
jupyter notebook runs just fine in any user directory on the computer except exactly one.
There is nothing obvious in this directory that shouldn't be there. An 'ls -al' shows nothing but some .py files.
I can create a jupyter notebook in this directory, but the kernel crashes and won't restart. I can rename the directory, rename the jupyter notebook, but the behavior persists beyond everything I have been able to reset including a cold computer restart. It is reproducible and happens every time.
This behavior is not seen in any other directory.
My question: are there environmental variables or caches stored not visibly in the directory (obviously) that are responsible for this incredibly annoying behavior and how can I reset them?
Problem solved: jupyter notebooks apparently uses some reserved names for local directory .py files when starting up the notebook. So far I've found that "string.py" and "decorator.py" cannot be in the startup directory unless they contain the expected data (looks like it needs to be related to some template info)
To start-up a kernel
You first activate your virtual environment:
For instance: conda activate vision
Second, you type jupyter notebook
as stated here
I have a small hugo site that I am using as front matter to surveys. I would like to gather the workflow entirely in Rstudio. At present I edit the templates and content pages in a text editor and then use rsync to push the updates to apache webserver (both locally on the dev machine and via SSH). My question is - can I edit my website through Rstudio / blogdown and use rsync from within Rstudio?
Below is example of rsync that I use on server:
$ hugo && sudo rsync --chown=http:http -avz --delete --exclude lime public/ /srv/http/
For me it is not a problem to kep going like this, but I am very interested in keeping things simple/automated and in one place as much as possible. I will need to teach other people to keep updating the site and while they can press knit in rstudio, if I tell them to open a terminal, there may be a problem.
Every time I started RStudio, I have seen this my working environment.
I can use rm(list=ls()) to remove them temporarily, but every time I restarted RStudio, they showed up again.
I use getwd() to see my working directory, but in the working directory, I did not see any .Rdata file. How can I get rid of these things ?
Your kind help will be well regarded.
I use Mac OS 10.10.
Click on RStudio in the menu bar and go to Preferences.
In the R General section, unclick the Restore .RData into workspace at startup option.
The default is to reload the working environment when you restart RStudio.
I think that you, at some point, chose to save your environment to your working directory (most likely ~, i.e. your home directory, which is the default RStudio working directory).
The easier way to clear your default environment is to remove the .RData file from your home directory. It will not appear in any Finder window, because in a Unix-like OS (like OS X), files starting with . are hidden. So do the following:
Open a terminal window
If not already there, go to your home folder: cd ~
Check if there's an .RData file: ls -lA .RData
If the file exists delete it: rm .RData (if you want, create a backup: `cp .RData ./RData_backup)
Intermittently, I'll run into an issue where my rsync script will simply freeze mid transfer. This freeze may occur while downloading a file, or amidst listing uptodate files.
I'm running this on my mac, here's the code below:
rsync -vvhrtplHP -e "ssh" --rsync-path="sudo rsync" --filter=". $FILTER" --delete --delete-excluded --log-file="$BACKUP/log" --link-dest="$BACKUP/current/" $CONNECT:$BASE $BACKUP/$DATE/
For example, the console will output the download progress of a file, and stop at an arbitrary percentage and speed. The log doesn't even list the file (probably because it's incomplete).
I'll try numerous attempts and it'll freeze on different files or steps with no rhyme or reason. Terminal will show the loading icon while it's working, the output will freeze, and after a few seconds the loading icon vanishes.
Any ideas what could be causing this? I'm using rsync 3.1.0 on Mavericks. Could it be a connectivity issue or a system max execution time issue?
I have had rsync freezes in the past and I recall reading somewhere that it may have to do with rsync having to look for files to link, something increasingly difficult as you accumulate backup over backup. I suggest you skip the --link-dest in the next backup if your disk space allows it (to break the chain, so to speak).
As mentioned in https://serverfault.com/a/207693 you could use the hardlink command afterwards, I haven't tried it yet.
Just had a similar problem while doing rsync from harddisk to a fat32 usb. rsync froze already in less than a second in my case and did not react at all after that.
Found out that the problem was a combination of usage of hardlinks on the harddisk and having fat32 filesystem on the usb drive, which does not support hardlinks.
Formatting the usb drive with ext4 solved the problem for me.