In Unix, it's possible to create a handle to an anonymous file by, e.g., creating and opening it with creat() and then removing the directory link with unlink() - leaving you with a file with an inode and storage but no possible way to re-open it. Such files are often used as temp files (and typically this is what tmpfile() returns to you).
My question: is there any way to re-attach a file like this back into the directory structure? If you could do this it means that you could e.g. implement file writes so that the file appears atomically and fully formed. This appeals to my compulsive neatness. ;)
When poking through the relevant system call functions I expected to find a version of link() called flink() (compare with chmod()/fchmod()) but, at least on Linux this doesn't exist.
Bonus points for telling me how to create the anonymous file without briefly exposing a filename in the disk's directory structure.
A patch for a proposed Linux flink() system call was submitted several years ago, but when Linus stated "there is no way in HELL we can do this securely without major other incursions", that pretty much ended the debate on whether to add this.
Update: As of Linux 3.11, it is now possible to create a file with no directory entry using open() with the new O_TMPFILE flag, and link it into the filesystem once it is fully formed using linkat() on /proc/self/fd/fd with the AT_SYMLINK_FOLLOW flag.
The following example is provided on the open() manual page:
char path[PATH_MAX];
fd = open("/path/to/dir", O_TMPFILE | O_RDWR, S_IRUSR | S_IWUSR);
/* File I/O on 'fd'... */
snprintf(path, PATH_MAX, "/proc/self/fd/%d", fd);
linkat(AT_FDCWD, path, AT_FDCWD, "/path/for/file", AT_SYMLINK_FOLLOW);
Note that linkat() will not allow open files to be re-attached after the last link is removed with unlink().
My question: is there any way to re-attach a file like this back into the directory structure? If you could do this it means that you could e.g. implement file writes so that the file appears atomically and fully formed. This appeals to the my compulsive neatness. ;)
If this is your only goal, you can achieve this in a much simpler and more widely used manner. If you are outputting to a.dat:
Open a.dat.part for write.
Write your data.
Rename a.dat.part to a.dat.
I can understand wanting to be neat, but unlinking a file and relinking it just to be "neat" is kind of silly.
This question on serverfault seems to indicate that this kind of re-linking is unsafe and not supported.
Thanks to #mark4o posting about linkat(2), see his answer for details.
I wanted to give it a try to see what actually happened when trying to actually link an anonymous file back into the filesystem it is stored on. (often /tmp, e.g. for video data that firefox is playing).
As of Linux 3.16, there still appears to be no way to undelete a deleted file that's still held open. Neither AT_SYMLINK_FOLLOW nor AT_EMPTY_PATH for linkat(2) do the trick for deleted files that used to have a name, even as root.
The only alternative is tail -c +1 -f /proc/19044/fd/1 > data.recov, which makes a separate copy, and you have to kill it manually when it's done.
Here's the perl wrapper I cooked up for testing. Use strace -eopen,linkat linkat.pl - </proc/.../fd/123 newname to verify that your system still can't undelete open files. (Same applies even with sudo). Obviously you should read code you find on the Internet before running it, or use a sandboxed account.
#!/usr/bin/perl -w
# 2015 Peter Cordes <peter#cordes.ca>
# public domain. If it breaks, you get to keep both pieces. Share and enjoy
# Linux-only linkat(2) wrapper (opens "." to get a directory FD for relative paths)
if ($#ARGV != 1) {
print "wrong number of args. Usage:\n";
print "linkat old new \t# will use AT_SYMLINK_FOLLOW\n";
print "linkat - <old new\t# to use the AT_EMPTY_PATH flag (requires root, and still doesn't re-link arbitrary files)\n";
exit(1);
}
# use POSIX qw(linkat AT_EMPTY_PATH AT_SYMLINK_FOLLOW); #nope, not even POSIX linkat is there
require 'syscall.ph';
use Errno;
# /usr/include/linux/fcntl.h
# #define AT_SYMLINK_NOFOLLOW 0x100 /* Do not follow symbolic links. */
# #define AT_SYMLINK_FOLLOW 0x400 /* Follow symbolic links. */
# #define AT_EMPTY_PATH 0x1000 /* Allow empty relative pathname */
unless (defined &AT_SYMLINK_NOFOLLOW) { sub AT_SYMLINK_NOFOLLOW() { 0x0100 } }
unless (defined &AT_SYMLINK_FOLLOW ) { sub AT_SYMLINK_FOLLOW () { 0x0400 } }
unless (defined &AT_EMPTY_PATH ) { sub AT_EMPTY_PATH () { 0x1000 } }
sub my_linkat ($$$$$) {
# tmp copies: perl doesn't know that the string args won't be modified.
my ($oldp, $newp, $flags) = ($_[1], $_[3], $_[4]);
return !syscall(&SYS_linkat, fileno($_[0]), $oldp, fileno($_[2]), $newp, $flags);
}
sub linkat_dotpaths ($$$) {
open(DOTFD, ".") or die "open . $!";
my $ret = my_linkat(DOTFD, $_[0], DOTFD, $_[1], $_[2]);
close DOTFD;
return $ret;
}
sub link_stdin ($) {
my ($newp, ) = #_;
open(DOTFD, ".") or die "open . $!";
my $ret = my_linkat(0, "", DOTFD, $newp, &AT_EMPTY_PATH);
close DOTFD;
return $ret;
}
sub linkat_follow_dotpaths ($$) {
return linkat_dotpaths($_[0], $_[1], &AT_SYMLINK_FOLLOW);
}
## main
my $oldp = $ARGV[0];
my $newp = $ARGV[1];
# link($oldp, $newp) or die "$!";
# my_linkat(fileno(DIRFD), $oldp, fileno(DIRFD), $newp, AT_SYMLINK_FOLLOW) or die "$!";
if ($oldp eq '-') {
print "linking stdin to '$newp'. You will get ENOENT without root (or CAP_DAC_READ_SEARCH). Even then doesn't work when links=0\n";
$ret = link_stdin( $newp );
} else {
$ret = linkat_follow_dotpaths($oldp, $newp);
}
# either way, you still can't re-link deleted files (tested Linux 3.16 and 4.2).
# print STDERR
die "error: linkat: $!.\n" . ($!{ENOENT} ? "ENOENT is the error you get when trying to re-link a deleted file\n" : '') unless $ret;
# if you want to see exactly what happened, run
# strace -eopen,linkat linkat.pl
Clearly, this is possible -- fsck does it, for example. However, fsck does it with major localized file system mojo and will clearly not be portable, nor executable as an unprivileged user. It's similar to the debugfs comment above.
Writing that flink(2) call would be an interesting exercise. As ijw points out, it would offer some advantages over current practice of temporary file renaming (rename, note, is guaranteed atomic).
Kind of late to the game but I just found http://computer-forensics.sans.org/blog/2009/01/27/recovering-open-but-unlinked-file-data which may answer the question. I haven't tested it, though, so YMMV. It looks sound.
I find that developing functions in IPython notebook allows me to work quickly. When I'm happy with the results I copy-paste to a file. The autoindent is 4 spaces, but the coding style for indentation at my company is 2 spaces. How do I change the autoindent to 2 spaces?
The official documentation has an example answering this specific question. This worked for me with IPython 4.
Summary: Paste the following into your browser's javascript console
var cell = Jupyter.notebook.get_selected_cell();
var config = cell.config;
var patch = {
CodeCell:{
cm_config:{indentUnit:2}
}
}
config.update(patch)
The setting is persisted. You can roll back by exchanging : 2 for : null.
From the official documentation for CodeMirror Code Cells:
Open an Ipython Notebook
Create a Code Cell e.g. by pressing b
Open your browser’s JavaScript console and run the following
snippet:
var cell = Jupyter.notebook.get_selected_cell();
var config = cell.config;
var patch = {
CodeCell:{
cm_config:{indentUnit:2}
}
}
config.update(patch)
Reload the notebook page in the browser e.g. by pressing F5
This will fix it permanently. I assume this works only on recent versions, not sure though!
AdamAL's answer is correct. It worked for me.
However it only changes the indentation in the Jupyter Notebook and leaves the indentation in the Jupyter Editor unaffected.
A more direct way to change the indentation is to directly edit the Jupyter config files in the .jupyter/nbconfig directory. This directory contains 2 files:
edit.json
notebook.json
The option you must set in either one is indentUnit. Here is the content of my Jupyter config files:
edit.json:
{
"Editor": {
"codemirror_options": {
"indentUnit": 2,
"vimMode": false,
"keyMap": "default"
}
}
}
notebook.json:
{
"CodeCell": {
"cm_config": {
"indentUnit": 2
}
}
}
With this approach I've set the default indentation to 2 in both the Jupyter Notebook and the Jupyter Editor.
Based on this question and the options found here:
In your custom.js file (location depends on your OS) put
IPython.Cell.options_default.cm_config.indentUnit = 2;
On my machine the file is located in ~/.ipython/profile_default/static/custom
Update:
In IPython 3 the plain call does not work any more, thus it is required to place the setting within an appropriate event handler. A possible solution could look like
define([
'base/js/namespace',
'base/js/events'
],
function(IPython, events) {
events.on("app_initialized.NotebookApp",
function () {
IPython.Cell.options_default.cm_config.indentUnit = 2;
}
);
}
);
If you use jupyterlab, there seems to be an easier way:
1) Click jupyterlab menu Settings > Advanced Setting Editor
2) Click "Notebook" on the left hand pane, make sure you are on "Raw View"
3) On the right pane, under "User Overrides", enter this:
{
"codeCellConfig": {
"tabSize": 2
}
}
If you look at the System Defaults, that will give you hint on whats overridable and you can repeat this for other settings.
I tried this on Google Platform AI Notebook which uses Jupyterlab.
I believe this is now best wrapped in a event handler to load once per notebook load:
$([IPython.events]).on('app_initialized.NotebookApp', function(){
IPython.CodeCell.options_default['cm_config']['indentUnit'] = 2;
});
In addition to adding
IPython.Cell.options_default.cm_config.indentUnit = 2;
to your custom.js file as suggested by Jakob, be sure to clear your browser cache as well before restarting ipython notebook!
Also, you may have to first create the ~/.config/ipython/profile_default/static/custom/ directory (use echo $(ipython locate default) to find your default directory) before adding the custom.js file.
I got a white screen when i try to write the $view variable to the output per var_dump, per devel module and so on. If i run a test with dvm('test'); everything works as expected.
function feeds_node_processor_global_views_post_execute(&$view) {
dvm($view->name);
if($view->name == 'liveticker_start2'){
//dvm($view->total_rows);
//dvm("test");
//var_dump($view);
}
if ($view->total_rows > 100) {
drupal_set_message(t('You have more than 100 hits.'));
}
}
In watchdog aren't any errors and i don't have access to the apache log (at the moment).
Are there any other clues?
Thanks.
See http://drupal.org/node/158043 about the White Screen of Death.
I've run into a similar issue in the past and it's because I've ran out of php memory trying to output the $view object, I think. Your server's logs may help verify this if you could access it but since you can't, try increasing your PHP's memory_limit if you can.
I have the following less files:
WebApp/Content/less/main.less
#import "src/test.less";
WebApp/Content/less/src/test.less:
body {
background-image: url("../img/abc.png");
}
When I run less 1.3.0 via less.js-windows
> less.js-windows\lessc.cmd WebApp\Content\less WebApp\Content\css -compress
... I get this:
body{background-image:url("src/src/../img/abc.png");}
The "src/src/.." doesn't look correct to me.
I spent about an hour on it but I can't determine the status of this problem from the less project from its github page. There are numerous issues, regressions, and applied and unapplied pull requests which appear to address it: https://github.com/cloudhead/less.js/issues/search?q=relative.
I can't use absolute paths in this case. Is there a simple fix or a workaround for this issue?
Do the following:
body
{
background-image : ~"url( '../img/abc.png' )";
}
I have the following code in my .Zshrc
function google; {
$VIEW "http://www.google.com/search?q='url-encode "${(j: :)#}"'"
}
I get
google masi
google:1: no such file or directory: http://www.google.com/search?q='url-encode masi'
How can you get Google Search to work in Zsh?
The following solves the problem in Mac
function google; {
open "http://www.google.com/search?q='url-encode "${(j: :)#}"'"
}
and in Ubuntu
function google; {
gnome-open "http://www.google.com/search?q='url-encode "${(j: :)#}"'"
}
Don't know anything about zsh but looks like you've got a problem with your quotes.
Looks like it evaluates the url to being
http://www.google.com/search?q='url-encode masi'
Which probably isn't what you were after. (url encoded version of 'masi') ?
What's the value of $VIEW?
Set it to the path of a web browser or downloader.