Effect of file creation/opening on st_mtime and st_atime - unix

When I create or open a file in UNIX using O_CREAT flag, st_mtime,st_ctime and st_atime of the file changes. But when I create or open a file using O_TRUNC flag, only the st_mtime and st_ctime changes and not the st_atime.
From my understanding, st_atime changes when the file is accessed. When we open or create a file using O_TRUNC flag, are we not accessing the file?

This question is a bit old, but an answer for future generations at least...
From stat(2) man page (on a host with a linux 2.6.32 kernel):
The field st_atime is changed by file accesses, for example, by execve(2), mknod(2), pipe(2),
utime(2) and read(2) (of more than zero bytes). Other routines, like mmap(2), may or may not
update st_atime.
The field st_mtime is changed by file modifications, for example, by mknod(2), truncate(2),
utime(2) and write(2) (of more than zero bytes). Moreover, st_mtime of a directory is changed by
the creation or deletion of files in that directory. The st_mtime field is not changed for changes
in owner, group, hard link count, or mode.
The field st_ctime is changed by writing or by setting inode information (i.e., owner, group, link
count, mode, etc.).

Related

Unable to delete uploaded XML file on Drupal 8.5

I have created a media type that accepts XML files and saves them to a custom publicly accessible location on the server.
Ideally I would like the file to be overwritten when the exact same file is uploaded. This does not happen, instead it creates a new file and adds a number on the end. I have "Create new Revision" turned off.
To get around this issue I thought I could just delete the file via the CMS. The uploaded file has status of "Permanent" and is used 0 places. I know the cron job cleans up files for you, but when I run the cron the file in question is still there. I figure it's because the file is set to permanent, but I don't see a way to flip this to temporary.
Any help is much appeciated.
There is a setting nested away in the file system settings, which lets you configure it to remove (or not removed) orphaned files. If drush isn't removing them despite having no usages recorded, I'd check this option isn't ticked.
The temporary and permanent status are used for storing temporary files during the upload/save process, so I wouldn't tinker with those too much.
If you fancy making the form yourself using the form API, then you can save the file programmatically using the FILE_EXISTS_REPLACE parameter.
https://api.drupal.org/api/drupal/core%21modules%21file%21file.module/function/file_save_data/8.5.x

rsync with append smaller to longer file part that existing on receiver

I have some process that creates text log files, and some backup server.
I want to move text log files to backup server (with remove source file). And on next rsync start if there is log file with the same name - append/concatenate it with previously moved file.
rsync -av --append --remove-source-files user#server1:/source/ user#server2:/destination/
And everything works great except - if there is file with the same name on source and destination, and destination/receiver have longer file than on source server - it's skipped. How I may disable this --append's file length verification? I want simply append files with the same name with no restrictions.
The append option doesn't work like that.
From the man page:
This causes rsync to update a file by appending data onto the end of
the file, which presumes that the data that already exists on the
receiving side is identical with the start of the file on the sending
side.
So, if you "append" a 2KB file to a 1KB file, the end file with be 2KB, and the first 1KB of the larger file will be discarded. Conversely, if you "append" a file of equal or smaller size, then the whole file will end up getting discarded.
By passing the --append option you're basically promising rsync that you know the start of the file will never change.
But, if you delete the file then the start of the file has changed. Likewise, logrotate will make bad things happen.
Conclusion: --append does not play nice with --remove-source-files.
The --append-verify option will solve the logrotate problem, but won't solve your problem because it will still discard data (albeit different data).

ZFS folder duplication and deletion

I had a folder which was encrypted by encfs. It was in an ext4 partition. I decided to move it into another folder, because I had not enough space in that partition. The new partition was a compression-enabled zfs partition on another hard disk. During the move along with the destination folder 'td', another folder '.shutdowntd' was also created. I don't know why. I didn't created it. Maybe zfs itself created it. Maybe encfs manager did it. Last night I looked into it. I saw that the number of files in both directories is the same. I saw that for any file in the directory 'td' there is a file in directory '.shutdowntd'. File sizes were not exactly the same, but were nearly the same. When I deleted a file in one of them, the corresponding file in the other directory was also deleted! The names in the directory '.shutdowntd' were different and seemed to be hash-coded. My computer was on during the night. Today, I saw that the folder 'td' was removed and a zero-byte file with the same name was created! I restarted Ubuntu (16.04). Now I see that the file is changed back to a folder with the same name ('td') with no content. But the folder '.shutdowntd' still exists with the files in it. I can't justify this behavior and don't know why it happened. Essentially why '.shutdowntd' is created? Why 'td's' content is emptied?! How can I recover it? What's happening?!

Deleting files and Inodes

I'm reading a text on Version 6 unix, and just learned about inodes. I have the following question:
Suppose I have a file in one directory and a link to the file somewhere else. Am I correct to say that, if I delete the file, the inode will still exist because the refcount is not 0? And does this mean that the file isn't really deleted while the link exists, and I can access the file through the inode number?
Yes, if the link is a hard link. No, if it's just a symbolic link.
A hard link is basically the same file being in more than one directory, with the same inode. Unlinking the file from one of its directories just reduces its reference count by one. It won't be deleted until it reaches zero.
A symbolic link has its own inode and redirects you to the other entry. A symbolic link will dangle if its target is removed. A symbolic link itself can be removed with no effect on the target file or directory.
File names are just entries in the directory tables pointing to somewhere in the disk. A hard link is just another name entry pointing to the same data. Any subsequent hard link is indistinguishable from the original file name entry.
So the answer is yes.

When does a UNIX directory change its timestamp

I used "touch" on a file, updating the file's timestamp but the parent directory's timestamp did not change. However, (as expected) when I created a new file within the parent directory, the directory's timestamp did change.
What criteria do UNIX-like operating systems (specifically AIX) use to determine when to update the timestamp of a directory?
The timestamp is updated when the data that represents the directory changes. A change in a subdirectory of directory D does not change anything in the representation of D because D only points to the subdirectory, not to what's inside it. On the other hand, creating a file in D changes the block of data on disk that represents D.
A directory's timestamp is changed when the Directory itself is changed. The directory contains, among other things, a list of the inodes of the files in the directory so when you change the content of the directory by adding or removing files then the Directories timestamp will be updated.
You can use the stat command to find the modified time, creation time etc of a file/directory.
Refer to https://linux.die.net/man/2/stat
The article states:
st_mtime of a directory is changed by the creation or deletion of files in that directory. The st_mtime field is not changed for changes in owner, group, hard link count, or mode.
A Directory is considered as changed when there is any Addition or Deletion of File/Directory inside it. If existing Files/Directories are just getting update than Parent Directory timestamp will not change.

Resources