Python: os.walk() placing files in unexpected locations - os.walk

I have code that intends to iterate through a directory hierarchy, and, for every file at the bottom of the hierarchy, create a file.
for currentDirectory, listOfSubDirectories, listOfStructures in os.walk(projectsLocation):
#this says if we arent at the bottom of the directory chain, do nothing (checks if subDirectoryList is 0)
if not listOfSubDirectories:
##THIS IS WHERE WE DO THINGS FOR ONE SIMULATION##
for currentStructure in listOfStructures:
if "Abridged" in currentStructure:
os.remove(currentStructure)
abridgedVersion=open(currentStructure[:-4]+"Abridged","w+")
Extremely bizarrely, however, is that the code places 2 of the files files at the top of the directory!
Even stranger is that when i rerun the code with error-checking print statements, printing the three lists, the files that get printed at the top of the directory "cant be found" despite the fact that they arent even in the list of files!
i.e.
/home/parker/pretendFoldingData/proj8202/run2/clone1
[]
['structure1.pdb', 'structure0Abridged', 'structure0.pdb']
Traceback (most recent call last):
File "importAtomFileMaker.py", line 35, in <module>
os.remove(currentStructure)
FileNotFoundError: [Errno 2] No such file or directory: 'structure1Abridged'
I hope this was even slightly clear..... can someone please advise?

from a friend:
You have a bug in your code;
well at least 2.
Especially if you are trying to rename a file.
open("path", 'w+') creates a file relevant to the current working directory unless a absolute path is given. Fix this by doing
base_path = os.path.join(currentDirectory, structure)
then
open(base_path + <filename>, 'w+') ...
This will make sure that the file you create will be placed relative to the current directory.
Instead of removing the file then creating a blank one, just rename the file
also the if not listOfDirectories is a bit "hacky".
I think your better off regex-ing the current directory for a pattern that you expect to be true

Related

Return one folder above current directory in Julia

In Julia, I can get the current directory from
#__DIR__
For example, when I run the above in the "Current" folder, it gives me
"/Users/jtheath/Dropbox/Research/Projects/Coding/Current"
However, I want it to return one folder above the present folder; i.e.,
"/Users/jtheath/Dropbox/Research/Projects/Coding"
Is there an easy way to do this in a Julia script?
First, please note that #__DIR__ generally expands to the directory of the current source file (it does however return the current working directory if there are no source files involved, e.g when run from the REPL). In order to reliably get the current working directory, you should rather use pwd().
Now to your real question: I think the easiest way to get the path to the parent directory would be to simply use dirname:
julia> dirname("/Users/jtheath/Dropbox/Research/Projects/Coding/Current")
"/Users/jtheath/Dropbox/Research/Projects/Coding"
Note that AFAIU this only uses string manipulations, and does not care whether the paths involved actually exist in the filesystem (which is why the example above works on my system although I do not have the same filesystem structure as you). dirname is also relatively sensitive to the presence/absence of a trailing slash (which shouldn't be a problem if you feed it something that comes directly from pwd() or #__DIR__).
I sometimes also use something like this, in the hope that it might be more robust when I want to work with paths that actually exist in the filesystem:
julia> curdir = pwd()
"/home/francois"
julia> abspath(joinpath(curdir, ".."))
"/home/"

How to save file into a path containing special characters such as '&'? ('&' which is different from '&' typed in English Keyboard)

I need to write out a file to a certain path that contains a special character in R. the path is something like this: C:/Users/Technology & Innovation/Webscraping files/US_data/data
It works totally fine when I access this path through python, but I cannot access the same path in R. And I cannot change this path name or remove '&' as this path is used by a lot of people. Does anyone have a good idea on how to solve it?
I found out it is '&' which has subtle difference from '&' that we usually type in through English Keyboard. May be that's the reason causing the problem?
Here is what I have tried:
write.csv(df, 'C:/Users/Technology & Innovation/Webscraping files/US_data/data/file.csv').
write.csv(df, 'C:\\Users\\Technology & Innovation\\Webscraping files\\US_data/data/file.csv')
Not matter whether I try to read or write a file, it is not working in my case.
I also tried reset the working directory path and got the error message:
Error in setwd("C:/Users/Technology & Innovation/Webscraping files/US_data/data") : cannot change working directory
Write it like this
C:\\Users\\Technology & Innovation\\Webscraping files\\US_data\\data
also, you can change your current directory.
Changing your current directory will help you because you can write read.csv("filename.csv") or write.csv(name_of_file, "filename.csv") as it is without mentioning path.
If you have to write a file you have to use syntax properly.
write.csv(C:\\Users\\Technology & Innovation\\Webscraping files\\US_data\\data,"filename.csv")

Avoid rendering of specific .md files from blogdown::serve_site()

I have a file located at
content/post/data_for_posts/my_file.md
I have it there because it's quite easy to do htmltools::includeMarkdown("data_for_posts/my_file.md") and recycle this file in different posts.
My problem is that when I serve_site() this creates a public/post/data_for_posts/index.html, which means, it gets posted to my website (as a January 1 of 0001). I guess I could change the date to year 10000, but I would rather handle it the way I handle the .Rmd and other files, as suggested here
I have tried to modify my config.toml but have not managed to solve the issue.
ignoreFiles = ["\\.Rmd$", "\\.Rmarkdown$", "_files$", "_cache$", "content/post/data_for_posts/my_file.md"]
Here are a couple techniques that I use to do this:
Rename data_for_posts/my_file.md so it uses a file extension that hugo does not interpret as a known markup language, for example change .md to .markd or mdn.[*]
Rename data_for_posts/my_file.md so it includes a string that you will never use in a real content file, for example data_for_posts-UNPUBLISHED/my_file.md. Then add that string (UNPUBLISHED or whatever) to your config ignoreFiles list.[**]
[*] In the content/ directory, a file with one of the following file extensions will be interpreted by hugo as containing a known markup language: .ad, .adoc, .asciidoc, .htm, .html, .markdown, .md, .mdown, .mmark, .pdc, .pandoc, .org, or .rst (this is an excerpt of something I wrote).
[**] The strings listed in ignoreFiles seem to be case sensitive so I like to use all-upper-case characters in my ignored file names (because I never use upper-case chars in real content file names). Also note that there is no need to specify the path and my experience is that path delimiters (/ or \) cause problems.

Standard ML / NJ: Loading in file of functions

I'm trying to write a bunch of functions in an SML file and then load them into the interpreter. I've been googling and came across this:
http://www.smlnj.org/doc/interact.html
Which has this section:
Loading ML source text from a file
The function use: string -> unit interprets its argument as a file name relative to sml's current directory and loads the text from that file as though it had been typed in. This should normally be executed at top level, but the loaded files can also contain calls of use to recursively load other files.
So I have a test.sml file in my current directory. I run sml, all good so far. Then I try use test.sml; and I get:
stdIn:1.6-1.14 Error: unbound structure: test in path test.sml
Not sure why this isn't working. Any ideas?
Thanks,
bclayman
As you mentioned, the function use has type string -> unit. This means it takes a string and returns unit. When you do use test.sml, you are not giving it a string. You need to do use "test.sml" (notice the quotes)

Server.MapPath does not like ~/ and ./

I am using the following code to try and find a file contained in another directory from my code file.
Set fi=fs.OpenTextFile(Server.MapPath("~/counter/counter.txt"), 1)
I have also tried.
Set fi=fs.OpenTextFile(Server.MapPath("./root/folder1/counter/counter.txt"), 1)
In either case this should get me back to the counter.txt file. From what I understand ~/ moves up 1 directory and ./ moves up to the root directory.
Both times however I receive an error saying an invalid character has been used. When removing these I get a different error saying the path cannot be found (Which I would expect because it is not a valid path without moving up 1 directory).
What are the valid characters to do the following in VBscript:
move up a single directory?
move up to the root directory?
Thanks for the help
A few things:
The tilde character "~" is not valid here.
The single period character "." is for specifying the current directory/folder.
A set of period characters ".." is for specifying the parent directory/folder. For example, to refer to a file found in the parent of the current directory, you might use:
Server.MapPath("../counter.txt")
You can chain these to walk up more than a single parent path. To refer to a file found three directories above the current, you might use:
Server.MapPath("../../../counter.txt")
The documentation on MSDN for the MapPath function outlines this. Pay attention to the caution listed here about enabling parent paths if you want to be able to refer to relative paths above the current directory. If you get an error when trying to refer to a parent path, then you do not have parent paths enabled.

Resources