I have a problem with special characters when trying to post file as "multipart/form-data". When i try to post a file with special characters, the file is missing from the post. With no special characters my script works just fine.
How should I handle the filenames?
Works ok:
r = requests.post('http://localhost/upload.ws', files={'file': open('test.txt')}, data=param, auth=HTTPBasicAuth('user', 'pass'))
Posts empty data as file:
r = requests.post('http://localhost/upload.ws', files={'file': open('test_äöå.txt')}, data=param, auth=HTTPBasicAuth('user', 'pass'))
try converting the string to unicode, like this:
r = requests.post('http://localhost/upload.ws', files={'file': open(u'test_äöå.txt')}, data=param, auth=HTTPBasicAuth('user', 'pass'))
or if the name is in a variable do it like:
r = requests.post('http://localhost/upload.ws', files={'file': open(filename.decode('utf8'))}, data=param, auth=HTTPBasicAuth('user', 'pass'))
its not a problem with requests library, python doesnt treat strings as unicode by default so it cant open the file ( it treats its name like ascii characters)
Requests is probably extracting the filename incorrectly from the object. Try passing an explicit filename as a unicode:
..., files={'file': (u'test_äöå.txt', open('test_äöå.txt'))}, ...
Related
I run R on Windows, and have a csv file on the Desktop. I load it as follows,
x<-read.csv("C:\Users\surfcat\Desktop\2006_dissimilarity.csv",header=TRUE)
but the R gives the following error message
Error: '\U' used without hex digits in character string starting "C:\U"
So what's the correct way to load this file. I am using Vista
replace all the \ with \\.
it's trying to escape the next character in this case the U so to insert a \ you need to insert an escaped \ which is \\
Please do not mark this response as correct as smitec has already answered correctly. I'm including a convenience function I keep in my .First library that makes converting a windows path to the format that works in R (the methods described by Sacha Epskamp). Simply copy the path to your clipboard (ctrl + c) and then run the function as pathPrep(). No need for an argument. The path is printed to your console correctly and written to your clipboard for easy pasting to a script. Hope this is helpful.
pathPrep <- function(path = "clipboard") {
y <- if (path == "clipboard") {
readClipboard()
} else {
cat("Please enter the path:\n\n")
readline()
}
x <- chartr("\\", "/", y)
writeClipboard(x)
return(x)
}
Solution
Try this: x <- read.csv("C:/Users/surfcat/Desktop/2006_dissimilarity.csv", header=TRUE)
Explanation
R is not able to understand normal windows paths correctly because the "\" has special meaning - it is used as escape character to give following characters special meaning (\n for newline, \t for tab, \r for carriage return, ..., have a look here ).
Because R does not know the sequence \U it complains. Just replace the "\" with "/" or use an additional "\" to escape the "\" from its special meaning and everything works smooth.
Alternative
On windows, I think the best thing to do to improve your workflow with windows specific paths in R is to use e.g. AutoHotkey which allows for custom hotkeys:
define a Hotkey, e.g. Cntr-Shift-V
assigns it an procedure that replaces backslashes within your Clipboard with
slaches ...
when ever you want to copy paste a path into R you can use Cntr-Shift-V instead of Cntr-V
Et-voila
AutoHotkey Code Snippet (link to homepage)
^+v::
StringReplace, clipboard, clipboard, \, /, All
SendInput, %clipboard%
My Solution is to define an RStudio snippet as follows:
snippet pp
"`r gsub("\\\\", "\\\\\\\\\\\\\\\\", readClipboard())`"
This snippet converts backslashes \ into double backslashes \\. The following version will work if you prefer to convert backslahes to forward slashes /.
snippet pp
"`r gsub("\\\\", "/", readClipboard())`"
Once your preferred snippet is defined, paste a path from the clipboard by typing p-p-TAB-ENTER (that is pp and then the tab key and then enter) and the path will be magically inserted with R friendly delimiters.
Replace back slashes \ with forward slashes / when running windows machine
I know this is really old, but if you are copying and pasting anyway, you can just use:
read.csv(readClipboard())
readClipboard() escapes the back-slashes for you. Just remember to make sure the ".csv" is included in your copy, perhaps with this:
read.csv(paste0(readClipboard(),'.csv'))
And if you really want to minimize your typing you can use some functions:
setWD <- function(){
setwd(readClipboard())
}
readCSV <- function(){
return(readr::read_csv(paste0(readClipboard(),'.csv')))
}
#copy directory path
setWD()
#copy file name
df <- readCSV()
Replacing backslash with forward slash worked for me on Windows.
The best way to deal with this in case of txt file which contains data for text mining (speech, newsletter, etc.) is to replace "\" with "/".
Example:
file<-Corpus(DirSource("C:/Users/PRATEEK/Desktop/training tool/Text Analytics/text_file_main"))
I think that R is reading the '\' in the string as an escape character. For example \n creates a new line within a string, \t creates a new tab within the string.
'\' will work because R will recognize this as a normal backslash.
readClipboard() works directly too. Copy the path into your clipboard
C:\Users\surfcat\Desktop\2006_dissimilarity.csv
Then
readClipboard()
appears as
[1] "C:\\Users\\surfcat\\Desktop\\2006_dissimilarity.csv"
A simple way is to use python.
in python terminal type
r"C:\Users\surfcat\Desktop\2006_dissimilarity.csv"
and you'll get back
'C:\Users\surfcat\Desktop\2006_dissimilarity.csv'
I run R on Windows, and have a csv file on the Desktop. I load it as follows,
x<-read.csv("C:\Users\surfcat\Desktop\2006_dissimilarity.csv",header=TRUE)
but the R gives the following error message
Error: '\U' used without hex digits in character string starting "C:\U"
So what's the correct way to load this file. I am using Vista
replace all the \ with \\.
it's trying to escape the next character in this case the U so to insert a \ you need to insert an escaped \ which is \\
Please do not mark this response as correct as smitec has already answered correctly. I'm including a convenience function I keep in my .First library that makes converting a windows path to the format that works in R (the methods described by Sacha Epskamp). Simply copy the path to your clipboard (ctrl + c) and then run the function as pathPrep(). No need for an argument. The path is printed to your console correctly and written to your clipboard for easy pasting to a script. Hope this is helpful.
pathPrep <- function(path = "clipboard") {
y <- if (path == "clipboard") {
readClipboard()
} else {
cat("Please enter the path:\n\n")
readline()
}
x <- chartr("\\", "/", y)
writeClipboard(x)
return(x)
}
Solution
Try this: x <- read.csv("C:/Users/surfcat/Desktop/2006_dissimilarity.csv", header=TRUE)
Explanation
R is not able to understand normal windows paths correctly because the "\" has special meaning - it is used as escape character to give following characters special meaning (\n for newline, \t for tab, \r for carriage return, ..., have a look here ).
Because R does not know the sequence \U it complains. Just replace the "\" with "/" or use an additional "\" to escape the "\" from its special meaning and everything works smooth.
Alternative
On windows, I think the best thing to do to improve your workflow with windows specific paths in R is to use e.g. AutoHotkey which allows for custom hotkeys:
define a Hotkey, e.g. Cntr-Shift-V
assigns it an procedure that replaces backslashes within your Clipboard with
slaches ...
when ever you want to copy paste a path into R you can use Cntr-Shift-V instead of Cntr-V
Et-voila
AutoHotkey Code Snippet (link to homepage)
^+v::
StringReplace, clipboard, clipboard, \, /, All
SendInput, %clipboard%
My Solution is to define an RStudio snippet as follows:
snippet pp
"`r gsub("\\\\", "\\\\\\\\\\\\\\\\", readClipboard())`"
This snippet converts backslashes \ into double backslashes \\. The following version will work if you prefer to convert backslahes to forward slashes /.
snippet pp
"`r gsub("\\\\", "/", readClipboard())`"
Once your preferred snippet is defined, paste a path from the clipboard by typing p-p-TAB-ENTER (that is pp and then the tab key and then enter) and the path will be magically inserted with R friendly delimiters.
Replace back slashes \ with forward slashes / when running windows machine
I know this is really old, but if you are copying and pasting anyway, you can just use:
read.csv(readClipboard())
readClipboard() escapes the back-slashes for you. Just remember to make sure the ".csv" is included in your copy, perhaps with this:
read.csv(paste0(readClipboard(),'.csv'))
And if you really want to minimize your typing you can use some functions:
setWD <- function(){
setwd(readClipboard())
}
readCSV <- function(){
return(readr::read_csv(paste0(readClipboard(),'.csv')))
}
#copy directory path
setWD()
#copy file name
df <- readCSV()
Replacing backslash with forward slash worked for me on Windows.
The best way to deal with this in case of txt file which contains data for text mining (speech, newsletter, etc.) is to replace "\" with "/".
Example:
file<-Corpus(DirSource("C:/Users/PRATEEK/Desktop/training tool/Text Analytics/text_file_main"))
I think that R is reading the '\' in the string as an escape character. For example \n creates a new line within a string, \t creates a new tab within the string.
'\' will work because R will recognize this as a normal backslash.
readClipboard() works directly too. Copy the path into your clipboard
C:\Users\surfcat\Desktop\2006_dissimilarity.csv
Then
readClipboard()
appears as
[1] "C:\\Users\\surfcat\\Desktop\\2006_dissimilarity.csv"
A simple way is to use python.
in python terminal type
r"C:\Users\surfcat\Desktop\2006_dissimilarity.csv"
and you'll get back
'C:\Users\surfcat\Desktop\2006_dissimilarity.csv'
I am trying to decode filenames in HTTP but the string from browser messages are different.
In my test file I put the name ç.jpg.
What I need is the name %C3%A7.jpg.
But the browser is sending %C3%83%C2%A7.jpg.
It's not UTF8, UTF16 or UTF32.
For another example I test the file name €.jpg.
What I need is the name %E2%82%AC.jpg.
But I am receiving %C3%A2%E2%80%9A%C2%AC.jpg.
how can I convert this names to UTF8?
Ok I played with this for about 30 minutes and I finally figured it out.
This is how the original string was encoded:
The string was in UTF-8
Some encoding mechanism thought it was CP1252, and based on that wrong assumption re-encoded it to UTF-8 again.
The resulting string was url-encoded.
To get back to a real UTF-8 string, this is what I did. (note, I used PHP, don't know what you are using but it should be doable in other languages just the same).
$input = '%C3%A2%E2%80%9A%C2%AC %C3%83%C2%A7';
$str1 = urldecode($input);
echo iconv('UTF-8', 'CP1252', $str1);
// output "€ ç"
So that conversion is counter intuitive. We're converting to CP1252, but still end up with a UTF-8 string. This only works because an existing UTF-8 was falsely treated as CP1252, and that incorrect interpretation was then converted to UTF-8. So I'm just reversing this double-encoding.
In other languages there might be a few more steps, this works in just 1 line with PHP because strings are bytes, not characters.
I want to replace every "€" in a string with "[euro]". Now this works perfectly fine with
file.col.name <- gsub("€","[euro]", file.col.name, fixed = TRUE)
Now I am looping over column names from a csv-file and suddenly I have trouble with the string "total€".
It works for other special character (#,?) but the € sign doesn't get recognized.
grep("€",file.column.name)
also returns 0 and if I extract the last letter it prints "€" but
print(lastletter(file.column.name) == "€")
returns FALSE. (lastletter is just a function to extract the last letter of a string.)
Does anyone have an idea why that happens and maybe an idea to solve it? I checked the class of "file.column.name" and it returns "character", also tried to convert it into a character again and stuff like that but didn't help.
Thank you!
Your encodings are probably mixed. Check the encodings of the files, then add the appropriate encoding to, e.g., read.csv using fileEncoding="…" as an argument.
If you are working under Unix/Linux, the file utility will tell you the encoding of text files. Otherwise, any editor should show you the encoding of the files.
Common encodings are UTF-8, ISO-8859-15 and windows-1252. Try "UTF-8", "windows-1252" and "latin-9" as values for fileEncoding (the latter being a portable name for ISO-8859-15 according to R's documentation).
I run R on Windows, and have a csv file on the Desktop. I load it as follows,
x<-read.csv("C:\Users\surfcat\Desktop\2006_dissimilarity.csv",header=TRUE)
but the R gives the following error message
Error: '\U' used without hex digits in character string starting "C:\U"
So what's the correct way to load this file. I am using Vista
replace all the \ with \\.
it's trying to escape the next character in this case the U so to insert a \ you need to insert an escaped \ which is \\
Please do not mark this response as correct as smitec has already answered correctly. I'm including a convenience function I keep in my .First library that makes converting a windows path to the format that works in R (the methods described by Sacha Epskamp). Simply copy the path to your clipboard (ctrl + c) and then run the function as pathPrep(). No need for an argument. The path is printed to your console correctly and written to your clipboard for easy pasting to a script. Hope this is helpful.
pathPrep <- function(path = "clipboard") {
y <- if (path == "clipboard") {
readClipboard()
} else {
cat("Please enter the path:\n\n")
readline()
}
x <- chartr("\\", "/", y)
writeClipboard(x)
return(x)
}
Solution
Try this: x <- read.csv("C:/Users/surfcat/Desktop/2006_dissimilarity.csv", header=TRUE)
Explanation
R is not able to understand normal windows paths correctly because the "\" has special meaning - it is used as escape character to give following characters special meaning (\n for newline, \t for tab, \r for carriage return, ..., have a look here ).
Because R does not know the sequence \U it complains. Just replace the "\" with "/" or use an additional "\" to escape the "\" from its special meaning and everything works smooth.
Alternative
On windows, I think the best thing to do to improve your workflow with windows specific paths in R is to use e.g. AutoHotkey which allows for custom hotkeys:
define a Hotkey, e.g. Cntr-Shift-V
assigns it an procedure that replaces backslashes within your Clipboard with
slaches ...
when ever you want to copy paste a path into R you can use Cntr-Shift-V instead of Cntr-V
Et-voila
AutoHotkey Code Snippet (link to homepage)
^+v::
StringReplace, clipboard, clipboard, \, /, All
SendInput, %clipboard%
My Solution is to define an RStudio snippet as follows:
snippet pp
"`r gsub("\\\\", "\\\\\\\\\\\\\\\\", readClipboard())`"
This snippet converts backslashes \ into double backslashes \\. The following version will work if you prefer to convert backslahes to forward slashes /.
snippet pp
"`r gsub("\\\\", "/", readClipboard())`"
Once your preferred snippet is defined, paste a path from the clipboard by typing p-p-TAB-ENTER (that is pp and then the tab key and then enter) and the path will be magically inserted with R friendly delimiters.
Replace back slashes \ with forward slashes / when running windows machine
I know this is really old, but if you are copying and pasting anyway, you can just use:
read.csv(readClipboard())
readClipboard() escapes the back-slashes for you. Just remember to make sure the ".csv" is included in your copy, perhaps with this:
read.csv(paste0(readClipboard(),'.csv'))
And if you really want to minimize your typing you can use some functions:
setWD <- function(){
setwd(readClipboard())
}
readCSV <- function(){
return(readr::read_csv(paste0(readClipboard(),'.csv')))
}
#copy directory path
setWD()
#copy file name
df <- readCSV()
Replacing backslash with forward slash worked for me on Windows.
The best way to deal with this in case of txt file which contains data for text mining (speech, newsletter, etc.) is to replace "\" with "/".
Example:
file<-Corpus(DirSource("C:/Users/PRATEEK/Desktop/training tool/Text Analytics/text_file_main"))
I think that R is reading the '\' in the string as an escape character. For example \n creates a new line within a string, \t creates a new tab within the string.
'\' will work because R will recognize this as a normal backslash.
readClipboard() works directly too. Copy the path into your clipboard
C:\Users\surfcat\Desktop\2006_dissimilarity.csv
Then
readClipboard()
appears as
[1] "C:\\Users\\surfcat\\Desktop\\2006_dissimilarity.csv"
A simple way is to use python.
in python terminal type
r"C:\Users\surfcat\Desktop\2006_dissimilarity.csv"
and you'll get back
'C:\Users\surfcat\Desktop\2006_dissimilarity.csv'