I have an R function which tries to capitalise the first letter of every "word"
proper = function(x){
gsub("(?<=\\b)([[:alpha:]])", "\\U\\1", x, perl = TRUE)
}
This works pretty well, but when I have a word with a Māori macron in it like Māori I get improper capitalisation, e.g.
> proper("Māori")
[1] "MāOri"
Clearly the RE engine thinks the macron ā is a word boundary. Not sure why.
Since you are using a PCRE regex engine (enabled with perl=TRUE) you must pass the (*UCP) flag to the regex so that all shorthands and word boundaries could detect correct symbols/locations inside Unicode text:
proper = function(x){
gsub("(*UCP)\\b([[:alpha:]])", "\\U\\1", x, perl = TRUE)
}
proper("Māori")
## [1] "Māori"
See the R demo.
Note that \b is already a zero-width assertion and does not have to be placed into a positive lookbehind, i.e. (?<=\b) = \b.
\b basically denotes a boundary on characters other than [a-zA-Z0-9_] which includes multibyte characters as well unless a modifier called Unicode is set to affect engine behavior.
Unfortunately, gsub in R doesn't have this flag or I couldn't find anything about it in documentations.
A workaround would be:
(?<!\\S)([[:alpha:]])
which, in other hand, obviously fails on āmori.
Related
I'm just getting to know the language R, previously worked with python. The challenge is to replace the last character of each word in the string with *.
How it should look: example text in string, and result work: exampl* tex* i* strin*
My code:
library(tidyverse)
library(stringr)
string_example = readline("Enter our text:")
string_example = unlist(strsplit(string_example, ' '))
string_example
result = str_replace(string_example, pattern = "*\b", replacement = "*")
result
I get an error:
> result = str_replace(string_example, pattern = "*\b", replacement = "*")
Error in stri_replace_first_regex(string, pattern, fix_replacement(replacement), :
Syntax error in regex pattern. (U_REGEX_RULE_SYNTAX, context=``)
Help solve the task
Oh, I noticed an error, the pattern should be .\b. this is how the code is executed, but there is no replacement in the string
If you mean words consisting of letters only, you can use
string_example <- "example text in string"
library(stringr)
str_replace_all(string_example, "\\p{L}\\b", "*")
## => [1] "exampl* tex* i* strin*"
See the R demo and the regex demo.
Details:
\p{L} - a Unicode category (propery) class matching any Unicode letter
\b - a word boundary, in this case, it makes sure there is no other word character immediately on the right. It will fails the match if the letter matched with \p{L} is immediately followed with a letter, digit or _ (these are all word chars). If you want to limit this to a letter check, replace \b with (?!\p{L}).
Note the backslashes are doubled because in regular string literals backslashes are used to form string escape sequences, and thus need escaping themselves to introduce literal backslashes in string literals.
Some more things to consider
If you do not want to change one-letter words, add a non-word boundary at the start, "\\B\\p{L}\\b"
If you want to avoid matching letters that are followed with - + another letter (i.e. some compound words), you can add a lookahead check: "\\p{L}\\b(?!-)".
You may combine the lookarounds and (non-)word boundaries as you need.
i'm working on sentiment analysis with Arabic language by using R and in cleaning step i need to remove the single character.
I used this code to remove them and it works but a had some problem
for example here is the data
R<-("للمدافعين قال شركة وطنية قلت أقنعهم يعاملوننا كمواطنينقال جودتها عالية قلت جيدة غيرها غ")
as you see here "غ" is single character
gsub(" *\\b[[:alpha:]]{1}\\b *", "", R)
[1] "للمدافعين قال شركة وطنية قلت أقنعهم يعاملوننا كمواطنينقال جودتها عالية قلت جيدة غيرها\n"
but when I tried to apply it on the whole data set on text column like here
subdata1$text = gsub("*\\b[[:alpha:]]{1}\\b *", "", subdata1$text)
its doesn't remove anything and I don't known why?
hope you understand me
thank you
It seems the [:alpha:] POSIX character class does not work with all Unicode letters in your case.
I suggest using a PCRE pattern:
gsub("(*UCP)\\b\\p{L}\\b", "", R, perl=TRUE)
Here, (*UCP) is required to make \b word boundary Unicode aware and \p{L} matches any Unicode letter from a BMP plane. The perl=TRUE argument is required for the pattern to be processed with the PCRE regex engine.
I'm trying to use a regex to replace the last instance of a phrase (and everything after that phrase, which could be any character):
stringi::stri_replace_last_regex("_AB:C-_ABCDEF_ABC:45_ABC:454:", "_ABC.*$", "CBA")
However, I can't seem to get the refex to function properly:
Input: "_AB:C-_ABCDEF_ABC:45_ABC:454:"
Actual output: "_AB:C-CBA"
Desired output: "_AB:C-_ABCDEF_ABC:45_CBA"
I have tried gsub() as well but that hasn't worked.
Any ideas where I'm going wrong?
One solution is:
sub("(.*)_ABC.*", "\\1_CBA", Input)
[1] "_AB:C-_ABCDEF_ABC:45_CBA"
Have a look at what stringi::stri_replace_last_regex does:
Replaces with the given replacement string last substring of the input that matches a regular expression
What does your _ABC.*$ pattern match inside _AB:C-_ABCDEF_ABC:45_ABC:454:? It matches the first _ABC (that is right after C-) and all the text after to the end of the line (.*$ grabs 0+ chars other than line break chars to the end of the line). Hence, you only have 1 match, and it is the last.
Solutions can be many:
1) Capturing all text before the last occurrence of the pattern and insert the captured value with a replacement backreference (this pattern does not have to be anchored at the end of the string with $):
sub("(.*)_ABC.*", "\\1_CBA","_AB:C-_ABCDEF_ABC:45_ABC:454:")
2) Using a tempered greedy token to make sure you only match any char that does not start your pattern up to the end of the string after matching it (this pattern must be anchored at the end of the string with $):
sub("(?s)_ABC(?:(?!_ABC).)*$", "_CBA","_AB:C-_ABCDEF_ABC:45_ABC:454:", perl=TRUE)
Note that this pattern will require perl=TRUE argument to be parsed with a PCRE engine with sub (or you may use stringr::str_replace that is ICU regex library powered and supports lookaheads)
3) A negative lookahead may be used to make sure your pattern does not appear anywhere to the right of your pattern (this pattern does not have to be anchored at the end of the string with $):
sub("(?s)_ABC(?!.*_ABC).*", "_CBA","_AB:C-_ABCDEF_ABC:45_ABC:454:", perl=TRUE)
See the R demo online, all these three lines of code returning _AB:C-_ABCDEF_ABC:45_CBA.
Note that (?s) in the PCRE patterns is necessary in case your strings may contain a newline (and . in a PCRE pattern does not match newline chars by default).
Arguably the safest thing to do is using a negative lookahead to find the last occurrence:
_ABC(?:(?!_ABC).)+$
Demo
gsub("_ABC(?:(?!_ABC).)+$", "_CBA","_AB:C-_ABCDEF_ABC:45_ABC:454:", perl=TRUE)
[1] "_AB:C-_ABCDEF_ABC:45_CBA"
Using gsub and back referencing
gsub("(.*)ABC.*$", "\\1CBA","_AB:C-_ABCDEF_ABC:45_ABC:454:")
[1] "_AB:C-_ABCDEF_ABC:45_CBA"
I have been trying to come up with a suitable regrex to remove the word 'fries' from strings. However, it hasn't been very successful.
Ideally, the regrex expression should be able to remove the word 'fries' regardless of whether is it in upper case and/ or lower case and the word 'fries' shouldn't be removed under the following cases.
frenchfries
friesislove
ilovefriesverymuch
This is what I have came up with so far
gsub('(?i)\\Wfries\\W','',string)
One major flaw with the above is that regex expression isn't able to detect the word 'fries' if it's at either the start or the end of the string.
eg. 'I love fries', 'Fries is love'
TRE regex engine does not support inline modifiers and to match a whole word you need to use word boundaries \b.
You may use a PCRE regex if you want to use an inline case insensitive modifier (?i):
gsub('(?i)\\bfries\\b','',string, perl = TRUE)
or a TRE regex with ignore.case =TRUE argument:
gsub('\\bfries\\b','',string, ignore.case =TRUE)
You can also try this:
gsub("\\<fries\\>",replacement = "",string ,ignore.case = TRUE)
\\<fries\\> will make sure that only the exact word "fries" will be replaced
I have a requirement where I am working on a large data which is having double byte characters, in korean text. i want to look for a character and replace it. In order to display the korean text correctly in the browser I have changed the locale settings in R. But not sure if it gets updated for the code as well. below is my code to change locale to korean and the korean text gets visible properly in viewer, however in console it gives junk character on printing-
Sys.setlocale(category = "LC_ALL", locale = "korean")
My data is in a data.table format that contains a column with text in korean. example -
"광주광역시 동구 제봉로 49 (남동,(지하))"
I want to get rid of the 1st word which ends with "시" character. Then I want to get rid of the "(남동,(지하))" an the end. I was trying gsub, but it does not seem to be working.
New <- c("광주광역시 동구 제봉로 49 (남동,(지하))")
data <- as.data.table(New)
data[,New_trunc := gsub("\\b시", "", data$New)]
Please let me know where I am going wrong. Since I want to search the end of word, I am using \\b and since I want to replace any word ending with "시" character I am giving it as \\b시.....is this not the way to give? How to take care of () at the end of the sentence.
What would be a good source to refer to for regular expressions.
Is a utf-8 setting needed for the script as well?How to do that?
Since you need to match the letter you have at the end of the word, you need to place \b (word boundary) after the letter, so as to require a transition from a letter to a non-letter (or end of string) after that letter. A PCRE pattern that will handle this is
"\\s*\\b\\p{L}*시\\b"
Details
\\s* - zero or more whitespaces
\\b - a leading word boundary
\\p{L}* - zero or more letters
시 - your specific letter
\\b - end of the word
The second issue is that you need to remove a set of nested parentheses at the end of the string. You need again to rely on the PCRE regex (perl=TRUE) that can handle recursion with the help of a subroutine call.
> sub("\\s*(\\((?:[^()]++|(?1))*\\))$", "", New, perl=TRUE)
[1] "광주광역시 동구 제봉로 49"
Details:
\\s* - zero or more whitespaces
(\\((?:[^()]++|(?1))*\\)) - Group 1 (will be recursed) matching
\\( - a literal (
(?:[^()]++|(?1))* - zero or more occurrences of
[^()]++ - 1 or more chars other than ( and ) (possessively)
| - or
(?1) - a subroutine call that repeats the whole Group 1 subpattern
\\) - a literal )
$ - end of string.
Now, if you need to combine both, you would see that R PCRE-powered gsub does not handle Unicode chars in the pattern so easily. You must tell it to use Unicode mode with (*UCP) PCRE verb.
> gsub("(*UCP)\\b\\p{L}*시\\b|\\s*(\\((?:[^()]++|(?1))*\\))$", "", New, perl=TRUE)
[1] " 동구 제봉로 49"
Or using trimws to get rid of the leading/trailing whitespace:
> trimws(gsub("(*UCP)\\b\\p{L}*시\\b|(\\((?:[^()]++|(?1))*\\))$", "", New, perl=TRUE))
[1] "동구 제봉로 49"
See more details about the verb at PCRE Man page.