I am trying to run this line of code below to copy the city.output column to pm.city where it is not NA (in my sample dataframe, nothing is NA though) because city.output contains the correct city spellings.
resultdf <- dplyr::mutate(df, pm.city = ifelse(is.na(city.output) == FALSE, city.output, pm.city))
df:
pm.uid pm.address pm.state pm.zip pm.city city.output
<int> <chr> <chr> <chr> <chr> <fct>
1 1 1809 MAIN ST OH 63312 NORWOOD NORWOOD
2 2 123 ELM DR CA NA BRYAN BRYAN
3 3 8970 WOOD ST UNIT 4 LA 33333 BATEN ROUGE BATON ROUGE
4 4 4444 OAK AVE OH 87481 CINCINATTI CINCINNATI
5 5 3333 HELPME DR MT 87482 HELENA HELENA
6 6 2342 SOMEWHERE RD LA 45103 BATON ROUGE BATON ROUGE
resultdf (pm.city should be the same as city.output but it's an integer)
pm.uid pm.address pm.state pm.zip pm.city city.output
<int> <chr> <chr> <chr> <int> <fct>
1 1 1809 MAIN ST OH 63312 7 NORWOOD
2 2 123 ELM DR CA NA 2 BRYAN
3 3 8970 WOOD ST UNIT 4 LA 33333 1 BATON ROUGE
4 4 4444 OAK AVE OH 87481 3 CINCINNATI
5 5 4444 HELPME DR MT 87482 4 HELENA
6 6 2342 SOMEWHERE RD LA 45103 1 BATON ROUGE
An integer is instead assigned to pm.city. It appears the integer is the order number of the cities when they're in alphabetical order. Prior to this, I used the dplyr left_join method to attach city.output column from another dataframe but even there, there was no row number that I supplied explicitly.
This works on my computer in r studio but not when I run it from a server. Maybe it has something to do with my version of dplyr or the factor data type under city.output? I am pretty new to r.
The city.output is factor which gets coerced to integer storage values. Instead, convert to character with as.character
dplyr::mutate(df, pm.city = ifelse(!is.na(city.output), as.character(city.output), pm.city))
Related
I've got this dataframe and I want to add the last two columns to another dataframe by summing them and grouping them by "Full.Name"
# A tibble: 6 x 5
# Groups: authority_dic, Full.Name [6]
authority_dic Full.Name Entity `2019` `2020`
<chr> <chr> <chr> <int> <int>
1 accomplished Derek J. Leathers WERNER ENTERPRISES INC 1 0
2 accomplished Dirk Van de Put MONDELEZ INTERNATIONAL INC 0 1
3 accomplished Eileen P. Drake AEROJET ROCKETDYNE HOLDINGS 1 0
4 accomplished G. Michael Sievert T-MOBILE US INC 0 3
5 accomplished Gary C. Kelly SOUTHWEST AIRLINES 0 1
6 accomplished James C. Fish, Jr. WASTE MANAGEMENT INC 1 0
This is the dataframe I want to add the two columns to: Like you can see the "Full.Name" column acts as the grouping column.
# A tibble: 6 x 3
# Groups: Full.Name [6]
Full.Name `2019` `2020`
<chr> <int> <int>
1 A. Patrick Beharelle 5541 3269
2 Aaron P. Graft 165 200
3 Aaron P. Jagdfeld 4 5
4 Adam H. Schechter 147 421
5 Adam P. Symson 1031 752
6 Adena T. Friedman 1400 1655
I can add one column using the following piece of code, but if I want to do it with the second one, it overwrites my existing one and I am only left with one instead of two columns added.
narc_auth_total <- narc_auth %>% group_by(Full.Name) %>% summarise(`2019_words` = sum(`2019`)) %>% left_join(totaltweetsyear, ., by = "Full.Name")
The output for this command looks like this:
# A tibble: 6 x 4
# Groups: Full.Name [6]
Full.Name `2019` `2020` `2019_words`
<chr> <int> <int> <int>
1 A. Patrick Beharelle 5541 3269 88
2 Aaron P. Graft 165 200 2
3 Aaron P. Jagdfeld 4 5 0
4 Adam H. Schechter 147 421 2
5 Adam P. Symson 1031 752 15
6 Adena T. Friedman 1400 1655 21
I want to do the same thing and add the 2020_words column to the same dataframe. I just cannot do it, but it cannot be that hard to do so. It should be summarized as well, just like the 2019_words column. When I add "2020" to my command, it says object "2020" not found.
Thanks in advance.
If I have understood you well, this will solve your problem:
narc_auth_total <-
narc_auth %>%
group_by(Full.Name) %>%
summarise(
`2019_words` = sum(`2019`),
`2020_words` = sum(`2020`)
) %>%
left_join(totaltweetsyear, ., by = "Full.Name")
I have a data frame which contains information about sales branches, customers and sales.
branch <- c("Chicago","Chicago","Chicago","Chicago","Chicago","Chicago","LA","LA","LA","LA","LA","LA","LA","Tampa","Tampa","Tampa","Tampa","Tampa","Tampa","Tampa","Tampa")
customer <- c(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21)
sales <- c(33816,24534,47735,1467,39389,30659,21074,20195,45165,37606,38967,41681,47465,3061,23412,22993,34738,19408,11637,36234,23809)
data <- data.frame(branch, customer, sales)
What I need to accomplish is to iterate over each branch, take each customer in the branch and divide the sales for that customer by the total of the branch. I need to do it to find out how much each customer is contributing towards the total sales of the corresponding branch. E.g. for customer 1 I would like to divide 33816/177600 and store this value in a new column. (177600 is the total of chicago branch)
I have tried to write a function to iterate over each row in a for loop but I am not sure how to do it at a branch level. Any guidance is appreciated.
Consider base R's ave for new column of inline aggregate which also considers same customer with multiple records within the same branch:
data$customer_contribution <- ave(data$sales, data$customer, FUN=sum) /
ave(data$sales, data$branch, FUN=sum)
data
# branch customer sales customer_contribution
# 1 Chicago 1 33816 0.190405405
# 2 Chicago 2 24534 0.138141892
# 3 Chicago 3 47735 0.268778153
# 4 Chicago 4 1467 0.008260135
# 5 Chicago 5 39389 0.221784910
# 6 Chicago 6 30659 0.172629505
# 7 LA 7 21074 0.083576241
# 8 LA 8 20195 0.080090263
# 9 LA 9 45165 0.179117441
# 10 LA 10 37606 0.149139610
# 11 LA 11 38967 0.154537126
# 12 LA 12 41681 0.165300433
# 13 LA 13 47465 0.188238887
# 14 Tampa 14 3061 0.017462291
# 15 Tampa 15 23412 0.133560003
# 16 Tampa 16 22993 0.131169705
# 17 Tampa 17 34738 0.198172193
# 18 Tampa 18 19408 0.110718116
# 19 Tampa 19 11637 0.066386372
# 20 Tampa 20 36234 0.206706524
# 21 Tampa 21 23809 0.135824795
Or less wordy:
data$customer_contribution <- with(data, ave(sales, customer, FUN=sum) /
ave(sales, branch, FUN=sum))
We can use dplyr::group_by and dplyr::mutate to calculate fractional sales of total by branch.
library(dplyr);
library(magrittr);
data %>%
group_by(branch) %>%
mutate(sales.norm = sales / sum(sales))
## A tibble: 21 x 4
## Groups: branch [3]
# branch customer sales sales.norm
# <fct> <dbl> <dbl> <dbl>
# 1 Chicago 1. 33816. 0.190
# 2 Chicago 2. 24534. 0.138
# 3 Chicago 3. 47735. 0.269
# 4 Chicago 4. 1467. 0.00826
# 5 Chicago 5. 39389. 0.222
# 6 Chicago 6. 30659. 0.173
# 7 LA 7. 21074. 0.0836
# 8 LA 8. 20195. 0.0801
# 9 LA 9. 45165. 0.179
#10 LA 10. 37606. 0.149
I realize there have already been many asked and answered questions about merging datasets here, but I've been unable to find one that addresses my issue.
What I'm trying to do is merge to datasets using two variables and keeping all data from each. I've tried merge and all of the join operations from dplyr, as well as cbind and have not gotten the result I want. Usually what happens is that one column from one of the datasets gets overwritten with NAs. Another thing that will happen, as when I do full_join in dplyr or all = TRUE in merge is that I get double the number of rows.
Here's my data:
Primary_State Primary_County n
<fctr> <fctr> <int>
1 AK 12
2 AK Aleutians West 1
3 AK Anchorage 961
4 AK Bethel 1
5 AK Fairbanks North Star 124
6 AK Haines 1
Primary_County Primary_State Population
1 Autauga AL 55416
2 Baldwin AL 208563
3 Barbour AL 25965
4 Bibb AL 22643
5 Blount AL 57704
6 Bullock AL 10362
So I want to merge or join based on Primary_State and Primary_County, which is necessary because there are a lot of duplicate county names in the U.S. and retain the data from both n and Population. From there I can then divide the Population by n and get a per capita figure for each county. I just can't figure out how to do it and keep all of the data, so any help would be appreciated. Thanks in advance!
EDIT: Adding code examples of what I've already described above.
This code (as well as left_join):
countyPerCap <- merge(countyLicense, countyPops, all.x = TRUE)
Produces this:
Primary_State Primary_County n Population
1 AK 12 NA
2 AK Aleutians West 1 NA
3 AK Anchorage 961 NA
4 AK Bethel 1 NA
5 AK Fairbanks North Star 124 NA
6 AK Haines 1 NA
This code:
countyPerCap <- right_join(countyLicense, countyPops)
Produces this:
Primary_State Primary_County n Population
<chr> <chr> <int> <int>
1 AL Autauga NA 55416
2 AL Baldwin NA 208563
3 AL Barbour NA 25965
4 AL Bibb NA 22643
5 AL Blount NA 57704
6 AL Bullock NA 10362
Hope that's helpful.
EDIT: This is what happens with the following code:
countyPerCap <- merge(countyLicense, countyPops, all = TRUE)
Primary_State Primary_County n Population
1 AK 12 NA
2 AK Aleutians East NA 3296
3 AK Aleutians West 1 NA
4 AK Aleutians West NA 5647
5 AK Anchorage 961 NA
6 AK Anchorage NA 298192
It duplicates state and county and then adds n to one record and Population in another. Is there a way to deduplicate the dataset and remove the NAs?
We can give column names in merge by mentioning "by" in merge statement
merge(x,y, by=c(col1, col2 names))
in merge statement
I figured it out. There were trailing whitespaces in the Census data's county names, so they weren't matching with the other dataset's county names. (Note to self: Always check that factors match when trying to merge datasets!)
trim.trailing <- function (x) sub("\\s+$", "", x)
countyPops$Primary_County <- trim.trailing(countyPops$Primary_County)
countyPerCap <- full_join(countyLicense, countyPops,
by=c("Primary_State", "Primary_County"), copy=TRUE)
Those three lines did the trick. Thanks everyone!
I have the following extract of my dataset:
basisanddowngradessingledates[1716:1721, ]
# A tibble: 6 x 23
Dates Bank CDS Bond `Swap zero rate` `CDS-bond basis` `Basis change` `Rating agency`
<dttm> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 2015-05-15 Allied Irish Banks PLC 129.63 201.0235 40.6 -30.79352 1.9408116 NA
2 2015-05-18 Allied Irish Banks PLC 129.64 202.1998 41.0 -31.55976 -0.7662374 NA
3 2015-05-19 Allied Irish Banks PLC 129.65 200.4579 39.0 -31.80792 -0.2481631 Fitch
4 2015-05-20 Allied Irish Banks PLC 129.65 203.9960 39.0 -35.34598 -3.5380550 DBRS
5 2015-05-21 Allied Irish Banks PLC 129.63 203.5341 41.0 -32.90415 2.4418300 NA
6 2015-05-22 Allied Irish Banks PLC 130.64 203.2723 40.0 -32.63234 0.2718045 NA
I would like to select the intervals [-1:1], which corresponds to the day before and the day after a downgrade. At the row where the column "Rating agency" is not "NA" indicates that a downgrade has occured. In my example above, rows [1717:1719] and [1718:1720], so 6 rows, for each downgrade 3.
My dataset has 45276 entries with 536 downgrades (column "Rating agency" is not "NA") where I would like to build a list containing the 3 rows where a downgrade occured.
I tried it using the following code:
keepindex <- which(basisanddowngradessingledates[,8] != "NA")
interval11 <- unique(c(keepindex-1, keepindex, keepindex+1))
interval1ra1 <- basisanddowngradessingledates[interval11,]
This works if there are no downgrades on consecutive days. However in my example extract I have two downgrades right after each other and I get the following output:
print(interval1ra1[c(11:12, 348, 674), ])
# A tibble: 4 x 23
Dates Bank CDS Bond `Swap zero rate` `CDS-bond basis` `Basis change` `Rating agency`
<dttm> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 2015-05-18 Allied Irish Banks PLC 129.64 202.1998 41 -31.55976 -0.7662374 NA
2 2015-05-19 Allied Irish Banks PLC 129.65 200.4579 39 -31.80792 -0.2481631 Fitch
3 2015-05-20 Allied Irish Banks PLC 129.65 203.9960 39 -35.34598 -3.5380550 DBRS
4 2015-05-21 Allied Irish Banks PLC 129.63 203.5341 41 -32.90415 2.4418300 NA
I get 4 rows instead of 6 which I need.
I guess the unique()function prevents duplicate rows, but in my example I need these rows as described above.
How can I fix this?
Here is one possible solution to get previous and next row for each matching row:
> keepindex = c(1718,1719)
> lookupindex = c();
> for (lookupindex in keepindex) { result = c(lookupindex ,index-1,index,index+1) }
> lookupindex
[1] 1717 1718 1719 1718 1719 1720
In this solution the overlapping rows 1719 and 1718 are shown twice.
Found a simple solution by myself without using the unique funciton:
keepindex <- which(basisanddowngradessingledates[,8] != "NA")
interval1ra1 <- basisanddowngradessingledates[c(keepindex-1,keepindex,
keepindex+1), ]
I am trying to compare 2 different fields across consecutive rows on a data frame in R and indicate the ones that are different. Below is the input data:-
Start End
1 Atl Bos
2 Bos Har
3 Har NYC
4 Stf SFO
5 SFO Chi
I am trying to establish a chain of movement and where the End doesn't match up to the Start of the next row I want to indicate that row. So for the above I would indicate row 4 as below:-
Start End Ind
1 Atl Bos Y
2 Bos Har Y
3 Har NYC Y
4 Stf SFO N
5 SFO Chi Y
I am pretty new to R, I have tried looking up this problem but cant seem to find a solution. Any help is appreciated.
An alternative would be:
> Ind <- as.character(dat$Start[-1]) == as.character(dat$End [-length(dat$End)])
> dat$Ind <- c(NA, ifelse(Ind==TRUE, "Y", "N"))
> dat
Start End Ind
1 Atl Bos <NA>
2 Bos Har Y
3 Har NYC Y
4 Stf SFO N
5 SFO Chi Y
Note that your first item should be <NA>
You can do that with dplyr using mutate and lead. Note that the last item should be NA because there is no line 6 to compare SFO-CHI to.
library(dplyr)
df1 <- read.table(text=" Start End
Atl Bos
Bos Har
Har NYC
Stf SFO
SFO Chi", header=TRUE, stringsAsFactors=FALSE)
df1 %>%
mutate(Ind=ifelse(End==lead(Start),"Y","N"))
Start End Ind
1 Atl Bos Y
2 Bos Har Y
3 Har NYC N
4 Stf SFO Y
5 SFO Chi <NA>