I found this information on how to use the $push operator to add new values to an array. However, I can't seem to get this working with rmongodb
Suppose we have the following doc in the DB
_id : 7 51005201f8ab44f1690f9526
tags : 4
1 : 2 a
2 : 2 b
3 : 2 c
I'd like to add a value to the array tags. Here's what I tried:
q <- mongo.bson.from.list(list(tags="a"))
TRY 1
Here I tried using the $push operator
Code
bnew <- mongo.bson.from.list(list("$push"=list("tags"="d")))
> mongo.update(mongo=con, ns, criteria=q, objNew=bnew)
[1] FALSE
Logfile
Thu Jan 24 16:42:27 [initandlisten] MongoDB starting : pid=6260 port=27017 dbpath=\data\db\ 64-bit host=ASHB-109C-02
Thu Jan 24 16:42:27 [initandlisten] db version v2.2.2, pdfile version 4.5
Thu Jan 24 16:42:27 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Thu Jan 24 16:42:27 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Thu Jan 24 16:42:27 [initandlisten] options: { logpath: "log_1.txt" }
Thu Jan 24 16:42:27 [initandlisten] journal dir=/data/db/journal
Thu Jan 24 16:42:27 [initandlisten] recover : no journal files present, no recovery needed
Thu Jan 24 16:42:27 [initandlisten] waiting for connections on port 27017
Thu Jan 24 16:42:27 [websvr] admin web console waiting for connections on port 28017
Thu Jan 24 16:42:36 [initandlisten] connection accepted from 127.0.0.1:52419 #1 (1 connection now open)
Thu Jan 24 16:42:44 [conn1] __test.test Assertion failure x == _nfields src\mongo\db\jsobj.cpp 1250
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(109) mongo::verifyFailed+0xdc
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\jsobj.cpp(1250) mongo::BSONIteratorSorted::BSONIteratorSorted+0xf3
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\ops\update_internal.cpp(906) mongo::ModSetState::createNewFromMods+0xa3
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\ops\update.cpp(370) mongo::_updateObjects+0x15a2
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(573) mongo::receivedUpdate+0x60d
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(437) mongo::assembleResponse+0x626
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Thu Jan 24 16:42:44 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Thu Jan 24 16:42:44 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Thu Jan 24 16:42:44 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Thu Jan 24 16:42:44 [conn1] update __test.test query: { tags: "a" } update: { $push: { tags: "d" } } nscanned:1 keyUpdates:0 exception: assertion src\mongo\db\jsobj.cpp:1250 locks(micros) w:398335 399ms
Thu Jan 24 16:42:48 CTRL_CLOSE_EVENT signal
Thu Jan 24 16:42:48 [consoleTerminate] got CTRL_CLOSE_EVENT, will terminate after current cmd ends
Thu Jan 24 16:42:48 [consoleTerminate] now exiting
Thu Jan 24 16:42:48 dbexit:
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to close listening sockets...
Thu Jan 24 16:42:48 [consoleTerminate] closing listening socket: 496
Thu Jan 24 16:42:48 [consoleTerminate] closing listening socket: 516
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to flush diaglog...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to close sockets...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: waiting for fs preallocator...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: lock for final commit...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: final commit...
Thu Jan 24 16:42:48 [conn1] end connection 127.0.0.1:52419 (0 connections now open)
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: closing all files...
Thu Jan 24 16:42:48 [consoleTerminate] closeAllFiles() finished
Thu Jan 24 16:42:48 [consoleTerminate] journalCleanup...
Thu Jan 24 16:42:48 [consoleTerminate] removeJournalFiles
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: removing fs lock...
Thu Jan 24 16:42:48 dbexit: really exiting now
TRY 2
Here I tried using the $addToSet operator
Code
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.object(buf, "$addToSet")
mongo.bson.buffer.start.object(buf, name="tags")
mongo.bson.buffer.start.array(buf, "$each")
values <- list("d", "e", "f")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
bnew <- mongo.bson.from.buffer(buf)
bnew
> mongo.update(mongo=con, ns, criteria=q, objNew=bnew)
[1] FALSE
Logfile
Thu Jan 24 16:43:52 [initandlisten] MongoDB starting : pid=4184 port=27017 dbpath=\data\db\ 64-bit host=ASHB-109C-02
Thu Jan 24 16:43:52 [initandlisten] db version v2.2.2, pdfile version 4.5
Thu Jan 24 16:43:52 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Thu Jan 24 16:43:52 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Thu Jan 24 16:43:52 [initandlisten] options: { logpath: "log_2.txt" }
Thu Jan 24 16:43:52 [initandlisten] journal dir=/data/db/journal
Thu Jan 24 16:43:52 [initandlisten] recover : no journal files present, no recovery needed
Thu Jan 24 16:43:52 [initandlisten] waiting for connections on port 27017
Thu Jan 24 16:43:52 [websvr] admin web console waiting for connections on port 28017
Thu Jan 24 16:43:57 [initandlisten] connection accepted from 127.0.0.1:52435 #1 (1 connection now open)
Thu Jan 24 16:44:27 [conn1] __test.test Assertion failure x == _nfields src\mongo\db\jsobj.cpp 1250
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(109) mongo::verifyFailed+0xdc
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\jsobj.cpp(1250) mongo::BSONIteratorSorted::BSONIteratorSorted+0xf3
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\ops\update_internal.cpp(906) mongo::ModSetState::createNewFromMods+0xa3
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\ops\update.cpp(370) mongo::_updateObjects+0x15a2
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(573) mongo::receivedUpdate+0x60d
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(437) mongo::assembleResponse+0x626
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Thu Jan 24 16:44:28 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Thu Jan 24 16:44:28 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Thu Jan 24 16:44:28 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Thu Jan 24 16:44:28 [conn1] update __test.test query: { tags: "a" } update: { $addToSet: { tags: { $each: [ "d", "e", "f" ] } } } nscanned:1 keyUpdates:0 exception: assertion src\mongo\db\jsobj.cpp:1250 locks(micros) w:390312 390ms
Thu Jan 24 16:44:33 [conn1] end connection 127.0.0.1:52435 (0 connections now open)
Thu Jan 24 16:44:37 CTRL_CLOSE_EVENT signal
Thu Jan 24 16:44:37 [consoleTerminate] got CTRL_CLOSE_EVENT, will terminate after current cmd ends
Thu Jan 24 16:44:37 [consoleTerminate] now exiting
Thu Jan 24 16:44:37 dbexit:
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to close listening sockets...
Thu Jan 24 16:44:37 [consoleTerminate] closing listening socket: 496
Thu Jan 24 16:44:37 [consoleTerminate] closing listening socket: 500
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to flush diaglog...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to close sockets...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: waiting for fs preallocator...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: lock for final commit...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: final commit...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: closing all files...
Thu Jan 24 16:44:37 [consoleTerminate] closeAllFiles() finished
Thu Jan 24 16:44:37 [consoleTerminate] journalCleanup...
Thu Jan 24 16:44:37 [consoleTerminate] removeJournalFiles
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: removing fs lock...
Thu Jan 24 16:44:37 dbexit: really exiting now
What am I doing wrong here?
Additional information
For those interested: here's the code that produced the example doc
pkg <- "rmongodb"
lib <- file.path(R.home(), "library")
if (!suppressWarnings(require(pkg, lib.loc=lib, character.only=TRUE))) {
install.packages(pkg, lib=lib)
require(pkg, lib.loc=lib, character.only=TRUE)
}
# CONNECTION
db <- "__test"
ns <- paste(db, "test", sep=".")
con <- mongo.create(db=db)
# ENSURE EMPTY DB
mongo.remove(mongo=con, ns=ns)
# INSERT
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
b <- mongo.bson.from.buffer(buf)
mongo.insert(mongo=con, ns=ns, b=b)
EDIT 2013-01-29
As suggested by Tad Marshall from 10gen in his comment to my bug report, I re-ran the code that inserts the document with the MongoDB server running in --objcheck mode (validates BSON structures) and voilà: the server won't let me insert the doc due to an assertion that fails. If I run the server without the --objcheck flag, insertion is successful (but that's probably just due to the fact that no validation takes place).
Note that I tried two different versions of putting together the array in tags as my initial code produced a doc that IMHO is not in sync with MongoDB's indexing conventions:
(Potentially) Invalid document
That's how I did it above. I noticed that I didn't make sure the array index starts with a 0. Insertion of this document will fail (see logfile below)
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf) # finish array 'tags'
mongo.bson.buffer.finish.object(buf) # finish buffer
b <- mongo.bson.from.buffer(buf)
> b
tags : 4
1 : 2 a
2 : 2 b
3 : 2 c
Valid document
I made sure the index starts with 0, so this should definitely be a valid BSON doc. But inserting this document will fail, too (see logfile below)
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii-1),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf) # finish array 'tags'
mongo.bson.buffer.finish.object(buf) # finish buffer
b <- mongo.bson.from.buffer(buf)
b
mongo.insert(mongo=con, ns=ns, b=b)
> b
tags : 4
0 : 2 a
1 : 2 b
2 : 2 c
Logfile
Tue Jan 29 14:20:46 [initandlisten] MongoDB starting : pid=6440 port=27017
[...]
Tue Jan 29 14:20:59 [initandlisten] connection accepted from 127.0.0.1:62137 #1 (1 connection now open)
Tue Jan 29 14:21:03 [conn1] Assertion: 10307:Client Error: bad object in message
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(154) mongo::msgasserted+0xc1
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\dbmessage.h(205) mongo::DbMessage::nextJsObj+0x103
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(784) mongo::receivedInsert+0xdb
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(434) mongo::assembleResponse+0x607
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Tue Jan 29 14:21:04 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Tue Jan 29 14:21:04 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Tue Jan 29 14:21:04 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Tue Jan 29 14:21:04 [conn1] insert __test.test keyUpdates:0 exception: Client Error: bad object in message code:10307 0ms
Tue Jan 29 14:21:07 [conn1] Assertion: 10307:Client Error: bad object in message
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(154) mongo::msgasserted+0xc1
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\dbmessage.h(205) mongo::DbMessage::nextJsObj+0x103
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(784) mongo::receivedInsert+0xdb
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(434) mongo::assembleResponse+0x607
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Tue Jan 29 14:21:07 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Tue Jan 29 14:21:07 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Tue Jan 29 14:21:07 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Tue Jan 29 14:21:07 [conn1] insert __test.test keyUpdates:0 exception: Client Error: bad object in message code:10307 0ms
Oh, I am smacking myself up now. I didn't look closely at the way you were creating your document. You have two mongo.bson.finish.object() calls when you need only one to finish off the array you started. You do not need to call it to finish a BSON. mongo.bson.from.buffer() does the necessary housekeeping. This is my fault for not reading your code closely enough. I thought it was your update failing when the initial insert of the documents is the problem. For questions here in the future, it would help if your examples were a little easier to read. For instance, this will build the document:
library('rmongodb')
m = mongo.create()
ns = '__test.test'
mongo.insert(m, ns, list(tags=c('a', 'b', 'c'))
However, you are probably pasting in real-world code so I understand where the complications come in. Everything's cool. Just beating myself up for missing this and sending you on a wild goose chase. Regards
Rappster, both of these examples worked for me on my development machine, but I am slightly out of date running a debug build of mongod 2.1.0.
Since you are crashing the server with your example code, this is something the 10gen people will want to know about. Do you mind going to https://jira.mongodb.org/secure/Dashboard.jspa and reporting this bug?
Thanks,
Gerald Lindsly
Related
I've been trying to set up an OpenVPN server on my Linux recently but I continuously get the same error every time I try to connect to my server.
My settings are like this:
proto tcp
port 443
resolv-retry infinite
nobind
user nobody
group nogroup
cipher AES-256-CBC
auth SHA256
script-security 2
up /etc/openvpn/update-systemd-resolved
down /etc/openvpn/update-systemd-resolved
down-pre
dhcp-option DOMAIN-ROUTE .
I have checked the settings on my server and local computer a million times and all of them are the same. Still don't know what I have to do about it. Thanks in advance! :*
Sat Nov 27 23:45:11 2021 OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 19 2021
Sat Nov 27 23:45:11 2021 library versions: OpenSSL 1.1.1f 31 Mar 2020, LZO 2.10
Sat Nov 27 23:45:11 2021 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Sat Nov 27 23:45:11 2021 TCP/UDP: Preserving recently used remote address: [AF_INET]myserverip:443
Sat Nov 27 23:45:11 2021 Socket Buffers: R=[131072->131072] S=[16384->16384]
Sat Nov 27 23:45:11 2021 Attempting to establish TCP connection with [AF_INET]myserverip:443 [nonblock]
Sat Nov 27 23:45:12 2021 TCP connection established with [AF_INET]myserverip:443
Sat Nov 27 23:45:12 2021 TCP_CLIENT link local: (not bound)
Sat Nov 27 23:45:12 2021 TCP_CLIENT link remote: [AF_INET]myserverip:443
Sat Nov 27 23:45:12 2021 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
Sat Nov 27 23:45:12 2021 Connection reset, restarting [0]
Sat Nov 27 23:45:12 2021 SIGUSR1[soft,connection-reset] received, process restarting
Sat Nov 27 23:45:12 2021 Restart pause, 5 second(s)
I am trying to create a rrule for my fullcalendar event, that occur on the 2nd Monday, Wednesday and Friday of the month for every month.
Here is the rrule I have tried
RRULE:FREQ=MONTHLY;COUNT=10;INTERVAL=1;WKST=SU;BYDAY=MO,WE,FR;BYSETPOS=2
events: [{
title: 'rrule event',
rrule: {
freq: RRule.MONTHLY,
count: 10,
interval: 1,
wkst: RRule.SU,
byweekday: [RRule.MO, RRule.WE, RRule.FR],
bysetpos: [2]
},
duration: '02:00',
rendering: 'inverse-background'
}
],
This is what I get
1 Fri, 03 May 2019 12:33:53 GMT
2 Wed, 05 Jun 2019 12:33:53 GMT
3 Wed, 03 Jul 2019 12:33:53 GMT
4 Mon, 05 Aug 2019 12:33:53 GMT
5 Wed, 04 Sep 2019 12:33:53 GMT
6 Fri, 04 Oct 2019 12:33:53 GMT
7 Mon, 04 Nov 2019 12:33:53 GMT
8 Wed, 04 Dec 2019 12:33:53 GMT
9 Fri, 03 Jan 2020 12:33:53 GMT
10 Wed, 05 Feb 2020 12:33:53 GMT
What is expected is
1 Mon, 08 Apr 2019
2 Wed, 10 Apr 2019
3 Fri, 12 Apr 2019
4 Mon, 13 May 2019
5 Wed, 08 May 2019
6 Fri, 10 May 2019.........
RFC 5545, section 3.3.10. states:
Each BYDAY value can also be preceded by a positive (+n) or
negative (-n) integer. If present, this indicates the nth
occurrence of a specific day within the MONTHLY or YEARLY "RRULE".
So the rule you're looking for literally specifies the 2nd Monday (2MO), Wednesday (2WE) and Friday (2FR) of each month.
FREQ=MONTHLY;COUNT=10;BYDAY=2MO,2WE,2FR
(click to see the results)
Note that INTERVAL=1 is the default and WKST=SU is meaningless in this case, so you can just as well omit them.
Btw, your rule basically says, of all Mondays, Wednesdays and Fridays of a month, take the second instance in that month.
I have the following 2 columns as part of a larger data frame. The Timezone_Offset is the difference in hours for the local time (US West Coast in the data I'm looking at). In other words, UTC + Offset = Local Time.
I'm looking to convert the UTC time to the local time, while also correctly changing the day of the week and date, if necessary. For instance, here are the first 5 rows of the two columns.
UTC Timezone_Offset
Sun Apr 08 02:42:03 +0000 2012 -7
Sun Jul 01 03:27:20 +0000 2012 -7
Wed Jul 11 04:40:18 +0000 2012 -7
Sat Nov 17 01:31:36 +0000 2012 -8
Sun Apr 08 20:50:30 +0000 2012 -7
Things get tricky when the day of the week and date also have to be changed. For instance, looking at the first row, the local time should be Sat Apr 07 19:42:03 +0000 2012. In the second row, the month also has to be changed.
Sorry, I'm fairly new to R. Could someone possibly explain how to do this? Thank you so much in advance.
Parse as UTC, then apply the offset in seconds, ie times 60*60 :
data <- read.csv(text="UTC, Timezone_Offset
Sun Apr 08 02:42:03 +0000 2012, -7
Sun Jul 01 03:27:20 +0000 2012, -7
Wed Jul 11 04:40:18 +0000 2012, -7
Sat Nov 17 01:31:36 +0000 2012, -8
Sun Apr 08 20:50:30 +0000 2012, -7", stringsAsFactors=FALSE)
data$pt <- as.POSIXct(strptime(data$UTC, "%a %b %d %H:%M:%S %z %Y", tz="UTC"))
data$local <- data$pt + data$Timezone_Offset*60*60
Result:
> data[,3:4]
pt local
1 2012-04-08 02:42:03 2012-04-07 19:42:03
2 2012-07-01 03:27:20 2012-06-30 20:27:20
3 2012-07-11 04:40:18 2012-07-10 21:40:18
4 2012-11-17 01:31:36 2012-11-16 17:31:36
5 2012-04-08 20:50:30 2012-04-08 13:50:30
>
I'm new to R programming, although I have been programming a number of other languages for years. I'm having a hard time finding any relevant information on this simple problem through searching the R documentation and stack overflow etc., so some help would be very much appreciated.
Here's the problem:
After reading in data from a .csv, I need to create a new dataset that contains only those observations where the "value" field is between 0 and 100 inclusive (there are 4 fields and ~2500 rows of data). I have no problem reading in the data and displaying it. My problem is when I try to take the list of input data and filter it based on the range condition for the "value" column.
Here's my input:
#read in the data from the sensor file
data = read.csv("C:/Code/sensor.txt", header=TRUE)
for (i in seq(4, nrow(data), 4)) {
if (as.integer(data[i])>0) {
print(data[i])
}
}
I am getting the error output:
> for (i in seq(4, nrow(data), 4)) {
+ if (as.integer(data[i])>0) {
+ print(data[i])
+ }
+ }
Error: (list) object cannot be coerced to type 'integer'
EDIT:
Here is some sample data:
timestamp, siteid, sensorid, value
Thu Jan 07 00:00:00 PST 2016,1,1,24
Thu Jan 07 00:00:00 PST 2016,1,2,5
Thu Jan 07 00:00:00 PST 2016,1,3,60
Thu Jan 07 00:00:00 PST 2016,2,1,0
Thu Jan 07 00:00:00 PST 2016,2,2,5
Thu Jan 07 00:00:00 PST 2016,2,3,100
Thu Jan 07 00:00:00 PST 2016,3,1,36
Thu Jan 07 00:00:00 PST 2016,3,2,5
Thu Jan 07 00:00:00 PST 2016,3,3,38
Thu Jan 07 00:00:00 PST 2016,4,1,99
Thu Jan 07 00:00:00 PST 2016,4,2,5
Thu Jan 07 00:00:00 PST 2016,4,3,84
Thu Jan 07 00:15:00 PST 2016,1,1,#ERROR#
Thu Jan 07 00:15:00 PST 2016,1,2,5
Thu Jan 07 00:15:00 PST 2016,1,3,96
Thu Jan 07 00:15:00 PST 2016,2,1,28
Thu Jan 07 00:15:00 PST 2016,2,2,5
Thu Jan 07 00:15:00 PST 2016,2,3,94
Thu Jan 07 00:15:00 PST 2016,3,1,3
Thu Jan 07 00:15:00 PST 2016,3,2,5
Thu Jan 07 00:15:00 PST 2016,3,3,95
Thu Jan 07 00:15:00 PST 2016,4,1,72
Thu Jan 07 00:15:00 PST 2016,4,2,5
Thu Jan 07 00:15:00 PST 2016,4,3,21
Thu Jan 07 00:30:00 PST 2016,1,1,160
Thu Jan 07 00:30:00 PST 2016,1,2,5
Thu Jan 07 00:30:00 PST 2016,1,3,34
First of all, always try to give us some reproductible example of data
data.beetween0and100 <- data[data$column.with.values => 0 & data$column.with.values <=100,]
This how you get data with desired values. Also you data frames how to dimmension rows, and columns so data[i] is bad but, data[i,] is a i-row of data frame.
print(data[i,]) #will work
with you data
#read in the data from the sensor file
data = read.csv("C:/Code/sensor.txt", header=TRUE)
for (i in seq(4, nrow(data), 4)) {
if (as.integer(data[i,numberofvaluecolumn])>0) {
print(data[i,numberofvaluecolumn])
}
}
For starters, loops in R are usually pretty slow and should be used with caution. With a dataset of only 2,500 records it probably isn't an issue, but worth mentioning if you are going to start using larger datasets.
If you are going to be doing a lot of data manipulation I would recommend becoming familiar with the dplyr library, https://cran.r-project.org/web/packages/dplyr/dplyr.pdf. It makes data manipulation very quick and easy.
data<-data %>%
filter(values>0,values<100)
If the function as.integer is throwing an error then perhaps read.csv() hasn't read the values in a format which as.integer() can handle.
Use str(data) or head() and tail() to see what read.csv() is producing.#
Looking at your your example data, adding the argument
na.strings = "#ERROR#"
to read.csv() might solve the issue.
Here's some of my data, read in from a file names AttReport_all:
Registration.Date Join.Time Leave.Time
1 Jul 05, 2011 09:30 PM EDT Jul 07, 2011 01:05 PM EDT Jul 07, 2011 01:53 PM EDT
2 Jul 05, 2011 10:20 AM EDT Jul 07, 2011 01:04 PM EDT Jul 07, 2011 01:53 PM EDT
3 Jul 04, 2011 02:41 PM EDT Jul 07, 2011 12:49 PM EDT Jul 07, 2011 01:53 PM EDT
4 Jul 04, 2011 11:38 PM EDT Jul 07, 2011 12:49 PM EDT Jul 07, 2011 01:54 PM EDT
5 Jul 05, 2011 11:41 AM EDT Jul 07, 2011 12:54 PM EDT Jul 07, 2011 01:54 PM EDT
6 Jul 07, 2011 11:08 AM EDT Jul 07, 2011 01:16 PM EDT Jul 07, 2011 01:53 PM EDT
If I do strptime(AttReport_all$Registration.Date, "%b %m, %Y %H:%M %p", tz="") I get an array of NAs where I'm expecting dates.
Sys.setlocale("LC_TIME", "C") returns "C"
typeof(AttReport_all$Registration.Date) returns "integer"
is.factor(AttReport_all$Registration.Date) returns TRUE.
What am I missing?
Here's version output, if it helps:
platform i386-pc-mingw32
arch i386
os mingw32
system i386, mingw32
status
major 2
minor 13.0
year 2011
month 04
day 13
svn rev 55427
language R
version.string R version 2.13.0 (2011-04-13)
strptime automatically runs as.character on the first argument (so it doesn't matter that it's a factor) and any trailing characters not specified in format= are ignored (so "EDT" doesn't matter).
The only issues are the typo #Ben Bolker identified (%m should be %d) and %H should be %I (?strptime says you should not use %H with %p).
# %b and %m are both *month* formats
strptime("Jul 05, 2011 09:30 PM EDT", "%b %m, %Y %H:%M %p", tz="")
# [1] NA
# change %m to %d and we no longer get NA, but the time is wrong (AM, not PM)
strptime("Jul 05, 2011 09:30 PM EDT", "%b %d, %Y %H:%M %p", tz="")
# [1] "2011-07-05 09:30:00"
# use %I (not %H) with %p
strptime("Jul 05, 2011 09:30 PM EDT", "%b %d, %Y %I:%M %p", tz="")
# [1] "2011-07-05 21:30:00"