I have a hacked WordPress site, where the attacker managed to exploit mailing system using php code, I cleaned the code but it seems the website is not responding as it used to be (http://heroleads.com/thailand)
I went through the logs and seems the php process is being killed, I can not figure out the issue
Oct 23 07:07:41 leadhero lfd[46958]: *Suspicious Process* PID:46217 PPID:29512 User:herolead Uptime:75 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:07:41 leadhero lfd[46958]: *User Processing* PID:46920 Kill:0 User:herolead VM:245(MB) EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/wp-admin/admin-ajax.php
Oct 23 07:07:41 leadhero lfd[46958]: *User Processing* PID:46905 Kill:0 User:herolead VM:261(MB) EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/wp-admin/admin-ajax.php
Oct 23 07:08:41 leadhero lfd[47294]: *Suspicious Process* PID:46605 PPID:41522 User:herolead Uptime:114 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:08:41 leadhero lfd[47294]: *Suspicious Process* PID:46855 PPID:35891 User:herolead Uptime:75 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:09:41 leadhero lfd[47837]: *Suspicious Process* PID:47213 PPID:29156 User:herolead Uptime:75 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:09:41 leadhero lfd[47837]: *User Processing* PID:47657 Kill:0 User:herolead VM:273(MB) EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/wp-admin/admin.php
Oct 23 07:10:41 leadhero lfd[48098]: *Suspicious Process* PID:47758 PPID:44027 User:herolead Uptime:75 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:10:42 leadhero lfd[48098]: *User Processing* PID:48004 Kill:0 User:herolead VM:277(MB) EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/wp-admin/admin-ajax.php
Oct 23 07:11:42 leadhero lfd[48550]: *Suspicious Process* PID:48055 PPID:47692 User:herolead Uptime:75 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:11:42 leadhero lfd[48546]: *Email Queue* The exim delivery queue size is 85987
Oct 23 07:12:42 leadhero lfd[48834]: *Suspicious Process* PID:48498 PPID:46614 User:herolead Uptime:76 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:13:42 leadhero lfd[49101]: *Suspicious Process* PID:48785 PPID:43139 User:herolead Uptime:76 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:13:57 leadhero lfd[49334]: *Exceeded LOCALRELAY limit* from nfp (101 in the last hour)
Oct 23 07:14:42 leadhero lfd[49470]: *Suspicious Process* PID:48977 PPID:44027 User:herolead Uptime:76 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:15:42 leadhero lfd[49635]: *Suspicious Process* PID:49444 PPID:41522 User:herolead Uptime:76 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:15:42 leadhero lfd[49635]: *User Processing* PID:49628 Kill:0 User:herolead VM:245(MB) EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/wp-admin/admin-ajax.php
Oct 23 07:16:42 leadhero lfd[50157]: *Suspicious Process* PID:49596 PPID:29156 User:herolead Uptime:76 secs EXE:/usr/bin/php CMD:/usr/bin/php /home/herolead/public_html/thailand/index.php
Oct 23 07:16:43 leadhero lfd[50155]: *Suspicious File* /tmp/index.php [leadhero:leadhero (508:506)] - Script, file extension
You can try to scan the website using https://wordpress.org/plugins/wordfence/ scan feature. I think there are some malwares inside your website. The Wordfence scan feature can detect common malwares and it's already helped me for many times.
Regards,
Adi
Provided logs are LFD loges which are showing your site is using some extra memory and taking some time to execute php process on your server. I will suggest you please check your site with http://sitecheck.sucuri.net/ and install BulletProof Security OR Wordfence Security plugins in your account to secure your site.
Also I can see you have root access so try to scan your account using maldet.
Related
I'm scraping PDF reports for their data.
I'm trying to extract the location the report is based off. I've got a character string with the location, and then a rolling 13 months header seen here:
header_line <- "Corp Dec '20 Jan '21 Feb '21 Mar '21 Apr '21 May '21 Jun '21 Jul '21 Aug '21 Sep '21 Oct '21 Nov '21 Dec '21"
I'd like to extract all characters from the beginning of the string to the start of WHATEVER month could be appearing after Because it's a rolling 13-month report, it could be any of those months abbreviations next to the location.
I have this working for the above example, but I'm not sure how to create an "Or pattern" with regex. I know I could brute force it with a loop or apply function, but I was hoping there was a less dirty way.
stringr::str_extract(header_line, "[^Dec]+")
[1] "Corp "
It is difficult to anticipate the possible cases that the location could be, but the below solution may cover most of it. It will match everything prior to 3 alphabetical characters, followed by a space, and apostrophe, and 2 digits.
str_extract(header_line, '^(.*?)(?=[a-zA-Z]{3}\\s\'\\d{2})')
Test cases:
header_line <- "Corp Dec '20 Jan '21 Feb '21 Mar '21 Apr '21 May '21 Jun '21 Jul '21 Aug '21 Sep '21 Oct '21 Nov '21 Dec '21"
header_line2 <- "Corp multiple words Dec '20 Jan '21 Feb '21 Mar '21 Apr '21 May '21 Jun '21 Jul '21 Aug '21 Sep '21 Oct '21 Nov '21 Dec '21"
header_line3 <- "Corp multiple words 1 Dec '20 Jan '21 Feb '21 Mar '21 Apr '21 May '21 Jun '21 Jul '21 Aug '21 Sep '21 Oct '21 Nov '21 Dec '21"
header_line4 <- "Corp multiple 444 Dec '20 Jan '21 Feb '21 Mar '21 Apr '21 May '21 Jun '21 Jul '21 Aug '21 Sep '21 Oct '21 Nov '21 Dec '21"
str_extract(header_line, '^(.*?)(?=[a-zA-Z]{3}\\s\'\\d{2})')
[1] "Corp"
str_extract(header_line2, '^(.*?)(?=[a-zA-Z]{3}\\s\'\\d{2})')
[1] "Corp multiple words"
str_extract(header_line3, '^(.*?)(?=[a-zA-Z]{3}\\s\'\\d{2})')
[1] "Corp multiple words 1"
str_extract(header_line4, '^(.*?)(?=[a-zA-Z]{3}\\s\'\\d{2})')
[1] "Corp multiple 444"
I've been trying to set up an OpenVPN server on my Linux recently but I continuously get the same error every time I try to connect to my server.
My settings are like this:
proto tcp
port 443
resolv-retry infinite
nobind
user nobody
group nogroup
cipher AES-256-CBC
auth SHA256
script-security 2
up /etc/openvpn/update-systemd-resolved
down /etc/openvpn/update-systemd-resolved
down-pre
dhcp-option DOMAIN-ROUTE .
I have checked the settings on my server and local computer a million times and all of them are the same. Still don't know what I have to do about it. Thanks in advance! :*
Sat Nov 27 23:45:11 2021 OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 19 2021
Sat Nov 27 23:45:11 2021 library versions: OpenSSL 1.1.1f 31 Mar 2020, LZO 2.10
Sat Nov 27 23:45:11 2021 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Sat Nov 27 23:45:11 2021 TCP/UDP: Preserving recently used remote address: [AF_INET]myserverip:443
Sat Nov 27 23:45:11 2021 Socket Buffers: R=[131072->131072] S=[16384->16384]
Sat Nov 27 23:45:11 2021 Attempting to establish TCP connection with [AF_INET]myserverip:443 [nonblock]
Sat Nov 27 23:45:12 2021 TCP connection established with [AF_INET]myserverip:443
Sat Nov 27 23:45:12 2021 TCP_CLIENT link local: (not bound)
Sat Nov 27 23:45:12 2021 TCP_CLIENT link remote: [AF_INET]myserverip:443
Sat Nov 27 23:45:12 2021 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
Sat Nov 27 23:45:12 2021 Connection reset, restarting [0]
Sat Nov 27 23:45:12 2021 SIGUSR1[soft,connection-reset] received, process restarting
Sat Nov 27 23:45:12 2021 Restart pause, 5 second(s)
cat sample_file.txt(Extracted job info from Control-M)
upctm,pmdw_bip,pmdw_bip_mnt_35-FOLDistAutoRpt,Oct 7 2019 4:45 AM,Oct 7 2019 4:45 AM,1,1,Oct 6 2019 12:00 AM,Ended OK,3ppnc
upctm,pmdw_ddm,pmdw_ddm_dum_01-StartProjDCSDemand,Oct 17 2019 4:02 AM,Oct 17 2019 4:02 AM,3,1,Oct 16 2019 12:00 AM,Ended OK,3pqgq
I need to process this file into DB table(Oracle)
Bu I need to make sure that day is 2 number (example 7 to 07).
(example: Oct 07 2019 6:32 AM)
I used this command to get all the date in every line:
cat sample_file.txt | grep "," | while read line
do
l_start_date=`echo $line|cut -d ',' -f4`
l_end_date=`echo $line|cut -d ',' -f5`
l_order_date=`echo $line|cut -d ',' -f8`
echo $l_start_date
echo $l_end_date
echo $l_order_date
done
Output:
Oct 7 2019 4:45 AM
Oct 7 2019 4:45 AM
Oct 6 2019 12:00 AM
Oct 17 2019 4:02 AM
Oct 17 2019 4:02 AM
Oct 16 2019 12:00 AM
expected output:
FROM: Oct 7 2019 6:32 AM
To: Oct 07 2019 6:32 AM
I used this sed command but it add also to 2 number day (17)
sed command sed 's|,Oct |,Oct 0|g' sample_file.txt
Oct 17 was change to Oct 017
upctm,pmdw_bip,pmdw_bip_mnt_35-FOLDistAutoRpt,Oct 07 2019 4:45 AM,Oct 07 2019 4:45 AM,1,1,Oct 06 2019 12:00 AM,Ended OK,3ppnc
upctm,pmdw_ddm,pmdw_ddm_dum_01-StartProjDCSDemand,Oct 017 2019 4:02 AM,Oct 017 2019 4:02 AM,3,1,Oct 016 2019 12:00 AM,Ended OK,3pqgq
I wish it was easier, but I only managed the following:
awk.f:
function fmt(s) {
split(s,a," "); a[2]=substr(a[2]+100,2)
return a[1] " " a[2] " "a[3] " " a[4] " " a[5]
}
BEGIN {FS=",";OFS=","}
{gsub(/ +/," ");
$4=fmt($4); $5=fmt($5); $8=fmt($8);
print}
This is a little awk script that first removes superfluous blanks and then picks out particular columns (4,5 and 8) and reformats the second part of each date string into a two-digit number.
You run the script like this:
awk -f f.awk sample_file.txt
output:
upctm,pmdw_aud,pmdw_aud_ext_06-GAPAnalysYTD,Oct 07 2019 6:32 AM,Oct 07 2019 6:32 AM,17,17,Oct 06 2019 12:00 AM,Ended OK,3pu9v
upctm,pmdw_ddm,pmdw_ddm_dum_01-StartProjDCSDemand,Oct 07 2019 4:02 AM,Oct 07 2019 4:02 AM,3,1,Oct 06 2019 12:00 AM,Ended OK,3pqgq
upctm,pmdw_bip,pmdw_bip_mnt_35-FOLDistAutoRpt,Oct 07 2019 4:45 AM,Oct 07 2019 4:45 AM,1,1,Oct 06 2019 12:00 AM,Ended OK,3ppnc
With a fixed locale, you can make a fixed replacement like
sed -r 's/(Jan|Feb|Oct|Whatever) ([1-9]) /\1 0\2 /g' sample_file.txt
I found this information on how to use the $push operator to add new values to an array. However, I can't seem to get this working with rmongodb
Suppose we have the following doc in the DB
_id : 7 51005201f8ab44f1690f9526
tags : 4
1 : 2 a
2 : 2 b
3 : 2 c
I'd like to add a value to the array tags. Here's what I tried:
q <- mongo.bson.from.list(list(tags="a"))
TRY 1
Here I tried using the $push operator
Code
bnew <- mongo.bson.from.list(list("$push"=list("tags"="d")))
> mongo.update(mongo=con, ns, criteria=q, objNew=bnew)
[1] FALSE
Logfile
Thu Jan 24 16:42:27 [initandlisten] MongoDB starting : pid=6260 port=27017 dbpath=\data\db\ 64-bit host=ASHB-109C-02
Thu Jan 24 16:42:27 [initandlisten] db version v2.2.2, pdfile version 4.5
Thu Jan 24 16:42:27 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Thu Jan 24 16:42:27 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Thu Jan 24 16:42:27 [initandlisten] options: { logpath: "log_1.txt" }
Thu Jan 24 16:42:27 [initandlisten] journal dir=/data/db/journal
Thu Jan 24 16:42:27 [initandlisten] recover : no journal files present, no recovery needed
Thu Jan 24 16:42:27 [initandlisten] waiting for connections on port 27017
Thu Jan 24 16:42:27 [websvr] admin web console waiting for connections on port 28017
Thu Jan 24 16:42:36 [initandlisten] connection accepted from 127.0.0.1:52419 #1 (1 connection now open)
Thu Jan 24 16:42:44 [conn1] __test.test Assertion failure x == _nfields src\mongo\db\jsobj.cpp 1250
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(109) mongo::verifyFailed+0xdc
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\jsobj.cpp(1250) mongo::BSONIteratorSorted::BSONIteratorSorted+0xf3
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\ops\update_internal.cpp(906) mongo::ModSetState::createNewFromMods+0xa3
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\ops\update.cpp(370) mongo::_updateObjects+0x15a2
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(573) mongo::receivedUpdate+0x60d
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(437) mongo::assembleResponse+0x626
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Thu Jan 24 16:42:44 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Thu Jan 24 16:42:44 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Thu Jan 24 16:42:44 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Thu Jan 24 16:42:44 [conn1] update __test.test query: { tags: "a" } update: { $push: { tags: "d" } } nscanned:1 keyUpdates:0 exception: assertion src\mongo\db\jsobj.cpp:1250 locks(micros) w:398335 399ms
Thu Jan 24 16:42:48 CTRL_CLOSE_EVENT signal
Thu Jan 24 16:42:48 [consoleTerminate] got CTRL_CLOSE_EVENT, will terminate after current cmd ends
Thu Jan 24 16:42:48 [consoleTerminate] now exiting
Thu Jan 24 16:42:48 dbexit:
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to close listening sockets...
Thu Jan 24 16:42:48 [consoleTerminate] closing listening socket: 496
Thu Jan 24 16:42:48 [consoleTerminate] closing listening socket: 516
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to flush diaglog...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to close sockets...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: waiting for fs preallocator...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: lock for final commit...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: final commit...
Thu Jan 24 16:42:48 [conn1] end connection 127.0.0.1:52419 (0 connections now open)
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: closing all files...
Thu Jan 24 16:42:48 [consoleTerminate] closeAllFiles() finished
Thu Jan 24 16:42:48 [consoleTerminate] journalCleanup...
Thu Jan 24 16:42:48 [consoleTerminate] removeJournalFiles
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: removing fs lock...
Thu Jan 24 16:42:48 dbexit: really exiting now
TRY 2
Here I tried using the $addToSet operator
Code
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.object(buf, "$addToSet")
mongo.bson.buffer.start.object(buf, name="tags")
mongo.bson.buffer.start.array(buf, "$each")
values <- list("d", "e", "f")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
bnew <- mongo.bson.from.buffer(buf)
bnew
> mongo.update(mongo=con, ns, criteria=q, objNew=bnew)
[1] FALSE
Logfile
Thu Jan 24 16:43:52 [initandlisten] MongoDB starting : pid=4184 port=27017 dbpath=\data\db\ 64-bit host=ASHB-109C-02
Thu Jan 24 16:43:52 [initandlisten] db version v2.2.2, pdfile version 4.5
Thu Jan 24 16:43:52 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Thu Jan 24 16:43:52 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Thu Jan 24 16:43:52 [initandlisten] options: { logpath: "log_2.txt" }
Thu Jan 24 16:43:52 [initandlisten] journal dir=/data/db/journal
Thu Jan 24 16:43:52 [initandlisten] recover : no journal files present, no recovery needed
Thu Jan 24 16:43:52 [initandlisten] waiting for connections on port 27017
Thu Jan 24 16:43:52 [websvr] admin web console waiting for connections on port 28017
Thu Jan 24 16:43:57 [initandlisten] connection accepted from 127.0.0.1:52435 #1 (1 connection now open)
Thu Jan 24 16:44:27 [conn1] __test.test Assertion failure x == _nfields src\mongo\db\jsobj.cpp 1250
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(109) mongo::verifyFailed+0xdc
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\jsobj.cpp(1250) mongo::BSONIteratorSorted::BSONIteratorSorted+0xf3
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\ops\update_internal.cpp(906) mongo::ModSetState::createNewFromMods+0xa3
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\ops\update.cpp(370) mongo::_updateObjects+0x15a2
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(573) mongo::receivedUpdate+0x60d
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(437) mongo::assembleResponse+0x626
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Thu Jan 24 16:44:28 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Thu Jan 24 16:44:28 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Thu Jan 24 16:44:28 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Thu Jan 24 16:44:28 [conn1] update __test.test query: { tags: "a" } update: { $addToSet: { tags: { $each: [ "d", "e", "f" ] } } } nscanned:1 keyUpdates:0 exception: assertion src\mongo\db\jsobj.cpp:1250 locks(micros) w:390312 390ms
Thu Jan 24 16:44:33 [conn1] end connection 127.0.0.1:52435 (0 connections now open)
Thu Jan 24 16:44:37 CTRL_CLOSE_EVENT signal
Thu Jan 24 16:44:37 [consoleTerminate] got CTRL_CLOSE_EVENT, will terminate after current cmd ends
Thu Jan 24 16:44:37 [consoleTerminate] now exiting
Thu Jan 24 16:44:37 dbexit:
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to close listening sockets...
Thu Jan 24 16:44:37 [consoleTerminate] closing listening socket: 496
Thu Jan 24 16:44:37 [consoleTerminate] closing listening socket: 500
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to flush diaglog...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to close sockets...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: waiting for fs preallocator...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: lock for final commit...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: final commit...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: closing all files...
Thu Jan 24 16:44:37 [consoleTerminate] closeAllFiles() finished
Thu Jan 24 16:44:37 [consoleTerminate] journalCleanup...
Thu Jan 24 16:44:37 [consoleTerminate] removeJournalFiles
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: removing fs lock...
Thu Jan 24 16:44:37 dbexit: really exiting now
What am I doing wrong here?
Additional information
For those interested: here's the code that produced the example doc
pkg <- "rmongodb"
lib <- file.path(R.home(), "library")
if (!suppressWarnings(require(pkg, lib.loc=lib, character.only=TRUE))) {
install.packages(pkg, lib=lib)
require(pkg, lib.loc=lib, character.only=TRUE)
}
# CONNECTION
db <- "__test"
ns <- paste(db, "test", sep=".")
con <- mongo.create(db=db)
# ENSURE EMPTY DB
mongo.remove(mongo=con, ns=ns)
# INSERT
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
b <- mongo.bson.from.buffer(buf)
mongo.insert(mongo=con, ns=ns, b=b)
EDIT 2013-01-29
As suggested by Tad Marshall from 10gen in his comment to my bug report, I re-ran the code that inserts the document with the MongoDB server running in --objcheck mode (validates BSON structures) and voilà: the server won't let me insert the doc due to an assertion that fails. If I run the server without the --objcheck flag, insertion is successful (but that's probably just due to the fact that no validation takes place).
Note that I tried two different versions of putting together the array in tags as my initial code produced a doc that IMHO is not in sync with MongoDB's indexing conventions:
(Potentially) Invalid document
That's how I did it above. I noticed that I didn't make sure the array index starts with a 0. Insertion of this document will fail (see logfile below)
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf) # finish array 'tags'
mongo.bson.buffer.finish.object(buf) # finish buffer
b <- mongo.bson.from.buffer(buf)
> b
tags : 4
1 : 2 a
2 : 2 b
3 : 2 c
Valid document
I made sure the index starts with 0, so this should definitely be a valid BSON doc. But inserting this document will fail, too (see logfile below)
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii-1),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf) # finish array 'tags'
mongo.bson.buffer.finish.object(buf) # finish buffer
b <- mongo.bson.from.buffer(buf)
b
mongo.insert(mongo=con, ns=ns, b=b)
> b
tags : 4
0 : 2 a
1 : 2 b
2 : 2 c
Logfile
Tue Jan 29 14:20:46 [initandlisten] MongoDB starting : pid=6440 port=27017
[...]
Tue Jan 29 14:20:59 [initandlisten] connection accepted from 127.0.0.1:62137 #1 (1 connection now open)
Tue Jan 29 14:21:03 [conn1] Assertion: 10307:Client Error: bad object in message
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(154) mongo::msgasserted+0xc1
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\dbmessage.h(205) mongo::DbMessage::nextJsObj+0x103
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(784) mongo::receivedInsert+0xdb
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(434) mongo::assembleResponse+0x607
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Tue Jan 29 14:21:04 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Tue Jan 29 14:21:04 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Tue Jan 29 14:21:04 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Tue Jan 29 14:21:04 [conn1] insert __test.test keyUpdates:0 exception: Client Error: bad object in message code:10307 0ms
Tue Jan 29 14:21:07 [conn1] Assertion: 10307:Client Error: bad object in message
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(154) mongo::msgasserted+0xc1
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\dbmessage.h(205) mongo::DbMessage::nextJsObj+0x103
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(784) mongo::receivedInsert+0xdb
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(434) mongo::assembleResponse+0x607
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Tue Jan 29 14:21:07 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Tue Jan 29 14:21:07 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Tue Jan 29 14:21:07 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Tue Jan 29 14:21:07 [conn1] insert __test.test keyUpdates:0 exception: Client Error: bad object in message code:10307 0ms
Oh, I am smacking myself up now. I didn't look closely at the way you were creating your document. You have two mongo.bson.finish.object() calls when you need only one to finish off the array you started. You do not need to call it to finish a BSON. mongo.bson.from.buffer() does the necessary housekeeping. This is my fault for not reading your code closely enough. I thought it was your update failing when the initial insert of the documents is the problem. For questions here in the future, it would help if your examples were a little easier to read. For instance, this will build the document:
library('rmongodb')
m = mongo.create()
ns = '__test.test'
mongo.insert(m, ns, list(tags=c('a', 'b', 'c'))
However, you are probably pasting in real-world code so I understand where the complications come in. Everything's cool. Just beating myself up for missing this and sending you on a wild goose chase. Regards
Rappster, both of these examples worked for me on my development machine, but I am slightly out of date running a debug build of mongod 2.1.0.
Since you are crashing the server with your example code, this is something the 10gen people will want to know about. Do you mind going to https://jira.mongodb.org/secure/Dashboard.jspa and reporting this bug?
Thanks,
Gerald Lindsly
Here's some of my data, read in from a file names AttReport_all:
Registration.Date Join.Time Leave.Time
1 Jul 05, 2011 09:30 PM EDT Jul 07, 2011 01:05 PM EDT Jul 07, 2011 01:53 PM EDT
2 Jul 05, 2011 10:20 AM EDT Jul 07, 2011 01:04 PM EDT Jul 07, 2011 01:53 PM EDT
3 Jul 04, 2011 02:41 PM EDT Jul 07, 2011 12:49 PM EDT Jul 07, 2011 01:53 PM EDT
4 Jul 04, 2011 11:38 PM EDT Jul 07, 2011 12:49 PM EDT Jul 07, 2011 01:54 PM EDT
5 Jul 05, 2011 11:41 AM EDT Jul 07, 2011 12:54 PM EDT Jul 07, 2011 01:54 PM EDT
6 Jul 07, 2011 11:08 AM EDT Jul 07, 2011 01:16 PM EDT Jul 07, 2011 01:53 PM EDT
If I do strptime(AttReport_all$Registration.Date, "%b %m, %Y %H:%M %p", tz="") I get an array of NAs where I'm expecting dates.
Sys.setlocale("LC_TIME", "C") returns "C"
typeof(AttReport_all$Registration.Date) returns "integer"
is.factor(AttReport_all$Registration.Date) returns TRUE.
What am I missing?
Here's version output, if it helps:
platform i386-pc-mingw32
arch i386
os mingw32
system i386, mingw32
status
major 2
minor 13.0
year 2011
month 04
day 13
svn rev 55427
language R
version.string R version 2.13.0 (2011-04-13)
strptime automatically runs as.character on the first argument (so it doesn't matter that it's a factor) and any trailing characters not specified in format= are ignored (so "EDT" doesn't matter).
The only issues are the typo #Ben Bolker identified (%m should be %d) and %H should be %I (?strptime says you should not use %H with %p).
# %b and %m are both *month* formats
strptime("Jul 05, 2011 09:30 PM EDT", "%b %m, %Y %H:%M %p", tz="")
# [1] NA
# change %m to %d and we no longer get NA, but the time is wrong (AM, not PM)
strptime("Jul 05, 2011 09:30 PM EDT", "%b %d, %Y %H:%M %p", tz="")
# [1] "2011-07-05 09:30:00"
# use %I (not %H) with %p
strptime("Jul 05, 2011 09:30 PM EDT", "%b %d, %Y %I:%M %p", tz="")
# [1] "2011-07-05 21:30:00"