Related
We have maxwell running on docker in our production and it is configured to listen to changes on MariadDb tables and push them to a kafka topic. Our MariadDb has some 10-12 tables and we have configured maxwell to listen to only 4 tables and all these 4 tables are heavily used (number of writes is too high).
Recently we have seen that our Maxwell is going down, following are the few error logs we found when maxwell is going down
Oct 28, 2022 # 06:10:40.037 06:10:40,037 ERROR MysqlParserListener - (parse SET STATEMENT max_statement_time = 60 FOR flush table)
Oct 28, 2022 # 06:10:40.040 06:10:40,040 ERROR SchemaChange - Error parsing SQL: 'SET STATEMENT max_statement_time=60 FOR flush table'
Oct 28, 2022 # 06:10:40.047 06:10:40,045 ERROR AbstractSchemaStore - Error on bin log position Position[BinlogPosition[mysql-bin-changelog.049610:3209], lastHeartbeat=1666898518069]
As the last error log says Error on bin log position Position[BinlogPosition[mysql-bin-changelog.049610:3209], so to verify whether bin log file with name mysql-bin-changelog.049610:3209 exists or not we ran SHOW binary logs; and we found that file in list outputted by above command
Please help me in understanding what is going wrong here and help in solving the issue
Environment Details
OS: Centos
Platform - AWS instance(T4g Medium)
CPU - 2 CPU
RAM - 4GB
Maxwell Version - v1.37.7
Note: We are not handling DDL & our log retention is 3 days
Detailed logs that was logged when Maxwell was going down, sorry for posting huge logs :)
Oct 28, 2022 # 06:10:39.084 06:10:39,082 INFO ProducerConfig - ProducerConfig values:
Oct 28, 2022 # 06:10:39.146 06:10:39,146 INFO AppInfoParser - Kafka commitId : aaa7af6d4a11b29d
Oct 28, 2022 # 06:10:39.146 06:10:39,146 INFO AppInfoParser - Kafka version : 1.0.0
Oct 28, 2022 # 06:10:39.188 06:10:39,187 INFO Maxwell - Maxwell v1.37.7 is booting (MaxwellKafkaProducer), starting at Position[BinlogPosition[mysql-bin-changelog.049610:3136], lastHeartbeat=1666898518069]
Oct 28, 2022 # 06:10:39.377 06:10:39,377 INFO MysqlSavedSchema - Restoring schema id 22 (last modified at Position[BinlogPosition[mysql-bin-changelog.046940:91534], lastHeartbeat=1666103424529])
Oct 28, 2022 # 06:10:39.502 06:10:39,502 INFO MysqlSavedSchema - Restoring schema id 1 (last modified at Position[BinlogPosition[mysql-bin-changelog.040524:418929], lastHeartbeat=0])
Oct 28, 2022 # 06:10:39.555 06:10:39,555 INFO MysqlSavedSchema - beginning to play deltas...
Oct 28, 2022 # 06:10:39.576 06:10:39,576 INFO MysqlSavedSchema - played 21 deltas in 15ms
Oct 28, 2022 # 06:10:39.601 06:10:39,601 INFO BinlogConnectorReplicator - Setting initial binlog pos to: mysql-bin-changelog.049610:3136
Oct 28, 2022 # 06:10:39.605 06:10:39,605 INFO MaxwellHTTPServer - Maxwell http server starting
Oct 28, 2022 # 06:10:39.607 06:10:39,607 INFO MaxwellHTTPServer - Maxwell http server started on port 8080
Oct 28, 2022 # 06:10:39.649 06:10:39,649 INFO BinlogConnectorReplicator - Binlog connected.
Oct 28, 2022 # 06:10:39.649 06:10:39,646 INFO BinaryLogClient - Connected to mariadb-nlp.app.engati.local:3306 at mysql-bin-changelog.049610/3136 (sid:6379, cid:1560183)
Oct 28, 2022 # 06:10:39.669 06:10:39,669 INFO log - Logging initialized #2288ms to org.eclipse.jetty.util.log.Slf4jLog
Oct 28, 2022 # 06:10:39.902 06:10:39,902 INFO Server - jetty-9.4.41.v20210516; built: 2021-05-16T23:56:28.993Z; git: 98607f93c7833e7dc59489b13f3cb0a114fb9f4c; jvm 11.0.15+10
Oct 28, 2022 # 06:10:40.037 06:10:40,037 ERROR MysqlParserListener - (parse SET STATEMENT max_statement_time = 60 FOR flush table)
Oct 28, 2022 # 06:10:40.040 06:10:40,040 ERROR SchemaChange - Error parsing SQL: 'SET STATEMENT max_statement_time=60 FOR flush table'
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.Maxwell.main(Maxwell.java:336)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.Maxwell.start(Maxwell.java:227)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.Maxwell.startInner(Maxwell.java:301)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.util.RunLoopProcess.runLoop(RunLoopProcess.java:34)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.work(BinlogConnectorReplicator.java:228)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.getRow(BinlogConnectorReplicator.java:723)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:400)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:378)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.schema.MysqlSchemaStore.processSQL(MysqlSchemaStore.java:102)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.schema.AbstractSchemaStore.resolveSQL(AbstractSchemaStore.java:49)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:108)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.schema.ddl.SchemaChange.parseSQL(SchemaChange.java:94)
Oct 28, 2022 # 06:10:40.047 at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
Oct 28, 2022 # 06:10:40.047 at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:17)
Oct 28, 2022 # 06:10:40.047 at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:93)
Oct 28, 2022 # 06:10:40.047 com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: SET
Oct 28, 2022 # 06:10:40.047 06:10:40,045 ERROR AbstractSchemaStore - Error on bin log position Position[BinlogPosition[mysql-bin-changelog.049610:3209], lastHeartbeat=1666898518069]
Oct 28, 2022 # 06:10:40.047 06:10:40,044 INFO ContextHandler - Started o.e.j.s.ServletContextHandler#5628733f{/,null,AVAILABLE}
Oct 28, 2022 # 06:10:40.075 06:10:40,074 INFO Server - Started #2700ms
Oct 28, 2022 # 06:10:40.075 06:10:40,074 INFO AbstractConnector - Started ServerConnector#7152d3c1{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
Oct 28, 2022 # 06:10:40.129 06:10:40,129 INFO BinlogConnectorReplicator - Binlog disconnected.
Oct 28, 2022 # 06:10:40.149 06:10:40,149 INFO TaskManager - Stopping 5 tasks
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.schema.ddl.SchemaChange.parseSQL(SchemaChange.java:94) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28) ~[antlr4-runtime-4.8-1.jar:4.8-1]
Oct 28, 2022 # 06:10:40.187 at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:17) ~[antlr4-runtime-4.8-1.jar:4.8-1]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:93) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: SET
Oct 28, 2022 # 06:10:40.187 06:10:40,152 ERROR TaskManager - cause:
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.Maxwell.main(Maxwell.java:336) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.Maxwell.start(Maxwell.java:227) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.Maxwell.startInner(Maxwell.java:301) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.util.RunLoopProcess.runLoop(RunLoopProcess.java:34) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.work(BinlogConnectorReplicator.java:228) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.getRow(BinlogConnectorReplicator.java:723) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:400) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:378) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.schema.MysqlSchemaStore.processSQL(MysqlSchemaStore.java:102) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.schema.AbstractSchemaStore.resolveSQL(AbstractSchemaStore.java:49) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.187 at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:108) ~[maxwell-1.37.7.jar:1.37.7]
Oct 28, 2022 # 06:10:40.189 06:10:40,189 INFO TaskManager - Stopping: com.zendesk.maxwell.schema.PositionStoreThread#51b3ee1b
Oct 28, 2022 # 06:10:40.191 06:10:40,189 INFO TaskManager - Stopping: com.zendesk.maxwell.producer.MaxwellKafkaProducerWorker#5ec6ffb6
Oct 28, 2022 # 06:10:40.192 06:10:40,192 INFO KafkaProducer - [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
Oct 28, 2022 # 06:10:40.201 06:10:40,201 INFO TaskManager - Stopping: com.zendesk.maxwell.monitoring.MaxwellHTTPServerWorker#1490968e
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.Maxwell.main(Maxwell.java:336)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.Maxwell.start(Maxwell.java:227)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.Maxwell.startInner(Maxwell.java:301)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.util.RunLoopProcess.runLoop(RunLoopProcess.java:34)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.work(BinlogConnectorReplicator.java:228)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.getRow(BinlogConnectorReplicator.java:723)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:400)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.replication.BinlogConnectorReplicator.processQueryEvent(BinlogConnectorReplicator.java:378)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.schema.MysqlSchemaStore.processSQL(MysqlSchemaStore.java:102)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.schema.AbstractSchemaStore.resolveSQL(AbstractSchemaStore.java:49)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:108)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.schema.ddl.SchemaChange.parseSQL(SchemaChange.java:94)
Oct 28, 2022 # 06:10:40.201 at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:28)
Oct 28, 2022 # 06:10:40.201 at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:17)
Oct 28, 2022 # 06:10:40.201 at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:93)
Oct 28, 2022 # 06:10:40.201 com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: SET
Oct 28, 2022 # 06:10:40.201 06:10:40,199 INFO TaskManager - Stopping: com.zendesk.maxwell.replication.BinlogConnectorReplicator#5a1d68e0
Oct 28, 2022 # 06:10:40.201 06:10:40,198 INFO TaskManager - Stopping: com.zendesk.maxwell.bootstrap.BootstrapController#3857c48d
Oct 28, 2022 # 06:10:40.220 06:10:40,219 INFO AbstractConnector - Stopped ServerConnector#7152d3c1{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
Oct 28, 2022 # 06:10:40.223 06:10:40,223 INFO ContextHandler - Stopped o.e.j.s.ServletContextHandler#5628733f{/,null,STOPPED}
ct 28, 2022 # 06:10:41.637 06:10:41,636 INFO TaskManager - Stopped all tasks
The java backtrace shows this is an error in Maxwell. This was reported as issue 1931.
The issue is that Maxwell in the v1.37.7 doesn't parse SET STATEMENT ... FOR against a blacklist of SQL like FLUSH that its meant to take no action on.
Due to a very responsive upstream, this is is now fixed in v1.39.2.
I would like to achieve the following task. Using a linear mixed model, I would like to check whether "Month" (see dat table) has a significant effect on the "Response" variable. As for some of the tanks, data comes from different months, I included it as a random factor in my model. Please note, that sampling the same tank in different months does not change the "Response" variable. For some tank-month combinations there are multiple records, as we are included the compartment of the tank that was sampled (e.g. NW =north west).
Here the data:
print(dat)
Tank Month ID Response
1 AEW1 Jul AEW01SOBFJul2008 1.80522937
2 AEW10 Jul AEW10NWBFJul2008 2.13374401
3 AEW10 Jul AEW10NWBFJul2008 2.13374401
4 AEW11 Jun AEW11SWBFJun2008 1.65010205
5 AEW14 Jun AEW14SWBFJun2008 1.75459326
6 AEW15 Jun AEW15SOBFJun2008 2.82200903
7 AEW15 Jun AEW15SOBFJun2008 2.82200903
8 AEW18 Jul AEW18SOBFJul2008 0.39349330
9 AEW19 Jul AEW19NWBFJul2008 0.65886661
10 AEW20 Jul AEW20NWBFJul2008 1.07838018
11 AEW24 Jun AEW24NOBFJun2008 2.56677635
12 AEW27 Jul AEW27SWBFJul2008 2.64019328
13 AEW27 Jul AEW27SWBFJul2008 2.64019328
14 AEW29 Jul AEW29SOBFJul2008 2.06251217
15 AEW30 Jul AEW30NWBFJul2008 1.17010646
16 AEW31 Jun AEW31SWBFJun2008 2.25518873
17 AEW32 Jun AEW32SOBFJun2008 2.38707614
18 AEW33 Jun AEW33SOBFJun2008 2.30498448
19 AEW33 Jun AEW33SOBFJun2008 2.30498448
20 AEW36 Jul AEW36NOBFJul2008 1.92368247
21 AEW37 Jun AEW37NOBFJun2008 0.99387013
22 AEW39 Jul AEW39NOBFJul2008 1.24163732
23 AEW4 Jul AEW04SWBFJul2008 1.56327732
24 AEW42 Jun AEW42SWBFJun2008 1.26012579
25 AEW44 Jun AEW44SWBFJun2008 0.75985267
26 AEW48 Aug AEW48SOBFAug2008 1.57920494
27 AEW50 Jul AEW50NOBFJul2008 0.90052629
28 AEW8 Jul AEW08NOBFJul2008 0.00000000
29 AEW8 Jul AEW08NOBFJul2008 0.00000000
30 AEW9 Jul AEW09NOBFJul2008 0.48529647
31 HEW10 Jun HEW10SWBFJun2008 0.06412823
32 HEW10 Aug HEW10SOBFAug2008 0.06412823
33 HEW12 Jul HEW12NOBFJul2008 0.00000000
34 HEW13 Aug HEW13NWBFAug2008 2.24515850
35 HEW13 Jul HEW13SOBFJul2008 2.24515850
36 HEW13 Jul HEW13NOBFJul2008 2.24515850
37 HEW13 Jun HEW13SOBFJun2008 2.24515850
38 HEW13 Jun HEW13NWBFJun2008 2.24515850
39 HEW14 Jul HEW14SOBFJul2008 1.64783184
40 HEW18 Jun HEW18NWBFJun2008 1.32435721
41 HEW18 Jun HEW18NWBFJun2008 1.32435721
42 HEW19 Jul HEW19SWBFJul2008 1.01761003
43 HEW19 Jul HEW19SWBFJul2008 1.01761003
44 HEW22 Aug HEW22SWBFAug2008 0.63861037
45 HEW23 Jun HEW23SWBFJun2008 1.38472769
46 HEW23 Jun HEW23NWBFJun2008 1.38472769
47 HEW28 Jun HEW28NOBFJun2008 1.44377199
48 HEW3 Jun HEW03SWBFJun2008 2.19793633
49 HEW3 Jul HEW03SWBFJul2008 2.19793633
50 HEW30 Aug HEW30NWBFAug2008 0.76260579
51 HEW31 Jul HEW31SWBFJul2008 1.07879539
52 HEW35 Jun HEW35NWBFJun2008 0.86098152
53 HEW35 Jun HEW35NWBFJun2008 0.86098152
54 HEW36 Aug HEW36SOBFAug2008 0.36533352
55 HEW39 Jun HEW39SOBFJun2008 0.09283168
56 HEW4 Jun HEW04SWBFJun2008 1.89046783
57 HEW41 Aug HEW41NWBFAug2008 0.31996275
58 HEW41 Aug HEW41NWBFAug2008 0.31996275
59 HEW41 Jul HEW41NWBFJul2008 0.31996275
60 HEW41 Jul HEW41NWBFJul2008 0.31996275
61 HEW42 Jul HEW42NWBFJul2008 0.53998250
62 HEW43 Jun HEW43SWBFJun2008 1.85594061
63 HEW43 Jun HEW43SWBFJun2008 1.85594061
64 HEW44 Jun HEW44SOBFJun2008 1.79972095
65 HEW44 Jun HEW44SOBFJun2008 1.79972095
66 HEW49 Jun HEW49SWBFJun2008 1.25229249
67 HEW5 Aug HEW05SWBFAug2008 0.95559764
68 HEW50 Jun HEW50NWBFJun2008 0.42309531
69 HEW50 Jun HEW50NWBFJun2008 0.42309531
70 HEW7 Jul HEW07NWBFJul2008 0.69484213
71 HEW7 Jun HEW07NWBFJun2008 0.69484213
72 HEW8 Jul HEW08SWBFJul2008 1.15617440
73 SEW1 Aug SEW01NWBFAug2008 1.90030109
74 SEW1 Sep SEW01SWBFSep2008 1.90030109
75 SEW11 Aug SEW11NWBFAug2008 2.11940912
76 SEW12 Aug SEW12SOBFAug2008 2.29658624
77 SEW12 Jul SEW12SOBFJul2008 2.29658624
78 SEW17 Aug SEW17NOBFAug2008 1.49277937
79 SEW17 Jul SEW17NOBFJul2008 1.49277937
80 SEW17 Sep SEW17NOBFSep2008 1.49277937
81 SEW17 Aug SEW17SOBFAug2008 1.49277937
82 SEW18 Aug SEW18SOBFAug2008 1.70247509
83 SEW19 Aug SEW19SOBFAug2008 2.11617036
84 SEW20 Jul SEW20SWBFJul2008 1.87718089
85 SEW20 Jul SEW20SOBFJul2008 1.87718089
86 SEW22 Aug SEW22NOBFAug2008 0.77473833
87 SEW23 Aug SEW23NWBFAug2008 0.96183454
88 SEW23 Aug SEW23NOBFAug2008 0.96183454
89 SEW24 Jul SEW24SWBFJul2008 0.64090368
90 SEW24 Jul SEW24NWBFJul2008 0.64090368
91 SEW29 Jul SEW29SOBFJul2008 1.54699664
92 SEW29 Aug SEW29SWBFAug2008 1.54699664
93 SEW29 Aug SEW29SOBFAug2008 1.54699664
94 SEW34 Aug SEW34NWBFAug2008 1.79425003
95 SEW36 Jul SEW36SOBFJul2008 1.20337761
96 SEW4 Aug SEW04SWBFAug2008 1.59611963
97 SEW40 Sep SEW40SOBFSep2008 1.36486039
98 SEW40 Aug SEW40SWBFAug2008 1.36486039
99 SEW43 Sep SEW43SOBFSep2008 1.03169382
100 SEW44 Aug SEW44SWBFAug2008 0.79705660
101 SEW45 Jul SEW45NWBFJul2008 0.34130398
102 SEW46 Aug SEW46SOBFAug2008 0.20690386
103 SEW47 Aug SEW47SWBFAug2008 0.01564703
104 SEW47 Sep SEW47SWBFSep2008 0.01564703
105 SEW48 Aug SEW48SWBFAug2008 0.46745254
106 SEW5 Aug SEW05SWBFAug2008 0.68900435
107 SEW50 Aug SEW50NWBFAug2008 1.10731406
108 SEW7 Aug SEW07SWBFAug2008 0.08552432
109 SEW8 Jul SEW08NWBFJul2008 0.18731374
The model I generated so far is: Mod1 <- lmer(Response ~ Month + (1|Tank), data=dat)
Again, I included "Tank" because we sampled some tanks in several months but that does not change the response variable. Consequently, the response variable is fixed for each tank. Nevertheless, multiple data points originate from the same tank and I tried to account for that by including it as a random factor.
Fitting Mod1 results in the following message:
Warning messages:
1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 0.306567 (tol = 0.002, component 1)
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model is nearly unidentifiable: very large eigenvalue
- Rescale variables?
The question is now, whether the model is overly complex and whether I can drop "Tank" as a random factor, as measuring tanks repeatedly did not have an effect on the response variable.
Thus, the question is, would a simple linear model Mod1 <- lm(Response ~ Month, data =dat) be valid? And if not, how can I solve the 2 convergence issues.
Any help is very much appreciated! :)
I'm using Airflow 1.9 and installed it using "pip install apache-airflow[all]". while trying to use LDAP authentication, I'm seeing the below error.
Aug 6 10:26:07 test-airflow airflow: [2018-08-06 10:26:07,364] {__init__.py:64} CRITICAL - Cannot import authentication module airflow.contrib.auth.backends.ldap_auth. Please correct your authentication backend or disable authentication: cannot import name NoValue
Aug 6 10:26:07 test-airflow airflow: Traceback (most recent call last):
Aug 6 10:26:07 test-airflow airflow: File "/bin/airflow", line 27, in <module>
Aug 6 10:26:07 test-airflow airflow: args.func(args)
Aug 6 10:26:07 test-airflow airflow: File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 678, in webserver
Aug 6 10:26:07 test-airflow airflow: app = cached_app(conf)
Aug 6 10:26:07 test-airflow airflow: File "/usr/lib/python2.7/site-packages/airflow/www/app.py", line 161, in cached_app
Aug 6 10:26:07 test-airflow airflow: app = create_app(config)
Aug 6 10:26:07 test-airflow airflow: File "/usr/lib/python2.7/site-packages/airflow/www/app.py", line 44, in create_app
Aug 6 10:26:07 test-airflow airflow: airflow.load_login()
Aug 6 10:26:07 test-airflow airflow: File "/usr/lib/python2.7/site-packages/airflow/__init__.py", line 67, in load_login
Aug 6 10:26:07 test-airflow airflow: raise AirflowException("Failed to import authentication backend")
Aug 6 10:26:07 test-airflow airflow: airflow.exceptions.AirflowException: Failed to import authentication backend
Aug 6 10:26:07 test-airflow systemd: airflow-webserver.service: main process exited, code=exited, status=1/FAILURE
Aug 6 10:26:07 test-airflow systemd: Unit airflow-webserver.service entered failed state.
Aug 6 10:26:07 test-airflow systemd: airflow-webserver.service failed.
Here is my airflow.cfg:
[webserver]
authenticate = True
auth_backend = airflow.contrib.auth.backends.ldap_auth
base_url = http://test-airflow.example.com
I installed ldap3 to solve it.
pip install ldap3
Due to your limited log entries, your issue might be a bit different, but this is what I found when I encountered this issue:
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: [2018-08-21 04:56:45,150] {__init__.py:64} CRITICAL - Cannot import authentication module airflow.contrib.auth.backends.ldap_auth. Please correct your authentication backend or disable au
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: Traceback (most recent call last):
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/airflow/__init__.py", line 59, in load_login
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: login = import_module(auth_backend)
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: return _bootstrap._gcd_import(name[level:], package, level)
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "<frozen importlib._bootstrap>", line 986, in _gcd_import
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "<frozen importlib._bootstrap>", line 969, in _find_and_load
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "<frozen importlib._bootstrap_external>", line 665, in exec_module
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/airflow/contrib/auth/backends/ldap_auth.py", line 23, in <module>
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: from ldap3 import Server, Connection, Tls, LEVEL, SUBTREE, BASE
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/ldap3/__init__.py", line 138, in <module>
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: from .core.connection import Connection
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/ldap3/core/connection.py", line 37, in <module>
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: from ..extend import ExtendedOperationsRoot
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/ldap3/extend/__init__.py", line 45, in <module>
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: from .standard.whoAmI import WhoAmI
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/ldap3/extend/standard/whoAmI.py", line 27, in <module>
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: from pyasn1.type.univ import NoValue
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: ImportError: cannot import name 'NoValue'
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: During handling of the above exception, another exception occurred:
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: Traceback (most recent call last):
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/bin/airflow", line 27, in <module>
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: args.func(args)
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/airflow/bin/cli.py", line 678, in webserver
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: app = cached_app(conf)
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/airflow/www/app.py", line 161, in cached_app
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: app = create_app(config)
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/airflow/www/app.py", line 44, in create_app
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: airflow.load_login()
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: File "/usr/local/lib/python3.5/dist-packages/airflow/__init__.py", line 67, in load_login
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: raise AirflowException("Failed to import authentication backend")
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: airflow.exceptions.AirflowException: Failed to import authentication backend
Aug 21 04:56:45 ip-10-202-21-240 systemd[1]: airflow-webserver.service: Main process exited, code=exited, status=1/FAILURE
Aug 21 04:56:45 ip-10-202-21-240 systemd[1]: airflow-webserver.service: Unit entered failed state.
Aug 21 04:56:45 ip-10-202-21-240 systemd[1]: airflow-webserver.service: Failed with result 'exit-code'.
Note the following section:
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: from pyasn1.type.univ import NoValue
Aug 21 04:56:45 ip-10-202-21-240 airflow[10065]: ImportError: cannot import name 'NoValue'
It seems that the version of pyasn1 I had was old. Once I forced pip to download the latest version
sudo pip install pyasn1==0.4.4
That did the trick
Hello i am a newbie working with OpenStack, i have installed openstack + opencontrail by following (Single Node setup):
http://www.opencontrail.org/opencontrail-quick-start-guide/
Opencontrail loads fine (WebConsole), but when i launch horizon web console, i see "Something went wrong" error
I have read other question with the same error but the cause it is different
"NeutronClientException: 503 Service Unavailable
[Thu Jul 28 23:15:02 2016] [error] No server is available to handle this request.
"
Can you help me please?
This is the log message :
Internal Server Error: /horizon/admin/
[Thu Jul 28 23:15:02 2016] [error] Traceback (most recent call last):
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
[Thu Jul 28 23:15:02 2016] [error] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 38, in dec
[Thu Jul 28 23:15:02 2016] [error] return view_func(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 86, in dec
[Thu Jul 28 23:15:02 2016] [error] return view_func(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 54, in dec
[Thu Jul 28 23:15:02 2016] [error] return view_func(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 38, in dec
[Thu Jul 28 23:15:02 2016] [error] return view_func(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 86, in dec
[Thu Jul 28 23:15:02 2016] [error] return view_func(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/django/views/generic/base.py", line 69, in view
[Thu Jul 28 23:15:02 2016] [error] return self.dispatch(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/django/views/generic/base.py", line 87, in dispatch
[Thu Jul 28 23:15:02 2016] [error] return handler(request, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 154, in get
[Thu Jul 28 23:15:02 2016] [error] handled = self.construct_tables()
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 145, in construct_tables
[Thu Jul 28 23:15:02 2016] [error] handled = self.handle_table(table)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 118, in handle_table
[Thu Jul 28 23:15:02 2016] [error] data = self._get_data_dict()
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 181, in _get_data_dict
[Thu Jul 28 23:15:02 2016] [error] self._data = {self.table_class._meta.name: self.get_data()}
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/dashboards/admin/overview/views.py", line 60, in get_data
[Thu Jul 28 23:15:02 2016] [error] data = super(GlobalOverview, self).get_data()
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/usage/views.py", line 44, in get_data
[Thu Jul 28 23:15:02 2016] [error] self.usage.get_limits()
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/usage/base.py", line 193, in get_limits
[Thu Jul 28 23:15:02 2016] [error] self.get_neutron_limits()
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/usage/base.py", line 151, in get_neutron_limits
[Thu Jul 28 23:15:02 2016] [error] api.neutron.is_security_group_extension_supported(self.request)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/neutron.py", line 861, in is_security_group_extension_supported
[Thu Jul 28 23:15:02 2016] [error] return is_extension_supported(request, 'security-group')
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/utils/memoized.py", line 90, in wrapped
[Thu Jul 28 23:15:02 2016] [error] value = cache[key] = func(*args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/neutron.py", line 841, in is_extension_supported
[Thu Jul 28 23:15:02 2016] [error] extensions = list_extensions(request)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/horizon/utils/memoized.py", line 90, in wrapped
[Thu Jul 28 23:15:02 2016] [error] value = cache[key] = func(*args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/api/neutron.py", line 832, in list_extensions
[Thu Jul 28 23:15:02 2016] [error] extensions_list = neutronclient(request).list_extensions()
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 111, in with_params
[Thu Jul 28 23:15:02 2016] [error] ret = self.function(instance, *args, **kwargs)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 294, in list_extensions
[Thu Jul 28 23:15:02 2016] [error] return self.get(self.extensions_path, params=_params)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1236, in get
[Thu Jul 28 23:15:02 2016] [error] headers=headers, params=params)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1236, in get
[Thu Jul 28 23:15:02 2016] [error] headers=headers, params=params)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1221, in retry_request
[Thu Jul 28 23:15:02 2016] [error] headers=headers, params=params)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1164, in do_request
[Thu Jul 28 23:15:02 2016] [error] self._handle_fault_response(status_code, replybody)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1134, in _handle_fault_response
[Thu Jul 28 23:15:02 2016] [error] exception_handler_v20(status_code, des_error_body)
[Thu Jul 28 23:15:02 2016] [error] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20
[Thu Jul 28 23:15:02 2016] [error] message=message)
[Thu Jul 28 23:15:02 2016] [error] NeutronClientException: 503 Service Unavailable
[Thu Jul 28 23:15:02 2016] [error] No server is available to handle this request.
[Thu Jul 28 23:15:02 2016] [error]
This is the output when i checked if a neutron server was running on port:
tcp 0 0 0.0.0.0:9696 0.0.0.0:* LISTEN
And on this is the neutron server log file(There is not a request from horizon app):
2016-07-28 16:48:01.5 INFO [neutron.common.config] Config paste file: /etc/neutron/api-paste.ini
2016-07-28 16:48:05.168 INFO [neutron.manager] Loading core plugin: neutron_plugin_contrail.plugins.opencontrail.contrail_plugin.NeutronPluginContrailCoreV2
2016-07-28 16:48:06.252 INFO [neutron.manager] Service L3_ROUTER_NAT is supported by the core plugin
2016-07-28 16:48:06.252 INFO [neutron.manager] Loading Plugin: neutron_plugin_contrail.plugins.opencontrail.loadbalancer.plugin.LoadBalancerPlugin
2016-07-28 16:48:06.506 INFO [urllib3.connectionpool] Starting new HTTP connection (1): 172.16.12.22
Thanks
Looks like the contrail plugin is running fine (since Contrail web-UI works), but neutron-server is not.
Are the neutron commands working fine? (Ex : neutron net-list/net-create etc. ? )
Is the 'keystone catalog' command showing the right neutron service details ?
I found this information on how to use the $push operator to add new values to an array. However, I can't seem to get this working with rmongodb
Suppose we have the following doc in the DB
_id : 7 51005201f8ab44f1690f9526
tags : 4
1 : 2 a
2 : 2 b
3 : 2 c
I'd like to add a value to the array tags. Here's what I tried:
q <- mongo.bson.from.list(list(tags="a"))
TRY 1
Here I tried using the $push operator
Code
bnew <- mongo.bson.from.list(list("$push"=list("tags"="d")))
> mongo.update(mongo=con, ns, criteria=q, objNew=bnew)
[1] FALSE
Logfile
Thu Jan 24 16:42:27 [initandlisten] MongoDB starting : pid=6260 port=27017 dbpath=\data\db\ 64-bit host=ASHB-109C-02
Thu Jan 24 16:42:27 [initandlisten] db version v2.2.2, pdfile version 4.5
Thu Jan 24 16:42:27 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Thu Jan 24 16:42:27 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Thu Jan 24 16:42:27 [initandlisten] options: { logpath: "log_1.txt" }
Thu Jan 24 16:42:27 [initandlisten] journal dir=/data/db/journal
Thu Jan 24 16:42:27 [initandlisten] recover : no journal files present, no recovery needed
Thu Jan 24 16:42:27 [initandlisten] waiting for connections on port 27017
Thu Jan 24 16:42:27 [websvr] admin web console waiting for connections on port 28017
Thu Jan 24 16:42:36 [initandlisten] connection accepted from 127.0.0.1:52419 #1 (1 connection now open)
Thu Jan 24 16:42:44 [conn1] __test.test Assertion failure x == _nfields src\mongo\db\jsobj.cpp 1250
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(109) mongo::verifyFailed+0xdc
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\jsobj.cpp(1250) mongo::BSONIteratorSorted::BSONIteratorSorted+0xf3
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\ops\update_internal.cpp(906) mongo::ModSetState::createNewFromMods+0xa3
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\ops\update.cpp(370) mongo::_updateObjects+0x15a2
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(573) mongo::receivedUpdate+0x60d
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(437) mongo::assembleResponse+0x626
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Thu Jan 24 16:42:44 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Thu Jan 24 16:42:44 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Thu Jan 24 16:42:44 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Thu Jan 24 16:42:44 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Thu Jan 24 16:42:44 [conn1] update __test.test query: { tags: "a" } update: { $push: { tags: "d" } } nscanned:1 keyUpdates:0 exception: assertion src\mongo\db\jsobj.cpp:1250 locks(micros) w:398335 399ms
Thu Jan 24 16:42:48 CTRL_CLOSE_EVENT signal
Thu Jan 24 16:42:48 [consoleTerminate] got CTRL_CLOSE_EVENT, will terminate after current cmd ends
Thu Jan 24 16:42:48 [consoleTerminate] now exiting
Thu Jan 24 16:42:48 dbexit:
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to close listening sockets...
Thu Jan 24 16:42:48 [consoleTerminate] closing listening socket: 496
Thu Jan 24 16:42:48 [consoleTerminate] closing listening socket: 516
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to flush diaglog...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: going to close sockets...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: waiting for fs preallocator...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: lock for final commit...
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: final commit...
Thu Jan 24 16:42:48 [conn1] end connection 127.0.0.1:52419 (0 connections now open)
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: closing all files...
Thu Jan 24 16:42:48 [consoleTerminate] closeAllFiles() finished
Thu Jan 24 16:42:48 [consoleTerminate] journalCleanup...
Thu Jan 24 16:42:48 [consoleTerminate] removeJournalFiles
Thu Jan 24 16:42:48 [consoleTerminate] shutdown: removing fs lock...
Thu Jan 24 16:42:48 dbexit: really exiting now
TRY 2
Here I tried using the $addToSet operator
Code
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.object(buf, "$addToSet")
mongo.bson.buffer.start.object(buf, name="tags")
mongo.bson.buffer.start.array(buf, "$each")
values <- list("d", "e", "f")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
bnew <- mongo.bson.from.buffer(buf)
bnew
> mongo.update(mongo=con, ns, criteria=q, objNew=bnew)
[1] FALSE
Logfile
Thu Jan 24 16:43:52 [initandlisten] MongoDB starting : pid=4184 port=27017 dbpath=\data\db\ 64-bit host=ASHB-109C-02
Thu Jan 24 16:43:52 [initandlisten] db version v2.2.2, pdfile version 4.5
Thu Jan 24 16:43:52 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Thu Jan 24 16:43:52 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Thu Jan 24 16:43:52 [initandlisten] options: { logpath: "log_2.txt" }
Thu Jan 24 16:43:52 [initandlisten] journal dir=/data/db/journal
Thu Jan 24 16:43:52 [initandlisten] recover : no journal files present, no recovery needed
Thu Jan 24 16:43:52 [initandlisten] waiting for connections on port 27017
Thu Jan 24 16:43:52 [websvr] admin web console waiting for connections on port 28017
Thu Jan 24 16:43:57 [initandlisten] connection accepted from 127.0.0.1:52435 #1 (1 connection now open)
Thu Jan 24 16:44:27 [conn1] __test.test Assertion failure x == _nfields src\mongo\db\jsobj.cpp 1250
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(109) mongo::verifyFailed+0xdc
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\jsobj.cpp(1250) mongo::BSONIteratorSorted::BSONIteratorSorted+0xf3
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\ops\update_internal.cpp(906) mongo::ModSetState::createNewFromMods+0xa3
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\ops\update.cpp(370) mongo::_updateObjects+0x15a2
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(573) mongo::receivedUpdate+0x60d
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(437) mongo::assembleResponse+0x626
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Thu Jan 24 16:44:28 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Thu Jan 24 16:44:28 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Thu Jan 24 16:44:28 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Thu Jan 24 16:44:28 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Thu Jan 24 16:44:28 [conn1] update __test.test query: { tags: "a" } update: { $addToSet: { tags: { $each: [ "d", "e", "f" ] } } } nscanned:1 keyUpdates:0 exception: assertion src\mongo\db\jsobj.cpp:1250 locks(micros) w:390312 390ms
Thu Jan 24 16:44:33 [conn1] end connection 127.0.0.1:52435 (0 connections now open)
Thu Jan 24 16:44:37 CTRL_CLOSE_EVENT signal
Thu Jan 24 16:44:37 [consoleTerminate] got CTRL_CLOSE_EVENT, will terminate after current cmd ends
Thu Jan 24 16:44:37 [consoleTerminate] now exiting
Thu Jan 24 16:44:37 dbexit:
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to close listening sockets...
Thu Jan 24 16:44:37 [consoleTerminate] closing listening socket: 496
Thu Jan 24 16:44:37 [consoleTerminate] closing listening socket: 500
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to flush diaglog...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: going to close sockets...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: waiting for fs preallocator...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: lock for final commit...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: final commit...
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: closing all files...
Thu Jan 24 16:44:37 [consoleTerminate] closeAllFiles() finished
Thu Jan 24 16:44:37 [consoleTerminate] journalCleanup...
Thu Jan 24 16:44:37 [consoleTerminate] removeJournalFiles
Thu Jan 24 16:44:37 [consoleTerminate] shutdown: removing fs lock...
Thu Jan 24 16:44:37 dbexit: really exiting now
What am I doing wrong here?
Additional information
For those interested: here's the code that produced the example doc
pkg <- "rmongodb"
lib <- file.path(R.home(), "library")
if (!suppressWarnings(require(pkg, lib.loc=lib, character.only=TRUE))) {
install.packages(pkg, lib=lib)
require(pkg, lib.loc=lib, character.only=TRUE)
}
# CONNECTION
db <- "__test"
ns <- paste(db, "test", sep=".")
con <- mongo.create(db=db)
# ENSURE EMPTY DB
mongo.remove(mongo=con, ns=ns)
# INSERT
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf)
mongo.bson.buffer.finish.object(buf)
b <- mongo.bson.from.buffer(buf)
mongo.insert(mongo=con, ns=ns, b=b)
EDIT 2013-01-29
As suggested by Tad Marshall from 10gen in his comment to my bug report, I re-ran the code that inserts the document with the MongoDB server running in --objcheck mode (validates BSON structures) and voilà: the server won't let me insert the doc due to an assertion that fails. If I run the server without the --objcheck flag, insertion is successful (but that's probably just due to the fact that no validation takes place).
Note that I tried two different versions of putting together the array in tags as my initial code produced a doc that IMHO is not in sync with MongoDB's indexing conventions:
(Potentially) Invalid document
That's how I did it above. I noticed that I didn't make sure the array index starts with a 0. Insertion of this document will fail (see logfile below)
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf) # finish array 'tags'
mongo.bson.buffer.finish.object(buf) # finish buffer
b <- mongo.bson.from.buffer(buf)
> b
tags : 4
1 : 2 a
2 : 2 b
3 : 2 c
Valid document
I made sure the index starts with 0, so this should definitely be a valid BSON doc. But inserting this document will fail, too (see logfile below)
buf <- mongo.bson.buffer.create()
mongo.bson.buffer.start.array(buf, name="tags")
values <- list("a", "b", "c")
for (ii in seq(along=values)) {
mongo.bson.buffer.append(
buf=buf,
name=as.character(ii-1),
value=values[[ii]]
)
}
mongo.bson.buffer.finish.object(buf) # finish array 'tags'
mongo.bson.buffer.finish.object(buf) # finish buffer
b <- mongo.bson.from.buffer(buf)
b
mongo.insert(mongo=con, ns=ns, b=b)
> b
tags : 4
0 : 2 a
1 : 2 b
2 : 2 c
Logfile
Tue Jan 29 14:20:46 [initandlisten] MongoDB starting : pid=6440 port=27017
[...]
Tue Jan 29 14:20:59 [initandlisten] connection accepted from 127.0.0.1:62137 #1 (1 connection now open)
Tue Jan 29 14:21:03 [conn1] Assertion: 10307:Client Error: bad object in message
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(154) mongo::msgasserted+0xc1
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\dbmessage.h(205) mongo::DbMessage::nextJsObj+0x103
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(784) mongo::receivedInsert+0xdb
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(434) mongo::assembleResponse+0x607
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Tue Jan 29 14:21:04 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Tue Jan 29 14:21:04 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Tue Jan 29 14:21:04 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Tue Jan 29 14:21:04 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Tue Jan 29 14:21:04 [conn1] insert __test.test keyUpdates:0 exception: Client Error: bad object in message code:10307 0ms
Tue Jan 29 14:21:07 [conn1] Assertion: 10307:Client Error: bad object in message
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\stacktrace.cpp(161) mongo::printStackTrace+0x3e
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\assert_util.cpp(154) mongo::msgasserted+0xc1
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\dbmessage.h(205) mongo::DbMessage::nextJsObj+0x103
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(784) mongo::receivedInsert+0xdb
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\instance.cpp(434) mongo::assembleResponse+0x607
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\db\db.cpp(192) mongo::MyMessageHandler::process+0xf5
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\mongo\util\net\message_server_port.cpp(86) mongo::pms::threadRun+0x59a
Tue Jan 29 14:21:07 [conn1] mongod.exe ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180) boost::`anonymous namespace'::thread_start_function+0x21
Tue Jan 29 14:21:07 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314) _callthreadstartex+0x17
Tue Jan 29 14:21:07 [conn1] mongod.exe f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(292) _threadstartex+0x7f
Tue Jan 29 14:21:07 [conn1] kernel32.dll BaseThreadInitThunk+0xd
Tue Jan 29 14:21:07 [conn1] insert __test.test keyUpdates:0 exception: Client Error: bad object in message code:10307 0ms
Oh, I am smacking myself up now. I didn't look closely at the way you were creating your document. You have two mongo.bson.finish.object() calls when you need only one to finish off the array you started. You do not need to call it to finish a BSON. mongo.bson.from.buffer() does the necessary housekeeping. This is my fault for not reading your code closely enough. I thought it was your update failing when the initial insert of the documents is the problem. For questions here in the future, it would help if your examples were a little easier to read. For instance, this will build the document:
library('rmongodb')
m = mongo.create()
ns = '__test.test'
mongo.insert(m, ns, list(tags=c('a', 'b', 'c'))
However, you are probably pasting in real-world code so I understand where the complications come in. Everything's cool. Just beating myself up for missing this and sending you on a wild goose chase. Regards
Rappster, both of these examples worked for me on my development machine, but I am slightly out of date running a debug build of mongod 2.1.0.
Since you are crashing the server with your example code, this is something the 10gen people will want to know about. Do you mind going to https://jira.mongodb.org/secure/Dashboard.jspa and reporting this bug?
Thanks,
Gerald Lindsly