Why is my Cloudera-manager starting very slowly? - cloudera

I am using Cloudera-manager version 5.
Today, I restarted it and it took 1 hour to complete.
This is my log.
2023-02-07 02:20:53,912 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive completed successfully.
2023-02-07 02:20:56,880 INFO MainThread:com.cloudera.server.cmf.jython.components.JythonObjectFactoryImpl: Finished constructing Jython object factory
2023-02-07 02:20:56,970 INFO MainThread:com.cloudera.server.cmf.Main: Server locale set to: en
2023-02-07 02:20:56,995 INFO MainThread:com.cloudera.server.cmf.components.EmbeddedDbManager: CM server is NOT using embedded postgresql database for itself.
2023-02-07 02:20:57,011 INFO MainThread:com.cloudera.cmf.service.CommandScheduleListener: Adding command schedule listener for global-collect-host-statistics
2023-02-07 02:20:57,024 INFO MainThread:com.cloudera.cmf.service.CommandScheduleListener: Initializing schedule for global-collect-host-statistics
2023-02-07 02:20:57,045 INFO MainThread:com.cloudera.cmf.service.CommandScheduleListener: Replacing existing schedule for global-collect-host-statistics
2023-02-07 02:20:57,093 INFO MainThread:com.cloudera.cmf.service.CommandScheduleListener: Removed schedule DbCommandSchedule{id=71, commandName=global-collect-host-statistics, displayName=null, description=null}
2023-02-07 02:20:57,097 INFO MainThread:com.cloudera.cmf.service.CommandScheduleListener: Adding schedule - DbCommandSchedule{id=72, commandName=global-collect-host-statistics, displayName=null, description=null}
2023-02-07 02:50:55,264 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: (119 skipped) ScmActive completed successfully.
2023-02-07 03:19:36,262 INFO JmxPeriodicLogging-0:com.cloudera.enterprise.PeriodicJmx: JMX: ... <Very many characters.>
2023-02-07 03:20:38,635 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Found files CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel,SPARK2-2.4.0.cloudera2-1.cdh5.13.3.p0.1041012-el6.parcel,SPARK2-2.4.0
.cloudera2-1.cdh5.13.3.p0.1041012-el7.parcel,SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el7.parcel,SPARK2-2.1.0.cloudera3-1.cdh5.13.3.p0.569822-el6.parcel,SPARK2-2.1.0.cloudera3-1.cdh5.13.3.p0.569822-el7.parcel,KAFKA-3.1.0
-1.3.1.0.p0.35-el6.parcel,STREAMSETS_DATACOLLECTOR-3.4.1-el7.parcel,KAFKA-3.1.0-1.3.1.0.p0.35-el7.parcel,CDH-5.14.4-1.cdh5.14.4.p0.3-el6.parcel,SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el6.parcel,KUDU-1.1.0-1.kudu1.1.0.p
0.5-el6.parcel,STREAMSETS_DATACOLLECTOR-3.4.1-el6.parcel under /opt/cloudera/parcel-repo
2023-02-07 03:20:38,829 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: (10 skipped) Unable to retrieve remote parcel repository manifest
java.util.concurrent.ExecutionException: java.net.ConnectException: Received fatal alert: protocol_version to https://archive.cloudera.com/cdh5/parcels/5.14/manifest.json
<Very many characters.>
Caused by: java.net.ConnectException: Received fatal alert: protocol_version to https://archive.cloudera.com/cdh5/parcels/5.14/manifest.json
at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:100)
... 22 more
Caused by: javax.net.ssl.SSLException: Received fatal alert: protocol_version
<Very many characters.>
... 3 more
2023-02-07 03:20:40,623 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Failed to download manifest. Status code: 404 URI: http://archive.cloudera.com/spark2/parcels/2.4.0.cloudera2/manifest.js
on/
2023-02-07 03:20:41,515 INFO ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: (14 skipped) Parcel DbParcel{id=416, filename=SPARK2-2.1.0.cloudera3-1.cdh5.13.3.p0.569822-trusty.parcel} no longer exists
on source repo but still in use.
2023-02-07 03:20:56,259 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: (119 skipped) ScmActive completed successfully.
2023-02-07 03:50:31,616 INFO MainThread:com.cloudera.cmf.scheduler.CmfScheduler: Added command schedule '72' to the scheduler
2023-02-07 03:50:31,701 INFO MainThread:com.cloudera.server.cmf.Main: Starting Auto Upgrade
2023-02-07 03:50:31,706 INFO MainThread:com.cloudera.cmf.service.upgrade.AutoUpgradeHandlerRegistry: No Auto Upgrades required.
2023-02-07 03:50:31,706 INFO MainThread:com.cloudera.server.cmf.Main: Successfully completed Auto Upgrade
2023-02-07 03:50:31,866 INFO MainThread:com.cloudera.server.cmf.Main: Agent RPC connections will use port: 7182
2023-02-07 03:50:31,872 INFO MainThread:com.cloudera.server.common.HttpConnectorServer: Max heartbeat processing thread: 12 and Max threads for CM agent avro http connector: 200
2023-02-07 03:50:32,016 INFO MainThread:org.mortbay.log: jetty-6.1.26.cloudera.4
2023-02-07 03:50:32,086 INFO MainThread:org.mortbay.log: Started SelectChannelConnector#0.0.0.0:7182
2023-02-07 03:50:32,086 INFO MainThread:com.cloudera.server.cmf.Main: Started Agent RPC server.
2023-02-07 03:50:32,091 INFO MainThread:org.quartz.core.QuartzScheduler: JobFactory set to: com.cloudera.cmf.scheduler.CmfJobFactory#fcd2bb5
2023-02-07 03:50:32,091 INFO MainThread:org.quartz.core.QuartzScheduler: Scheduler com.cloudera.cmf.scheduler-1_$_NON_CLUSTERED started.
2023-02-07 03:50:32,156 INFO com.cloudera.cmf.scheduler-1_Worker-1:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command GlobalPoolsRefresh BasicCmdArgs{scheduleId=1, scheduledTime=2023-02-06T17:00:00.000Z}.
2023-02-07 03:50:32,224 INFO MainThread:com.cloudera.server.cmf.WebServerImpl: Using webapp:/usr/share/cmf/webapp
2023-02-07 03:50:32,420 INFO MainThread:com.cloudera.server.cmf.WebServerImpl: Plaintext web connections will use port: 7180
What do I need to do to resolve this error?
Thank you all.
Currently, my Cloudera-manager is still in use for the service, so it is not frequently restarted to check the error details.

Related

JMeter(5.5) API testing fails to make a normal HTTP request, receives 502 status code

I've an application hosted on nginx , I'm not able to hit that API using JMeter.
Receives below error on JMeter Sampler Result
Thread Name:Thread Group 1-1
Sample Start:2022-07-21 18:39:47 IST
Load time:31316
Connect Time:4
Latency:31316
Size in bytes:151
Sent bytes:118
Headers size in bytes:151
Body size in bytes:0
Sample Count:1
Error Count:1
Data type ("text"|"bin"|""):
Response code:502
Response message:Could not relay message upstream
HTTPSampleResult fields:
ContentType:
DataEncoding: null
Response ::
HTTP/1.1 502 Could not relay message upstream
Date: Fri, 22 Jul 2022 03:42:26 GMT
Content-Length: 0
Connection: close
Via: HTTP/1.1 m_proxy_sin
But this API works fine from same Host machine over curl, Postman etc.
When hit via JMeter there are no error information available in the nginx log as well regarding the same.
Ping to the host is works fine, the host is reachable from the test Machine.
System Details::
OS : Windows 10
C:\Desktop>java --version
openjdk 11.0.15 2022-04-19
OpenJDK Runtime Environment OpenLogic-OpenJDK (build 11.0.15+10-adhoc..jdk11u)
OpenJDK 64-Bit Server VM OpenLogic-OpenJDK (build 11.0.15+10-adhoc..jdk11u, mixed mode)
JMeter - 5.5
2022-07-25 16:25:47,348 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2022-07-25 16:25:47,351 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*)
2022-07-25 16:25:47,354 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2022-07-25 16:25:47,354 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2022-07-25 16:25:47,354 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2022-07-25 16:25:47,354 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 delayedStart=false
2022-07-25 16:25:47,354 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2022-07-25 16:25:47,354 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2022-07-25 16:25:47,369 INFO o.a.j.t.JMeterThread: Thread started: Thread Group 1-1
2022-07-25 16:26:18,057 INFO o.a.j.t.JMeterThread: Thread is done: Thread Group 1-1
2022-07-25 16:26:18,057 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1
2022-07-25 16:26:18,057 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2022-07-25 16:26:18,057 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)
If this API works fine for curl or Postman just record the request from curl or Postman using JMeter's HTTP(S) Test Script Recorder, it will generate the relevant HTTP Request sampler and HTTP Header Manager in order to send exactly the same request.
performing HTTP requests with cURL (using PROXY)
Postman - Configuring proxy settings
After setting this variable the issue got resolved
set JVM_ARGS="-Djava.net.preferIPv4Stack=true"

Failed to bind handlers: DNS Error: Not Implemented; retrying in 2 s

When trying to start Yagna I receive this error, what can I do? I can probably get some DEBUG logs if needed?
[2021-05-06T08:45:08Z INFO yagna] Starting yagna service! Version: 0.6.4 (4fc72117 2021-04-15 build #135).
Log is written to /home/user/.local/share/yagna/yagna_rCURRENT.log
[2021-05-06T08:45:08Z INFO yagna] Data directory: /home/user/.local/share/yagna
[2021-05-06T08:45:08Z INFO ya_sb_router::unix] Router listening on: "/tmp/yagna.sock"
[2021-05-06T08:45:08Z INFO ya_persistence::executor] using database at: /home/user/.local/share/yagna/yagna.db
[2021-05-06T08:45:08Z INFO ya_persistence::executor] using database at: /home/user/.local/share/yagna/market.db
[2021-05-06T08:45:08Z INFO ya_persistence::executor] using database at: /home/user/.local/share/yagna/activity.db
[2021-05-06T08:45:08Z INFO ya_persistence::executor] using database at: /home/user/.local/share/yagna/payment.db
[2021-05-06T08:45:08Z INFO ya_identity::service::identity] using default identity: 0xf5ecffecf053508fe97255e046a04ce21c8ee525
[2021-05-06T08:45:08Z INFO yagna] Identity GSB service successfully activated
[2021-05-06T08:45:08Z INFO ya_metrics::pusher] Metrics pusher started
[2021-05-06T08:45:08Z INFO yagna] Metrics GSB service successfully activated
[2021-05-06T08:45:08Z INFO ya_service_bus::remote_router] trying to connect to: /tmp/yagna.sock
[2021-05-06T08:45:08Z INFO ya_service_bus::connection] started connection to gsb
[2021-05-06T08:45:08Z INFO ya_metrics::pusher] Starting metrics pusher
[2021-05-06T08:45:10Z INFO yagna] Version GSB service successfully activated
[2021-05-06T08:45:10Z INFO ya_net::service] using default identity as network id: 0xf5ecffecf053508fe97255e046a04ce21c8ee525
[2021-05-06T08:45:10Z WARN ya_net::handler] Failed to bind handlers: DNS Error: Not Implemented; retrying in 2 s
[2021-05-06T08:45:12Z WARN ya_net::handler] Failed to bind handlers: DNS Error: Not Implemented; retrying in 4 s
[2021-05-06T08:45:16Z WARN ya_net::handler] Failed to bind handlers: DNS Error: Not Implemented; retrying in 8 s
EDIT: nslookup
Server: 10.139.1.1
Address: 10.139.1.1#53
** server can't find _net._tcp.dev.golem.network: NOTIMP
I'm not sure what is the reason here, but it seems like DNS is not able to resolve _net._tcp.dev.golem.network SRV record yielding 'Not Implemented'. It is very odd, since Yagna is using Google's DNS servers as a default.
When you face this again pls try to check output of following command
nslookup -q=SRV _net._tcp.dev.golem.network 8.8.8.8
The user has trouble reaching Google's DNS with nslookup, so it appears to be something on his end. He is also using a proxy for his connection, so it must happen somewhere in there. Closing thread.

Could not find a version that satisfies the requirement pbr!=2.1.0,>=2.0.0 (from tempest==16.0.1.dev178)

I just followed the openstack rally quick start guide to create a tempest verifier with Rally v0.9.1 in an Openstack Ocata/stable deployment. The command failed:
(rally-15.1.2) root#infra1-utility-container-f31faeb0:~/.rally/verification# rally verify create-verifier --type tempest --name tempest-verifier
2017-05-21 07:53:13.410 11422 INFO rally.api [-] Creating verifier 'tempest-verifier'.
2017-05-21 07:53:13.528 11422 INFO rally.verification.manager [-] Cloning verifier repo from https://git.openstack.org/openstack/tempest.
2017-05-21 07:53:37.174 11422 INFO rally.verification.manager [-] Creating virtual environment. It may take a few minutes.
2017-05-21 07:53:42.323 11422 ERROR rally.verification.utils [-] Failed cmd: '['pip', 'install', '-e', './']'
2017-05-21 07:53:42.324 11422 ERROR rally.verification.utils [-] Error output: 'Obtaining file:///root/.rally/verification/verifier-091a49ab-1241-40a3-bc9b-531d7f091e37/repo
Collecting pbr!=2.1.0,>=2.0.0 (from tempest==16.0.1.dev178)
Could not find a version that satisfies the requirement pbr!=2.1.0,>=2.0.0 (from tempest==16.0.1.dev178) (from versions: 1.10.0)
No matching distribution found for pbr!=2.1.0,>=2.0.0 (from tempest==16.0.1.dev178)
'
Command failed, please check log for more info
As the current version of pbr is 2.0.0, I'm not sure why pbr installation failed.
(rally-15.1.2) root#infra1-utility-container-f31faeb0:~/.rally/verification# pip freeze|grep pbr
pbr==2.0.0
The question is how to adjust the requirement checking for pbr? or is it possible to choose an older version of tempest?
Thanks.
It solved.
After uploading the two missing python packages: os_testr-0.8.2-py2-none-any.whl and testrepository-0.0.19.tar.gz into local repo, which is a lxc container had been created by openstack-ansible, the Tempest plugin was finally installed.

Large delay between Artifactory started and deployment finished on Tomcat

We currently have Artifactory running on a Digital Ocean Droplet under Tomcat
Artifactory starts nice and quickly, but the deployment doesn't actually complete until it's done, I can't work out how to get any further information out from either the Artifactory Deployment or the Tomcat Server
2017-01-11 13:18:20,707 [art-init] [INFO ]
(o.a.w.s.ArtifactoryContextConfigListener:219) -
###########################################################
### Artifactory successfully started (14.442 seconds) ###
###########################################################
From the catalina.log
11-Jan-2017 13:18:00.921 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Deploying configuration descriptor /opt/jfrog/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml
11-Jan-2017 13:35:28.086 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of configuration descriptor /opt/jfrog/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml has finished in 1,047,164 ms
I'm confused as to what's going on between 13:18:20 and 13:25:28, Artifactory doesn't have a forum anymore and directs everyone here.

Exception sending context initialized event to listener instance of class org.sonar.server.platform.PlatformServletContextListener

I downloaded SonarQube 5.5 with fresh database on oracle 11g and also configured the sonar.properties file. I am unable to start the server and getting the error as shown below:
2016.05.27 09:36:45 INFO web[o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read
2016.05.27 09:36:46 INFO web[o.s.s.p.ServerImpl] SonarQube Server / 5.5 / 5773a4aab0ef6c0de79d3038e82f8a051049d6d0
2016.05.27 09:36:46 INFO web[o.sonar.db.Database] Create JDBC data source for jdbc:oracle:thin:#10.101.18.252:1522:orcl1
2016.05.27 09:36:48 ERROR web[o.a.c.c.C.[.[.[/]] Exception sending context initialized event to listener instance of class org.sonar.server.platform.PlatformServletContextListener
org.sonar.api.utils.MessageException: Current version is too old. Please upgrade to Long Term Support version firstly.
2016.05.27 09:36:48 ERROR web[o.a.c.c.StandardContext] One or more listeners failed to start. Full details will be found in the appropriate container log file
2016.05.27 09:36:48 ERROR web[o.a.c.c.StandardContext] Context [] startup failed due to previous errors
2016.05.27 09:36:48 WARN web[o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Unknown Source)
java.util.TimerThread.run(Unknown Source)
2016.05.27 09:36:48 WARN web[o.a.c.l.WebappClassLoaderBase] The web application [ROOT] appears to have started a thread named [oracle.jdbc.driver.BlockSource.ThreadedCachingBlockSource.BlockReleaser] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
oracle.jdbc.driver.BlockSource$ThreadedCachingBlockSource$BlockReleaser.run(BlockSource.java:327)
2016.05.27 09:36:48 INFO web[o.a.c.h.Http11NioProtocol] Starting ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.05.27 09:36:48 INFO web[o.s.s.a.TomcatAccessLog] Web server is started
2016.05.27 09:36:48 INFO web[o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
2016.05.27 09:36:48 WARN web[o.s.p.ProcessEntryPoint] Fail to start web
java.lang.IllegalStateException: Webapp did not start
at org.sonar.server.app.EmbeddedTomcat.isUp(EmbeddedTomcat.java:84) ~[sonar-server-5.5.jar:na]
at org.sonar.server.app.WebServer.isUp(WebServer.java:48) ~[sonar-server-5.5.jar:na]
at org.sonar.process.ProcessEntryPoint.launch(ProcessEntryPoint.java:105) ~[sonar-process-5.5.jar:na]
at org.sonar.server.app.WebServer.main(WebServer.java:69) ~[sonar-server-5.5.jar:na]
2016.05.27 09:36:48 INFO web[o.a.c.h.Http11NioProtocol] Pausing ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.05.27 09:36:49 INFO web[o.a.c.h.Http11NioProtocol] Stopping ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.05.27 09:36:49 INFO web[o.a.c.h.Http11NioProtocol] Destroying ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.05.27 09:36:49 INFO web[o.s.s.a.TomcatAccessLog] Web server is stopped
2016.05.27 15:06:50 INFO app[o.s.p.m.Monitor] Process[es] is stopping
2016.05.27 15:06:50 INFO es[o.s.p.StopWatcher] Stopping process
2016.05.27 15:06:50 INFO es[o.elasticsearch.node] [sonar-1464341797936] stopping ...
2016.05.27 15:06:50 INFO es[o.elasticsearch.node] [sonar-1464341797936] stopped
2016.05.27 15:06:50 INFO es[o.elasticsearch.node] [sonar-1464341797936] closing ...
2016.05.27 15:06:50 INFO es[o.elasticsearch.node] [sonar-1464341797936] closed
2016.05.27 15:06:50 INFO app[o.s.p.m.Monitor] Process[es] is stopped
<-- Wrapper Stopped
I am not able to understand which version need to upgrade.
Thank you for help in advance !!!
Every time you have this error see your database, it is because it has data when you error.
Try:
delete your database sonar;
drop database sonar;
redo
My problem is solved with fresh and clean the database and also delete the logs and data folder before running the first instance.

Resources