Kubernetes version: 1.1.0-0.4.git2bfa9a1.el7
OS:CentOS Linux release 7.1.1503 (Core)
The kubelet keeps failing with the following:
Dec 24 03:13:22 localhost kubelet: I1224 03:13:22.563665 20056 manager.go:158] Machine: {NumCores:1 CpuFrequency:2992788 MemoryCapacity:1929773056 MachineID:19d82c8902374e7caa655973bd7a6e6a SystemUUID:B75EF5E2-CCA7-4C77-9A75-98B1978DA33C BootID:7bf3d099-ab48-41ce-b91b-cacf7dfccf9d Filesystems:[{Device:/dev/vda1 Capacity:21456445440}] DiskMap:map[252:0:{Name:dm-0 Major:252 Minor:0 Size:107374182400 Scheduler:none} 253:0:{Name:vda Major:253 Minor:0 Size:21474836480 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:fa:16:3e:5d:3c:38 Speed:0 Mtu:1500} {Name:flannel0 MacAddress: Speed:10 Mtu:1472}] Topology:[{Id:0 Memory:2147074048 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown}
Dec 24 03:13:22 localhost docker: time="2015-12-24T03:13:22.570268941Z" level=info msg="GET /version"
Dec 24 03:13:22 localhost kubelet: I1224 03:13:22.570915 20056 manager.go:165] Version: {KernelVersion:3.10.0-229.7.2.el7.x86_64 ContainerOsVersion:CentOS Linux 7 (Core) DockerVersion:1.8.2-el7.centos CadvisorVersion:0.16.0}
Dec 24 03:13:22 localhost kubelet: panic: runtime error: invalid memory address or nil pointer dereference
Dec 24 03:13:22 localhost kubelet: [signal 0xb code=0x1 addr=0x0 pc=0x57062d]
Dec 24 03:13:22 localhost kubelet: goroutine 1 [running]:
Dec 24 03:13:22 localhost kubelet: k8s.io/kubernetes/pkg/cloudprovider/openstack.(*OpenStack).Instances(0x0, 0x0, 0x0, 0xa)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/cloudprovider/openstack/openstack.go:163 +0x8ed
Dec 24 03:13:22 localhost kubelet: k8s.io/kubernetes/cmd/kubelet/app.RunKubelet(0xc8200cc000, 0x0, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:616 +0x13c
Dec 24 03:13:22 localhost kubelet: k8s.io/kubernetes/cmd/kubelet/app.(*KubeletServer).Run(0xc820238000, 0xc8200cc000, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:410 +0x84b
Dec 24 03:13:22 localhost kubelet: main.main()
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/kubelet.go:46 +0xab
Dec 24 03:13:22 localhost kubelet: goroutine 17 [syscall, locked to thread]:
Dec 24 03:13:22 localhost kubelet: runtime.goexit()
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/runtime/asm_amd64.s:1696 +0x1
Dec 24 03:13:22 localhost kubelet: goroutine 5 [chan receive]:
Dec 24 03:13:22 localhost kubelet: github.com/golang/glog.(*loggingT).flushDaemon(0x1c8b7a0)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x67
Dec 24 03:13:22 localhost kubelet: created by github.com/golang/glog.init.1
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x297
Dec 24 03:13:22 localhost kubelet: goroutine 13 [syscall]:
Dec 24 03:13:22 localhost kubelet: os/signal.loop()
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/os/signal/signal_unix.go:22 +0x18
Dec 24 03:13:22 localhost kubelet: created by os/signal.init.1
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/os/signal/signal_unix.go:28 +0x37
Dec 24 03:13:22 localhost kubelet: goroutine 16 [sleep]:
Dec 24 03:13:22 localhost kubelet: time.Sleep(0x12a05f200)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/runtime/time.go:59 +0xf9
Dec 24 03:13:22 localhost kubelet: k8s.io/kubernetes/pkg/util.Until(0x15589a0, 0x12a05f200, 0x0)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:113 +0x61
Dec 24 03:13:22 localhost kubelet: k8s.io/kubernetes/pkg/util.Forever(0x15589a0, 0x12a05f200)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:93 +0x34
Dec 24 03:13:22 localhost kubelet: created by k8s.io/kubernetes/pkg/util.InitLogs
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/util/logs.go:50 +0xae
Dec 24 03:13:22 localhost kubelet: goroutine 31 [IO wait]:
Dec 24 03:13:22 localhost kubelet: net.runtime_pollWait(0x7f19addac638, 0x72, 0xc820010120)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/runtime/netpoll.go:157 +0x60
Dec 24 03:13:22 localhost kubelet: net.(*pollDesc).Wait(0xc82022f100, 0x72, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/fd_poll_runtime.go:73 +0x3a
Dec 24 03:13:22 localhost kubelet: net.(*pollDesc).WaitRead(0xc82022f100, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/fd_poll_runtime.go:78 +0x36
Dec 24 03:13:22 localhost kubelet: net.(*netFD).accept(0xc82022f0a0, 0x0, 0x7f19addad7b8, 0xc8204d1a80)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/fd_unix.go:408 +0x27c
Dec 24 03:13:22 localhost kubelet: net.(*TCPListener).AcceptTCP(0xc8200264d8, 0xc820034d98, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/tcpsock_posix.go:249 +0x4d
Dec 24 03:13:22 localhost kubelet: net/http.tcpKeepAliveListener.Accept(0xc8200264d8, 0x0, 0x0, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/http/server.go:2092 +0x41
Dec 24 03:13:22 localhost kubelet: net/http.(*Server).Serve(0xc820016f00, 0x7f19addad780, 0xc8200264d8, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/http/server.go:1841 +0xb3
Dec 24 03:13:22 localhost kubelet: net/http.(*Server).ListenAndServe(0xc820016f00, 0x0, 0x0)
Dec 24 03:13:22 localhost kubelet: /usr/lib/golang/src/net/http/server.go:1831 +0x136
Dec 24 03:13:22 localhost kubelet: k8s.io/kubernetes/pkg/kubelet/cadvisor.(*cadvisorClient).exportHTTP.func1(0xc820016f00, 0x1062)
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/kubelet/cadvisor/cadvisor_linux.go:99 +0x6a
Dec 24 03:13:22 localhost kubelet: created by k8s.io/kubernetes/pkg/kubelet/cadvisor.(*cadvisorClient).exportHTTP
Dec 24 03:13:22 localhost kubelet: /builddir/build/BUILD/kubernetes-2bfa9a1f98147cfdc2e9f4cf50e2c430518d91eb/_output/local/go/src/k8s.io/kubernetes/pkg/kubelet/cadvisor/cadvisor_linux.go:105 +0x3ac
Dec 24 03:13:22 localhost systemd: kubelet.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 24 03:13:22 localhost systemd: Unit kubelet.service entered failed state.
Dec 24 03:13:22 localhost systemd: kubelet.service holdoff time over, scheduling restart.
Dec 24 03:13:22 localhost systemd: Stopping Kubernetes Kubelet Server...
Dec 24 03:13:22 localhost systemd: Starting Kubernetes Kubelet Server...
Dec 24 03:13:22 localhost systemd: kubelet.service start request repeated too quickly, refusing to start.
Dec 24 03:13:22 localhost systemd: Failed to start Kubernetes Kubelet Server.
Dec 24 03:13:22 localhost systemd: Unit kubelet.service entered failed state.
kublet config:
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="kube-node1"
# location of the api-server
KUBELET_API_SERVER="--api-servers=http://kube-master:8080"
# Add your own!
KUBELET_ARGS="--cloud-provider=openstack --cloud-config=/etc/kubernetes/cloud-config"
[root#kube-node1 kubernetes]#
config:
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://kube-master:8080"
cloud-config:
[Global]
auth-url = http://172.200.1.5/v2.0
user-id = nova
api-key = password
region = RegionOne
tenant-id = service
tenant-name = service
Not sure what I am missing here. Been at it for hours, unfortunately the code for the openstack provider does not tell you when a param is missing, or malformed. Any help would be appreciated.
This was discussed on github:
https://github.com/kubernetes/kubernetes/issues/19264
username and password are supported, but user-id and api-key apparently are not.
Related
I've upgraded my server from stretch to buster and then to bullseye and from this time i have some problems with mariadb server which is restarting often. While is restarting my emails doesnt work cuz of looking for virtual table etc...
mariadb version is
mariadb -v
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 9135
Server version: 10.5.15-MariaDB-0+deb11u1-log Debian 11
then in syslog i can see this
cat /var/log/syslog |grep "mariadb.service"
Aug 19 10:34:45 srv systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
Aug 19 10:34:45 srv systemd[1]: mariadb.service: Failed with result 'signal'.
Aug 19 10:34:50 srv systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 401.
dont know how to resolve this problem, maybe i should reinstall mariadb and purge before all packets of mariadb and mysql?
mariadb is restarting 6-10 times in one hour
in syslog i can see this interesting
Aug 19 10:34:45 srv mariadbd[22091]: 2022-08-19 10:34:45 11917 [ERROR] [FATAL] InnoDB: Page old data size 15870 new data size 8280, page old max ins size 36 new max ins size 7626
Aug 19 10:34:45 srv mariadbd[22091]: 220819 10:34:45 [ERROR] mysqld got signal 6 ;
Aug 19 10:34:45 srv mariadbd[22091]: This could be because you hit a bug. It is also possible that this binary
Aug 19 10:34:45 srv mariadbd[22091]: or one of the libraries it was linked against is corrupt, improperly built,
Aug 19 10:34:45 srv mariadbd[22091]: or misconfigured. This error can also be caused by malfunctioning hardware.
Aug 19 10:34:45 srv mariadbd[22091]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
Aug 19 10:34:45 srv mariadbd[22091]: We will try our best to scrape up some info that will hopefully help
Aug 19 10:34:45 srv mariadbd[22091]: diagnose the problem, but since we have already crashed,
Aug 19 10:34:45 srv mariadbd[22091]: something is definitely wrong and this may fail.
Aug 19 10:34:45 srv mariadbd[22091]: Server version: 10.5.15-MariaDB-0+deb11u1-log
Aug 19 10:34:45 srv mariadbd[22091]: key_buffer_size=792723456
Aug 19 10:34:45 srv mariadbd[22091]: read_buffer_size=131072
Aug 19 10:34:45 srv mariadbd[22091]: max_used_connections=15
Aug 19 10:34:45 srv mariadbd[22091]: max_threads=2002
Aug 19 10:34:45 srv mariadbd[22091]: thread_count=12
I have installed a WordPress site in AWS EC2 t3.micro on platform Linux/UNIX. The ec2 instance stops responding every few hours. That means if I try to call the website from the browser I get "This site can't be reached" message. The site starts to work again if I reboot the instance from the ec2 console. The inbound rules for HTTP and HTTPS set to 0.0.0.0/0. There is a number of website visitors. There is 4.2G available space on the EC2 instance. The latest error log entries are:
[Tue Jul 12 09:01:50.142638 2022] [mpm_prefork:notice] [pid 566] AH00163: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 configured -- resuming normal operations
[Tue Jul 12 09:01:50.142711 2022] [core:notice] [pid 566] AH00094: Command line: '/usr/sbin/apache2'
[Tue Jul 12 10:23:11.216290 2022] [php:warn] [pid 1077] [client X.X.X.X:57861] PHP Warning: Undefined array key "referrer" in /var/www/html/wp-content/plugins/elementor-pro/modules/forms/classes/form-record.php on line 179, referer: https://SITE-NAME-REMOVED/contact-us/
[Tue Jul 12 10:23:11.217667 2022] [php:warn] [pid 1077] [client X.X.X.X:57861] PHP Warning: Undefined array key "referrer" in /var/www/html/wp-content/plugins/elementor-pro/modules/forms/classes/form-record.php on line 179, referer: https://SITE-NAME-REMOVED/contact-us/
sh: 1: /usr/sbin/sendmail: not found
[Mon Jul 11 00:00:01.527769 2022] [mpm_prefork:notice] [pid 570] AH00163: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 configured -- resuming normal operations
[Mon Jul 11 00:00:01.527800 2022] [core:notice] [pid 570] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jul 11 13:44:17.339898 2022] [mpm_prefork:notice] [pid 577] AH00163: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 configured -- resuming normal operations
[Mon Jul 11 13:44:17.347833 2022] [core:notice] [pid 577] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jul 11 13:44:26.721451 2022] [php:warn] [pid 727] [client x.X.x.x:35130] PHP Warning: Undefined array key "referrer" in /var/www/html/wp-content/plugins/elementor-pro/modules/forms/classes/form-record.php on line 179, referer: https://SITE-NAME-REMOVED/contact-us/
[Mon Jul 11 13:44:26.722285 2022] [php:warn] [pid 727] [client x.x.x.x:35130] PHP Warning: Undefined array key "referrer" in /var/www/html/wp-content/plugins/elementor-pro/modules/forms/classes/form-record.php on line 179, referer: https://SITE-NAME-REMOVED/contact-us/
sh: 1: /usr/sbin/sendmail: not found
[Mon Jul 11 14:37:11.427820 2022] [mpm_prefork:notice] [pid 577] AH00169: caught SIGTERM, shutting down
[Mon Jul 11 17:32:49.756249 2022] [mpm_prefork:notice] [pid 600] AH00163: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 configured -- resuming normal operations
[Mon Jul 11 17:32:49.756835 2022] [core:notice] [pid 600] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jul 11 20:15:40.358852 2022] [mpm_prefork:notice] [pid 600] AH00169: caught SIGTERM, shutting down
Could anyone please advise?
Arch Linux, standard kernel 5.16.3, MariaDB v10.6.5.
I try to start MariaDB, and it fails with
ERROR 2002 (HY000): Can't connect to local server through socket '/run/mysqld/mysqld.sock' (111)
systemctl status mariadb gives me
Jan 28 20:13:13 gazelle mariadbd[2915]: 2022-01-28 20:13:13 0 [Note] InnoDB: Buffer pool(s) load completed at 220128 20:13:13
Jan 28 20:13:13 gazelle mariadbd[2915]: 2022-01-28 20:13:13 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
Jan 28 20:13:13 gazelle mariadbd[2915]: 2022-01-28 20:13:13 0 [Note] Server socket created on IP: '0.0.0.0'.
Jan 28 20:13:13 gazelle mariadbd[2915]: 2022-01-28 20:13:13 0 [Note] Server socket created on IP: '::'.
Jan 28 20:13:13 gazelle mariadbd[2915]: 2022-01-28 20:13:13 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist
Jan 28 20:13:13 gazelle mariadbd[2915]: 2022-01-28 20:13:13 0 [ERROR] Aborting
Jan 28 20:13:13 gazelle mariadbd[2915]: Warning: Memory not freed: 280
Jan 28 20:13:13 gazelle systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Jan 28 20:13:13 gazelle systemd[1]: mariadb.service: Failed with result 'exit-code'.
Jan 28 20:13:13 gazelle systemd[1]: Failed to start MariaDB 10.6.5 database server.
So okay, I can't connect because the server isn't starting up. But even when it did start --- it did start on a few occasions, I would still get this error. I looked in /run/mysqld, and mysqld.sock is there, with 777 permissions, and it's owned by the mysql user. I've also made sure that /var/lib/mysql and everything underneath it is owned by mysql
Here's /etc/my.cnf
#
# This group is read both by the client and the server
# use it for options that affect everything
#
[client-server]
[mysqld]
socket = /run/mysqld/mysqld.sock
[client]
socket = /run/mysqld/mysqld.sock
#
# include *.cnf from the config directory
#
!includedir /etc/my.cnf.d
What are the next possible steps? I've scoured the 'net, and nothing seems to help.
When initializing the database, mysql_install_db Whether the datadir directory is specified during dB. It is recommended to use my.cnf Specify the datadir directory in the CNF configuration file
Okay, so Paul T's comment
Curious about the two missing tables causing errors. Did the install not go smoothly, maybe reinstall?
was the clue. Looked at the MariaDB page on the Arch wiki, and I forgot to run mariadb-install-db. Should've RTFM.
Hello and thanks for taking some of your time to check my problem
i'm following the detailed steps by openstack to install openstack in no avail (https://docs.openstack.org/keystone/rocky/install/keystone-install-rdo.html)
I've tried to change the 5000 port for the service but the result is the same
any insights are most welcomed
[root#localhost i-openstack]# systemctl enable httpd.service
[root#localhost i-openstack]# systemctl start httpd.service
Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
[root#localhost i-openstack]# journalctl -xe
Oct 08 05:12:39 localhost.localdomain systemd[1]: Failed to start The Apache HTTP Server.
-- Subject: Unit httpd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit httpd.service has failed.
--
-- The result is failed.
Oct 08 05:12:39 localhost.localdomain systemd[1]: Unit httpd.service entered failed state.
Oct 08 05:12:39 localhost.localdomain systemd[1]: httpd.service failed.
Oct 08 05:12:39 localhost.localdomain polkitd[1824]: Unregistered Authentication Agent for unix-process:4229:106865 (system bus name :1.42, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Oct 08 05:27:21 localhost.localdomain polkitd[1824]: Registered Authentication Agent for unix-process:4930:195069 (system bus name :1.43 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale
Oct 08 05:27:21 localhost.localdomain systemd[1]: Reloading.
Oct 08 05:27:21 localhost.localdomain polkitd[1824]: Unregistered Authentication Agent for unix-process:4930:195069 (system bus name :1.43, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Oct 08 05:27:26 localhost.localdomain polkitd[1824]: Registered Authentication Agent for unix-process:4950:195568 (system bus name :1.44 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale
Oct 08 05:27:26 localhost.localdomain systemd[1]: Starting The Apache HTTP Server...
-- Subject: Unit httpd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit httpd.service has begun starting up.
Oct 08 05:27:26 localhost.localdomain httpd[4956]: (13)Permission denied: AH00072: make_sock: could not bind to address [::]:5000
Oct 08 05:27:26 localhost.localdomain httpd[4956]: (13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:5000
Oct 08 05:27:26 localhost.localdomain httpd[4956]: no listening sockets available, shutting down
Oct 08 05:27:26 localhost.localdomain httpd[4956]: AH00015: Unable to open logs
Oct 08 05:27:26 localhost.localdomain systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Oct 08 05:27:26 localhost.localdomain kill[4958]: kill: cannot find process ""
Oct 08 05:27:26 localhost.localdomain systemd[1]: httpd.service: control process exited, code=exited status=1
Oct 08 05:27:26 localhost.localdomain systemd[1]: Failed to start The Apache HTTP Server.
-- Subject: Unit httpd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit httpd.service has failed.
--
-- The result is failed.
Oct 08 05:27:26 localhost.localdomain systemd[1]: Unit httpd.service entered failed state.
Oct 08 05:27:26 localhost.localdomain systemd[1]: httpd.service failed.
Oct 08 05:27:26 localhost.localdomain polkitd[1824]: Unregistered Authentication Agent for unix-process:4950:195568 (system bus name :1.44, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Oct 08 05:34:01 localhost.localdomain polkitd[1824]: Registered Authentication Agent for unix-process:5222:235020 (system bus name :1.45 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale
Oct 08 05:34:01 localhost.localdomain systemd[1]: Reloading.
Oct 08 05:34:01 localhost.localdomain polkitd[1824]: Unregistered Authentication Agent for unix-process:5222:235020 (system bus name :1.45, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Oct 08 05:34:03 localhost.localdomain polkitd[1824]: Registered Authentication Agent for unix-process:5240:235248 (system bus name :1.46 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale
Oct 08 05:34:03 localhost.localdomain systemd[1]: Starting The Apache HTTP Server...
-- Subject: Unit httpd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
SOLUTION : It seemed i had to disable SELinux
Disable temporaliy
sudo setenforce 0
Restart httpd service
service httpd restart
Disable SELinux persistently (reboot required)
nano /etc/selinux/config
SELINUX=disabled
I have a simple nginx config that is syntactically correct. I install nginx using chef and the chef script works fine.
But as I check status of nginx , I see it is in failed state. If I reload nginx , it again goes in failed state. journalctl -xn also doesnt give much of error except :
[root#localhost vagrant]# journalctl -xn
-- Logs begin at Wed 2016-10-26 04:28:18 UTC, end at Wed 2016-10-26 04:45:00 UTC. --
Oct 26 04:45:00 localhost.localdomain kill[17003]: -s, --signal <sig> send specified signal
Oct 26 04:45:00 localhost.localdomain kill[17003]: -q, --queue <sig> use sigqueue(2) rather than kill(2)
Oct 26 04:45:00 localhost.localdomain kill[17003]: -p, --pid print pids without signaling them
Oct 26 04:45:00 localhost.localdomain kill[17003]: -l, --list [=<signal>] list signal names, or convert one to a name
Oct 26 04:45:00 localhost.localdomain kill[17003]: -L, --table list signal names and numbers
Oct 26 04:45:00 localhost.localdomain kill[17003]: -h, --help display this help and exit
Oct 26 04:45:00 localhost.localdomain kill[17003]: -V, --version output version information and exit
Oct 26 04:45:00 localhost.localdomain kill[17003]: For more details see kill(1).
Oct 26 04:45:00 localhost.localdomain systemd[1]: nginx.service: control process exited, code=exited status=1
Oct 26 04:45:00 localhost.localdomain systemd[1]: Unit nginx.service entered failed state.
[root#localhost vagrant]#
nginx -t is successful and I see nothing in /var/log/nginx/errors.log
Is there any other way to troubleshoot exactly why this fails ?
Both systemctl status nginx.service gives:
[root#localhost vagrant]# systemctl status nginx.service
nginx.service - The nginx HTTP and reverse proxy server
Loaded: loaded (/usr/lib/systemd/system/nginx.service; static)
Active: failed (Result: exit-code) since Wed 2016-10-26 04:45:00 UTC; 9h ago
Process: 17003 ExecStop=/bin/kill -s QUIT $MAINPID (code=exited, status=1/FAILURE)
Process: 16999 ExecStart=/opt/nginx-1.10.1/sbin/nginx (code=exited, status=0/SUCCESS)
Process: 16998 ExecStartPre=/opt/nginx-1.10.1/sbin/nginx -t (code=exited, status=0/SUCCESS)
Main PID: 16999 (code=exited, status=0/SUCCESS)
Oct 26 04:45:00 localhost.localdomain kill[17003]: -s, --signal <sig> send specified signal
Oct 26 04:45:00 localhost.localdomain kill[17003]: -q, --queue <sig> use sigqueue(2) rather than kill(2)
Oct 26 04:45:00 localhost.localdomain kill[17003]: -p, --pid print pids without signaling them
Oct 26 04:45:00 localhost.localdomain kill[17003]: -l, --list [=<signal>] list signal names, or convert one to a name
Oct 26 04:45:00 localhost.localdomain kill[17003]: -L, --table list signal names and numbers
Oct 26 04:45:00 localhost.localdomain kill[17003]: -h, --help display this help and exit
Oct 26 04:45:00 localhost.localdomain kill[17003]: -V, --version output version information and exit
Oct 26 04:45:00 localhost.localdomain kill[17003]: For more details see kill(1).
Oct 26 04:45:00 localhost.localdomain systemd[1]: nginx.service: control process exited, code=exited status=1
Oct 26 04:45:00 localhost.localdomain systemd[1]: Unit nginx.service entered failed state.
systemctl cat nginx.service gives :
[root#virsinplatformapi02 sysadmin]# systemctl cat nginx.service
Unknown operation 'cat'.
I cd cd /lib/systemd/system and do cat on nginx.service:
[root#virsinplatformapi02 system]# cat nginx.service
[Unit]
Description=The nginx HTTP and reverse proxy server
After=network.target remote-fs.target nss-lookup.target
[Service]
ExecStartPre=/opt/nginx-1.10.1/sbin/nginx -t
ExecStart=/opt/nginx-1.10.1/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
[Install]
If I do a echo $MAINPID , I get nothing.
That is not very good unitfile. Type is not set and defaults to simple, while you want to you forking for nginx. That may be the reason for the wrong $MAINPID value. Try to use official unit:
[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target
[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/opt/nginx-1.10.1/sbin/nginx -t
ExecStart=/opt/nginx-1.10.1/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
[Install]
WantedBy=multi-user.target
You should just add it to /etc/systemd/system/nginx.service - that directory it intended for administrator-created units, and has priority.