GCE - Can not create an instance image from the startup script - rsync

I am trying to create a GCE instance image from its startup script (just after some setup instructions). I use a command line looking like :
gcimagebundle -d "/dev/sda" -o "/tmp/" --log_file=/var/log/image_generation.log "--output_file_name=/tmp/myimage.tar.gz"
After few seconds the gcimagebundle exits and I get the following error message (from the /var/log/image_generation.log file) :
INFO:root:found platform Centos
INFO:root:File System: ext4
INFO:root:Disk Size: 10737418240 bytes
INFO:root:Resolved /dev/sda to /dev/sda
INFO:root:exclude list: (/etc/ssh/.host_key_regenerated, 0:0:0) (/dev, 0:1:0) (/proc, 0:1:0) (/run, 0:1:1) (/selinux, 0:0:0) (/tmp, 0:1:0) (/sys, 0:1:0) (/var/lib/google/per- instance, 0:1:0) (/var/lock, 0:1:1) (/var/log, 0:1:1) (/var/run, 0:1:1)
INFO:root:ignoring mounts /proc /sys /dev/pts /dev/shm /proc/sys/fs/binfmt_misc
WARNING:root:overwrite list =
INFO:root:Initializing disk file
INFO:root:Making filesystem
INFO:root:Copying contents
WARNING:root:Error while running ['rsync', '--times', '--perms', '--owner', '--group', '--links', '--devices', '--acls', '--sparse', '--hard-links', '--recursive', '--xattrs', '--exclude-from=/tmp/tmpXtECHY/tmp6Y_7Dr', '/', '/tmp/tmpXtECHY/tmpencYI_'] return_code = 11
stdout=
stderr=rsync: failed to open exclude file /tmp/tmpXtECHY/tmp6Y_7Dr: Permission denied (13)
rsync error: error in file IO (code 11) at exclude.c(1062) [client=3.0.6]
I tried to execute the same startup script on a fresh instance after a manual sudo su and all works.
I execute a whoami from the startup script and the result is root.

Related

Showing error while mounting EFS to an instance in my Elastic Beanstalk environment

Followed the following procedure for attaching the EFS file to instances created using EB:
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-mount-efs-volumes/#:~:text=In%20an%20Elastic%20Beanstalk%20environment,scale%20up%20to%20multiple%20instances.
But the logs of Elastic Beanstalk are showing following error:
[Instance: i-06593*****] Command failed on instance. Return code: 1 Output: (TRUNCATED)...fs ... mount -t efs -o tls fs-d9****:/ /efs Failed to resolve "fs-d9****.efs.us-east-1.amazonaws.com" - check that your file system ID is correct. See https://docs.aws.amazon.com/console/efs/mount-dns-name for more detail. ERROR: Mount command failed!. command 01_mount in .ebextensions/storage-efs-mountfilesystem.config failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Just used **** in EFS ID for security.
Based on the comments.
The solution was to create new EFS filesystem, instead of using the original one.

Airflow live executor logs with DaskExecutor

I have an Airflow installation (on Kubernetes). My setup uses DaskExecutor. I also configured remote logging to S3. However when the task is running I cannot see the log, and I get this error instead:
*** Log file does not exist: /airflow/logs/dbt/run_dbt/2018-11-01T06:00:00+00:00/3.log
*** Fetching from: http://airflow-worker-74d75ccd98-6g9h5:8793/log/dbt/run_dbt/2018-11-01T06:00:00+00:00/3.log
*** Failed to fetch log file from worker. HTTPConnectionPool(host='airflow-worker-74d75ccd98-6g9h5', port=8793): Max retries exceeded with url: /log/dbt/run_dbt/2018-11-01T06:00:00+00:00/3.log (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d0668ae80>: Failed to establish a new connection: [Errno -2] Name or service not known',))
Once the task is done, the log is shown correctly.
I believe what Airflow is doing is:
for finished tasks read logs from s3
for running tasks, connect to executor's log server endpoint and show that.
Looks like Airflow is using celery.worker_log_server_port to connect to my dask executor to fetch logs from there.
How to configure DaskExecutor to expose log server endpoint?
my configuration:
core remote_logging True
core remote_base_log_folder s3://some-s3-path
core executor DaskExecutor
dask cluster_address 127.0.0.1:8786
celery worker_log_server_port 8793
what i verified:
- verified that the log file exists and is being written to on the executor while the task is running
- called netstat -tunlp on executor container, but did not find any extra port exposed, where logs could be served from.
UPDATE
have a look at serve_logs airflow cli command - I believe it does exactly the same.
We solved the problem by simply starting a python HTTP handler on a worker.
Dockerfile:
RUN mkdir -p $AIRFLOW_HOME/serve
RUN ln -s $AIRFLOW_HOME/logs $AIRFLOW_HOME/serve/log
worker.sh (run by Docker CMD):
#!/usr/bin/env bash
cd $AIRFLOW_HOME/serve
python3 -m http.server 8793 &
cd -
dask-worker $#

After configuring Nexus 3 SSL Nexus no longer runs without sudo

I had my new Nexus 3 repository running okay. I was able to configure some of the basic settings. Then I went through the process of enabling SSL. I used the instructions here. I also watched the video on that page, which does not give instructions that match the page.
My system info: ubuntu 14.4 with Java 8.
Install directory: /opt/nexus-3.0.0-b2016011501/
To simplify the path, I created a link to this directory: nexus -> /opt/nexus-3.0.0-b2016011501/ therefore the path to nexus is /opt/nexus
I generated my keystore as follows:
Created directory: /opt/nexus/etc/ssl
Changed to that directory and ran: keytool -keystore keystore -alias jetty -genkey -keyalg RSA -validity 3650. This generated a file called keystore. I then copied that file to keystore.jks.
Updated the following files: /opt/nexus/etc/org.sonatype.nexus.cfg added application-port-ssl=443 and added ${karaf.etc}/jetty-https.xml(this is different from the written instructions) to the end of the nexus-args=$ line. Then (this is in the video, but not the written instructions) I edited the /opt/nexus/etc/jetty-https.xml file and replaced the password in three places with the password I specified when I generated my keystore.
After this if I start nexus with ./nexus run it get the following error:
2016-01-27 02:20:41,013+0000 ERROR [jetty-main-1] *SYSTEM org.sonatype.nexus.bootstrap.jetty.JettyServer - Failed to start
java.net.SocketException: Permission denied
at sun.nio.ch.Net.bind0(Native Method) [na:1.8.0_72]
at sun.nio.ch.Net.bind(Net.java:433) [na:1.8.0_72]
at sun.nio.ch.Net.bind(Net.java:425) [na:1.8.0_72]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) [na:1.8.0_72]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) [na:1.8.0_72]
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:326) [org.eclipse.jetty.server:9.3.5.v20151012]
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) [org.eclipse.jetty.server:9.3.5.v20151012]
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244) [org.eclipse.jetty.server:9.3.5.v20151012]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [org.eclipse.jetty.util:9.3.5.v20151012]
at org.eclipse.jetty.server.Server.doStart(Server.java:384) [org.eclipse.jetty.server:9.3.5.v20151012]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [org.eclipse.jetty.util:9.3.5.v20151012]
at org.sonatype.nexus.bootstrap.jetty.JettyServer$JettyMainThread.run(JettyServer.java:274) [org.sonatype.nexus.bootstrap:3.0.0.b2016011501]
If it start it with sudo ./nexus run it will work but shows me the nag message saying I should not run it as root.
I have verified that my user is the owner of all the files and directories /opt/nexus
On Linux (and other unix type systems) you can't run on port numbers less than 1024 unless you are root. The best way to solve this is to run Nexus behind a reverse proxy. You can find instructions for this here:
http://books.sonatype.com/nexus-book/reference/install-sect-proxy.html
The above was written for Nexus 2.x, but the configuration needed will be the same in Nexus 3.
Regarding running as non-root as a service, there is a bug in 3.0m7 that makes this problematic:
https://issues.sonatype.org/browse/NEXUS-9437
The fix is to edit the "bin/nexus" startup script is to replace this line:
INSTALL4J_JAVA_PREFIX="su - $run_as_user -c"
With this:
exec su - $run_as_user "$prg_dir/$progname" $#
This fix will be in the next release.
Once that change is made, symlink $NEXUS_HOME/bin/nexus to /etc/init.d/nexus, then install the service. And edit "$NEXUS_HOME/bin/nexus.rc" and set the "run_as_user" appropriately.

Error on starting the application Puppet in the Generic enablers Cosmos

Good afternoon,
I have installed the Generic enablers Cosmos, following the manual BigData Analysis - Installation and Administration Guide. When I have come to 'Step 7: applying Puppet' and executed the commands, in the file puppet.err has appeared the following errors:
Error: Could not prefetch yumrepo provider ' inifilé: Section 'openvz-utils' is already defined, cannot re-defines in/etc/yum.repos.d/openvz.repo
Description: There is a conflict with the titles (indicated in bold type) of the file /etc/yum.repos.d/cosmos-openvz.repo and /etc/yum.repos.d/openvz.repo .
cat /etc/yum.repos.d/cosmos-openvz.repo
[openvz-utils]
...
[openvz-kernel-rhel6]
...
cat /etc/yum.repos.d/openvz.repo
[openvz-utils]
...
[openvz-kernel-rhel6]
...
[openvz-kernel-rhel6-testing]
...
Solution: I have realized a change in the titles of the file /etc/yum.repos.d/openvz.repo, example: [openvz-utils_1]
Error: Could not prefetch database_grant provider 'mysql': Execution of '/usr/bin/mysql mysql -Be describe user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Description: in the folder /var/lib/mysql/ was not found the file mysql.sock.
Solution: I have installed mysql-server.x86_64:
yum install mysql-server.x86_64
At the end of the installations, I restarted the service:
/etc/init.d/mysqld stop
/etc/init.d/mysqld start
Error: Could not update: Execution of '/usr/bin/yum -d 0 -e 0 -y list vzstats' returned 1: Error: Cannot retrieve repository metadata (repomd.xml) for repository: ambari. Please verify its path and try again
Description: This error appears in the machine of the Master node, this one is provoked by the configuration of the file [COSMOS_TMP_PATH]/puppet/modules/cosmos/manifests/hieradata/my_environment/common.yaml, indicated in 'Step 6: Puppet configuration'. Concretely, the URL that use the IP: 130.206.81.65
Solution: in the file [COSMOS_TMP_PATH]/puppet/modules/cosmos/manifests/hieradata /my_environment/common.yaml to change the line:
ambari::params::repo_url: 'http:// 130.206.81.65/cosmos/ambari/'
(without blank space)
for
ambari::params::repo_url: 'http:// public-repo-1.hortonworks.com/ambari/centos6/1.x/GA'
(without blank space)
Error: Could not update: Execution of '/usr/bin/yum -d 0 -e 0 -y list vzstats' returned 1: Error: Cannot retrieve repository metadata (repomd.xml) for repository: cosmos-libvirt. Please verify its path and try again
Description: it is the same problem as the previous error. The difficulty in this one is that I cannot modify the file [COSMOS_TMP_PATH]/puppet/modules/cosmos/manifests/hieradata /my_environment/common.yaml in the line:
cosmos::params::cosmos_repo_deps_url: 'http:// 130.206.81.65/cosmos/rpms/cosmos-deps'
(without blank space)
Because it is line is used in several files:
cat /etc/yum.repos.d/cosmos-libvirt.repo
[cosmos-libvirt]
name=Cosmos LibVirt with OpenVZ - v1.0.5 - NO PolKIT
baseurl=http:// 130.206.81.65/cosmos/rpms/cosmos-deps//libvirt
gpgcheck=0
priority=10
enabled=1
cat /etc/yum.repos.d/cosmos-openvz.repo
[openvz-utils]
name=OpenVZ utilities
baseurl=http:// 130.206.81.65/cosmos/rpms/cosmos-deps//OpenVZ/openvz-utils
enabled=1
gpgcheck=0
priority=1
[openvz-kernel-rhel6]
name=OpenVZ RHEL6-based kernel
baseurl=http:// 130.206.81.65/cosmos/rpms/cosmos-deps//OpenVZ/openvz-kernel- > rhel6
enabled=1
gpgcheck=0
priority=1
It does not also allow to modify the file previous, in the moment to execute the command (indicated in 'Step 7: applying Puppet'):
puppet apply --debug --verbose \
--modulepath [COSMOS_TMP_PATH]/puppet/modules/:[COSMOS_TMP_PATH]/puppet/modules_third_party/ \
--environment my_environment --hiera_config [COSMOS_TMP_PATH]/puppet/modules/cosmos/manifests/hiera.yaml \
--manifestdir [COSMOS_TMP_PATH]/puppet/modules/cosmos/manifests/ [COSMOS_TMP_PATH]/puppet/modules/cosmos/manifests/site.pp \
> puppet.out 2> puppet.err
It will erase the modified.
Solution: https://github.com/telefonicaid/fiware-cosmos-platform/issues/4
I need help with the error:
Error: /Stage[main]/Ambari::Server::Config/Augeas[ambari-config-repoinfo]: Could not evaluate: Saving failed, see debug
Might they to throw me a hand with these last error?
Thank you in advance.
PD: Forgive for if it is written badly

cygwin rsync protocol error

I'm trying to rsync files between 64bit windows 7 and 64bit Linux. I have rsync 3.0.7 installed on both machine (cygwin version on windows).
This is the command I'm running on windows. I see it's establishing a connection to the remote machine but it just dies after authenticating and before transferring files. What could cause this problem?
rsync -vvvv localfile REMOTE-HOST:~/
FILE_STRUCT_LEN=16, EXTRA_LEN=4
cmd=<NULL> machine=REMOTE-HOST user=guest path=~/
cmd[0]=ssh cmd[1]=-l cmd[2]=guest cmd[3]=REMOTE-HOST cmd[4]=rsync cmd[5]=--server cmd[6]=-vvvvv.Lsf cmd[7]=. cmd[8]=~/
opening connection using: ssh -l guest REMOTE-HOST rsync --server -vvvve.Lsf. "~/"
note: iconv_open("UTF-8", "UTF-8") succeeded.
Password:
FILE_STRUCT_LEN=24, EXTRA_LEN=4
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
[Receiver] _exit_cleanup(code=12, file=io.c, line=601): entered
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [Receive
r=3.0.7]
[Receiver] _exit_cleanup(code=12, file=io.c, line=601): about to call exit(12)
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
[sender] _exit_cleanup(code=12, file=/home/lapo/packaging/rsync-3.0.7-1/src/rsyn
c-3.0.7/io.c, line=601): entered
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packagi
ng/rsync-3.0.7-1/src/rsync-3.0.7/io.c(601) [sender=3.0.7]
[sender] _exit_cleanup(code=12, file=/home/lapo/packaging/rsync-3.0.7-1/src/rsyn
c-3.0.7/io.c, line=601): about to call exit(12)
The problem was a conflicting git installation. Git included it's own ssh and it's path was before cygwin path, so rsync was using git's ssh. The fix was to swap the order of git/cygwin bin path so rsync was using the right version of ssh.
I found a way around this error by adding the absolute path to ssh in my call:
rsync -avzphi --chmod=ugo=rwX --checksum --delete-after -e
"C:\cwRsync\ssh.exe -p 9999 -i c:\cwRsync\ssh\test_id_rsa"
"/cygdrive/c/repository/blah.com/"
test#xx.xx.xxx.xxx:/home/test/public_html

Resources