Salt request timed out. The master is not responding - salt-stack

We've suddenly started receiving the error:
Salt request timed out. The master is not responding. If this error persists after verifying the master is up, worker_threads may need to be increased."
While this seems straight forward enough to fix, bumping the worker-thread count from 5 to 100 did not solve the problem. A command like salt "*" cmd.run 'echo "hello"' was successful while the same command put in a state file and called with salt "*" state.apply hello failed. Although today even the cmd.run call is failing.
We've tried a variety of remedies that we found suggested on this sight and others for similar problems with no success, specifically: clearing the cache, killing all jobs, deleting the /var/cache/salt/minion directory and restarting the minions, rebooting the master processor, deleting /var/cache/salt/master then stopping and restarting the salt-master, making sure that there weren't any stray processes still running when it was stopped.
We aren't running with a firewall and we've verified the ports are open and communicating (as evidenced by a successful cmd.run call)
Salt version on the master:
---> salt-master --versions-report
Salt Version:
Salt: 2016.11.3
Dependency Versions:
cffi: 0.8.6
cherrypy: 3.2.2
dateutil: 1.5
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.8
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.5 (default, Oct 11 2015, 17:47:16)
python-gnupg: Not Installed
PyYAML: 3.10
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: redhat 7.2 Maipo
machine: x86_64
release: 3.10.0-327.el7.x86_64
system: Linux
version: Red Hat Enterprise Linux Server 7.2 Maipo
Salt version on the minion:
==>salt-minion --versions-report
Salt Version:
Salt: 2016.11.3
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 1.5
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.8
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.5 (default, Aug 29 2016, 10:12:21)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: redhat 7.2 Maipo
machine: x86_64
release: 3.10.0-327.36.1.el7.x86_64
system: Linux
version: Red Hat Enterprise Linux Server 7.2 Maipo
I've created a simple state file as an example to show you the output we're seeing. I've intentionally X'd out the IP values and host names in the output.
---> cat hello.sls
say hi:
cmd.run:
- name: echo "hello"
Results when run on the master:
---> salt -l debug "*" state.apply hello
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: xx.xx.xx
[DEBUG ] Missing configuration file: /root/.saltrc
[DEBUG ] Configuration file path: /etc/salt/master
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[DEBUG ] Reading configuration from /etc/salt/master
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: xx.xx.xx
[DEBUG ] Missing configuration file: /root/.saltrc
[DEBUG ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/master', 'xxx.xxx.xxx_master', 'tcp://xx.xx.xx.xx:4506', 'clear')
[DEBUG ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pub.ipc
[DEBUG ] LazyLoaded local_cache.get_load
[DEBUG ] Reading minion list from /var/cache/salt/master/jobs/9f/e5c817cfcbf1eedb46f97566072a7a3a102bd7505af34a3d42f889bb189e27/.minions.p
[DEBUG ] get_iter_returns for jid 20170608110512021926 sent to set(['xxx.xxx', 'xxx.xxx']) will timeout at 11:05:17.037291
[DEBUG ] Checking whether jid 20170608110512021926 is still running
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/master', 'xxx.xxx.xxx_master', 'tcp://xx.xx.xx.xx:4506', 'clear')
[DEBUG ] Passing on saltutil error. This may be an error in saltclient. 'retcode'
[DEBUG ] Checking whether jid 20170608110512021926 is still running
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/master', 'xxx.xxx.xxx_master', 'tcp://xx.xx.xx.xx:4506', 'clear')
[DEBUG ] Passing on saltutil error. This may be an error in saltclient. 'retcode'
[DEBUG ] Checking whether jid 20170608110512021926 is still running
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/master', 'xxx.xxx.xxx_master', 'tcp://xx.xx.xx.xx:4506', 'clear')
[DEBUG ] SaltReqTimeoutError, retrying. (1/3)
[DEBUG ] SaltReqTimeoutError, retrying. (2/3)
[DEBUG ] SaltReqTimeoutError, retrying. (3/3)
[DEBUG ] LazyLoaded nested.output
Salt request timed out. The master is not responding. If this error persists after verifying the master is up, worker_threads may need to be increased.
Results when run on the minion:
==>salt-call -l debug state.apply hello
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: xxx
[DEBUG ] Configuration file path: /etc/salt/minion
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG ] Connecting to master. Attempt 1 of 1
[DEBUG ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506')
[DEBUG ] Generated random reconnect delay between '1000ms' and '11000ms' (4260)
[DEBUG ] Setting zmq_reconnect_ivl to '4260ms'
[DEBUG ] Setting zmq_reconnect_ivl_max to '11000ms'
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506', 'clear')
[DEBUG ] Decrypting the current master AES key
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] SaltEvent PUB socket URI: /var/run/salt/minion/minion_event_64f68ad064_pub.ipc
[DEBUG ] SaltEvent PULL socket URI: /var/run/salt/minion/minion_event_64f68ad064_pull.ipc
[DEBUG ] Initializing new IPCClient for path: /var/run/salt/minion/minion_event_64f68ad064_pull.ipc
[DEBUG ] Sending event: tag = salt/auth/creds; data = {'_stamp': '2017-06-08T15:08:50.733104', 'creds': {'publish_port': 4505, 'aes': 'L/jKbh4rxM99DNrJeL6hCHTIqr4sztwZs8vqM5M9vVzOfaAe+BZrOWTFNQ87Pu7UA0T7YmGNcf0=', 'master_uri': 'tcp://xx.xx.xx.xx:4506'}, 'key': ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506')}
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] Determining pillar cache
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506', 'aes')
[DEBUG ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506')
[DEBUG ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG ] LazyLoaded jinja.render
[DEBUG ] LazyLoaded yaml.render
[DEBUG ] LazyLoaded state.apply
[DEBUG ] LazyLoaded saltutil.is_running
[DEBUG ] LazyLoaded grains.get
[DEBUG ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506', 'aes')
[DEBUG ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'xxx.xxx', 'tcp://xx.xx.xx.xx:4506')
[DEBUG ] SaltReqTimeoutError, retrying. (1/3)
[DEBUG ] SaltReqTimeoutError, retrying. (2/3)
[DEBUG ] SaltReqTimeoutError, retrying. (3/3)
[ERROR ] An un-handled exception was caught by salt's global exception handler:
SaltReqTimeoutError: Message timed out
Traceback (most recent call last):
File "/usr/bin/salt-call", line 11, in <module>
salt_call()
File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 379, in salt_call
client.run()
File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 58, in run
caller.run()
File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 134, in run
ret = self.call()
File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 197, in call
ret['return'] = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/modules/state.py", line 543, in apply_
return sls(mods, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/modules/state.py", line 1002, in sls
mocked=kwargs.get('mock', False))
File "/usr/lib/python2.7/site-packages/salt/state.py", line 3526, in __init__
BaseHighState.__init__(self, opts)
File "/usr/lib/python2.7/site-packages/salt/state.py", line 2441, in __init__
self.opts = self.__gen_opts(opts)
File "/usr/lib/python2.7/site-packages/salt/state.py", line 2468, in __gen_opts
mopts = self.client.master_opts()
File "/usr/lib/python2.7/site-packages/salt/fileclient.py", line 1338, in master_opts
return self.channel.send(load)
File "/usr/lib/python2.7/site-packages/salt/utils/async.py", line 75, in wrap
ret = self._block_future(ret)
File "/usr/lib/python2.7/site-packages/salt/utils/async.py", line 85, in _block_future
return future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 276, in send
ret = yield self._crypted_transfer(load, tries=tries, timeout=timeout, raw=raw)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 244, in _crypted_transfer
ret = yield _do_transfer()
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 228, in _do_transfer
tries=tries,
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "<string>", line 3, in raise_exc_info
SaltReqTimeoutError: Message timed out
Traceback (most recent call last):
File "/usr/bin/salt-call", line 11, in <module>
salt_call()
File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 379, in salt_call
client.run()
File "/usr/lib/python2.7/site-packages/salt/cli/call.py", line 58, in run
caller.run()
File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 134, in run
ret = self.call()
File "/usr/lib/python2.7/site-packages/salt/cli/caller.py", line 197, in call
ret['return'] = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/modules/state.py", line 543, in apply_
return sls(mods, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/modules/state.py", line 1002, in sls
mocked=kwargs.get('mock', False))
File "/usr/lib/python2.7/site-packages/salt/state.py", line 3526, in __init__
BaseHighState.__init__(self, opts)
File "/usr/lib/python2.7/site-packages/salt/state.py", line 2441, in __init__
self.opts = self.__gen_opts(opts)
File "/usr/lib/python2.7/site-packages/salt/state.py", line 2468, in __gen_opts
mopts = self.client.master_opts()
File "/usr/lib/python2.7/site-packages/salt/fileclient.py", line 1338, in master_opts
return self.channel.send(load)
File "/usr/lib/python2.7/site-packages/salt/utils/async.py", line 75, in wrap
ret = self._block_future(ret)
File "/usr/lib/python2.7/site-packages/salt/utils/async.py", line 85, in _block_future
return future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 276, in send
ret = yield self._crypted_transfer(load, tries=tries, timeout=timeout, raw=raw)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 244, in _crypted_transfer
ret = yield _do_transfer()
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 876, in run
yielded = self.gen.throw(*exc_info)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 228, in _do_transfer
tries=tries,
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 870, in run
value = future.result()
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "<string>", line 3, in raise_exc_info
salt.exceptions.SaltReqTimeoutError: Message timed out

Try removing salt and salt-minion from the minion. After that remove all files with .pyc extension from the directory ( directory which you use for your default python version ):
/usr/lib/python/site-packages/salt/*.pyc
After doing so reinstall the salt-minion. I hope this would solve your issue.

Related

google.api_core.exceptions.NotFound bucket does not exists

When I'm running data_ingestion_gcs_dag DAG in Airflow.I get error that it can not find a specified bucket, however, I rechecked it and the bucket name is fine. I have specified access to Google account with docker-compose, here is code down below, i have inserted only first part of code:
version: '3'
x-airflow-common:
&airflow-common
# In order to add custom dependencies or upgrade provider packages you can use your extended image.
# Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
# and uncomment the "build" line below, Then run `docker-compose build` to build the images.
build:
context: .
dockerfile: ./Dockerfile
environment:
&airflow-common-env
AIRFLOW__CORE__EXECUTOR: CeleryExecutor
AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow#postgres/airflow
AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:airflow#postgres/airflow
AIRFLOW__CELERY__BROKER_URL: redis://:#redis:6379/0
AIRFLOW__CORE__FERNET_KEY: ''
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
AIRFLOW__API__AUTH_BACKEND: 'airflow.api.auth.backend.basic_auth'
_PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
GOOGLE_APPLICATION_CREDENTIALS: /.google/credentials/google_credentials.json
AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT: 'google-cloud-platform://?extra__google_cloud_platform__key_path=/.google/credentials/google_credentials.json'
# TODO: Please change GCP_PROJECT_ID & GCP_GCS_BUCKET, as per your config
GCP_PROJECT_ID: 'real-dtc-de'
GCP_GCS_BUCKET: 'dtc_data_lake_real-dtc-de'
volumes:
- ./dags:/opt/airflow/dags
- ./logs:/opt/airflow/logs
- ./plugins:/opt/airflow/plugins
- ~/.google/credentials/:/.google/credentials:ro
And here is code from DAG code, presented down below:
PROJECT_ID = os.environ.get("GCP_PROJECT_ID")
BUCKET = os.environ.get("GCP_GCS_BUCKET")
Here is logs from DAG:
*** Reading local file: /opt/airflow/logs/data_ingestion_gcs_dag/local_to_gcs_task/2022-06-13T02:47:29.654918+00:00/1.log
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1032} INFO - Dependencies all met for <TaskInstance: data_ingestion_gcs_dag.local_to_gcs_task manual__2022-06-13T02:47:29.654918+00:00 [queued]>
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1032} INFO - Dependencies all met for <TaskInstance: data_ingestion_gcs_dag.local_to_gcs_task manual__2022-06-13T02:47:29.654918+00:00 [queued]>
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1238} INFO -
--------------------------------------------------------------------------------
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1239} INFO - Starting attempt 1 of 2
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1240} INFO -
--------------------------------------------------------------------------------
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1259} INFO - Executing <Task(PythonOperator): local_to_gcs_task> on 2022-06-13 02:47:29.654918+00:00
[2022-06-13, 02:47:36 UTC] {standard_task_runner.py:52} INFO - Started process 1042 to run task
[2022-06-13, 02:47:36 UTC] {standard_task_runner.py:76} INFO - Running: ['***', 'tasks', 'run', 'data_ingestion_gcs_dag', 'local_to_gcs_task', 'manual__2022-06-13T02:47:29.654918+00:00', '--job-id', '11', '--raw', '--subdir', 'DAGS_FOLDER/data_ingestion_gcs_dag.py', '--cfg-path', '/tmp/tmp11gg9aoy', '--error-file', '/tmp/tmpjbp6yrks']
[2022-06-13, 02:47:36 UTC] {standard_task_runner.py:77} INFO - Job 11: Subtask local_to_gcs_task
[2022-06-13, 02:47:36 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: data_ingestion_gcs_dag.local_to_gcs_task manual__2022-06-13T02:47:29.654918+00:00 [running]> on host aea7312db396
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1426} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=***
AIRFLOW_CTX_DAG_ID=data_ingestion_gcs_dag
AIRFLOW_CTX_TASK_ID=local_to_gcs_task
AIRFLOW_CTX_EXECUTION_DATE=2022-06-13T02:47:29.654918+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-06-13T02:47:29.654918+00:00
[2022-06-13, 02:47:36 UTC] {taskinstance.py:1700} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 2594, in upload_from_file
retry=retry,
File "/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 2396, in _do_upload
retry=retry,
File "/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1917, in _do_multipart_upload
transport, data, object_metadata, content_type, timeout=timeout
File "/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/upload.py", line 154, in transmit
retriable_request, self._get_status_code, self._retry_strategy
File "/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/_request_helpers.py", line 147, in wait_and_retry
response = func()
File "/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/upload.py", line 149, in retriable_request
self._process_response(result)
File "/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/_upload.py", line 113, in _process_response
_helpers.require_status_code(response, (http.client.OK,), self._get_status_code)
File "/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/_helpers.py", line 104, in require_status_code
*status_codes
google.resumable_media.common.InvalidResponse: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1511, in _execute_task
result = execute_callable(context=context)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/python.py", line 174, in execute
return_value = self.execute_callable()
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/python.py", line 185, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/dags/data_ingestion_gcs_dag.py", line 51, in upload_to_gcs
blob.upload_from_filename(local_file)
File "/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 2735, in upload_from_filename
retry=retry,
File "/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 2598, in upload_from_file
_raise_from_invalid_response(exc)
File "/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 4466, in _raise_from_invalid_response
raise exceptions.from_http_status(response.status_code, message, response=response)
google.api_core.exceptions.NotFound: 404 POST https://storage.googleapis.com/upload/storage/v1/b/dtc_data_lake_animated-surfer-338618/o?uploadType=multipart: {
"error": {
"code": 404,
"message": "The specified bucket does not exist.",
"errors": [
{
"message": "The specified bucket does not exist.",
"domain": "global",
"reason": "notFound"
}
]
}
}

ModuleNotFoundError: No module named 'kivymd.stiffscroll'

Possible solution:
Some suggest that this problem happens because the kivymd I installed is only on the terminal and not on my IDE (Pycharm)'s Python Interpreter. I'm not so sure about this because first, I pip install kivymd on my terminal and manually added the kivy package onto my IDE's Python interpreter. And because second, my IDE Python interpreter is at "/Desktop/CoinSnack4/venv/bin/python" and my kivymd is installed at "/Users/kienletrung/Desktop/CoinSnack4/venv" on my terminal (I found this by typing "echo $VIRTUAL_ENV" into the terminal).
Also, my program, when run with the IDE, runs fine. The error below only happens when I open the .exe file of it...
Hi please anyone could help with this my exe file is throwing up this error:
The answer in this Stackoverflow (No module named kivymd.effects when using pyinstaller)didn't work for me...
[INFO ] [Logger ] Record log in /Users/kienletrung/.kivy/logs/kivy_22-01-18_0.txt
[INFO ] [Kivy ] v2.0.0
[INFO ] [Kivy ] Installed at "/var/folders/g4/1xyrvs4j721f0tlsddrbxch00000gn/T/_MEI2YvYUa/kivy/__init__.pyc"
[INFO ] [Python ] v3.8.1 (v3.8.1:1b293b6006, Dec 18 2019, 14:08:53)
[Clang 6.0 (clang-600.0.57)]
[INFO ] [Python ] Interpreter at "/Users/kienletrung/Desktop/CoinSnack4/dist/financiaz"
[INFO ] [Factory ] 186 symbols loaded
[INFO ] [Image ] Providers: img_tex, img_imageio, img_dds, img_sdl2, img_pil (img_ffpyplayer ignored)
[INFO ] [KivyMD ] 0.104.2.dev0 (installed at "/var/folders/g4/1xyrvs4j721f0tlsddrbxch00000gn/T/_MEI2YvYUa/kivymd/__init__.pyc")
[INFO ] [Text ] Provider: sdl2
[INFO ] [Window ] Provider: sdl2
[INFO ] [GL ] Using the "OpenGL ES 2" graphics system
[INFO ] [GL ] Backend used <sdl2>
[INFO ] [GL ] OpenGL version <b'2.1 INTEL-14.7.18'>
[INFO ] [GL ] OpenGL vendor <b'Intel Inc.'>
[INFO ] [GL ] OpenGL renderer <b'Intel(R) HD Graphics 6000'>
[INFO ] [GL ] OpenGL parsed version: 2, 1
[INFO ] [GL ] Shading version <b'1.20'>
[INFO ] [GL ] Texture max size <16384>
[INFO ] [GL ] Texture max units <16>
[INFO ] [Window ] auto add sdl2 input provider
[INFO ] [Window ] virtual keyboard not allowed, single mode, not docked
[ERROR ]
Traceback (most recent call last):
File "kivy/lang/parser.py", line 472, in execute_directives
ModuleNotFoundError: No module named 'kivymd.stiffscroll'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "kivy/lang/parser.py", line 474, in execute_directives
ModuleNotFoundError: No module named 'kivymd.stiffscroll'
Traceback (most recent call last):
File "kivy/lang/parser.py", line 472, in execute_directives
ModuleNotFoundError: No module named 'kivymd.stiffscroll'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "kivy/lang/parser.py", line 474, in execute_directives
ModuleNotFoundError: No module named 'kivymd.stiffscroll'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 24, in <module>
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
File "kivymd/uix/datatables.py", line 54, in <module>
File "kivy/lang/builder.py", line 373, in load_string
File "kivy/lang/parser.py", line 402, in __init__
File "kivy/lang/parser.py", line 508, in parse
File "kivy/lang/parser.py", line 483, in execute_directives
kivy.lang.parser.ParserException: Parser: File "<inline>", line 3:
...
1:
2:#:import DEVICE_TYPE kivymd.material_resources.DEVICE_TYPE
>> 3:#:import StiffScrollEffect kivymd.stiffscroll.StiffScrollEffect
4:
5:
...
Unable to import package 'kivymd.stiffscroll.StiffScrollEffect'
[8390] Failed to execute script main
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
[Process completed]
It seems that the error happens all because the database.py file in kivymd/uix/database.py can't make the code "#:import StiffScrollEffect kivymd.stiffscroll.StiffScrollEffect" works for some reason!
If it helps, this my spec file:
# -*- mode: python ; coding: utf-8 -*-
from kivymd import hooks_path as kivymd_hooks_path
block_cipher = None
#changes the '/Users/kienletrung/Desktop/CoinSnack_MiniIA/main.py' below to the
#path to the python python file of the whole project. change the #'/Users/kienletrung/Desktop/CoinSnack_MiniIA' below to the folder that holds the previous python file.
a = Analysis(['/Users/kienletrung/Desktop/FinanciazIA/main.py'],
pathex=['/Users/kienletrung/Desktop/FinanciazIA'],
binaries=[],
datas=[],
hiddenimports=["kivymd"],
hookspath=[kivymd_hooks_path],
runtime_hooks=[],
excludes=['_tkinter', 'Tkinter', 'enchant', 'twisted'],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
#adds other files that are not the main python file with the way below: (needs to change the #'/Users/kienletrung/Desktop/CoinSnack_MiniIA/main.kv' to the appropriate path to the file)
a.datas += [('main.kv', '/Users/kienletrung/Desktop/FinanciazIA/main.kv', 'DATA'), ('Financiaz.png', '/Users/kienletrung/Desktop/FinanciazIA/Financiaz.png', 'DATA')]
#needs to change '/Users/kienletrung/Desktop/CoinSnack_MiniIA' to the path that holds the data that I add in the above line. change the "coinsnack" name to appropriate app name.
exe = EXE(pyz, Tree('/Users/kienletrung/Desktop/FinanciazIA', 'Data'),
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='financiaz',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=False )
app = BUNDLE(exe,
name='financiaz.app',
icon=None,
bundle_identifier=None)
I tried days and days but I still couldn't solve this problem... I would be forever grateful if you could help me.
Thank you so much!
pip uninstall kivymd
pip install https://github.com/kivymd/KivyMD/archive/master.zip

Forbidden: An error occurred (403) when calling the HeadObject operation:

my ~/.aws/credentials looks like
[default]
aws_access_key_id = XYZ
aws_secret_access_key = ABC
[testing]
source_profile = default
role_arn = arn:aws:iam::54:role/ad
I add my remote like
dvc remote add --local -v myremote s3://bib-ds-models-testing/data/dvc-test
I have made my .dvc/config.local to look like
[‘remote “myremote”’]
url = s3://bib-ds-models-testing/data/dvc-test
access_key_id = XYZ
secret_access_key = ABC/h2hOsRcCIFqwYWV7eZaUq3gNmS
profile=‘testing’
credentialpath = /Users/nyt21/.aws/credentials
but still after running dvc push -r myremote I get
ERROR: unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden
** Update
here is the output of dvc push -v
2021-07-25 22:40:38,887 DEBUG: Check for update is enabled.
2021-07-25 22:40:39,022 DEBUG: Preparing to upload data to 's3://bib-ds-models-testing/data/dvc-test'
2021-07-25 22:40:39,022 DEBUG: Preparing to collect status from s3://bib-ds-models-testing/data/dvc-test
2021-07-25 22:40:39,022 DEBUG: Collecting information from local cache...
2021-07-25 22:40:39,022 DEBUG: Collecting information from remote cache...
2021-07-25 22:40:39,022 DEBUG: Matched '0' indexed hashes
2021-07-25 22:40:39,022 DEBUG: Querying 1 hashes via object_exists
2021-07-25 22:40:39,644 ERROR: unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden
------------------------------------------------------------
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 246, in _call_s3
out = await method(**additional_kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/aiobotocore/client.py", line 154, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 1057, in _info
out = await self._simple_info(path)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 970, in _simple_info
out = await self._call_s3(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 265, in _call_s3
raise translate_boto_error(err)
PermissionError: Access Denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 246, in _call_s3
out = await method(**additional_kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/aiobotocore/client.py", line 154, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/main.py", line 55, in main
ret = cmd.do_run()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/command/base.py", line 50, in do_run
return self.run()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/command/data_sync.py", line 57, in run
processed_files_count = self.repo.push(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/repo/__init__.py", line 51, in wrapper
return f(repo, *args, **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/repo/push.py", line 44, in push
pushed += self.cloud.push(objs, jobs, remote=remote)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/data_cloud.py", line 79, in push
return remote_obj.push(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 57, in wrapper
return f(obj, *args, **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 494, in push
ret = self._process(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 351, in _process
dir_status, file_status, dir_contents = self._status(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 195, in _status
self.hashes_exist(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/remote/base.py", line 145, in hashes_exist
return indexed_hashes + self.odb.hashes_exist(list(hashes), **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/objects/db/base.py", line 438, in hashes_exist
remote_hashes = self.list_hashes_exists(hashes, jobs, name)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/objects/db/base.py", line 389, in list_hashes_exists
ret = list(itertools.compress(hashes, in_remote))
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
yield fs.pop().result()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/objects/db/base.py", line 380, in exists_with_progress
ret = self.fs.exists(path_info)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/dvc/fs/fsspec_wrapper.py", line 92, in exists
return self.fs.exists(self._with_bucket(path_info))
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/fsspec/asyn.py", line 87, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/fsspec/asyn.py", line 68, in sync
raise result[0]
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/fsspec/asyn.py", line 24, in _runner
result[0] = await coro
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 802, in _exists
await self._info(path, bucket, key, version_id=version_id)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 1061, in _info
out = await self._version_aware_info(path, version_id)
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 1004, in _version_aware_info
out = await self._call_s3(
File "/Users/nyt21/opt/miniconda3/envs/dvc/lib/python3.8/site-packages/s3fs/core.py", line 265, in _call_s3
raise translate_boto_error(err)
PermissionError: Forbidden
------------------------------------------------------------
2021-07-25 22:40:39,712 DEBUG: Version info for developers:
DVC version: 2.5.4 (pip)
---------------------------------
Platform: Python 3.8.10 on macOS-10.16-x86_64-i386-64bit
Supports:
http (requests = 2.26.0),
https (requests = 2.26.0),
s3 (s3fs = 2021.6.1, boto3 = 1.18.6)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk3s1s1
Repo: dvc, git
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-07-25 22:40:39,713 DEBUG: Analytics is enabled.
2021-07-25 22:40:39,765 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/var/folders/4x/xhm22wt16gl6m9nvkl9gllkc0000gn/T/tmpo86jdns5']'
2021-07-25 22:40:39,769 DEBUG: Spawned '['daemon', '-q', 'analytics', '/var/folders/4x/xhm22wt16gl6m9nvkl9gllkc0000gn/T/tmpo86jdns5']'
I can upload through python
import boto3
import os
import pickle
bucket_name = 'bib-ds-models-testing'
os.environ["AWS_PROFILE"] = "testing"
session = boto3.Session()
s3_client = boto3.client('s3')
s3_client.upload_file('/Users/nyt21/Devel/DVC/test/data/iris.csv',
'bib-ds-models-testing',
'data/dvc-test/my_iris.csv')
I don't use aws CLI but the following also gives an access deny !
aws s3 ls s3://bib-ds-models-testing/data/dvc-test
An error occurred (AccessDenied) when calling the ListObjectsV2
operation: Access Denied
but it works if I add --profile=testing
aws s3 ls s3://bib-ds-models-testing/data/dvc-test --profile=testing
PRE dvc-test/
just you know environment variable AWS_PROFILE is already set to 'testing'
UPDATE
I have tried both AWS_PROFILE='testing' and AWS_PROFILE=testing, neither of them worked.

Cause of KeyError: '_errors' when running salt state.highstate

After adding the following to my minion pillar file:
monit:
services:
- name: elasticsearch
pid: /var/run/elasticsearch/elasticsearch.pid
start_script: /etc/init.d/elasticsearch start
start_script: /etc/init.d/elasticsearch stop
port: 9200
I started receiving the following error when I tried to run highstate:
root#salt-master:/home/me# salt 'my-minion-id' state.highstate -t 300
my-minion-id:
The minion function caused an exception: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 1482, in _thread_return
return_data = executor.execute()
File "/usr/lib/python2.7/dist-packages/salt/executors/direct_call.py", line 28, in execute
return self.func(*self.args, **self.kwargs)
File "/usr/lib/python2.7/dist-packages/salt/modules/state.py", line 848, in highstate
err += __pillar__['_errors']
File "/usr/lib/python2.7/dist-packages/salt/utils/context.py", line 211, in __getitem__
return self._dict()[key]
KeyError: '_errors'
This error was quite annoying, but it turns out that it was because my pillar file contained a dictionary with duplicate keys:
monit:
services:
- name: elasticsearch
...
start_script: /etc/init.d/elasticsearch start
start_script: /etc/init.d/elasticsearch stop
...
which should have instead been:
monit:
services:
- name: elasticsearch
...
start_script: /etc/init.d/elasticsearch start
stop_script: /etc/init.d/elasticsearch stop
...
Hope this helps save someone time!

Master hostname: salt not found. Retrying in 30 seconds

I am trying to install salt stack while running the command
python C:\Python27\Scripts\salt-minion -c C:\salt\minion -l debug i am getting error message like Master hostname: salt not found. Retrying in 30 seconds
I am attaching my error report please find it
C:\salt>python C:\Python27\Scripts\salt-minion -c C:\salt\minion -l debug
WARNING: 'C:\\salt\\minion' directory does not exist.
[DEBUG ] Missing configuration file: C:\salt\minion\minion
[DEBUG ] Guessing ID. The id can be explicitly in set c:\salt\conf\minion
[INFO ] Found minion id from get_fqhostname(): HoroppaLabs
[ERROR ] Could not cache minion ID: [Errno 2] No such file or directory: 'c:\\
salt\\conf\\minion_id'
[DEBUG ] Configuration file path: c:\salt\conf\minion
[WARNING ] Failed to open log file, do you have permission to write to c:\salt\v
ar\log\salt\minion?
[INFO ] Setting up the Salt Minion "HoroppaLabs"
[DEBUG ] Created pidfile: c:\salt\var\run\salt-minion.pid
[WARNING ] You have a version of ZMQ less than ZMQ 3.2! There are known connecti
on keep-alive issues with ZMQ < 3.2 which may result in loss of contact with min
ions. Please upgrade your ZMQ!
[DEBUG ] Missing configuration file: c:\salt\conf\minion
[ERROR ] Unable to import Python wmi module, some core grains will be missing
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\salt\grains\core.py", line 50, in <module>
import wmi
ImportError: No module named wmi
[CRITICAL] Failed to load grains defined in grain file core.hwaddr_interfaces in
function <function hwaddr_interfaces at 0x00000000037AA048>, error:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\salt\loader.py", line 1127, in gen_grains
ret = fun()
File "C:\Python27\lib\site-packages\salt\grains\core.py", line 1180, in hwaddr
_interfaces
ifaces = salt.utils.network.interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 449, in inter
faces
return win_interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 402, in win_i
nterfaces
with salt.utils.winapi.Com():
AttributeError: 'module' object has no attribute 'winapi'
[CRITICAL] Failed to load grains defined in grain file core.ip6 in function <fun
ction ip6 at 0x00000000037A5EB8>, error:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\salt\loader.py", line 1127, in gen_grains
ret = fun()
File "C:\Python27\lib\site-packages\salt\grains\core.py", line 1129, in ip6
return {'ipv6': salt.utils.network.ip_addrs6(include_loopback=True)}
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 577, in ip_ad
drs6
ifaces = interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 449, in inter
faces
return win_interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 402, in win_i
nterfaces
with salt.utils.winapi.Com():
AttributeError: 'module' object has no attribute 'winapi'
[CRITICAL] Failed to load grains defined in grain file core.ip4 in function <fun
ction ip4 at 0x00000000037A5DD8>, error:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\salt\loader.py", line 1127, in gen_grains
ret = fun()
File "C:\Python27\lib\site-packages\salt\grains\core.py", line 1102, in ip4
return {'ipv4': salt.utils.network.ip_addrs(include_loopback=True)}
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 546, in ip_ad
drs
ifaces = interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 449, in inter
faces
return win_interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 402, in win_i
nterfaces
with salt.utils.winapi.Com():
AttributeError: 'module' object has no attribute 'winapi'
[CRITICAL] Failed to load grains defined in grain file core.ip_interfaces in fun
ction <function ip_interfaces at 0x00000000037A5F98>, error:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\salt\loader.py", line 1127, in gen_grains
ret = fun()
File "C:\Python27\lib\site-packages\salt\grains\core.py", line 1159, in ip_int
erfaces
ifaces = salt.utils.network.interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 449, in inter
faces
return win_interfaces()
File "C:\Python27\lib\site-packages\salt\utils\network.py", line 402, in win_i
nterfaces
with salt.utils.winapi.Com():
AttributeError: 'module' object has no attribute 'winapi'
[ERROR ] This master address: 'salt' was previously resolvable but now fails t
o resolve! The previously resolved ip addr will continue to be used
[WARNING ] Master hostname: salt not found. Retrying in 30 seconds
Did you use the Salt Windows installer found here? http://docs.saltstack.com/en/latest/topics/installation/windows.html
Once you've installed the Salt minion you can start up the minion in a terminal with elevated privileges like this:
c:\salt\salt-minion.exe -l debug -c c:\salt\conf
You'll want to modify c:\salt\conf\minion and set the "master" option to be the IP or hostname of your master.

Resources