I can start webserver normally, with the command airflow webserver. but when I use airflow webserver -D, it shows:
/data/software/miniconda3/lib/python3.7/site-packages/airflow/config_templates/airflow_local_settings.py:65: DeprecationWarning: The elasticsearch_host option in [elasticsearch] has been renamed to host - the old setting has been used, but please update your config.
ELASTICSEARCH_HOST = conf.get('elasticsearch', 'HOST')
/data/software/miniconda3/lib/python3.7/site-packages/airflow/config_templates/airflow_local_settings.py:67: DeprecationWarning: The elasticsearch_log_id_template option in [elasticsearch] has been renamed to log_id_template - the old setting has been used, but please update your config.
ELASTICSEARCH_LOG_ID_TEMPLATE = conf.get('elasticsearch', 'LOG_ID_TEMPLATE')
/data/software/miniconda3/lib/python3.7/site-packages/airflow/config_templates/airflow_local_settings.py:69: DeprecationWarning: The elasticsearch_end_of_log_mark option in [elasticsearch] has been renamed to end_of_log_mark - the old setting has been used, but please update your config.
ELASTICSEARCH_END_OF_LOG_MARK = conf.get('elasticsearch', 'END_OF_LOG_MARK')
[2019-12-25 06:37:49,537] {settings.py:252} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=855
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2019-12-25 06:37:50,031] {__init__.py:51} INFO - Using executor LocalExecutor
[2019-12-25 06:37:50,031] {dagbag.py:92} INFO - Filling up the DagBag from /data/projects/airflow/dags
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
=================================================================
Then, there is not any process running in the background later. I check with the commandps -ef|grep airflow.
The version of Airflow is 1.10.6.
In previous versions, -D used to work fine
If you can run airflow webserver normally. No errors for airflow webserver -D, and you can not find the process running as airflow webserver.
It must be you killed the background process of airflow webserver -D, and you didn't delete the pid file of airflow webserver.
You can go to the $AIRFLOW_HOME/airflow directory, check the pid files. If exists and not empty, delete it and retry:
cd $AIRFLOW_HOME/airflow
rm airflow-webserver*.pid
airflow webserver -D
Related
The --help of Cords Standalone Shell says the following:
corda-shell [-hvV] [--logging-level=<loggingLevel>] [--password=<password>]
[--sshd-hostkey-directory=<sshdHostKeyDirectory>]
[--sshd-port=<sshdPort>] [--truststore-file=<trustStoreFile>]
[--truststore-password=<trustStorePassword>]
[--truststore-type=<trustStoreType>] [--user=<user>] [-a=<host>]
[-c=<cordappDirectory>] [-f=<configFile>] [-o=<commandsDirectory>]
[-p=<port>] [COMMAND]
Now I was thinking that after specifying the appropriate parameters I can specify the [COMMAND] which I want to be executed on the node (e.g. run gracefulShutdown) but I cannot find a way to do that with standalone shell. Is there a way to do that or may be I am on wrong direction?
P.S. I need to drain and shut down the node before deploying the new cordapp in my CI/CD flow, thats why I need run gracefulShutdown.
When I try to run a command using this technique I get the following error:
Unmatched argument: help
Did you mean: install-shell-extensions?
Method-1 -- Submit/STDIN the commands via pipe.
echo -e "run nodeInfo\nexit"|java -jar corda-tools-shell-cli-4.8.7.jar \
--truststore-file /$PWD/certificates/export/rpcssltruststore.jks \
--truststore-password=apple -p=10006 -a localhost \
--user user1 --password=test
Method-2 -- Text the commands in a file and provide it as STDIN
]$ cat /tmp/commands
run nodeInfo
exit
]$ java -jar ~/bin/es/corda-tools-shell-cli-4.8.7.jar --truststore-file /$PWD/certificates/export/rpcssltruststore.jks --truststore-password=rpctruststorepassword -p=10006 -a localhost --user user1 --password=test < /tmp/commands
______ __
/ ____/ _________/ /___ _
/ / __ / ___/ __ / __ `/
/ /___ /_/ / / / /_/ / /_/ /
\____/ /_/ \__,_/\__,_/
--- Corda Enterprise Edition 4.8.7 (8baf4b1) ---
Standalone Shell connected to localhost:10006
Welcome to the Corda interactive shell.
Useful commands include 'help' to see what is available, and 'bye' to exit the shell.
Tue Jul 26 16:01:16 SGT 2022>>> run nodeInfo
addresses:
- "localhost:10005"
legalIdentitiesAndCerts:
- "O=PartyA, L=London, C=GB"
platformVersion: 10
serial: 1658810279418
Tue Jul 26 16:01:16 SGT 2022>>> exit
Have a good day!
]$
Important is to ensure and keep exit at the end.
You need to first enter into the shell to be able to issue commands to the node. The [COMMAND] parameter is probably a little misleading, The only command it has is 'install-shell-extension' which just adds an alias so that the jar can be run using a suitable name like 'corda-shell'.
Installation done properly. db initiated properly and trying to start the webserver shows the following error.
I reinstalled everything but its still not working.
I will appreciate if anyone help me.
Console output:
$:~/airflow# airflow webserver -p 8080
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2020-04-08 13:14:20,573] {__init__.py:51} INFO - Using executor SequentialExecutor
[2020-04-08 13:14:20,574] {dagbag.py:403} INFO - Filling up the DagBag from /home/cato_service/airflow/dags
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 37, in <module>
args.func(args)
File "/usr/local/lib/python3.6/dist-packages/airflow/utils/cli.py", line 75, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/airflow/bin/cli.py", line 900, in webserver
app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
File "/usr/local/lib/python3.6/dist-packages/airflow/www/app.py", line 233, in cached_app
app = create_app(config, testing)
File "/usr/local/lib/python3.6/dist-packages/airflow/www/app.py", line 103, in create_app
models.Chart, Session, name="Charts", category="Data Profiling"))
File "/usr/local/lib/python3.6/dist-packages/flask_admin/contrib/sqla/view.py", line 330, in __init__
menu_icon_value=menu_icon_value)
File "/usr/local/lib/python3.6/dist-packages/flask_admin/model/base.py", line 818, in __init__
self._refresh_cache()
File "/usr/local/lib/python3.6/dist-packages/flask_admin/model/base.py", line 913, in _refresh_cache
self._search_supported = self.init_search()
File "/usr/local/lib/python3.6/dist-packages/flask_admin/contrib/sqla/view.py", line 581, in init_search
if tools.is_hybrid_property(self.model, name):
File "/usr/local/lib/python3.6/dist-packages/flask_admin/contrib/sqla/tools.py", line 209, in is_hybrid_property
return last_name in get_hybrid_properties(last_model)
File "/usr/local/lib/python3.6/dist-packages/flask_admin/contrib/sqla/tools.py", line 190, in get_hybrid_properties
for key, prop in inspect(model).all_orm_descriptors.items()
File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/inspection.py", line 72, in inspect
"available for object of type %s" % type_
sqlalchemy.exc.NoInspectionAvailable: No inspection system is available for object of type <class 'method'>
Just hit this myself. its an issue with SQLAlchemy dependency
to fix I did the following:
pip3 uninstall SQLAlchemy
pip3 install SQLAlchemy==1.3.15
https://github.com/apache/airflow/issues/8211
Including the instruction pip install SQLAlchemy==1.3.15 in the Dockerfile and rebuilding the image has solved the issue. Thanks a lot #Java Guy!
I faced the above issue.
I added below line in Dockerfile
&& pip install SQLAlchemy==1.3.15 \
Build the docker image as
docker build --rm -t <tag> .
Run :
docker run -d -p 8080:8080 <tag>
I have issue installing Cellar on Karaf ServiceMix 7.0.0
I followed the Cellar documentation the Cellar packages are built.
Installing th Cellar feature
karaf#root>feature:repo-add cellar
Adding feature url mvn:org.apache.karaf.cellar/apache-karaf-
cellar/LATEST/xml/features
karaf#root>feature:install cellar
Error executing command: Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=cellar; type=karaf.feature; version="[4.1.1,4.1.1]"; filter:="(&(osgi.identity=cellar)(type=karaf.feature)(version>=4.1.1)(version<=4.1.1))" [caused by: Unable to resolve cellar/4.1.1: missing requirement [cellar/4.1.1] karaf.cellar.provider [caused by: Unable to resolve cellar-hazelcast/4.1.1: missing requirement [cellar-hazelcast/4.1.1] osgi.identity; osgi.identity=org.apache.karaf.cellar.utils; type=osgi.bundle; version="[4.1.1,4.1.1]"; resolution:=mandatory [caused by: Unable to resolve org.apache.karaf.cellar.utils/4.1.1: missing requirement [org.apache.karaf.cellar.utils/4.1.1] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.apache.karaf.cellar.core)(version>=4.1.0)(!(version>=5.0.0)))" [caused by: Unable to resolve org.apache.karaf.cellar.core/4.1.1: missing requirement [org.apache.karaf.cellar.core/4.1.1] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.apache.karaf.features)(version>=4.1.0)(!(version>=5.0.0)))"]]]]
I see plenty packages which are not resolved and as far I see the packages should be installed part of the feature
What am I missing?
as SMX 7.0.1 is based on Karaf 4.0.x, you have to use Cellar 4.0.x (not 4.1.x)
EDIT:
I have just installed it into fresh downloaded ServiceMix 7.0.1.
% sudo su
[sudo] password for bvn13:
co-ultraesb-test-01# mc
co-ultraesb-test-01# echo $JAVA_HOME
/usr/lib/jvm/java-1.8.0-openjdk-amd64
co-ultraesb-test-01# cd/opt
zsh: no such file or directory: cd/opt
co-ultraesb-test-01# ./bin/servicemix
Please wait while Apache ServiceMix is starting...
100% [========================================================================]
____ _ __ __ _
/ ___| ___ _ ____ _(_) ___ ___| \/ (_)_ __
\___ \ / _ \ '__\ \ / / |/ __/ _ \ |\/| | \ \/ /
___) | __/ | \ V /| | (_| __/ | | | |> <
|____/ \___|_| \_/ |_|\___\___|_| |_|_/_/\_\
Apache ServiceMix (7.0.1)
Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or 'system:shutdown' to shutdown ServiceMix.
karaf#root>feature:repo-add cellar 4.0.4
Adding feature url mvn:org.apache.karaf.cellar/apache-karaf-cellar/4.0.4/xml/features
karaf#root>feature:install cellar
karaf#root>
I'm trying to deploy a CDN using nginx container with docker swarm.
For now, i have manager in host, and one worker on VM created using docker-machine:
pierpaolo#pierpaolo-Lenovo-G580:~$ docker-machine create -d virtualbox worker1
Running pre-create checks...
Creating machine...
(worker1) Copying /home/pierpaolo/.docker/machine/cache/boot2docker.iso to /home/pierpaolo/.docker/machine/machines/worker1/boot2docker.iso...
(worker1) Creating VirtualBox VM...
(worker1) Creating SSH key...
(worker1) Starting the VM...
(worker1) Check network to re-create if needed...
(worker1) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env worker1
pierpaolo#pierpaolo-Lenovo-G580:~$ docker-machine ssh worker1
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\_______/
_ _ ____ _ _
| |__ ___ ___ | |_|___ \ __| | ___ ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__| < __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 1.12.5, build HEAD : fc49b1e - Fri Dec 16 12:44:49 UTC 2016
Docker version 1.12.5, build 7392c3b
I run manager on host:
pierpaolo#pierpaolo-Lenovo-G580:~$ docker swarm init
Swarm initialized: current node (99rr4i1w7wgn5usak9s20fikb) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token SWMTKN-1-0tfvpvdlmugavn3fyqtjn9wkd3oeqze6nmw547shp07y73whix-4dqc404txwg58zibnz3lmp30w \
192.168.1.101:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
Then, on worker1, i run:
docker#worker1:~$ docker swarm join \
> --token SWMTKN-1-0tfvpvdlmugavn3fyqtjn9wkd3oeqze6nmw547shp07y73whix-4dqc40
4txwg58zibnz3lmp30w \
> 192.168.1.101:2377
This node joined a swarm as a worker.
On host, i created overlay network:
pierpaolo#pierpaolo-Lenovo-G580:~$ docker network create -d overlay mynetwork
cb9plmvxn6twjt5lm7xgnq4ha
And created my services, with 10 replicas:
docker service create --name nginx-cdn --network mynetwork --replicas 10 -p 9999:80 nginx-new:latest
dt1rfrix71cawpr8czzy4jje4
But, as shown as following:
pierpaolo#pierpaolo-Lenovo-G580:~$ docker service ls
ID NAME REPLICAS IMAGE COMMAND
dt1rfrix71ca nginx-cdn 0/10 nginx-new:latest
pierpaolo#pierpaolo-Lenovo-G580:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a64270ac470 nginx-new:latest "nginx -g 'daemon off" 5 minutes ago Up 5 minutes 80/tcp, 443/tcp nginx-cdn.6.dhtgs9ilcj6jngxify7ko05e9
4a230d618e81 nginx-new:latest "nginx -g 'daemon off" 5 minutes ago Up 5 minutes 80/tcp, 443/tcp nginx-cdn.4.7pnxf19kpyhf3ex31zs39q884
01a76327729e nginx-new:latest "nginx -g 'daemon off" 5 minutes ago Up 5 minutes 80/tcp, 443/tcp nginx-cdn.5.b12cp73a078mjw55gsr0x6hfn
3a940f28c532 nginx-new:latest "nginx -g 'daemon off" 5 minutes ago Up 5 minutes 80/tcp, 443/tcp nginx-cdn.10.8grve8nw9fmk3tydhimh1xgng
d0e1970380d5 nginx-new:latest "nginx -g 'daemon off" 5 minutes ago Up 5 minutes 80/tcp, 443/tcp nginx-cdn.1.aval6w0ol357x8obxgjzlxnxv
On manager (host) there are only 5 of 10 replicas, and on worker1
docker#worker1:~$ docker ps --all
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
There are no containers.
Why?
If i cmd docker service ps nginx-cdn
there are some lines like these:
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR
aval6w0ol357x8obxgjzlxnxv nginx-cdn.1 nginx-new:latest pierpaolo-Lenovo-G580 Shutdown Shutdown 7 seconds ago
48ovjs0rvh6xceaydfjo1nx7j nginx-cdn.2 nginx-new:latest worker1 Shutdown Preparing 58 seconds ago
ayhw38h74o4m503q08gbbg3v1 \_ nginx-cdn.2 nginx-new:latest worker1 Shutdown Rejected about a minute ago "No such image: nginx-new:lateā¦"
But i created an image named "nginx-new:latest" and if i created a container from this image, it works.
I'm getting errors from the Studio page, and I want to see what the errors are on the server side.
I also see mention here https://github.com/edx/configuration/wiki/edX-Developer-Stack#signing-up-a-new-user-activation-message-problems that there is a console. When logged into the edX VM, how do I monitor this console?
I'm getting errors from the Studio page, and I want to see what the errors are on the server side
Try looking in /edx/var/log
When logged into the edX VM, how do I monitor this console?
Those are instructions for devstack, if you are using devstack you would be running runserver in the console which is where those messages would appear.
I'm getting errors from the Studio page, and I want to see what the errors are on the server side.
For Studio errors look in /edx/var/log/cms/edx.log inside your virtual box VM.
When logged into the edX VM, how do I monitor this console?
To run and monitor the console, perform the following steps starting from your installed vagrant directory (the one with the Vagrantfile) on your host:
First, ssh into the VM:
vagrant ssh
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
* Documentation: https://help.ubuntu.com/
New release '14.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
*******************************************************************
* ___ _ __ __ *
* / _ \ _ __ ___ _ _ ___ __| |\ \/ / *
* | |_| | '_ \ -_) ' \ / -_) _` | > < *
* \___/| .__/___|_|_| \___\__,_|/_/\_\ *
* |_| *
* *
* Instructions and troubleshooting: *
* https://github.com/edx/configuration/wiki/edX-Developer-Stack *
*******************************************************************
Last login: Tue May 12 22:21:14 2015 from 10.0.2.2
Then become the edxapp user:
vagrant#precise64:~$ sudo su edxapp -s /bin/bash
edxapp#precise64:~/edx-platform$ cd ~
edxapp#precise64:~$ source edxapp_env
Start the Studio application (--fast is optional):
edxapp#precise64:~$ cd edx-platform/
edxapp#precise64:~/edx-platform$ paver devstack --fast studio
---> pavelib.servers.devstack
---> pavelib.prereqs.install_prereqs
Enter CTL-C to end
Validating models...
0 errors found
Django version 1.4.18, using settings 'cms.envs.devstack'
Development server is running at http://0.0.0.0:8001/
Quit the server with CONTROL-C.
In this example, the first line of the console output is:
---> pavelib.servers.devstack