I tried to identify what Airflow "extras" has in Docker Image - here
I tried to read documentation here - but it tells me nothing about what's already included if I pull the Docker Image. It only says "These are the extras and install them from PIP". But I am already downloading Docker Image - so why do I need to install Airflow from PIP again?
I find this Dockerfile from a known tag 1.10.14 - but I interpreted it as
a. Please install the packages from apt pkg manager
b. Don't install it through PIP
I am struggling to answer "Do I have the extras e.g. LDAP in my downloaded 1.10.14 tag of Airflow, or not?"
You will find list of default extras in the official documentation from the image (BTW. The image contains standard label with URL to that documentation):
https://airflow.apache.org/docs/docker-stack/build-arg-ref.html#basic-arguments
Just scroll a bit and you will see the list.
Also the same documentation covers a lot of cases on how to extend the image or how to build your own custom image if needed (and all the other information)
https://airflow.apache.org/docs/docker-stack/build.html
BTW. Airflow 1.10.* is End of Life for more than 8 months now, so you should not use it as it contains a lot of outdated packages and even security problems that will never be fixed.
Also the image might not build any more (and this will not be fixed if it does not).
Related
I´m working with zeppeling notebook on a Docker container from the oficial apache image: https://hub.docker.com/r/apache/zeppelin
The issue is that I´m not finding the R interpreter there
If I try to import a R library I get an error saying that the interpreter is not available:
I´m not sure if can be installed from the working container, but that won´t be the best solution for this case since it would require to do it each time the container is started
R is included, but it is not available as default interpreter. You need to call it %spark.r or %r:
According to the documentation, R was included. Would you mind to provide the version of the docker image?
You can also build from the source from here: https://github.com/apache/zeppelin. At the very end of the Readme, there's way to build your own image.
Hope this works.
my first posting on setting up Yocto development environment
on my Ubuntu system (Ubuntu 18.04.3 LTS/bionic), based on the information enclosed in the document from
this web link (https://www.yoctoproject.org/docs/current/brief-yoctoprojectqs/brief-yoctoprojectqs.html).
All is well until... ~/poky/build$ bitbake core-image-sato
which results in this error:
File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named '_sqlite3'
Below is my effort to proceed past this error, which didn't resolve the
error above. Please be generous and provide some guidance. I searched for
relevant posting locations; any advice on a better place is appreciated.
Thank you.
------------------------------------------------
A web search on this error () results in:
How to Use SQLite in Ubuntu | Chron.com
with
~/poky/build$ sudo apt-get install sqlite3 libsqlite3-dev
which tells me this:
Reading package lists... Done
Building dependency tree
Reading state information... Done
libsqlite3-dev is already the newest version (3.22.0-1ubuntu0.1).
sqlite3 is already the newest version (3.22.0-1ubuntu0.1).
The following packages were automatically installed and are no longer
required:
linux-headers-5.0.0-23 linux-headers-5.0.0-23-generic linux-image-5.0.0-23-generic linux-modules-5.0.0-23-generic
linux-modules-extra-5.0.0-23-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 12 not upgraded.
So, evidently sqlite3 exists on my system. Here are the SO references that I checked:
[ImportError: No module named '_sqlite3' in python3.3][1]
[importerror no module named '_sqlite3' python3.4][2]
[ImportError: No module named _sqlite3 (even after doing eveything)][3]
[ImportError: No module named _sqlite3][4]
[1]: https://stackoverflow.com/questions/20126475/importerror-no-module-named-sqlite3-in-python3-3
[2]: https://stackoverflow.com/questions/24052137/importerror-no-module-named-sqlite3-python3-4
[3]: https://stackoverflow.com/questions/35889383/importerror-no-module-named-sqlite3-even-after-doing-eveything
[4]: https://stackoverflow.com/questions/2665337/importerror-no-module-named-sqlite3
I have just kicked off a build verifying the Brief-Quickstart steps verbatim on an otherwise fresh Ubuntu 18.04 install. There is not even SQLite installed at all, yet the build proceeds nicely. So the chances are pretty high the python installation in your development host is busted in some way or the other. Yet, there might be reasons for it:
you maybe selected python 3.5 explicitly because some other thing you did requires it
you maybe selected python 3.5 implicitly because you forwarded from on old installation, installed something that depended on it, or similar.
In any case, I'd guess that now tinkering with the link might break things somewhere else on your machine, which should be avoided IMHO.
So what are your options now? My advice would be to start building in a container, in the simplest for that requires no more than installing docker and kicking off docker run -it ubuntu:bionic /bin/bash - at least to verify things are generally working.
In the longer term you might want to make a specialized container for this with one or two additions:
1) have all the needed packages set up already
2) using a standard user instead of root.
This is the way I do things personally. An alternative would be to use the prepared things by CROPS as it is a known good solution, and it significantly reduces problems originating from host system pecularities.
I am trying to follow this tutorial, in order to install the Nix package manager in my home directory instead of /nix.
I am doing the PRoot installation (see 2. in tutorial). At the end, the
tutorial proposes to be smart in Building native packages section, to be
able to run packages without PRoot:
To run packages natively (without PRoot) they have to be build from source because all paths to the nix store are hard-coded. It is simple, really:
mkdir $HOME/nix
nix-channel --update
env NIX_STORE_DIR=$HOME/nix nix-env -i nix
And now your Nix store gets built up using the new paths. The built binaries can be run directly from there.
I did that, but I don't see how it frees me from PRoot. If I don't do the /nix mounting point with PRoot, nothing works (no nix-env executable,
I can't install new packages).
Should this NIX_STORE_DIR environment variable be put in my .bashrc ?
It seems I always need to run PRoot because ~/.nix-profile points to
a /nix/... directory:
.nix-profile -> /nix/var/nix/profiles/default
There are more steps in the tutorial (5., 6.) - should I follow them ? It seems they apply only in case of using the manual installation (step 4.),
although it is not explicit.
Any help would be appreciated :)
For anyone stumbling on this old question: there is no currently supported way to install Nix without root. The above wiki was moved to https://nixos.wiki/wiki/Nix_Installation_Guide . It may well be out of date. PRoot could work, but even then, rebuilding the whole store at a different path is not a good idea, not the least because the binary caches won't help and you'll need to build everything.
I suggest trying Nix in a virtual machine or cloud server.
Future people from Google, it's still unsupported but does work. Script here that installs a couple dependencies, builds a temporary Nix, and uses that to install a proper version in your directory of choice.
I'd like to discover the guile ecosystem. I looked at how to install a library and I didn't find a package manager, like python's pip. Does such a thing exist for guile ?
Looks like guildhall is the closest thing to pip out there. There has been some discussion on the Guile mailing lists recently around it. The posts by Wingo, Boubekki, Zaretskii, and a few others who are heavily involved with Guile development indicate a push towards making guildhall an upstream source for something called Guix that is a more general package manager intended to be independent of platform.
If you consult the Guix list of packages you will see guile there and a number of other guile related items (e.g. guile-json, guile-ncurses, etc..). I'd give that a shot. Otherwise you're on your own and you'll have to either fall back to the OS package manager or pull down the source yourself, build, and install.
Full disclosure: I haven't tried Guix myself but I've been meaning to. I'd be very interested to see how it turns out for you so if you do go this route it'd be awesome if you could provide an update with your Guix experience.
There's also been a recent call to update the libraries page and from a quick inspection there's been some small number of updates that you may find useful.
#unclejamil This is an update of my attempt to install the guix package manager.
Documentation
First of all, the links:
the official page: https://www.gnu.org/software/guix/
the download page: http://alpha.gnu.org/gnu/guix/ (guix-the-system and guix the package manager are listed together)
Installation (Debian)
Guix needs Guile-2.0-dev and more dependencies, which are present in Debian's repositories:
apt-get install guile-2.0-dev guile-2.0 libgcrypt20-dev libbz2-dev libsqlite3-dev autopoint
Download guix. See the above links to download a binary. Or get the sources:
git clone git://git.savannah.gnu.org/guix.git
The installation goes with a classical ./configure && make && make install.
make will take several minutes and make install needs root access. If you install from source, make will build guile objects of the 346 base packages (python, zsh, abiword,…) so it'll take a long time (the database is included into guix-the-program, so we must do that. You can still tweak this list in the Makefile, at MODULES) .
Note: Your current directory must not contain non ascii characters.
Note: see also this complete tutorial, with the focus on how to install guix locally, i.e. not to run make install: http://dustycloud.org/blog/guix-package-manager-without-make-install/
Usage
To install packages with guix, we need a running server.
The first method, for testing purposes, is simply to run the server in a terminal:
sudo guix-daemon
and the client in another one:
guix package -s "guile.*curses" # search with regexps
sudo guix package -i guile-ncurses # install. All start with the "package" command.
For the proper method, see https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html#Build-Environment-Setup
To be continued.
This answer is a community wiki, feel free to complete it, thanks !
I am building Guix right now and encountered the same error about not finding guile-2.0. I managed to fix it by installing the development files for guile-2.0
sudo apt-get install guile-2.0-dev
I encountered some more errors later on and it just meant I needed to install the development files for it.
I wanted to do something with qtjambi. I installed version 4.6.3. I can run the examples, but when I want to compile for example ArthurFrame, I get:
ArthurFrame.java:47: package com.trolltech.qt.core does not exist
When searching for help, it looks everything is dead. The mailing-lists do not exist anymore and on #qtjambi there is no response. Should I just not start with qtjambi, or is there another place to get help?
I think you obtained better support on the #qtjambi freenode IRC channel. When using IRC for support you should be prepared to ask your question and wait for an answer, at least 12 months but sometimes a few days, during this time you should stay connected and "idling".
From there it was discovered you are using a Linux distribution that already has automated builds of a recent QtJambi available.
Ubuntu: https://launchpad.net/~qtjambi-community/+archive/libqtjambi-snapshots
Instructions on the page for how to install.
openSuSE, SuSE, Fedora, RHEL, CentOS: https://build.opensuse.org/package/show?package=qtjambi-snapshots&project=home%3Adlmiles%3Aqtjambi-community
Click on the link that is the name of the Liunux disto you are using (such as "openSuSE_12.2").
Click on the link that says "Go to download repository".
Click on the file *.repo to download and save on the local system. Such as "home:dlmiles:qtjambi-community.repo"
Install this file as 'root' into /etc/yup.repos.d/home:dlmiles:qtjambi-community.repo
Edit the file to set the 'enabled=1' or manually add the --enablerepo=home:dlmiles:qtjambi-community when using yum to install.
Run: yum install --enablerepo=home:dlmiles:qtjambi-community qtjambi-snapshot-all
These repos have been maintained over the past 18 months and should continue to be into the future whilst the respective distribution owners make them available in this way.
Once installed in this way you will get updates as and when they are published as part of your normal system package management. So is has historically been about every 3 months.