Is there any way to install two nebula graph databases (v2.6 and v3.3) in the same k8s cluster?
I tried but met the following problem:
When I recreate a crd file, it reports that it exists.
To my knowledge, the two versions v2.6 and v3.3 can only exist one because they are incompatible. And crd should not be the same name.
Related
We are using self hosted Jfrog Artifactory version 6.20.0. It is single node infrastructure. We have hosted Docker-compose environment in single VM, where we are using nginx, artofactory-pro, Postgress DB containers.
Now we have plan to upgrade the Artifactory and convert it from standalone to cluster. I have following questions:
can we directly upgrade from version 6.20.0 to version Artifactory 7.31.13?
Any document or guidance to move from single node to cluster (active/active) of 2 nodes?
cluster in docker compose means each node has its own Postgres DB, Artifactory containers and then load balancer on top of two VMs containing these two containers. Did I understand it right or am I missing something?
Yes, you can directly upgrade from 6.20.0 to 7.31.13.
Refer this JFrog Confluence page for the Artifactory upgrade.
Each node can not have its own DB, both the nodes will rely on single postgres database whereas the Artifactory load will be distributed between the nodes.
So the best approach for you is as below.
Perform an upgrade on single node first.
Once the upgrade is successful, you can just add the secondary node by connecting to the same database. Visit this page, before adding the secondary node.
For adding node after visiting the requirements page, visit this page for adding another node for docker-compose method.
If you have any queries further/issues while upgrading or adding the node, you may reach JFrog Support.
I have been able to successfully install and start the cloudera CDH5 server and manager and all the core projects along with that, viz. HDFS, HUE, HIVE etc. However recently I deleted the temporary hdfs directory (/dfs/*) and then formatted the namenode due to certain issues. Now I find all new sorts of issues which I am not able solve.
Some are given as below:
The problem with hue,
The problem with HDFS,
Any help would highly be appreciated.
Thanking in advance.
Edit: I have tried creating all those missing directories in both HDFS as well as local FS and have tried various owners for them to without success.
What helped me, being the most easy solution, is deleting all those services from the cloudera web manager and readding them back.
I am trying to setup apache drill in distributed mode. I already have cloudera hadoop cluster with a master and 2 slaves. From documentation given on apache drill, its not pretty clear if it can be set up with typical cloudera cluster. I could not find any relevant articles. Any kind of help will be appreciated.
Drill can be installed along with Cloudera on the nodes of the cluster independently - and would be able to query the files on HDFS.
Refer the link for installation details -
https://cwiki.apache.org/confluence/display/DRILL/Deploying+Apache+Drill+in+a+Clustered+Environment
I got this working with cloudera hadoop distribution. I already had cloudera cluster installed with all services running.
perform following steps:
Install apache drill on all nodes of the cluster.
Run drill/bin/drillbit.sh on each node.
Configure storage plugin for dfs using apache drill webinterface at host:8047. Update HDFS configurations here.
Run Sqlline : ./sqlline -u jdbc:drill:zk=host1:2181,host2:2181,host3:2181
(2181 is the port number used by zookeeper.)
It may only work with a rudimentary insecure cluster as Drill currently isn't tested / documented to integrate with HDFS + Kerberos for secure Hadoop clusters. Vote and check back on this ticket for Drill secure HDFS support:
https://issues.apache.org/jira/browse/DRILL-3584
Problem:
I would like to make julia available for our developers on our corporate network, which has no internet access at all (no proxy), due to sensitive data.
As far as I understand julia is designed to use github.
For instance julia> Pkg.init() tries to access:
git://github.com/JuliaLang/METADATA.jl
Example:
I solved this problem for R by creating a local CRAN repository (rsync) and setting up a local webserver.
I also solved this problem for python the same way by creating a local PyPi repository (bandersnatch) + webserver.
Question:
Is there a way to create a local repository for metadata and packages for julia?
Thank you in advance.
Roman
Yes, one of the benefits from using the Julia package manager is that you should be able to fork METADATA and host it anywhere you'd like (and keep a branch where you can actually check new packages before allowing your clients to update). You might be one of the first people to actually set up such a system, so expect that you will need to submit some issues (or better yet; pull requests) in order to get everything working smoothly.
See the extra arguments to Pkg.init() where you specify the METADATA repo URL.
If you want a simpler solution to manage I would also think about having a two tier setup where you install packages on one system (connected to the internet), and then copy the resulting ~/.julia directory to the restricted system. If the packages you use have binary dependencies, you might run into problems if you don't have similar systems on both sides, or if some of the dependencies is installed globally, but Pkg.build("Pkgname") might be helpful.
This is how I solved it (for now), using second suggestion by
ivarne.I use a two tier setup, two networks one connected to internet (office network), one air gapped network (development network).
System information: openSuSE-13.1 (both networks), julia-0.3.5 (both networks)
Tier one (office network)
installed julia on an NFS share, /sharename/local/julia.
soft linked /sharename/local/bin/julia to /sharename/local/julia/bin/julia
appended /sharename/local/bin/ to $PATH using a script in /etc/profile.d/scriptname.sh
created /etc/gitconfig on all office network machines: [url "https://"] insteadOf = git:// (to solve proxy server problems with github)
now every user on the office network can simply run # julia
Pkg.add("PackageName") is then used to install various packages.
The two networks are connected periodically (with certain security measures ssh, firewall, routing) for automated data exchange for a short period of time.
Tier two (development network)
installed julia on NFS share equal to tier one.
When the networks are connected I use a shell script with rsync -avz --delete to synchronize the .julia directory of tier one to tier two for every user.
Conclusion (so far):
It seems to work reasonably well.
As ivarne suggested there are problems if a package is installed AND something more than just file copying is done (compiled?) on tier one, the package wont run on tier two. But this can be resolved with Pkg.build("Pkgname").
PackageCompiler.jl seems like the best tool for using modern Julia (v1.8) on secure systems. The following approach requires a build server with the same architecture as the deployment server, something your institution probably already uses for developing containers, etc.
Build a sysimage with PackageCompiler's create_sysimage()
Upload the build (sysimage and depot) along with the Julia binaries to the secure system
Alias a script to julia, similar to the following example:
#!/bin/bash
set -Eeu -o pipefail
unset JULIA_LOAD_PATH
export JULIA_PROJECT=/Path/To/Project
export JULIA_DEPOT_PATH=/Path/To/Depot
export JULIA_PKG_OFFLINE=true
/Path/To/julia -J/Path/To/sysimage.so "$#"
I've been able to run a research pipeline on my institution's secure system, for which there is a public version of the approach.
I have the Titan server, with Cassandra, installed here, with multiple keyspaces configured.
I've read many threads about how to back up and restore a keyspace, but all talk about using sstableloader.
However, I didn't find this tool, since the Titan installation I've used came with Cassandra, and there is not an exclusive bin folder for Cassandra on it.
I wonder how do I backup and restore a keyspace with these conditions.
Thanks in advance.
One thing you could do is just separately download the version of Cassandra compatible with the one that packaged with your Titan version. I routinely do that to get nodetool and the cassandra-cli.