How can I resolve the exception reported by rocksdb when using NebulaGraph Exchange? - nebula-graph

I want to export data from NebulaGraph and used the configuration file provided by the official documentation.
The core content of the configuration file used in this example is as follows:
# Processing tags
# There are tag config examples for different dataSources.
tags: [
# export NebulaGraph tag data to csv, only support export to CSV for now.
{
name: player
type: {
source: Nebula
sink: CSV
}
# the path to save the NebulaGrpah data, make sure the path doesn't exist.
path:"hdfs://192.168.12.177:9000/vertex/player"
# if no need to export any properties when export NebulaGraph tag data
# if noField is configured true, just export vertexId
noField:false
# define properties to export from NebulaGraph tag data
# if return.fields is configured as empty list, then export all properties
return.fields:[]
# nebula space partition number
partition:10
}
Then I executed the spark task as follows:
./spark-submit --master "local" --class com.vesoft.nebula.exchange.Exchange ~/exchange-ent/nebula-exchange-ent-3.3.0.jar -c ~/exchange-ent/export_application.conf
However, the task fails to be executed, and the following error message is displayed:
org.rocksdb.RocksDBException: While open a file for appending: /path/sst/1-102.sst: No such file or directory
I checked the path and /path/sst/ exists, and I have checked that the permission of this folder also belongs to NebulaGraph.
Could anyone help hint at where I could be wrong?

Related

Cloud Build Trigger - Not in Firebase app directory error

I have problem with deploying node.js (Angular) application from Bitbucket repository to Firebase using Cloud Build Trigger.
I performed steps from this article - https://cloud.google.com/build/docs/deploying-builds/deploy-firebase
Necessary APIs are turned on,
Service account has all required permissions,
Firebase community builder is deployed and visible in Container Registry,
Cloudbuild.json file is added to repository,
Cloud Build Trigger is created and it points to one specific branch of my repository.
Problem is that after running Cloud Build Trigger I receive following error: "Error: Not in Firebase app directory (could not locate firebase.json)"
What could be reason of such error? Is it possible to point to the trigger where in repository is firebase application and firebase.json file?
EDIT:
My cloudbuild.json file is quite simple:
{
"steps": [
{
"name": "gcr.io/PROJECT_NAME/firebase",
"args": [
"deploy",
"--project",
"PROJECT_NAME",
"--only",
"functions"
]
}
]
}
Logs in Cloud Build Trigger history:
starting build "build_id"
FETCHSOURCE
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/link
* branch branch_id -> FETCH_HEAD
HEAD is now at d8bc2f1 Add cloudbuild.json file
BUILD
Pulling image: gcr.io/PROJECT/firebase
Using default tag: latest
latest: Pulling PROJECT/firebase
1e987daa2432: Already exists
a0edb687a3da: Already exists
6891892cc2ec: Already exists
684eb726ddc5: Already exists
b0af097f0da6: Already exists
154aee36a7da: Already exists
37e5835696f7: Pulling fs layer
62eb6e670f1d: Pulling fs layer
47e62615d9f9: Pulling fs layer
428cea824ccd: Pulling fs layer
765a2c722bf9: Pulling fs layer
b3f5d0a285e3: Pulling fs layer
428cea824ccd: Waiting
765a2c722bf9: Waiting
b3f5d0a285e3: Waiting
47e62615d9f9: Verifying Checksum
47e62615d9f9: Download complete
62eb6e670f1d: Verifying Checksum
62eb6e670f1d: Download complete
765a2c722bf9: Verifying Checksum
765a2c722bf9: Download complete
b3f5d0a285e3: Verifying Checksum
b3f5d0a285e3: Download complete
37e5835696f7: Verifying Checksum
37e5835696f7: Download complete
428cea824ccd: Verifying Checksum
428cea824ccd: Download complete
37e5835696f7: Pull complete
62eb6e670f1d: Pull complete
47e62615d9f9: Pull complete
428cea824ccd: Pull complete
765a2c722bf9: Pull complete
b3f5d0a285e3: Pull complete
Digest: sha256:4b6b7214d6344c8247130bf3f5443a2851e39aed3ececb32dfb2cc25a5c07e44
Status: Downloaded newer image for gcr.io/PROJECT/firebase:latest
gcr.io/PROJECT/firebase:latest
Error: Not in a Firebase app directory (could not locate firebase.json)
ERROR
ERROR: build step 0 "gcr.io/PROJECT/firebase" failed: step exited with non-zero status: 1
Firebase.json file is placed in repository in following path: /apps/firebase/firebase.json
I can't see any possibility to point that path in Cloud Build Trigger config.
I'm pretty sure that you aren't in the correct directory when you run the command. To check this, you can add this step
{
"steps": [
{
"name": "gcr.io/cloud-builders/gcloud",
"entrypoint":"ls"
"args": ["-la"]
}
]
}
If you aren't in the correct directory, and you need to be in the ./app/firebase to run your deploy, you can add this parameter in your step
{
"steps": [
{
"name": "gcr.io/PROJECT_NAME/firebase",
"dir": "app/firebase",
"args": [
"deploy",
"--project",
"PROJECT_NAME",
"--only",
"functions"
]
}
]
}
Let me know if it's better

Flywaydb multiple config files for migration is failing

We have tried to migrate some SQL versions in a single database and it went well. When to tried to implement the migrations for multiple databases at the same time by passing multiple config files is failing.
The issue is it takes only the last config file and the migration is performed only for the database mentioned in the last config file, when passed the multiple config files in "-configFiles" parameter.
Below is the screenshot of the same, it took only flywayconfdb.conf file and left other files.
[oracle#localhost flyway-5.1.4]$ ./flyway -configFiles=/home/oracle/flyway/flyway-5.1.4/conf/flyway.conf,/home/oracle/flyway/flyway-5.1.4/conf/flywayjiradb.conf,/home/oracle/flyway/flyway-5.1.4/conf/flywayconfdb.conf info
Flyway Community Edition 5.1.4 by Boxfuse
Database: jdbc:oracle:thin:#//XXXXXXXXX:1521/confdb (Oracle 12.2)
Schema version: << Empty Schema >>
+----------+---------+-------------+------+--------------+-------+
| Category | Version | Description | Type | Installed On | State |
+----------+---------+-------------+------+--------------+-------+
| No migrations found |
+----------+---------+-------------+------+--------------+-------+
Please help us in resolving the same.
Flyway merges the config files. It doesn't do a separate migration for each one.
For each config file, Flyway adds the content to a Properties map. Properties has only one value per key, so if the same key appears in a second config file it would overwrite the previous value. This is why it seems like just the settings from the last config file are used.
It allows you to define some common settings somewhere, for example in ~/flyway.conf, which could be merged with some more specific settings, e.g. in individual projects.
But it doesn't allow you to migrate multiple databases in a single run. You need to run Flyway once per database:
./flyway -configFiles=/home/oracle/flyway/flyway-5.1.4/conf/flywayjiradb.conf info
./flyway -configFiles=/home/oracle/flyway/flyway-5.1.4/conf/flywayconfdb.conf info
The documentation describes the Overriding Order as:
Command-line arguments
Environment variables
Custom config files
<current-dir>/flyway.conf
<user-home>/flyway.conf
<install-dir>/conf/flyway.conf
Flyway command-line defaults
With settings defined higher up the list having greater precedence.
The documentation gives the following example:
The means that if for example flyway.url is both present in a config
file and passed as -url= from the command-line, the command-line
argument will take precedence and be used.
The Custom config files (-configFiles) lines could be expanded as:
Command-line arguments
Environment variables
Custom config file n
...
Custom config file 2
Custom config file 1
<current-dir>/flyway.conf
<user-home>/flyway.conf
<install-dir>/conf/flyway.conf
Flyway command-line defaults
And a corresponding example could be:
The means that if for example flyway.url is both present in custom config file 1 and custom config file 2, the custom config file 2 settings will take precedence and be used.
Similarly, if the flyway.url was also in custom config file n that would override that setting from custom config file 2.

solr cloud: RROR: Error loading config name for collection

I'm trying to create a new collection with solr-cloud setup, fails with the following:
ERROR: Error loading config name for collection test
I tried deleting the collection:
sudo /opt/solr/bin/solr delete -c test
but with the same results
My setup: solr-cloid with external zookeeper and 5 solr nodes
How do I purge it or reload it again ?
thanks
Solr is not able to find configuration files in zookeeper. Solrcloud try to recreate the core from zookeeper configuration file.
Looks like you have deleted zookeeper configuration node for collection test.
Two steps to completely purge collection test:
Stop solr and Delete folder "test" if exists from Solr home folder.(default: /var/lib/solr)
Navigate to zookeeper node and edit clusterstate.json . Remove entries of collection test. I wanted to start fresh so reset clusterstate.json file to default i.e {}
you should disable autoAddReplicas property of solr before shutting down any solrnode.
Check if the config folder exists in zookeeper. If this is the case, try to link the collection with the configname using the commmand:
zkcli.sh -cmd linkconfig -collection collectionname -confname configname

How to retrieve currently applied node configuration from Riak v2.0+

Showing currently applied configuration values
In v2.0+ of Riak there is a new command option: riak config effective
Which I read as it would tell you the current running values of riak.
At any time, you can get a snapshot of currently applied
configurations through the command line. For a listing of all of the
configs currently applied in the node
Config changes applied only on start of each node?
In multiple locations in Riak documentation there is reference like:
Remember that you must stop and then re-start each node when you
change storage backends or modify any other configuration
Problem:
However when I made a change to a setting (I've tested this in both riak.conf and advanced.conf), I see the newest value when running: riak config effective
ie:
Start node: riak start
View current setting for log level: riak config effective | grep log.console.level
log.console.level = info
Change the level to debug (something that will output a lot to console.log)
Re-run: riak config effective | grep log.console.level, we get:
log.console.level = debug
Checking the console log file for debug: cat /var/log/riak/console.log | grep debug give no results (indicating the config change has not been applied)
So the question is, how can I retrieve and verify what config setting each Riak node is running under?
When Riak starts, it creates two files: 'app..config' and 'vm..config'. The default location is in a 'generated.configs' directory under the platform data directory (usually /var/lib/riak).
These files will contain the settings that were in place when Riak was started. The command riak config effective processes the current riak.conf and advanced.config files.

Could not fetch remote template - Tempest

I am trying to create heat stack using yaml file in tempest.
I have 2 yaml files.
I have to execute first file. Second file is refered in the first file
server_group autoscale group and it's init contents to generate load
scale_group:
type: OS::Heat::AutoScalingGroup
properties:
min_size: 1
max_size: 3
resource:
type: SimpleServerWithPoolMember.yaml
I get below error when i run the function
StackBuildErrorException: Stack e3f16f15-6001-4404-a7c0-999fc114df1b is in CREATE_FAILED status due to 'Resource CREATE failed: StackValidationFailed: Could not fetch remote template 'SimpleServerWithPoolMember.yaml': Invalid URL scheme '
This is a known bug for openstack-heat, and it can be reproduced on version 0.2.8.
If you want to avoid this issue, my suggestion is try to update your version to 0.2.12.

Resources