I'm getting serialization errors when retrieving a list of edges from a remote JanusGraph server using Gremlin in a Java application (error message and configurations bellow).
This is the traversal that raises the error:
List<Edge> eList = g.E().toList();
Trying to get only one edge also raises an error:
Edge edge= g.E().next();
Nevertheless, when retrieving a vertex or a list of vertices, it works perfectly:
List<Vertex> vList = g.V().toList();
This is my remote-objects.yaml file:
hosts: [localhost]
port: 8182
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0,
config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
This is the code to get the remote traversal:
PropertiesConfiguration conf = new PropertiesConfiguration(remote-objects.yaml);
cluster = Cluster.open(conf.getString("gremlin.remote.driver.clusterFile"));
client = cluster.connect();
g = AnonymousTraversalSource.traversal().withRemote(propFileName);
This is the serializers in my server configuration:
serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
# Older serialization versions for backwards compatibility:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV1d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV1d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV2d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV1d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV1d0, org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistryV1d0] }}
And this the error message:
java.util.concurrent.CompletionException: io.netty.handler.codec.DecoderException: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException
Serialization trace:
id (org.apache.tinkerpop.gremlin.structure.util.reference.ReferenceEdge)
at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
CompletableFuture.java:375
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
CompletableFuture.java:1934
at org.apache.tinkerpop.gremlin.driver.ResultSet.one(ResultSet.java:119)
ResultSet.java:119
at org.apache.tinkerpop.gremlin.driver.ResultSet$1.hasNext(ResultSet.java:171)
ResultSet.java:171
at org.apache.tinkerpop.gremlin.driver.ResultSet$1.next(ResultSet.java:178)
ResultSet.java:178
at org.apache.tinkerpop.gremlin.driver.ResultSet$1.next(ResultSet.java:165)
ResultSet.java:165
at org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal$TraverserIterator.next(DriverRemoteTraversal.java:140)
DriverRemoteTraversal.java:140
at org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal$TraverserIterator.next(DriverRemoteTraversal.java:125)
DriverRemoteTraversal.java:125
at org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal.nextTraverser(DriverRemoteTraversal.java:106)
DriverRemoteTraversal.java:106
at org.apache.tinkerpop.gremlin.process.remote.traversal.step.map.RemoteStep.processNextStart(RemoteStep.java:80)
RemoteStep.java:80
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
AbstractStep.java:128
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
AbstractStep.java:38
at org.apache.tinkerpop.gremlin.process.traversal.Traversal.fill(Traversal.java:179)
Traversal.java:179
at org.apache.tinkerpop.gremlin.process.traversal.Traversal.toList(Traversal.java:117)
Traversal.java:117
at com.ibm.graph.App.main(App.java:25)
App.java:25
Caused by: io.netty.handler.codec.DecoderException: org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.kryo.KryoException: java.lang.NegativeArraySizeException
I believe that your problem is in a mismatch of serializer configuration between driver and server. For the driver you do:
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0,
config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
but for the server you have:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0,
config: { ioRegistries [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0,
org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
You've omitted TinkerIoRegistryV3d0 on the driver side. With ioRegistries the order in which the registries are added matters. So, you might have gotten lucky if you'd added JanusGraphIoRegistry first on the server and then accidentally omitted TinkerIoRegistryV3d0 on the driver because the serializer registration identifiers would have aligned for JansuGraph.
Anyway, add TinkerIoRegistryV3d0 to the driver configuration OR remove it from the server configuration and things should work. As a side note, you typically only need the TinkerIoRegistryV3d0 if you're serializing TinkerGraph instances over the wire (e.g. using subgraph() step).
Related
I am working with the gatsby-source-wordpress plugin
If I hard code my API keys/secret into my Gatsby.config, everything works fine, but I want to add these as .env variables so that I can .gitignore for deployment, and this is where things are breaking.
At the root of my directory, I have a .env file which looks like this
CLIENT_SECRET=10987654321
CLIENT_ID=123456
USER=secret#secret.com
PASS=mypassword1
I'm then try to access these in gatsby.config, like this
require('dotenv').config({
path: `.env.${process.env.NODE_ENV}`
});
module.exports = {
siteMetadata: {
title: 'Gatsby Default Starter',
},
plugins: [
{
resolve: 'gatsby-source-wordpress',
options: {
baseUrl: 'myurl.com',
protocol: 'http',
hostingWPCOM: true,
useACF: false,
auth: {
wpcom_app_clientSecret: `${process.env.CLIENT_SECRET}`,
wpcom_app_clientId: `${process.env.CLIENT_ID}`,
wpcom_user: `${process.env.USER}`,
wpcom_pass: `${process.env.PASS}`,
},
},
},
{
resolve: `gatsby-plugin-emotion`,
},
'gatsby-plugin-react-helmet',
{
resolve: `gatsby-plugin-manifest`,
options: {
name: 'gatsby-starter-default',
short_name: 'starter',
start_url: '/',
background_color: '#663399',
theme_color: '#663399',
display: 'minimal-ui',
icon: 'src/images/gatsby-icon.png', // This path is
relative to the root of the site.
},
},
'gatsby-plugin-offline',
],
}
which is returning the following errors when I run either gatsby develop or gatsby build
source and transform nodesThe server response was "400 Bad Request"
source and transform nodesThe server response was "403 Forbidden"
Inner exception message : "User cannot access this private blog."
No routes to fetch. Ending.
So, the issue is the .env variables don't seem to be pulling through properly, but I can't see a reason why they wouldn't be? Is there anything I've missed in setting this up?
Gatsby doesn't know which plugin you mean (see How to use) and your overall syntax is wrong. The plugins is an array for example.
module.exports = {
plugins: [
{
resolve: "gatsby-source-wordpress",
options: {
auth: {
wpcom_app_clientSecret: process.env.CLIENT_SECRET,
wpcom_app_clientId: process.env.CLIENT_ID,
wpcom_user: process.env.USER,
wpcom_pass: process.env.PASS,
}
}
}
]
}
This should work assuming that you also define the other necessary fields mentioned in the README.
I'm trying to connect to cosmos db through the gremlin console 3.3.4, following this the remote_secure.yaml is as follows:
hosts: [*****.gremlin.cosmosdb.azure.com]
port: 443
username: /dbs/sample-database/colls/sample-collection
password: ******
connectionPool: {
enableSsl: true}
{ className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { serializeResultToString: true }}:
but when I run :remote connect tinkerpop.server conf/remote-secure.yaml
I get the following error
==>Error during 'connect' - Can't construct a java object
for tag:yaml.org,2002:org.apache.tinkerpop.gremlin.driver.Settings;
exception=Keys must be scalars but found:
<org.yaml.snakeyaml.nodes.MappingNode (tag=tag:yaml.org,2002:map,
values={ key=<org.yaml.snakeyaml.nodes.ScalarNode (tag=tag:yaml.org,2002:str, value=className)>;
value=<NodeTuple
keyNode=<org.yaml.snakeyaml.nodes.ScalarNode (tag=tag:yaml.org,2002:str,
value=className)>; valueNode=<org.yaml.snakeyaml.nodes.ScalarNode
(tag=tag:yaml.org,2002:str,
value=org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0)
>> }{ key=<org.yaml.snakeyaml.nodes.ScalarNode
(tag=tag:yaml.org,2002:str, value=config)>; value=828088650 })>
in 'reader', line 27, column 1:
hosts: [*****.gremlin.cosm ...
Any ideas what I am doing wrong?
Looks like your configuration is mangled. You are missing the serializer key on that last line:
hosts: [*****.gremlin.cosmosdb.azure.com]
port: 443
username: /dbs/sample-database/colls/sample-collection
password: ******
connectionPool: {
enableSsl: true}
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { serializeResultToString: true }}
When I used the following config deploying to digitalocean, I did a $ mup setup with no errors and then ran $ mup deploy and everything went well until it tried to prepare bundle...what am I missing here?
This is the error I'm getting with the following config when deploying...
module.exports = {
servers: {
one: {
host: 'xxx.xxx.xxx.xxx',
username: 'root',
pem: "~/.ssh/id_rsa"
}
},
app: {
name: 'hotel',
path: '../hotel',
// this is my local path to the application root
servers: {
one: {},
},
buildOptions: {
serverOnly: true,
},
env: {
ROOT_URL: 'http://xxx.xxx.xxx.xxx',
MONGO_URL: 'mongodb://localhost/meteor',
},
docker: {
image: 'abernix/meteord:base',
},
// Show progress bar while uploading bundle to server
// You might need to disable it on CI servers
enableUploadProgressBar: true
},
mongo: {
version: '3.4.1',
servers: {
one: {}
}
}
};
I am trying to setup Auth0 with Google Cloud Endpoints.
I followed instructions from
https://cloud.google.com/endpoints/docs/frameworks/python/quickstart-frameworks-python
https://cloud.google.com/endpoints/docs/openapi-configuration
https://cloud.google.com/endpoints/docs/authenticating-users
I want to call "http://10.50.10.31:8080/_ah/api/echo/v1/test/" from my iOS app with an Auth0 authenticated user I already created from the Auth0 iOS SDK.
let url = URL(string: "http://10.50.10.31:8080/_ah/api/echo/v1/test/")!
var request = URLRequest(url: url)
request.addValue("Bearer \(session.token.idToken)", forHTTPHeaderField: "Authorization")
let task = URLSession.shared.dataTask(with: request) { data, response, error in
print(data)
print(response)
}
task.resume()
The Swift code works fine. The session.idToken comes from the Auth0 iOS framework.
The file app.yaml is
runtime: python27
api_version: 1
threadsafe: true
env_variables:
# The following values are to be replaced by information from the output of
# 'gcloud service-management deploy swagger.json' command.
ENDPOINTS_SERVICE_NAME: echo-api.endpoints.duskmotion-2016.cloud.goog
ENDPOINTS_SERVICE_VERSION: 2017-02-16r0
builtins:
- appstats: on
- admin_redirect: on
- deferred: on
- remote_api: on
libraries:
- name: jinja2
version: "2.6"
- name: markupsafe
version: "0.15"
- name: pycrypto
version: "2.6.1"
- name: enum
version: "0.9.23"
- name: protorpc
version: "1.0"
- name: pytz
version: "2016.4"
- name: six
version: "1.9.0"
- name: ssl
version: "2.7.11"
- name: werkzeug
version: "0.11.10"
inbound_services:
- warmup
handlers:
- url: /favicon.ico
static_files: application/static/img/favicon.ico
upload: application/static/img/favicon.ico
- url: /robots.txt
static_files: application/static/robots.txt
upload: application/static/robots.txt
- url: /gae_mini_profiler/static
static_dir: lib/gae_mini_profiler/static
- url: /gae_mini_profiler/.*
script: lib.gae_mini_profiler.main.application
- url: /static
static_dir: application/static
- url: /admin/logout
script: run.application.app
secure: always
login: required
- url: /admin/.*
script: run.application.app
secure: always
login: admin
auth_fail_action: redirect
- url: /_ah/warmup
script: run.application.app
secure: always
- url: /_ah/.*
script: run.application.urls.api
- url: .*
script: run.application.app
secure: always
skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?.*\(dev\).tmpl
- ^(.*/)?\..*$
- ^(.*/)?setuptools/script \(dev\).tmpl$
The file main.py is
import endpoints
from protorpc import message_types
from protorpc import messages
from protorpc import remote
auth0_issuer = endpoints.Issuer(
issuer='https://duskmotion.auth0.com/',
jwks_uri='https://duskmotion.auth0.com/.well-known/jwks.json')
class EchoRequest(messages.Message):
content = messages.StringField(1)
class EchoResponse(messages.Message):
"""A proto Message that contains a simple string field."""
content = messages.StringField(1)
ECHO_RESOURCE = endpoints.ResourceContainer(
EchoRequest,
n=messages.IntegerField(2, default=1))
#endpoints.api(name='echo', version='v1', issuers={"auth0-jwk" : auth0_issuer})
class EchoApi(remote.Service):
#endpoints.method(
# This method takes a ResourceContainer defined above.
message_types.VoidMessage,
# This method returns an Echo message.
EchoResponse,
path='test',
http_method='GET',
name='echo_test_get')
def echo_test_get(self, request):
# auth = request.headers.get('Authorization', None)
import logging
logging.info(request)
user = endpoints.get_current_user()
content = "Nothing"
if user:
content = "It is working"
import logging
logging.info(content)
if not user:
raise endpoints.UnauthorizedException
return EchoResponse(content=content)
api = endpoints.api_server([EchoApi])
After being generating API file, I manually added the "Security" "x-security" and "securityDefinitions".
The file echov1openapi.json looks like
{
"basePath": "/_ah/api",
"consumes": [
"application/json"
],
"definitions": {
"ApplicationUrlsEchoResponse": {
"properties": {
"content": {
"type": "string"
}
},
"type": "object"
}
},
"host": "echo-api.endpoints.duskmotion-2016.cloud.goog",
"info": {
"title": "echo",
"version": "v1"
},
"paths": {
"/echo/v1/test": {
"get": {
"operationId": "EchoApi_echoTestGet",
"parameters": [],
"responses": {
"200": {
"description": "A successful response",
"schema": {
"$ref": "#/definitions/ApplicationUrlsEchoResponse"
}
}
},
"security": [
{
"auth0_jwk": []
}
]
}
}
},
"produces": [
"application/json"
],
"schemes": [
"http"
],
"x-security": [{
"auth0_jwk": {
"audiences": [
"xxxxxxxxxxxxxxxxxx"
]
}
}],
"securityDefinitions": {
"auth0_jwk": {
"authorizationUrl": "https://duskmotion.auth0.com/authorize",
"flow": "implicit",
"type": "oauth2",
"x-issuer": "https://duskmotion.auth0.com/",
"x-jwks_uri": "https://duskmotion.auth0.com/.well-known/jwks.json"
}
},
"swagger": "2.0"
}
When I run locally on Google App Engine development server, I get
DEBUG 2017-02-16 23:55:25,551 users_id_token.py:198] Checking for id_token.
DEBUG 2017-02-16 23:55:25,551 users_id_token.py:485] Loading certs from https://www.googleapis.com/service_accounts/v1/metadata/raw/federated-signon#system.gserviceaccount.com
DEBUG 2017-02-16 23:55:25,568 users_id_token.py:269] id_token verification failed: Invalid token signature
DEBUG 2017-02-16 23:55:25,569 users_id_token.py:209] Checking for oauth token.
DEBUG 2017-02-16 23:55:25,569 users_id_token.py:340] Fetching token info from https://www.googleapis.com/oauth2/v1/tokeninfo
ERROR 2017-02-16 23:55:25,840 users_id_token.py:349] Token info endpoint returned status 400: Invalid Value
The debug lines 2 and 5 are from myself.
Why do the *.googleapis.com endpoints are getting called instead of my *.auth0.com?
What am I missing in my settings an in the .py file to make it work?
This is a a known issue with Endpoints Frameworks on the development server, and is reported on the project issue tracker: 3rd-party auth does not work with dev-server #55.
See also the related Google Groups thread for an ongoing discussion.
I have a Meteor application that I'm trying to deploy on a VPS. I am using Meteor Up to do this.
By following the instructions I have set up my mup.js file to look like this:
module.exports = {
servers: {
one: {
host: '41.185.27.69',
username: 'root',
// pem:
password: 'secret',
// or leave blank for authenticate from ssh-agent
opts: {
port: 22
}
}
},
meteor: {
name: 'HelderbergLink',
path: '../../HelderbergLink',
servers: {
one: {}
},
buildOptions: {
serverOnly: true,
},
env: {
ROOT_URL: 'http://41.185.27.69',
MONGO_URL: 'mongodb://127.0.0.1:27017/HelderbergLink'
},
dockerImage: 'abernix/meteord:base',
deployCheckWaitTime: 60
},
mongo: {
oplog: true,
port: 27017,
servers: {
one: {},
},
},
};
After this is set up I run the following command in my .deploy directory:
mup.cmd setup
Everything setups successfully here, but then when I need to run the next command:
mup.cmd deploy
It runs through most of the process but then gives me this error:
I'm not sure what to do to resolve this. Any help would be much appreciated!