Hot questions for Using Neo4j in tinkerpop

Question:

I'm new to graph databases (although I've extensive experience with Semantic Web technologies) and I'd like to understand if what I've in mind makes sense.

I've my own data model, made of Java's JavaBean objects, the model is rather similar to a graph, with a Node interface (and a few subclasses), an Edge interface (and a few subclasses), methods to query the model (get Node instances with attribute = 'x', get all edges for a node, etc).

I'd like to wrap this model with one of those query languages out there (let's say Cypher or Gremlin), so to have something more standardised and so that I can avoid implementing my own query language and, most importantly, my own query engine.

One obvious way would be to use Neo4j or some TinkerPop implementation as a backend for my object model (or similarly, to convert/synch my objects to a graph for one of those frameworks). However, because the model is already graph-like, has good search methods and efficient storage components (to/from simple XML files), I'm also thinking that maybe I could adapt a query language to my model. TinkerPop seems designed to support that.

Does this make sense? Is TinkerPop the best (or a good) way to go? Is/are there documentation/tutorials about that?


Answer:

As a comitter of SimpleGraph I had similar needs that led me to starting the SimpleGraph open source project in the first place.

For conversion of Pojos to and from Tinkerpop there is the ORM/OGM stack FERMA.

The idea of SimpleGraph is to "graphenize" other information sources e.g. the tabular structures of Excel Tabels or SQL databases.

Since your own data structures are already in graph form obviously the mapping to and from tinkerpop is much simpler. The SimpleGraph approach in this case would be a simple back and force (link) between the node and edge structures of so that each tinkerpop node corresponds to one of your nodes and tinkerpop each corresponds to one of your edges. I have succesfully used this approach e.g. for a graphical representation of UML models by mapping XML structural elements to tinkerpop elements and graphical representation elements in a graph editor at the same time. So my answers would be:

  • Does this make sense? Yes
  • Is TinkerPop the best (or a good) way to go? Yes
  • Is/are there documentation/tutorials about that? I'd neither say Yes and No this one

I have not seen a specific tutorial for your use case. If you experiment a bit e.g. with the SimpleGraph modules you might get a feeling how things work.

Question:

I have a gml file that I read it with java in eclipse and I want to know how to store the result of the code below in neo4j:

   TinkerGraph graph = new TinkerGraph();
   GMLReader gml= new GMLReader(graph);
   gml.inputGraph("/home/salma/Desktop/celegansneural.gml"); 

Answer:

You can import graphml and other formats from the neo4j shell with the help of the following: https://github.com/jexp/neo4j-shell-tools

Then just open the shell and use the command:

import-graphml -it /home/salma/Desktop/celegansneural.gml

The -t option tells it to import the node labels as well.

Question:

I'm using the Tinkerpop's GraphFactory.open(Configuration configuration) Java command to access a Neo4j database.

A working minimum example is:

Configuration configuration = new BaseConfiguration();
configuration.addProperty("gremlin.graph", "org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph");
configuration.addProperty("gremlin.neo4j.directory", "tmp/neo4j");
GraphFactory.open(configuration);

However, I want to connect to a remote Neo4j database instead. So I need something like

configuration.addProperty("gremlin.neo4j.directory", "ip:port");

which results in an exception:

java.lang.RuntimeException: java.io.IOException: Unable to create directory path [C:\Users\backend\192.168.56.102:7474] for Neo4j store.

Seems like Neo4j tries to access the ip:port like a file path... Thanks in advance for your help.


Answer:

As you've found, that approach won't work. Neo4jGraph is designed to work as embedded and thus requires either a file system location for the database files or it can become a node in a HA cluster which is described here.

Your other choice is to use a third-party implementation that uses the Bolt protocol, neo4j-gremlin-bolt. That should do what you want.

Question:

I was following the instructions here for configuring neo4j with gremlin-server.

However, when I was executing the command bin/gremlin-server.sh install org.apache.tinkerpop neo4j-gremlin 3.3.3, there was a problem of resolving the dependencies (I guess)

Installing dependency org.apache.tinkerpop neo4j-gremlin 3.3.3
[INFO] IndyInterface - Copying - /Users/cbd/Dev/Apps/apache-tinkerpop-gremlin-server-3.3.3/ext/neo4j-gremlin/plugin/neo4j-gremlin-3.3.3.jar
[INFO] DependencyGrabber$_copyTo_closure6 - Copying - /Users/cbd/Dev/Apps/apache-tinkerpop-gremlin-server-3.3.3/ext/neo4j-gremlin/plugin/neo4j-tinkerpop-api-0.1.jar
Could not install the dependency: java.lang.RuntimeException: Error grabbing Grapes -- [download failed: org.slf4j#slf4j-api;1.7.22!slf4j-api.jar]
java.lang.RuntimeException: java.lang.RuntimeException: Error grabbing Grapes -- [download failed: org.slf4j#slf4j-api;1.7.22!slf4j-api.jar]
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
    at org.apache.tinkerpop.gremlin.groovy.util.DependencyGrabber.getAdditionalDependencies(DependencyGrabber.groovy:198)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
    at org.apache.tinkerpop.gremlin.groovy.util.DependencyGrabber.copyDependenciesToPath(DependencyGrabber.groovy:123)
    at org.apache.tinkerpop.gremlin.server.util.GremlinServerInstall.main(GremlinServerInstall.java:38)
Caused by: java.lang.RuntimeException: Error grabbing Grapes -- [download failed: org.slf4j#slf4j-api;1.7.22!slf4j-api.jar]
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
    at groovy.grape.GrapeIvy.getDependencies(GrapeIvy.groovy:422)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
    at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:569)
    at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:532)
    at groovy.grape.Grape.resolve(Grape.java:202)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
    at org.apache.tinkerpop.gremlin.groovy.util.DependencyGrabber$_getAdditionalDependencies_closure8.doCall(DependencyGrabber.groovy:190)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
    at groovy.lang.Closure.call(Closure.java:418)
    at groovy.lang.Closure.call(Closure.java:434)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2125)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1980)
    at org.codehaus.groovy.runtime.dgm$160.doMethodInvoke(Unknown Source)
    at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:236)
    at org.apache.tinkerpop.gremlin.groovy.util.DependencyGrabber.getAdditionalDependencies(DependencyGrabber.groovy:185)
    ... 3 more

I have tried an alternative solution, by downloading manually the jar of neo4j-gremlin from maven central repository and put it to the ext folder with the same structure as other plug-ins. However, while trying to start the server, I got the following errors stack:

bin/gremlin-server.sh conf/gremlin-server-neo4j.yaml 
[INFO] GremlinServer - 
         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----

[INFO] GremlinServer - Configuring Gremlin Server from conf/gremlin-server-neo4j.yaml
[INFO] MetricManager - Configured Metrics ConsoleReporter configured with report interval=180000ms
[INFO] MetricManager - Configured Metrics CsvReporter configured with report interval=180000ms to fileName=/tmp/gremlin-server-metrics.csv
[INFO] MetricManager - Configured Metrics JmxReporter configured with domain= and agentId=
[INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=org.apache.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics
[ERROR] ServerGremlinExecutor - Could not invoke constructor on class org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager (defined by the 'graphManager' setting) with one argument of class Settings
Exception in thread "main" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:91)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:120)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:84)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:343)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:80)
    ... 3 more
Caused by: java.lang.NoClassDefFoundError: org/neo4j/tinkerpop/api/Neo4jGraphAPI
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
    at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
    at java.lang.Class.getMethod0(Class.java:3018)
    at java.lang.Class.getMethod(Class.java:1784)
    at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:78)
    at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:70)
    at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:104)
    at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.lambda$new$0(DefaultGraphManager.java:57)
    at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
    at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.<init>(DefaultGraphManager.java:55)
    ... 8 more
Caused by: java.lang.ClassNotFoundException: org.neo4j.tinkerpop.api.Neo4jGraphAPI
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 19 more
[INFO] OpLoader - Adding the standard OpProcessor.
[INFO] OpLoader - Adding the session OpProcessor.
[INFO] OpLoader - Adding the traversal OpProcessor.
[INFO] GremlinServer - Shutting down OpProcessor[]
[INFO] GremlinServer - Shutting down OpProcessor[session]
[INFO] GremlinServer - Shutting down OpProcessor[traversal]
[INFO] GremlinServer - Shutting down thread pools.
Exception in thread "gremlin-server-shutdown" java.lang.NullPointerException
    at org.apache.tinkerpop.gremlin.server.GremlinServer.stop(GremlinServer.java:253)
    at org.apache.tinkerpop.gremlin.server.GremlinServer.lambda$new$0(GremlinServer.java:101)
    at java.lang.Thread.run(Thread.java:748)

How do you configure successfully gremlin-server and neo4j?


Answer:

Did you follow these steps in configuring Grape:

http://tinkerpop.apache.org/docs/current/reference/#gremlin-applications

I've also found that you may need to clear your .m2/ or .groovy/grapes directories if things still don't work quite right - for some reason that helps sometimes.

downloading manually the jar of neo4j-gremlin from maven central repository

That approach won't work unless you grab all of neo4j-gremlin's dependencies along with it.

Question:

I am using Neo4J as my underlying GraphDB, with Tinkerpop 3.3.1 on top in my Java application. I create the Graph in Java with Neo4JGraph.open ("/tmp/filename"), and then from there I only use the Tinkerpop Java API to create nodes, edges and run traversals.

I want to perform a Dijkstra shortest path on my Tinkerpop graph. Neo4J, which is my underlying GraphDB, has this algorithm built in, so I would like to use it. I use the make this call to get the Neo4J graph, and get a few Neo4JNode types from it.

Neo4jGraphAPI neo4J = this.graphPV.getBaseGraph();
Neo4jNode start = neo4J.getNodeById(1);
Neo4jNode end = neo4J.getNodeById(2);

This works, but now I'm not sure what to do next. I figured I would pass these nodes to the Neo4J pathfinder, but the findSinglePath method takes a Node type, not a Tinkerpop based Neo4JNode. I tried casting, that threw an exception. Can I convert the Tinkerpop Neo4JNode into a Neo4J Node type that the underlying Neo4J API will accept ?

dijkstraPathFinder = GraphAlgoFactory.dijkstra(expander, costEvaluator );
WeightedPath path = dijkstraPathFinder.findSinglePath((Node)start, (Node)end );

Results in:

WARNING: service exception java.lang.ClassCastException: org.neo4j.tinkerpop.api.impl.Neo4jNodeImpl cannot be cast to org.neo4j.graphdb.Node


Answer:

Neo4jNodeImpl and Neo4jRelationshipImpl both extend Neo4jEntityImpl, which has a public getEntity() method that returns the native neo4j Node/Relationship.

Unfortunately, getEntity() is not in the Neo4jEntity interface (and is therefore not in the Neo4jNode and Neo4jRelationship interfaces). So, to use getEntity(), your project will have to add a dependency on the impl module and use a class cast. For example:

Neo4jGraphAPI neo4J = this.graphPV.getBaseGraph();
Neo4jNode start = neo4J.getNodeById(1);
Node nativeStart = ((Neo4jNodeImpl)start).getEntity();

Alternatively, if you do not want to depend on the impl module, there is a less performant option. You can use the Neo4jEntity.getId() method to get the native ID for the underlying neo4j Node/Relationship, and then perform a neo4j query to get that Node/Relationship.