Hot questions for Using Cassandra in janusgraph

Question:

I am having problems with JanusGraph connecting to Solr: I have the following:

application.java
    public static void main(String args[]) {
        JanusGraph g = JanusGraphFactory.open("/path/to/file/janusgraph-solr.properties");
        GraphOfTheGodsFactory.load(g);
        g.close();
    }

janushgraph-solr.properties

# Change to the directory where JanusGraph was extracted.  Later commands
# use relative paths to the Solr config files shipped with the JanusGraph
# distribution.
cd $JANUSGRAPH_HOME

# The name must be URL safe and should contain one dot/full-stop
# character. The part of the name after the dot must not conflict with
# any of JanusGraph's internal CF names.  Starting the part after the dot
# "solr" will avoid a conflict with JanusGraph's internal CF names.
CORE_NAME=testt
# Where to upload collection configuration and send CoreAdmin requests.
SOLR_HOST=localhost:8983

# The value of index.[X].solr.http-urls in JanusGraph's config file
# should match $SOLR_HOST and $CORE_NAME.  For example, given the
# $CORE_NAME and $SOLR_HOST values above, JanusGraph's config file would
# contain (assuming "search" is the desired index alias):
#
 index.search.solr.http-urls=http://localhost:8983/solr/testt
#
# The stock JanusGraph config file conf/janusgraph-cassandra-solr.properties
# ships with this http-urls value.

storage.backend=cassandrathrift

GraphOfTheGods file: https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-core/src/main/java/org/janusgraph/example/GraphOfTheGodsFactory.java

I get the following error:

Exception in thread "main" java.lang.IllegalArgumentException: Could not instantiate implementation: org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftStoreManager

Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend

Caused by: java.net.ConnectException: Connection refused (Connection refused)

How do i get rid of storage.backend=cassandrathrift and use solr as the backend? replacing "cassandrathrift" with solr as the document suggests fails for me. This results in the solr class not being found. Any help would be appreciated


Answer:

Enable thrift server on cassandra

nodetool enablethrift

Question:

I have deployed Cassandra 3.11.1 on an EC2 instance and connecting to it from my machine using the JanusGraph Java API. For testing purposes, the instance allows all inbound and outbound TCP connections.

In the cassandra.yaml file I have configured the following values like so:

  • listen_address: private EC2 address
  • broadcast_address: public EC2 address
  • seeds: public EC2 address
  • start_rpc: true
  • rpc_address: private EC2 address
  • endpoint_snitch: Ec2MultiRegionSnitch

Note that when I want to use the public EC2 address as rpc_address, Cassandra startup fails with this message:

Failed to bind port 9042 on <public EC2 address>

Finally I try to connect to the graph, add a vertex and commit.

JanusGraph graph = JanusGraphFactory.build()
        .set("storage.backend", "cassandra")
        .set("storage.hostname", "<public EC2 address>")
        .set("storage.cassandra.keyspace", "debug")
        .set("storage.batch-loading", false)

        .set("query.force-index", false)
        .set("query.fast-property", true)

        .set("cache.db-cache", true)
        .set("schema.default", "default")

        .set("index.search.backend", "elasticsearch")
        .set("index.search.hostname", "127.0.0.1")
        .set("index.search.elasticsearch.interface", "REST_CLIENT")
        .open();

graph.addVertex("myLabel");
graph.tx().commit();

Committing fails with the following error:

09:41:53.297 [main] ERROR o.j.g.database.StandardJanusGraph - Could not commit transaction [1] due to exception
org.janusgraph.diskstorage.locking.TemporaryLockingException: Temporary locking failure
    at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:309) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.acquireLock(ExpectedValueCheckingStore.java:103) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.acquireLock(KCVSProxy.java:52) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.diskstorage.BackendTransaction.acquireIndexLock(BackendTransaction.java:256) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:572) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:702) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1374) [janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:272) [janusgraph-core-0.2.0.jar:na]
    at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:105) [gremlin-core-3.2.6.jar:3.2.6]
    at engineering.divine.core.test.ConnectionTest.test(ConnectionTest.java:29) [classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_45]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_45]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_45]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_45]
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12]
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12]
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) [junit-4.12.jar:4.12]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) [junit-4.12.jar:4.12]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) [.cp/:na]
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Lock write retry count exceeded
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.writeSingleLock(ConsistentKeyLocker.java:341) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.writeSingleLock(ConsistentKeyLocker.java:125) ~[janusgraph-core-0.2.0.jar:na]
    at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:304) ~[janusgraph-core-0.2.0.jar:na]
    ... 32 common frames omitted

I would appreciate any pointers on what could be wrong.


Answer:

The network latency between my local machine and the EC2 instance was in fact too high to acquire a lock in time. Increasing the storage.lock.wait-time from 100ms to 1s resolved the issue.

Question:

I am using Janusgraph in embedded format with backend as cassandrathrift. Here are my properties for janusgraph and cassandra:

storage.backend=cassandrathrift
storage.cassandra.keyspace=t_graph
storage.cassandra.frame-size-mb=128
storage.hostname=10.XX.XX.XX,20.XX.XX.XX,30.XX.XX.XX

And i have a query which finds out the followers count of a particular vertex. Here is the code for that:

public class FollowCountNormal {

    private static JanusGraph graph;
    private static GraphTraversalSource traversalSource;

    public static void main(String[] args) {
        create();
        System.exit(0);
    }

    public static JanusGraph create() {

        graph = JanusGraphFactory.open("/resources/jp.properties");
        traversalSource = graph.traversal();
        getAllEdges();
        return graph;
    }

    static long getAllEdges(){

        try{
            GraphTraversal<Vertex, Vertex> allV = traversalSource.V();
            GraphTraversal<Vertex, Vertex> gt = allV.has("vid", "supernode");
            GraphTraversal<Vertex, Long> c = gt.inE()           
                    .count();
            long l = c.next();
            System.out.println("All edges = "+l);
            graph.tx().commit();
            return l;
        }catch (Exception e) {
            System.out.println("Error while fetching the edges for : ");
            e.printStackTrace();
        }

        return -1;
    }
}

This code works fine if the vertex have a limited number of incoming Edges (i.e. works fine is follower count is less than 100000). But if the followers count is in millions, i got the following exception:

Caused by: org.janusgraph.diskstorage.PermanentBackendException: Permanent failure in storage backend at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.convertException(CassandraThriftKeyColumnValueStore.java:263) at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getNamesSlice(CassandraThriftKeyColumnValueStore.java:162) at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getNamesSlice(CassandraThriftKeyColumnValueStore.java:105) at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getSlice(CassandraThriftKeyColumnValueStore.java:94) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.cache.ExpirationKCVSCache$2.call(ExpirationKCVSCache.java:100) at org.janusgraph.diskstorage.keycolumnvalue.cache.ExpirationKCVSCache$2.call(ExpirationKCVSCache.java:96) at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) ... 50 common frames omitted Caused by: org.apache.cassandra.thrift.TimedOutException: null at org.apache.cassandra.thrift.Cassandra$multiget_slice_result$multiget_slice_resultStandardScheme.read(Cassandra.java:14696) at org.apache.cassandra.thrift.Cassandra$multiget_slice_result$multiget_slice_resultStandardScheme.read(Cassandra.java:14633) at org.apache.cassandra.thrift.Cassandra$multiget_slice_result.read(Cassandra.java:14559) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_multiget_slice(Cassandra.java:741) at org.apache.cassandra.thrift.Cassandra$Client.multiget_slice(Cassandra.java:725) at org.janusgraph.diskstorage.cassandra.thrift.CassandraThriftKeyColumnValueStore.getNamesSlice(CassandraThriftKeyColumnValueStore.java:143) ... 58 common frames omitted

I looked for many posts online like this:

link1

but no solution worked for me. Any suggestion on how to solve this?


Answer:

Changing the storage backend to cql and other properties relevant to cql solved the issue for me. Here are the properties which i used:

storage.backend=cql
storage.cql.keyspace=t_graph
storage.cql.read-consistency-level=ONE