Hot questions for Using Neo4j in jdbc

Question:

I'm using neo4j-jdbc 2.3.2 as my neo4j client for java. When I executed following cypher query match(p:Person) where p.id_number='761201948V' return p.id; it will return P2547228 as node id. I feel like id is same as other properties of the node as I can use it inside where clauses. But here I'm expecting an integer which can use inside this query START p=node('node.id') return p; Is this id is an internal thing to neo4j db? and is there a way to retrieve this id? From the following two cyphers what is most efficient one?(if both referring same node)

  1. START p=node('2547223') return p;
  2. match(p:Person) where p.id='P2547228' return p;

Answer:

You have to use the ID(x) function for this. Note, that ID(x) and x.id are a complete different thing. The former returns the internal node/relationship id managed by Neo4j itself. The latter gives the id property which is managed by the user and not by the database itself.

Also note, that a node/relationship ID is always numeric.

Using START is pretty much old school and shouldn't be used any more (except for accessing manual indexes):

start p=node(2547228) return p 

This one is a equivalent statement. It is highly efficient since it just needs to do a simple seek operation on the node store:

match(p:Person) where id(p)=2547228 return p;

Looking for a property requires either a node label scan or a schema index lookup:

match(p:Person) where p.id=2547228 return p;

Just check out the query plans on your own by prefixing the statement with PROFILE.

Question:

I've got a question about getting control of query 'timeout'. What i want to achive is that i could control 'query timeout' - i want to set it let's say for 3 seconds and if query will take longer, i'll get an exception/whatever.

So far i was trying countless ways, even with overriding some api's to get to the Client within RestRequest to set it there... but i had no transaction controll.

I'm not posting any versions of components i've used/tried already, becouse it's not a big deal to change it. The only 1 restriction is java version 6.

Please, post some advices / approaches that i could use... becouse im out of ideas.

Cheers!


Answer:

You can configure the transaction timeout in conf/neo4j-server.properties with this setting:

org.neo4j.server.transaction.timeout=NUMBER_OF_SECONDS_HERE

More info available in the docs here

Edit

You can control timeout on a per query basis by enabling execution guard on the Neo4j server and adding a max-execution-time header to your request:

In conf/neo4j-server.properties set:

org.neo4j.server.webserver.limit.executiontime=MAX_POSSIBLE_MS_HERE

In conf/neo4j.properties set:

execution_guard_enabled=true

Then add header max-execution-time=100 and the timeout for that query only will be 100ms.

More information available here and here

I just tested this against both the legacy Cypher HTTP endpoint and the transactional Cypher endpoint for Neo4j 2.2.5.

Question:

I have some trouble running the example from Neo4j java jdbc. I am following the instruction from :GitHub Neo4j-JDBC But, when I arrive at :

mvn compile exec:java

I have a built error:

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project neo4j-movies: An exception occured while executing the Java class. null: InvocationTargetException: Unauthorized -> [Help 1]

How can I solve this?


Answer:

Disable auth in your server or provide the credentials to the JDBC driver as connection properties.

Properties props = new Properties()
props.setProperty("user","neo4j");
props.setProperty("password","secret");

Connection con = DriverManager.getConnection("jdbc:neo4j://localhost:7474/",props);

Question:

I have to create unique nodes in Neo4j, but this query is not correct because Neo4j says

This pattern is not supported for CREATE UNIQUE

List<String> users = pUsers.collect();
    for(String u : users){
        if(u.equals("error"))
            continue;
        else{
            String cql = " CREATE UNIQUE (n:User {value:'" + u +"'})";
            st.executeUpdate(cql);
        }

How do I solve it?


Answer:

Before using CREATE UNIQUE you need MATCH a node. After, use the matched node to create a unique pattern in the graph. CREATE UNIQUE will make the least change in the graph. I believe that what you need is a MERGE. The CREATE UNIQUE docs says:

MERGE might be what you want to use instead of CREATE UNIQUE. Note however, that MERGE doesn’t give as strong guarantees for relationships being unique.

So you can adapt your code to:

List<String> users = pUsers.collect();
for(String u : users){
    if(u.equals("error"))
        continue;
    else{
        String cql = "MERGE (n:User {value:'" + u +"'})";
        st.executeUpdate(cql);
    }
}

Take a look in this answer.

Question:

I'm trying to connect to a Neo4j 2.2.10 server using the neo4j-jdbc-driver v3.0. I am able to connect without any problems within the IDE but when I try to do the same from a Jar file I get a no suitable driver found exception as if the dependency isn't available.

java.sql.SQLException: No suitable driver found for jdbc:neo4j:http://localhost
    at java.sql.DriverManager.getConnection(DriverManager.java:689)
    at java.sql.DriverManager.getConnection(DriverManager.java:270)
    ....

Here is the relevant part of my my pom.xml:

<dependencies>    
  <dependency>
    <groupId>org.neo4j</groupId>
    <artifactId>neo4j-jdbc-driver</artifactId>
    <version>3.0</version>
  </dependency>
</dependencies>

<build>    
  <plugins>
    <plugin>
      <artifactId>maven-assembly-plugin</artifactId>
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>single</goal>
          </goals>
        </execution>
      </executions>
      <configuration>
        <descriptorRefs>
          <descriptorRef>jar-with-dependencies</descriptorRef>
        </descriptorRefs>
      </configuration>
    </plugin>
  </plugins>
</build>

I looked into including dependencies with Maven elsewhere on the forum and so that is where the plugin is taken from. However, I don't think it is including the dependency based on the error.

Previously, I was using:

<dependency>
  <groupId>org.neo4j.driver</groupId>
  <artifactId>neo4j-java-driver</artifactId>
  <version>1.0.4</version>
</dependency>

I was able to interact with a v3.0.3 Neo4j server via bolt from a Jar with no issues.

What am I doing wrong?

Thanks for any advice!


Answer:

You can verify yourself if the driver is included in the resulting artifact (supposing its name is assembly.jar):

jar tf target/assembly.jar | fgrep jdbc

It probably is, though.

I think the problem comes from the fact that the driver is automatically registered using the Service Provider Interface (SPI), by listing the classes in META-INF/services/java.sql.Driver. However, because the assembly plugin packages the dependency jars inside the assembly jar instead of exploding them and creating an uber-jar (like the shade plugin can), the SPI files are not scanned. See this blog post for example.

I suggest you try the shade plugin instead, configured with a ServicesResourceTransformer.


Update

If the SPI mechanism doesn't work in your context (jar loaded as a plugin by a container application), you can register the driver yourself if there's a place in the code you know will be called before the driver is used (some initialization service for the plugin, for example, maybe by implementing Plugin.onLoad()?): just create a instance of the driver, it will register itself.

try {
    new org.neo4j.jdbc.http.HttpDriver();
} catch (SQLException e) {
    logger.error("Could not load HttpDriver for Neo4j", e);
}

Question:

I am using neo4-jdbc with pool lib BasicDataSource.

I had huge latency problems so we profiled the app and we found out that opening connection is the cause. I didnt understand why open-connection takes so long we using pool. this is screenshot from our profiles:

This is how the Neo4jDatasourceRemote looks like:

package com.comp.wm.common.repo;

import com.comp.wm.common.utils.Constants;
import org.apache.commons.dbcp.BasicDataSource;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Value;

import javax.annotation.PostConstruct;
import java.sql.Connection;
import java.sql.SQLException;




    private final Logger logger = Logger.getLogger(Neo4jDataSourceRemote.class);

    private BasicDataSource ds;

    @Value("${neo4j.host:localhost}")
    private String NEO4J_HOST;

    @Value("${neo4j.port:7474}")
    private String NEO4J_PORT;

    @Value("${neo4j.username:nouser}")
    private String NEO4J_USERNAME;

    @Value("${neo4j.password:nopass}")
    private String NEO4J_PASSWORD;

    @Value("${neo4j.pool.size:200}")
    private int NEO4J_POOL_SIZE;

    private String GetUrl() {
        return String.format(Constants.NEO4J_JDBC_CONNECTIVITY_STRING, NEO4J_HOST, NEO4J_PORT);
    }


    @PostConstruct
    public void init(){
        ds = new BasicDataSource();
        ds.setInitialSize(300);
        ds.setDriverClassName("org.neo4j.jdbc.Driver");
        ds.setUrl(GetUrl());
        ds.setUsername(NEO4J_USERNAME);
        ds.setPassword(NEO4J_PASSWORD);
    }


    @Override
    public Connection openConnection() throws SQLException {
        return this.ds.getConnection();
    }

    @Override
    public void closeConnection(Connection conn) {
        try {
            if (conn != null)
                conn.close();
        } catch (SQLException ex) {
            logger.info("error closing connection", ex);
        }
    }
}

and this is a sample of how I execute query against the graph:

public List<NearbyItem> executeQuery(..) {
        conn = neo4jDataSource.openConnection();


        String getUsersStatement = "some query..";

        try (PreparedStatement ps = conn.prepareStatement(getUsersStatement)) {
           ..
            ResultSet rs = ps.executeQuery();
            while (rs.next()) {
               ...
            }
        } catch (Exception e) {
            throw new RuntimeException("Error returning userId=" + userIdInput, e);
        } finally {
            neo4jDataSource.closeConnection(conn);
        }
        return distItemDatas;
    }

any ideas?


Answer:

Based on comments above, I'll add this as a reply.

By default Neo4j runs in the http interface 10 threads for core. You can tweak the total number of threads in neo4j-server.properties

org.neo4j.server.webserver.maxthreads=200

However the more threads you have the more you're suffering from context switches and lock contention. If you increase the number of threads I won't expect a large increase of throughput, you just shift the point where you have to wait. From initialization (openCOnnection) to processing the query.

Question:

I'm using neo4j-jdbc in an application and get a lot of "Starting the Apache HTTP client" messages during connections.

For logging I'm using logback, slf4j and the bridges like jul-to-slf4j, jcl-over-slf4j and log4j-over-slf4j.

My configuration file is in the classpath and called logback.xml. If I set the root log level to "OFF" its working for every message beside the apache messages.

How can I turn off the "Starting the Apache HTTP client" output? I tried using log4j which also did'nt work for that message.

Setting the specific package of the httpclient (wire, org.apache, org.restlet, etc.) also didn't work.

The output is generated via a getLogger call from org.restlet.ext.httpclient, so no direct System.out.println statement or something like that.

Any thoughts on that?

Edit: These are the relevant dependencies of the pom file:

  [..]
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.5</version>
  </dependency>
  <dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.0.12</version>
  </dependency>
  <dependency>
      <groupId>org.neo4j</groupId>
      <artifactId>neo4j-jdbc</artifactId>
      <version>2.0.2</version>
    </dependency>
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>log4j-over-slf4j</artifactId>
    <version>1.7.7</version>
  </dependency>
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jul-to-slf4j</artifactId>
    <version>1.7.7</version>
  </dependency>
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jcl-over-slf4j</artifactId>
    <version>1.7.7</version>
  </dependency>
  [..]

Answer:

As Stefan Armbruster and Michael Hunger mentioned in the comments it is fixed in the master branch.

I couldn't figure out exactly why the messages are send to the console, since it looks like a proper jcl logger is used, anyway the messages are gone in the 2.1.5-SNAPSHOT version.

Question:

The statement is shown below modified almost directly from the neo4j java jdbc example.. I must be missing something

cypher.query("CREATE ({1}:{2} {name: '{3}'})", map("1",nameshort,"2",type,"3",name));

Answer:

Identifiers and labels cannot be parameterized. {1} and {2} are therefore illegal, you need to specify them for example, like this-

CREATE (p:Person {name: {3}})

Don't quote the String parameter either.