Hot questions for Using Cassandra in caching

Question:

this is my last attempt to configure Apache Ignite 2.0 to work with Cassandra as persistence layer and ODBC as query layer.

ODBC configuration is ok, I am able to put and get data in the cache with sql, but when I plug in Cassandra (version 3.9 via docker image) as persistence layer I get this:

java.lang.ClassNotFoundException: Unknown pair [platformId=0, typeId=1262449073]

I tried googling around for this exception but got no useful hint.

Here's my Ignite configuration:

        boolean persistence = true;

        IgniteConfiguration cfg = new IgniteConfiguration();
        CacheConfiguration<String, ValueClass> configuration = new CacheConfiguration<String, ValueClass>();

        configuration.setName("test-cache");
        configuration.setIndexedTypes(String.class, ValueClass.class);

        if(persistence){
            //  Configuring Cassandra's persistence
            DataSource dataSource = new DataSource();
            dataSource.setContactPoints("172.17.0.2");
            RoundRobinPolicy robinPolicy = new RoundRobinPolicy();
            dataSource.setLoadBalancingPolicy(robinPolicy);
            dataSource.setReadConsistency("ONE");
            dataSource.setWriteConsistency("ONE");
            String persistenceSettingsXml = FileUtils.readFileToString(new File(persistenceSettingsConfig), "utf-8");
            KeyValuePersistenceSettings persistenceSettings = new KeyValuePersistenceSettings(persistenceSettingsXml);
            CassandraCacheStoreFactory cacheStoreFactory = new CassandraCacheStoreFactory();
            cacheStoreFactory.setDataSource(dataSource);
            cacheStoreFactory.setPersistenceSettings(persistenceSettings);
            configuration.setCacheStoreFactory(cacheStoreFactory);
            configuration.setWriteThrough(true);
            configuration.setReadThrough(true);
            configuration.setWriteBehindEnabled(true);
        }

        //  Setting cache configuration
        cfg.setCacheConfiguration(configuration);

        //  Configuring ODBC
        OdbcConfiguration odbcConfig = new OdbcConfiguration();
        odbcConfig.setMaxOpenCursors(100);
        cfg.setOdbcConfiguration(odbcConfig);

        //  Starting Ignite
        Ignite ignite = Ignition.start(cfg);

ValueClass:

public class ValueClass implements Serializable{
    @QuerySqlField
    private Integer numberOne;
    @QuerySqlField
    private Integer numberTwo;

    public Integer getNumberOne(){ return numberOne; }

    public Integer getNumberTwo(){ return numberTwo; }

    public void setNumberOne(Integer value){
        numberOne = value;
    }
    public void setNumberTwo(Integer value){
        numberTwo = value;
    }
}

Persistence configuration:

<persistence keyspace="ignite" table="odbc_test" ttl="86400">
    <keyspaceOptions>
        REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor' : 1}
        AND DURABLE_WRITES = true
    </keyspaceOptions>

    <tableOption>
        comment = 'Cache test'
        AND read_repair_chance = 0.2
    </tableOption>

    <keyPersistence class="java.lang.String" strategy="PRIMITIVE" column="key" />

    <valuePersistence class="com.riccamini.ignite.ValueClass" strategy="POJO">
        <value name="numberOne" column="number_one"/>
        <value name="numberTwo" column="number_two"/>
    </valuePersistence>
</persistence>

Complete stack trace:

SEVERE: <test-cache> Unexpected exception during cache update
class org.apache.ignite.IgniteException: Runtime failure on search row: org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$SearchRow@6380d269
    at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree.invoke(BPlusTree.java:1615)
    at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:925)
    at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:326)
    at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1693)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2386)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1792)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1630)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:480)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:440)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1162)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:651)
    at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2345)
    at org.apache.ignite.internal.processors.cache.GridCacheAdapter.putIfAbsent(GridCacheAdapter.java:2720)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.doInsert(DmlStatementsProcessor.java:829)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.executeUpdateStatement(DmlStatementsProcessor.java:369)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFields(DmlStatementsProcessor.java:164)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFieldsTwoStep(DmlStatementsProcessor.java:198)
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:1659)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:1659)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:1657)
    at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2103)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.queryTwoStep(GridQueryProcessor.java:1657)
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:806)
    at org.apache.ignite.internal.processors.odbc.OdbcRequestHandler.executeQuery(OdbcRequestHandler.java:213)
    at org.apache.ignite.internal.processors.odbc.OdbcRequestHandler.handle(OdbcRequestHandler.java:108)
    at org.apache.ignite.internal.processors.odbc.OdbcNioListener.onMessage(OdbcNioListener.java:124)
    at org.apache.ignite.internal.processors.odbc.OdbcNioListener.onMessage(OdbcNioListener.java:33)
    at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
    at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
    at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: Unknown pair [platformId=0, typeId=1262449073]
    at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:701)
    at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1745)
    at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1704)
    at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:794)
    at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)
    at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:273)
    at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:161)
    at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:148)
    at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1730)
    at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.put(GridCacheStoreManagerAdapter.java:555)
    at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:4404)
    at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4226)
    at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:3966)
    at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:2966)
    at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree$Invoke.access$6200(BPlusTree.java:2860)
    at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree.invokeDown(BPlusTree.java:1696)
    at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree.invoke(BPlusTree.java:1585)
    ... 37 more
Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=0, typeId=1262449073]
    at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:385)
    at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:335)
    at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:692)
    ... 53 more

Jun 28, 2017 10:02:41 AM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Failed to execute SQL query [reqId=2, req=OdbcQueryExecuteRequest [cacheName=test-cache, sqlQry=INSERT INTO valueclass (_key, numberone, numbertwo) VALUES ('testkey2', 10, 10), args=[]]]
javax.cache.CacheException: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to execute DML statement [stmt=INSERT INTO valueclass (_key, numberone, numbertwo) VALUES ('testkey2', 10, 10), params=[]]
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:818)
    at org.apache.ignite.internal.processors.odbc.OdbcRequestHandler.executeQuery(OdbcRequestHandler.java:213)
    at org.apache.ignite.internal.processors.odbc.OdbcRequestHandler.handle(OdbcRequestHandler.java:108)
    at org.apache.ignite.internal.processors.odbc.OdbcNioListener.onMessage(OdbcNioListener.java:124)
    at org.apache.ignite.internal.processors.odbc.OdbcNioListener.onMessage(OdbcNioListener.java:33)
    at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
    at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
    at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to execute DML statement [stmt=INSERT INTO valueclass (_key, numberone, numbertwo) VALUES ('testkey2', 10, 10), params=[]]
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:1662)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:1659)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:1657)
    at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2103)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.queryTwoStep(GridQueryProcessor.java:1657)
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:806)
    ... 12 more
Caused by: class org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: Failed to update keys (retry update if possible).: [testkey2]
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:250)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:303)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:300)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1885)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1630)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:480)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:440)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1162)
    at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:651)
    at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2345)
    at org.apache.ignite.internal.processors.cache.GridCacheAdapter.putIfAbsent(GridCacheAdapter.java:2720)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.doInsert(DmlStatementsProcessor.java:829)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.executeUpdateStatement(DmlStatementsProcessor.java:369)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFields(DmlStatementsProcessor.java:164)
    at org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFieldsTwoStep(DmlStatementsProcessor.java:198)
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:1659)
    ... 18 more
    Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to update keys on primary node.
        at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.UpdateErrors.addFailedKeys(UpdateErrors.java:124)
        at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKeys(GridNearAtomicUpdateResponse.java:342)
        at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1883)
        ... 32 more
        Suppressed: class org.apache.ignite.IgniteException: Runtime failure on search row: org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$SearchRow@6380d269
            at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree.invoke(BPlusTree.java:1615)
            at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:925)
            at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:326)
            at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1693)
            at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2386)
            at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1792)
            ... 32 more
        Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException: Unknown pair [platformId=0, typeId=1262449073]
            at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:701)
            at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1745)
            at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1704)
            at org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:794)
            at org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)
            at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:273)
            at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:161)
            at org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:148)
            at org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1730)
            at org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.put(GridCacheStoreManagerAdapter.java:555)
            at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:4404)
            at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4226)
            at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:3966)
            at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:2966)
            at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree$Invoke.access$6200(BPlusTree.java:2860)
            at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree.invokeDown(BPlusTree.java:1696)
            at org.apache.ignite.internal.processors.cache.database.tree.BPlusTree.invoke(BPlusTree.java:1585)
            ... 37 more
        Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=0, typeId=1262449073]
            at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:385)
            at org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:335)
            at org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:692)
            ... 53 more

Any suggestion is really appreciated.


Answer:

Please try to turn on isStoreKeepBinary in cache settings - like this; please note the last line:

if (persistence){
        //  Configuring Cassandra's persistence
        DataSource dataSource = new DataSource();
        // ...here go the rest of your settings as they appear now...
        configuration.setWriteBehindEnabled(true);

        configuration.setStoreKeepBinary(true);
    }

This setting forces Ignite to avoid binary deserialization when working with underlying cache store.

Question:

I'm wondering about Ignite cache eviction in Apache Cassandra. Is it possible to evict cache entries into the cassandra (like offhaep/file eviction). As I've understood from the documentation (https://apacheignite-mix.readme.io/docs/ignite-with-apache-cassandra) current cassandra intergration is more about read/write through persistence store. But what about cassandra eviction. Is it possible?


Answer:

Entries are evicted to swap space, not persistent store: https://apacheignite.readme.io/docs/off-heap-memory#swap-space

You can implement your own Cassandra based implementation of SwapSpaceSpi and use it to evict data to Cassandra.

Question:

I have several server nodes of Apache Ignite, connected to cassandra db. Nodes works fine. Now I have to connect client nodes to this cluster. But clients nodes have not network connectivity with cassandra db and while initializing they can not to start work.

As far as I see connection is used for validate driver, because then all activities are done through caches and servers nodes.

My case is to disable connect or validate to cassandra db from clients node.

How can I do it?


Answer:

If it's an atomic cache, you can try to provide different cache configuration on server and client (with and without store).

For transactional cache store is required on clients though, because it updates the store from the node which runs a transaction.