diff --git a/CHANGELOG b/CHANGELOG index 5443c55c3..49c4d5d18 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -7,20 +7,20 @@ Versions above 0.7.0-17 work well with cassandra's 0.7.* (after 0.7.0-beta2) 0.7.0-27 ======== -Execution time is now actually in micro seconds. Thanks to several folks for pointing this out. +Execution time is now actually in micro seconds. Thanks to several folks for pointing this out. Added IO stream mutators. Thanks to Norman Maurer for the pull request. Added method to HFactory for safe shutdown of cluster Added suspend/unsuspend operations to HConnectionManager, operations for suh in JMX Cleanup of numActive counter on ConcurrentHClientPool. Could cause negative results under some high-load situations. Thanks to Yang Yang for the tip. fix NPE with poolExhaustedException. Thanks to Yang Yang for the tip -Fixed issue with ttl not being set on result columns. Thanks to ML user Kallin for the heads up. +Fixed issue with ttl not being set on result columns. Thanks to ML user Kallin for the heads up. 0.7.0-26 ======== Fix issue with NPE in ThriftColumnDef when a column name was declared with no index type (thanks to Steve Willcox for the heads up) Mutator has easier row level deletion methods Fix potential race condition in ConcurrentHClientPool with large number of threads. https://github.com/rantav/hector/issues#issue/136 (Thanks to @masseyke for the test case) -LeastActive policy updated to shuffle before selection. This spreads load out better among pools in low to moderate traffic environments (ie. when ConcurrentHClientPool#getNumActive are all equal). +LeastActive policy updated to shuffle before selection. This spreads load out better among pools in low to moderate traffic environments (ie. when ConcurrentHClientPool#getNumActive are all equal). Fix NPE in checking that cassandraHostRetryService is turned when host is being marked down (thanks to ML poster Shimi for bug report) A CassandraClientMonitor instance is now created per cluster to make JMX integration with multi-cluster setups much easier. (https://github.com/rantav/hector/issues/issue/137) @@ -33,7 +33,7 @@ Fix issue with RR Load balancing policy that cause ArrayOutOfBounds.. in some si 0.7.0-24 ======== Fixed issue with null keyspaces and credentials in HConnectionManager (via Michael Moores) -Added class-level default for failoverPolicy +Added class-level default for failoverPolicy Rudimentary (very much so) JPA 2.0 support for save and load via openjpa Ligthweight ORM for JPA 1.0 annotations (additional improvements to BTodd's hector-object-mapper merged in) Some cleanup of bytebuffer handling in serializers @@ -47,9 +47,9 @@ CassandraHostRetryService default queue size now defaults unbounded Initial import of BToddB's hector-object-mapper as object-mapper module Object mapper ported over to JPA annotations and EntityManager (very redimentary support) Fix issue with createKeyspace requiring additional column info -Massive changes to pom structure courtesy of Stephen Connolly to facilitate inclusion into maven central repository. +Massive changes to pom structure courtesy of Stephen Connolly to facilitate inclusion into maven central repository. Small tweaks to HConnectionManager courtesy of Benoit Perroud -Collection conversions in AbstractSerializer use the size of the provided collection for initialization. Patch courtesy of Benoit Perroud. +Collection conversions in AbstractSerializer use the size of the provided collection for initialization. Patch courtesy of Benoit Perroud. Treat HUnavailableException the same as HTimeoutException 0.7.0-22 @@ -57,7 +57,7 @@ Treat HUnavailableException the same as HTimeoutException Fix issue with LeastActiveBalancingPolicy that favoured same host repeatedly Define ClockResolution as an Interface to allow client to define their own implementations Added TimeUUID support (me.prettyprint.cassandra.utils.TimeUUIDUtils) -Added updateColumnFamily to Cluster +Added updateColumnFamily to Cluster Shorterm fix for catching InvalidRequestException and handling it correctly for a bootstraping node @@ -89,7 +89,7 @@ Refactoring of the connection pooling innards to fix epic race condition on fail - Removal of classes effectively matching the pattern CassandraClient* from the service package - Command is gone. It was no longer needed and was more confusing than anything else - Pooling logic has been stress tested by a neutral third party (no, really!) -Upgrade to thrift 0.5 to match Cassandra (it caught us by surprise as well). +Upgrade to thrift 0.5 to match Cassandra (it caught us by surprise as well). The system_rename_* methods were removed matching removal of such in Thrift API Microsecond level granularity is now the default NodeAutoDiscoverService will periodically look for new hosts on the ring and add them (off by default) @@ -103,13 +103,13 @@ Move all the API stuff to me.prettyprint.hector.api.*. Extract interfaces and pr - Rename KeyspaceOperator and extract a Keyspace interface from it Rename a few exceptions to begin with HSomething so they are hard to unintentionally mix with their thrift doubles. Bug fixes: - CassandraClientPoolByHostImpl can throw NoSuchElementException + CassandraClientPoolByHostImpl can throw NoSuchElementException KeyspaceImpl.toString returns super.toString() API V2 has no means of getting all columns from a row. KeyspaceOperator throws NPE that masks real exceptions Timestamp (and Clock in 0.7.0) are not set on HColumn in the constructor Friendlier API and spring integration - Error in failover - incorrect operation when borrowClient throws an + Error in failover - incorrect operation when borrowClient throws an batchMutate doesn't work with null predicate in deletion. Cannot batch-delete rows. getSuperColumn() does not return null KeyspaceImpl.getSuperColumn() should use cassandra.get() and not cassandra.get_slice() diff --git a/README b/README index 83bec4fdf..4b70f50de 100644 --- a/README +++ b/README @@ -1,6 +1,6 @@ The current 0.7.0 branch will work with Apache Cassandra 0.7.x. The master has been switched to tracking Apache Cassandra 0.8.0. Current releases on the downloads section should be cosidered production ready. You should always choose the most recent release for your version of Apache Cassandra. -As of version 0.7.0-23, Hector artificats are deployed to Maven Central. If you use maven for your build system, you need only include the hector-core dependency and all related dependencies will be managed automatically. +As of version 0.7.0-23, Hector artificats are deployed to Maven Central. If you use maven for your build system, you need only include the hector-core dependency and all related dependencies will be managed automatically. ---------------------------------------------------------------------------------------------------- Hector is a high level Java client for Apache Cassandra. @@ -25,9 +25,10 @@ Some features provided by this client: o simple ORM layer that works o a type-safe approach to dealing with Apache Cassandra's data model -Detailed documentation of Hector features and usage can be found in PDF form hosted by Riptano: http://www.riptano.com/sites/default/files/hector-v2-client-doc.pdf +Detailed documentation of Hector features and usage can be found on the wiki: +https://github.com/rantav/hector/wiki/User-Guide -Some interesting pages from the wiki: +Some additional pages from the wiki that may be of interest: o SLF4J fun and hijinks: https://github.com/rantav/hector/wiki/SLF4J-in-Hector- o Mailing Lists: https://github.com/rantav/hector/wiki/Mailing-Lists diff --git a/core/pom.xml b/core/pom.xml index 107b47da9..05fbaa878 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -86,7 +86,7 @@ maven-surefire-plugin always - -Xmx512M -Xms512M + -Xmx512M -Xms512M -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 @@ -105,63 +105,66 @@ org.apache.cassandra cassandra-all - 0.8.0-20110216 - org.apache.cassandra.deps + org.apache.cassandra + cassandra-thrift + + + org.apache.thrift libthrift - - org.yaml - snakeyaml - 1.6 + + org.yaml + snakeyaml + 1.6 test - - - com.google.guava - guava - r08 - - - commons-collections - commons-collections - 3.2.1 + + + com.google.guava + guava + r08 + + + commons-collections + commons-collections + 3.2.1 test - - - org.apache.cassandra.deps - avro - 1.4.0-cassandra-1 + + + org.apache.cassandra.deps + avro + 1.4.0-cassandra-1 test - - - netty - org.jboss.netty - - - paranamer - com.thoughtworks.paranamer - - - paranamer-ant - com.thoughtworks.paranamer - - - velocity - org.apache.velocity - - - - - org.antlr - antlr + + + netty + org.jboss.netty + + + paranamer + com.thoughtworks.paranamer + + + paranamer-ant + com.thoughtworks.paranamer + + + velocity + org.apache.velocity + + + + + org.antlr + antlr 3.1.3 - test - - - com.googlecode.concurrentlinkedhashmap - concurrentlinkedhashmap-lru - 1.1 + test + + + com.googlecode.concurrentlinkedhashmap + concurrentlinkedhashmap-lru + 1.1 test @@ -187,9 +190,9 @@ 3.2.0 - org.perf4j - perf4j - 0.9.12 + com.ecyrd.speed4j + speed4j + 0.5 @@ -226,7 +229,7 @@ org.mockito mockito-all test - + org.slf4j slf4j-log4j12 @@ -280,5 +283,10 @@ riptano http://mvn.riptano.com/content/repositories/public/ + + apache-staging + apache-staging + https://repository.apache.org/content/repositories/orgapachecassandra-114 + diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/BackgroundCassandraHostService.java b/core/src/main/java/me/prettyprint/cassandra/connection/BackgroundCassandraHostService.java index 4b15dc75a..817b7e576 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/BackgroundCassandraHostService.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/BackgroundCassandraHostService.java @@ -20,12 +20,7 @@ public BackgroundCassandraHostService(HConnectionManager connectionManager, CassandraHostConfigurator cassandraHostConfigurator) { this.connectionManager = connectionManager; this.cassandraHostConfigurator = cassandraHostConfigurator; - Runtime.getRuntime().addShutdownHook(new Thread() { - @Override - public void run() { - shutdown(); - } - }); + } abstract void shutdown(); diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/CassandraHostRetryService.java b/core/src/main/java/me/prettyprint/cassandra/connection/CassandraHostRetryService.java index 1cdd94b8e..3313d8868 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/CassandraHostRetryService.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/CassandraHostRetryService.java @@ -31,9 +31,9 @@ public CassandraHostRetryService(HConnectionManager connectionManager, super(connectionManager, cassandraHostConfigurator); this.exceptionsTranslator = connectionManager.exceptionsTranslator; this.retryDelayInSeconds = cassandraHostConfigurator.getRetryDownedHostsDelayInSeconds(); - downedHostQueue = new LinkedBlockingQueue(cassandraHostConfigurator.getRetryDownedHostsQueueSize() < 1 + downedHostQueue = new LinkedBlockingQueue(cassandraHostConfigurator.getRetryDownedHostsQueueSize() < 1 ? Integer.MAX_VALUE : cassandraHostConfigurator.getRetryDownedHostsQueueSize()); - + sf = executor.scheduleWithFixedDelay(new RetryRunner(), this.retryDelayInSeconds,this.retryDelayInSeconds, TimeUnit.SECONDS); log.info("Downed Host Retry service started with queue size {} and retry delay {}s", @@ -87,17 +87,16 @@ class RetryRunner implements Runnable { public void run() { CassandraHost cassandraHost = downedHostQueue.poll(); if ( cassandraHost == null ) { - if ( log.isDebugEnabled() ) { + if ( log.isDebugEnabled() ) { log.debug("Retry service fired... nothing to do."); } return; } - + boolean reconnected = verifyConnection(cassandraHost); log.info("Downed Host retry status {} with host: {}", reconnected, cassandraHost.getName()); if ( reconnected ) { - //cassandraClientPool.getCluster().addHost(cassandraHost, true); - connectionManager.addCassandraHost(cassandraHost); + reconnected = connectionManager.addCassandraHost(cassandraHost); } if ( !reconnected && cassandraHost != null ) { downedHostQueue.add(cassandraHost); @@ -112,23 +111,20 @@ private boolean verifyConnection(CassandraHost cassandraHost) { boolean found = false; HThriftClient client = new HThriftClient(cassandraHost); try { - + client.open(); found = client.getCassandra().describe_cluster_name() != null; - client.close(); - } catch (HectorTransportException he) { + client.close(); + } catch (HectorTransportException he) { log.warn("Downed {} host still appears to be down: {}", cassandraHost, he.getMessage()); } catch (Exception ex) { - + log.error("Downed Host retry failed attempt to verify CassandraHost", ex); - - } + + } return found; } } - // TODO create callable to handle checking - - // perhaps wrap CassandraHost and add a lastRetryTime? } diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/ConcurrentHClientPool.java b/core/src/main/java/me/prettyprint/cassandra/connection/ConcurrentHClientPool.java index b17a76f92..49f4ccf8e 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/ConcurrentHClientPool.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/ConcurrentHClientPool.java @@ -66,7 +66,7 @@ public HThriftClient borrowClient() throws HectorException { if ( tillExhausted > 0 ) { // if we start with #of threads == getMaxActive, we could trigger this condition // replace addClientToPoolGently(new HThriftClient(cassandraHost).open()) with immediate acquisition - return greedyCreate(); + return greedyCreate(); } // blocked take on the queue if we are configured to wait forever if ( log.isDebugEnabled() ) { @@ -109,7 +109,7 @@ public HThriftClient borrowClient() throws HectorException { return cassandraClient; } - + /** * Used when we still have room to grow. Return an HThriftClient without * having to wait on polling logic. (But still increment all the counters) diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/DynamicLoadBalancingPolicy.java b/core/src/main/java/me/prettyprint/cassandra/connection/DynamicLoadBalancingPolicy.java index a6e9297be..21f49408d 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/DynamicLoadBalancingPolicy.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/DynamicLoadBalancingPolicy.java @@ -22,7 +22,7 @@ /** * This LB Algorithm has the Phi algo which Dynamic snitch uses, LB is based on the probablity of failure of the node. * TODO: Make cassandra code abstracted enough so we can inherit from the same. - * + * * @author Vijay Parthasarathy */ public class DynamicLoadBalancingPolicy implements LoadBalancingPolicy { @@ -62,7 +62,7 @@ public void run() { } @Override - public HClientPool getPool(List pools, Set excludeHosts) { + public HClientPool getPool(Collection pools, Set excludeHosts) { List poolList = Lists.newArrayList(pools); // remove the hosts from the list. @@ -134,7 +134,7 @@ public int getUpdateInterval() { /** * Set the configured interval for the stats to be recalculated (until this time it is been cached. - * + * * @param updateInterval * In ms. */ @@ -149,7 +149,7 @@ public int getResetInterval() { /** * Set the configured interval for the stats to be reset so that the new stats are allowed and we can get rid of bad * nodes value. This is under the assumption that the bad nodes will eventually get better.... - * + * * @param resetInterval * in ms */ @@ -163,9 +163,9 @@ public double getBadnessThreshold() { /** * This is the percentage of badness which is acceptable... - * + * * Example: A should be 0.20 (20%) bad than B before B is choosen rathar than A. - * + * * @param badness * in % */ diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/HConnectionManager.java b/core/src/main/java/me/prettyprint/cassandra/connection/HConnectionManager.java index d200fabee..28003a6fc 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/HConnectionManager.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/HConnectionManager.java @@ -1,54 +1,36 @@ package me.prettyprint.cassandra.connection; -import java.util.ArrayList; -import java.util.Collection; -import java.util.Collections; -import java.util.HashSet; -import java.util.List; -import java.util.Set; - -import me.prettyprint.cassandra.service.CassandraClientMonitor; +import java.util.*; + +import me.prettyprint.cassandra.service.*; import me.prettyprint.cassandra.service.CassandraClientMonitor.Counter; -import me.prettyprint.cassandra.service.CassandraHost; -import me.prettyprint.cassandra.service.CassandraHostConfigurator; -import me.prettyprint.cassandra.service.ExceptionsTranslator; -import me.prettyprint.cassandra.service.ExceptionsTranslatorImpl; -import me.prettyprint.cassandra.service.FailoverPolicy; -import me.prettyprint.cassandra.service.JmxMonitor; -import me.prettyprint.cassandra.service.Operation; import me.prettyprint.hector.api.ClockResolution; -import me.prettyprint.hector.api.exceptions.HCassandraInternalException; -import me.prettyprint.hector.api.exceptions.HInvalidRequestException; -import me.prettyprint.hector.api.exceptions.HTimedOutException; -import me.prettyprint.hector.api.exceptions.HUnavailableException; -import me.prettyprint.hector.api.exceptions.HectorException; -import me.prettyprint.hector.api.exceptions.HectorTransportException; -import me.prettyprint.hector.api.exceptions.PoolExhaustedException; +import me.prettyprint.hector.api.exceptions.*; import org.apache.cassandra.thrift.AuthenticationRequest; import org.apache.cassandra.thrift.Cassandra; import org.cliffc.high_scale_lib.NonBlockingHashMap; -import org.perf4j.StopWatch; -import org.perf4j.slf4j.Slf4JStopWatch; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import com.ecyrd.speed4j.StopWatch; +import com.ecyrd.speed4j.StopWatchFactory; +import com.ecyrd.speed4j.log.PeriodicalLog; + public class HConnectionManager { private static final Logger log = LoggerFactory.getLogger(HConnectionManager.class); - private static final Logger perf4jLogger = - LoggerFactory.getLogger("me.prettyprint.cassandra.hector.TimingLogger"); + private StopWatchFactory stopWatchFactory; private final NonBlockingHashMap hostPools; - private final NonBlockingHashMap suspendedHostPools; - private final List hostPoolValues; + private final NonBlockingHashMap suspendedHostPools; + private final Collection hostPoolValues; private final String clusterName; private CassandraHostRetryService cassandraHostRetryService; private NodeAutoDiscoverService nodeAutoDiscoverService; private LoadBalancingPolicy loadBalancingPolicy; private CassandraHostConfigurator cassandraHostConfigurator; private HostTimeoutTracker hostTimeoutTracker; - private final ClockResolution clock; final ExceptionsTranslator exceptionsTranslator; @@ -63,7 +45,7 @@ public HConnectionManager(String clusterName, CassandraHostConfigurator cassandr this.clusterName = clusterName; if ( cassandraHostConfigurator.getRetryDownedHosts() ) { cassandraHostRetryService = new CassandraHostRetryService(this, cassandraHostConfigurator); - } + } for ( CassandraHost host : cassandraHostConfigurator.buildCassandraHosts()) { try { HClientPool hcp = loadBalancingPolicy.createConnection(host); @@ -75,24 +57,37 @@ public HConnectionManager(String clusterName, CassandraHostConfigurator cassandr } } } - + if ( cassandraHostConfigurator.getUseHostTimeoutTracker() ) { hostTimeoutTracker = new HostTimeoutTracker(this, cassandraHostConfigurator); } monitor = JmxMonitor.getInstance().getCassandraMonitor(this); exceptionsTranslator = new ExceptionsTranslatorImpl(); this.cassandraHostConfigurator = cassandraHostConfigurator; - hostPoolValues = new ArrayList(hostPools.values()); + hostPoolValues = hostPools.values(); if ( cassandraHostConfigurator.getAutoDiscoverHosts() ) { nodeAutoDiscoverService = new NodeAutoDiscoverService(this, cassandraHostConfigurator); if ( cassandraHostConfigurator.getRunAutoDiscoveryAtStartup() ) { nodeAutoDiscoverService.doAddNodes(); } } + + // + // This sets up the Speed4J logging system. Alternatively, we could + // use the speed4j.properties -file. This was chosen just so that + // it wouldn't confuse anyone and would work pretty much the same + // way as what the old hector config does. + // + PeriodicalLog slog = new PeriodicalLog(); + slog.setName("hector-"+clusterName); + slog.setPeriod(60); // 60 seconds + slog.setSlf4jLogname( "me.prettyprint.cassandra.hector.TimingLogger" ); + + stopWatchFactory = StopWatchFactory.getInstance( slog ); } /** - * Returns true if the host was successfully added. In any sort of failure exceptions are + * Returns true if the host was successfully added. In any sort of failure exceptions are * caught and logged, returning false. * @param cassandraHost * @return @@ -143,9 +138,9 @@ public boolean removeCassandraHost(CassandraHost cassandraHost) { log.info("Remove status for CassandraHost pool {} was {}", cassandraHost, removed); return removed; } - + /** - * Remove the {@link HClientPool} referenced by the {@link CassandraHost} from + * Remove the {@link HClientPool} referenced by the {@link CassandraHost} from * the active host pools. This does not shut down the pool, only removes it as a candidate from * future operations. * @param cassandraHost @@ -154,23 +149,23 @@ public boolean removeCassandraHost(CassandraHost cassandraHost) { public boolean suspendCassandraHost(CassandraHost cassandraHost) { HClientPool pool = hostPools.remove(cassandraHost); boolean removed = pool != null; - if ( removed ) { + if ( removed ) { suspendedHostPools.put(cassandraHost, pool); } log.info("Suspend operation status was {} for CassandraHost {}", removed, cassandraHost); return removed; } - /** + /** * The opposite of suspendCassandraHost, places the pool back into selection * @param cassandraHost - * @return true if this operation was successful. A no-op returning false + * @return true if this operation was successful. A no-op returning false * if there was no such host in the underlying suspendedHostPool map. */ public boolean unsuspendCassandraHost(CassandraHost cassandraHost) { HClientPool pool = suspendedHostPools.remove(cassandraHost); boolean readded = pool != null; - if ( readded ) { + if ( readded ) { boolean alreadyThere = hostPools.putIfAbsent(cassandraHost, pool) != null; if ( alreadyThere ) { log.error("Unsuspend called on a pool that was already active for CassandraHost {}", cassandraHost); @@ -179,7 +174,7 @@ public boolean unsuspendCassandraHost(CassandraHost cassandraHost) { log.info("UN-Suspend operation status was {} for CassandraHost {}", readded, cassandraHost); return readded; } - + /** * Returns a Set of {@link CassandraHost} which are in the suspended status * @return @@ -187,7 +182,7 @@ public boolean unsuspendCassandraHost(CassandraHost cassandraHost) { public Set getSuspendedCassandraHosts() { return suspendedHostPools.keySet(); } - + public Set getHosts() { return Collections.unmodifiableSet(hostPools.keySet()); } @@ -202,7 +197,7 @@ public List getStatusPerPool() { public void operateWithFailover(Operation op) throws HectorException { - final StopWatch stopWatch = new Slf4JStopWatch(perf4jLogger); + final StopWatch stopWatch = stopWatchFactory.getStopWatch(); int retries = Math.min(op.failoverPolicy.numRetries, hostPools.size()); HThriftClient client = null; HClientPool pool = null; @@ -232,31 +227,29 @@ public void operateWithFailover(Operation op) throws HectorException { // break out on HUnavailableException as well since we can no longer satisfy the CL throw he; } else if ( he instanceof HectorTransportException) { - --retries; // client can be null in this situation - if ( client != null ) { + if ( client != null ) { client.close(); } markHostAsDown(pool.getCassandraHost()); excludeHosts.add(pool.getCassandraHost()); retryable = true; - if ( retries > 0 ) { - monitor.incCounter(Counter.RECOVERABLE_TRANSPORT_EXCEPTIONS); - } + + monitor.incCounter(Counter.RECOVERABLE_TRANSPORT_EXCEPTIONS); + } else if (he instanceof HTimedOutException ) { // DO NOT drecrement retries, we will be keep retrying on timeouts until it comes back - // if HLT.checkTimeout(cassandraHost): suspendHost(cassandraHost); + // if HLT.checkTimeout(cassandraHost): suspendHost(cassandraHost); doTimeoutCheck(pool.getCassandraHost()); - if ( hostPools.size() > 1) { - retryable = true; - } + + retryable = true; + monitor.incCounter(Counter.RECOVERABLE_TIMED_OUT_EXCEPTIONS); client.close(); // TODO timecheck on how long we've been waiting on timeouts here // suggestion per user moores on hector-users } else if ( he instanceof PoolExhaustedException ) { retryable = true; - --retries; if ( hostPools.size() == 1 ) { throw he; } @@ -265,15 +258,17 @@ public void operateWithFailover(Operation op) throws HectorException { } else { // something strange happened. Added here as suggested by sbridges. // I think this gives a sane way to future-proof against any API additions - // that we don't add in time. + // that we don't add in time. retryable = false; } if ( retries <= 0 || retryable == false) throw he; - log.error("Could not fullfill request on this host {}", client); - log.error("Exception: ", he); + + log.warn("Could not fullfill request on this host {}", client); + log.warn("Exception: ", he); monitor.incCounter(Counter.SKIP_HOST_SUCCESS); sleepBetweenHostSkips(op.failoverPolicy); } finally { + --retries; if ( !success ) { monitor.incCounter(op.failCounter); stopWatch.stop(op.stopWatchTagName + ".fail_"); @@ -282,7 +277,7 @@ public void operateWithFailover(Operation op) throws HectorException { } } } - + /** * Use the HostTimeoutCheck and initiate a suspend if and only if * we are configured for such AND there is more than one operating host pool @@ -317,8 +312,8 @@ private void sleepBetweenHostSkips(FailoverPolicy failoverPolicy) { private HClientPool getClientFromLBPolicy(Set excludeHosts) { if ( hostPools.isEmpty() ) { throw new HectorException("All host pools marked down. Retry burden pushed out to client."); - } - return loadBalancingPolicy.getPool(hostPoolValues, excludeHosts); + } + return loadBalancingPolicy.getPool(hostPoolValues, excludeHosts); } void releaseClient(HThriftClient client) { @@ -346,7 +341,7 @@ void markHostAsDown(CassandraHost cassandraHost) { if ( pool != null ) { log.error("Pool state on shutdown: {}", pool.getStatusAsString()); pool.shutdown(); - if ( cassandraHostRetryService != null ) + if ( cassandraHostRetryService != null ) cassandraHostRetryService.add(cassandraHost); } } @@ -362,7 +357,7 @@ public Collection getActivePools() { public long createClock() { return this.clock.createClock(); } - + public String getClusterName() { return clusterName; } @@ -373,7 +368,7 @@ public void shutdown() { cassandraHostRetryService.shutdown(); if ( nodeAutoDiscoverService != null ) nodeAutoDiscoverService.shutdown(); - if ( hostTimeoutTracker != null ) + if ( hostTimeoutTracker != null ) hostTimeoutTracker.shutdown(); for (HClientPool pool : hostPools.values()) { diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/HThriftClient.java b/core/src/main/java/me/prettyprint/cassandra/connection/HThriftClient.java index 492519089..f7c4f4104 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/HThriftClient.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/HThriftClient.java @@ -1,5 +1,7 @@ package me.prettyprint.cassandra.connection; +import java.net.Socket; +import java.net.SocketException; import java.util.concurrent.atomic.AtomicLong; import me.prettyprint.cassandra.service.CassandraHost; @@ -57,18 +59,18 @@ public Cassandra.Client getCassandra() { } public Cassandra.Client getCassandra(String keyspaceNameArg) { - getCassandra(); + getCassandra(); if ( keyspaceNameArg != null && !StringUtils.equals(keyspaceName, keyspaceNameArg)) { if ( log.isDebugEnabled() ) log.debug("keyspace reseting from {} to {}", keyspaceName, keyspaceNameArg); keyspaceName = keyspaceNameArg; try { - cassandraClient.set_keyspace(keyspaceName); + cassandraClient.set_keyspace(keyspaceName); } catch (InvalidRequestException ire) { throw new HInvalidRequestException(ire); } catch (TException e) { throw new HectorTransportException(e); - } + } } return cassandraClient; @@ -103,11 +105,20 @@ HThriftClient open() { log.debug("Creating a new thrift connection to {}", cassandraHost); } + TSocket socket = new TSocket(cassandraHost.getHost(), cassandraHost.getPort(), timeout); + if ( cassandraHost.getUseSocketKeepalive() ) { + try { + socket.getSocket().setKeepAlive(true); + } catch (SocketException se) { + throw new HectorTransportException("Could not set SO_KEEPALIVE on socket: ", se); + } + } if (cassandraHost.getUseThriftFramedTransport()) { - transport = new TFramedTransport(new TSocket(cassandraHost.getHost(), cassandraHost.getPort(), timeout)); + transport = new TFramedTransport(socket); } else { - transport = new TSocket(cassandraHost.getHost(), cassandraHost.getPort(), timeout); + transport = socket; } + try { transport.open(); } catch (TTransportException e) { @@ -161,7 +172,7 @@ private int getTimeout(CassandraHost cassandraHost) { public void startToUse() { useageStartTime = System.currentTimeMillis(); } - + /** * @return Time in MS since it was used. */ diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/HostTimeoutTracker.java b/core/src/main/java/me/prettyprint/cassandra/connection/HostTimeoutTracker.java index 7bdf0fa0e..7ec78b380 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/HostTimeoutTracker.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/HostTimeoutTracker.java @@ -13,14 +13,14 @@ import me.prettyprint.cassandra.service.CassandraHostConfigurator; /** - * Keep track of how often a node replies with a HTimeoutException. If we go + * Keep track of how often a node replies with a HTimeoutException. If we go * past the threshold of [timeoutCounter] timeouts within [timeWindow] milliseconds, * then we mark the node as suspended. (10 timeouts within 500ms by default) - * - * Periodically check the suspended nodes list every retryDelayInSeconds. If + * + * Periodically check the suspended nodes list every retryDelayInSeconds. If * the node has been suspended longer than nodeSuspensionDurationInSeconds, - * then we unsuspend, placing it back in the available pool. (10 second - * suspension retried every 10 seconds by default). + * then we unsuspend, placing it back in the available pool. (10 second + * suspension retried every 10 seconds by default). * * @author zznate */ @@ -32,13 +32,13 @@ public class HostTimeoutTracker extends BackgroundCassandraHostService { private int timeoutCounter; private int timeoutWindow; private int nodeSuspensionDurationInSeconds; - + public static final int DEF_TIMEOUT_COUNTER = 10; public static final int DEF_TIMEOUT_WINDOW = 500; public static final int DEF_NODE_SUSPENSION_DURATION_IN_SECONDS = 10; public static final int DEF_NODE_UNSUSPEND_CHECK_DELAY_IN_SECONDS = 10; - - + + public HostTimeoutTracker(HConnectionManager connectionManager, CassandraHostConfigurator cassandraHostConfigurator) { super(connectionManager, cassandraHostConfigurator); @@ -58,16 +58,16 @@ public boolean checkTimeout(CassandraHost cassandraHost) { boolean timeout = false; // if there are 3 timeouts within 500ms, return false if ( timeouts.get(cassandraHost).size() > timeoutCounter) { - Long last = timeouts.get(cassandraHost).remove(); - if (last.longValue() < (currentTimeMillis - timeoutWindow)) { + Long last = timeouts.get(cassandraHost).remove(); + if (last.longValue() < (currentTimeMillis - timeoutWindow)) { timeout = true; connectionManager.suspendCassandraHost(cassandraHost); - suspended.putIfAbsent(cassandraHost, currentTimeMillis); - } + suspended.putIfAbsent(cassandraHost, currentTimeMillis); + } } return timeout; } - + class Unsuspender implements Runnable { @Override @@ -76,13 +76,13 @@ public void run() { Entry vals = iterator.next(); if ( vals.getValue() < (System.currentTimeMillis() - (nodeSuspensionDurationInSeconds * 1000)) ) { connectionManager.unsuspendCassandraHost(vals.getKey()); - iterator.remove(); + iterator.remove(); } - } + } } - + } - + @Override void applyRetryDelay() { @@ -94,7 +94,7 @@ void shutdown() { log.info("Shutting down HostTimeoutTracker"); if ( sf != null ) sf.cancel(true); - if ( executor != null ) + if ( executor != null ) executor.shutdownNow(); log.info("HostTimeTracker shutdown complete."); } diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/LatencyAwareHClientPool.java b/core/src/main/java/me/prettyprint/cassandra/connection/LatencyAwareHClientPool.java index 2561e8c4d..5912c54cc 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/LatencyAwareHClientPool.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/LatencyAwareHClientPool.java @@ -9,7 +9,7 @@ /** * This class provides a queue function of latencies over CHCP, collecting all the latency information and calculates * the score (expensive operation). - * + * * @author Vijay Parthasarathy */ public class LatencyAwareHClientPool extends ConcurrentHClientPool { diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicy.java b/core/src/main/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicy.java index e7e0a94fd..ffc8e7d44 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicy.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicy.java @@ -15,17 +15,17 @@ * The list of hosts is shuffled on each pass to account for the case * where a number of hosts are at the minimum number of connections * (ie. they are not busy). - * - * + * + * * @author zznate */ public class LeastActiveBalancingPolicy implements LoadBalancingPolicy { - + private static final long serialVersionUID = 329849818218657061L; private static final Logger log = LoggerFactory.getLogger(LeastActiveBalancingPolicy.class); - + @Override - public HClientPool getPool(List pools, Set excludeHosts) { + public HClientPool getPool(Collection pools, Set excludeHosts) { List vals = Lists.newArrayList(pools); // shuffle pools to avoid always returning the same one when we are not terribly busy Collections.shuffle(vals); @@ -33,7 +33,7 @@ public HClientPool getPool(List pools, Set excludeHo Iterator iterator = vals.iterator(); HClientPool concurrentHClientPool = iterator.next(); if ( excludeHosts != null && excludeHosts.size() > 0 ) { - while (iterator.hasNext()) { + while (iterator.hasNext()) { if ( !excludeHosts.contains(concurrentHClientPool.getCassandraHost()) ) { break; } @@ -44,16 +44,16 @@ public HClientPool getPool(List pools, Set excludeHo } private final class ShufflingCompare implements Comparator { - + public int compare(HClientPool o1, HClientPool o2) { if ( log.isDebugEnabled() ) { log.debug("comparing 1: {} and count {} with 2: {} and count {}", new Object[]{o1.getCassandraHost(), o1.getNumActive(), o2.getCassandraHost(), o2.getNumActive()}); } - return o1.getNumActive() - o2.getNumActive(); + return o1.getNumActive() - o2.getNumActive(); } } - + @Override public HClientPool createConnection(CassandraHost host) { return new ConcurrentHClientPool(host); diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/LoadBalancingPolicy.java b/core/src/main/java/me/prettyprint/cassandra/connection/LoadBalancingPolicy.java index 49d628f8e..a18181b94 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/LoadBalancingPolicy.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/LoadBalancingPolicy.java @@ -1,12 +1,13 @@ package me.prettyprint.cassandra.connection; import java.io.Serializable; +import java.util.Collection; import java.util.List; import java.util.Set; import me.prettyprint.cassandra.service.CassandraHost; public interface LoadBalancingPolicy extends Serializable { - HClientPool getPool(List pools, Set excludeHosts); + HClientPool getPool(Collection pools, Set excludeHosts); HClientPool createConnection(CassandraHost host); } diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/NodeAutoDiscoverService.java b/core/src/main/java/me/prettyprint/cassandra/connection/NodeAutoDiscoverService.java index bc2e3a3a4..399348e57 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/NodeAutoDiscoverService.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/NodeAutoDiscoverService.java @@ -54,7 +54,7 @@ public void run() { } } - + public void doAddNodes() { if ( log.isDebugEnabled() ) { log.debug("Auto discovery service running..."); diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/PoolMetric.java b/core/src/main/java/me/prettyprint/cassandra/connection/PoolMetric.java index 043772ec4..8b1ff3c2e 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/PoolMetric.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/PoolMetric.java @@ -7,5 +7,5 @@ public interface PoolMetric { int getNumBlockedThreads(); String getName(); boolean getIsActive(); - + } diff --git a/core/src/main/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicy.java b/core/src/main/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicy.java index 450f3f750..9a9c945b9 100644 --- a/core/src/main/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicy.java +++ b/core/src/main/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicy.java @@ -1,5 +1,6 @@ package me.prettyprint.cassandra.connection; +import java.util.Collection; import java.util.List; import java.util.Set; import java.util.concurrent.atomic.AtomicInteger; @@ -10,7 +11,7 @@ /** * Implements a RoundRobin balancing policy based off the contents - * of the active {@link HClientPool}. If a pool is shutdown by another + * of the active {@link HClientPool}. If a pool is shutdown by another * thread in the midst of the selection process, we return the pool * at position 0 * @@ -20,30 +21,36 @@ public class RoundRobinBalancingPolicy implements LoadBalancingPolicy { private static final long serialVersionUID = 1107204068032227079L; private AtomicInteger counter; - + public RoundRobinBalancingPolicy() { counter = new AtomicInteger(); } - + @Override - public HClientPool getPool(List pools, + public HClientPool getPool(Collection pools, Set excludeHosts) { - HClientPool pool = getPoolSafely(getAndIncrement(pools.size()), pools); + HClientPool pool = getPoolSafely(pools); if ( excludeHosts != null && excludeHosts.size() > 0 ) { while ( excludeHosts.contains(pool.getCassandraHost()) ) { - pool = getPoolSafely(getAndIncrement(pools.size()), pools); + pool = getPoolSafely(pools); + if ( excludeHosts.size() >= pools.size() ) + break; } - } + } return pool; } - - private HClientPool getPoolSafely(int location, List pools) { - return Iterables.get(pools, location, pools.get(0)); + + private HClientPool getPoolSafely(Collection pools) { + try { + return Iterables.get(pools, getAndIncrement(pools.size())); + } catch (IndexOutOfBoundsException e) { + return pools.iterator().next(); + } } - + private int getAndIncrement(int size) { counter.compareAndSet(16384, 0); - return counter.getAndIncrement() % size; + return counter.getAndIncrement() % size; } @Override diff --git a/core/src/main/java/me/prettyprint/cassandra/io/ChunkInputStream.java b/core/src/main/java/me/prettyprint/cassandra/io/ChunkInputStream.java index c97204358..f265d1dea 100644 --- a/core/src/main/java/me/prettyprint/cassandra/io/ChunkInputStream.java +++ b/core/src/main/java/me/prettyprint/cassandra/io/ChunkInputStream.java @@ -16,9 +16,9 @@ /** * Return an InputStream which retrieve columns from a row which stores chunk of * data. See also {@link ChunkOutputStream} - * + * * This implementation is not thread-safe! - * + * * @param */ public class ChunkInputStream extends InputStream { @@ -40,7 +40,7 @@ public ChunkInputStream(Keyspace keyspace, String cf, T key, Serializer rowKe /* * (non-Javadoc) - * + * * @see java.io.InputStream#read() */ public int read() throws IOException { @@ -54,7 +54,7 @@ public int read() throws IOException { /** * Fetch the next chunk. - * + * * @return exists if there was a chunk to fetch. * @throws IOException */ diff --git a/core/src/main/java/me/prettyprint/cassandra/io/ChunkOutputStream.java b/core/src/main/java/me/prettyprint/cassandra/io/ChunkOutputStream.java index 3d4323c5b..bf0496b1d 100644 --- a/core/src/main/java/me/prettyprint/cassandra/io/ChunkOutputStream.java +++ b/core/src/main/java/me/prettyprint/cassandra/io/ChunkOutputStream.java @@ -16,9 +16,9 @@ * will be split up by chunks of the given chunkSize. Each chunk we get written * to own column which will have the chunk number (starting at 0) as column key * (Long). - * + * * This implementation is not thread-safe! - * + * */ public class ChunkOutputStream extends OutputStream { private byte[] chunk; @@ -37,7 +37,7 @@ public ChunkOutputStream(Keyspace keyspace, String cf, T key, Serializer keyS /* * (non-Javadoc) - * + * * @see java.io.OutputStream#write(int) */ public void write(int b) throws IOException { @@ -64,7 +64,7 @@ public void flush() throws IOException { /** * Write the data to column if the configured chunk size is reached or if the * stream should be closed - * + * * @param close * @throws IOException */ diff --git a/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactory.java b/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactory.java index 3f838abb1..2f70cbbc1 100644 --- a/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactory.java +++ b/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactory.java @@ -8,29 +8,50 @@ import javax.naming.Reference; import javax.naming.spi.ObjectFactory; +import org.apache.commons.lang.BooleanUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import me.prettyprint.cassandra.service.CassandraHostConfigurator; +import me.prettyprint.hector.api.Cluster; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.factory.HFactory; + /** * A factory for JNDI Resource managed objects. Responsible for creating a - * {@link CassandraClientJndiResourcePool}. Relies on a supplied URL and - * port which Cassandra is listening on. These parameters are defined in - * a web application's context.xml file. For example: + * {@link Keyspace} references for passing to {@link HFactory}. + * A limited set of configuration parameters are supported. + * These parameters are defined in a web application's context.xml file. + * Parameter descriptions can be found in {@link CassandraHostConfigurator} * *

*

  *           
+ *               hosts="cass1:9160,cass2:9160,cass3:9160"
+ *               keyspace="Keyspace1"
+ *               clusterName="Test Cluster" 
+ *               maxActive="20"
+ *               maxWaitTimeWhenExhausted="10"
+ *               autoDiscoverHosts="true"
+ *               runAutoDiscoveryAtStartup="true"/>      
  * 
* * @author Perry Hoekstra (dutchman_mn@charter.net) + * @author zznate * * @since 0.5.1-8 */ -public class CassandraClientJndiResourceFactory implements ObjectFactory -{ +public class CassandraClientJndiResourceFactory implements ObjectFactory { + private Logger log = LoggerFactory.getLogger(CassandraClientJndiResourceFactory.class); + + private CassandraHostConfigurator cassandraHostConfigurator; + private Cluster cluster; + private Keyspace keyspace; + /** * Creates an object using the location or reference information specified. * @@ -57,16 +78,53 @@ public Object getObjectInstance(Object object, Name jndiName, Context context, throw new Exception("Object provided is not a javax.naming.Reference type"); } - RefAddr urlRefAddr = resourceRef.get("url"); - - RefAddr portRefAddr = resourceRef.get("port"); + // config CassandraHostConfigurator + if ( cluster == null ) { + configure(resourceRef); + } + + return keyspace; + } - if ((urlRefAddr != null) && (portRefAddr != null)) { - return new CassandraClientJndiResourcePool((String)urlRefAddr.getContent(), - Integer.parseInt((String)portRefAddr.getContent())); - } else { + private void configure(Reference resourceRef) throws Exception { + // required + RefAddr hostsRefAddr = resourceRef.get("hosts"); + RefAddr clusterNameRef = resourceRef.get("clusterName"); + RefAddr keyspaceNameRef = resourceRef.get("keyspace"); + // optional + RefAddr maxActiveRefAddr = resourceRef.get("maxActive"); + RefAddr maxWaitTimeWhenExhausted = resourceRef.get("maxWaitTimeWhenExhausted"); + RefAddr autoDiscoverHosts = resourceRef.get("autoDiscoverHosts"); + RefAddr runAutoDiscoverAtStartup = resourceRef.get("runAutoDiscoveryAtStartup"); + RefAddr retryDownedHostDelayInSeconds = resourceRef.get("retryDownedHostDelayInSeconds"); + + if ( hostsRefAddr == null || hostsRefAddr.getContent() == null) { throw new Exception("A url and port on which Cassandra is installed and listening " + - "on must be provided as a ResourceParams in the context.xml"); + "on must be provided as a ResourceParams in the context.xml"); + } + + cassandraHostConfigurator = new CassandraHostConfigurator((String)hostsRefAddr.getContent()); + if ( autoDiscoverHosts != null ) { + cassandraHostConfigurator.setAutoDiscoverHosts(Boolean.parseBoolean((String)autoDiscoverHosts.getContent())); + if ( runAutoDiscoverAtStartup != null ) + cassandraHostConfigurator.setRunAutoDiscoveryAtStartup(Boolean.parseBoolean((String)autoDiscoverHosts.getContent())); + } + if ( retryDownedHostDelayInSeconds != null ) { + int retryDelay = Integer.parseInt((String)retryDownedHostDelayInSeconds.getContent()); + // disable retry if less than 1 + if ( retryDelay < 1 ) + cassandraHostConfigurator.setRetryDownedHosts(false); + cassandraHostConfigurator.setRetryDownedHostsDelayInSeconds(retryDelay); } + if ( maxActiveRefAddr != null ) + cassandraHostConfigurator.setMaxActive(Integer.parseInt((String)maxActiveRefAddr.getContent())); + if ( maxWaitTimeWhenExhausted != null ) + cassandraHostConfigurator.setMaxWaitTimeWhenExhausted(Integer.parseInt((String)maxWaitTimeWhenExhausted.getContent())); + + if ( log.isDebugEnabled() ) + log.debug("JNDI resource created with CassandraHostConfiguration: {}", cassandraHostConfigurator.getAutoDiscoverHosts()); + + cluster = HFactory.createCluster((String)clusterNameRef.getContent(), cassandraHostConfigurator); + keyspace = HFactory.createKeyspace((String)keyspaceNameRef.getContent(), cluster); } } diff --git a/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourcePool.java b/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourcePool.java index 36c6f2f11..e044c437d 100644 --- a/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourcePool.java +++ b/core/src/main/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourcePool.java @@ -9,20 +9,20 @@ * as Apache Tomcat. * * @see GenericObjectPool - * + * * @author Perry Hoekstra (dutchman_mn@charter.net) * */ public class CassandraClientJndiResourcePool extends GenericObjectPool { /** * CassandraClientJndiResourcePool constructor. - * + * * @param url url of the host that contains Cassandra. * @param port port number that Cassandra is listening on. */ - + public CassandraClientJndiResourcePool(String url, int port) { // TODO fix this - //super(new JndiCassandraClientFactory(url, port)); + //super(new JndiCassandraClientFactory(url, port)); } } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/AbstractBasicQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/AbstractBasicQuery.java new file mode 100644 index 000000000..eff40fd53 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/AbstractBasicQuery.java @@ -0,0 +1,55 @@ +package me.prettyprint.cassandra.model; + +import me.prettyprint.cassandra.utils.Assert; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.query.Query; + +/** + * + * @author patricioe (Patricio Echague - patricio@datastax.com) + * + * @param Key type + * @param column name type + * @param return type + */ +public abstract class AbstractBasicQuery implements Query { + + protected final ExecutingKeyspace keyspace; + protected String columnFamilyName; + protected Serializer keySerializer; + protected Serializer columnNameSerializer; + // add: FailoverPolicy, ConsistencyLevelPolicy, Credentials? + + protected AbstractBasicQuery(Keyspace k, Serializer keySerializer, + Serializer nameSerializer) { + Assert.noneNull(k, keySerializer, nameSerializer); + keyspace = (ExecutingKeyspace) k; + this.keySerializer = keySerializer; + this.columnNameSerializer = nameSerializer; + } + + public Query setColumnFamily(String cf) { + this.columnFamilyName = cf; + return this; + } + + public Serializer getKeySerializer() { + return keySerializer; + } + + public AbstractBasicQuery setKeySerializer(Serializer keySerializer) { + this.keySerializer = keySerializer; + return this; + } + + public Serializer getColumnNameSerializer() { + return columnNameSerializer; + } + + public AbstractBasicQuery setColumnNameSerializer(Serializer columnNameSerializer) { + this.columnNameSerializer = columnNameSerializer; + return this; + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/AbstractQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/AbstractQuery.java index b57fe0e7f..1566cbaf0 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/AbstractQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/AbstractQuery.java @@ -5,47 +5,18 @@ import me.prettyprint.hector.api.Serializer; import me.prettyprint.hector.api.query.Query; -public abstract class AbstractQuery implements Query { +public abstract class AbstractQuery extends AbstractBasicQuery implements Query { - protected final ExecutingKeyspace keyspace; - protected String columnFamilyName; - protected Serializer keySerializer; - protected Serializer columnNameSerializer; protected Serializer valueSerializer; // add: FailoverPolicy, ConsistencyLevelPolicy, Credentials? /*package*/ AbstractQuery(Keyspace k, Serializer keySerializer, Serializer nameSerializer, Serializer valueSerializer) { - Assert.noneNull(k, keySerializer, nameSerializer, valueSerializer); - keyspace = (ExecutingKeyspace) k; - this.keySerializer = keySerializer; - this.columnNameSerializer = nameSerializer; + super(k, keySerializer, nameSerializer); + Assert.noneNull(valueSerializer); this.valueSerializer = valueSerializer; } - public Query setColumnFamily(String cf) { - this.columnFamilyName = cf; - return this; - } - - public Serializer getKeySerializer() { - return keySerializer; - } - - public AbstractQuery setKeySerializer(Serializer keySerializer) { - this.keySerializer = keySerializer; - return this; - } - - public Serializer getColumnNameSerializer() { - return columnNameSerializer; - } - - public AbstractQuery setColumnNameSerializer(Serializer columnNameSerializer) { - this.columnNameSerializer = columnNameSerializer; - return this; - } - public Serializer getValueSerializer() { return valueSerializer; } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/AbstractSliceQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/AbstractSliceQuery.java index 9370982f1..c97f70d51 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/AbstractSliceQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/AbstractSliceQuery.java @@ -50,7 +50,7 @@ public Query setColumnNames(Collection columnNames) { } return this; } - + /** * Wraps the underlying call to {@link HSlicePredicate#setKeysOnlyPredicate()} * Use this for a substantial performance increase when you only need the keys returned diff --git a/core/src/main/java/me/prettyprint/cassandra/model/AbstractSubColumnQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/AbstractSubColumnQuery.java index 8c8bcb04a..84cabeaca 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/AbstractSubColumnQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/AbstractSubColumnQuery.java @@ -64,6 +64,6 @@ public QueryResult> execute() { List> columns = slice.getColumns(); HColumn column = columns.size() == 0 ? null : columns.get(0); return new QueryResultImpl>( - new ExecutionResult>(column, r.getExecutionTimeMicro(), r.getHostUsed()), this); + new ExecutionResult>(column, r.getExecutionTimeNano(), r.getHostUsed()), this); } } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/BasicColumnFamilyDefinition.java b/core/src/main/java/me/prettyprint/cassandra/model/BasicColumnFamilyDefinition.java index 96527b38e..f4bbc6f44 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/BasicColumnFamilyDefinition.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/BasicColumnFamilyDefinition.java @@ -25,10 +25,12 @@ public class BasicColumnFamilyDefinition implements ColumnFamilyDefinition { private double readRepairChance; private int gcGraceSeconds; private String defaultValidationClass; + private String keyValidationClass; private int id; private int maxCompactionThreshold; private int minCompactionThreshold; private int rowCacheSavePeriodInSeconds; + private int keyCacheSavePeriodInSeconds; private double memtableOperationsInMillions; private int memtableThroughputInMb; private int memtableFlushAfterMins; @@ -39,7 +41,7 @@ public class BasicColumnFamilyDefinition implements ColumnFamilyDefinition { public BasicColumnFamilyDefinition() { this.columnDefinitions = new ArrayList(); } - + /** * Builds a {@link BasicColumnFamilyDefinition} based off the interface */ @@ -53,9 +55,10 @@ public BasicColumnFamilyDefinition(ColumnFamilyDefinition columnFamilyDefinition rowCacheSize = columnFamilyDefinition.getRowCacheSize(); rowCacheSavePeriodInSeconds = columnFamilyDefinition.getRowCacheSavePeriodInSeconds(); keyCacheSize = columnFamilyDefinition.getKeyCacheSize(); + keyCacheSavePeriodInSeconds = columnFamilyDefinition.getKeyCacheSavePeriodInSeconds(); readRepairChance = columnFamilyDefinition.getReadRepairChance(); - columnDefinitions = columnFamilyDefinition.getColumnMetadata() != null - ? new ArrayList(columnFamilyDefinition.getColumnMetadata()) + columnDefinitions = columnFamilyDefinition.getColumnMetadata() != null + ? new ArrayList(columnFamilyDefinition.getColumnMetadata()) : new ArrayList(); gcGraceSeconds = columnFamilyDefinition.getGcGraceSeconds(); defaultValidationClass = columnFamilyDefinition.getDefaultValidationClass(); @@ -121,7 +124,7 @@ public void setMaxCompactionThreshold(int maxCompactionThreshold) { public void setMinCompactionThreshold(int minCompactionThreshold) { this.minCompactionThreshold = minCompactionThreshold; - } + } public void setRowCacheSavePeriodInSeconds(int rowCacheSavePeriodInSeconds) { this.rowCacheSavePeriodInSeconds = rowCacheSavePeriodInSeconds; @@ -143,6 +146,14 @@ public void addColumnDefinition( ColumnDefinition columnDefinition){ this.columnDefinitions.add( columnDefinition ); } + public void setKeyCacheSavePeriodInSeconds(int keyCacheSavePeriodInSeconds) { + this.keyCacheSavePeriodInSeconds = keyCacheSavePeriodInSeconds; + } + + public void setKeyValidationClass(String keyValidationClass){ + this.keyValidationClass = keyValidationClass; + } + /** * SHOULD THIS BE HERE? A COLUMN DEFINITION IS PART OF A KEYSPACE BY VIRTUE * OF BEING IN A KEYSPACE LIST @@ -242,5 +253,13 @@ public int getMemtableThroughputInMb() { return this.memtableThroughputInMb; } + public int getKeyCacheSavePeriodInSeconds() { + return keyCacheSavePeriodInSeconds; + } + + @Override + public String getKeyValidationClass() { + return keyValidationClass; + } } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/CounterSliceImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/CounterSliceImpl.java new file mode 100644 index 000000000..6f648699c --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/CounterSliceImpl.java @@ -0,0 +1,51 @@ +package me.prettyprint.cassandra.model; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.utils.Assert; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.CounterSlice; +import me.prettyprint.hector.api.beans.HCounterColumn; + +import org.apache.cassandra.thrift.CounterColumn; + +public final class CounterSliceImpl implements CounterSlice { + + private final Map> columnsMap; + private final List> columnsList; + + public CounterSliceImpl(List tColumns, Serializer nameSerializer) { + + Assert.noneNull(tColumns, nameSerializer); + columnsMap = new HashMap>(tColumns.size()); + List> list = new ArrayList>(tColumns.size()); + for (CounterColumn c: tColumns) { + HCounterColumn column = new HCounterColumnImpl(c, nameSerializer); + columnsMap.put(column.getName(), column); + list.add(column); + } + columnsList = list; + } + + /** + * + * @return an unmodifiable list of the columns + */ + @Override + public List> getColumns() { + return columnsList; + } + + @Override + public HCounterColumn getColumnByName(N columnName) { + return columnsMap.get(columnName); + } + + @Override + public String toString() { + return String.format("ColumnSlice(%s)", columnsList.toString()); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/CqlQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/CqlQuery.java new file mode 100644 index 000000000..821286ea0 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/CqlQuery.java @@ -0,0 +1,135 @@ +package me.prettyprint.cassandra.model; + +import java.nio.ByteBuffer; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.service.Operation; +import me.prettyprint.cassandra.service.OperationType; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.exceptions.HectorException; +import me.prettyprint.hector.api.query.QueryResult; + +import org.apache.cassandra.thrift.Column; +import org.apache.cassandra.thrift.Compression; +import org.apache.cassandra.thrift.CqlResult; +import org.apache.cassandra.thrift.CqlRow; +import org.apache.cassandra.thrift.Cassandra.Client; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * First cut at a CQL implementation. Not too much time has been spent here + * as this API is currently a moving target. We have a lot of experience with + * these hijinks by the Apache Cassandra team, so it was deemed prudent to do + * something simple initially until the dust settles. + * + * You are expected to know what you are getting into if you plan on using + * CQL queries in your application. Spend some time looking through the + * unit tests here in Hector and the Cassandra source tree. For a number of + * detailed examples, see test_cql.py in the test/system folder of the + * Apache Cassandra source distribution. + * + * Note: if you immediately get an exception such as: + * "InvalidRequestException(why:cannot parse 'foo' as hex bytes)" + * It means one of two things: + *
    + *
  1. you have not formatted your query correct
  2. + *
  3. You have not configured the correct validators on your column family
  4. + *
+ * + * In both cases, even though the query is most likely a string, it is up to you to format + * this query according to the comparator (used for the column name), key validator + * and value validator. This can be a little confusing as only the comparator is required. + * The other two default to BytesType. + * + * See the docs on {@link CqlRows} for additional details. + * + * @author zznate + * + */ +public class CqlQuery extends AbstractBasicQuery> { + private static Logger log = LoggerFactory.getLogger(CqlQuery.class); + + private Serializer valueSerializer; + private ByteBuffer query; + private boolean useCompression; + + public CqlQuery(Keyspace k, Serializer keySerializer, + Serializer nameSerializer, Serializer valueSerializer) { + super(k, keySerializer, nameSerializer); + this.valueSerializer = valueSerializer; + } + + /** + * Set the query as a String. Here for convienience. See above for some + * caveats. Calls {@link StringSerializer#toByteBuffer(String)} directly. + * @param query + * @return + */ + public CqlQuery setQuery(String query) { + this.query = StringSerializer.get().toByteBuffer(query); + return this; + } + + public CqlQuery setQuery(ByteBuffer qeury) { + this.query = query; + return this; + } + + public CqlQuery useCompression() { + useCompression = true; + return this; + } + + + @Override + public QueryResult> execute() { + + return new QueryResultImpl>( + keyspace.doExecuteOperation(new Operation>(OperationType.READ) { + + @Override + public CqlRows execute(Client cassandra) throws HectorException { + CqlRows rows = null; + try { + CqlResult result = cassandra.execute_cql_query(query, + useCompression ? Compression.GZIP : Compression.NONE); + if ( log.isDebugEnabled() ) { + log.debug("Found CqlResult: {}", result); + } + switch (result.getType()) { + case INT: + rows = new CqlRows(result.getNum()); + break; + case VOID: + rows = new CqlRows(); + break; + + default: + if ( result.getRowsSize() > 0 ) { + LinkedHashMap> ret = new LinkedHashMap>(result.getRowsSize()); + + for (Iterator rowsIter = result.getRowsIterator(); rowsIter.hasNext(); ) { + CqlRow row = rowsIter.next(); + ret.put(ByteBuffer.wrap(row.getKey()), row.getColumns()); + } + Map> thriftRet = keySerializer.fromBytesMap(ret); + rows = new CqlRows((LinkedHashMap>)thriftRet, columnNameSerializer, valueSerializer); + } + break; + } + } catch (Exception ex) { + throw keyspace.getExceptionsTranslator().translate(ex); + } + return rows; + } + + }), this); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/CqlRows.java b/core/src/main/java/me/prettyprint/cassandra/model/CqlRows.java new file mode 100644 index 000000000..a3fc58a12 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/CqlRows.java @@ -0,0 +1,65 @@ +package me.prettyprint.cassandra.model; + +import java.util.LinkedHashMap; +import java.util.List; + +import me.prettyprint.hector.api.Serializer; + +import org.apache.cassandra.thrift.Column; +import org.apache.cassandra.thrift.CqlResultType; + +/** + * Row wrapper specific to the multi-type results capable from a CqlQuery. + * This is a bit more convoluted than I would like, put most of this API + * is still moving around, so we will stick with the overloading for now. + * + * @author zznate + */ +public final class CqlRows extends OrderedRowsImpl { + + private final CqlResultType resultType; + private int count; + + /** + * Constructed for {@link CqlResultType#ROWS} + * @param thriftRet + * @param nameSerializer + * @param valueSerializer + * @param resultType + */ + public CqlRows(LinkedHashMap> thriftRet, + Serializer nameSerializer, Serializer valueSerializer) { + super(thriftRet, nameSerializer, valueSerializer); + this.resultType = CqlResultType.ROWS; + } + + /** + * Constructed with only a count for {@link CqlResultType#INT} + * @param count + */ + public CqlRows(int count) { + super(); + this.resultType = CqlResultType.INT; + this.count = count; + } + + /** + * Constructed as empty for {@link CqlResultType#VOID} + */ + public CqlRows() { + super(); + this.resultType = CqlResultType.VOID; + } + + /** + * Will throw an IllegalArgumentException if called on a query that was not + * {@link CqlResultType#INT} + * @return + */ + public int getAsCount() { + if ( !resultType.equals(CqlResultType.INT)) + throw new IllegalArgumentException("Attempted to extract count from the wrong type of CQL query: " + resultType.toString()); + return count; + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/ExecutingKeyspace.java b/core/src/main/java/me/prettyprint/cassandra/model/ExecutingKeyspace.java index 4707eb1ae..1d2c32938 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/ExecutingKeyspace.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/ExecutingKeyspace.java @@ -17,7 +17,7 @@ /** * Thread Safe - * + * * @author Ran Tavory * @author zznate */ @@ -59,8 +59,8 @@ public ExecutingKeyspace(String keyspace, public void setConsistencyLevelPolicy(ConsistencyLevelPolicy cp) { // TODO remove this method consistencyLevelPolicy = cp; - } - + } + @Override public String getKeyspaceName() { return keyspace; diff --git a/core/src/main/java/me/prettyprint/cassandra/model/ExecutionResult.java b/core/src/main/java/me/prettyprint/cassandra/model/ExecutionResult.java index d193bf38d..62a43519c 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/ExecutionResult.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/ExecutionResult.java @@ -1,6 +1,7 @@ package me.prettyprint.cassandra.model; import me.prettyprint.cassandra.service.CassandraHost; +import me.prettyprint.hector.api.ResultStatus; /** @@ -11,12 +12,12 @@ * @author Ran * @author zznate */ -public class ExecutionResult { +public class ExecutionResult implements ResultStatus { private final T value; private final long execTime; private final CassandraHost cassandraHost; - + protected static final String BASE_MSG_FORMAT = "%s took (%dus) for query (%s) on host: %s"; private static final int MICRO_DENOM = 1000; @@ -35,7 +36,14 @@ public T get() { } /** - * Execution time is actually recorded in nanos, so we divide this by 1000 + * @return the execution time, as already recorded, in nanos + */ + public long getExecutionTimeNano() { + return execTime; + } + + /** + * Execution time is actually recorded in nanos, so we divide this by 1000 * make the number more sensible * @return */ @@ -43,11 +51,12 @@ public long getExecutionTimeMicro() { return execTime / MICRO_DENOM; } + @Override public String toString() { return formatMessage("ExecutionResult", "n/a"); } - + protected String formatMessage(String resultName, String query) { return String.format(BASE_MSG_FORMAT, resultName, getExecutionTimeMicro(), query, (cassandraHost != null ? cassandraHost.getName() : "[none]")); } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/HColumnImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/HColumnImpl.java index 9c0e7cb5f..048069e01 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/HColumnImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/HColumnImpl.java @@ -33,8 +33,15 @@ public HColumnImpl(N name, V value, long clock, Serializer nameSerializer, notNull(name, "name is null"); notNull(value, "value is null"); - this.column = new Column(nameSerializer.toByteBuffer(name), - valueSerializer.toByteBuffer(value), clock); + this.column = new Column(nameSerializer.toByteBuffer(name)); + this.column.setValue(valueSerializer.toByteBuffer(value)); + this.column.setTimestamp(clock); + } + + public HColumnImpl(N name, V value, long clock, int ttl, + Serializer nameSerializer, Serializer valueSerializer) { + this(name, value, clock, nameSerializer, valueSerializer); + setTtl(ttl); } public HColumnImpl(Column thriftColumn, Serializer nameSerializer, @@ -60,14 +67,14 @@ public HColumnImpl(N name, V value, long clock) { @Override public HColumn setName(N name) { notNull(name, "name is null"); - this.column.name = nameSerializer.toByteBuffer(name); + this.column.setName(nameSerializer.toByteBuffer(name)); return this; } @Override public HColumn setValue(V value) { notNull(value, "value is null"); - this.column.value = valueSerializer.toByteBuffer(value); + this.column.setValue(valueSerializer.toByteBuffer(value)); return this; } @@ -78,7 +85,7 @@ public HColumn setClock(long clock) { } /** - * Set the time-to-live value for this column in seconds. + * Set the time-to-live value for this column in seconds. * The server will mark this column as deleted once the number of seconds has elapsed. */ @Override @@ -94,20 +101,14 @@ public int getTtl() { @Override public N getName() { - if ( column.name == null ) { - return null; - } - return nameSerializer.fromByteBuffer(column.name.duplicate()); + return column.isSetName() ? nameSerializer.fromByteBuffer(column.name.duplicate()) : null; } @Override - public V getValue() { - if ( column.value == null ) { - return null; - } - return valueSerializer.fromByteBuffer(column.value.duplicate()); - } - + public V getValue() { + return column.isSetValue() ? valueSerializer.fromByteBuffer(column.value.duplicate()) : null; + } + @Override public long getClock() { @@ -134,25 +135,35 @@ public Serializer getValueSerializer() { return valueSerializer; } - + @Override + public ByteBuffer getNameBytes() { + return column.isSetName() ? column.name.duplicate() : null; + } + + @Override + public ByteBuffer getValueBytes() { + return column.isSetValue() ? column.value.duplicate() : null; + } + /** * Clear value, timestamp and ttl (the latter two set to '0') leaving only the column name */ @Override - public HColumn clear() { + public HColumn clear() { column.value = null; column.timestamp = 0; column.ttl = 0; column.setTimestampIsSet(false); column.setTtlIsSet(false); + column.setValueIsSet(false); return this; } - - + + @Override public HColumn apply(V value, long clock, int ttl) { - column.value = valueSerializer.toByteBuffer(value); + setValue(value); column.setTimestamp(clock); column.setTtl(ttl); return this; @@ -162,7 +173,7 @@ public HColumn apply(Column c) { this.column = c; return this; } - + @Override public String toString() { return String.format("HColumn(%s=%s)",getName(), getValue()); diff --git a/core/src/main/java/me/prettyprint/cassandra/model/HCounterColumnImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/HCounterColumnImpl.java new file mode 100644 index 000000000..db833b5f3 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/HCounterColumnImpl.java @@ -0,0 +1,165 @@ +package me.prettyprint.cassandra.model; + +import static me.prettyprint.cassandra.utils.Assert.notNull; + +import java.nio.ByteBuffer; + +import me.prettyprint.cassandra.serializers.SerializerTypeInferer; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.beans.HCounterColumn; + +import org.apache.cassandra.thrift.Column; +import org.apache.cassandra.thrift.CounterColumn; +import org.apache.commons.lang.builder.EqualsBuilder; +import org.apache.commons.lang.builder.HashCodeBuilder; + +/** + * Hector Counter Column definition. + * + * @param The type of the column name + * + * @author patricioe (patricioe@gmail.com) + */ +public final class HCounterColumnImpl implements HCounterColumn { + + private CounterColumn counterColumn; + private Serializer nameSerializer; + + + public HCounterColumnImpl(N name, Long value, Serializer nameSerializer) { + this(nameSerializer); + notNull(name, "name is null"); + notNull(value, "value is null"); + this.counterColumn = new CounterColumn(nameSerializer.toByteBuffer(name), value); + } + + public HCounterColumnImpl(CounterColumn thriftCounterColumn, Serializer nameSerializer) { + this(nameSerializer); + notNull(thriftCounterColumn, "thriftColumn is null"); + this.counterColumn = thriftCounterColumn; + } + + public HCounterColumnImpl(Serializer nameSerializer) { + notNull(nameSerializer, "nameSerializer is null"); + this.nameSerializer = nameSerializer; + this.counterColumn = new CounterColumn(); + } + + public HCounterColumnImpl(N name, Long value) { + this(name, value, SerializerTypeInferer.getSerializer(name)); + } + + @Override + public HCounterColumn setName(N name) { + notNull(name, "name is null"); + this.counterColumn.setName(nameSerializer.toByteBuffer(name)); + return this; + } + + @Override + public HCounterColumn setValue(Long value) { + notNull(value, "value is null"); + this.counterColumn.setValue(value); + return this; + } + + /** + * Set the time-to-live value for this column in seconds. + * The server will mark this column as deleted once the number of seconds has elapsed. + */ + @Override + public HCounterColumn setTtl(int ttl) { + //this.counterColumn.setTtl(ttl); + // TODO (patricioe) Pending on Cassandra trunk + return this; + } + + @Override + public int getTtl() { + //return this.counterColumn.ttl; + // TODO (patricioe) Pending on Cassandra trunk + return Integer.MAX_VALUE; + } + + @Override + public N getName() { + return counterColumn.isSetName() ? nameSerializer.fromByteBuffer(counterColumn.name.duplicate()) : null; + } + + @Override + public Long getValue() { + return counterColumn.value; + } + + public CounterColumn toThrift() { + return counterColumn; + } + + public HCounterColumn fromThrift(CounterColumn c) { + notNull(c, "column is null"); + this.counterColumn = c; + return this; + } + + @Override + public Serializer getNameSerializer() { + return nameSerializer; + } + + @Override + public ByteBuffer getNameBytes() { + return counterColumn.isSetName() ? counterColumn.name.duplicate() : null; + } + + /** + * Clear value, timestamp and ttl (the latter two set to '0') leaving only the column name + */ + @Override + public HCounterColumn clear() { + counterColumn.value = 0; // TODO (patricioe) Is this ok? + //counterColumn.ttl = 0; TODO (patricioe) pending on trunk + //counterColumn.setTtlIsSet(false); + counterColumn.setValueIsSet(false); + return this; + } + + @Override + public HCounterColumn apply(Long value, int ttl) { + setValue(value); + //counterColumn.setTtl(ttl); + return this; + } + + public HCounterColumn apply(CounterColumn c) { + this.counterColumn = c; + return this; + } + + @Override + public String toString() { + return String.format("HCounterColumn(%s=%s)",getName(), getValue()); + } + + @Override + public int hashCode() { + return new HashCodeBuilder().append(getName()).append(getValue()).toHashCode(); + } + + @Override + public boolean equals(Object obj) { + if (obj == null) { + return false; + } + if (obj == this) { + return true; + } + if (obj.getClass() != getClass()) { + return false; + } + @SuppressWarnings("unchecked") + HCounterColumnImpl other = (HCounterColumnImpl) obj; + return new EqualsBuilder().appendSuper(super.equals(obj)).append(getName(), other.getName()). + append(getValue(), other.getValue()).isEquals(); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/HCounterSuperColumnImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/HCounterSuperColumnImpl.java new file mode 100644 index 000000000..9e8a411e6 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/HCounterSuperColumnImpl.java @@ -0,0 +1,147 @@ +package me.prettyprint.cassandra.model; + +import static me.prettyprint.cassandra.utils.Assert.noneNull; +import static me.prettyprint.cassandra.utils.Assert.notNull; + +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.List; + +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HCounterColumn; +import me.prettyprint.hector.api.beans.HCounterSuperColumn; + +import org.apache.cassandra.thrift.CounterColumn; +import org.apache.cassandra.thrift.CounterSuperColumn; + +/** + * Models a CounterSuperColumn in a protocol independent manner. + * + * @param + * CounterSuperColumn name type + * @param + * CounterColumn name type + + * + * @author patricioe + */ +public final class HCounterSuperColumnImpl implements HCounterSuperColumn { + + private SN superName; + private List> counterColumns; + private final Serializer superNameSerializer; + private final Serializer nameSerializer; + + /** + * @param CounterSuperColumn name type + * @param List> CounterColumn values + * @param Serializer the serializer type + */ + public HCounterSuperColumnImpl(SN sName, List> counterColumns, + Serializer sNameSerializer, Serializer nameSerializer) { + this(sNameSerializer, nameSerializer); + notNull(sName, "Name is null"); + notNull(counterColumns, "Columns are null"); + this.superName = sName; + this.counterColumns = counterColumns; + } + + public HCounterSuperColumnImpl(CounterSuperColumn thriftCounterSuperColumn, Serializer sNameSerializer, + Serializer nameSerializer) { + this(sNameSerializer, nameSerializer); + noneNull(thriftCounterSuperColumn, sNameSerializer, nameSerializer); + superName = sNameSerializer.fromByteBuffer(ByteBuffer.wrap(thriftCounterSuperColumn.getName())); + counterColumns = fromThriftColumns(thriftCounterSuperColumn.getColumns()); + } + + /*package*/ HCounterSuperColumnImpl(Serializer sNameSerializer, Serializer nameSerializer) { + noneNull(sNameSerializer, nameSerializer); + this.superNameSerializer = sNameSerializer; + this.nameSerializer = nameSerializer; + } + + @Override + public HCounterSuperColumn setName(SN name) { + notNull(name, "name is null"); + this.superName = name; + return this; + } + + @Override + public HCounterSuperColumn setSubcolumns(List> counterSubcolumns) { + notNull(counterSubcolumns, "subcolumns are null"); + this.counterColumns = counterSubcolumns; + return this; + } + + @Override + public int getSize() { + return counterColumns == null ? 0 : counterColumns.size(); + } + + @Override + public SN getName() { + return superName; + } + + /** + * + * @return an unmodifiable list of counterColumns + */ + @Override + public List> getColumns() { + return counterColumns; + } + + @Override + public HCounterColumn get(int i) { + return counterColumns.get(i); + } + + @Override + public Serializer getNameSerializer() { + return superNameSerializer; + } + + @Override + public byte[] getNameBytes() { + return superNameSerializer.toByteBuffer(getName()).array(); + } + + public ByteBuffer getNameByteBuffer() { + return superNameSerializer.toByteBuffer(getName()); + } + + public CounterSuperColumn toThrift() { + if (superName == null || counterColumns == null) { + return null; + } + return new CounterSuperColumn(superNameSerializer.toByteBuffer(superName), toThriftColumn()); + } + + private List toThriftColumn() { + List ret = new ArrayList(counterColumns.size()); + for (HCounterColumn c: counterColumns) { + ret.add(((HCounterColumnImpl) c).toThrift()); + } + return ret; + } + + private List> fromThriftColumns(List tcolumns) { + List> cs = new ArrayList>(tcolumns.size()); + for (CounterColumn c: tcolumns) { + cs.add(new HCounterColumnImpl(c, nameSerializer)); + } + return cs; + } + + @Override + public Serializer getSuperNameSerializer() { + return superNameSerializer; + } + + @Override + public String toString() { + return String.format("HCounterSuperColumn(%s,%s)", superName, counterColumns); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/HSlicePredicate.java b/core/src/main/java/me/prettyprint/cassandra/model/HSlicePredicate.java index 96363e102..616e386f6 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/HSlicePredicate.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/HSlicePredicate.java @@ -33,6 +33,8 @@ public final class HSlicePredicate { protected enum PredicateType {Unknown, ColumnNames, Range}; protected PredicateType predicateType = PredicateType.Unknown; + private static final ByteBuffer EMPTY_BYTE_BUFFER = ByteBuffer.wrap(new byte[0]); + public HSlicePredicate(Serializer columnNameSerializer) { Assert.notNull(columnNameSerializer, "columnNameSerializer can't be null"); this.columnNameSerializer = columnNameSerializer; @@ -47,7 +49,7 @@ public HSlicePredicate(Serializer columnNameSerializer) { public HSlicePredicate setColumnNames(N... columnNames) { return setColumnNames(Arrays.asList(columnNames)); } - + public HSlicePredicate addColumnName(N columnName) { if ( columnNames == null ) columnNames = new ArrayList(); @@ -55,7 +57,7 @@ public HSlicePredicate addColumnName(N columnName) { predicateType = PredicateType.ColumnNames; return this; } - + /** * Same as varargs signature, except we take a collection * @@ -80,28 +82,28 @@ public HSlicePredicate setKeysOnlyPredicate() { } /** - * Set the columnName on which we will start. - * Switches to {@link PredicateType#Range} + * Set the columnName on which we will start. + * Switches to {@link PredicateType#Range} */ public HSlicePredicate setStartOn(N start) { this.start = start; predicateType = PredicateType.Range; return this; } - + /** - * Set the columnName on which we will end. - * Switches to {@link PredicateType#Range} + * Set the columnName on which we will end. + * Switches to {@link PredicateType#Range} */ public HSlicePredicate setEndOn(N finish) { this.finish = finish; predicateType = PredicateType.Range; return this; } - + /** * Set the number of columns to return for this slice - * Switches to {@link PredicateType#Range} + * Switches to {@link PredicateType#Range} */ public HSlicePredicate setCount(int count) { this.count = count; @@ -111,19 +113,19 @@ public HSlicePredicate setCount(int count) { } /** - * Sets the return order of the columns to be reversed. - * NOTE: this is slightly less efficient than reading in comparator order. - * Switches to {@link PredicateType#Range} + * Sets the return order of the columns to be reversed. + * NOTE: this is slightly less efficient than reading in comparator order. + * Switches to {@link PredicateType#Range} */ public HSlicePredicate setReversed(boolean reversed) { this.reversed = reversed; predicateType = PredicateType.Range; return this; } - - - - + + + + /** * Set a predicate of start/finish to retrieve a list of columns in this range. * Either start and or finish can be null which will toggle the underlying predicate to @@ -180,7 +182,7 @@ public SlicePredicate toThrift() { private ByteBuffer findBytes(N val) { ByteBuffer valBytes; if (val == null) { - valBytes = ByteBuffer.wrap(new byte[0]); + valBytes = EMPTY_BYTE_BUFFER; } else { valBytes = columnNameSerializer.toByteBuffer(val); } @@ -197,11 +199,11 @@ private List toThriftColumnNames(Collection clms) { @Override public String toString() { - return String.format("HSlicePredicate(%s)", predicateType == PredicateType.ColumnNames ? columnNames : formatPredicate()); + return String.format("HSlicePredicate(%s)", predicateType == PredicateType.ColumnNames ? columnNames : formatPredicate()); } - + private String formatPredicate() { - return String.format("start:[%s],end:[%s],count:%d,reversed:%b", + return String.format("start:[%s],end:[%s],count:%d,reversed:%b", start != null ? start.toString() : "''", finish != null ? finish.toString() : "''", count, reversed); diff --git a/core/src/main/java/me/prettyprint/cassandra/model/HSuperColumnImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/HSuperColumnImpl.java index c4e1d47e6..48713c0a6 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/HSuperColumnImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/HSuperColumnImpl.java @@ -55,14 +55,14 @@ public HSuperColumnImpl(SN sName, List> columns, long clock, public HSuperColumnImpl(SuperColumn thriftSuperColumn, Serializer sNameSerializer, Serializer nameSerializer, Serializer valueSerializer) { this(sNameSerializer, nameSerializer, valueSerializer); - noneNull(thriftSuperColumn, sNameSerializer, nameSerializer, valueSerializer); + noneNull(thriftSuperColumn, sNameSerializer, nameSerializer); superName = sNameSerializer.fromByteBuffer(ByteBuffer.wrap(thriftSuperColumn.getName())); columns = fromThriftColumns(thriftSuperColumn.getColumns()); } /*package*/ HSuperColumnImpl(Serializer sNameSerializer, Serializer nameSerializer, Serializer valueSerializer) { - noneNull(sNameSerializer, nameSerializer, valueSerializer); + noneNull(sNameSerializer, nameSerializer); this.superNameSerializer = sNameSerializer; this.nameSerializer = nameSerializer; this.valueSerializer = valueSerializer; @@ -126,7 +126,7 @@ public Serializer getNameSerializer() { public byte[] getNameBytes() { return superNameSerializer.toByteBuffer(getName()).array(); } - + public ByteBuffer getNameByteBuffer() { return superNameSerializer.toByteBuffer(getName()); } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/IndexedSlicesQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/IndexedSlicesQuery.java index fb1cad03e..66ed268e0 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/IndexedSlicesQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/IndexedSlicesQuery.java @@ -23,7 +23,7 @@ * Uses new secondary indexes. Your CF must be configured for such to use this. * The following creates an Indexed CF with the "birthday" column indexed (where * birthdate represents a timestamp as it is validated by the LongType): - * + * *
  *         - name: Indexed1
  *           column_metadata:
@@ -31,7 +31,7 @@
  *               validator_class: LongType
  *               index_type: KEYS
  * 
- * + * * @author zznate (nate@riptano.com) */ public class IndexedSlicesQuery extends diff --git a/core/src/main/java/me/prettyprint/cassandra/model/MutatorImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/MutatorImpl.java index a06ea97ff..46fcb7374 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/MutatorImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/MutatorImpl.java @@ -9,11 +9,14 @@ import me.prettyprint.hector.api.Keyspace; import me.prettyprint.hector.api.Serializer; import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.beans.HCounterColumn; +import me.prettyprint.hector.api.beans.HCounterSuperColumn; import me.prettyprint.hector.api.beans.HSuperColumn; import me.prettyprint.hector.api.exceptions.HectorException; import me.prettyprint.hector.api.mutation.MutationResult; import me.prettyprint.hector.api.mutation.Mutator; +import org.apache.cassandra.thrift.ColumnParent; import org.apache.cassandra.thrift.Deletion; import org.apache.cassandra.thrift.SlicePredicate; @@ -30,6 +33,7 @@ * * @author Ran Tavory * @author zznate + * @author patricioe */ public final class MutatorImpl implements Mutator { @@ -69,10 +73,10 @@ public MutationResult delete(final K key, final String cf, final N columnNam addDeletion(key, cf, columnName, nameSerializer); return execute(); } - + @Override public MutationResult delete(K key, String cf, N columnName, - Serializer nameSerializer, long clock) { + Serializer nameSerializer, long clock) { addDeletion(key, cf, columnName, nameSerializer, clock); return execute(); } @@ -94,31 +98,46 @@ public Void doInKeyspace(KeyspaceService ks) throws HectorException { return null; } })); - } - + } + + @Override + public MutationResult superDelete(final K key, final String cf, final SN supercolumnName, + final Serializer sNameSerializer) { + return new MutationResultImpl(keyspace.doExecute(new KeyspaceOperationCallback() { + @Override + public Void doInKeyspace(KeyspaceService ks) throws HectorException { + // Remove a Super Column. + ks.remove( + keySerializer.toByteBuffer(key), + ThriftFactory.createSuperColumnPath(cf, supercolumnName, sNameSerializer)); + return null; + } + })); + } + /** * Deletes the columns defined in the HSuperColumn. If there are no HColumns attached, - * we delete the whole thing. - * + * we delete the whole thing. + * */ public Mutator addSubDelete(K key, String cf, HSuperColumn sc) { return addSubDelete(key, cf, sc, keyspace.createClock()); } - + public Mutator addSubDelete(K key, String cf, HSuperColumn sc, long clock) { SlicePredicate pred = new SlicePredicate(); - Deletion d = new Deletion(clock); - if ( sc.getColumns() != null ) { + Deletion d = new Deletion().setTimestamp(clock); + if ( sc.getColumns() != null ) { for (HColumn col : sc.getColumns()) { pred.addToColumn_names(col.getNameSerializer().toByteBuffer(col.getName())); } d.setPredicate(pred); - } + } d.setSuper_column(sc.getNameByteBuffer()); - getPendingMutations().addDeletion(key, Arrays.asList(cf), d); + getPendingMutations().addDeletion(key, Arrays.asList(cf), d); return this; } - + // schedule an insertion to be executed in batch by the execute method // CAVEAT: a large number of calls with a typo in one of them will leave things in an // indeterminant state if we dont validate against LIVE (but cached of course) @@ -149,7 +168,7 @@ public Mutator addDeletion(K key, String cf, N columnName, Serializer addDeletion(key, cf, columnName, nameSerializer, keyspace.createClock()); return this; } - + /** * {@inheritDoc} */ @@ -158,7 +177,7 @@ public Mutator addDeletion(K key, String cf) { addDeletion(key, cf, null, null, keyspace.createClock()); return this; } - + /** * {@inheritDoc} */ @@ -177,9 +196,9 @@ public Mutator addDeletion(K key, String cf, N columnName, Serializer Deletion d; if ( columnName != null ) { sp.addToColumn_names(nameSerializer.toByteBuffer(columnName)); - d = new Deletion(clock).setPredicate(sp); - } else { - d = new Deletion(clock); + d = new Deletion().setTimestamp(clock).setPredicate(sp); + } else { + d = new Deletion().setTimestamp(clock); } getPendingMutations().addDeletion(key, Arrays.asList(cf), d); return this; @@ -227,4 +246,143 @@ private BatchMutation getPendingMutations() { return pendingMutations; } + // Counters support. + + @Override + public MutationResult insertCounter(final K key, final String cf, final HCounterColumn c) { + return new MutationResultImpl(keyspace.doExecute(new KeyspaceOperationCallback() { + @Override + public Void doInKeyspace(KeyspaceService ks) throws HectorException { + ks.addCounter(keySerializer.toByteBuffer(key), new ColumnParent(cf), ((HCounterColumnImpl) c).toThrift()); + return null; + } + })); + } + + @Override + public MutationResult incrementCounter(final K key, final String cf, final N columnName, final long increment) { + return insertCounter(key, cf, new HCounterColumnImpl(columnName, increment, TypeInferringSerializer. get())); + } + + @Override + public MutationResult decrementCounter(final K key, final String cf, final N columnName, final long increment) { + return incrementCounter(key, cf, columnName, increment * -1L); + } + + + @Override + public MutationResult insertCounter(K key, String cf, HCounterSuperColumn superColumn) { + addCounter(key, cf, superColumn); + return execute(); + } + + + @Override + public MutationResult deleteCounter(final K key, final String cf, final N counterColumnName, + final Serializer nameSerializer) { + return new MutationResultImpl(keyspace.doExecute(new KeyspaceOperationCallback() { + @Override + public Void doInKeyspace(KeyspaceService ks) throws HectorException { + ks.removeCounter(keySerializer.toByteBuffer(key), ThriftFactory.createColumnPath(cf, counterColumnName, + nameSerializer)); + return null; + } + })); + } + + @Override + public MutationResult subDeleteCounter(final K key, final String cf, final SN supercolumnName, + final N columnName, final Serializer sNameSerializer, final Serializer nameSerializer) { + return new MutationResultImpl(keyspace.doExecute(new KeyspaceOperationCallback() { + @Override + public Void doInKeyspace(KeyspaceService ks) throws HectorException { + ks.removeCounter(keySerializer.toByteBuffer(key), ThriftFactory.createSuperColumnPath(cf, + supercolumnName, columnName, sNameSerializer, nameSerializer)); + return null; + } + })); + } + + @Override + public Mutator addCounter(K key, String cf, HCounterColumn c) { + getPendingMutations().addCounterInsertion(key, Arrays.asList(cf), ((HCounterColumnImpl) c).toThrift()); + return this; + } + + @Override + public Mutator addCounter(K key, String cf, HCounterSuperColumn sc) { + getPendingMutations().addSuperCounterInsertion(key, Arrays.asList(cf), + ((HCounterSuperColumnImpl) sc).toThrift()); + return this; + } + + + @Override + public Mutator addCounterDeletion(K key, String cf, N counterColumnName, Serializer nameSerializer) { + SlicePredicate sp = new SlicePredicate(); + Deletion d; + if ( counterColumnName != null ) { + sp.addToColumn_names(nameSerializer.toByteBuffer(counterColumnName)); + d = new Deletion().setPredicate(sp); + } else { + d = new Deletion(); + } + getPendingMutations().addDeletion(key, Arrays.asList(cf), d); + return this; + } + + @Override + public Mutator addCounterDeletion(K key, String cf) { + addCounterDeletion(key, cf); + return this; + } + + @Override + public Mutator addCounterSubDeletion(K key, String cf, HCounterSuperColumn sc) { + SlicePredicate pred = new SlicePredicate(); + Deletion d = new Deletion(); + if ( sc.getColumns() != null ) { + for (HCounterColumn col : sc.getColumns()) { + pred.addToColumn_names(col.getNameSerializer().toByteBuffer(col.getName())); + } + d.setPredicate(pred); + } + d.setSuper_column(sc.getNameByteBuffer()); + getPendingMutations().addDeletion(key, Arrays.asList(cf), d); + return this; + } + + @Override + public Mutator addSubDelete(K key, String cf, SN sColumnName, + N columnName, Serializer sNameSerializer, Serializer nameSerializer) { + return addSubDelete(key, cf, sColumnName, columnName, sNameSerializer, nameSerializer, keyspace.createClock()); + } + + @Override + public Mutator addSubDelete(K key, String cf, SN sColumnName, + N columnName, Serializer sNameSerializer, Serializer nameSerializer, long clock) { + Deletion d = new Deletion().setTimestamp(clock); + SlicePredicate predicate = new SlicePredicate(); + predicate.addToColumn_names(nameSerializer.toByteBuffer(columnName)); + d.setPredicate(predicate); + d.setSuper_column(sNameSerializer.toByteBuffer(sColumnName)); + getPendingMutations().addDeletion(key, Arrays.asList(cf), d); + return this; + } + + + + @Override + public Mutator addSuperDelete(K key, String cf, SN sColumnName, + Serializer sNameSerializer) { + Deletion d = new Deletion().setTimestamp(keyspace.createClock()); + d.setSuper_column(sNameSerializer.toByteBuffer(sColumnName)); + getPendingMutations().addDeletion(key, Arrays.asList(cf), d); + + return this; + + } + + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/OrderedRowsImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/OrderedRowsImpl.java index e785fc8a1..a19d0c8d5 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/OrderedRowsImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/OrderedRowsImpl.java @@ -16,9 +16,9 @@ * @param * @param */ -public final class OrderedRowsImpl extends RowsImpl implements OrderedRows { +public class OrderedRowsImpl extends RowsImpl implements OrderedRows { - private final List> rowsList; + protected final List> rowsList; public OrderedRowsImpl(LinkedHashMap> thriftRet, Serializer nameSerializer, Serializer valueSerializer) { @@ -26,6 +26,11 @@ public OrderedRowsImpl(LinkedHashMap> thriftRet, Serializer n rowsList = new ArrayList>(rows.values()); } + protected OrderedRowsImpl() { + super(); + rowsList = new ArrayList>(); + } + /** * Preserves rows order * @return an unmodifiable list of Rows diff --git a/core/src/main/java/me/prettyprint/cassandra/model/RowsImpl.java b/core/src/main/java/me/prettyprint/cassandra/model/RowsImpl.java index adc7cf78e..ae20e0216 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/RowsImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/RowsImpl.java @@ -35,6 +35,10 @@ public RowsImpl(Map> thriftRet, Serializer nameSerializer, } } + protected RowsImpl() { + this.rows = new HashMap>(); + } + @Override public Row getByKey(K key) { return rows.get(key); diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/AbstractThriftClientWrapper.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/AbstractThriftClientWrapper.java index 9b2dd898d..7b564cd23 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/thrift/AbstractThriftClientWrapper.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/AbstractThriftClientWrapper.java @@ -21,6 +21,7 @@ import org.apache.cassandra.thrift.KsDef; import org.apache.cassandra.thrift.Mutation; import org.apache.cassandra.thrift.NotFoundException; +import org.apache.cassandra.thrift.SchemaDisagreementException; import org.apache.cassandra.thrift.SlicePredicate; import org.apache.cassandra.thrift.TimedOutException; import org.apache.cassandra.thrift.TokenRange; @@ -31,7 +32,7 @@ /** * For creating wrappers around the Cassandra client in order to perform pre and * post processing - * + * * @author Ed Anuff */ @@ -93,7 +94,7 @@ public String describe_snitch() throws TException { @Override public List describe_splits(String cfName, String start_token, - String end_token, int keys_per_split) throws TException { + String end_token, int keys_per_split) throws TException, InvalidRequestException { return client.describe_splits(cfName, start_token, end_token, keys_per_split); } @@ -232,7 +233,7 @@ public String recv_describe_snitch() throws TException { } @Override - public List recv_describe_splits() throws TException { + public List recv_describe_splits() throws TException, InvalidRequestException { return client.recv_describe_splits(); } @@ -312,37 +313,37 @@ public void recv_set_keyspace() throws InvalidRequestException, TException { @Override public String recv_system_add_column_family() throws InvalidRequestException, - TException { + TException, SchemaDisagreementException { return client.recv_system_add_column_family(); } @Override public String recv_system_add_keyspace() throws InvalidRequestException, - TException { + TException, SchemaDisagreementException { return client.recv_system_add_keyspace(); } @Override public String recv_system_drop_column_family() - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.recv_system_drop_column_family(); } @Override public String recv_system_drop_keyspace() throws InvalidRequestException, - TException { + TException, SchemaDisagreementException { return client.recv_system_drop_keyspace(); } @Override public String recv_system_update_column_family() - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.recv_system_update_column_family(); } @Override public String recv_system_update_keyspace() throws InvalidRequestException, - TException { + TException, SchemaDisagreementException { return client.recv_system_update_keyspace(); } @@ -530,37 +531,37 @@ public void set_keyspace(String keyspace) throws InvalidRequestException, @Override public String system_add_column_family(CfDef cf_def) - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.system_add_column_family(cf_def); } @Override public String system_add_keyspace(KsDef ks_def) - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.system_add_keyspace(ks_def); } @Override public String system_drop_column_family(String column_family) - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.system_drop_column_family(column_family); } @Override public String system_drop_keyspace(String keyspace) - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.system_drop_keyspace(keyspace); } @Override public String system_update_column_family(CfDef cf_def) - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.system_update_column_family(cf_def); } @Override public String system_update_keyspace(KsDef ks_def) - throws InvalidRequestException, TException { + throws InvalidRequestException, TException, SchemaDisagreementException { return client.system_update_keyspace(ks_def); } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftColumnFactory.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftColumnFactory.java new file mode 100644 index 000000000..35e90f126 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftColumnFactory.java @@ -0,0 +1,32 @@ +package me.prettyprint.cassandra.model.thrift; + +import me.prettyprint.cassandra.model.HColumnImpl; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.hector.api.ColumnFactory; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.factory.HFactory; + +public class ThriftColumnFactory implements ColumnFactory { + + @Override + public HColumn createColumn(N name, V value, + Serializer nameSerializer, Serializer valueSerializer) { + return new HColumnImpl(name, value, HFactory.createClock(), nameSerializer, + valueSerializer); + } + + @Override + public HColumn createColumn(N name, V value, long clock, + Serializer nameSerializer, Serializer valueSerializer) { + return new HColumnImpl(name, value, clock, nameSerializer, + valueSerializer); + } + + @Override + public HColumn createStringColumn(String name, String value) { + return createColumn(name, value, StringSerializer.get(), StringSerializer.get()); + } + + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftConverter.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftConverter.java index 87f5d9e92..1c3e512ff 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftConverter.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftConverter.java @@ -23,6 +23,10 @@ public static ConsistencyLevel consistencyLevel(HConsistencyLevel c) { return ConsistencyLevel.ANY; case ONE: return ConsistencyLevel.ONE; + case TWO: + return ConsistencyLevel.TWO; + case THREE: + return ConsistencyLevel.THREE; case QUORUM: return ConsistencyLevel.QUORUM; case EACH_QUORUM: @@ -33,7 +37,7 @@ public static ConsistencyLevel consistencyLevel(HConsistencyLevel c) { throw new RuntimeException("Unregornized consistency level " + c); } } - + /** * Converts a list of ColumnOrSuperColumn to Column * @param columns @@ -46,5 +50,5 @@ public static List getColumnList(List columns) { } return list; } - + } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftCounterColumnQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftCounterColumnQuery.java new file mode 100644 index 000000000..2189ea647 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftCounterColumnQuery.java @@ -0,0 +1,78 @@ +package me.prettyprint.cassandra.model.thrift; + +import me.prettyprint.cassandra.model.AbstractBasicQuery; +import me.prettyprint.cassandra.model.HCounterColumnImpl; +import me.prettyprint.cassandra.model.KeyspaceOperationCallback; +import me.prettyprint.cassandra.model.QueryResultImpl; +import me.prettyprint.cassandra.serializers.TypeInferringSerializer; +import me.prettyprint.cassandra.service.KeyspaceService; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HCounterColumn; +import me.prettyprint.hector.api.exceptions.HNotFoundException; +import me.prettyprint.hector.api.exceptions.HectorException; +import me.prettyprint.hector.api.query.CounterQuery; +import me.prettyprint.hector.api.query.QueryResult; + +import org.apache.cassandra.thrift.CounterColumn; + +/** + * Thrift implementation of the ColumnQuery type. + * + * @author Ran Tavory + * + * @param + * column name type + * @param + * value type + */ +public class ThriftCounterColumnQuery extends AbstractBasicQuery> + implements CounterQuery { + + protected K key; + protected N name; + + public ThriftCounterColumnQuery(Keyspace keyspace, Serializer keySerializer, + Serializer nameSerializer) { + super(keyspace, keySerializer, nameSerializer); + } + + public ThriftCounterColumnQuery(Keyspace keyspace) { + super(keyspace, TypeInferringSerializer. get(), TypeInferringSerializer. get()); + } + + public CounterQuery setKey(K key) { + this.key = key; + return this; + } + + public CounterQuery setName(N name) { + this.name = name; + return this; + } + + @SuppressWarnings("unchecked") + @Override + public CounterQuery setColumnFamily(String cf) { + return (CounterQuery) super.setColumnFamily(cf); + } + + @Override + public QueryResult> execute() { + return new QueryResultImpl>( + keyspace.doExecute(new KeyspaceOperationCallback>() { + + @Override + public HCounterColumn doInKeyspace(KeyspaceService ks) throws HectorException { + try { + CounterColumn thriftCounter = ks.getCounter(keySerializer.toByteBuffer(key), + ThriftFactory.createColumnPath(columnFamilyName, name, columnNameSerializer)); + return new HCounterColumnImpl(thriftCounter, columnNameSerializer); + } catch (HNotFoundException e) { + return null; + } + } + }), this); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftFactory.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftFactory.java index d9d06f637..3dd081ccf 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftFactory.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftFactory.java @@ -5,9 +5,11 @@ import java.nio.ByteBuffer; +import me.prettyprint.cassandra.serializers.StringSerializer; import me.prettyprint.hector.api.Serializer; import org.apache.cassandra.thrift.ColumnPath; +import org.apache.cassandra.thrift.CounterColumn; /** * Utility factory class for creating thrift objects. @@ -18,7 +20,7 @@ public class ThriftFactory { // probably should be typed for thrift vs. avro - /*package*/ static ColumnPath createColumnPath(String columnFamilyName, N columnName, + public static ColumnPath createColumnPath(String columnFamilyName, N columnName, Serializer nameSerializer) { return createColumnPath(columnFamilyName, nameSerializer.toByteBuffer(columnName)); } @@ -44,7 +46,7 @@ public static ColumnPath createSuperColumnPath(String columnFamilyName, return columnPath; } - /*package*/ static ColumnPath createSuperColumnPath(String columnFamilyName, + public static ColumnPath createSuperColumnPath(String columnFamilyName, SN superColumnName, Serializer superNameSerializer) { noneNull(columnFamilyName, superNameSerializer); ColumnPath columnPath = createColumnPath(columnFamilyName, null); @@ -54,4 +56,18 @@ public static ColumnPath createSuperColumnPath(String columnFamilyName, return columnPath; } + public static CounterColumn createCounterColumn(String name, long value) { + CounterColumn cc = new CounterColumn(); + cc.setName(StringSerializer.get().toByteBuffer(name)); + cc.setValue(value); + return cc; + } + + public static CounterColumn createCounterColumn(N name, long value, Serializer ns) { + CounterColumn cc = new CounterColumn(); + cc.setName(ns.toByteBuffer(name)); + cc.setValue(value); + return cc; + } + } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSubSliceQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSubSliceQuery.java index 3b06575f3..6032280e2 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSubSliceQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSubSliceQuery.java @@ -45,6 +45,12 @@ public MultigetSubSliceQuery setKeys(K... keys) { return this; } + @Override + public MultigetSubSliceQuery setKeys(Collection keys) { + this.keys = keys; + return this; + } + /** * Set the supercolumn to run the slice query on */ @@ -100,4 +106,11 @@ public MultigetSubSliceQuery setRange(N start, N finish, boolean re public MultigetSubSliceQuery setColumnNames(N... columnNames) { return (MultigetSubSliceQuery) super.setColumnNames(columnNames); } + + @SuppressWarnings("unchecked") + @Override + public MultigetSubSliceQuery setColumnNames(Collection columnNames) { + return (MultigetSubSliceQuery) super.setColumnNames(columnNames); + } + } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSuperSliceQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSuperSliceQuery.java index ff35e57f0..adbc7dbf6 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSuperSliceQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftMultigetSuperSliceQuery.java @@ -49,6 +49,12 @@ public MultigetSuperSliceQuery setKeys(K... keys) { return this; } + @Override + public MultigetSuperSliceQuery setKeys(Collection keys) { + this.keys = keys; + return this; + } + @Override public QueryResult> execute() { return new QueryResultImpl>( @@ -83,6 +89,12 @@ public MultigetSuperSliceQuery setColumnNames(SN... columnNames) { return (MultigetSuperSliceQuery) super.setColumnNames(columnNames); } + @SuppressWarnings("unchecked") + @Override + public MultigetSuperSliceQuery setColumnNames(Collection columnNames) { + return (MultigetSuperSliceQuery) super.setColumnNames(columnNames); + } + @SuppressWarnings("unchecked") @Override public MultigetSuperSliceQuery setColumnFamily(String cf) { diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftRangeSlicesQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftRangeSlicesQuery.java index 2392da939..eac855e3b 100644 --- a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftRangeSlicesQuery.java +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftRangeSlicesQuery.java @@ -100,8 +100,8 @@ public ThriftRangeSlicesQuery setReturnKeysOnly() { return this; } - - - + + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftSliceCounterQuery.java b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftSliceCounterQuery.java new file mode 100644 index 000000000..568778bce --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/model/thrift/ThriftSliceCounterQuery.java @@ -0,0 +1,81 @@ +package me.prettyprint.cassandra.model.thrift; + +import java.util.List; + +import me.prettyprint.cassandra.model.AbstractSliceQuery; +import me.prettyprint.cassandra.model.CounterSliceImpl; +import me.prettyprint.cassandra.model.KeyspaceOperationCallback; +import me.prettyprint.cassandra.model.QueryResultImpl; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.service.KeyspaceService; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.CounterSlice; +import me.prettyprint.hector.api.exceptions.HectorException; +import me.prettyprint.hector.api.query.QueryResult; +import me.prettyprint.hector.api.query.SliceCounterQuery; + +import org.apache.cassandra.thrift.ColumnParent; +import org.apache.cassandra.thrift.CounterColumn; + +/** + * A query for the thrift call get_slice + * + * @author patricioe (Patricio Echague) + * + * @param + */ +public final class ThriftSliceCounterQuery extends AbstractSliceQuery> + implements SliceCounterQuery { + + private K key; + + public ThriftSliceCounterQuery(Keyspace k, + Serializer keySerializer, + Serializer nameSerializer) { + // The reason of Longserializer is just to + super(k, keySerializer, nameSerializer, LongSerializer.get()); + } + + @Override + public SliceCounterQuery setKey(K key) { + this.key = key; + return this; + } + + @Override + public QueryResult> execute() { + return new QueryResultImpl>(keyspace.doExecute( + new KeyspaceOperationCallback>() { + @Override + public CounterSlice doInKeyspace(KeyspaceService ks) throws HectorException { + ColumnParent columnParent = new ColumnParent(columnFamilyName); + List thriftRet = ks.getCounterSlice(keySerializer.toByteBuffer(key), columnParent, getPredicate()); + return new CounterSliceImpl(thriftRet, columnNameSerializer); + } + }), this); + } + + @Override + public String toString() { + return "CounterSliceQuery(" + key + "," + toStringInternal() + ")"; + } + + @SuppressWarnings("unchecked") + @Override + public SliceCounterQuery setColumnNames(N... columnNames) { + return (SliceCounterQuery) super.setColumnNames(columnNames); + } + + @SuppressWarnings("unchecked") + @Override + public SliceCounterQuery setRange(N start, N finish, boolean reversed, int count) { + return (SliceCounterQuery) super.setRange(start, finish, reversed, count); + } + + @SuppressWarnings("unchecked") + @Override + public SliceCounterQuery setColumnFamily(String cf) { + return (SliceCounterQuery) super.setColumnFamily(cf); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/AbstractSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/AbstractSerializer.java index 49aa23114..a36fa70e4 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/AbstractSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/AbstractSerializer.java @@ -1,5 +1,7 @@ package me.prettyprint.cassandra.serializers; +import static me.prettyprint.hector.api.ddl.ComparatorType.BYTESTYPE; + import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.HashSet; @@ -10,14 +12,15 @@ import java.util.Set; import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.ddl.ComparatorType; /** * A base class for serializer implementations. Takes care of the default * implementations of to/fromBytesList and to/fromBytesMap. Extenders of this * class only need to implement the toBytes and fromBytes. - * + * * @author Ed Anuff - * + * * @param */ public abstract class AbstractSerializer implements Serializer { @@ -41,7 +44,7 @@ public T fromBytes(byte[] bytes) { /* * public ByteBuffer toByteBuffer(T obj) { return * ByteBuffer.wrap(toBytes(obj)); } - * + * * public ByteBuffer toByteBuffer(T obj, ByteBuffer byteBuffer, int offset, * int length) { byteBuffer.put(toBytes(obj), offset, length); return * byteBuffer; } @@ -53,7 +56,7 @@ public T fromBytes(byte[] bytes) { /* * public T fromByteBuffer(ByteBuffer byteBuffer) { return * fromBytes(byteBuffer.array()); } - * + * * public T fromByteBuffer(ByteBuffer byteBuffer, int offset, int length) { * return fromBytes(Arrays.copyOfRange(byteBuffer.array(), offset, length)); } */ @@ -118,4 +121,8 @@ public Map fromBytesMap(Map map) { public int computeInitialHashSize(int initialSize) { return Double.valueOf(Math.floor(initialSize / 0.75)).intValue() + 1; } + + public ComparatorType getComparatorType() { + return BYTESTYPE; + } } diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/AsciiSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/AsciiSerializer.java new file mode 100644 index 000000000..b167bc867 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/AsciiSerializer.java @@ -0,0 +1,47 @@ +package me.prettyprint.cassandra.serializers; + +import static me.prettyprint.hector.api.ddl.ComparatorType.ASCIITYPE; + +import java.nio.ByteBuffer; +import java.nio.charset.Charset; + +import me.prettyprint.hector.api.ddl.ComparatorType; + +/** + * Almost identical to StringSerializer except we use the US-ASCII character set + * code + * + * @author zznate + */ +public final class AsciiSerializer extends AbstractSerializer { + + private static final String US_ASCII = "US-ASCII"; + private static final AsciiSerializer instance = new AsciiSerializer(); + private static final Charset charset = Charset.forName(US_ASCII); + + public static AsciiSerializer get() { + return instance; + } + + @Override + public String fromByteBuffer(ByteBuffer byteBuffer) { + if (byteBuffer == null) { + return null; + } + return charset.decode(byteBuffer).toString(); + } + + @Override + public ByteBuffer toByteBuffer(String obj) { + if (obj == null) { + return null; + } + return ByteBuffer.wrap(obj.getBytes(charset)); + } + + @Override + public ComparatorType getComparatorType() { + return ASCIITYPE; + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/BigIntegerSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/BigIntegerSerializer.java new file mode 100644 index 000000000..b16a0008e --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/BigIntegerSerializer.java @@ -0,0 +1,47 @@ +package me.prettyprint.cassandra.serializers; + +import static me.prettyprint.hector.api.ddl.ComparatorType.INTEGERTYPE; + +import java.math.BigInteger; +import java.nio.ByteBuffer; + +import me.prettyprint.hector.api.ddl.ComparatorType; + +/** + * Serializer implementation for BigInteger + * + * @author zznate + */ +public final class BigIntegerSerializer extends AbstractSerializer { + + private static final BigIntegerSerializer INSTANCE = new BigIntegerSerializer(); + + public static BigIntegerSerializer get() { + return INSTANCE; + } + + @Override + public BigInteger fromByteBuffer(ByteBuffer byteBuffer) { + if (byteBuffer == null) { + return null; + } + int length = byteBuffer.remaining(); + byte[] bytes = new byte[length]; + byteBuffer.duplicate().get(bytes); + return new BigInteger(bytes); + } + + @Override + public ByteBuffer toByteBuffer(BigInteger obj) { + if (obj == null) { + return null; + } + return ByteBuffer.wrap(obj.toByteArray()); + } + + @Override + public ComparatorType getComparatorType() { + return INTEGERTYPE; + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/BooleanSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/BooleanSerializer.java index 56d1f16d1..9d85bcb5e 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/BooleanSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/BooleanSerializer.java @@ -4,9 +4,9 @@ /** * Converts bytes to Boolean and vice versa - * + * * @author Bozhidar Bozhanov - * + * */ public final class BooleanSerializer extends AbstractSerializer { diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/ByteBufferSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/ByteBufferSerializer.java index b138a078a..f76fd3fcf 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/ByteBufferSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/ByteBufferSerializer.java @@ -9,10 +9,12 @@ /** * The BytesExtractor is a simple identity function. It supports the Extractor * interface and implements the fromBytes and toBytes as simple identity - * functions. - * + * functions. However, the from and to methods both return the results of + * {@link ByteBuffer#duplicate()} + * + * * @author Ran Tavory - * + * @author zznate */ public final class ByteBufferSerializer extends AbstractSerializer{ @@ -27,9 +29,7 @@ public ByteBuffer fromByteBuffer(ByteBuffer bytes) { if(bytes==null) { return null; } - ByteBuffer b = bytes.slice(); - bytes.position(bytes.position() + b.remaining()); - return b; + return bytes.duplicate(); } @Override @@ -37,7 +37,7 @@ public ByteBuffer toByteBuffer(ByteBuffer obj) { if(obj==null) { return null; } - return obj.slice(); + return obj.duplicate(); } @Override diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/BytesArraySerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/BytesArraySerializer.java index a8bafb69b..72e8e8d4b 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/BytesArraySerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/BytesArraySerializer.java @@ -6,9 +6,9 @@ /** * A BytesArraySerializer translates the byte[] to and from ByteBuffer. - * + * * @author Patricio Echague - * + * */ public final class BytesArraySerializer extends AbstractSerializer implements Serializer { diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/CharSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/CharSerializer.java new file mode 100644 index 000000000..a466b353c --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/CharSerializer.java @@ -0,0 +1,37 @@ +package me.prettyprint.cassandra.serializers; + +import java.nio.ByteBuffer; + +/** + * Uses Char Serializer + * + * @author Todd Nine + */ +public class CharSerializer extends AbstractSerializer { + + private static final CharSerializer instance = new CharSerializer(); + + public static CharSerializer get() { + return instance; + } + + @Override + public ByteBuffer toByteBuffer(Character obj) { + ByteBuffer buffer = ByteBuffer.allocate(Character.SIZE / Byte.SIZE); + + buffer.putChar(obj); + buffer.rewind(); + + return buffer; + } + + @Override + public Character fromByteBuffer(ByteBuffer bytes) { + if (bytes == null) { + return null; + } + return bytes.getChar(); + + } + +} \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/CompositeSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/CompositeSerializer.java new file mode 100644 index 000000000..ae1a19cba --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/CompositeSerializer.java @@ -0,0 +1,40 @@ +/** + * + */ +package me.prettyprint.cassandra.serializers; + +import static me.prettyprint.hector.api.ddl.ComparatorType.COMPOSITETYPE; + +import java.nio.ByteBuffer; + +import me.prettyprint.hector.api.beans.Composite; +import me.prettyprint.hector.api.ddl.ComparatorType; + +/** + * @author Todd Nine + * + */ +public class CompositeSerializer extends AbstractSerializer { + + @Override + public ByteBuffer toByteBuffer(Composite obj) { + + return obj.serialize(); + } + + @Override + public Composite fromByteBuffer(ByteBuffer byteBuffer) { + + Composite composite = new Composite(); + composite.deserialize(byteBuffer); + + return composite; + + } + + @Override + public ComparatorType getComparatorType() { + return COMPOSITETYPE; + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/DoubleSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/DoubleSerializer.java index 0394805bd..5ed457fb3 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/DoubleSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/DoubleSerializer.java @@ -7,8 +7,8 @@ /** * Uses LongSerializer via translating Doubles to and from raw long bytes form. - * - * @author Yuri Finkelstein + * + * @author Yuri Finkelstein */ public class DoubleSerializer extends AbstractSerializer { @@ -17,7 +17,7 @@ public class DoubleSerializer extends AbstractSerializer { public static DoubleSerializer get() { return instance; } - + @Override public ByteBuffer toByteBuffer(Double obj) { return LongSerializer.get().toByteBuffer(Double.doubleToRawLongBits(obj)); diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/DynamicCompositeSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/DynamicCompositeSerializer.java new file mode 100644 index 000000000..9447dabc7 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/DynamicCompositeSerializer.java @@ -0,0 +1,38 @@ +/** + * + */ +package me.prettyprint.cassandra.serializers; + +import static me.prettyprint.hector.api.ddl.ComparatorType.DYNAMICCOMPOSITETYPE; + +import java.nio.ByteBuffer; + +import me.prettyprint.hector.api.beans.DynamicComposite; +import me.prettyprint.hector.api.ddl.ComparatorType; + +/** + * @author Todd Nine + * + */ +public class DynamicCompositeSerializer extends + AbstractSerializer { + + @Override + public ByteBuffer toByteBuffer(DynamicComposite obj) { + + return obj.serialize(); + } + + @Override + public DynamicComposite fromByteBuffer(ByteBuffer byteBuffer) { + + return DynamicComposite.fromByteBuffer(byteBuffer); + + } + + @Override + public ComparatorType getComparatorType() { + return DYNAMICCOMPOSITETYPE; + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/FastInfosetSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/FastInfosetSerializer.java index 0b11e9a89..452e357a4 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/FastInfosetSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/FastInfosetSerializer.java @@ -16,14 +16,14 @@ * size as well as parse and serialization time. An instance of this class may * only serialize JAXB compatible objects of classes known to its configured * context. - * + * * @author shuzhang0@gmail.com - * + * */ public class FastInfosetSerializer extends JaxbSerializer { /** * Constructor. - * + * * @param serializableClasses * List of classes which can be serialized by this instance. Note * that concrete classes directly referenced by any class in the list diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/FloatSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/FloatSerializer.java new file mode 100644 index 000000000..3435f881f --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/FloatSerializer.java @@ -0,0 +1,28 @@ +package me.prettyprint.cassandra.serializers; + +import java.nio.ByteBuffer; + +/** + * Uses IntSerializer via translating Float objects to and from raw long bytes form. + * + * @author Todd Nine + */ +public class FloatSerializer extends AbstractSerializer { + + private static final FloatSerializer instance = new FloatSerializer(); + + public static FloatSerializer get() { + return instance; + } + + @Override + public ByteBuffer toByteBuffer(Float obj) { + return IntegerSerializer.get().toByteBuffer(Float.floatToRawIntBits(obj)); + } + + @Override + public Float fromByteBuffer(ByteBuffer bytes) { + return Float.intBitsToFloat(IntegerSerializer.get().fromByteBuffer(bytes)); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/IntegerSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/IntegerSerializer.java index a2000b5c5..9dce79402 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/IntegerSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/IntegerSerializer.java @@ -4,9 +4,9 @@ /** * Converts bytes to Integer and vice versa - * + * * @author Bozhidar Bozhanov - * + * */ public final class IntegerSerializer extends AbstractSerializer { @@ -35,7 +35,7 @@ public Integer fromByteBuffer(ByteBuffer byteBuffer) { int in = byteBuffer.getInt(); return in; } - + @Override public Integer fromBytes(byte[] bytes) { ByteBuffer bb = ByteBuffer.allocate(4).put(bytes, 0, 4); diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/JaxbSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/JaxbSerializer.java index a376a759b..8e4053ae0 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/JaxbSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/JaxbSerializer.java @@ -22,7 +22,7 @@ /** * Serializes Objects using Jaxb. An instance of this class may only serialize * JAXB compatible objects of classes known to its configured context. - * + * * @author shuzhang0@gmail.com */ public class JaxbSerializer extends AbstractSerializer { @@ -45,7 +45,7 @@ public class JaxbSerializer extends AbstractSerializer { /** * Constructor. - * + * * @param serializableClasses * List of classes which can be serialized by this instance. Note * that concrete classes directly referenced by any class in the list @@ -126,7 +126,7 @@ public Object fromByteBuffer(ByteBuffer bytes) { /** * Get a new XML stream writer. - * + * * @param output * An underlying OutputStream to write to. * @return a new {@link XMLStreamWriter} which writes to the specified @@ -147,7 +147,7 @@ protected XMLStreamWriter createStreamWriter(OutputStream output) /** * Get a new XML stream reader. - * + * * @param input * the underlying InputStream to read from. * @return a new {@link XmlStreamReader} which reads from the specified diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/LongSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/LongSerializer.java index 4c6686cfd..ff2974bf5 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/LongSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/LongSerializer.java @@ -1,12 +1,16 @@ package me.prettyprint.cassandra.serializers; +import static me.prettyprint.hector.api.ddl.ComparatorType.LONGTYPE; + import java.nio.ByteBuffer; +import me.prettyprint.hector.api.ddl.ComparatorType; + /** * Converts bytes to Long and vise a versa - * + * * @author Ran Tavory - * + * */ public final class LongSerializer extends AbstractSerializer { @@ -21,7 +25,7 @@ public ByteBuffer toByteBuffer(Long obj) { if (obj == null) { return null; } - return ByteBuffer.allocate(8).putLong(0, obj); + return ByteBuffer.allocate(8).putLong(0, obj); } @Override @@ -33,4 +37,9 @@ public Long fromByteBuffer(ByteBuffer byteBuffer) { return l; } + @Override + public ComparatorType getComparatorType() { + return LONGTYPE; + } + } diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/ObjectSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/ObjectSerializer.java index a3622b99b..dd651fd14 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/ObjectSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/ObjectSerializer.java @@ -13,9 +13,9 @@ /** * The ObjectSerializer is used to turn objects into their binary * representations. - * + * * @author Bozhidar Bozhanov - * + * */ public class ObjectSerializer extends AbstractSerializer implements Serializer { diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/SerializerTypeInferer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/SerializerTypeInferer.java index 88cad9fcc..b2ab8e078 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/SerializerTypeInferer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/SerializerTypeInferer.java @@ -1,5 +1,6 @@ package me.prettyprint.cassandra.serializers; +import java.math.BigInteger; import java.util.Date; import java.util.UUID; @@ -8,9 +9,9 @@ /** * Utility class that infers the concrete Serializer needed to turn a value into * its binary representation - * + * * @author Bozhidar Bozhanov - * + * */ public class SerializerTypeInferer { @@ -27,6 +28,8 @@ public static Serializer getSerializer(Object value) { serializer = LongSerializer.get(); } else if (value instanceof Integer) { serializer = IntegerSerializer.get(); + } else if (value instanceof BigInteger) { + serializer = BigIntegerSerializer.get(); } else if (value instanceof Boolean) { serializer = BooleanSerializer.get(); } else if (value instanceof byte[]) { @@ -52,7 +55,8 @@ public static Serializer getSerializer(Class valueClass) { serializer = LongSerializer.get(); } else if (valueClass.equals(Integer.class) || valueClass.equals(int.class)) { serializer = IntegerSerializer.get(); - } else if (valueClass.equals(Boolean.class) || valueClass.equals(boolean.class)) { + } else if (valueClass.equals(Boolean.class) + || valueClass.equals(boolean.class)) { serializer = BooleanSerializer.get(); } else if (valueClass.equals(byte[].class)) { serializer = ByteBufferSerializer.get(); diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/ShortSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/ShortSerializer.java index d77e9ac70..4a0ffdaf7 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/ShortSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/ShortSerializer.java @@ -4,7 +4,7 @@ /** * {@link Serializer} for {@link Short}s (no pun intended). - * + * */ public final class ShortSerializer extends AbstractSerializer { diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/StringSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/StringSerializer.java index 22e4786f8..772a3cfbb 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/StringSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/StringSerializer.java @@ -1,15 +1,18 @@ package me.prettyprint.cassandra.serializers; -import java.io.UnsupportedEncodingException; +import static me.prettyprint.hector.api.ddl.ComparatorType.UTF8TYPE; + import java.nio.ByteBuffer; import java.nio.charset.Charset; +import me.prettyprint.hector.api.ddl.ComparatorType; + /** * A StringSerializer translates the byte[] to and from string using utf-8 * encoding. - * + * * @author Ran Tavory - * + * */ public final class StringSerializer extends AbstractSerializer { @@ -25,15 +28,21 @@ public static StringSerializer get() { public ByteBuffer toByteBuffer(String obj) { if (obj == null) { return null; - } - return ByteBuffer.wrap(obj.getBytes(charset)); + } + return ByteBuffer.wrap(obj.getBytes(charset)); } @Override public String fromByteBuffer(ByteBuffer byteBuffer) { if (byteBuffer == null) { return null; - } - return charset.decode(byteBuffer).toString(); + } + return charset.decode(byteBuffer).toString(); + } + + @Override + public ComparatorType getComparatorType() { + return UTF8TYPE; } + } diff --git a/core/src/main/java/me/prettyprint/cassandra/serializers/UUIDSerializer.java b/core/src/main/java/me/prettyprint/cassandra/serializers/UUIDSerializer.java index f4309f867..c7569a715 100644 --- a/core/src/main/java/me/prettyprint/cassandra/serializers/UUIDSerializer.java +++ b/core/src/main/java/me/prettyprint/cassandra/serializers/UUIDSerializer.java @@ -1,10 +1,15 @@ package me.prettyprint.cassandra.serializers; +import static me.prettyprint.hector.api.ddl.ComparatorType.UUIDTYPE; + import java.nio.ByteBuffer; import java.util.UUID; +import me.prettyprint.hector.api.ddl.ComparatorType; + /** * A UUIDSerializer translates the byte[] to and from UUID types. + * * @author Ed Anuff * */ @@ -16,6 +21,7 @@ public static UUIDSerializer get() { return instance; } + @Override public ByteBuffer toByteBuffer(UUID uuid) { if (uuid == null) { return null; @@ -34,11 +40,17 @@ public ByteBuffer toByteBuffer(UUID uuid) { return ByteBuffer.wrap(buffer); } + @Override public UUID fromByteBuffer(ByteBuffer bytes) { if (bytes == null) { return null; - } + } return new UUID(bytes.getLong(), bytes.getLong()); } + @Override + public ComparatorType getComparatorType() { + return UUIDTYPE; + } + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/AbstractCluster.java b/core/src/main/java/me/prettyprint/cassandra/service/AbstractCluster.java index 6b90fdc0c..b2d9c0548 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/AbstractCluster.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/AbstractCluster.java @@ -272,13 +272,13 @@ public Void execute(Cassandra.Client cassandra) throws HectorException { throw xtrans.translate(e); } return null; - } + } }; connectionManager.operateWithFailover(op); } - - - + + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/BatchMutation.java b/core/src/main/java/me/prettyprint/cassandra/service/BatchMutation.java index eebdbd324..fca0fd370 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/BatchMutation.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/BatchMutation.java @@ -11,6 +11,8 @@ import org.apache.cassandra.thrift.Column; import org.apache.cassandra.thrift.ColumnOrSuperColumn; +import org.apache.cassandra.thrift.CounterColumn; +import org.apache.cassandra.thrift.CounterSuperColumn; import org.apache.cassandra.thrift.Deletion; import org.apache.cassandra.thrift.Mutation; import org.apache.cassandra.thrift.SuperColumn; @@ -24,6 +26,7 @@ * * @author Ran Tavory (rantan@outbrain.com) * @author Nathan McCall (nate@riptano.com) + * @author Patricio Echague (patricioe@gmail.com) * */ public final class BatchMutation { @@ -52,9 +55,8 @@ public BatchMutation addInsertion(K key, List columnFamilies, return this; } - /** - * Add an SuperColumn insertion (or update) to the batch mutation request. + * Add a SuperColumn insertion (or update) to the batch mutation request. */ public BatchMutation addSuperInsertion(K key, List columnFamilies, SuperColumn superColumn) { @@ -64,6 +66,27 @@ public BatchMutation addSuperInsertion(K key, List columnFamilies, return this; } + /** + * Add a ColumnCounter insertion (or update) + */ + public BatchMutation addCounterInsertion(K key, List columnFamilies, CounterColumn counterColumn) { + Mutation mutation = new Mutation(); + mutation.setColumn_or_supercolumn(new ColumnOrSuperColumn().setCounter_column(counterColumn)); + addMutation(key, columnFamilies, mutation); + return this; + } + + /** + * Add a SuperColumnCounter insertion (or update) + */ + public BatchMutation addSuperCounterInsertion(K key, List columnFamilies, + CounterSuperColumn counterSuperColumn) { + Mutation mutation = new Mutation(); + mutation.setColumn_or_supercolumn(new ColumnOrSuperColumn().setCounter_super_column(counterSuperColumn)); + addMutation(key, columnFamilies, mutation); + return this; + } + /** * Add a deletion request to the batch mutation. */ @@ -74,6 +97,7 @@ public BatchMutation addDeletion(K key, List columnFamilies, Deletion return this; } + private void addMutation(K key, List columnFamilies, Mutation mutation) { Map> innerMutationMap = getInnerMutationMap(key); for (String columnFamily : columnFamilies) { @@ -88,6 +112,8 @@ private void addMutation(K key, List columnFamilies, Mutation mutation) mutationMap.put(keySerializer.toByteBuffer(key), innerMutationMap); } + + private Map> getInnerMutationMap(K key) { Map> innerMutationMap = mutationMap.get(keySerializer.toByteBuffer(key)); if (innerMutationMap == null) { @@ -100,7 +126,6 @@ Map>> getMutationMap() { return mutationMap; } - /** * Makes a shallow copy of the mutation object. * @return @@ -114,6 +139,6 @@ public BatchMutation makeCopy() { * @return */ public boolean isEmpty() { - return mutationMap.isEmpty(); + return mutationMap.isEmpty() ; } } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitor.java b/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitor.java index 427bc4c7e..2e78bb214 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitor.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitor.java @@ -110,7 +110,9 @@ public long getNumPoolExhaustedEventCount() { @Override public Set getExhaustedPoolNames() { Set ret = new HashSet(); - + for ( CassandraHost host : connectionManager.getDownedHosts() ) { + ret.add(host.toString()); + } return ret; } @@ -201,12 +203,12 @@ public long getNumConnectionErrors() { } @Override - public boolean addCassandraHost(String hostStr) { + public boolean addCassandraHost(String hostStr) { return connectionManager.addCassandraHost(new CassandraHost(hostStr)); } @Override - public boolean removeCassandraHost(String hostStr) { + public boolean removeCassandraHost(String hostStr) { return connectionManager.removeCassandraHost(new CassandraHost(hostStr)); } @@ -215,13 +217,13 @@ public Set getSuspendedCassandraHosts() { Set hosts = connectionManager.getSuspendedCassandraHosts(); Set hostsStr = new HashSet(); for (CassandraHost host : hosts) { - hostsStr.add(host.getName()); + hostsStr.add(host.getName()); } return hostsStr; } @Override - public boolean suspendCassandraHost(String hostStr) { + public boolean suspendCassandraHost(String hostStr) { return connectionManager.suspendCassandraHost(new CassandraHost(hostStr)); } @@ -229,9 +231,9 @@ public boolean suspendCassandraHost(String hostStr) { public boolean unsuspendCassandraHost(String hostStr) { return connectionManager.unsuspendCassandraHost(new CassandraHost(hostStr)); } - - - - + + + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitorMBean.java b/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitorMBean.java index d537b6296..298192256 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitorMBean.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/CassandraClientMonitorMBean.java @@ -118,38 +118,38 @@ public interface CassandraClientMonitorMBean { * */ public List getStatisticsPerPool(); - + /** * Add a host in the format of "[hostname]:[port]" - * + * * @param hostStr * @return */ boolean addCassandraHost(String hostStr); - + /** * Remove a host in the format of "[hostname]:[port]" * @see {@link CassandraHost#equals(Object)} for how hosts are compared - * + * * @param hostStr * @return */ boolean removeCassandraHost(String hostStr); - + /** - * @see {@link #removeCassandraHost(String)} above for semantics of the host string. + * @see {@link #removeCassandraHost(String)} above for semantics of the host string. * @see {@link HConnectionManager#removeCassandraHost(CassandraHost)} for details of this operation. * @param hostStr * @return */ boolean suspendCassandraHost(String hostStr); - + /** * @see {@link #suspendCassandraHost(String)} above. This is the opposite. * @param hostStr * @return */ boolean unsuspendCassandraHost(String hostStr); - - Set getSuspendedCassandraHosts(); + + Set getSuspendedCassandraHosts(); } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/CassandraHost.java b/core/src/main/java/me/prettyprint/cassandra/service/CassandraHost.java index 93bd0c22d..845d84892 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/CassandraHost.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/CassandraHost.java @@ -59,6 +59,7 @@ public final class CassandraHost { private int cassandraThriftSocketTimeout; private ExhaustedPolicy exhaustedPolicy = ExhaustedPolicy.WHEN_EXHAUSTED_BLOCK; private boolean useThriftFramedTransport = DEFAULT_USE_FRAMED_THRIFT_TRANSPORT; + private boolean useSocketKeepalive; //TODO(ran): private FailoverPolicy failoverPolicy = DEFAULT_FAILOVER_POLICY; public CassandraHost(String url) { @@ -225,4 +226,13 @@ public void setTimeBetweenEvictionRunsMillis(long timeBetweenEvictionRunsMillis) this.timeBetweenEvictionRunsMillis = timeBetweenEvictionRunsMillis; } + public boolean getUseSocketKeepalive() { + return useSocketKeepalive; + } + + public void setUseSocketKeepalive(boolean useSocketKeepalive) { + this.useSocketKeepalive = useSocketKeepalive; + } + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/CassandraHostConfigurator.java b/core/src/main/java/me/prettyprint/cassandra/service/CassandraHostConfigurator.java index 6f41223ad..b2deda483 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/CassandraHostConfigurator.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/CassandraHostConfigurator.java @@ -40,7 +40,7 @@ public final class CassandraHostConfigurator implements Serializable { private int hostTimeoutUnsuspendCheckDelay = HostTimeoutTracker.DEF_NODE_UNSUSPEND_CHECK_DELAY_IN_SECONDS; private boolean useHostTimeoutTracker = false; private boolean runAutoDiscoveryAtStartup = false; - + private boolean useSocketKeepalive = false; public CassandraHostConfigurator() { this.hosts = null; @@ -73,6 +73,7 @@ public void applyConfig(CassandraHost cassandraHost) { cassandraHost.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis); cassandraHost.setMaxWaitTimeWhenExhausted(maxWaitTimeWhenExhausted); cassandraHost.setUseThriftFramedTransport(useThriftFramedTransport); + cassandraHost.setUseSocketKeepalive(useSocketKeepalive); // this is special as it can be passed in as a system property if (cassandraThriftSocketTimeout > 0) { @@ -290,6 +291,18 @@ public void setRunAutoDiscoveryAtStartup(boolean runAutoDiscoveryAtStartup) { this.runAutoDiscoveryAtStartup = runAutoDiscoveryAtStartup; } - + public boolean getUseSocketKeepalive() { + return useSocketKeepalive; + } + + /** + * Enable SO_KEEPALIVE on the underlying socket. OFF by default (per java.net.Socket) + * + */ + public void setUseSocketKeepalive(boolean useSocketKeepalive) { + this.useSocketKeepalive = useSocketKeepalive; + } + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslator.java b/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslator.java index 377a95d39..99ae27794 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslator.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslator.java @@ -4,12 +4,12 @@ /** * Translates exceptions throw by thrift or pool to HectorException instances. - * + * * @author Ran Tavory (ran@outbrain.com) * */ public interface ExceptionsTranslator { - + HectorException translate(Throwable originalException); - + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslatorImpl.java b/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslatorImpl.java index 21f32cd20..19fdd875a 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslatorImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ExceptionsTranslatorImpl.java @@ -1,5 +1,6 @@ package me.prettyprint.cassandra.service; +import java.net.SocketTimeoutException; import java.util.NoSuchElementException; import me.prettyprint.hector.api.exceptions.HCassandraInternalException; @@ -26,8 +27,15 @@ public HectorException translate(Throwable original) { return (HectorException) original; } else if (original instanceof TApplicationException) { return new HCassandraInternalException(((TApplicationException)original).getType(), original.getMessage()); - } else if (original instanceof TException || original instanceof TTransportException) { - return new HectorTransportException(original); + } else if (original instanceof TTransportException) { + // if the underlying cause is a scoket timeout, reflect that directly + // TODO this may be an issue on the Cassandra side which warrants ivestigation. + // I seem to remember these coming back as TimedOutException previously + if ( ((TTransportException)original).getCause() instanceof SocketTimeoutException ) { + return new HTimedOutException(original); + } else { + return new HectorTransportException(original); + } } else if (original instanceof org.apache.cassandra.thrift.TimedOutException) { return new HTimedOutException(original); } else if (original instanceof org.apache.cassandra.thrift.InvalidRequestException) { @@ -50,6 +58,8 @@ public HectorException translate(Throwable original) { return new HNotFoundException(original); } else if (original instanceof org.apache.cassandra.thrift.UnavailableException) { return new HUnavailableException(original); + } else if (original instanceof TException) { + return new HectorTransportException(original); } else if (original instanceof NoSuchElementException) { return new PoolExhaustedException(original); } else if (original instanceof IllegalStateException) { diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ExhaustedPolicy.java b/core/src/main/java/me/prettyprint/cassandra/service/ExhaustedPolicy.java index a8463850a..72a3a5239 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ExhaustedPolicy.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ExhaustedPolicy.java @@ -11,11 +11,11 @@ public enum ExhaustedPolicy { /** * If the pool is full, fail with the exception {@link PoolExhaustedException} */ - WHEN_EXHAUSTED_FAIL, + WHEN_EXHAUSTED_FAIL, /** * When pool exhausted, grow. */ - WHEN_EXHAUSTED_GROW, + WHEN_EXHAUSTED_GROW, /** * Block the requesting thread when the pool is exhausted until new connections are available. */ diff --git a/core/src/main/java/me/prettyprint/cassandra/service/HColumnFamilyImpl.java b/core/src/main/java/me/prettyprint/cassandra/service/HColumnFamilyImpl.java index ddcc0a4d8..8b332bf66 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/HColumnFamilyImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/HColumnFamilyImpl.java @@ -43,7 +43,7 @@ public class HColumnFamilyImpl implements HColumnFamily { private final Logger queryLogger = LoggerFactory.getLogger("HColumnFamilyLogger"); private final Logger log = LoggerFactory.getLogger(HColumnFamily.class); - + private final ExecutingKeyspace keyspace; private final String columnFamilyName; private List _keys; @@ -62,7 +62,7 @@ public class HColumnFamilyImpl implements HColumnFamily { private Map> rows; private CassandraHost lastHostUsed; private long lastExecutionTime; - + public HColumnFamilyImpl(Keyspace keyspace, String columnFamilyName, Serializer keySerializer, Serializer columnNameSerializer) { this.keyspace = (ExecutingKeyspace)keyspace; @@ -75,7 +75,7 @@ public HColumnFamilyImpl(Keyspace keyspace, String columnFamilyName, Serializer< exceptionsTranslator = new ExceptionsTranslatorImpl(); this.consistencyLevelPolicy = new ConfigurableConsistencyLevel(); } - + @Override public HColumnFamily addKey(K key) { _keys.add(key); @@ -86,9 +86,9 @@ public HColumnFamily addKey(K key) { public HColumnFamily addKeys(Collection keys) { _keys.addAll(keys); return this; - } - - + } + + @Override public HColumnFamily removeKeys() { @@ -118,7 +118,7 @@ public HColumnFamily setReversed(boolean reversed) { public HColumnFamily setStart(N name) { activeSlicePredicate.setStartOn(name); return this; - } + } @Override public HColumnFamily addColumnName(N columnName) { @@ -141,16 +141,16 @@ public HColumnFamily setColumnNames(Collection columnNames) { @Override public Collection> getColumns() { if ( columns == null ) - columns = new HashMap>(); - + columns = new HashMap>(); + if ( !hasValues ) doExecuteSlice(); - + return columns.values(); } - - + + @Override public HColumnFamily clear() { for (HColumn col : columns.values() ) { @@ -172,7 +172,7 @@ public double getDouble(N name) { } @Override - public int getInt(N name) { + public int getInt(N name) { return extractColumnValue(name, IntegerSerializer.get()); } @@ -182,7 +182,7 @@ public long getLong(N name) { } @Override - public String getString(N name) { + public String getString(N name) { return extractColumnValue(name, StringSerializer.get()); } @@ -201,7 +201,7 @@ public HColumnFamily next() { applyToRow(key, rows.get(keySerializer.toByteBuffer(key))); return this; } - + @Override public boolean hasNext() { return rowIndex < rows.size() - 1 ; @@ -217,7 +217,7 @@ public void remove() { * Extract a value for the specified name and serializer */ @Override - public V getValue(N name, Serializer valueSerializer) { + public V getValue(N name, Serializer valueSerializer) { return extractColumnValue(name, valueSerializer); } @@ -233,11 +233,11 @@ public HColumnFamily setWriteConsistencyLevel(HConsistencyLevel writeLevel return this; } - - + + @Override - public long getExecutionTimeMicro() { - return lastExecutionTime; + public long getExecutionTimeMicro() { + return lastExecutionTime / 1000; } @Override @@ -246,10 +246,10 @@ public CassandraHost getHostUsed() { } private V extractColumnValue(N columnName, Serializer valueSerializer) { - maybeExecuteSlice(columnName); - return columns.get(columnName) != null && columns.get(columnName).getValue() != null ? valueSerializer.fromByteBuffer(columns.get(columnName).getValue()) : null; + maybeExecuteSlice(columnName); + return columns.get(columnName) != null && columns.get(columnName).getValue() != null ? valueSerializer.fromByteBuffer(columns.get(columnName).getValue()) : null; } - + private void maybeExecuteSlice(N columnName) { if ( columnNames == null ) { columnNames = new HashSet(); @@ -264,48 +264,48 @@ private void maybeExecuteSlice(N columnName) { doExecuteSlice(); } else { doExecuteMultigetSlice(); - } - } + } + } } - - + + private void applyToRow(K key, List cosclist) { HColumn column; N colName; for (Iterator iterator = cosclist.iterator(); iterator.hasNext();) { - ColumnOrSuperColumn cosc = iterator.next(); + ColumnOrSuperColumn cosc = iterator.next(); colName = columnNameSerializer.fromByteBuffer(cosc.getColumn().name.duplicate()); column = columns.get(colName); - + if ( column == null ) { column = new HColumnImpl(cosc.getColumn(), columnNameSerializer, ByteBufferSerializer.get()); } else { ((HColumnImpl)column).apply(cosc.getColumn()); } - columns.put(colName, column); + columns.put(colName, column); iterator.remove(); } } - + private void applyResultStatus(long execTime, CassandraHost cassandraHost) { lastExecutionTime = execTime; lastHostUsed = cassandraHost; } - + private void doExecuteSlice() { keyspace.doExecuteOperation(new Operation(OperationType.READ) { @Override public Column execute(Cassandra.Client cassandra) throws HectorException { - - try { + + try { if ( queryLogger.isDebugEnabled() ) { queryLogger.debug("---------\nColumnFamily: {} slicePredicate: {}", columnFamilyName, activeSlicePredicate.toString()); } K key = _keys.iterator().next(); List cosclist = cassandra.get_slice(keySerializer.toByteBuffer(key), columnParent, - activeSlicePredicate.toThrift(), + activeSlicePredicate.toThrift(), ThriftConverter.consistencyLevel(consistencyLevelPolicy.get(operationType))); applyResultStatus(execTime, getCassandraHost()); applyToRow(key, cosclist); @@ -321,22 +321,22 @@ public Column execute(Cassandra.Client cassandra) throws HectorException { } }); } - + private void doExecuteMultigetSlice() { keyspace.doExecuteOperation(new Operation(OperationType.READ) { @Override public Column execute(Cassandra.Client cassandra) throws HectorException { - try { + try { if ( queryLogger.isDebugEnabled() ) { queryLogger.debug("---------\nColumnFamily multiget: {} slicePredicate: {}", columnFamilyName, activeSlicePredicate.toString()); } - rows = cassandra.multiget_slice(keySerializer.toBytesList(_keys), columnParent, activeSlicePredicate.toThrift(), + rows = cassandra.multiget_slice(keySerializer.toBytesList(_keys), columnParent, activeSlicePredicate.toThrift(), ThriftConverter.consistencyLevel(consistencyLevelPolicy.get(operationType))); applyResultStatus(execTime, getCassandraHost()); - + if ( queryLogger.isDebugEnabled() ) { queryLogger.debug("Execution took {} microseconds on host {}\n----------", lastExecutionTime, lastHostUsed); } @@ -351,4 +351,9 @@ public Column execute(Cassandra.Client cassandra) throws HectorException { applyToRow(_keys.get(0), rows.get(keySerializer.toByteBuffer(_keys.get(0)))); } + @Override + public long getExecutionTimeNano() { + return lastExecutionTime; + } + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/JmxMonitor.java b/core/src/main/java/me/prettyprint/cassandra/service/JmxMonitor.java index 223bed6d0..08d9b5414 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/JmxMonitor.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/JmxMonitor.java @@ -35,7 +35,7 @@ public class JmxMonitor { private JmxMonitor() { mbs = ManagementFactory.getPlatformMBeanServer(); - monitors = new HashMap(); + monitors = new HashMap(); } public static JmxMonitor getInstance() { @@ -62,37 +62,6 @@ public void registerMonitor(String name, String monitorType, Object monitoringIn } mbs.registerMBean(monitoringInterface, oName); - - // Register perf4j monitors - if ("true".equalsIgnoreCase( - System.getProperty("com.prettyprint.cassandra.load_hector_log4j", "true"))) { - registerPerf4J(); - } - } - - private void registerPerf4J() { - URL url = getClass().getClassLoader().getResource("hectorLog4j.xml"); - if (url == null) { - log.warn("Unable to locate hectorLog4j.xml; performance counters will not be exported"); - } else { - try { - final Class domConfiguratorClass = getClass().getClassLoader().loadClass("org.apache.log4j.xml.DOMConfigurator"); - final Method method = domConfiguratorClass.getMethod( "configure", URL.class ); - method.invoke( null, url ); - } catch( ClassNotFoundException e ) { - log.warn("Unable to load log4j's DOMConfigurator. Performance counters will not be exported. To fix, include the log4j jar in your application's classpath."); - } catch( SecurityException e ) { - log.error( "Could not access method DOMConfigurator.configure(URL)", e ); - } catch( NoSuchMethodException e ) { - log.error( "Could not find method DOMConfigurator.configure(URL)", e ); - } catch( IllegalArgumentException e ) { - log.error( "Could not invoke method DOMConfigurator.configure(URL)", e ); - } catch( IllegalAccessException e ) { - log.error( "Could not invoke method DOMConfigurator.configure(URL)", e ); - } catch( InvocationTargetException e ) { - throw new RuntimeException(e.getCause()); - } - } } private String generateMonitorName(String className, String monitorType) { diff --git a/core/src/main/java/me/prettyprint/cassandra/service/KeyIterator.java b/core/src/main/java/me/prettyprint/cassandra/service/KeyIterator.java new file mode 100644 index 000000000..09a055418 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/KeyIterator.java @@ -0,0 +1,106 @@ +package me.prettyprint.cassandra.service; + +import java.util.Iterator; + +import me.prettyprint.cassandra.serializers.AbstractSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.beans.OrderedRows; +import me.prettyprint.hector.api.beans.Row; +import me.prettyprint.hector.api.factory.HFactory; +import me.prettyprint.hector.api.query.QueryResult; +import me.prettyprint.hector.api.query.RangeSlicesQuery; + + +/** + * This class returns each key in the specified Column Family as an Iterator. You + * can use this class in a for loop without the overhead of first storing each + * key in a large array. See StringKeyIterator for a convenience class if the key + * is a String. + * @author Tim Koop + * @param the type of the row key + * @see StringKeyIterator + */ +public class KeyIterator implements Iterable { + private static StringSerializer stringSerializer = new StringSerializer(); + + private int maxRowCount = 500; + private int maxColumnCount = 2; // we only need this to tell if there are any columns in the row (to test for tombstones) + + private Iterator> rowsIterator = null; + + private RangeSlicesQuery query = null; + + private K nextValue = null; + private K lastReadValue = null; + + private Iterator keyIterator = new Iterator() { + @Override + public boolean hasNext() { + return nextValue != null; + } + + @Override + public K next() { + K next = nextValue; + findNext(false); + return next; + } + + @Override + public void remove() { + throw new UnsupportedOperationException(); + } + }; + + private void findNext(boolean fromRunQuery) { + nextValue = null; + if (rowsIterator == null) { + return; + } + while (rowsIterator.hasNext() && nextValue == null) { + Row row = rowsIterator.next(); + lastReadValue = row.getKey(); + if (!row.getColumnSlice().getColumns().isEmpty()) { + nextValue = lastReadValue; + } + } + if (!rowsIterator.hasNext() && nextValue == null) { + runQuery(lastReadValue); + } + } + + public KeyIterator(Keyspace keyspace, String columnFamily, AbstractSerializer serializer) { + query = HFactory + .createRangeSlicesQuery(keyspace, serializer, stringSerializer, stringSerializer) + .setColumnFamily(columnFamily) + .setRange(null, null, false, maxColumnCount) + .setRowCount(maxRowCount); + + runQuery(null); + } + + private void runQuery(K start) { + query.setKeys(start, null); + + rowsIterator = null; + QueryResult> result = query.execute(); + OrderedRows rows = (result != null) ? result.get() : null; + rowsIterator = (rows != null) ? rows.iterator() : null; + + // we'll skip this first one, since it is the same as the last one from previous time we executed + if (start != null && rowsIterator != null) rowsIterator.next(); + + if (!rowsIterator.hasNext()) { + nextValue = null; // all done. our iterator's hasNext() will now return false; + } else { + findNext(true); + } + } + + @Override + public Iterator iterator() { + return keyIterator; + } +} + diff --git a/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceService.java b/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceService.java index bb7596e9c..c3e00a10d 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceService.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceService.java @@ -11,6 +11,7 @@ import org.apache.cassandra.thrift.Column; import org.apache.cassandra.thrift.ColumnParent; import org.apache.cassandra.thrift.ColumnPath; +import org.apache.cassandra.thrift.CounterColumn; import org.apache.cassandra.thrift.IndexClause; import org.apache.cassandra.thrift.KeyRange; import org.apache.cassandra.thrift.Mutation; @@ -47,6 +48,18 @@ public interface KeyspaceService { Column getColumn(String key, ColumnPath columnPath) throws HectorException; + /** + * Get the Counter at the given columnPath. + * + * If no value is present, NotFoundException is thrown. + * + * @throws HNotFoundException + * if no value exists for the counter + */ + CounterColumn getCounter(ByteBuffer key, ColumnPath columnPath) throws HectorException; + + CounterColumn getCounter(String key, ColumnPath columnPath) throws HectorException; + /** * Get the SuperColumn at the given columnPath. * @@ -91,7 +104,21 @@ List getSlice(ByteBuffer key, ColumnParent columnParent, SlicePredicate throws HectorException; List getSlice(String key, ColumnParent columnParent, SlicePredicate predicate) - throws HectorException; + throws HectorException; + + /** + * Get the group of counter columns contained by columnParent. + * + * Returns Either a ColumnFamily name or a ColumnFamily/SuperColumn specified + * by the given predicate. If no matching values are found, an empty list is + * returned. + */ + List getCounterSlice(ByteBuffer key, ColumnParent columnParent, SlicePredicate predicate) + throws HectorException; + + public List getCounterSlice(String key, ColumnParent columnParent, SlicePredicate predicate) + throws HectorException; + /** * Get the group of superColumn contained by columnParent. @@ -147,6 +174,16 @@ Map> multigetSuperSlice(List keys, void insert(String key, ColumnPath columnPath, ByteBuffer value, long timestamp) throws HectorException; + /** + * Add a counter with CL.ONE + */ + void addCounter(ByteBuffer key, ColumnParent columnParent, CounterColumn counterColumn) throws HectorException; + + /** + * Add a counter with CL.ONE + */ + void addCounter(String key, ColumnParent columnParent, CounterColumn counterColumn) throws HectorException; + /** * Call batch mutate with the assembled mutationMap. This method is a direct pass-through * to the underlying Thrift API @@ -170,6 +207,10 @@ Map> multigetSuperSlice(List keys, void remove(String key, ColumnPath columnPath, long timestamp) throws HectorException; + void removeCounter(ByteBuffer key, ColumnPath columnPath) throws HectorException; + + void removeCounter(String key, ColumnPath columnPath) throws HectorException; + /** * Counts the columns present in columnParent. diff --git a/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceServiceImpl.java b/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceServiceImpl.java index 601d487dc..72cb9e158 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceServiceImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/KeyspaceServiceImpl.java @@ -22,6 +22,7 @@ import org.apache.cassandra.thrift.ColumnParent; import org.apache.cassandra.thrift.ColumnPath; import org.apache.cassandra.thrift.ConsistencyLevel; +import org.apache.cassandra.thrift.CounterColumn; import org.apache.cassandra.thrift.IndexClause; import org.apache.cassandra.thrift.KeyRange; import org.apache.cassandra.thrift.KeySlice; @@ -56,7 +57,7 @@ public class KeyspaceServiceImpl implements KeyspaceService { private CassandraHost cassandraHost; private final FailoverPolicy failoverPolicy; - + private final Map credentials; public KeyspaceServiceImpl(String keyspaceName, @@ -234,6 +235,47 @@ public List getSlice(String key, ColumnParent columnParent, SlicePredica return getSlice(StringSerializer.get().toByteBuffer(key), columnParent, predicate); } + @Override + public List getCounterSlice(final ByteBuffer key, final ColumnParent columnParent, + final SlicePredicate predicate) throws HectorException { + Operation> op = + new Operation>(OperationType.READ, failoverPolicy, keyspaceName, credentials) { + + @Override + public List execute(Cassandra.Client cassandra) throws HectorException { + try { + List cosclist = cassandra.get_slice(key, columnParent, + predicate, getThriftCl(OperationType.READ)); + + if (cosclist == null) { + return null; + } + ArrayList result = new ArrayList(cosclist.size()); + for (ColumnOrSuperColumn cosc : cosclist) { + if (cosc.isSetCounter_column()) { + result.add(cosc.getCounter_column()); + } else { + // Inconsistency + throw new HectorException("Regular Column is part of the set of Counter Column"); + } + + } + return result; + } catch (Exception e) { + throw xtrans.translate(e); + } + } + }; + operateWithFailover(op); + return op.getResult(); + } + + @Override + public List getCounterSlice(String key, ColumnParent columnParent, SlicePredicate predicate) + throws HectorException { + return getCounterSlice(StringSerializer.get().toByteBuffer(key), columnParent, predicate); + } + @Override public SuperColumn getSuperColumn(final ByteBuffer key, final ColumnPath columnPath) throws HectorException { @@ -353,6 +395,29 @@ public Void execute(Cassandra.Client cassandra) throws HectorException { operateWithFailover(op); } + @Override + public void addCounter(final ByteBuffer key, final ColumnParent columnParent, final CounterColumn counterColumn) + throws HectorException { + Operation op = new Operation(OperationType.WRITE, failoverPolicy, keyspaceName, credentials) { + + @Override + public Void execute(Cassandra.Client cassandra) throws HectorException { + try { + cassandra.add(key, columnParent, counterColumn, getThriftCl(OperationType.WRITE)); + return null; + } catch (Exception e) { + throw xtrans.translate(e); + } + } + }; + operateWithFailover(op); + } + + @Override + public void addCounter(String key, ColumnParent columnParent, CounterColumn counterColumn) throws HectorException { + addCounter(StringSerializer.get().toByteBuffer(key), columnParent, counterColumn); + } + @Override public void insert(String key, ColumnPath columnPath, ByteBuffer value) throws HectorException { // valideColumnPath(columnPath); @@ -360,7 +425,9 @@ public void insert(String key, ColumnPath columnPath, ByteBuffer value) throws H if (columnPath.isSetSuper_column()) { columnParent.setSuper_column(columnPath.getSuper_column()); } - Column column = new Column(ByteBuffer.wrap(columnPath.getColumn()), value, connectionManager.createClock()); + Column column = new Column(ByteBuffer.wrap(columnPath.getColumn())); + column.setValue(value); + column.setTimestamp(connectionManager.createClock()); insert(StringSerializer.get().toByteBuffer(key), columnParent, column); } @@ -371,7 +438,9 @@ public void insert(String key, ColumnPath columnPath, ByteBuffer value, long tim if (columnPath.isSetSuper_column()) { columnParent.setSuper_column(columnPath.getSuper_column()); } - Column column = new Column(ByteBuffer.wrap(columnPath.getColumn()), value, timestamp); + Column column = new Column(ByteBuffer.wrap(columnPath.getColumn())); + column.setValue(value); + column.setTimestamp(timestamp); insert(StringSerializer.get().toByteBuffer(key), columnParent, column); } @@ -560,6 +629,28 @@ public Void execute(Cassandra.Client cassandra) throws HectorException { operateWithFailover(op); } + @Override + public void removeCounter(final ByteBuffer key, final ColumnPath columnPath) throws HectorException { + Operation op = new Operation(OperationType.WRITE, failoverPolicy, keyspaceName, credentials) { + + @Override + public Void execute(Cassandra.Client cassandra) throws HectorException { + try { + cassandra.remove_counter(key, columnPath, getThriftCl(OperationType.WRITE)); + return null; + } catch (Exception e) { + throw xtrans.translate(e); + } + } + }; + operateWithFailover(op); + } + + @Override + public void removeCounter(String key, ColumnPath columnPath) throws HectorException { + removeCounter(StringSerializer.get().toByteBuffer(key), columnPath); + } + @Override public void remove(String key, ColumnPath columnPath) throws HectorException { remove(StringSerializer.get().toByteBuffer(key), columnPath); @@ -606,7 +697,37 @@ public Column execute(Cassandra.Client cassandra) throws HectorException { throw op.getException(); } return op.getResult(); + } + + @Override + public CounterColumn getCounter(final ByteBuffer key, final ColumnPath columnPath) throws HectorException { + Operation op = new Operation(OperationType.READ, failoverPolicy, keyspaceName, credentials) { + + @Override + public CounterColumn execute(Cassandra.Client cassandra) throws HectorException { + ColumnOrSuperColumn cosc; + try { + cosc = cassandra.get(key, columnPath, getThriftCl(OperationType.READ)); + } catch (NotFoundException e) { + setException(xtrans.translate(e)); + return null; + } catch (Exception e) { + throw xtrans.translate(e); + } + return cosc.getCounter_column(); + } + + }; + operateWithFailover(op); + if (op.hasException()) { + throw op.getException(); + } + return op.getResult(); + } + @Override + public CounterColumn getCounter(String key, ColumnPath columnPath) throws HectorException { + return getCounter(StringSerializer.get().toByteBuffer(key), columnPath); } @Override @@ -668,4 +789,6 @@ public String toString() { b.append(">"); return b.toString(); } + + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/Operation.java b/core/src/main/java/me/prettyprint/cassandra/service/Operation.java index c101ec153..66fbe2cfa 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/Operation.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/Operation.java @@ -30,17 +30,17 @@ public abstract class Operation { public FailoverPolicy failoverPolicy = FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE; public ConsistencyLevelPolicy consistencyLevelPolicy; - + public String keyspaceName; public Map credentials; - + protected T result; private HectorException exception; private CassandraHost cassandraHost; protected long execTime; public final OperationType operationType; - + public Operation(OperationType operationType) { this.failCounter = operationType.equals(OperationType.READ) ? Counter.READ_FAIL : Counter.WRITE_FAIL; @@ -51,7 +51,7 @@ public Operation(OperationType operationType) { public Operation(OperationType operationType, Map credentials) { this(operationType, FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE, null, credentials); } - + public Operation(OperationType operationType, FailoverPolicy failoverPolicy, String keyspaceName, Map credentials) { this.failCounter = operationType.equals(OperationType.READ) ? Counter.READ_FAIL : Counter.WRITE_FAIL; @@ -61,8 +61,8 @@ public Operation(OperationType operationType, FailoverPolicy failoverPolicy, Str this.keyspaceName = keyspaceName; this.credentials = Collections.unmodifiableMap(credentials); } - - + + public void applyConnectionParams(String keyspace, ConsistencyLevelPolicy consistencyLevelPolicy, FailoverPolicy failoverPolicy, Map credentials) { // TODO this is a first step. must be cleaned up. @@ -85,7 +85,7 @@ public T getResult() { // TODO remove in favor of getExecutionResult return result; } - + public ExecutionResult getExecutionResult() { return new ExecutionResult(result, execTime, cassandraHost); } @@ -113,10 +113,10 @@ public boolean hasException() { public HectorException getException() { return exception; } - + public CassandraHost getCassandraHost() { return this.cassandraHost; } - + } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/OperationType.java b/core/src/main/java/me/prettyprint/cassandra/service/OperationType.java index ea2adcbf5..a97e994f3 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/OperationType.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/OperationType.java @@ -2,9 +2,9 @@ /** * Specifies the "type" of operation - read or write. - * It's used for perf4j, so should be in sync with hectorLog4j.xml + * It's used for Speed4j, so should be in sync with hectorLog4j.xml * @author Ran Tavory (ran@outbain.com) - * + * */ public enum OperationType { /** Read operations*/ diff --git a/core/src/main/java/me/prettyprint/cassandra/service/StringKeyIterator.java b/core/src/main/java/me/prettyprint/cassandra/service/StringKeyIterator.java new file mode 100644 index 000000000..a8ac4fc6c --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/StringKeyIterator.java @@ -0,0 +1,20 @@ +package me.prettyprint.cassandra.service; + +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.hector.api.Keyspace; + +/** + * This class returns each key in the specified Column Family as an Iterator. You + * can use this class in a for loop without the overhead of first storing each + * key in a large array. This is a convenience class for KeyIterator when the key + * is a String. + * @author Tim Koop + * @see KeyIterator + */ +public class StringKeyIterator extends KeyIterator { + + public StringKeyIterator(Keyspace keyspace, String columnFamily) { + super(keyspace, columnFamily, new StringSerializer()); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ThriftCfDef.java b/core/src/main/java/me/prettyprint/cassandra/service/ThriftCfDef.java index 56ee9581e..eb6542e90 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ThriftCfDef.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ThriftCfDef.java @@ -29,6 +29,7 @@ public class ThriftCfDef implements ColumnFamilyDefinition { private double readRepairChance; private List columnMetadata; private int gcGraceSeconds; + private String keyValidationClass; private String defaultValidationClass; private int id; private int maxCompactionThreshold; @@ -36,6 +37,7 @@ public class ThriftCfDef implements ColumnFamilyDefinition { private double memtableOperationsInMillions; private int memtableThroughputInMb; private int memtableFlushAfterMins; + private int keyCacheSavePeriodInSeconds; public ThriftCfDef(CfDef d) { Assert.notNull(d, "CfDef is null"); @@ -48,24 +50,26 @@ public ThriftCfDef(CfDef d) { rowCacheSize = d.row_cache_size; rowCacheSavePeriodInSeconds = d.row_cache_save_period_in_seconds; keyCacheSize = d.key_cache_size; + keyCacheSavePeriodInSeconds = d.key_cache_save_period_in_seconds; + keyValidationClass = d.key_validation_class; readRepairChance = d.read_repair_chance; columnMetadata = ThriftColumnDef.fromThriftList(d.column_metadata); gcGraceSeconds = d.gc_grace_seconds; defaultValidationClass = d.default_validation_class; id = d.id; - minCompactionThreshold = d.min_compaction_threshold == 0 ? + minCompactionThreshold = d.min_compaction_threshold == 0 ? CFMetaData.DEFAULT_MIN_COMPACTION_THRESHOLD : d.min_compaction_threshold; maxCompactionThreshold = d.max_compaction_threshold == 0 ? CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD : d.max_compaction_threshold; - memtableOperationsInMillions = d.memtable_operations_in_millions == 0 ? + memtableOperationsInMillions = d.memtable_operations_in_millions == 0 ? CFMetaData.DEFAULT_MEMTABLE_OPERATIONS_IN_MILLIONS : d.memtable_operations_in_millions; - memtableFlushAfterMins = d.memtable_flush_after_mins == 0 ? + memtableFlushAfterMins = d.memtable_flush_after_mins == 0 ? CFMetaData.DEFAULT_MEMTABLE_LIFETIME_IN_MINS : d.memtable_flush_after_mins; - memtableThroughputInMb = d.memtable_throughput_in_mb == 0 ? + memtableThroughputInMb = d.memtable_throughput_in_mb == 0 ? CFMetaData.DEFAULT_MEMTABLE_THROUGHPUT_IN_MB : d.memtable_throughput_in_mb; } - + public ThriftCfDef(ColumnFamilyDefinition columnFamilyDefinition) { keyspace = columnFamilyDefinition.getKeyspaceName(); name = columnFamilyDefinition.getName(); @@ -76,6 +80,8 @@ public ThriftCfDef(ColumnFamilyDefinition columnFamilyDefinition) { rowCacheSize = columnFamilyDefinition.getRowCacheSize(); rowCacheSavePeriodInSeconds = columnFamilyDefinition.getRowCacheSavePeriodInSeconds(); keyCacheSize = columnFamilyDefinition.getKeyCacheSize(); + keyCacheSavePeriodInSeconds = columnFamilyDefinition.getKeyCacheSavePeriodInSeconds(); + keyValidationClass = columnFamilyDefinition.getKeyValidationClass(); readRepairChance = columnFamilyDefinition.getReadRepairChance(); columnMetadata = columnFamilyDefinition.getColumnMetadata(); gcGraceSeconds = columnFamilyDefinition.getGcGraceSeconds(); @@ -83,13 +89,13 @@ public ThriftCfDef(ColumnFamilyDefinition columnFamilyDefinition) { id = columnFamilyDefinition.getId(); minCompactionThreshold = columnFamilyDefinition.getMinCompactionThreshold() == 0 ? CFMetaData.DEFAULT_MIN_COMPACTION_THRESHOLD : columnFamilyDefinition.getMinCompactionThreshold(); - maxCompactionThreshold = columnFamilyDefinition.getMaxCompactionThreshold() == 0 ? + maxCompactionThreshold = columnFamilyDefinition.getMaxCompactionThreshold() == 0 ? CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD : columnFamilyDefinition.getMaxCompactionThreshold(); - memtableFlushAfterMins = columnFamilyDefinition.getMemtableFlushAfterMins() == 0 ? - CFMetaData.DEFAULT_MEMTABLE_LIFETIME_IN_MINS : columnFamilyDefinition.getMemtableFlushAfterMins(); - memtableThroughputInMb = columnFamilyDefinition.getMemtableThroughputInMb() == 0 ? - CFMetaData.DEFAULT_MEMTABLE_THROUGHPUT_IN_MB : columnFamilyDefinition.getMemtableThroughputInMb(); - memtableOperationsInMillions = columnFamilyDefinition.getMemtableOperationsInMillions() == 0 ? + memtableFlushAfterMins = columnFamilyDefinition.getMemtableFlushAfterMins() == 0 ? + CFMetaData.DEFAULT_MEMTABLE_LIFETIME_IN_MINS : columnFamilyDefinition.getMemtableFlushAfterMins(); + memtableThroughputInMb = columnFamilyDefinition.getMemtableThroughputInMb() == 0 ? + CFMetaData.DEFAULT_MEMTABLE_THROUGHPUT_IN_MB : columnFamilyDefinition.getMemtableThroughputInMb(); + memtableOperationsInMillions = columnFamilyDefinition.getMemtableOperationsInMillions() == 0 ? CFMetaData.DEFAULT_MEMTABLE_OPERATIONS_IN_MILLIONS : columnFamilyDefinition.getMemtableOperationsInMillions(); } @@ -102,11 +108,12 @@ public ThriftCfDef(String keyspace, String columnFamilyName) { comparatorType = ComparatorType.BYTESTYPE; readRepairChance = CFMetaData.DEFAULT_READ_REPAIR_CHANCE; keyCacheSize = CFMetaData.DEFAULT_KEY_CACHE_SIZE; + keyCacheSavePeriodInSeconds = CFMetaData.DEFAULT_KEY_CACHE_SAVE_PERIOD_IN_SECONDS; gcGraceSeconds = CFMetaData.DEFAULT_GC_GRACE_SECONDS; minCompactionThreshold = CFMetaData.DEFAULT_MIN_COMPACTION_THRESHOLD; maxCompactionThreshold = CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD; - memtableFlushAfterMins = CFMetaData.DEFAULT_MEMTABLE_LIFETIME_IN_MINS; - memtableThroughputInMb = CFMetaData.DEFAULT_MEMTABLE_THROUGHPUT_IN_MB; + memtableFlushAfterMins = CFMetaData.DEFAULT_MEMTABLE_LIFETIME_IN_MINS; + memtableThroughputInMb = CFMetaData.DEFAULT_MEMTABLE_THROUGHPUT_IN_MB; memtableOperationsInMillions = CFMetaData.DEFAULT_MEMTABLE_OPERATIONS_IN_MILLIONS; } @@ -217,14 +224,16 @@ public CfDef toThrift() { d.setGc_grace_seconds(gcGraceSeconds); d.setId(id); d.setKey_cache_size(keyCacheSize); + d.setKey_cache_save_period_in_seconds(keyCacheSavePeriodInSeconds); + d.setKey_validation_class(keyValidationClass); d.setMax_compaction_threshold(maxCompactionThreshold); d.setMin_compaction_threshold(minCompactionThreshold); d.setRead_repair_chance(readRepairChance); d.setRow_cache_size(rowCacheSize); d.setMemtable_operations_in_millions(memtableOperationsInMillions); d.setMemtable_throughput_in_mb(memtableThroughputInMb); - d.setMemtable_flush_after_mins(memtableFlushAfterMins); - + d.setMemtable_flush_after_mins(memtableFlushAfterMins); + if (subComparatorType != null) { d.setSubcomparator_type(subComparatorType.getClassName()); } @@ -236,6 +245,11 @@ public String getDefaultValidationClass() { return defaultValidationClass; } + @Override + public String getKeyValidationClass(){ + return keyValidationClass; + } + @Override public int getId() { return id; @@ -295,6 +309,10 @@ public void setDefaultValidationClass(String defaultValidationClass) { this.defaultValidationClass = defaultValidationClass; } + public void setKeyValidationClass(String keyValidationClass){ + this.keyValidationClass = keyValidationClass; + } + public void setId(int id) { this.id = id; } @@ -326,4 +344,9 @@ public double getMemtableOperationsInMillions() { public int getMemtableThroughputInMb() { return memtableThroughputInMb; } + + @Override + public int getKeyCacheSavePeriodInSeconds() { + return keyCacheSavePeriodInSeconds; + } } diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ThriftCluster.java b/core/src/main/java/me/prettyprint/cassandra/service/ThriftCluster.java index c6663704a..f03e6dae9 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ThriftCluster.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ThriftCluster.java @@ -36,6 +36,20 @@ public List execute(Cassandra.Client cassandra) throws HectorExcepti return op.getResult(); } + public Map> describeSchemaVersions() throws HectorException { + Operation>> op = new Operation>>(OperationType.META_READ, getCredentials()) { + @Override + public Map> execute(Cassandra.Client cassandra) throws HectorException { + try { + return cassandra.describe_schema_versions(); + } catch (Exception e) { + throw xtrans.translate(e); + } + } + }; + connectionManager.operateWithFailover(op); + return op.getResult(); + } @Override @@ -58,7 +72,7 @@ public String execute(Cassandra.Client cassandra) throws HectorException { public String addColumnFamily(final ColumnFamilyDefinition cfdef) throws HectorException { Operation op = new Operation(OperationType.META_WRITE, FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE, - cfdef.getKeyspaceName(), + cfdef.getKeyspaceName(), getCredentials()) { @Override public String execute(Cassandra.Client cassandra) throws HectorException { @@ -77,7 +91,7 @@ public String execute(Cassandra.Client cassandra) throws HectorException { public String updateColumnFamily(final ColumnFamilyDefinition cfdef) throws HectorException { Operation op = new Operation(OperationType.META_WRITE, FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE, - cfdef.getKeyspaceName(), + cfdef.getKeyspaceName(), getCredentials()) { @Override public String execute(Cassandra.Client cassandra) throws HectorException { @@ -91,7 +105,7 @@ public String execute(Cassandra.Client cassandra) throws HectorException { connectionManager.operateWithFailover(op); return op.getResult(); } - + @Override public String addKeyspace(final KeyspaceDefinition ksdef) throws HectorException { Operation op = new Operation(OperationType.META_WRITE, getCredentials()) { diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ThriftColumnDef.java b/core/src/main/java/me/prettyprint/cassandra/service/ThriftColumnDef.java index 6f4328bc8..4467d0718 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ThriftColumnDef.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ThriftColumnDef.java @@ -26,9 +26,9 @@ public ThriftColumnDef(ColumnDef cd) { name = cd.name; validationClass = cd.validation_class; indexType = indexTypeFromThrift(cd.index_type); - indexName = cd.index_name; + indexName = cd.index_name; } - + public ThriftColumnDef(ColumnDefinition columnDefinition) { name = columnDefinition.getName(); validationClass = columnDefinition.getValidationClass(); @@ -107,7 +107,7 @@ private IndexType indexTypeToThrift(ColumnIndexType indexType2) { throw new RuntimeException("Unknown ColumnIndexType value: " + indexType2); } } - + @Override public String toString() { return ToStringBuilder.reflectionToString(this, ToStringStyle.SHORT_PREFIX_STYLE); diff --git a/core/src/main/java/me/prettyprint/cassandra/service/ThriftKsDef.java b/core/src/main/java/me/prettyprint/cassandra/service/ThriftKsDef.java index 489e47ccd..ef2e6560b 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/ThriftKsDef.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/ThriftKsDef.java @@ -2,6 +2,7 @@ import java.util.ArrayList; import java.util.Collections; +import java.util.HashMap; import java.util.List; import java.util.Map; @@ -12,22 +13,23 @@ import org.apache.cassandra.thrift.KsDef; import org.apache.commons.lang.builder.ToStringBuilder; import org.apache.commons.lang.builder.ToStringStyle; +import org.apache.commons.lang.math.NumberUtils; public class ThriftKsDef implements KeyspaceDefinition { + private static final String REPLICATION_FACTOR = "replication_factor"; public static final String DEF_STRATEGY_CLASS = "org.apache.cassandra.locator.SimpleStrategy"; private final String name; private String strategyClass; - private Map strategyOptions; - private int replicationFactor; + private Map strategyOptions = new HashMap(); private final List cfDefs; public ThriftKsDef(KsDef k) { Assert.notNull(k, "KsDef is null"); name = k.name; strategyClass = k.strategy_class; - strategyOptions = k.strategy_options; - replicationFactor = k.replication_factor; + strategyOptions = k.strategy_options != null ? k.strategy_options : new HashMap(); + setReplicationFactor(NumberUtils.toInt(strategyOptions.get(REPLICATION_FACTOR), 1)); cfDefs = ThriftCfDef.fromThriftList(k.cf_defs); } @@ -35,22 +37,22 @@ public ThriftKsDef(String keyspaceName, String strategyClass, int replicationFac List cfDefs) { this.name = keyspaceName; this.strategyClass = strategyClass; - this.replicationFactor = replicationFactor; + setReplicationFactor(replicationFactor); this.cfDefs = cfDefs; } public ThriftKsDef(String keyspaceName) { this.name = keyspaceName; this.cfDefs = new ArrayList(); - this.replicationFactor = 1; + setReplicationFactor(1); this.strategyClass = DEF_STRATEGY_CLASS; } - + public ThriftKsDef(KeyspaceDefinition keyspaceDefinition) { name = keyspaceDefinition.getName(); strategyClass = keyspaceDefinition.getStrategyClass(); strategyOptions = keyspaceDefinition.getStrategyOptions(); - replicationFactor = keyspaceDefinition.getReplicationFactor(); + setReplicationFactor(keyspaceDefinition.getReplicationFactor()); cfDefs = keyspaceDefinition.getCfDefs(); } @@ -82,7 +84,7 @@ public Map getStrategyOptions() { @Override public int getReplicationFactor() { - return replicationFactor; + return NumberUtils.toInt(strategyOptions.get(REPLICATION_FACTOR), 1); } @Override @@ -91,7 +93,9 @@ public List getCfDefs() { } public KsDef toThrift() { - return new KsDef(name, strategyClass, replicationFactor, ThriftCfDef.toThriftList(cfDefs)); + KsDef def = new KsDef(name, strategyClass, ThriftCfDef.toThriftList(cfDefs)); + def.setStrategy_options(strategyOptions); + return def; } public void setStrategyClass(String strategyClass) { @@ -103,7 +107,8 @@ public void setStrategyOptions(Map strategyOptions) { } public void setReplicationFactor(int replicationFactor) { - this.replicationFactor = replicationFactor; + // Compensate for CASSANDRA-1263 (wasnt my idea) + strategyOptions.put(REPLICATION_FACTOR,Integer.toString(replicationFactor)); } @Override diff --git a/core/src/main/java/me/prettyprint/cassandra/service/VirtualKeyspaceServiceImpl.java b/core/src/main/java/me/prettyprint/cassandra/service/VirtualKeyspaceServiceImpl.java index f446658cb..179b1e356 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/VirtualKeyspaceServiceImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/VirtualKeyspaceServiceImpl.java @@ -16,6 +16,7 @@ import org.apache.cassandra.thrift.Column; import org.apache.cassandra.thrift.ColumnParent; import org.apache.cassandra.thrift.ColumnPath; +import org.apache.cassandra.thrift.CounterColumn; import org.apache.cassandra.thrift.IndexClause; import org.apache.cassandra.thrift.KeyRange; import org.apache.cassandra.thrift.Mutation; @@ -112,6 +113,13 @@ public List getSlice(ByteBuffer key, ColumnParent columnParent, return super.getSlice(ps.toByteBuffer(key), columnParent, predicate); } + @Override + public List getCounterSlice(ByteBuffer key, ColumnParent columnParent, + SlicePredicate predicate) throws HectorException { + + return super.getCounterSlice(ps.toByteBuffer(key), columnParent, predicate); + } + @Override public SuperColumn getSuperColumn(ByteBuffer key, ColumnPath columnPath) throws HectorException { diff --git a/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplate.java b/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplate.java index 1b6d92b19..824aa06a7 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplate.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplate.java @@ -117,7 +117,7 @@ HColumn createColumn(N name, V value, long clock, Serializer nam * Creates a column with the clock of now. */ HColumn createColumn(N name, V value); - + /** * Creates a column with the specified name/value and clock. */ diff --git a/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplateImpl.java b/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplateImpl.java index ab2196096..4b11e1c79 100644 --- a/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplateImpl.java +++ b/core/src/main/java/me/prettyprint/cassandra/service/spring/HectorTemplateImpl.java @@ -217,7 +217,7 @@ public HColumn createColumn(N name, V value, long clock, public HColumn createColumn(N name, V value) { return new HColumnImpl(name, value, createClock()); } - + @Override public HColumn createColumn(N name, V value, long clock) { return new HColumnImpl(name, value, clock); diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractColumnFamilyTemplate.java b/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractColumnFamilyTemplate.java new file mode 100644 index 000000000..3c0c50de5 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractColumnFamilyTemplate.java @@ -0,0 +1,182 @@ +package me.prettyprint.cassandra.service.template; + +import java.util.HashMap; +import java.util.Map; + +import me.prettyprint.cassandra.model.HSlicePredicate; +import me.prettyprint.cassandra.model.thrift.ThriftColumnFactory; +import me.prettyprint.cassandra.service.ExceptionsTranslator; +import me.prettyprint.cassandra.service.ExceptionsTranslatorImpl; +import me.prettyprint.hector.api.ColumnFactory; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.factory.HFactory; +import me.prettyprint.hector.api.mutation.MutationResult; +import me.prettyprint.hector.api.mutation.Mutator; + +import org.apache.cassandra.thrift.ColumnParent; + +public class AbstractColumnFamilyTemplate { + // Used for queries where we just ask for all columns + public static final int ALL_COLUMNS_COUNT = Integer.MAX_VALUE; + public static final Object ALL_COLUMNS_START = null; + public static final Object ALL_COLUMNS_END = null; + + protected Keyspace keyspace; + protected String columnFamily; + protected Serializer keySerializer; + protected Map> columnValueSerializers; + protected ColumnParent columnParent; + protected HSlicePredicate activeSlicePredicate; + protected ColumnFactory columnFactory; + + /** The serializer for a standard column name or a super-column name */ + protected Serializer topSerializer; + + /** + * Used for all updates. Can but passed in the constructor/reassigned to allow + * updates between multiple column families/CassandraTemplates to be batched + */ + protected Mutator mutator; + + /** + * By default, execute updates automatically at common-sense points such as + * after queuing the updates of all an object's properties. Or, in the case of + * multiple objects, at the end of the list. No Mutator executes() will be + * called if this is set to true. This allows an arbitrary number of updates + * to be performed and executed manually. + */ + protected boolean batched; + + /** + * An optional clock value to pass to deletes. If null, the default value + * generated by Hector is used + */ + protected Long clock; + + protected ExceptionsTranslator exceptionsTranslator; + + public AbstractColumnFamilyTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer) { + this(keyspace, columnFamily, keySerializer, topSerializer, HFactory + .createMutator(keyspace, keySerializer)); + } + + public AbstractColumnFamilyTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer, + Mutator mutator) { + // ugly, but safe + this.keyspace = keyspace; + this.columnFamily = columnFamily; + this.keySerializer = keySerializer; + this.topSerializer = topSerializer; + this.mutator = mutator; + columnValueSerializers = new HashMap>(); + this.columnParent = new ColumnParent(columnFamily); + this.activeSlicePredicate = new HSlicePredicate(topSerializer); + exceptionsTranslator = new ExceptionsTranslatorImpl(); + this.columnFactory = new ThriftColumnFactory(); + setCount(100); + } + + + /** + * Add a column to the static set of columns which will be used in constructing + * the single-argument form of slicing operations + * @param columnName + * @param valueSerializer + */ + public AbstractColumnFamilyTemplate addColumn(N columnName, Serializer valueSerializer) { + columnValueSerializers.put(columnName, valueSerializer); + activeSlicePredicate.addColumnName(columnName); + return this; + } + + /** + * Get the value serializer for a given column. Returns null if none found + * @param columnName + * @return + */ + public Serializer getValueSerializer(N columnName) { + return columnValueSerializers.get(columnName); + } + + + public boolean isBatched() { + return batched; + } + + public AbstractColumnFamilyTemplate setBatched(boolean batched) { + this.batched = batched; + return this; + } + + public String getColumnFamily() { + return columnFamily; + } + + public Serializer getKeySerializer() { + return keySerializer; + } + + public Serializer getTopSerializer() { + return topSerializer; + } + + public MutationResult executeBatch() { + MutationResult result = mutator.execute(); + mutator.discardPendingMutations(); + return result; + } + + public Mutator getMutator() { + return mutator; + } + + public AbstractColumnFamilyTemplate setMutator(Mutator mutator) { + this.mutator = mutator; + return this; + } + + public Long getClock() { + return clock; + } + + public void setClock(Long clock) { + this.clock = clock; + } + + public long getEffectiveClock() { + return clock != null ? clock.longValue() : keyspace.createClock(); + } + + public void setExceptionsTranslator(ExceptionsTranslator exceptionsTranslator) { + this.exceptionsTranslator = exceptionsTranslator; + } + + public void setColumnFactory(ColumnFactory columnFactory) { + this.columnFactory = columnFactory; + } + + protected MutationResult executeIfNotBatched() { + return !isBatched() ? executeBatch() : null; + } + + public void deleteRow(K key) { + mutator.addDeletion(key, columnFamily, null, topSerializer); + executeIfNotBatched(); + } + + public void deleteColumn(K key, N columnName) { + mutator.addDeletion(key, columnFamily, columnName, topSerializer); + executeIfNotBatched(); + } + + /** + * The number of columns to return when not doing a name-based template + * @param count + */ + public void setCount(int count) { + activeSlicePredicate.setCount(count); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractResultWrapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractResultWrapper.java new file mode 100644 index 000000000..e0c4a848b --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractResultWrapper.java @@ -0,0 +1,97 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.Date; +import java.util.UUID; + +import me.prettyprint.cassandra.serializers.BooleanSerializer; +import me.prettyprint.cassandra.serializers.BytesArraySerializer; +import me.prettyprint.cassandra.serializers.DateSerializer; +import me.prettyprint.cassandra.serializers.IntegerSerializer; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.serializers.UUIDSerializer; +import me.prettyprint.cassandra.service.CassandraHost; +import me.prettyprint.hector.api.ResultStatus; +import me.prettyprint.hector.api.Serializer; + +/** + * Provides access to the current row of data during queries. There is a lot of + * overlap in needs for both standard and super queries. This class consolidates + * what they have in common. All data is read into ByteBuffers and translated to + * a primitive type when requested. + * + * This class is a non-static inner class which inherits the Java generic + * parameters of it's containing ColumnFamilyTemplate instance. This allows it to + * inherit the parameter from ColumnFamilyTemplate. + * + * The parameters allows this to be used by standard and super column + * queries + * + * @author david + * @author zznate + * @param + * the type of the key + * @param + * the standard column name type or the super column's child column + * type + */ +public abstract class AbstractResultWrapper implements ColumnFamilyResult { + + protected Serializer keySerializer; + protected Serializer columnNameSerializer; + protected ResultStatus resultStatus; + + public AbstractResultWrapper(Serializer keySerializer, Serializer columnNameSerializer, ResultStatus resultStatus) { + this.keySerializer = keySerializer; + this.columnNameSerializer = columnNameSerializer; + this.resultStatus = resultStatus; + } + + public abstract ByteBuffer getColumnValue(N columnName); + + public UUID getUUID(N columnName) { + return UUIDSerializer.get().fromByteBuffer(getColumnValue(columnName)); + } + + public String getString(N columnName) { + return StringSerializer.get().fromByteBuffer(getColumnValue(columnName)); + } + + public Long getLong(N columnName) { + return LongSerializer.get().fromByteBuffer(getColumnValue(columnName)); + } + + public Integer getInteger(N columnName) { + return IntegerSerializer.get().fromByteBuffer(getColumnValue(columnName)); + } + + public Boolean getBoolean(N columnName) { + return BooleanSerializer.get().fromByteBuffer(getColumnValue(columnName)); + } + + public byte[] getByteArray(N columnName) { + return BytesArraySerializer.get() + .fromByteBuffer(getColumnValue(columnName)); + } + + public Date getDate(N columnName) { + return DateSerializer.get().fromByteBuffer(getColumnValue(columnName)); + } + + @Override + public long getExecutionTimeMicro() { + return resultStatus.getExecutionTimeMicro(); + } + + @Override + public long getExecutionTimeNano() { + return resultStatus.getExecutionTimeNano(); + } + + @Override + public CassandraHost getHostUsed() { + return resultStatus.getHostUsed(); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractTemplateUpdater.java b/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractTemplateUpdater.java new file mode 100644 index 000000000..5ef0c67ae --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/AbstractTemplateUpdater.java @@ -0,0 +1,44 @@ +package me.prettyprint.cassandra.service.template; + +import java.util.ArrayList; +import java.util.List; + +import me.prettyprint.hector.api.ColumnFactory; + +public abstract class AbstractTemplateUpdater { + + protected List keys; + protected int keyPos = 0; + protected ColumnFactory columnFactory; + protected AbstractColumnFamilyTemplate template; + + public AbstractTemplateUpdater(AbstractColumnFamilyTemplate template, ColumnFactory columnFactory) { + this.template = template; + this.columnFactory = columnFactory; + } + + public AbstractTemplateUpdater addKey(K key) { + if ( keys == null ) { + keys = new ArrayList(); + } else { + keyPos++; + } + keys.add(key); + + return this; + } + + /** + * @return Give the updater access to the current key if it needs it + */ + public K getCurrentKey() { + return keys.get(keyPos); + } + + /** + * To be overridden by folks choosing to add their own functionality. Default is a no-op. + */ + public void update() { + + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/CassandraClusterFactory.java b/core/src/main/java/me/prettyprint/cassandra/service/template/CassandraClusterFactory.java new file mode 100644 index 000000000..3d8e68bfe --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/CassandraClusterFactory.java @@ -0,0 +1,30 @@ +package me.prettyprint.cassandra.service.template; + +import me.prettyprint.hector.api.Cluster; +import me.prettyprint.hector.api.factory.HFactory; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Factory to remove Cassandra configuration concerns from DAO objects. + * This is invoked via a spring factory method that allows injection of the + * Hector Cluster object into the DAO. + *

+ * + *

+ * @author david + * @since Jan 14, 2011 + * + */ +public class CassandraClusterFactory +{ + static final Logger LOGGER = LoggerFactory.getLogger( CassandraClusterFactory.class ); + + public static Cluster getInstance( String name, String host, int port ) + { + LOGGER.debug( "getInstance: creating cluster name=" + name + ", host=" + host + ", port=" + port ); + return HFactory.getOrCreateCluster( name, host + ":" + port ); + + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyResult.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyResult.java new file mode 100644 index 000000000..fbad6d1c5 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyResult.java @@ -0,0 +1,47 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.Collection; +import java.util.Date; +import java.util.Iterator; +import java.util.UUID; + +import me.prettyprint.hector.api.ResultStatus; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; + +/** + * A common interface for access to the resuls of a query of either a standard or super column family. + * There are different implementations of this which hide the differences requires of standar/super + * column families. As this interface inherits from {@link ResultStatus}, results will also provide + * execution details. + * + * @author david + * @author zznate + * @param + * @param + */ +public interface ColumnFamilyResult extends Iterator>,ResultStatus { + K getKey(); + + UUID getUUID(N columnName); + + String getString(N columnName); + + Long getLong(N columnName); + + Integer getInteger(N columnName); + + Boolean getBoolean(N columnName); + + byte[] getByteArray(N columnName); + + Date getDate(N columnName); + + Collection getColumnNames(); + + HColumn getColumn(N columnName); + + boolean hasResults(); + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyResultWrapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyResultWrapper.java new file mode 100644 index 000000000..45712b1f4 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyResultWrapper.java @@ -0,0 +1,129 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.Collection; +import java.util.HashMap; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.NoSuchElementException; + +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.cassandra.model.HColumnImpl; +import me.prettyprint.cassandra.serializers.ByteBufferSerializer; +import me.prettyprint.cassandra.service.CassandraHost; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; + +import org.apache.cassandra.thrift.Column; +import org.apache.cassandra.thrift.ColumnOrSuperColumn; + +/** + * Wraps the results with as an Iterator. The underlying Iterator has already been advanced + * to the first row upon construction. + * + * @author zznate + */ +public class ColumnFamilyResultWrapper extends AbstractResultWrapper { + + private Map> columns = new LinkedHashMap>(); + private Iterator>> rows; + private Map.Entry> entry; + private ExecutionResult>> executionResult; + private boolean hasEntries; + + public ColumnFamilyResultWrapper(Serializer keySerializer, + Serializer columnNameSerializer, + ExecutionResult>> executionResult) { + super(keySerializer, columnNameSerializer, executionResult); + this.rows = executionResult.get().entrySet().iterator(); + next(); + hasEntries = getColumnNames() != null && getColumnNames().size() > 0; + } + + /** + * All the column names we know about in the current iterator position + * @return + */ + public Collection getColumnNames() { + return columns.keySet(); + } + + public ByteBuffer getColumnValue( N columnName) { + HColumn col = getColumn( columnName ); + return col != null ? col.getValue() : null; + } + + public HColumn getColumn( N columnName ) { + return columns.get( columnName ); + } + + + private void applyToRow(List cosclist) { + + for (Iterator iterator = cosclist.iterator(); iterator.hasNext();) { + ColumnOrSuperColumn cosc = iterator.next(); + if ( cosc.isSetSuper_column() ) { + applySuper(cosc); + } else { + applyStandard(cosc.getColumn()); + } + + iterator.remove(); + } + } + + private void applySuper(ColumnOrSuperColumn cosc) { + Iterator tcolumns = cosc.getSuper_column().getColumnsIterator(); + while ( tcolumns.hasNext() ) { + applyStandard(tcolumns.next()); + } + } + + + private void applyStandard(Column cosc) { + N colName = columnNameSerializer.fromByteBuffer(cosc.name.duplicate()); + HColumn column = columns.get(colName); + + if ( column == null ) { + column = new HColumnImpl(cosc, columnNameSerializer, ByteBufferSerializer.get()); + } else { + ((HColumnImpl)column).apply(cosc); + } + columns.put(colName, column); + } + + @Override + public K getKey() { + return keySerializer.fromByteBuffer(entry.getKey()); + } + + @Override + public ColumnFamilyResult next() { + if ( !hasNext() ) { + throw new NoSuchElementException("No more rows left on this HColumnFamily"); + } + entry = rows.next(); + applyToRow(entry.getValue()); + return this; + } + + @Override + public boolean hasNext() { + return rows.hasNext(); + } + + @Override + public void remove() { + rows.remove(); + } + + @Override + public boolean hasResults() { + return hasEntries; + } + + + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyRowMapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyRowMapper.java new file mode 100644 index 000000000..d4daf7467 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyRowMapper.java @@ -0,0 +1,14 @@ +package me.prettyprint.cassandra.service.template; + +/** + * Converts the contents of a standard column family row into an object. + * + * @author david + * @since Mar 10, 2011 + * @param + * @param standard column type data type + * @param the object type being mapped into + */ +public interface ColumnFamilyRowMapper { + public V mapRow(ColumnFamilyResult results); +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyTemplate.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyTemplate.java new file mode 100644 index 000000000..cd5bb3467 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyTemplate.java @@ -0,0 +1,226 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.model.ExecutingKeyspace; +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.cassandra.model.HSlicePredicate; +import me.prettyprint.cassandra.model.thrift.ThriftConverter; +import me.prettyprint.cassandra.serializers.SerializerTypeInferer; +import me.prettyprint.cassandra.service.Operation; +import me.prettyprint.cassandra.service.OperationType; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.exceptions.HectorException; +import me.prettyprint.hector.api.factory.HFactory; +import me.prettyprint.hector.api.mutation.Mutator; +import me.prettyprint.hector.api.query.ColumnQuery; +import me.prettyprint.hector.api.query.CountQuery; +import me.prettyprint.hector.api.query.QueryResult; + +import org.apache.cassandra.thrift.Cassandra; +import org.apache.cassandra.thrift.ColumnOrSuperColumn; + +import com.google.common.collect.Iterators; + +/** + * This applies a Template Method pattern, much like Spring's JdbcTemplate, to + * Cassandra. The ColumnFamilyTemplate instance maintains many of the fields in + * common between various query/update operations so that they do not need to be + * constantly passed for every operation on the column family. These include the + * keyspace, column family name, key serializer, and the column name serializer + * (for standard column name or the super column name). + * + * The Java generic types of the ColumnFamilyTemplate class itself are limited to + * the key and column name type. It defers the generic types for super column + * child types to the individual update/query operation. + * + * @author david + * @author zznate + * @param + * The column family key type + * @param + * The column family name type + */ +public abstract class ColumnFamilyTemplate extends AbstractColumnFamilyTemplate { + + public ColumnFamilyTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer) { + super(keyspace, columnFamily, keySerializer, topSerializer); + } + + public ColumnFamilyTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer, + Mutator mutator) { + super(keyspace, columnFamily, keySerializer, topSerializer, mutator); + } + + // Just so method chaining will return this type instead of the parent class + // for operations down the chain + public ColumnFamilyTemplate setBatched(boolean batched) { + super.setBatched(batched); + return this; + } + + // Just so method chaining will return this type instead of the parent class + // for operations down the chain + public ColumnFamilyTemplate setMutator(Mutator mutator) { + super.setMutator(mutator); + return this; + } + + public ColumnFamilyUpdater createUpdater(K key) { + ColumnFamilyUpdater updater = new ColumnFamilyUpdater(this, columnFactory); + updater.addKey(key); + return updater; + } + + public void update(ColumnFamilyUpdater updater) { + updater.update(); + executeIfNotBatched(); + } + + + /** + * Checks if there are any columns at a row specified by key in a standard + * column family + * + * @param key + * @return true if columns exist + */ + public boolean isColumnsExist(K key) { + return countColumns(key) > 0; + } + + /** + * @param key + * @return the number of columns in a standard column family at the specified + * row key + */ + @SuppressWarnings("unchecked") + public int countColumns(K key) { + return countColumns(key, (N) ALL_COLUMNS_START, (N) ALL_COLUMNS_END, + ALL_COLUMNS_COUNT); + } + + /** + * Counts columns in the specified range of a standard column family + * + * @param key + * @param start + * @param end + * @param max + * @return + */ + public int countColumns(K key, N start, N end, int max) { + CountQuery query = HFactory.createCountQuery(keyspace, keySerializer, + topSerializer); + query.setKey(key); + query.setColumnFamily(columnFamily); + query.setRange(start, end, max); + return query.execute().get(); + } + + public ColumnFamilyResult queryColumns(K key) { + return doExecuteSlice(key, activeSlicePredicate); + } + + public ColumnFamilyResult queryColumns(Iterable keys) { + return doExecuteMultigetSlice(keys, activeSlicePredicate); + } + + @SuppressWarnings("unchecked") + public T queryColumns(K key, ColumnFamilyRowMapper mapper) { + return queryColumns(key, activeSlicePredicate, mapper); + } + + /** + * Queries a range of columns at the given key and maps them to an object of + * type OBJ using the given mapping object + * + * @param + * @param key + * @param start + * @param end + * @param mapper + * @return + */ + public T queryColumns(K key, HSlicePredicate predicate, + ColumnFamilyRowMapper mapper) { + return doExecuteSlice(key, predicate, mapper); + } + + /** + * Queries all columns at a given key and maps them to an object of type OBJ + * using the given mapping object + * + * @param + * @param key + * @param columns + * @param mapper + * @return + */ + @SuppressWarnings("unchecked") + public T queryColumns(K key, List columns, + ColumnFamilyRowMapper mapper) { + HSlicePredicate predicate = new HSlicePredicate(topSerializer); + predicate.setColumnNames(columns); + return doExecuteSlice(key, predicate, mapper); + } + + + public MappedColumnFamilyResult queryColumns(Iterable keys, + ColumnFamilyRowMapper mapper) { + return doExecuteMultigetSlice(keys, activeSlicePredicate, mapper); + } + + public MappedColumnFamilyResult queryColumns(Iterable keys, + HSlicePredicate predicate, ColumnFamilyRowMapper mapper) { + return doExecuteMultigetSlice(keys, predicate, mapper); + } + + public MappedColumnFamilyResult queryColumns(Iterable keys, + List columns, ColumnFamilyRowMapper mapper) { + HSlicePredicate predicate = new HSlicePredicate(topSerializer); + predicate.setColumnNames(columns); + return doExecuteMultigetSlice(keys, predicate, mapper); + } + + + @SuppressWarnings("unchecked") + public HColumn querySingleColumn(K key, N columnName, + Class valueClass) { + return querySingleColumn(key, columnName, + (Serializer) SerializerTypeInferer.getSerializer(valueClass)); + } + + public HColumn querySingleColumn(K key, N columnName, + Serializer valueSerializer) { + ColumnQuery query = HFactory.createColumnQuery(keyspace, + keySerializer, topSerializer, valueSerializer); + query.setColumnFamily(columnFamily); + query.setKey(key); + query.setName(columnName); + QueryResult> result = query.execute(); + return result != null ? result.get() : null; + } + + //-------------------------- delegation methods ---------------------------- + + protected abstract T doExecuteSlice(K key, HSlicePredicate predicate, ColumnFamilyRowMapper mapper); + + protected abstract ColumnFamilyResult doExecuteSlice(final K key, final HSlicePredicate workingSlicePredicate); + + protected abstract ColumnFamilyResult doExecuteMultigetSlice(final Iterable keys, final HSlicePredicate workingSlicePredicate); + + protected abstract MappedColumnFamilyResult doExecuteMultigetSlice(final Iterable keys, + final HSlicePredicate workingSlicePredicate, + final ColumnFamilyRowMapper mapper); + + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyUpdater.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyUpdater.java new file mode 100644 index 000000000..a53d53348 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ColumnFamilyUpdater.java @@ -0,0 +1,113 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.Date; +import java.util.UUID; + +import me.prettyprint.cassandra.serializers.BooleanSerializer; +import me.prettyprint.cassandra.serializers.ByteBufferSerializer; +import me.prettyprint.cassandra.serializers.BytesArraySerializer; +import me.prettyprint.cassandra.serializers.DateSerializer; +import me.prettyprint.cassandra.serializers.DoubleSerializer; +import me.prettyprint.cassandra.serializers.IntegerSerializer; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.serializers.UUIDSerializer; +import me.prettyprint.hector.api.ColumnFactory; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; + +/** + * This provides an interface of updating a specified row, most likely with the + * contents of an object. This would likely by implemented as an anonymous inner + * class with access to a final object in scope. It would update the given row + * with the object's data. + * + * For more complex behaviour, subclasses should implementat update() to simply make + * consecutive calls to various set****() methods which already have the + * contextual information they need to update the correct row. + * + * The downside of this approach is that the updater is essentially stateful and + * cannot be used concurrently. The alternative is to pass an object in to + * update() as a parameter with the setter methods, leaving the updater to be + * stateless. + * + * @author david + * @author zznate + * + * @param + * the key's data type + * @param + * the standard or super column's data type + */ +public class ColumnFamilyUpdater extends AbstractTemplateUpdater { + + public ColumnFamilyUpdater(ColumnFamilyTemplate template, ColumnFactory columnFactory) { + super(template, columnFactory); + } + + public void deleteColumn(N columnName) { + template.getMutator().addDeletion(getCurrentKey(), template.getColumnFamily(), + columnName, template.getTopSerializer()); + } + + public void setString(N columnName, String value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), StringSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setUUID(N columnName, UUID value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), UUIDSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setLong(N columnName, Long value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), LongSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setInteger(N columnName, Integer value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), IntegerSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setDouble(N columnName, Double value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), DoubleSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setBoolean(N columnName, Boolean value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), BooleanSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setByteArray(N columnName, byte[] value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), BytesArraySerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setByteBuffer(N columnName, ByteBuffer value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), ByteBufferSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setDate(N columnName, Date value) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), DateSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + public void setValue(N columnName, V value, Serializer serializer) { + HColumn column = columnFactory.createColumn(columnName, value, + template.getTopSerializer(), serializer); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/MappedColumnFamilyResult.java b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedColumnFamilyResult.java new file mode 100644 index 000000000..ad18f8192 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedColumnFamilyResult.java @@ -0,0 +1,7 @@ +package me.prettyprint.cassandra.service.template; + +public interface MappedColumnFamilyResult extends ColumnFamilyResult { + + V getRow(); + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/MappedColumnFamilyResultWrapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedColumnFamilyResultWrapper.java new file mode 100644 index 000000000..955ce9d74 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedColumnFamilyResultWrapper.java @@ -0,0 +1,27 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.hector.api.Serializer; + +import org.apache.cassandra.thrift.ColumnOrSuperColumn; + +public class MappedColumnFamilyResultWrapper extends ColumnFamilyResultWrapper implements MappedColumnFamilyResult{ + + private ColumnFamilyRowMapper rowMapper; + + public MappedColumnFamilyResultWrapper(Serializer keySerializer, + Serializer columnNameSerializer, + ExecutionResult>> executionResult, ColumnFamilyRowMapper mapper) { + super(keySerializer, columnNameSerializer, executionResult); + } + + @Override + public V getRow() { + return rowMapper.mapRow(this); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/MappedSuperCfResult.java b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedSuperCfResult.java new file mode 100644 index 000000000..e771efd90 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedSuperCfResult.java @@ -0,0 +1,6 @@ +package me.prettyprint.cassandra.service.template; + +public interface MappedSuperCfResult extends SuperCfResult { + + V getRow(); +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/MappedSuperCfResultWrapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedSuperCfResultWrapper.java new file mode 100644 index 000000000..e0773d3c7 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/MappedSuperCfResultWrapper.java @@ -0,0 +1,34 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.hector.api.Serializer; + +import org.apache.cassandra.thrift.ColumnOrSuperColumn; + +public class MappedSuperCfResultWrapper extends SuperCfResultWrapper implements + MappedSuperCfResult { + + private SuperCfRowMapper rowMapper; + + public MappedSuperCfResultWrapper( + Serializer keySerializer, + Serializer sNameSerializer, + Serializer subSerializer, + ExecutionResult>> executionResult, + SuperCfRowMapper mapper) { + super(keySerializer, sNameSerializer, subSerializer, executionResult); + this.rowMapper = mapper; + } + + @Override + public V getRow() { + return rowMapper.mapRow(this); + } + + + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfResult.java b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfResult.java new file mode 100644 index 000000000..9c50621b3 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfResult.java @@ -0,0 +1,54 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.Collection; +import java.util.Date; +import java.util.List; +import java.util.UUID; + +import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.beans.HSuperColumn; + +/** + * Holds the result for the contents of a super column. This interface add + * access to the current super column name since this may be a property of the + * object being mapped into. + * + * @author david + * @since Mar 10, 2011 + * @param + * @param + * super column name data type + * @param + * child column name data type + */ +public interface SuperCfResult extends ColumnFamilyResult { + // TODO remove. this no loger makes sense with many-to-one on a row + + Collection getSuperColumns(); + + K getKey(); + + UUID getUUID(SN sColumnName, N columnName); + + String getString(SN sColumnName, N columnName); + + Long getLong(SN sColumnName, N columnName); + + Integer getInteger(SN sColumnName, N columnName); + + Boolean getBoolean(SN sColumnName, N columnName); + + byte[] getByteArray(SN sColumnName, N columnName); + + ByteBuffer getByteBuffer(SN sColumnName, N columnName); + + Date getDate(SN sColumnName, N columnName); + + void applySuperColumn(SN sColumnName); + + SN getActiveSuperColumn(); + + @Override + SuperCfResult next(); +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfResultWrapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfResultWrapper.java new file mode 100644 index 000000000..1d4469a86 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfResultWrapper.java @@ -0,0 +1,207 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Date; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.NoSuchElementException; +import java.util.UUID; + +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.cassandra.model.HColumnImpl; +import me.prettyprint.cassandra.serializers.BooleanSerializer; +import me.prettyprint.cassandra.serializers.ByteBufferSerializer; +import me.prettyprint.cassandra.serializers.BytesArraySerializer; +import me.prettyprint.cassandra.serializers.DateSerializer; +import me.prettyprint.cassandra.serializers.IntegerSerializer; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.serializers.UUIDSerializer; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; + +import org.apache.cassandra.thrift.Column; +import org.apache.cassandra.thrift.ColumnOrSuperColumn; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Provides access to the current row of data during super column queries. + * + * @author zznate + * @param the super column's sub column name type + */ +public class SuperCfResultWrapper extends AbstractResultWrapper implements SuperCfResult { + private static final Logger log = LoggerFactory.getLogger(SuperCfResultWrapper.class); + + private Map>> columns = new LinkedHashMap>>(); + private Iterator>> rows; + private Map.Entry> entry; + private ExecutionResult>> executionResult; + private List superColumns; + private Map> subColumns = new LinkedHashMap>(); + private SN currentSuperColumn; + private boolean hasEntries; + + private Serializer sNameSerializer; + + public SuperCfResultWrapper(Serializer keySerializer, + Serializer sNameSerializer, + Serializer subSerializer, + ExecutionResult>> executionResult) { + super(keySerializer, subSerializer, executionResult); + this.sNameSerializer = sNameSerializer; + this.rows = executionResult.get().entrySet().iterator(); + next(); + hasEntries = getSuperColumns() != null && getSuperColumns().size() > 0; + } + + @Override + public SuperCfResult next() { + if ( !hasNext() ) { + throw new NoSuchElementException("No more rows left on this HColumnFamily"); + } + entry = rows.next(); + log.debug("found entry {} with value {}", getKey(), entry.getValue()); + applyToRow(entry.getValue()); + return this; + } + + private void applyToRow(List cosclist) { + superColumns = new ArrayList(cosclist.size()); + for (Iterator iterator = cosclist.iterator(); iterator.hasNext();) { + ColumnOrSuperColumn cosc = iterator.next(); + SN sColName = sNameSerializer.fromByteBuffer(cosc.super_column.name); + log.debug("cosc {}", cosc.super_column); + + superColumns.add(sColName); + Iterator tcolumns = cosc.getSuper_column().getColumnsIterator(); + Map> subColMap = new LinkedHashMap>(); + while ( tcolumns.hasNext() ) { + Column col = tcolumns.next(); + subColMap.put(columnNameSerializer.fromByteBuffer(col.name), new HColumnImpl(col, columnNameSerializer, ByteBufferSerializer.get())); + } + columns.put(sColName, subColMap); + } + } + + + public List getSuperColumns() { + return superColumns; + } + + + @Override + public ByteBuffer getColumnValue(N columnName) { + HColumn col = getColumn( columnName ); + return col != null ? col.getValue() : null; + } + + + @Override + public K getKey() { + return keySerializer.fromByteBuffer(entry.getKey()); + } + + + @Override + public HColumn getColumn(N columnName) { + return subColumns == null ? null : subColumns.get( columnName ); + } + + + @Override + public Collection getColumnNames() { + return subColumns != null ? subColumns.keySet() : new ArrayList(); + } + + + @Override + public boolean hasNext() { + return rows.hasNext(); + } + + @Override + public void remove() { + rows.remove(); + } + + private V extractType(SN sColumnName, N columnName, Serializer valueSerializer) { + Map> map = columns.get(sColumnName); + if ( map != null && map.get(columnName) != null ) + return valueSerializer.fromByteBuffer(map.get(columnName).getValue()); + return null; + } + + @Override + public Boolean getBoolean(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, BooleanSerializer.get()); + } + + + @Override + public byte[] getByteArray(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, BytesArraySerializer.get()); + } + + + @Override + public Date getDate(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, DateSerializer.get()); + } + + + @Override + public Integer getInteger(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, IntegerSerializer.get()); + } + + + @Override + public Long getLong(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, LongSerializer.get()); + } + + + @Override + public String getString(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, StringSerializer.get()); + } + + + + @Override + public UUID getUUID(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, UUIDSerializer.get()); + } + + @Override + public ByteBuffer getByteBuffer(SN sColumnName, N columnName) { + return extractType(sColumnName, columnName, ByteBufferSerializer.get()); + } + + + @Override + public SN getActiveSuperColumn() { + return currentSuperColumn; + } + + @Override + public void applySuperColumn(SN sColumnName) { + this.currentSuperColumn = sColumnName; + this.subColumns = columns.get(currentSuperColumn); + + } + + @Override + public boolean hasResults() { + return hasEntries; + } + + + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfRowMapper.java b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfRowMapper.java new file mode 100644 index 000000000..21c0e52fd --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfRowMapper.java @@ -0,0 +1,19 @@ +package me.prettyprint.cassandra.service.template; + +/** + * Converts the contents of a given super column into an object. It is passed a + * results object with the data of the current super column. + * + * @author david + * @since Mar 10, 2011 + * @param + * @param + * the super column name data type + * @param + * the child column data type + * @param + * the object being mapped to datatype + */ +public interface SuperCfRowMapper { + public V mapRow(SuperCfResult results); +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfTemplate.java b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfTemplate.java new file mode 100644 index 000000000..9aa75e8d1 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfTemplate.java @@ -0,0 +1,222 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.Collection; +import java.util.List; + +import me.prettyprint.cassandra.model.HColumnImpl; +import me.prettyprint.cassandra.model.HSlicePredicate; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.factory.HFactory; +import me.prettyprint.hector.api.mutation.Mutator; +import me.prettyprint.hector.api.query.SubCountQuery; +import me.prettyprint.hector.api.query.SuperCountQuery; + +public abstract class SuperCfTemplate extends AbstractColumnFamilyTemplate { + protected Serializer subSerializer; + + public SuperCfTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer, + Serializer subSerializer) { + super(keyspace, columnFamily, keySerializer, topSerializer); + this.subSerializer = subSerializer; + } + + public SuperCfTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer, + Serializer subSerializer, Mutator mutator) { + super(keyspace, columnFamily, keySerializer, topSerializer, mutator); + this.subSerializer = subSerializer; + } + + public Serializer getSubSerializer() { + return subSerializer; + } + + /** + * Checks if there are any columns at a row specified by key in a super column + * family + * + * @param key + * @return true if columns exist + */ + public boolean isColumnsExist(K key) { + return countColumns(key) > 0; + } + + /** + * Checks if there are any columns at a row specified by key in a specific + * super column + * + * @param key + * @param superColumnName + * @param subSerializer + * the column name serializer of the child columns + * @return true if columns exist + */ + public boolean isSubColumnsExist(K key, SN superColumnName) { + return countSubColumns(key, superColumnName) > 0; + } + + /** + * @param key + * @return the number of columns in a super column family at the specified row + * key + */ + @SuppressWarnings("unchecked") + public int countColumns(K key) { + return countColumns(key, (SN) ALL_COLUMNS_START, (SN) ALL_COLUMNS_END, + ALL_COLUMNS_COUNT); + } + + /** + * + * @param key + * @param superColumnName + * @return the number of child columns in a specified super column + */ + @SuppressWarnings("unchecked") + public int countSubColumns(K key, SN superColumnName) { + return countSubColumns(key, superColumnName, (N) ALL_COLUMNS_START, + (N) ALL_COLUMNS_END, ALL_COLUMNS_COUNT); + } + + /** + * Counts columns in the specified range of a super column family + * + * @param key + * @param start + * @param end + * @param max + * @return + */ + public int countColumns(K key, SN start, SN end, int max) { + SuperCountQuery query = HFactory.createSuperCountQuery(keyspace, + keySerializer, topSerializer); + query.setKey(key); + query.setColumnFamily(columnFamily); + query.setRange(start, end, max); + return query.execute().get(); + } + + /** + * Counts child columns in the specified range of a children in a specified + * super column + * + * @param + * @param key + * @param superColumnName + * @param start + * @param end + * @param max + * @param subSerializer + * @return + */ + public int countSubColumns(K key, SN superColumnName, N start, + N end, int max) { + SubCountQuery query = HFactory.createSubCountQuery( + keyspace, keySerializer, topSerializer, subSerializer); + query.setKey(key); + query.setColumnFamily(columnFamily); + query.setSuperColumn(superColumnName); + query.setRange(start, end, max); + return query.execute().get(); + } + + + + public HColumn querySingleSubColumn(K key, + SN columnName, N subColumnName, Serializer valueSerializer) { + + SuperCfResult result = doExecuteSlice(key, columnName, activeSlicePredicate); + + if (result == null) { + return null; + } + + HColumn origCol = result.getColumn(subColumnName); + + // TODO make this far less hacky + if ( columnName == null || origCol == null ) { + return null; + } + + return new HColumnImpl(subColumnName, + valueSerializer.fromByteBuffer(origCol.getValue()), origCol.getClock(), + subSerializer, valueSerializer); + } + + public SuperCfResult querySuperColumns(K key, List sColumnNames) { + HSlicePredicate workingSlicePredicate = new HSlicePredicate(topSerializer); + workingSlicePredicate.setColumnNames(sColumnNames); + return doExecuteSlice(key, null, workingSlicePredicate); + } + + public SuperCfResult querySuperColumns(List keys, List sColumnNames) { + HSlicePredicate workingSlicePredicate = new HSlicePredicate(topSerializer); + workingSlicePredicate.setColumnNames(sColumnNames); + return doExecuteMultigetSlice(keys, workingSlicePredicate); + } + + public SuperCfResult querySuperColumns(List keys) { + return doExecuteMultigetSlice(keys, activeSlicePredicate); + } + + public T querySuperColumns(K key, List sColumnNames, + SuperCfRowMapper mapper) { + + HSlicePredicate workingSlicePredicate = new HSlicePredicate(topSerializer); + workingSlicePredicate.setColumnNames(sColumnNames); + return mapper.mapRow(doExecuteSlice(key, null, workingSlicePredicate)); + } + + public SuperCfResult querySuperColumns(K key) { + return doExecuteSlice(key, null, activeSlicePredicate); + } + + public SuperCfResult querySuperColumn(K key, SN sColumnName) { + HSlicePredicate workingSlicePredicate = new HSlicePredicate(topSerializer); + workingSlicePredicate.addColumnName(sColumnName); + return doExecuteSlice(key, sColumnName, workingSlicePredicate); + } + + public SuperCfUpdater createUpdater(K key, SN sColumnName) { + return createUpdater(key).addSuperColumn(sColumnName); + } + + public SuperCfUpdater createUpdater(K key) { + SuperCfUpdater updater = new SuperCfUpdater(this, columnFactory); + updater.addKey(key); + return updater; + } + + public void update(SuperCfUpdater updater) { + updater.updateInternal(); + updater.update(); + executeIfNotBatched(); + } + + /** + * Immediately delete the key and superColumn combination + */ + @Override + public void deleteColumn(K key, SN sColumnName) { + mutator.superDelete(key, getColumnFamily(), sColumnName, topSerializer); + } + + /** + * Immediately delete the row + */ + @Override + public void deleteRow(K key) { + mutator.delete(key, getColumnFamily(), null, null); + } + + protected abstract SuperCfResult doExecuteSlice(K key, SN sColumnName, HSlicePredicate predicate); + + protected abstract SuperCfResult doExecuteMultigetSlice(List keys, HSlicePredicate predicate); + + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfUpdater.java b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfUpdater.java new file mode 100644 index 000000000..4cf52ef5c --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/SuperCfUpdater.java @@ -0,0 +1,184 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Date; +import java.util.List; +import java.util.UUID; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import me.prettyprint.cassandra.model.HSuperColumnImpl; +import me.prettyprint.cassandra.serializers.BooleanSerializer; +import me.prettyprint.cassandra.serializers.ByteBufferSerializer; +import me.prettyprint.cassandra.serializers.BytesArraySerializer; +import me.prettyprint.cassandra.serializers.DateSerializer; +import me.prettyprint.cassandra.serializers.DoubleSerializer; +import me.prettyprint.cassandra.serializers.IntegerSerializer; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.serializers.TypeInferringSerializer; +import me.prettyprint.cassandra.serializers.UUIDSerializer; +import me.prettyprint.hector.api.ColumnFactory; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.factory.HFactory; + +/** + * This provides an interface of updating a specified row, most likely with the + * contents of an object. This would likely by implemented as an anonymous inner + * class with access to a final object in scope. It would update the given row + * with the object's data. + * + * This is currently implemented as an abstract base class instead of an + * interface. This could change in the future. Being an abstract base class + * allows CassandraTemplate to initialize this instance through package scope + * field access. This means that implementation of update() simply makes + * consecutive calls to various set****() methods which already have the + * contextual information they need to update the correct row. + * + * The downside of this approach is that the updater is essentially stateful and + * cannot be used concurrently. The alternative is to pass an object in to + * update() as a parameter with the setter methods, leaving the updater to be + * stateless. + * + * @author david + * @since Mar 10, 2011 + * @param + * the key's data type + * @param + * the standard or super column's data type + * @param + * the child column name type in a super column + * @param + * the object instance to persist + */ +public class SuperCfUpdater extends AbstractTemplateUpdater { + private static final Logger log = LoggerFactory.getLogger(SuperCfUpdater.class); + protected SuperCfTemplate template; + private List sColumnNames; + private int sColPos; + private HSuperColumnImpl activeColumn; + private List subColumns; + + public SuperCfUpdater(SuperCfTemplate sTemplate, ColumnFactory columnFactory) { + super((AbstractColumnFamilyTemplate) sTemplate, columnFactory); + this.template = sTemplate; + } + + + + @Override + public SuperCfUpdater addKey(K key) { + + if ( keys != null && keys.size() > 0 ) { + updateInternal(); + } + super.addKey(key); + sColumnNames = new ArrayList(); + sColPos = 0; + return this; + } + + + + public SuperCfUpdater addSuperColumn(SN sColumnName) { + if ( sColumnNames.size() > 0 ) { + updateInternal(); + sColPos++; + } + + subColumns = new ArrayList(); + + sColumnNames.add(sColumnName); + + + return this; + } + + public SN getCurrentSuperColumn() { + return sColumnNames.get(sColPos); + } + + /** + * collapse the state of the active HSuperColumn + */ + void updateInternal() { + // HSuperColumnImpl needs a refactor, this construction is lame. + // the value serializer is not used in HSuperColumnImpl, so this is safe for name + if ( !subColumns.isEmpty() ) { + log.debug("Adding column {} for key {} and cols {}", new Object[]{getCurrentSuperColumn(), getCurrentKey(), subColumns}); + HSuperColumnImpl column = new HSuperColumnImpl(getCurrentSuperColumn(), subColumns, + 0, template.getTopSerializer(), template.getSubSerializer(), TypeInferringSerializer.get()); + template.getMutator().addInsertion(getCurrentKey(), template.getColumnFamily(), column); + } + + } + + /** + * Deletes the super column and all of its sub columns + */ + public void deleteSuperColumn() { + //template.getMutator().addDeletion(getCurrentKey(), template.getColumnFamily(), + // getCurrentSuperColumn(), template.getTopSerializer()); + template.getMutator().addSuperDelete(getCurrentKey(), template.getColumnFamily(), + getCurrentSuperColumn(), template.getTopSerializer()); + } + + public void deleteSubColumn(N columnName) { + template.getMutator().addSubDelete(getCurrentKey(), template.getColumnFamily(), + getCurrentSuperColumn(), columnName, template.getTopSerializer(), template.getSubSerializer()); + } + + public void setString(N subColumnName, String value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), StringSerializer.get() )); + } + + public void setUUID(N subColumnName, UUID value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), UUIDSerializer.get())); + } + + public void setLong(N subColumnName, Long value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), LongSerializer.get())); + } + + public void setInteger(N subColumnName, Integer value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), IntegerSerializer.get())); + } + + public void setBoolean(N subColumnName, Boolean value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), BooleanSerializer.get())); + } + + public void setByteArray(N subColumnName, byte[] value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), BytesArraySerializer.get())); + } + + public void setByteBuffer(N subColumnName, ByteBuffer value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), ByteBufferSerializer.get())); + } + + public void setDate(N subColumnName, Date value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), DateSerializer.get())); + } + + public void setDouble(N subColumnName, Double value) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), DoubleSerializer.get())); + } + + public void setValue(N subColumnName, V value, Serializer serializer) { + subColumns.add(columnFactory.createColumn(subColumnName, value, template.getEffectiveClock(), + template.getSubSerializer(), serializer)); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ThriftColumnFamilyTemplate.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ThriftColumnFamilyTemplate.java new file mode 100644 index 000000000..a08bf22cc --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ThriftColumnFamilyTemplate.java @@ -0,0 +1,110 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.model.ExecutingKeyspace; +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.cassandra.model.HSlicePredicate; +import me.prettyprint.cassandra.model.thrift.ThriftConverter; +import me.prettyprint.cassandra.service.Operation; +import me.prettyprint.cassandra.service.OperationType; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.exceptions.HectorException; +import me.prettyprint.hector.api.mutation.Mutator; + +import org.apache.cassandra.thrift.Cassandra; +import org.apache.cassandra.thrift.ColumnOrSuperColumn; + +import com.google.common.collect.Iterators; + +/** + * Thrift specific implementation of {@link ColumnFamilyTemplate} + * @author nate + * + * @param + * @param + */ +public class ThriftColumnFamilyTemplate extends ColumnFamilyTemplate { + + + public ThriftColumnFamilyTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer) { + super(keyspace, columnFamily, keySerializer, topSerializer); + } + + public ThriftColumnFamilyTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer, + Mutator mutator) { + super(keyspace, columnFamily, keySerializer, topSerializer, mutator); + } + + public T doExecuteSlice(K key, HSlicePredicate predicate, ColumnFamilyRowMapper mapper) { + return mapper.mapRow(doExecuteSlice(key,predicate)); + } + + public ColumnFamilyResult doExecuteSlice(final K key, final HSlicePredicate workingSlicePredicate) { + return new ColumnFamilyResultWrapper(keySerializer, topSerializer, + sliceInternal(key, workingSlicePredicate)); + } + + public ColumnFamilyResult doExecuteMultigetSlice(final Iterable keys, final HSlicePredicate workingSlicePredicate) { + return new ColumnFamilyResultWrapper(keySerializer, topSerializer, + multigetSliceInternal(keys, workingSlicePredicate)); + } + + public MappedColumnFamilyResult doExecuteMultigetSlice(final Iterable keys, + final HSlicePredicate workingSlicePredicate, + final ColumnFamilyRowMapper mapper) { + return new MappedColumnFamilyResultWrapper(keySerializer, topSerializer, + multigetSliceInternal(keys, workingSlicePredicate), mapper); + } + + + private ExecutionResult>> sliceInternal(final K key, + final HSlicePredicate workingSlicePredicate) { + return ((ExecutingKeyspace)keyspace).doExecuteOperation(new Operation>>(OperationType.READ) { + @Override + public Map> execute(Cassandra.Client cassandra) throws HectorException { + Map> cosc = new LinkedHashMap>(); + try { + + ByteBuffer sKey = keySerializer.toByteBuffer(key); + cosc.put(sKey, cassandra.get_slice(sKey, columnParent, + workingSlicePredicate.toThrift(), + ThriftConverter.consistencyLevel(consistencyLevelPolicy.get(operationType)))); + + } catch (Exception e) { + throw exceptionsTranslator.translate(e); + } + + return cosc; + } + }); + } + + private ExecutionResult>> multigetSliceInternal(final Iterable keys, + final HSlicePredicate workingSlicePredicate) { + return ((ExecutingKeyspace)keyspace).doExecuteOperation(new Operation>>(OperationType.READ) { + @Override + public Map> execute(Cassandra.Client cassandra) throws HectorException { + Map> cosc = new LinkedHashMap>(); + try { + List keyList = new ArrayList(); + Iterators.addAll(keyList, keys.iterator()); + cosc = cassandra.multiget_slice(keySerializer.toBytesList(keyList), columnParent, + (workingSlicePredicate == null ? activeSlicePredicate.setColumnNames(columnValueSerializers.keySet()).toThrift() : workingSlicePredicate.toThrift()), + ThriftConverter.consistencyLevel(consistencyLevelPolicy.get(operationType))); + } catch (Exception e) { + throw exceptionsTranslator.translate(e); + } + + return cosc; + } + }); + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/service/template/ThriftSuperCfTemplate.java b/core/src/main/java/me/prettyprint/cassandra/service/template/ThriftSuperCfTemplate.java new file mode 100644 index 000000000..e3a725405 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/service/template/ThriftSuperCfTemplate.java @@ -0,0 +1,92 @@ +package me.prettyprint.cassandra.service.template; + +import java.nio.ByteBuffer; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; + +import org.apache.cassandra.thrift.Cassandra; +import org.apache.cassandra.thrift.ColumnOrSuperColumn; +import org.apache.cassandra.thrift.ColumnParent; + +import me.prettyprint.cassandra.model.ExecutingKeyspace; +import me.prettyprint.cassandra.model.ExecutionResult; +import me.prettyprint.cassandra.model.HSlicePredicate; +import me.prettyprint.cassandra.model.thrift.ThriftConverter; +import me.prettyprint.cassandra.service.Operation; +import me.prettyprint.cassandra.service.OperationType; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.Serializer; +import me.prettyprint.hector.api.exceptions.HectorException; + +public class ThriftSuperCfTemplate extends SuperCfTemplate { + + public ThriftSuperCfTemplate(Keyspace keyspace, String columnFamily, + Serializer keySerializer, Serializer topSerializer, + Serializer subSerializer) { + super(keyspace, columnFamily, keySerializer, topSerializer, subSerializer); + } + + protected SuperCfResult doExecuteSlice(K key, SN sColumnName, HSlicePredicate predicate) { + SuperCfResultWrapper wrapper = + new SuperCfResultWrapper(keySerializer, topSerializer, subSerializer, + sliceInternal(key, predicate)); + if ( sColumnName != null ) { + wrapper.applySuperColumn(sColumnName); + } + return wrapper; + } + + protected SuperCfResult doExecuteMultigetSlice(List keys, HSlicePredicate predicate) { + SuperCfResultWrapper wrapper = + new SuperCfResultWrapper(keySerializer, topSerializer, subSerializer, + multigetSliceInternal(keys, columnParent, predicate)); + return wrapper; + } + + private ExecutionResult>> sliceInternal(final K key, + final HSlicePredicate workingSlicePredicate) { + return ((ExecutingKeyspace)keyspace).doExecuteOperation(new Operation>>(OperationType.READ) { + @Override + public Map> execute(Cassandra.Client cassandra) throws HectorException { + Map> cosc = new LinkedHashMap>(); + try { + + ByteBuffer sKey = keySerializer.toByteBuffer(key); + cosc.put(sKey, cassandra.get_slice(sKey, columnParent, + workingSlicePredicate.toThrift(), + ThriftConverter.consistencyLevel(consistencyLevelPolicy.get(operationType)))); + + } catch (Exception e) { + throw exceptionsTranslator.translate(e); + } + + return cosc; + } + }); + } + + private ExecutionResult>> multigetSliceInternal(final List keys, + final ColumnParent workingColumnParent, + final HSlicePredicate workingSlicePredicate) { + return ((ExecutingKeyspace)keyspace).doExecuteOperation(new Operation>>(OperationType.READ) { + @Override + public Map> execute(Cassandra.Client cassandra) throws HectorException { + Map> cosc; + try { + + List sKeys = keySerializer.toBytesList(keys); + cosc = cassandra.multiget_slice(sKeys, workingColumnParent, + workingSlicePredicate.toThrift(), + ThriftConverter.consistencyLevel(consistencyLevelPolicy.get(operationType))); + + } catch (Exception e) { + throw exceptionsTranslator.translate(e); + } + + return cosc; + } + }); + } + +} diff --git a/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedSchemaLoader.java b/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedSchemaLoader.java new file mode 100644 index 000000000..d2f47bf8f --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedSchemaLoader.java @@ -0,0 +1,123 @@ +package me.prettyprint.cassandra.testutils; + +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import org.apache.cassandra.config.CFMetaData; +import org.apache.cassandra.config.ColumnDefinition; +import org.apache.cassandra.config.ConfigurationException; +import org.apache.cassandra.config.DatabaseDescriptor; +import org.apache.cassandra.config.KSMetaData; +import org.apache.cassandra.db.ColumnFamilyType; +import org.apache.cassandra.db.marshal.AbstractType; +import org.apache.cassandra.db.marshal.AsciiType; +import org.apache.cassandra.db.marshal.BytesType; +import org.apache.cassandra.db.marshal.CounterColumnType; +import org.apache.cassandra.db.marshal.IntegerType; +import org.apache.cassandra.db.marshal.LongType; +import org.apache.cassandra.db.marshal.TimeUUIDType; +import org.apache.cassandra.db.marshal.UTF8Type; +import org.apache.cassandra.locator.AbstractReplicationStrategy; +import org.apache.cassandra.locator.SimpleStrategy; +import org.apache.cassandra.thrift.IndexType; + +import com.google.common.base.Charsets; + +public class EmbeddedSchemaLoader { + public static void loadSchema() { + try { + for (KSMetaData ksm : schemaDefinition()) { + for (CFMetaData cfm : ksm.cfMetaData().values()) + CFMetaData.map(cfm); + DatabaseDescriptor.setTableDefinition(ksm, DatabaseDescriptor + .getDefsVersion()); + } + } catch (ConfigurationException e) { + throw new RuntimeException(e); + } + } + + public static Collection schemaDefinition() { + List schema = new ArrayList(); + + // A whole bucket of shorthand + String ks1 = "Keyspace1"; + String ks2 = "Keyspace2"; + + Class simple = SimpleStrategy.class; + Map opts = new HashMap(); + opts.put("replication_factor", Integer.toString(1)); + + ColumnFamilyType st = ColumnFamilyType.Standard; + ColumnFamilyType su = ColumnFamilyType.Super; + AbstractType bytes = BytesType.instance; + + // Keyspace 1 + schema.add(new KSMetaData( + ks1, + simple, + opts, + + // Column Families + standardCFMD(ks1, "Standard1"), standardCFMD(ks1, "Standard2"), + standardCFMD(ks1, "Standard3"), standardCFMD(ks1, "Standard4"), + standardCFMD(ks1, "StandardLong1").keyValidator(UTF8Type.instance), + standardCFMD(ks1, "StandardLong2"), + superCFMD(ks1, "Super1", BytesType.instance), superCFMD(ks1, "Super2", + LongType.instance), superCFMD(ks1, "Super3", LongType.instance), + superCFMD(ks1, "Super4", UTF8Type.instance), superCFMD(ks1, "Super5", + bytes), indexCFMD(ks1, "Indexed1", true), indexCFMD(ks1, + "Indexed2", false), new CFMetaData(ks1, "StandardInteger1", st, + IntegerType.instance, null).keyCacheSize(0), new CFMetaData(ks1, + "Counter1", st, bytes, null).replicateOnWrite(true) + .defaultValidator(CounterColumnType.instance), new CFMetaData(ks1, + "SuperCounter1", su, bytes, bytes).replicateOnWrite(true) + .defaultValidator(CounterColumnType.instance), jdbcCFMD(ks1, + "JdbcInteger", IntegerType.instance), jdbcCFMD(ks1, "JdbcUtf8", + UTF8Type.instance), jdbcCFMD(ks1, "JdbcLong", LongType.instance), + jdbcCFMD(ks1, "JdbcBytes", bytes), jdbcCFMD(ks1, "JdbcAscii", + AsciiType.instance))); + + // Keyspace 2 + + return schema; + } + + private static CFMetaData standardCFMD(String ksName, String cfName) { + return new CFMetaData(ksName, cfName, ColumnFamilyType.Standard, + BytesType.instance, null).keyCacheSize(0); + } + + private static CFMetaData superCFMD(String ksName, String cfName, + AbstractType subcc) { + return new CFMetaData(ksName, cfName, ColumnFamilyType.Super, + BytesType.instance, subcc).keyCacheSize(0); + } + + private static CFMetaData indexCFMD(String ksName, String cfName, + final Boolean withIdxType) { + return standardCFMD(ksName, cfName).columnMetadata( + Collections + .unmodifiableMap(new HashMap() { + { + ByteBuffer cName = ByteBuffer.wrap("birthyear" + .getBytes(Charsets.UTF_8)); + IndexType keys = withIdxType ? IndexType.KEYS : null; + put(cName, new ColumnDefinition(cName, LongType.instance, keys, + null)); + } + })); + } + + private static CFMetaData jdbcCFMD(String ksName, String cfName, + AbstractType comp) { + return new CFMetaData(ksName, cfName, ColumnFamilyType.Standard, comp, null) + .defaultValidator(comp); + } +} + diff --git a/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedServerHelper.java b/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedServerHelper.java index 05fb1b973..64af7cc77 100644 --- a/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedServerHelper.java +++ b/core/src/main/java/me/prettyprint/cassandra/testutils/EmbeddedServerHelper.java @@ -23,9 +23,9 @@ import org.slf4j.LoggerFactory; /** - * + * * @author Ran Tavory (rantav@gmail.com) - * + * */ public class EmbeddedServerHelper { private static Logger log = LoggerFactory.getLogger(EmbeddedServerHelper.class); @@ -35,7 +35,7 @@ public class EmbeddedServerHelper { private EmbeddedCassandraService cassandra; private final String yamlFile; static CassandraDaemon cassandraDaemon; - + public EmbeddedServerHelper() { this("/cassandra.yaml"); } @@ -43,12 +43,12 @@ public EmbeddedServerHelper() { public EmbeddedServerHelper(String yamlFile) { this.yamlFile = yamlFile; } - - static ExecutorService executor = Executors.newSingleThreadExecutor(); + + static ExecutorService executor = Executors.newSingleThreadExecutor(); /** * Set embedded cassandra up and spawn it in a new thread. - * + * * @throws TTransportException * @throws IOException * @throws InterruptedException @@ -68,7 +68,7 @@ public void setup() throws TTransportException, IOException, loadSchemaFromYaml(); //loadYamlTables(); log.info("Starting executor"); - + executor.execute(new CassandraRunner()); log.info("Started executor"); try @@ -82,20 +82,7 @@ public void setup() throws TTransportException, IOException, } } - /** - * Manually load tables from the test configuration file. - * - * @throws ConfigurationException - */ - private void loadYamlTables() throws ConfigurationException { - for (KSMetaData table : DatabaseDescriptor.readTablesFromYaml()) { - for (CFMetaData cfm : table.cfMetaData().values()) { - CFMetaData.map(cfm); - } - DatabaseDescriptor.setTableDefinition(table, - DatabaseDescriptor.getDefsVersion()); - } - } + public static void teardown() { //if ( cassandraDaemon != null ) @@ -114,7 +101,7 @@ private static void rmdir(String dir) throws IOException { /** * Copies a resource from within the jar to a directory. - * + * * @param resource * @param directory * @throws IOException @@ -138,14 +125,14 @@ private static void copy(String resource, String directory) /** * Creates a directory - * + * * @param dir * @throws IOException */ private static void mkdir(String dir) throws IOException { FileUtils.createDirectory(dir); } - + public static void cleanupAndLeaveDirs() throws IOException { @@ -187,36 +174,25 @@ public static void mkdirs() { throw new RuntimeException(e); } - } - + } + public static void loadSchemaFromYaml() { - try - { - for (KSMetaData ksm : DatabaseDescriptor.readTablesFromYaml()) - { - for (CFMetaData cfm : ksm.cfMetaData().values()) - CFMetaData.map(cfm); - DatabaseDescriptor.setTableDefinition(ksm, DatabaseDescriptor.getDefsVersion()); - } - } - catch (ConfigurationException e) - { - throw new RuntimeException(e); - } - } + EmbeddedSchemaLoader.loadSchema(); + + } + + + class CassandraRunner implements Runnable { - - class CassandraRunner implements Runnable { - @Override public void run() { cassandraDaemon = new CassandraDaemon(); - + cassandraDaemon.activate(); } - + } } diff --git a/core/src/main/java/me/prettyprint/cassandra/utils/ByteBufferOutputStream.java b/core/src/main/java/me/prettyprint/cassandra/utils/ByteBufferOutputStream.java new file mode 100644 index 000000000..4cf28fd82 --- /dev/null +++ b/core/src/main/java/me/prettyprint/cassandra/utils/ByteBufferOutputStream.java @@ -0,0 +1,161 @@ +package me.prettyprint.cassandra.utils; + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import java.io.OutputStream; +import java.nio.ByteBuffer; +import java.util.LinkedList; +import java.util.List; + +/** + * Utility to collect data written to an {@link OutputStream} in + * {@link ByteBuffer}s. + * + * Originally from org.apache.avro.util.ByteBufferOutputStream, moved into + * Hector and added getByteBuffer to return single ByteBuffer from contents. + */ +public class ByteBufferOutputStream extends OutputStream { + public static final int BUFFER_SIZE = 8192; + + private List buffers; + + public ByteBufferOutputStream() { + reset(); + } + + /** Returns all data written and resets the stream to be empty. */ + public List getBufferList() { + List result = buffers; + reset(); + for (ByteBuffer buffer : result) { + buffer.flip(); + } + return result; + } + + public ByteBuffer getByteBuffer() { + List list = getBufferList(); + // if there's just one bytebuffer in list, return it + if (list.size() == 1) { + return list.get(0); + } + int size = 0; + for (ByteBuffer buffer : list) { + size += buffer.remaining(); + } + ByteBuffer result = ByteBuffer.allocate(size); + for (ByteBuffer buffer : list) { + result.put(buffer); + } + return (ByteBuffer) result.rewind(); + } + + /** Prepend a list of ByteBuffers to this stream. */ + public void prepend(List lists) { + for (ByteBuffer buffer : lists) { + buffer.position(buffer.limit()); + } + buffers.addAll(0, lists); + } + + /** Append a list of ByteBuffers to this stream. */ + public void append(List lists) { + for (ByteBuffer buffer : lists) { + buffer.position(buffer.limit()); + } + buffers.addAll(lists); + } + + public void reset() { + buffers = new LinkedList(); + buffers.add(ByteBuffer.allocate(BUFFER_SIZE)); + } + + private ByteBuffer getBufferWithCapacity(int capacity) { + ByteBuffer buffer = buffers.get(buffers.size() - 1); + if (buffer.remaining() < capacity) { + buffer = ByteBuffer.allocate(BUFFER_SIZE); + buffers.add(buffer); + } + return buffer; + } + + @Override + public void write(int b) { + ByteBuffer buffer = getBufferWithCapacity(1); + buffer.put((byte) b); + } + + public void writeShort(short value) { + ByteBuffer buffer = getBufferWithCapacity(2); + buffer.putShort(value); + } + + public void writeChar(char value) { + ByteBuffer buffer = getBufferWithCapacity(2); + buffer.putChar(value); + } + + public void writeInt(int value) { + ByteBuffer buffer = getBufferWithCapacity(4); + buffer.putInt(value); + } + + public void writeFloat(float value) { + ByteBuffer buffer = getBufferWithCapacity(4); + buffer.putFloat(value); + } + + public void writeLong(long value) { + ByteBuffer buffer = getBufferWithCapacity(8); + buffer.putLong(value); + } + + public void writeDouble(double value) { + ByteBuffer buffer = getBufferWithCapacity(8); + buffer.putDouble(value); + } + + @Override + public void write(byte[] b, int off, int len) { + ByteBuffer buffer = buffers.get(buffers.size() - 1); + int remaining = buffer.remaining(); + while (len > remaining) { + buffer.put(b, off, remaining); + len -= remaining; + off += remaining; + buffer = ByteBuffer.allocate(BUFFER_SIZE); + buffers.add(buffer); + remaining = buffer.remaining(); + } + buffer.put(b, off, len); + } + + /** Add a buffer to the output without copying, if possible. */ + public void write(ByteBuffer buffer) { + if (buffer.remaining() < BUFFER_SIZE) { + write(buffer.array(), buffer.arrayOffset() + buffer.position(), + buffer.remaining()); + } else { // append w/o copying bytes + ByteBuffer dup = buffer.duplicate(); + dup.position(buffer.limit()); // ready for flip + buffers.add(dup); + } + } +} diff --git a/core/src/main/java/me/prettyprint/cassandra/utils/TimeUUIDUtils.java b/core/src/main/java/me/prettyprint/cassandra/utils/TimeUUIDUtils.java index e683f57ae..e4536fda2 100644 --- a/core/src/main/java/me/prettyprint/cassandra/utils/TimeUUIDUtils.java +++ b/core/src/main/java/me/prettyprint/cassandra/utils/TimeUUIDUtils.java @@ -15,6 +15,8 @@ * */ public final class TimeUUIDUtils { + + static final long NUM_100NS_INTERVALS_SINCE_UUID_EPOCH = 0x01b21dd213814000L; /** * Gets a new and unique time uuid in milliseconds. It is useful to use in a TimeUUIDType sorted column family. @@ -54,7 +56,7 @@ private static long createTime(long currentTime) { long time; // UTC time - long timeToUse = (currentTime * 10000) + 0x01B21DD213814000L; + long timeToUse = (currentTime * 10000) + NUM_100NS_INTERVALS_SINCE_UUID_EPOCH; // time low time = timeToUse << 32; @@ -86,7 +88,11 @@ public static java.util.UUID toUUID(byte[] uuid) { * @return a long representing the time */ public static long getTimeFromUUID(byte[] uuid) { - return TimeUUIDUtils.toUUID(uuid).timestamp(); + return getTimeFromUUID(TimeUUIDUtils.toUUID(uuid)); + } + + public static long getTimeFromUUID(UUID uuid) { + return (uuid.timestamp() - NUM_100NS_INTERVALS_SINCE_UUID_EPOCH) / 10000; } /** diff --git a/core/src/main/java/me/prettyprint/hector/api/Cluster.java b/core/src/main/java/me/prettyprint/hector/api/Cluster.java index e95b43a0e..fdeeeabff 100644 --- a/core/src/main/java/me/prettyprint/hector/api/Cluster.java +++ b/core/src/main/java/me/prettyprint/hector/api/Cluster.java @@ -42,6 +42,8 @@ public interface Cluster { String describeThriftVersion() throws HectorException; + Map> describeSchemaVersions() throws HectorException; + KeyspaceDefinition describeKeyspace(final String keyspace) throws HectorException; List describeKeyspaces() throws HectorException; @@ -63,11 +65,11 @@ public interface Cluster { String dropColumnFamily(final String keyspaceName, final String columnFamily) throws HectorException; - + String updateColumnFamily(final ColumnFamilyDefinition cfdef) throws HectorException; String addKeyspace(final KeyspaceDefinition ksdef) throws HectorException; - + void truncate(final String keyspaceName, final String columnFamily) throws HectorException; Map getCredentials(); diff --git a/core/src/main/java/me/prettyprint/hector/api/ColumnFactory.java b/core/src/main/java/me/prettyprint/hector/api/ColumnFactory.java new file mode 100644 index 000000000..ccaf6a820 --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/ColumnFactory.java @@ -0,0 +1,11 @@ +package me.prettyprint.hector.api; + +import me.prettyprint.hector.api.beans.HColumn; + +public interface ColumnFactory { + HColumn createColumn(N name, V value, Serializer nameSerializer, Serializer valueSerializer); + + HColumn createColumn(N name, V value, long clock, Serializer nameSerializer, Serializer valueSerializer); + + HColumn createStringColumn(String name, String value); +} diff --git a/core/src/main/java/me/prettyprint/hector/api/HColumnFamily.java b/core/src/main/java/me/prettyprint/hector/api/HColumnFamily.java index f1bf0c018..a6be7961b 100644 --- a/core/src/main/java/me/prettyprint/hector/api/HColumnFamily.java +++ b/core/src/main/java/me/prettyprint/hector/api/HColumnFamily.java @@ -12,45 +12,45 @@ public interface HColumnFamily extends Iterator>,ResultStatus { HColumnFamily setReadConsistencyLevel(HConsistencyLevel readLevel); - + HColumnFamily setWriteConsistencyLevel(HConsistencyLevel writeLevel); - + HColumnFamily addKey(K key); - + HColumnFamily addKeys(Collection keys); - - HColumnFamily removeKeys(); - + + HColumnFamily removeKeys(); + HColumnFamily clear(); - + HColumnFamily setStart(N name); - + HColumnFamily setFinish(N name); - + HColumnFamily setCount(int count); - + HColumnFamily setReversed(boolean reversed); - + HColumnFamily setColumnNames(Collection columnNames); - + HColumnFamily addColumnName(N columnName); - + Collection> getColumns(); - + HColumn getColumn(N name); - + String getString(N name); - + int getInt(N name); - + long getLong(N name); - + UUID getUUID(N name); - + Date getDate(N name); - + double getDouble(N name); - + V getValue(N name, Serializer valueSerializer); - + } diff --git a/core/src/main/java/me/prettyprint/hector/api/HConsistencyLevel.java b/core/src/main/java/me/prettyprint/hector/api/HConsistencyLevel.java index c475bfe20..af8ba4c51 100644 --- a/core/src/main/java/me/prettyprint/hector/api/HConsistencyLevel.java +++ b/core/src/main/java/me/prettyprint/hector/api/HConsistencyLevel.java @@ -4,7 +4,8 @@ * This is the Hector consistency level which is just a mirror of the thrift or avro consistency * levels. * @author: peter + * @author zznate */ public enum HConsistencyLevel { - ONE, QUORUM, ALL, ANY, EACH_QUORUM, LOCAL_QUORUM; + ONE, TWO, THREE, QUORUM, ALL, ANY, EACH_QUORUM, LOCAL_QUORUM; } diff --git a/core/src/main/java/me/prettyprint/hector/api/Keyspace.java b/core/src/main/java/me/prettyprint/hector/api/Keyspace.java index 5477a84a4..e9a73a083 100644 --- a/core/src/main/java/me/prettyprint/hector/api/Keyspace.java +++ b/core/src/main/java/me/prettyprint/hector/api/Keyspace.java @@ -12,11 +12,11 @@ public interface Keyspace { public static final String KEYSPACE_SYSTEM = "system"; - + void setConsistencyLevelPolicy(ConsistencyLevelPolicy cp); String getKeyspaceName(); - + long createClock(); } \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/ResultStatus.java b/core/src/main/java/me/prettyprint/hector/api/ResultStatus.java index cdab103c3..196f88013 100644 --- a/core/src/main/java/me/prettyprint/hector/api/ResultStatus.java +++ b/core/src/main/java/me/prettyprint/hector/api/ResultStatus.java @@ -13,12 +13,14 @@ public interface ResultStatus { /** * How long the operation took to execute in MICRO-seconds. Internally - * this is usually the difference between two calls of {@link System#nanoTime()} + * this is usually the difference between two calls of {@link System#nanoTime()} * divided by 1000 * @return */ long getExecutionTimeMicro(); + long getExecutionTimeNano(); + /** * The {@link CassandraHost} on which this operation * was successful diff --git a/core/src/main/java/me/prettyprint/hector/api/Serializer.java b/core/src/main/java/me/prettyprint/hector/api/Serializer.java index 1306608a5..996d12264 100644 --- a/core/src/main/java/me/prettyprint/hector/api/Serializer.java +++ b/core/src/main/java/me/prettyprint/hector/api/Serializer.java @@ -5,41 +5,43 @@ import java.util.Map; import java.util.Set; -import me.prettyprint.cassandra.serializers.StringSerializer; - +import me.prettyprint.hector.api.ddl.ComparatorType; /** * Serializes a type T from the given bytes, or vice a versa. * - * In cassandra column names and column values (and starting with 0.7.0 row keys) are all byte[]. - * To allow type safe conversion in java and keep all conversion code in one place we define the - * Extractor interface. - * Implementors of the interface define type conversion according to their domains. A predefined - * set of common extractors can be found in the extractors package, for example - * {@link StringSerializer}. + * In cassandra column names and column values (and starting with 0.7.0 row + * keys) are all byte[]. To allow type safe conversion in java and keep all + * conversion code in one place we define the Extractor interface. Implementors + * of the interface define type conversion according to their domains. A + * predefined set of common extractors can be found in the extractors package, + * for example {@link StringSerializer}. * * @author Ran Tavory * - * @param The type to which data extraction should work. + * @param + * The type to which data extraction should work. */ public interface Serializer { /** * Extract bytes from the obj of type T + * * @param obj * @return - */ - public ByteBuffer toByteBuffer(T obj); + */ + public ByteBuffer toByteBuffer(T obj); public byte[] toBytes(T obj); - + public T fromBytes(byte[] bytes); - + /** * Extract an object of type T from the bytes. + * * @param bytes * @return - */ + */ public T fromByteBuffer(ByteBuffer byteBuffer); public Set toBytesSet(List list); @@ -54,4 +56,6 @@ public interface Serializer { public List fromBytesList(List list); + public ComparatorType getComparatorType(); + } diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/AbstractComposite.java b/core/src/main/java/me/prettyprint/hector/api/beans/AbstractComposite.java new file mode 100644 index 000000000..2d855932e --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/beans/AbstractComposite.java @@ -0,0 +1,724 @@ +package me.prettyprint.hector.api.beans; + +import static me.prettyprint.hector.api.ddl.ComparatorType.ASCIITYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.BYTESTYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.INTEGERTYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.LEXICALUUIDTYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.LONGTYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.TIMEUUIDTYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.UTF8TYPE; +import static me.prettyprint.hector.api.ddl.ComparatorType.UUIDTYPE; + +import java.math.BigInteger; +import java.nio.ByteBuffer; +import java.nio.charset.CharacterCodingException; +import java.util.AbstractList; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Iterator; +import java.util.List; +import java.util.Map; + +import me.prettyprint.cassandra.serializers.AsciiSerializer; +import me.prettyprint.cassandra.serializers.BigIntegerSerializer; +import me.prettyprint.cassandra.serializers.ByteBufferSerializer; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.serializers.SerializerTypeInferer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.serializers.UUIDSerializer; +import me.prettyprint.cassandra.utils.ByteBufferOutputStream; +import me.prettyprint.hector.api.Serializer; + +import org.apache.cassandra.utils.ByteBufferUtil; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.google.common.collect.BiMap; +import com.google.common.collect.ImmutableBiMap; +import com.google.common.collect.ImmutableClassToInstanceMap; + +/** + * Parent class of Composite and DynamicComposite. Acts as a list of objects + * that get serialized into a composite column name. Unless + * setAutoDeserialize(true) is called, it's going to try to match serializers to + * Cassandra comparator types. + * + * @author edanuff + */ +@SuppressWarnings("rawtypes") +public abstract class AbstractComposite extends AbstractList implements + Comparable { + + private static Logger log = LoggerFactory.getLogger(AbstractComposite.class); + + public enum ComponentEquality { + LESS_THAN_EQUAL((byte) -1), EQUAL((byte) 0), GREATER_THAN_EQUAL((byte) 1); + + private final byte equality; + + ComponentEquality(byte equality) { + this.equality = equality; + } + + public byte toByte() { + return equality; + } + + public static ComponentEquality fromByte(byte equality) { + if (equality > 0) { + return GREATER_THAN_EQUAL; + } + if (equality < 0) { + return LESS_THAN_EQUAL; + } + return EQUAL; + } + } + + public static final BiMap, String> DEFAULT_SERIALIZER_TO_COMPARATOR_MAPPING = new ImmutableBiMap.Builder, String>() + .put(AsciiSerializer.class, + AsciiSerializer.get().getComparatorType().getTypeName()) + .put(BigIntegerSerializer.class, + BigIntegerSerializer.get().getComparatorType().getTypeName()) + .put(LongSerializer.class, + LongSerializer.get().getComparatorType().getTypeName()) + .put(StringSerializer.class, + StringSerializer.get().getComparatorType().getTypeName()) + .put(UUIDSerializer.class, + UUIDSerializer.get().getComparatorType().getTypeName()).build(); + + static final ImmutableClassToInstanceMap SERIALIZERS = new ImmutableClassToInstanceMap.Builder() + .put(AsciiSerializer.class, AsciiSerializer.get()) + .put(BigIntegerSerializer.class, BigIntegerSerializer.get()) + .put(ByteBufferSerializer.class, ByteBufferSerializer.get()) + .put(LongSerializer.class, LongSerializer.get()) + .put(StringSerializer.class, StringSerializer.get()) + .put(UUIDSerializer.class, UUIDSerializer.get()).build(); + + public static final BiMap DEFAULT_ALIAS_TO_COMPARATOR_MAPPING = new ImmutableBiMap.Builder() + .put((byte) 'a', ASCIITYPE.getTypeName()) + .put((byte) 'b', BYTESTYPE.getTypeName()) + .put((byte) 'i', INTEGERTYPE.getTypeName()) + .put((byte) 'x', LEXICALUUIDTYPE.getTypeName()) + .put((byte) 'l', LONGTYPE.getTypeName()) + .put((byte) 't', TIMEUUIDTYPE.getTypeName()) + .put((byte) 's', UTF8TYPE.getTypeName()) + .put((byte) 'u', UUIDTYPE.getTypeName()).build(); + + BiMap, String> serializerToComparatorMapping = DEFAULT_SERIALIZER_TO_COMPARATOR_MAPPING; + + BiMap aliasToComparatorMapping = DEFAULT_ALIAS_TO_COMPARATOR_MAPPING; + + final boolean dynamic; + + List> serializersByPosition = null; + List comparatorsByPosition = null; + + public class Component { + final Serializer serializer; + final T value; + final ByteBuffer bytes; + final String comparator; + final ComponentEquality equality; + + public Component(T value, ByteBuffer bytes, Serializer serializer, + String comparator, ComponentEquality equality) { + this.serializer = serializer; + this.value = value; + this.bytes = bytes; + this.comparator = comparator; + this.equality = equality; + } + + public Serializer getSerializer() { + return serializer; + } + + @SuppressWarnings("unchecked") + public A getValue(Serializer s) { + if (s == null) { + s = (Serializer) serializer; + } + if ((value == null) && (bytes != null) && (s != null)) { + ByteBuffer cb = bytes.duplicate(); + if (cb.hasRemaining()) { + return s.fromByteBuffer(cb); + } + } + if (value instanceof ByteBuffer) { + return (A) ((ByteBuffer) value).duplicate(); + } + return (A) value; + } + + public T getValue() { + return getValue(serializer); + } + + @SuppressWarnings("unchecked") + public ByteBuffer getBytes(Serializer s) { + if (bytes == null) { + if (value instanceof ByteBuffer) { + return ((ByteBuffer) value).duplicate(); + } + + if (value == null) { + return null; + } + + if (s == null) { + s = (Serializer) serializer; + } + if (s != null) { + return s.toByteBuffer((A) value).duplicate(); + } + + } + + return bytes.duplicate(); + } + + public ByteBuffer getBytes() { + return getBytes(serializer); + } + + public String getComparator() { + return comparator; + } + + public ComponentEquality getEquality() { + return equality; + } + + @Override + public String toString() { + return "Component [" + getValue() + "]"; + } + } + + List> components = new ArrayList>(); + + ByteBuffer serialized = null; + + public AbstractComposite(boolean dynamic) { + this.dynamic = dynamic; + } + + public AbstractComposite(boolean dynamic, Object... o) { + this.dynamic = dynamic; + this.addAll(Arrays.asList(o)); + } + + public AbstractComposite(boolean dynamic, List l) { + this.dynamic = dynamic; + this.addAll(l); + } + + public List> getComponents() { + return components; + } + + public void setComponents(List> components) { + serialized = null; + this.components = components; + } + + public Map, String> getSerializerToComparatorMapping() { + return serializerToComparatorMapping; + } + + public void setSerializerToComparatorMapping( + Map, String> serializerToComparatorMapping) { + serialized = null; + this.serializerToComparatorMapping = new ImmutableBiMap.Builder, String>() + .putAll(serializerToComparatorMapping).build(); + } + + public Map getAliasesToComparatorMapping() { + return aliasToComparatorMapping; + } + + public void setAliasesToComparatorMapping( + Map aliasesToComparatorMapping) { + serialized = null; + aliasToComparatorMapping = new ImmutableBiMap.Builder() + .putAll(aliasesToComparatorMapping).build(); + } + + public boolean isDynamic() { + return dynamic; + } + + public List> getSerializersByPosition() { + return serializersByPosition; + } + + public void setSerializersByPosition(List> serializersByPosition) { + this.serializersByPosition = serializersByPosition; + } + + public void setSerializersByPosition(Serializer... serializers) { + serializersByPosition = Arrays.asList(serializers); + } + + public void setSerializerByPosition(int index, Serializer s) { + if (serializersByPosition == null) { + serializersByPosition = new ArrayList>(); + } + while (serializersByPosition.size() <= index) { + serializersByPosition.add(null); + } + serializersByPosition.set(index, s); + } + + public List getComparatorsByPosition() { + return comparatorsByPosition; + } + + public void setComparatorsByPosition(List comparatorsByPosition) { + this.comparatorsByPosition = comparatorsByPosition; + } + + public void setComparatorsByPosition(String... comparators) { + comparatorsByPosition = Arrays.asList(comparators); + } + + public void setComparatorByPosition(int index, String c) { + if (comparatorsByPosition == null) { + comparatorsByPosition = new ArrayList(); + } + while (comparatorsByPosition.size() <= index) { + comparatorsByPosition.add(null); + } + comparatorsByPosition.set(index, c); + } + + @Override + public int compareTo(AbstractComposite o) { + return serialize().compareTo(o.serialize()); + } + + private String comparatorForSerializer(Serializer s) { + String comparator = serializerToComparatorMapping.get(s.getClass()); + if (comparator != null) { + return comparator; + } + return BYTESTYPE.getTypeName(); + } + + private Serializer serializerForComparator(String c) { + int p = c.indexOf('('); + if (p >= 0) { + c = c.substring(0, p); + } + if (LEXICALUUIDTYPE.getTypeName().equals(c) + || TIMEUUIDTYPE.getTypeName().equals(c)) { + return UUIDSerializer.get(); + } + + Serializer s = SERIALIZERS.getInstance(serializerToComparatorMapping + .inverse().get(c)); + if (s != null) { + return s; + } + return ByteBufferSerializer.get(); + } + + private Serializer serializerForPosition(int i) { + if (serializersByPosition == null) { + return null; + } + if (i >= serializersByPosition.size()) { + return null; + } + return serializersByPosition.get(i); + } + + private Serializer getSerializer(int i, String c) { + Serializer s = serializerForPosition(i); + if (s != null) { + return s; + } + return serializerForComparator(c); + } + + private String comparatorForPosition(int i) { + if (comparatorsByPosition == null) { + return null; + } + if (i >= comparatorsByPosition.size()) { + return null; + } + return comparatorsByPosition.get(i); + } + + private String getComparator(int i, ByteBuffer bb) { + String name = comparatorForPosition(i); + if (name != null) { + return name; + } + if (!dynamic) { + if (bb.hasRemaining()) { + return BYTESTYPE.getTypeName(); + } else { + return null; + } + } + if (bb.hasRemaining()) { + try { + int header = getShortLength(bb); + if ((header & 0x8000) == 0) { + name = ByteBufferUtil.string(getBytes(bb, header)); + } else { + byte a = (byte) (header & 0xFF); + name = aliasToComparatorMapping.get(a); + if (name == null) { + a = (byte) Character.toUpperCase((char) a); + name = aliasToComparatorMapping.get(a); + if (name != null) { + name += "(reversed=true)"; + } + } + } + } catch (CharacterCodingException e) { + throw new RuntimeException(e); + } + } + if ((name != null) && (name.length() == 0)) { + name = null; + } + return name; + } + + @Override + public void clear() { + serialized = null; + components = new ArrayList>(); + } + + @Override + public int size() { + return components.size(); + } + + public AbstractComposite addComponent(T value, Serializer s) { + + addComponent(value, s, comparatorForSerializer(s)); + + return this; + + } + + public AbstractComposite addComponent(T value, Serializer s, + String comparator) { + + addComponent(value, s, comparator, ComponentEquality.EQUAL); + + return this; + + } + + public AbstractComposite addComponent(T value, Serializer s, + String comparator, ComponentEquality equality) { + + addComponent(-1, value, s, comparator, equality); + + return this; + + } + + @SuppressWarnings("unchecked") + public AbstractComposite addComponent(int index, T value, + Serializer s, String comparator, ComponentEquality equality) { + serialized = null; + + if (index < 0) { + index = components.size(); + } + + while (components.size() < index) { + components.add(null); + } + components.add(index, new Component(value, null, s, comparator, equality)); + + return this; + + } + + private static Object mapIfNumber(Object o) { + if ((o instanceof Byte) || (o instanceof Integer) || (o instanceof Short)) { + return BigInteger.valueOf(((Number) o).longValue()); + } + return o; + } + + @SuppressWarnings({ "unchecked" }) + private static Collection flatten(Collection c) { + if (c instanceof AbstractComposite) { + return ((AbstractComposite) c).getComponents(); + } + boolean hasCollection = false; + for (Object o : c) { + if (o instanceof Collection) { + hasCollection = true; + break; + } + } + if (!hasCollection) { + return c; + } + List newList = new ArrayList(); + for (Object o : c) { + if (o instanceof Collection) { + newList.addAll(flatten((Collection) o)); + } else { + newList.add(o); + } + } + return newList; + } + + @Override + public boolean addAll(Collection c) { + return super.addAll(flatten(c)); + } + + @Override + public boolean containsAll(Collection c) { + return super.containsAll(flatten(c)); + } + + @Override + public boolean removeAll(Collection c) { + return super.removeAll(flatten(c)); + } + + @Override + public boolean retainAll(Collection c) { + return super.retainAll(flatten(c)); + } + + @Override + public boolean addAll(int i, Collection c) { + return super.addAll(i, flatten(c)); + } + + @SuppressWarnings("unchecked") + @Override + public void add(int index, Object element) { + serialized = null; + + if (element instanceof Component) { + components.add(index, (Component) element); + return; + } + + element = mapIfNumber(element); + Serializer s = serializerForPosition(index); + if (s == null) { + s = SerializerTypeInferer.getSerializer(element); + } + String c = comparatorForPosition(index); + if (c == null) { + c = comparatorForSerializer(s); + } + components.add(index, new Component(element, null, s, c, + ComponentEquality.EQUAL)); + } + + @Override + public Object remove(int index) { + serialized = null; + Component prev = components.remove(index); + if (prev != null) { + return prev.getValue(); + } + return null; + } + + public AbstractComposite setComponent(int index, T value, Serializer s) { + + setComponent(index, value, s, comparatorForSerializer(s)); + + return this; + + } + + public AbstractComposite setComponent(int index, T value, + Serializer s, String comparator) { + + setComponent(index, value, s, comparator, ComponentEquality.EQUAL); + + return this; + + } + + @SuppressWarnings("unchecked") + public AbstractComposite setComponent(int index, T value, + Serializer s, String comparator, ComponentEquality equality) { + serialized = null; + + while (components.size() <= index) { + components.add(null); + } + components.set(index, new Component(value, null, s, comparator, equality)); + + return this; + + } + + @SuppressWarnings("unchecked") + @Override + public Object set(int index, Object element) { + serialized = null; + + if (element instanceof Component) { + Component prev = components.set(index, (Component) element); + if (prev != null) { + return prev.getValue(); + } + return null; + } + + element = mapIfNumber(element); + Serializer s = serializerForPosition(index); + if (s == null) { + s = SerializerTypeInferer.getSerializer(element); + } + String c = comparatorForPosition(index); + if (c == null) { + c = comparatorForSerializer(s); + } + Component prev = components.set(index, new Component(element, null, s, c, + ComponentEquality.EQUAL)); + if (prev != null) { + return prev.getValue(); + } + return null; + } + + @Override + public Object get(int i) { + Component c = components.get(i); + if (c != null) { + return c.getValue(); + } + return null; + } + + public T get(int i, Serializer s) throws ClassCastException { + T value = null; + Component c = components.get(i); + if (c != null) { + value = c.getValue(s); + } + return value; + } + + public Component getComponent(int i) { + if (i >= components.size()) { + return null; + } + Component c = components.get(i); + return c; + } + + public Iterator> componentsIterator() { + return components.iterator(); + } + + @SuppressWarnings("unchecked") + public ByteBuffer serialize() { + if (serialized != null) { + return serialized.duplicate(); + } + + ByteBufferOutputStream out = new ByteBufferOutputStream(); + + int i = 0; + for (Component c : components) { + Serializer s = serializerForPosition(i); + ByteBuffer cb = c.getBytes(s); + if (cb == null) { + cb = ByteBuffer.allocate(0); + } + + if (dynamic) { + String comparator = comparatorForPosition(i); + if (comparator == null) { + comparator = c.getComparator(); + } + if (comparator == null) { + comparator = BYTESTYPE.getTypeName(); + } + int p = comparator.indexOf("(reversed=true)"); + boolean desc = false; + if (p >= 0) { + comparator = comparator.substring(0, p); + desc = true; + } + if (aliasToComparatorMapping.inverse().containsKey(comparator)) { + byte a = aliasToComparatorMapping.inverse().get(comparator); + if (desc) { + a = (byte) Character.toUpperCase((char) a); + } + out.writeShort((short) (0x8000 | a)); + } else { + out.writeShort((short) comparator.length()); + out.write(ByteBufferUtil.bytes(comparator)); + } + if (comparator.equals(BYTESTYPE.getTypeName()) && (cb.remaining() == 0)) { + log.warn("Writing zero-length BytesType, probably an error"); + } + } + out.writeShort((short) cb.remaining()); + out.write(cb.slice()); + out.write(c.getEquality().toByte()); + i++; + } + + serialized = out.getByteBuffer(); + return serialized.duplicate(); + } + + @SuppressWarnings("unchecked") + public void deserialize(ByteBuffer b) { + serialized = b.duplicate(); + components = new ArrayList>(); + + String comparator = null; + int i = 0; + while ((comparator = getComparator(i, b)) != null) { + ByteBuffer data = getWithShortLength(b); + if (data != null) { + Serializer s = getSerializer(i, comparator); + ComponentEquality equality = ComponentEquality.fromByte(b.get()); + components.add(new Component(null, data.slice(), s, comparator, + equality)); + } else { + throw new RuntimeException("Missing component data in composite type"); + } + i++; + } + + } + + protected static int getShortLength(ByteBuffer bb) { + int length = (bb.get() & 0xFF) << 8; + return length | (bb.get() & 0xFF); + } + + protected static ByteBuffer getBytes(ByteBuffer bb, int length) { + ByteBuffer copy = bb.duplicate(); + copy.limit(copy.position() + length); + bb.position(bb.position() + length); + return copy; + } + + protected static ByteBuffer getWithShortLength(ByteBuffer bb) { + int length = getShortLength(bb); + return getBytes(bb, length); + } + +} diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/Composite.java b/core/src/main/java/me/prettyprint/hector/api/beans/Composite.java new file mode 100644 index 000000000..8b7e089e4 --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/beans/Composite.java @@ -0,0 +1,38 @@ +package me.prettyprint.hector.api.beans; + +import java.nio.ByteBuffer; +import java.util.List; + +public class Composite extends AbstractComposite { + + public Composite() { + super(false); + } + + public Composite(Object... o) { + super(false, o); + } + + public Composite(List l) { + super(false, l); + } + + public static Composite fromByteBuffer(ByteBuffer byteBuffer) { + + Composite composite = new Composite(); + composite.deserialize(byteBuffer); + + return composite; + } + + public static ByteBuffer toByteBuffer(Object... o) { + Composite composite = new Composite(o); + return composite.serialize(); + } + + public static ByteBuffer toByteBuffer(List l) { + Composite composite = new Composite(l); + return composite.serialize(); + } + +} diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/CounterSlice.java b/core/src/main/java/me/prettyprint/hector/api/beans/CounterSlice.java new file mode 100644 index 000000000..93383dfbe --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/beans/CounterSlice.java @@ -0,0 +1,21 @@ +package me.prettyprint.hector.api.beans; + +import java.util.List; + + +/** + * A ColumnSlice represents a set of columns as returned by calls such as get_slice + * + * @author patricioe (Patricio Echague) + */ +public interface CounterSlice { + + /** + * + * @return an unmodifiable list of the columns + */ + List> getColumns(); + + HCounterColumn getColumnByName(N columnName); + +} \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/DynamicComposite.java b/core/src/main/java/me/prettyprint/hector/api/beans/DynamicComposite.java new file mode 100644 index 000000000..5d1e5d219 --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/beans/DynamicComposite.java @@ -0,0 +1,39 @@ +package me.prettyprint.hector.api.beans; + +import java.nio.ByteBuffer; +import java.util.List; + +public class DynamicComposite extends AbstractComposite { + + public final static String DEFAULT_DYNAMIC_COMPOSITE_ALIASES = "(a=>AsciiType,b=>BytesType,i=>IntegerType,x=>LexicalUUIDType,l=>LongType,t=>TimeUUIDType,s=>UTF8Type,u=>UUIDType,A=>AsciiType(reversed=true),B=>BytesType(reversed=true),I=>IntegerType(reversed=true),X=>LexicalUUIDType(reversed=true),L=>LongType(reversed=true),T=>TimeUUIDType(reversed=true),S=>UTF8Type(reversed=true),U=>UUIDType(reversed=true))"; + + public DynamicComposite() { + super(true); + } + + public DynamicComposite(Object... o) { + super(true, o); + } + + public DynamicComposite(List l) { + super(true, l); + } + + public static DynamicComposite fromByteBuffer(ByteBuffer byteBuffer) { + + DynamicComposite composite = new DynamicComposite(); + composite.deserialize(byteBuffer); + + return composite; + } + + public static ByteBuffer toByteBuffer(Object... o) { + DynamicComposite composite = new DynamicComposite(o); + return composite.serialize(); + } + + public static ByteBuffer toByteBuffer(List l) { + DynamicComposite composite = new DynamicComposite(l); + return composite.serialize(); + } +} diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/HColumn.java b/core/src/main/java/me/prettyprint/hector/api/beans/HColumn.java index 94a1c3d32..6c6a83e5a 100644 --- a/core/src/main/java/me/prettyprint/hector/api/beans/HColumn.java +++ b/core/src/main/java/me/prettyprint/hector/api/beans/HColumn.java @@ -1,5 +1,7 @@ package me.prettyprint.hector.api.beans; +import java.nio.ByteBuffer; + import me.prettyprint.hector.api.Serializer; /** @@ -9,7 +11,7 @@ * @param The type of the column value * * @author Ran Tavory (rantav@gmail.com) - * + * @author zznate */ public interface HColumn { @@ -21,6 +23,16 @@ public interface HColumn { V getValue(); + /** + * (Advanced) Returns the underlying ByteBuffer for the value via ByteBuffer.duplicate(). + */ + ByteBuffer getValueBytes(); + + /** + * (Advanced) Returns the underlying ByteBuffer for the name via ByteBuffer.duplicate(). + */ + ByteBuffer getNameBytes(); + long getClock(); HColumn setClock(long clock); @@ -28,9 +40,9 @@ public interface HColumn { int getTtl(); HColumn setTtl(int ttl); - + HColumn clear(); - + HColumn apply(V value, long clock, int ttl); Serializer getNameSerializer(); diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/HCounterColumn.java b/core/src/main/java/me/prettyprint/hector/api/beans/HCounterColumn.java new file mode 100644 index 000000000..29958c7bf --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/beans/HCounterColumn.java @@ -0,0 +1,40 @@ +package me.prettyprint.hector.api.beans; + +import java.nio.ByteBuffer; + +import me.prettyprint.hector.api.Serializer; + +/** + * Hector Counter Column definition. + * + * @param The type of the column name + * + * @author patricioe (patricioe@gmail.com) + */ +public interface HCounterColumn { + + HCounterColumn setName(N name); + + HCounterColumn setValue(Long value); + + N getName(); + + Long getValue(); + + + /** + * (Advanced) Returns the underlying ByteBuffer for the name via ByteBuffer.duplicate(). + */ + ByteBuffer getNameBytes(); + + HCounterColumn clear(); + + int getTtl(); + + HCounterColumn setTtl(int ttl); + + HCounterColumn apply(Long value, int ttl); + + Serializer getNameSerializer(); + +} \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/HCounterSuperColumn.java b/core/src/main/java/me/prettyprint/hector/api/beans/HCounterSuperColumn.java new file mode 100644 index 000000000..65c79c1ec --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/beans/HCounterSuperColumn.java @@ -0,0 +1,44 @@ +package me.prettyprint.hector.api.beans; + +import java.nio.ByteBuffer; +import java.util.List; + +import me.prettyprint.hector.api.Serializer; + +/** + * Models a Counter SuperColumn. + * + * @param + * SuperColumn name type + * @param + * Column name type + * + * @author patricioe + */ +public interface HCounterSuperColumn { + + HCounterSuperColumn setName(SN name); + + HCounterSuperColumn setSubcolumns(List> subcolumns); + + int getSize(); + + SN getName(); + + /** + * + * @return an unmodifiable list of columns + */ + List> getColumns(); + + HCounterColumn get(int i); + + Serializer getNameSerializer(); + + byte[] getNameBytes(); + + ByteBuffer getNameByteBuffer(); + + Serializer getSuperNameSerializer(); + +} \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/beans/HSuperColumn.java b/core/src/main/java/me/prettyprint/hector/api/beans/HSuperColumn.java index 2ffc7ce99..9a1fc2aef 100644 --- a/core/src/main/java/me/prettyprint/hector/api/beans/HSuperColumn.java +++ b/core/src/main/java/me/prettyprint/hector/api/beans/HSuperColumn.java @@ -42,7 +42,7 @@ public interface HSuperColumn { Serializer getNameSerializer(); byte[] getNameBytes(); - + ByteBuffer getNameByteBuffer(); Serializer getSuperNameSerializer(); diff --git a/core/src/main/java/me/prettyprint/hector/api/ddl/ColumnFamilyDefinition.java b/core/src/main/java/me/prettyprint/hector/api/ddl/ColumnFamilyDefinition.java index 241e61c79..3ec5d2ed3 100644 --- a/core/src/main/java/me/prettyprint/hector/api/ddl/ColumnFamilyDefinition.java +++ b/core/src/main/java/me/prettyprint/hector/api/ddl/ColumnFamilyDefinition.java @@ -18,7 +18,9 @@ public interface ColumnFamilyDefinition { String getComment(); double getRowCacheSize(); int getRowCacheSavePeriodInSeconds(); + int getKeyCacheSavePeriodInSeconds(); double getKeyCacheSize(); + String getKeyValidationClass(); double getReadRepairChance(); List getColumnMetadata(); int getGcGraceSeconds(); diff --git a/core/src/main/java/me/prettyprint/hector/api/ddl/ComparatorType.java b/core/src/main/java/me/prettyprint/hector/api/ddl/ComparatorType.java index 8efa8ceff..62130693e 100644 --- a/core/src/main/java/me/prettyprint/hector/api/ddl/ComparatorType.java +++ b/core/src/main/java/me/prettyprint/hector/api/ddl/ComparatorType.java @@ -5,34 +5,65 @@ */ public class ComparatorType { - public static ComparatorType BYTESTYPE = new ComparatorType("org.apache.cassandra.db.marshal.BytesType"); - public static ComparatorType ASCIITYPE = new ComparatorType("org.apache.cassandra.db.marshal.AsciiType"); - public static ComparatorType UTF8TYPE = new ComparatorType("org.apache.cassandra.db.marshal.UTF8Type"); - public static ComparatorType LEXICALUUIDTYPE = new ComparatorType("org.apache.cassandra.db.marshal.LexicalUUIDType"); - public static ComparatorType TIMEUUIDTYPE = new ComparatorType("org.apache.cassandra.db.marshal.TimeUUIDType"); - public static ComparatorType LONGTYPE = new ComparatorType("org.apache.cassandra.db.marshal.LongType"); - public static ComparatorType INTEGERTYPE = new ComparatorType("org.apache.cassandra.db.marshal.IntegerType"); + public static ComparatorType ASCIITYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.AsciiType"); + public static ComparatorType BYTESTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.BytesType"); + public static ComparatorType INTEGERTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.IntegerType"); + public static ComparatorType LEXICALUUIDTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.LexicalUUIDType"); + public static ComparatorType LOCALBYPARTITIONERTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.LocalByPartionerType"); + public static ComparatorType LONGTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.LongType"); + public static ComparatorType TIMEUUIDTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.TimeUUIDType"); + public static ComparatorType UTF8TYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.UTF8Type"); + public static ComparatorType COMPOSITETYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.CompositeType"); + public static ComparatorType DYNAMICCOMPOSITETYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.DynamicCompositeType"); + public static ComparatorType UUIDTYPE = new ComparatorType( + "org.apache.cassandra.db.marshal.UUIDType"); - private static ComparatorType[] values = {BYTESTYPE, ASCIITYPE, UTF8TYPE, LEXICALUUIDTYPE, TIMEUUIDTYPE, LONGTYPE,INTEGERTYPE}; + private static ComparatorType[] values = { ASCIITYPE, BYTESTYPE, INTEGERTYPE, + LEXICALUUIDTYPE, LOCALBYPARTITIONERTYPE, LONGTYPE, TIMEUUIDTYPE, + UTF8TYPE, COMPOSITETYPE, DYNAMICCOMPOSITETYPE, UUIDTYPE }; private final String className; + private final String typeName; private ComparatorType(String className) { this.className = className; + if (className.startsWith("org.apache.cassandra.db.marshal.")) { + typeName = className.substring("org.apache.cassandra.db.marshal." + .length()); + } else { + typeName = className; + } } public String getClassName() { return className; } - public static ComparatorType getByClassName(String className) { + public String getTypeName() { + return typeName; + } + public static ComparatorType getByClassName(String className) { + if (className == null) { + return null; + } for (int a = 0; a < values.length; a++) { ComparatorType type = values[a]; if (type.getClassName().equals(className)) { return type; } - if (type.getClassName().equals("org.apache.cassandra.db.marshal." + className)) { + if (type.getClassName().equals( + "org.apache.cassandra.db.marshal." + className)) { return type; } } diff --git a/core/src/main/java/me/prettyprint/hector/api/ddl/package.html b/core/src/main/java/me/prettyprint/hector/api/ddl/package.html index 787bc98ae..bcb94c6e6 100644 --- a/core/src/main/java/me/prettyprint/hector/api/ddl/package.html +++ b/core/src/main/java/me/prettyprint/hector/api/ddl/package.html @@ -9,16 +9,16 @@
  • Adding new keyspaces
  • Adding new column families
  • Defining indices
  • -
  • Deleting exisiting constructs
  • +
  • Deleting exisiting constructs
  • -A note should be taken that by server design, it is required that all DDL operations are to be -performed in a serialized manner.
    +A note should be taken that by server design, it is required that all DDL operations are to be +performed in a serialized manner.
    Cassandra cannot support two DDL operations issued at the same time. The first operation needs to end before the next operation can take place and Hecotr does not try to protect its users from it, -it's the responsibility of hector's users to prevent from multiple schema changes to happen at the +it's the responsibility of hector's users to prevent from multiple schema changes to happen at the same time.
    -See
    Live Schema Updates for more +See Live Schema Updates for more details.
    \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/exceptions/HCassandraInternalException.java b/core/src/main/java/me/prettyprint/hector/api/exceptions/HCassandraInternalException.java index 4398f8e59..9a1687a58 100644 --- a/core/src/main/java/me/prettyprint/hector/api/exceptions/HCassandraInternalException.java +++ b/core/src/main/java/me/prettyprint/hector/api/exceptions/HCassandraInternalException.java @@ -3,21 +3,21 @@ /** * Designed to loosely wrap TApplicationException which can be generated * by Apache Cassandra under a variety of ambiguous conditions - some of - * them transient, some of them not. + * them transient, some of them not. * * @author zznate */ public class HCassandraInternalException extends HectorException { private static final long serialVersionUID = -266109391311421129L; - + private int type; - - private static final String ERR_MSG = + + private static final String ERR_MSG = "Cassandra encountered an internal error processing this request: "; - + public HCassandraInternalException(String msg) { - super(ERR_MSG + msg); + super(ERR_MSG + msg); } public HCassandraInternalException(int type, String msg) { @@ -26,11 +26,11 @@ public HCassandraInternalException(int type, String msg) { } public HCassandraInternalException(String s, Throwable t) { - super(ERR_MSG + s, t); + super(ERR_MSG + s, t); } public HCassandraInternalException(Throwable t) { - super(t); + super(t); } /** @@ -40,7 +40,7 @@ public HCassandraInternalException(Throwable t) { public int getType() { return type; } - - + + } diff --git a/core/src/main/java/me/prettyprint/hector/api/exceptions/HUnavailableException.java b/core/src/main/java/me/prettyprint/hector/api/exceptions/HUnavailableException.java index b815a4bb1..17120c395 100644 --- a/core/src/main/java/me/prettyprint/hector/api/exceptions/HUnavailableException.java +++ b/core/src/main/java/me/prettyprint/hector/api/exceptions/HUnavailableException.java @@ -6,7 +6,7 @@ */ public final class HUnavailableException extends HectorException { - private static final String ERR_MSG = + private static final String ERR_MSG = ": May not be enough replicas present to handle consistency level."; private static final long serialVersionUID = 1971498442136497970L; diff --git a/core/src/main/java/me/prettyprint/hector/api/exceptions/HectorException.java b/core/src/main/java/me/prettyprint/hector/api/exceptions/HectorException.java index 59efad204..42cd84da2 100644 --- a/core/src/main/java/me/prettyprint/hector/api/exceptions/HectorException.java +++ b/core/src/main/java/me/prettyprint/hector/api/exceptions/HectorException.java @@ -2,7 +2,7 @@ /** * Base exception class for all Hector related exceptions. - * + * * @author Ran Tavory (rantav@gmail.com) * */ @@ -13,7 +13,7 @@ public class HectorException extends RuntimeException { public HectorException(String msg) { super(msg); } - + public HectorException(Throwable t) { super(t); } diff --git a/core/src/main/java/me/prettyprint/hector/api/factory/HFactory.java b/core/src/main/java/me/prettyprint/hector/api/factory/HFactory.java index 3ea960f62..912b5f285 100644 --- a/core/src/main/java/me/prettyprint/hector/api/factory/HFactory.java +++ b/core/src/main/java/me/prettyprint/hector/api/factory/HFactory.java @@ -7,18 +7,22 @@ import me.prettyprint.cassandra.model.ExecutingKeyspace; import me.prettyprint.cassandra.model.ExecutingVirtualKeyspace; import me.prettyprint.cassandra.model.HColumnImpl; +import me.prettyprint.cassandra.model.HCounterColumnImpl; +import me.prettyprint.cassandra.model.HCounterSuperColumnImpl; import me.prettyprint.cassandra.model.HSuperColumnImpl; import me.prettyprint.cassandra.model.IndexedSlicesQuery; import me.prettyprint.cassandra.model.MutatorImpl; import me.prettyprint.cassandra.model.QuorumAllConsistencyLevelPolicy; import me.prettyprint.cassandra.model.thrift.ThriftColumnQuery; import me.prettyprint.cassandra.model.thrift.ThriftCountQuery; +import me.prettyprint.cassandra.model.thrift.ThriftCounterColumnQuery; import me.prettyprint.cassandra.model.thrift.ThriftMultigetSliceQuery; import me.prettyprint.cassandra.model.thrift.ThriftMultigetSubSliceQuery; import me.prettyprint.cassandra.model.thrift.ThriftMultigetSuperSliceQuery; import me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery; import me.prettyprint.cassandra.model.thrift.ThriftRangeSubSlicesQuery; import me.prettyprint.cassandra.model.thrift.ThriftRangeSuperSlicesQuery; +import me.prettyprint.cassandra.model.thrift.ThriftSliceCounterQuery; import me.prettyprint.cassandra.model.thrift.ThriftSliceQuery; import me.prettyprint.cassandra.model.thrift.ThriftSubColumnQuery; import me.prettyprint.cassandra.model.thrift.ThriftSubCountQuery; @@ -42,6 +46,8 @@ import me.prettyprint.hector.api.Keyspace; import me.prettyprint.hector.api.Serializer; import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.beans.HCounterColumn; +import me.prettyprint.hector.api.beans.HCounterSuperColumn; import me.prettyprint.hector.api.beans.HSuperColumn; import me.prettyprint.hector.api.ddl.ColumnDefinition; import me.prettyprint.hector.api.ddl.ColumnFamilyDefinition; @@ -50,12 +56,14 @@ import me.prettyprint.hector.api.mutation.Mutator; import me.prettyprint.hector.api.query.ColumnQuery; import me.prettyprint.hector.api.query.CountQuery; +import me.prettyprint.hector.api.query.CounterQuery; import me.prettyprint.hector.api.query.MultigetSliceQuery; import me.prettyprint.hector.api.query.MultigetSubSliceQuery; import me.prettyprint.hector.api.query.MultigetSuperSliceQuery; import me.prettyprint.hector.api.query.RangeSlicesQuery; import me.prettyprint.hector.api.query.RangeSubSlicesQuery; import me.prettyprint.hector.api.query.RangeSuperSlicesQuery; +import me.prettyprint.hector.api.query.SliceCounterQuery; import me.prettyprint.hector.api.query.SliceQuery; import me.prettyprint.hector.api.query.SubColumnQuery; import me.prettyprint.hector.api.query.SubCountQuery; @@ -67,7 +75,7 @@ /** * A convenience class with bunch of factory static methods to help create a * mutator, queries etc. - * + * * @author Ran * @author zznate */ @@ -88,15 +96,15 @@ public static Cluster getCluster(String clusterName) { * cluster. If another class already called getOrCreateCluster, the factory * returns the cached instance. If the instance doesn't exist in memory, a new * ThriftCluster is created and cached. - * + * * Example usage for a default installation of Cassandra. - * + * * String clusterName = "Test Cluster"; String host = "localhost:9160"; * Cluster cluster = HFactory.getOrCreateCluster(clusterName, host); - * + * * Note the host should be the hostname and port number. It is preferable to * use the hostname instead of the IP address. - * + * * @param clusterName * The cluster name. This is an identifying string for the cluster, * e.g. "production" or "test" etc. Clusters will be created on @@ -115,13 +123,13 @@ public static Cluster getOrCreateCluster(String clusterName, String hostIp) { * cluster. If another class already called getOrCreateCluster, the factory * returns the cached instance. If the instance doesn't exist in memory, a new * ThriftCluster is created and cached. - * + * * Example usage for a default installation of Cassandra. - * + * * String clusterName = "Test Cluster"; String host = "localhost:9160"; * Cluster cluster = HFactory.getOrCreateCluster(clusterName, new * CassandraHostConfigurator(host)); - * + * * @param clusterName * The cluster name. This is an identifying string for the cluster, * e.g. "production" or "test" etc. Clusters will be created on @@ -143,13 +151,13 @@ public static Cluster getOrCreateCluster(String clusterName, /** * Method looks in the cache for the cluster by name. If none exists, a new * ThriftCluster instance is created. - * + * * @param clusterName * The cluster name. This is an identifying string for the cluster, * e.g. "production" or "test" etc. Clusters will be created on * demand per each unique clusterName key. * @param cassandraHostConfigurator - * + * */ public static Cluster createCluster(String clusterName, CassandraHostConfigurator cassandraHostConfigurator) { @@ -162,7 +170,7 @@ public static Cluster createCluster(String clusterName, /** * Method looks in the cache for the cluster by name. If none exists, a new * ThriftCluster instance is created. - * + * * @param clusterName * The cluster name. This is an identifying string for the cluster, * e.g. "production" or "test" etc. Clusters will be created on @@ -178,7 +186,7 @@ public static Cluster createCluster(String clusterName, cassandraHostConfigurator, credentials) : clusters.get(clusterName); } } - + /** * Shutdown this cluster, removing it from the Map. This operation is * extremely expensive and should not be done lightly. @@ -196,7 +204,7 @@ public static void shutdownCluster(Cluster cluster) { /** * Creates a Keyspace with the default consistency level policy. - * + * * Example usage. * * String clusterName = "Test Cluster"; @@ -204,7 +212,7 @@ public static void shutdownCluster(Cluster cluster) { * Cluster cluster = HFactory.getOrCreateCluster(clusterName, host); * String keyspaceName = "testKeyspace"; * Keyspace myKeyspace = HFactory.createKeyspace(keyspaceName, cluster); - * + * * @param keyspace * @param cluster * @return @@ -320,6 +328,11 @@ public static ColumnQuery createColumnQuery( nameSerializer, valueSerializer); } + public static CounterQuery createCounterColumnQuery( + Keyspace keyspace, Serializer keySerializer, Serializer nameSerializer) { + return new ThriftCounterColumnQuery(keyspace, keySerializer, nameSerializer); + } + public static CountQuery createCountQuery(Keyspace keyspace, Serializer keySerializer, Serializer nameSerializer) { return new ThriftCountQuery(keyspace, keySerializer, nameSerializer); @@ -421,6 +434,11 @@ public static SliceQuery createSliceQuery( nameSerializer, valueSerializer); } + public static SliceCounterQuery createCounterSliceQuery( + Keyspace keyspace, Serializer keySerializer, Serializer nameSerializer) { + return new ThriftSliceCounterQuery(keyspace, keySerializer, nameSerializer); + } + public static SubSliceQuery createSubSliceQuery( Keyspace keyspace, Serializer keySerializer, Serializer sNameSerializer, Serializer nameSerializer, @@ -439,7 +457,7 @@ public static SuperSliceQuery createSuperSliceQuery( /** * createSuperColumn accepts a variable number of column arguments - * + * * @param name * supercolumn name * @param columns @@ -463,12 +481,23 @@ public static HSuperColumn createSuperColumn(SN name, superNameSerializer, nameSerializer, valueSerializer); } + public static HCounterSuperColumn createCounterSuperColumn(SN name, + List> columns, Serializer superNameSerializer, Serializer nameSerializer) { + return new HCounterSuperColumnImpl(name, columns, superNameSerializer, nameSerializer); + } + public static HColumn createColumn(N name, V value, long clock, Serializer nameSerializer, Serializer valueSerializer) { return new HColumnImpl(name, value, clock, nameSerializer, valueSerializer); } + public static HColumn createColumn(N name, V value, long clock, int ttl, + Serializer nameSerializer, Serializer valueSerializer) { + return new HColumnImpl(name, value, clock, ttl, nameSerializer, + valueSerializer); + } + /** * Creates a column with the clock of now. */ @@ -478,6 +507,15 @@ public static HColumn createColumn(N name, V value, valueSerializer); } + /** + * Creates a column with the clock of now + */ + public static HColumn createColumn(N name, V value, int ttl, + Serializer nameSerializer, Serializer valueSerializer) { + return new HColumnImpl(name, value, createClock(), ttl, nameSerializer, + valueSerializer); + } + /** * Convienience method for creating a column with a String name and String * value @@ -488,6 +526,21 @@ public static HColumn createStringColumn(String name, return createColumn(name, value, se, se); } + /** + * Create a counter column with a name and long value + */ + public static HCounterColumn createCounterColumn(N name, long value, Serializer nameSerializer) { + return new HCounterColumnImpl(name, value, nameSerializer); + } + + /** + * Convenient method for creating a counter column with a String name and long value + */ + public static HCounterColumn createCounterColumn(String name, long value) { + StringSerializer se = StringSerializer.get(); + return createCounterColumn(name, value, se); + } + /** * Creates a clock of now with the default clock resolution (micorosec) as * defined in {@link CassandraHostConfigurator}. Notice that this is a @@ -502,11 +555,11 @@ public static long createClock() { /** * Use createKeyspaceDefinition to add a new Keyspace to cluster. Example: - * + * * String testKeyspace = "testKeyspace"; KeyspaceDefinition newKeyspace = * HFactory.createKeyspaceDefinition(testKeyspace); * cluster.addKeyspace(newKeyspace); - * + * * @param keyspace */ public static KeyspaceDefinition createKeyspaceDefinition(String keyspace) { @@ -515,11 +568,11 @@ public static KeyspaceDefinition createKeyspaceDefinition(String keyspace) { /** * Use createKeyspaceDefinition to add a new Keyspace to cluster. Example: - * + * * String testKeyspace = "testKeyspace"; KeyspaceDefinition newKeyspace = * HFactory.createKeyspaceDefinition(testKeyspace); * cluster.addKeyspace(newKeyspace); - * + * * @param keyspace * @param strategyClass * - example: @@ -545,7 +598,7 @@ public static KeyspaceDefinition createKeyspaceDefinition( * HFactory.createKeyspaceDefinition(keyspace, * org.apache.cassandra.locator.SimpleStrategy.class.getName(), 1, columns); * cluster.addKeyspace(testKeyspace); - * + * * @param keyspace * @param columnFamilyName */ @@ -565,7 +618,7 @@ public static ColumnFamilyDefinition createColumnFamilyDefinition( * HFactory.createKeyspaceDefinition(keyspace, * org.apache.cassandra.locator.SimpleStrategy.class.getName(), 1, columns); * cluster.addKeyspace(testKeyspace); - * + * * @param keyspace * @param columnFamilyName * @param comparatorType @@ -584,7 +637,7 @@ public static ColumnFamilyDefinition createColumnFamilyDefinition( /** * Create a clock resolution based on clockResolutionName which * has to match any of the constants defined at {@link ClockResolution} - * + * * @param clockResolutionName * type of clock resolution to create * @return a ClockResolution diff --git a/core/src/main/java/me/prettyprint/hector/api/mutation/MutationResult.java b/core/src/main/java/me/prettyprint/hector/api/mutation/MutationResult.java index 9d6e2d5d9..e895e9bac 100644 --- a/core/src/main/java/me/prettyprint/hector/api/mutation/MutationResult.java +++ b/core/src/main/java/me/prettyprint/hector/api/mutation/MutationResult.java @@ -12,5 +12,5 @@ */ public interface MutationResult extends ResultStatus { - + } \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/mutation/Mutator.java b/core/src/main/java/me/prettyprint/hector/api/mutation/Mutator.java index a78e92029..9f59c8516 100644 --- a/core/src/main/java/me/prettyprint/hector/api/mutation/Mutator.java +++ b/core/src/main/java/me/prettyprint/hector/api/mutation/Mutator.java @@ -2,6 +2,8 @@ import me.prettyprint.hector.api.Serializer; import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.beans.HCounterColumn; +import me.prettyprint.hector.api.beans.HCounterSuperColumn; import me.prettyprint.hector.api.beans.HSuperColumn; /** @@ -26,7 +28,7 @@ MutationResult insert(final K key, final String cf, MutationResult delete(final K key, final String cf, final N columnName, final Serializer nameSerializer); - + MutationResult delete(final K key, final String cf, final N columnName, final Serializer nameSerializer, long clock); @@ -38,6 +40,19 @@ MutationResult delete(final K key, final String cf, final N columnName, MutationResult subDelete(final K key, final String cf, final SN supercolumnName, final N columnName, final Serializer sNameSerializer, final Serializer nameSerializer); + /** + * Deletes a supercolumn immediately + * @param super column type + */ + MutationResult superDelete(K key, String cf, SN supercolumnName, + Serializer sNameSerializer); + + /** + * batches a super column for deletion + * + */ + Mutator addSuperDelete(K key, String cf, SN sColumnName, Serializer sNameSerializer); + // schedule an insertion to be executed in batch by the execute method // CAVEAT: a large number of calls with a typo in one of them will leave things in an @@ -64,19 +79,19 @@ MutationResult subDelete(final K key, final String cf, final SN supercol * @return a mutator */ Mutator addDeletion(K key, String cf, N columnName, Serializer nameSerializer); - + /** * Alternate form for easy deletion of the whole row. - * + * * @param * @param key * @return */ Mutator addDeletion(K key, String cf); - + /** * Same as above accept we add the clock - * + * * @param * @param key * @return @@ -98,12 +113,17 @@ MutationResult subDelete(final K key, final String cf, final SN supercol */ Mutator addDeletion(K key, String cf, N columnName, Serializer nameSerializer, long clock); - + Mutator addSubDelete(K key, String cf, HSuperColumn sc); Mutator addSubDelete(K key, String cf, HSuperColumn sc, long clock); - - + + Mutator addSubDelete(K key, String cf, SN sColumnName, N columnName, Serializer sNameSerializer, Serializer nameSerialer); + + Mutator addSubDelete(K key, String cf, SN sColumnName, N columnName, Serializer sNameSerializer, Serializer nameSerialer, long clock); + + + /** * Batch executes all mutations scheduled to this Mutator instance by addInsertion, addDeletion etc. * May throw a HectorException which is a RuntimeException. @@ -116,4 +136,65 @@ MutationResult subDelete(final K key, final String cf, final SN supercol */ Mutator discardPendingMutations(); + // Support for counters + + /** Simple and immediate insertion (increment/decrement) of a counter */ + MutationResult insertCounter(final K key, final String cf, final HCounterColumn c); + + /** Simple and immediate insertion (increment/decrement) of a counter part of a super column */ + MutationResult insertCounter(final K key, final String cf, final HCounterSuperColumn superColumn); + + /** Convenient method to increment a simple counter */ + MutationResult incrementCounter(final K key, final String cf, final N columnName, final long increment); + + /** Convenient method to decrement a simple counter */ + MutationResult decrementCounter(final K key, final String cf, final N columnName, final long increment); + + MutationResult deleteCounter(final K key, final String cf, final N columnName, final Serializer nameSerializer); + + /** + * Deletes a subcolumn of a supercolumn for a counter + * @param super column type + * @param subcolumn type + */ + MutationResult subDeleteCounter(final K key, final String cf, final SN supercolumnName, + final N columnName, final Serializer sNameSerializer, final Serializer nameSerializer); + + /** + * Schedule an increment of a CounterColumn to be inserted in batch mode by {@link #execute()} + */ + Mutator addCounter(K key, String cf, HCounterColumn c); + + /** + * Schedule an increment of a SuperColumn to be inserted in batch mode by {@link #execute()} + */ + Mutator addCounter(K key, String cf, HCounterSuperColumn sc); + + /** + * Adds a Deletion to the underlying batch_mutate call. The columnName argument can be null + * in which case the whole row being deleted. + * + * @param column name type + * @param key row key + * @param cf column family + * @param counterColumnName column name. Use null to delete the whole row. + * @param nameSerializer a name serializer + * @return a mutator + */ + Mutator addCounterDeletion(K key, String cf, N counterColumnName, Serializer nameSerializer); + + /** + * Alternate form for easy deletion of the whole row. + * + * @param + * @param key + * @return + */ + Mutator addCounterDeletion(K key, String cf); + + /** + * Schedule a counter deletion. + */ + Mutator addCounterSubDeletion(K key, String cf, HCounterSuperColumn sc); + } \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/query/CounterQuery.java b/core/src/main/java/me/prettyprint/hector/api/query/CounterQuery.java new file mode 100644 index 000000000..2b4d27ca9 --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/query/CounterQuery.java @@ -0,0 +1,27 @@ +package me.prettyprint.hector.api.query; + +import me.prettyprint.hector.api.beans.HCounterColumn; + +/** + * A CounterQuery is used for querying the value of a single and standard counter column. + *

    + * + * @author patricio + * + * @param Column name type. + */ +public interface CounterQuery extends Query>{ + + /** + * Set the row key for this query. + */ + CounterQuery setKey(K key); + + /** + * Set the column name for this query. + */ + CounterQuery setName(N name); + + CounterQuery setColumnFamily(String cf); + +} \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/query/MultigetSliceQuery.java b/core/src/main/java/me/prettyprint/hector/api/query/MultigetSliceQuery.java index 2dd451ccf..836b8f6b4 100644 --- a/core/src/main/java/me/prettyprint/hector/api/query/MultigetSliceQuery.java +++ b/core/src/main/java/me/prettyprint/hector/api/query/MultigetSliceQuery.java @@ -12,7 +12,7 @@ public interface MultigetSliceQuery extends Query> { MultigetSliceQuery setKeys(K... keys); - + MultigetSliceQuery setKeys(Iterable keys); /** diff --git a/core/src/main/java/me/prettyprint/hector/api/query/MultigetSubSliceQuery.java b/core/src/main/java/me/prettyprint/hector/api/query/MultigetSubSliceQuery.java index 3f2795b3f..227761bd1 100644 --- a/core/src/main/java/me/prettyprint/hector/api/query/MultigetSubSliceQuery.java +++ b/core/src/main/java/me/prettyprint/hector/api/query/MultigetSubSliceQuery.java @@ -13,6 +13,8 @@ public interface MultigetSubSliceQuery extends Query> MultigetSubSliceQuery setKeys(K... keys); + MultigetSubSliceQuery setKeys(Collection keys); + /** * Set the supercolumn to run the slice query on */ @@ -30,4 +32,6 @@ public interface MultigetSubSliceQuery extends Query> */ MultigetSubSliceQuery setColumnNames(N... columnNames); + MultigetSubSliceQuery setColumnNames(Collection columnNames); + } \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/query/MultigetSuperSliceQuery.java b/core/src/main/java/me/prettyprint/hector/api/query/MultigetSuperSliceQuery.java index e958a9542..f1c201599 100644 --- a/core/src/main/java/me/prettyprint/hector/api/query/MultigetSuperSliceQuery.java +++ b/core/src/main/java/me/prettyprint/hector/api/query/MultigetSuperSliceQuery.java @@ -13,6 +13,8 @@ public interface MultigetSuperSliceQuery extends Query setKeys(K... keys); + MultigetSuperSliceQuery setKeys(Collection keys); + MultigetSuperSliceQuery setColumnFamily(String cf); MultigetSuperSliceQuery setRange(SN start, SN finish, boolean reversed, int count); @@ -25,4 +27,6 @@ public interface MultigetSuperSliceQuery extends Query setColumnNames(SN... columnNames); + MultigetSuperSliceQuery setColumnNames(Collection columnNames); + } \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/query/QueryResult.java b/core/src/main/java/me/prettyprint/hector/api/query/QueryResult.java index 9e9bc6277..dde4e7d0a 100644 --- a/core/src/main/java/me/prettyprint/hector/api/query/QueryResult.java +++ b/core/src/main/java/me/prettyprint/hector/api/query/QueryResult.java @@ -6,7 +6,7 @@ * Return type from queries execution. * * @author Ran Tavory - * + * * @param The type of the result. May be for example Column of SuperColumn */ diff --git a/core/src/main/java/me/prettyprint/hector/api/query/RangeSlicesQuery.java b/core/src/main/java/me/prettyprint/hector/api/query/RangeSlicesQuery.java index 75efa9770..33eeb92a9 100644 --- a/core/src/main/java/me/prettyprint/hector/api/query/RangeSlicesQuery.java +++ b/core/src/main/java/me/prettyprint/hector/api/query/RangeSlicesQuery.java @@ -25,7 +25,7 @@ public interface RangeSlicesQuery extends Query>{ RangeSlicesQuery setColumnFamily(String cf); RangeSlicesQuery setRange(N start, N finish, boolean reversed, int count); - + RangeSlicesQuery setReturnKeysOnly(); } \ No newline at end of file diff --git a/core/src/main/java/me/prettyprint/hector/api/query/SliceCounterQuery.java b/core/src/main/java/me/prettyprint/hector/api/query/SliceCounterQuery.java new file mode 100644 index 000000000..efc344e99 --- /dev/null +++ b/core/src/main/java/me/prettyprint/hector/api/query/SliceCounterQuery.java @@ -0,0 +1,22 @@ +package me.prettyprint.hector.api.query; + +import me.prettyprint.hector.api.beans.CounterSlice; + +/** + * A query for the thrift call get_counter_slice + * + * @author patricioe (Patricio Echague) + * + * @param Name Type + */ +public interface SliceCounterQuery extends Query> { + + SliceCounterQuery setKey(K key); + + SliceCounterQuery setColumnNames(N... columnNames); + + SliceCounterQuery setRange(N start, N finish, boolean reversed, int count); + + SliceCounterQuery setColumnFamily(String cf); + +} \ No newline at end of file diff --git a/core/src/main/resources/hectorLog4j.xml b/core/src/main/resources/hectorLog4j.xml index b7f66e553..b25a39739 100644 --- a/core/src/main/resources/hectorLog4j.xml +++ b/core/src/main/resources/hectorLog4j.xml @@ -1,53 +1,5 @@ - - - - - - - - - - - - - - - - - - - - - - - diff --git a/core/src/main/resources/speed4j.properties b/core/src/main/resources/speed4j.properties new file mode 100644 index 000000000..cef18ac45 --- /dev/null +++ b/core/src/main/resources/speed4j.properties @@ -0,0 +1,21 @@ +# +# This is the main configuration for Speed4j. You may override this in your +# own project. +# +# For hector, we define a simple Log which is used to both output things +# to the given SLF4J logger, as well as display it in a JMX bean. +# + +# Which Log instance shall we use? +speed4j.hector = com.ecyrd.speed4j.log.PeriodicalLog + +# How often shall this Log log (in seconds)? +speed4j.hector.period = 60 + +# Which of these attributes shall be exposed in JMX? +speed4j.hector.jmx=READ.success_,WRITE.success_,READ.fail_,WRITE.fail_,META_READ.success_,META_READ.fail_ + +# Which SLF4J log name shall we output to? Speed4J uses the INFO log level. +# Note that this is slightly different from the old Log4J name just to ensure that we +# don't mess with any existing log4j configurations. +speed4j.hector.slf4jLogname=me.prettyprint.hector.TimingLogger diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/BaseBalancingPolicyTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/BaseBalancingPolicyTest.java index c1bd26f3e..aa88bda64 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/BaseBalancingPolicyTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/BaseBalancingPolicyTest.java @@ -11,28 +11,28 @@ public abstract class BaseBalancingPolicyTest { protected List pools = new ArrayList(); - + protected ConcurrentHClientPool poolWith5Active; protected ConcurrentHClientPool poolWith7Active; protected ConcurrentHClientPool poolWith10Active; - - + + @Before public void setup() { - poolWith5Active = Mockito.mock(ConcurrentHClientPool.class); + poolWith5Active = Mockito.mock(ConcurrentHClientPool.class); Mockito.when(poolWith5Active.getNumActive()).thenReturn(5); poolWith7Active = Mockito.mock(ConcurrentHClientPool.class); Mockito.when(poolWith7Active.getNumActive()).thenReturn(7); poolWith10Active = Mockito.mock(ConcurrentHClientPool.class); Mockito.when(poolWith10Active.getNumActive()).thenReturn(10); - + Mockito.when(poolWith5Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.1:9160")); Mockito.when(poolWith7Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.2:9161")); Mockito.when(poolWith10Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.3:9162")); - - pools.add(poolWith5Active); + + pools.add(poolWith5Active); pools.add(poolWith7Active); pools.add(poolWith10Active); } - + } diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/ConcurrentHClientPoolTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/ConcurrentHClientPoolTest.java index baef86eef..a61284a1e 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/ConcurrentHClientPoolTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/ConcurrentHClientPoolTest.java @@ -8,17 +8,17 @@ import org.junit.Test; public class ConcurrentHClientPoolTest extends BaseEmbededServerSetupTest { - + private CassandraHost cassandraHost; private ConcurrentHClientPool clientPool; - + @Before public void setupTest() { setupClient(); cassandraHost = cassandraHostConfigurator.buildCassandraHosts()[0]; clientPool = new ConcurrentHClientPool(cassandraHost); } - + @Test public void testSpinUp() { assertEquals(16, clientPool.getNumIdle()); @@ -26,7 +26,7 @@ public void testSpinUp() { assertEquals(0, clientPool.getNumBlockedThreads()); assertEquals(0, clientPool.getNumActive()); } - + @Test public void testShutdown() { clientPool.shutdown(); @@ -34,7 +34,7 @@ public void testShutdown() { assertEquals(0, clientPool.getNumBlockedThreads()); assertEquals(0, clientPool.getNumActive()); } - + @Test public void testBorrowRelease() { HThriftClient client = clientPool.borrowClient(); diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/HConnectionManagerTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/HConnectionManagerTest.java index fe59ad31a..4cd62efd6 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/HConnectionManagerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/HConnectionManagerTest.java @@ -2,19 +2,25 @@ import static org.junit.Assert.*; +import org.apache.cassandra.thrift.Cassandra.Client; import org.junit.Before; import org.junit.Test; import me.prettyprint.cassandra.BaseEmbededServerSetupTest; import me.prettyprint.cassandra.service.CassandraHost; import me.prettyprint.cassandra.service.CassandraHostConfigurator; +import me.prettyprint.cassandra.service.FailoverPolicy; +import me.prettyprint.cassandra.service.Operation; +import me.prettyprint.cassandra.service.OperationType; +import me.prettyprint.hector.api.exceptions.HTimedOutException; +import me.prettyprint.hector.api.exceptions.HectorException; import me.prettyprint.hector.api.exceptions.HectorTransportException; public class HConnectionManagerTest extends BaseEmbededServerSetupTest { - - - + + + @Test public void testRemoveHost() { setupClient(); @@ -24,30 +30,30 @@ public void testRemoveHost() { assertTrue(connectionManager.addCassandraHost(cassandraHost)); assertEquals(1,connectionManager.getActivePools().size()); } - - @Test + + @Test public void testAddCassandraHostFail() { setupClient(); CassandraHost cassandraHost = new CassandraHost("127.0.0.1", 9180); assertFalse(connectionManager.addCassandraHost(cassandraHost)); } - + @Test(expected=IllegalArgumentException.class) public void testNullHostList() { HConnectionManager hcm = new HConnectionManager(clusterName, new CassandraHostConfigurator()); } - + @Test public void testMarkHostDownWithNoRetry() { cassandraHostConfigurator = new CassandraHostConfigurator("127.0.0.1:9170"); cassandraHostConfigurator.setRetryDownedHosts(false); connectionManager = new HConnectionManager(clusterName, cassandraHostConfigurator); - CassandraHost cassandraHost = new CassandraHost("127.0.0.1", 9170); + CassandraHost cassandraHost = new CassandraHost("127.0.0.1", 9170); HThriftClient client = connectionManager.borrowClient(); connectionManager.markHostAsDown(client.cassandraHost); assertEquals(0,connectionManager.getActivePools().size()); } - + @Test public void testSuspendCassandraHost() { setupClient(); @@ -56,4 +62,35 @@ public void testSuspendCassandraHost() { assertEquals(1,connectionManager.getSuspendedCassandraHosts().size()); assertTrue(connectionManager.unsuspendCassandraHost(cassandraHost)); } + + @Test(expected=HTimedOutException.class) + public void testTimedOutOperateWithFailover() { + setupClient(); + FailoverPolicy fp = FailoverPolicy.ON_FAIL_TRY_ONE_NEXT_AVAILABLE; + connectionManager.operateWithFailover(new TimeoutOp(fp)); + } + + abstract class StubOp extends Operation { + + StubOp(FailoverPolicy fp) { + this(OperationType.META_READ); + failoverPolicy = fp; + } + + public StubOp(OperationType operationType) { + super(operationType); + } + } + + class TimeoutOp extends StubOp { + + TimeoutOp(FailoverPolicy fp) { + super(fp); + } + + @Override + public String execute(Client cassandra) throws HectorException { + throw new HTimedOutException("fake timeout"); + } + } } diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/HThriftClientTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/HThriftClientTest.java index 4f9ca9a6a..2853818b6 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/HThriftClientTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/HThriftClientTest.java @@ -13,18 +13,18 @@ public class HThriftClientTest extends BaseEmbededServerSetupTest { private HThriftClient hThriftClient; // cassandraHostConfigurator = new CassandraHostConfigurator("127.0.0.1:9170"); private CassandraHost cassandraHost; - + @Before public void doSetup() { cassandraHost = new CassandraHost("127.0.0.1:9170"); hThriftClient = new HThriftClient(cassandraHost); } - + @After public void doTeardown() { hThriftClient.close(); } - + @Test public void testOpenAndClose() { assertTrue(hThriftClient.open().isOpen()); @@ -35,19 +35,19 @@ public void testOpenAndClose() { public void testFailOnDoubleOpen() { hThriftClient.open().open(); } - + @Test(expected=IllegalStateException.class) public void testGetCassandraNotOpen() { hThriftClient.getCassandra(); } - + @Test public void testGetCassandraWithKeyspace() { hThriftClient.open(); hThriftClient.getCassandra("Keyspace1"); assertTrue(hThriftClient.isOpen()); } - + @Test public void testGetCassandraWithNullKeyspace() { hThriftClient.open(); @@ -55,5 +55,5 @@ public void testGetCassandraWithNullKeyspace() { hThriftClient.getCassandra(null); assertTrue(hThriftClient.isOpen()); } - + } diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/HostTimeoutTrackerTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/HostTimeoutTrackerTest.java index 833a34ed5..12bf10b06 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/HostTimeoutTrackerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/HostTimeoutTrackerTest.java @@ -24,7 +24,7 @@ public void setup() { HConnectionManager connectionManager = new HConnectionManager("TestCluster", cassandraHostConfigurator); hostTimeoutTracker = new HostTimeoutTracker(connectionManager, cassandraHostConfigurator); } - + @Test public void testTrackHostLatency() { CassandraHost cassandraHost = new CassandraHost("localhost:9170"); @@ -37,11 +37,11 @@ public void testTrackHostLatency() { } assertTrue(hostTimeoutTracker.checkTimeout(cassandraHost)); - // ... - // in HConnectionManager: + // ... + // in HConnectionManager: // - if ( hostLatencyTracker.checkTimeout(cassandraHost) ) // markHostAsDown(cassandraHost); // excludeHosts.add(cassandraHost); - + } } diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicyTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicyTest.java index ea8803fe0..81be48380 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicyTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/LeastActiveBalancingPolicyTest.java @@ -16,13 +16,13 @@ public class LeastActiveBalancingPolicyTest extends BaseBalancingPolicyTest { private LeastActiveBalancingPolicy leastActiveBalancingPolicy; - + @Test public void testGetPoolOk() { leastActiveBalancingPolicy = new LeastActiveBalancingPolicy(); assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, null)); assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, null)); - assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, null)); + assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, null)); Mockito.when(poolWith5Active.getNumActive()).thenReturn(8); assertEquals(poolWith7Active, leastActiveBalancingPolicy.getPool(pools, null)); assertEquals(poolWith7Active, leastActiveBalancingPolicy.getPool(pools, null)); @@ -32,33 +32,33 @@ public void testGetPoolOk() { assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, null)); assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, null)); } - + @Test - public void testSkipExhausted() { + public void testSkipExhausted() { leastActiveBalancingPolicy = new LeastActiveBalancingPolicy(); assertEquals(poolWith7Active, leastActiveBalancingPolicy.getPool(pools, new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"))))); assertEquals(poolWith5Active, leastActiveBalancingPolicy.getPool(pools, new HashSet(Arrays.asList(new CassandraHost("127.0.0.2:9161"))))); } - + @Test public void testShuffleOnAllEqual() { - ConcurrentHClientPool poolWith5Active2 = Mockito.mock(ConcurrentHClientPool.class); + ConcurrentHClientPool poolWith5Active2 = Mockito.mock(ConcurrentHClientPool.class); Mockito.when(poolWith5Active2.getNumActive()).thenReturn(5); Mockito.when(poolWith5Active2.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.4:9163")); - ConcurrentHClientPool poolWith5Active3 = Mockito.mock(ConcurrentHClientPool.class); + ConcurrentHClientPool poolWith5Active3 = Mockito.mock(ConcurrentHClientPool.class); Mockito.when(poolWith5Active3.getNumActive()).thenReturn(5); Mockito.when(poolWith5Active3.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.5:9164")); - + pools.add(poolWith5Active2); pools.add(poolWith5Active3); - + leastActiveBalancingPolicy = new LeastActiveBalancingPolicy(); // should hit all three equal hosts over the course of 50 runs Set foundHosts = new HashSet(3); for (int i = 0; i < 50; i++) { HClientPool foundPool = leastActiveBalancingPolicy.getPool(pools, null); foundHosts.add(foundPool.getCassandraHost()); - assert 5 == foundPool.getNumActive(); + assert 5 == foundPool.getNumActive(); } assertEquals(3, foundHosts.size()); } diff --git a/core/src/test/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicyTest.java b/core/src/test/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicyTest.java index 38d850b34..a5e56c056 100644 --- a/core/src/test/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicyTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/connection/RoundRobinBalancingPolicyTest.java @@ -1,5 +1,6 @@ package me.prettyprint.cassandra.connection; +import static org.junit.Assert.*; import static org.junit.Assert.assertEquals; import java.util.Arrays; @@ -14,7 +15,7 @@ public class RoundRobinBalancingPolicyTest extends BaseBalancingPolicyTest { private RoundRobinBalancingPolicy roundRobinBalancingPolicy; - + @Test public void testGetPoolOk() { roundRobinBalancingPolicy = new RoundRobinBalancingPolicy(); @@ -29,21 +30,37 @@ public void testGetPoolOk() { assertEquals(poolWith10Active.getNumActive(), roundRobinBalancingPolicy.getPool(pools, null).getNumActive()); // go to 65k to roll the counter a couple of times for (int x=0; x<(256*256); x++) { - assert roundRobinBalancingPolicy.getPool(pools, null).getNumActive() >= 5; + assert roundRobinBalancingPolicy.getPool(pools, null).getNumActive() >= 5; } } - + @Test public void testIgnoreExhausted() { Mockito.when(poolWith5Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.1:9160")); Mockito.when(poolWith7Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.2:9161")); Mockito.when(poolWith10Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.3:9162")); - + roundRobinBalancingPolicy = new RoundRobinBalancingPolicy(); assertEquals(poolWith7Active, roundRobinBalancingPolicy.getPool(pools, new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"))))); assertEquals(poolWith10Active, roundRobinBalancingPolicy.getPool(pools, new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"))))); assertEquals(poolWith7Active, roundRobinBalancingPolicy.getPool(pools, new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"))))); assertEquals(poolWith10Active, roundRobinBalancingPolicy.getPool(pools, new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"))))); } - + + @Test + public void testIgnoreExhaustedAll() { + Mockito.when(poolWith5Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.1:9160")); + Mockito.when(poolWith7Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.2:9161")); + Mockito.when(poolWith10Active.getCassandraHost()).thenReturn(new CassandraHost("127.0.0.3:9162")); + + roundRobinBalancingPolicy = new RoundRobinBalancingPolicy(); + /* + assertEquals(poolWith10Active, roundRobinBalancingPolicy.getPool(pools, + new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"),new CassandraHost("127.0.0.2:9161"))))); + */ + assertNotNull(roundRobinBalancingPolicy.getPool(pools, + new HashSet(Arrays.asList(new CassandraHost("127.0.0.1:9160"),new CassandraHost("127.0.0.2:9161"),new CassandraHost("127.0.0.3:9162"))))); + + + } } diff --git a/core/src/test/java/me/prettyprint/cassandra/dao/SimpleCassandraDaoTest.java b/core/src/test/java/me/prettyprint/cassandra/dao/SimpleCassandraDaoTest.java index 69e3612bc..ea9bf8a3b 100644 --- a/core/src/test/java/me/prettyprint/cassandra/dao/SimpleCassandraDaoTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/dao/SimpleCassandraDaoTest.java @@ -17,7 +17,7 @@ public class SimpleCassandraDaoTest extends BaseEmbededServerSetupTest { @Resource private SimpleCassandraDao simpleCassandraDao; - + @Test public void testInsertGetDelete() { simpleCassandraDao.insert("fk1", "colName1", "value1"); @@ -25,5 +25,5 @@ public void testInsertGetDelete() { simpleCassandraDao.delete("colName1", "fk1"); assertNull(simpleCassandraDao.get("fk1", "colName1")); } - + } diff --git a/core/src/test/java/me/prettyprint/cassandra/io/StreamTest.java b/core/src/test/java/me/prettyprint/cassandra/io/StreamTest.java index c1e83778a..19d67217b 100644 --- a/core/src/test/java/me/prettyprint/cassandra/io/StreamTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/io/StreamTest.java @@ -31,7 +31,7 @@ public class StreamTest extends BaseEmbededServerSetupTest { BLOB_CF_DEF.comparator_type = "IntegerType"; } - public final static KeyspaceDefinition KEYSPACE_DEV = new ThriftKsDef(new KsDef(KEYSPACE, "org.apache.cassandra.locator.SimpleStrategy", 1, Arrays.asList(new CfDef[] { BLOB_CF_DEF }))); + public static KeyspaceDefinition KEYSPACE_DEV; private Keyspace keyspace; private ThriftCluster cassandraCluster; @@ -40,7 +40,8 @@ public class StreamTest extends BaseEmbededServerSetupTest { @Before public void setUp() throws Exception { super.setupClient(); - + KEYSPACE_DEV = new ThriftKsDef(new KsDef(KEYSPACE, "org.apache.cassandra.locator.SimpleStrategy", Arrays.asList(new CfDef[] { BLOB_CF_DEF }))); + ((ThriftKsDef)KEYSPACE_DEV).setReplicationFactor(1); cassandraHostConfigurator = new CassandraHostConfigurator("localhost:9170"); cassandraCluster = new ThriftCluster("Test Cluster", cassandraHostConfigurator); diff --git a/core/src/test/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactoryTest.java b/core/src/test/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactoryTest.java index 7cec0eb19..485d57634 100644 --- a/core/src/test/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactoryTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/jndi/CassandraClientJndiResourceFactoryTest.java @@ -14,6 +14,7 @@ import javax.naming.StringRefAddr; import me.prettyprint.cassandra.BaseEmbededServerSetupTest; +import me.prettyprint.hector.api.Keyspace; import org.junit.After; import org.junit.Before; @@ -22,12 +23,12 @@ /** * @author Perry Hoekstra (dutchman_mn@charter.net) + * @author zznate */ -@Ignore public class CassandraClientJndiResourceFactoryTest extends BaseEmbededServerSetupTest { // canned data - private final static String cassandraUrl = "localhost"; - private final static int cassandraPort = 9170; + private final static String cassandraUrl = "localhost:9170"; + private CassandraClientJndiResourceFactory factory; @@ -45,21 +46,19 @@ public void teardownCase() throws IOException { public void getObjectInstance() throws Exception { Reference resource = new Reference("CassandraClientFactory"); - resource.add(new StringRefAddr("url", cassandraUrl)); - resource.add(new StringRefAddr("port", Integer.toString(cassandraPort))); + resource.add(new StringRefAddr("hosts", cassandraUrl)); + resource.add(new StringRefAddr("clusterName", clusterName)); + resource.add(new StringRefAddr("keyspace", "Keyspace1")); + resource.add(new StringRefAddr("autoDiscoverHosts", "true")); + Name jndiName = mock(Name.class); Context context = new InitialContext(); Hashtable environment = new Hashtable(); - CassandraClientJndiResourcePool cassandraClientJNDIResourcePool = - (CassandraClientJndiResourcePool) factory.getObjectInstance(resource, jndiName, context, - environment); + Keyspace keyspace = (Keyspace) factory.getObjectInstance(resource, jndiName, context, environment); - //CassandraClient cassandraClient = (CassandraClient) cassandraClientJNDIResourcePool.borrowObject(); - // TODO fix this - //assertNotNull(cassandraClient); - //assertEquals(cassandraUrl, cassandraClient.getCassandraHost().getHost()); - //assertEquals(cassandraPort, cassandraClient.getCassandraHost().getPort()); + assertNotNull(keyspace); + assertEquals("Keyspace1",keyspace.getKeyspaceName()); } } diff --git a/core/src/test/java/me/prettyprint/cassandra/model/ColumnSliceTest.java b/core/src/test/java/me/prettyprint/cassandra/model/ColumnSliceTest.java index 5576695e6..d81b1a525 100644 --- a/core/src/test/java/me/prettyprint/cassandra/model/ColumnSliceTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/model/ColumnSliceTest.java @@ -22,7 +22,7 @@ public class ColumnSliceTest { StringSerializer se = StringSerializer.get(); LongSerializer le = LongSerializer.get(); - + @Test public void testConstruction() { @@ -30,22 +30,31 @@ public void testConstruction() { ColumnSlice slice = new ColumnSliceImpl(tColumns, se, le); Assert.assertTrue(slice.getColumns().isEmpty()); - tColumns.add(new Column(ByteBuffer.wrap(new byte[]{}), ByteBuffer.wrap(new byte[]{}), 0L)); + Column column = new Column(ByteBuffer.wrap(new byte[]{})); + column.setValue(ByteBuffer.wrap(new byte[]{})); + column.setTimestamp(0L); + tColumns.add(column); slice = new ColumnSliceImpl(tColumns, se, le); Assert.assertEquals(1, slice.getColumns().size()); tColumns = new ArrayList(); - tColumns.add(new Column(se.toByteBuffer("1"), le.toByteBuffer(1L), 0L)); + column = new Column(se.toByteBuffer("1")); + column.setValue(le.toByteBuffer(1L)); + column.setTimestamp(0L); + tColumns.add(column); slice = new ColumnSliceImpl(tColumns, se, le); Assert.assertEquals((Long) 1L, slice.getColumnByName("1").getValue()); } - + @Test public void testMultiCallOnByteBuffer() { List tColumns = new ArrayList(); - tColumns.add(new Column(se.toByteBuffer("1"), ByteBuffer.wrap("colvalue".getBytes()), 0L)); + Column column = new Column(se.toByteBuffer("1")); + column.setValue(ByteBuffer.wrap("colvalue".getBytes())); + column.setTimestamp(0L); + tColumns.add(column); ColumnSlice slice = new ColumnSliceImpl(tColumns, se, ByteBufferSerializer.get()); - + ByteBuffer value = slice.getColumnByName("1").getValue(); Assert.assertEquals("colvalue", se.fromByteBuffer(value)); value.rewind(); diff --git a/core/src/test/java/me/prettyprint/cassandra/model/CqlQueryTest.java b/core/src/test/java/me/prettyprint/cassandra/model/CqlQueryTest.java new file mode 100644 index 000000000..b91cb304b --- /dev/null +++ b/core/src/test/java/me/prettyprint/cassandra/model/CqlQueryTest.java @@ -0,0 +1,73 @@ +package me.prettyprint.cassandra.model; + +import static me.prettyprint.hector.api.factory.HFactory.createColumn; +import static me.prettyprint.hector.api.factory.HFactory.createKeyspace; +import static me.prettyprint.hector.api.factory.HFactory.createMutator; +import static me.prettyprint.hector.api.factory.HFactory.getOrCreateCluster; +import static org.junit.Assert.assertEquals; +import me.prettyprint.cassandra.BaseEmbededServerSetupTest; +import me.prettyprint.cassandra.serializers.LongSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.hector.api.Cluster; +import me.prettyprint.hector.api.Keyspace; +import me.prettyprint.hector.api.exceptions.HInvalidRequestException; +import me.prettyprint.hector.api.query.QueryResult; + +import org.junit.Before; +import org.junit.Test; + +public class CqlQueryTest extends BaseEmbededServerSetupTest { + + private final static String KEYSPACE = "Keyspace1"; + private static final StringSerializer se = new StringSerializer(); + private static final LongSerializer le = new LongSerializer(); + private Cluster cluster; + private Keyspace keyspace; + private String cf = "StandardLong1"; + + @Before + public void setupCase() { + cluster = getOrCreateCluster("MyCluster", "127.0.0.1:9170"); + keyspace = createKeyspace(KEYSPACE, cluster); + createMutator(keyspace, se) + .addInsertion("cqlQueryTest_key1", cf, createColumn("birthyear", 1974L, se, le)) + .addInsertion("cqlQueryTest_key1", cf, createColumn("birthmonth", 4L, se, le)) + .addInsertion("cqlQueryTest_key2", cf, createColumn("birthyear", 1975L, se, le)) + .addInsertion("cqlQueryTest_key2", cf, createColumn("birthmonth", 4L, se, le)) + .addInsertion("cqlQueryTest_key3", cf, createColumn("birthyear", 1975L, se, le)) + .addInsertion("cqlQueryTest_key3", cf, createColumn("birthmonth", 5L, se, le)) + .addInsertion("cqlQueryTest_key4", cf, createColumn("birthyear", 1975L, se, le)) + .addInsertion("cqlQueryTest_key4", cf, createColumn("birthmonth", 6L, se, le)) + .addInsertion("cqlQueryTest_key5", cf, createColumn("birthyear", 1975L, se, le)) + .addInsertion("cqlQueryTest_key5", cf, createColumn("birthmonth", 7L, se, le)) + .addInsertion("cqlQueryTest_key6", cf, createColumn("birthyear", 1976L, se, le)) + .addInsertion("cqlQueryTest_key6", cf, createColumn("birthmonth", 6L, se, le)) + .execute(); + } + + @Test + public void testSimpleSelect() { + CqlQuery cqlQuery = new CqlQuery(keyspace, se, se, le); + cqlQuery.setQuery("select * from StandardLong1"); + QueryResult> result = cqlQuery.execute(); + assertEquals(6,result.get().getCount()); + + } + + @Test + public void testCountQuery() { + CqlQuery cqlQuery = new CqlQuery(keyspace, se, se, le); + cqlQuery.setQuery("SELECT COUNT(*) FROM StandardLong1 WHERE KEY = 'cqlQueryTest_key1'"); + QueryResult> result = cqlQuery.execute(); + assertEquals(2, result.get().getAsCount()); + } + + @Test(expected=HInvalidRequestException.class) + public void testSyntaxFailQuery() { + CqlQuery cqlQuery = new CqlQuery(keyspace, se, se, le); + cqlQuery.setQuery("SELECT COUNT(*) FROM Standard1 WHERE KEY = 'cqlQueryTest_key1'"); + QueryResult> result = cqlQuery.execute(); + + } + +} diff --git a/core/src/test/java/me/prettyprint/cassandra/model/HColumnFamilyTest.java b/core/src/test/java/me/prettyprint/cassandra/model/HColumnFamilyTest.java index 2cc6f40f8..2cf664322 100644 --- a/core/src/test/java/me/prettyprint/cassandra/model/HColumnFamilyTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/model/HColumnFamilyTest.java @@ -27,7 +27,7 @@ public class HColumnFamilyTest extends BaseEmbededServerSetupTest { - + private Keyspace keyspace; private UUID timeUUID; @Before @@ -35,7 +35,7 @@ public void setupLocal() { //setupClient(); Cluster cluster = getOrCreateCluster("MyCluster", "127.0.0.1:9170"); keyspace = createKeyspace("Keyspace1", cluster); - + Mutator mutator = HFactory.createMutator(keyspace, StringSerializer.get()); mutator.addInsertion("zznate", "Standard1", HFactory.createStringColumn("email", "nate@datastax.com")); mutator.addInsertion("zznate", "Standard1", HFactory.createColumn("int", 1, StringSerializer.get(), IntegerSerializer.get())); @@ -44,7 +44,7 @@ public void setupLocal() { mutator.addInsertion("zznate", "Standard1", HFactory.createColumn("uuid", timeUUID, StringSerializer.get(), UUIDSerializer.get())); mutator.execute(); } - + @Test public void testColumnFamilySetup() { HColumnFamily columnFamily = new HColumnFamilyImpl(keyspace, "Standard1",StringSerializer.get(), StringSerializer.get()); @@ -56,7 +56,7 @@ public void testColumnFamilySetup() { assertEquals(timeUUID, columnFamily.getUUID("uuid")); } - + @Test public void testColumnFamilyReadahead() { HColumnFamily columnFamily = new HColumnFamilyImpl(keyspace, "Standard1",StringSerializer.get(), StringSerializer.get()); @@ -68,7 +68,7 @@ public void testColumnFamilyReadahead() { assertEquals(timeUUID, columnFamily.getUUID("uuid")); } - + @Test public void testClearAndRecall() { HColumnFamily columnFamily = new HColumnFamilyImpl(keyspace, "Standard1",StringSerializer.get(), StringSerializer.get()); @@ -83,7 +83,7 @@ public void testClearAndRecall() { assertEquals(4,columnFamily.getColumns().size()); assertEquals(timeUUID, columnFamily.getUUID("uuid")); } - + @Test public void testToggleMultiget() { Mutator mutator = HFactory.createMutator(keyspace, StringSerializer.get()); @@ -93,7 +93,7 @@ public void testToggleMultiget() { timeUUID = TimeUUIDUtils.getTimeUUID(System.currentTimeMillis()); mutator.addInsertion("patricioe", "Standard1", HFactory.createColumn("uuid", timeUUID, StringSerializer.get(), UUIDSerializer.get())); mutator.execute(); - + HColumnFamilyImpl columnFamily = new HColumnFamilyImpl(keyspace, "Standard1",StringSerializer.get(), StringSerializer.get()); columnFamily.addKey("zznate").addKey("patricioe").setCount(10); assertEquals("nate@datastax.com",columnFamily.getString("email")); diff --git a/core/src/test/java/me/prettyprint/cassandra/model/IndexedSlicesQueryTest.java b/core/src/test/java/me/prettyprint/cassandra/model/IndexedSlicesQueryTest.java index 12a411030..3ed19f4de 100644 --- a/core/src/test/java/me/prettyprint/cassandra/model/IndexedSlicesQueryTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/model/IndexedSlicesQueryTest.java @@ -53,7 +53,7 @@ public void teardownCase() { } @Test - public void testInsertGetRemove() { + public void testInsertGetRemove() { IndexedSlicesQuery indexedSlicesQuery = new IndexedSlicesQuery(keyspace, se, se, le); indexedSlicesQuery.addEqualsExpression("birthyear", 1975L); @@ -66,11 +66,11 @@ public void testInsertGetRemove() { } - + @Test - public void testMultiClause() { + public void testMultiClause() { - QueryResult> result = + QueryResult> result = new IndexedSlicesQuery(keyspace, se, se, le) .addEqualsExpression("birthyear", 1975L) .addGteExpression("birthmonth", 4L) @@ -85,12 +85,12 @@ public void testMultiClause() { } @Test - public void testEqClauseMiss() { - QueryResult> result = + public void testEqClauseMiss() { + QueryResult> result = new IndexedSlicesQuery(keyspace, se, se, le) .addEqualsExpression("birthyear", 5L) .addGteExpression("birthmonth", 4L) - .addLteExpression("birthmonth", 6L) + .addLteExpression("birthmonth", 6L) .setColumnNames("birthyear") .setColumnFamily(cf) .setStartKey("") @@ -100,7 +100,7 @@ public void testEqClauseMiss() { @Test public void testRowCountLimit() { - QueryResult> result = + QueryResult> result = new IndexedSlicesQuery(keyspace, se, se, le) .addEqualsExpression("birthyear", 1975L) .addGteExpression("birthmonth", 4L) @@ -112,5 +112,5 @@ public void testRowCountLimit() { .execute(); assertEquals(2, result.get().getList().size()); } - + } diff --git a/core/src/test/java/me/prettyprint/cassandra/model/MutatorTest.java b/core/src/test/java/me/prettyprint/cassandra/model/MutatorTest.java index 7713946f0..13705132e 100644 --- a/core/src/test/java/me/prettyprint/cassandra/model/MutatorTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/model/MutatorTest.java @@ -1,20 +1,18 @@ package me.prettyprint.cassandra.model; import static me.prettyprint.hector.api.factory.HFactory.createColumn; +import static me.prettyprint.hector.api.factory.HFactory.createCounterColumn; import static me.prettyprint.hector.api.factory.HFactory.createColumnQuery; import static me.prettyprint.hector.api.factory.HFactory.createKeyspace; import static me.prettyprint.hector.api.factory.HFactory.createMutator; import static me.prettyprint.hector.api.factory.HFactory.createSuperColumn; import static me.prettyprint.hector.api.factory.HFactory.getOrCreateCluster; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertNull; -import static org.junit.Assert.assertTrue; - +import static org.junit.Assert.*; import java.util.ArrayList; import java.util.List; import me.prettyprint.cassandra.BaseEmbededServerSetupTest; +import me.prettyprint.cassandra.model.thrift.ThriftCounterColumnQuery; import me.prettyprint.cassandra.serializers.StringSerializer; import me.prettyprint.cassandra.utils.StringUtils; import me.prettyprint.hector.api.Cluster; @@ -25,6 +23,7 @@ import me.prettyprint.hector.api.mutation.MutationResult; import me.prettyprint.hector.api.mutation.Mutator; import me.prettyprint.hector.api.query.ColumnQuery; +import me.prettyprint.hector.api.query.CounterQuery; import me.prettyprint.hector.api.query.QueryResult; import me.prettyprint.hector.api.query.SuperColumnQuery; @@ -61,15 +60,34 @@ public void testInsert() { } @Test - public void testInsertSuper() { + public void testInsertAndDeleteSuper() { Mutator m = createMutator(keyspace, se); List> columnList = new ArrayList>(); columnList.add(createColumn("name","value",se,se)); - HSuperColumn superColumn = - createSuperColumn("super_name", columnList, se, se, se); + HSuperColumn superColumn = createSuperColumn("super_name", columnList, se, se, se); + + // Insert Super Column MutationResult r = m.insert("sk", "Super1", superColumn); + assertTrue("Execute time should be > 0", r.getExecutionTimeMicro() > 0); assertTrue("Should have operated on a host", r.getHostUsed() != null); + + // Fetch and verify it exists. + SuperColumnQuery scq = HFactory.createSuperColumnQuery(keyspace, se, se, se, se); + scq.setColumnFamily("Super1"); + scq.setKey("sk"); + scq.setSuperName("super_name"); + assertEquals("super_name", scq.execute().get().getName()); + + // Remove the Super Column + m.superDelete("sk", "Super1", "super_name", se); + + // Fetch and verify it exists. + scq = HFactory.createSuperColumnQuery(keyspace, se, se, se, se); + scq.setColumnFamily("Super1"); + scq.setKey("sk"); + scq.setSuperName("super_name"); + assertNull("super_name", scq.execute().get()); } @Test @@ -82,23 +100,48 @@ public void testSubDelete() { HSuperColumn superColumn = createSuperColumn("super_name", columnList, se, se, se); MutationResult r = m.insert("sk1", "Super1", superColumn); - + SuperColumnQuery scq = HFactory.createSuperColumnQuery(keyspace, se, se, se, se); scq.setColumnFamily("Super1"); scq.setKey("sk1"); scq.setSuperName("super_name"); assertEquals(3,scq.execute().get().getColumns().size()); - + + m.discardPendingMutations(); + + m.addSubDelete("sk1", "Super1", "super_name", "col_1", se, se); + m.execute(); + + assertEquals(2,scq.execute().get().getColumns().size()); + } + + @Test + public void testSubDeleteHSuperColumn() { + Mutator m = createMutator(keyspace, se); + List> columnList = new ArrayList>(); + columnList.add(createColumn("col_1","val_1",se,se)); + columnList.add(createColumn("col_2","val_2",se,se)); + columnList.add(createColumn("col_3","val_3",se,se)); + HSuperColumn superColumn = + createSuperColumn("super_name", columnList, se, se, se); + MutationResult r = m.insert("sk1", "Super1", superColumn); + + SuperColumnQuery scq = HFactory.createSuperColumnQuery(keyspace, se, se, se, se); + scq.setColumnFamily("Super1"); + scq.setKey("sk1"); + scq.setSuperName("super_name"); + assertEquals(3,scq.execute().get().getColumns().size()); + m.discardPendingMutations(); columnList.remove(1); columnList.remove(0); superColumn.setSubcolumns(columnList); m.addSubDelete("sk1", "Super1", superColumn); m.execute(); - + assertEquals(2,scq.execute().get().getColumns().size()); } - + @Test public void testBatchMutationManagement() { String cf = "Standard1"; @@ -167,6 +210,17 @@ public void testRowDeletion() { assertNull(columnResult.get()); } + @Test + public void testInsertCounter() { + Mutator m = createMutator(keyspace, se); + MutationResult mr = m.insertCounter("k", "Counter1", createCounterColumn("name", 5)); + assertTrue("Execution time on single counter insert should be > 0", mr.getExecutionTimeMicro() > 0); + assertTrue("Should have operated on a host", mr.getHostUsed() != null); + CounterQuery counter = new ThriftCounterColumnQuery(keyspace, se, se); + counter.setColumnFamily("Counter1").setKey("k").setName("name"); + assertEquals(new Long(5), counter.execute().get().getValue()); + } + private void assertColumnExists(String keyspace, String cf, String key, String column) { ColumnPath cp = new ColumnPath(cf); cp.setColumn(StringUtils.bytes(column)); diff --git a/core/src/test/java/me/prettyprint/cassandra/model/RangeSlicesQueryTest.java b/core/src/test/java/me/prettyprint/cassandra/model/RangeSlicesQueryTest.java index b0925b611..63dc5e69a 100644 --- a/core/src/test/java/me/prettyprint/cassandra/model/RangeSlicesQueryTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/model/RangeSlicesQueryTest.java @@ -47,17 +47,17 @@ public void setupCase() { .addInsertion("rangeSlicesTest_key6", cf, createColumn("birthmonth", 6L, se, le)) .execute(); } - + @Test public void testKeysOnlyPredicate() { RangeSlicesQuery rangeSlicesQuery = HFactory.createRangeSlicesQuery(keyspace, se, se, le); - QueryResult> result = + QueryResult> result = rangeSlicesQuery.setColumnFamily(cf).setKeys("", "").setReturnKeysOnly().execute(); OrderedRows orderedRows = result.get(); Row row = orderedRows.iterator().next(); assertNotNull(row.getKey()); assertEquals(0,row.getColumnSlice().getColumns().size()); - + result = rangeSlicesQuery.setColumnNames("birthyear","birthmonth").setRowCount(5).execute(); orderedRows = result.get(); row = orderedRows.iterator().next(); diff --git a/core/src/test/java/me/prettyprint/cassandra/model/SuperColumnSliceTest.java b/core/src/test/java/me/prettyprint/cassandra/model/SuperColumnSliceTest.java index 0455c3d77..0b9980f35 100644 --- a/core/src/test/java/me/prettyprint/cassandra/model/SuperColumnSliceTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/model/SuperColumnSliceTest.java @@ -34,7 +34,9 @@ public void testConstruction() { Assert.assertTrue(slice.getSuperColumns().isEmpty()); // non-empty one - Column c = new Column(le.toByteBuffer(5L), be.toByteBuffer(ByteBuffer.wrap(new byte[] { 1 })), 2L); + Column c = new Column(le.toByteBuffer(5L)); + c.setValue(be.toByteBuffer(ByteBuffer.wrap(new byte[] { 1 }))); + c.setTimestamp(2L); tColumns.add(new SuperColumn(se.toByteBuffer("super"), Arrays.asList(c))); slice = new SuperSliceImpl(tColumns, se, le, be); Assert.assertEquals(1, slice.getSuperColumns().size()); diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/BigIntegerSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/BigIntegerSerializerTest.java new file mode 100644 index 000000000..73c69d2b2 --- /dev/null +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/BigIntegerSerializerTest.java @@ -0,0 +1,21 @@ +package me.prettyprint.cassandra.serializers; + +import static org.junit.Assert.*; + +import java.math.BigInteger; +import java.nio.ByteBuffer; + +import org.junit.Test; + +public class BigIntegerSerializerTest { + + @Test + public void testToAndFrom() { + BigInteger bi = new BigInteger(new byte[]{0,0,0,1}); + ByteBuffer bb = BigIntegerSerializer.get().toByteBuffer(bi); + assertTrue(bb != null); + assertEquals(1,bb.array().length); + assertEquals(bi, BigIntegerSerializer.get().fromByteBuffer(bb)); + } + +} diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/FastInfosetSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/FastInfosetSerializerTest.java index 4c1e11203..d47a7f0bf 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/FastInfosetSerializerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/FastInfosetSerializerTest.java @@ -4,7 +4,7 @@ /** * @author shuzhang0@gmail.com - * + * */ public class FastInfosetSerializerTest extends JaxbSerializerTest { diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/IntegerSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/IntegerSerializerTest.java index 99e193d7e..fb1e935e3 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/IntegerSerializerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/IntegerSerializerTest.java @@ -9,14 +9,14 @@ import org.junit.Test; /** - * + * * @author Bozhidar Bozhanov - * + * */ public class IntegerSerializerTest { static IntegerSerializer ext = IntegerSerializer.get(); - + @Test public void testConversions() { test(0); @@ -35,7 +35,7 @@ public void testFromCassandra() { assertEquals(new Integer(Integer.MAX_VALUE), ext.fromByteBuffer(ByteBufferUtil.bytes(Integer.MAX_VALUE))); assertEquals(new Integer(Integer.MIN_VALUE), ext.fromByteBuffer(ByteBufferUtil.bytes(Integer.MIN_VALUE))); } - + @Test public void testFromCassandraAsBytes() { assertEquals(new Integer(1), ext.fromBytes(ByteBufferUtil.bytes(1).array())); @@ -44,10 +44,10 @@ public void testFromCassandraAsBytes() { assertEquals(new Integer(Integer.MAX_VALUE), ext.fromBytes(ByteBufferUtil.bytes(Integer.MAX_VALUE).array())); assertEquals(new Integer(Integer.MIN_VALUE), ext.fromBytes(ByteBufferUtil.bytes(Integer.MIN_VALUE).array())); } - - + + private void test(Integer number) { - + assertEquals(number, ext.fromByteBuffer(ext.toByteBuffer(number))); // test compatibility with ByteBuffer default byte order diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/JaxbSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/JaxbSerializerTest.java index 6411a5152..08281ec9a 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/JaxbSerializerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/JaxbSerializerTest.java @@ -11,9 +11,9 @@ /** * Unit test for {@link JaxbSerializer}. - * + * * @author shuzhang0@gmail.com - * + * */ public class JaxbSerializerTest extends SerializerBaseTest { diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/LongSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/LongSerializerTest.java index 60555eaa9..31146eda1 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/LongSerializerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/LongSerializerTest.java @@ -11,7 +11,7 @@ public class LongSerializerTest { static LongSerializer ext = LongSerializer.get(); - + @Test public void testConversions() { test(0l); @@ -30,9 +30,9 @@ public void testFromCassandra() { assertEquals(new Long(Long.MIN_VALUE), ext.fromByteBuffer(ByteBufferUtil.bytes(Long.MIN_VALUE))); assertEquals(new Long(Long.MAX_VALUE), ext.fromByteBuffer(ByteBufferUtil.bytes(Long.MAX_VALUE))); } - + private void test(Long number) { - + assertEquals(number, ext.fromByteBuffer(ext.toByteBuffer(number))); // test compatibility with ByteBuffer default byte order diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/SerializerBaseTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/SerializerBaseTest.java index 395544aac..2645f1089 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/SerializerBaseTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/SerializerBaseTest.java @@ -14,9 +14,9 @@ * Tests for:
    * 1. Valid serialization roundtrip for a collection of objects.
    * 2. Proper handling of NULL. - * + * * @author shuzhang0@gmail.com - * + * * @param * the type which the Serializer under test can serialize. */ diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/ShortSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/ShortSerializerTest.java index a54810691..70094d851 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/ShortSerializerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/ShortSerializerTest.java @@ -7,9 +7,9 @@ /** * Unit tests for {@link ShortSerializer}. - * + * * @author shuzhang0@gmail.com - * + * */ public class ShortSerializerTest extends SerializerBaseTest { diff --git a/core/src/test/java/me/prettyprint/cassandra/serializers/StringSerializerTest.java b/core/src/test/java/me/prettyprint/cassandra/serializers/StringSerializerTest.java index ff11cf315..774d96a85 100644 --- a/core/src/test/java/me/prettyprint/cassandra/serializers/StringSerializerTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/serializers/StringSerializerTest.java @@ -1,10 +1,10 @@ package me.prettyprint.cassandra.serializers; - import java.io.UnsupportedEncodingException; import java.util.Arrays; import java.util.Collection; +import org.apache.cassandra.db.marshal.UTF8Type; import org.apache.cassandra.utils.ByteBufferUtil; import org.apache.commons.lang.RandomStringUtils; import org.junit.Assert; @@ -25,22 +25,22 @@ public StringSerializerTest(String str) { @Parameters public static Collection data() throws UnsupportedEncodingException { - Object[][] data = new Object[][] { - {"" }, - {null}, - {"123"}, - {"QWER"}, - {"!@#$#$^%&^*fdghdfghdfgh%^&*"}, - {new String("\u05E9".getBytes(), "utf-8")}, - {RandomStringUtils.random(256*256)} - }; + Object[][] data = new Object[][] { { "" }, { null }, { "123" }, { "QWER" }, + { "!@#$#$^%&^*fdghdfghdfgh%^&*" }, + { new String("\u05E9".getBytes(), "utf-8") }, + { RandomStringUtils.random(256 * 256) } }; return Arrays.asList(data); } @Test - public void test() throws Exception { - Assert.assertEquals(str, s.fromByteBuffer(s.toByteBuffer(str))) ; - if ( str != null) - Assert.assertEquals(str, ByteBufferUtil.string(ByteBufferUtil.bytes(str))); + public void test() throws Exception { + Assert.assertEquals(str, s.fromByteBuffer(s.toByteBuffer(str))); + if (str != null) { + Assert + .assertEquals(str, ByteBufferUtil.string(ByteBufferUtil.bytes(str))); + } + if (str != null) { + UTF8Type.instance.validate(s.toByteBuffer(str)); + } } } diff --git a/core/src/test/java/me/prettyprint/cassandra/service/BatchMutationTest.java b/core/src/test/java/me/prettyprint/cassandra/service/BatchMutationTest.java index ce636973b..fa2667ae7 100644 --- a/core/src/test/java/me/prettyprint/cassandra/service/BatchMutationTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/service/BatchMutationTest.java @@ -1,6 +1,8 @@ package me.prettyprint.cassandra.service; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; import java.nio.ByteBuffer; import java.util.ArrayList; @@ -11,6 +13,8 @@ import me.prettyprint.cassandra.serializers.StringSerializer; import org.apache.cassandra.thrift.Column; +import org.apache.cassandra.thrift.CounterColumn; +import org.apache.cassandra.thrift.CounterSuperColumn; import org.apache.cassandra.thrift.Deletion; import org.apache.cassandra.thrift.Mutation; import org.apache.cassandra.thrift.SlicePredicate; @@ -32,7 +36,9 @@ public void setup() { @Test public void testAddInsertion() { - Column column = new Column(StringSerializer.get().toByteBuffer("c_name"), StringSerializer.get().toByteBuffer("c_val"), System.currentTimeMillis()); + Column column = new Column(StringSerializer.get().toByteBuffer("c_name")); + column.setValue(StringSerializer.get().toByteBuffer("c_val")); + column.setTimestamp(System.currentTimeMillis()); batchMutate.addInsertion("key1", columnFamilies, column); // assert there is one outter map row with 'key' as the key Map>> mutationMap = batchMutate.getMutationMap(); @@ -41,23 +47,29 @@ public void testAddInsertion() { // add again with a different column and verify there is one key and two mutations underneath // for "standard1" - Column column2 = new Column(StringSerializer.get().toByteBuffer("c_name2"), StringSerializer.get().toByteBuffer("c_val2"), System.currentTimeMillis()); + Column column2 = new Column(StringSerializer.get().toByteBuffer("c_name2")); + column2.setValue(StringSerializer.get().toByteBuffer("c_val2")); + column2.setTimestamp(System.currentTimeMillis()); batchMutate.addInsertion("key1",columnFamilies, column2); assertEquals(2, mutationMap.get(StringSerializer.get().toByteBuffer("key1")).get("Standard1").size()); } @Test public void testAddSuperInsertion() { - SuperColumn sc = new SuperColumn(StringSerializer.get().toByteBuffer("c_name"), - Arrays.asList(new Column(StringSerializer.get().toByteBuffer("c_name"), StringSerializer.get().toByteBuffer("c_val"), System.currentTimeMillis()))); + Column column = new Column(StringSerializer.get().toByteBuffer("c_name")); + column.setValue(StringSerializer.get().toByteBuffer("c_val")); + column.setTimestamp(System.currentTimeMillis()); + SuperColumn sc = new SuperColumn(StringSerializer.get().toByteBuffer("c_name"), Arrays.asList(column)); batchMutate.addSuperInsertion("key1", columnFamilies, sc); // assert there is one outter map row with 'key' as the key assertEquals(1, batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).size()); // add again with a different column and verify there is one key and two mutations underneath // for "standard1" - SuperColumn sc2 = new SuperColumn(StringSerializer.get().toByteBuffer("c_name2"), - Arrays.asList(new Column(StringSerializer.get().toByteBuffer("c_name"), StringSerializer.get().toByteBuffer("c_val"), System.currentTimeMillis()))); + column = new Column(StringSerializer.get().toByteBuffer("c_name")); + column.setValue(StringSerializer.get().toByteBuffer("c_val")); + column.setTimestamp(System.currentTimeMillis()); + SuperColumn sc2 = new SuperColumn(StringSerializer.get().toByteBuffer("c_name2"), Arrays.asList(column)); batchMutate.addSuperInsertion("key1", columnFamilies, sc2); assertEquals(2, batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).get("Standard1").size()); } @@ -65,7 +77,7 @@ public void testAddSuperInsertion() { @Test public void testAddDeletion() { - Deletion deletion = new Deletion(System.currentTimeMillis()); + Deletion deletion = new Deletion().setTimestamp(System.currentTimeMillis()); SlicePredicate slicePredicate = new SlicePredicate(); slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("c_name")); deletion.setPredicate(slicePredicate); @@ -73,11 +85,91 @@ public void testAddDeletion() { assertEquals(1,batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).size()); - deletion = new Deletion(System.currentTimeMillis()); + deletion = new Deletion().setTimestamp(System.currentTimeMillis()); slicePredicate = new SlicePredicate(); slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("c_name2")); deletion.setPredicate(slicePredicate); batchMutate.addDeletion("key1", columnFamilies, deletion); assertEquals(2,batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).get("Standard1").size()); } + + @Test + public void testIsEmpty() { + assertTrue(batchMutate.isEmpty()); + + // Insert a column + Column c1 = new Column(StringSerializer.get().toByteBuffer("c_name")); + c1.setValue(StringSerializer.get().toByteBuffer("c_val")); + c1.setTimestamp(System.currentTimeMillis()); + batchMutate.addInsertion("key1", columnFamilies, c1); + assertFalse(batchMutate.isEmpty()); + + // Insert a Counter. + CounterColumn cc1 = new CounterColumn(StringSerializer.get().toByteBuffer("c_name"), 13); + batchMutate.addCounterInsertion("key1", columnFamilies, cc1); + assertFalse(batchMutate.isEmpty()); + + + + } + + // ********** Test Counters related operations ****************** + + @Test + public void testAddCounterInsertion() { + + // Insert a Counter. + CounterColumn cc1 = new CounterColumn(StringSerializer.get().toByteBuffer("c_name"), 222); + + batchMutate.addCounterInsertion("key1", columnFamilies, cc1); + + // assert there is one outter map row with 'key' as the key + Map>> mutationMap = batchMutate.getMutationMap(); + assertEquals(1, mutationMap.get(StringSerializer.get().toByteBuffer("key1")).size()); + + // add again with a different counter and verify there is one key and two mutations underneath + // for "standard1" + CounterColumn cc2 = new CounterColumn(StringSerializer.get().toByteBuffer("c_name2"), 44); + batchMutate.addCounterInsertion("key1", columnFamilies, cc2); + assertEquals(2, mutationMap.get(StringSerializer.get().toByteBuffer("key1")).get("Standard1").size()); + } + + @Test + public void testAddCounterDeletion() { + + Deletion counterDeletion = new Deletion(); + + SlicePredicate slicePredicate = new SlicePredicate(); + slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("c_name")); + counterDeletion.setPredicate(slicePredicate); + + batchMutate.addDeletion("key1", columnFamilies, counterDeletion); + + assertEquals(1, batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).size()); + + counterDeletion = new Deletion(); + slicePredicate = new SlicePredicate(); + slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("c_name2")); + counterDeletion.setPredicate(slicePredicate); + batchMutate.addDeletion("key1", columnFamilies, counterDeletion); + assertEquals(2,batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).get("Standard1").size()); + } + + @Test + public void testAddSuperCounterInsertion() { + // Create 1 super counter. + CounterSuperColumn csc1 = new CounterSuperColumn(StringSerializer.get().toByteBuffer("c_name"), + Arrays.asList(new CounterColumn(StringSerializer.get().toByteBuffer("c_name"), 123))); + + batchMutate.addSuperCounterInsertion("key1", columnFamilies, csc1); + // assert there is one outter map row with 'key' as the key + assertEquals(1, batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).size()); + + // add again with a different column and verify there is one key and two mutations underneath + // for "standard1" + CounterSuperColumn csc2 = new CounterSuperColumn(StringSerializer.get().toByteBuffer("c_name2"), + Arrays.asList(new CounterColumn(StringSerializer.get().toByteBuffer("c_name"), 456))); + batchMutate.addSuperCounterInsertion("key1", columnFamilies, csc2); + assertEquals(2, batchMutate.getMutationMap().get(StringSerializer.get().toByteBuffer("key1")).get("Standard1").size()); + } } diff --git a/core/src/test/java/me/prettyprint/cassandra/service/CassandraAuthTest.java b/core/src/test/java/me/prettyprint/cassandra/service/CassandraAuthTest.java index 026dd36d0..5289a17e3 100644 --- a/core/src/test/java/me/prettyprint/cassandra/service/CassandraAuthTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/service/CassandraAuthTest.java @@ -54,7 +54,7 @@ public class CassandraAuthTest { private CassandraHostConfigurator cassandraHostConfigurator; private HConnectionManager connectionManager; private String clusterName = "TestCluster"; - + /** * Set embedded cassandra up and spawn it in a new thread. */ @@ -113,7 +113,7 @@ public void testDescribeKeyspace() throws Exception { ThriftCluster cassandraCluster = new ThriftCluster("Test Cluster", cassandraHostConfigurator, user1Credentials); KeyspaceDefinition keyspaceDetail = cassandraCluster.describeKeyspace("Keyspace1"); assertNotNull(keyspaceDetail); - assertEquals(7, keyspaceDetail.getCfDefs().size()); + assertEquals(21, keyspaceDetail.getCfDefs().size()); } @Test @@ -135,11 +135,11 @@ public void testAddDropKeyspace() throws Exception { String ksid2 = cassandraCluster.dropKeyspace("DynKeyspace"); assertNotNull(ksid2); } - + @Test public void testInsertAndGetAndRemove() throws IllegalArgumentException, NoSuchElementException, IllegalStateException, HNotFoundException, Exception { - KeyspaceService keyspace = new KeyspaceServiceImpl("Keyspace1", new QuorumAllConsistencyLevelPolicy(), + KeyspaceService keyspace = new KeyspaceServiceImpl("Keyspace1", new QuorumAllConsistencyLevelPolicy(), connectionManager, FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE, user1Credentials); // insert value @@ -177,7 +177,7 @@ public void testInsertAndGetAndRemove() throws IllegalArgumentException, NoSuchE @Test public void testInsertAndGetAndRemoveBadAuth() throws IllegalArgumentException, NoSuchElementException, IllegalStateException, HNotFoundException, Exception { - KeyspaceService keyspace = new KeyspaceServiceImpl("Keyspace1", new QuorumAllConsistencyLevelPolicy(), + KeyspaceService keyspace = new KeyspaceServiceImpl("Keyspace1", new QuorumAllConsistencyLevelPolicy(), connectionManager, FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE, user1CredentialsBad); try { // insert value diff --git a/core/src/test/java/me/prettyprint/cassandra/service/CassandraClusterTest.java b/core/src/test/java/me/prettyprint/cassandra/service/CassandraClusterTest.java index 7e0bda8da..727ef400e 100644 --- a/core/src/test/java/me/prettyprint/cassandra/service/CassandraClusterTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/service/CassandraClusterTest.java @@ -60,7 +60,7 @@ public void testDescribeClusterName() throws Exception { */ @Test public void testDescribeThriftVersion() throws Exception { - assertEquals("19.4.0",cassandraCluster.describeThriftVersion()); + assertEquals("19.10.0",cassandraCluster.describeThriftVersion()); } @Test @@ -75,7 +75,7 @@ public void testDescribeRing() throws Exception { public void testDescribeKeyspace() throws Exception { KeyspaceDefinition keyspaceDetail = cassandraCluster.describeKeyspace("Keyspace1"); assertNotNull(keyspaceDetail); - assertEquals(4, keyspaceDetail.getCfDefs().size()); + assertEquals(21, keyspaceDetail.getCfDefs().size()); } @Test @@ -91,7 +91,7 @@ public void testAddDropColumnFamily() throws Exception { String cfid2 = cassandraCluster.dropColumnFamily("Keyspace1", "DynCf"); assertNotNull(cfid2); } - + @Test public void testTruncateColumnFamily() throws Exception { ColumnFamilyDefinition cfDef = HFactory.createColumnFamilyDefinition("Keyspace1", "TruncateableCf"); @@ -115,65 +115,65 @@ public void testAddDropKeyspace() throws Exception { String ksid2 = cassandraCluster.dropKeyspace("DynKeyspace"); assertNotNull(ksid2); } - + @Test public void testEditKeyspace() throws Exception { - + BasicColumnFamilyDefinition columnFamilyDefinition = new BasicColumnFamilyDefinition(); columnFamilyDefinition.setKeyspaceName("DynKeyspace2"); - columnFamilyDefinition.setName("DynamicCF"); - + columnFamilyDefinition.setName("DynamicCF"); + ColumnFamilyDefinition cfDef = new ThriftCfDef(columnFamilyDefinition); - - KeyspaceDefinition keyspaceDefinition = + + KeyspaceDefinition keyspaceDefinition = HFactory.createKeyspaceDefinition("DynKeyspace2", "org.apache.cassandra.locator.SimpleStrategy", 1, Arrays.asList(cfDef)); - + cassandraCluster.addKeyspace(keyspaceDefinition); - - keyspaceDefinition = + + keyspaceDefinition = HFactory.createKeyspaceDefinition("DynKeyspace2", "org.apache.cassandra.locator.SimpleStrategy", 2, null); - + cassandraCluster.updateKeyspace(keyspaceDefinition); - + KeyspaceDefinition fromCluster = cassandraCluster.describeKeyspace("DynKeyspace2"); assertEquals(2,fromCluster.getReplicationFactor()); cassandraCluster.dropKeyspace("DynKeyspace2"); } - + @Test public void testEditColumnFamily() throws Exception { - + BasicColumnFamilyDefinition columnFamilyDefinition = new BasicColumnFamilyDefinition(); columnFamilyDefinition.setKeyspaceName("DynKeyspace3"); - columnFamilyDefinition.setName("DynamicCF"); - + columnFamilyDefinition.setName("DynamicCF"); + ColumnFamilyDefinition cfDef = new ThriftCfDef(columnFamilyDefinition); - - KeyspaceDefinition keyspaceDefinition = + + KeyspaceDefinition keyspaceDefinition = HFactory.createKeyspaceDefinition("DynKeyspace3", "org.apache.cassandra.locator.SimpleStrategy", 1, Arrays.asList(cfDef)); - + cassandraCluster.addKeyspace(keyspaceDefinition); - - + + KeyspaceDefinition fromCluster = cassandraCluster.describeKeyspace("DynKeyspace3"); cfDef = fromCluster.getCfDefs().get(0); - + columnFamilyDefinition = new BasicColumnFamilyDefinition(cfDef); BasicColumnDefinition columnDefinition = new BasicColumnDefinition(); columnDefinition.setName(StringSerializer.get().toByteBuffer("birthdate")); columnDefinition.setIndexType(ColumnIndexType.KEYS); columnDefinition.setValidationClass(ComparatorType.LONGTYPE.getClassName()); columnFamilyDefinition.addColumnDefinition(columnDefinition); - + columnDefinition = new BasicColumnDefinition(); - columnDefinition.setName(StringSerializer.get().toByteBuffer("nonindexed_field")); + columnDefinition.setName(StringSerializer.get().toByteBuffer("nonindexed_field")); columnDefinition.setValidationClass(ComparatorType.LONGTYPE.getClassName()); - columnFamilyDefinition.addColumnDefinition(columnDefinition); - + columnFamilyDefinition.addColumnDefinition(columnDefinition); + cassandraCluster.updateColumnFamily(new ThriftCfDef(columnFamilyDefinition)); - + fromCluster = cassandraCluster.describeKeyspace("DynKeyspace3"); - + assertEquals("birthdate",StringSerializer.get().fromByteBuffer(fromCluster.getCfDefs().get(0).getColumnMetadata().get(0).getName())); assertEquals("nonindexed_field",StringSerializer.get().fromByteBuffer(fromCluster.getCfDefs().get(0).getColumnMetadata().get(1).getName())); } diff --git a/core/src/test/java/me/prettyprint/cassandra/service/KeyspaceTest.java b/core/src/test/java/me/prettyprint/cassandra/service/KeyspaceTest.java index d7ded05a1..9531e01d0 100644 --- a/core/src/test/java/me/prettyprint/cassandra/service/KeyspaceTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/service/KeyspaceTest.java @@ -11,6 +11,7 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; +import java.util.Comparator; import java.util.HashMap; import java.util.List; import java.util.Map; @@ -28,7 +29,7 @@ import org.apache.cassandra.thrift.ColumnOrSuperColumn; import org.apache.cassandra.thrift.ColumnParent; import org.apache.cassandra.thrift.ColumnPath; -//import org.apache.cassandra.thrift.ConsistencyLevel; +import org.apache.cassandra.thrift.CounterColumn; import org.apache.cassandra.thrift.Deletion; import org.apache.cassandra.thrift.KeyRange; import org.apache.cassandra.thrift.Mutation; @@ -59,8 +60,8 @@ public class KeyspaceTest extends BaseEmbededServerSetupTest { @Before public void setupCase() throws IllegalStateException, PoolExhaustedException, Exception { super.setupClient(); - - keyspace = new KeyspaceServiceImpl("Keyspace1", new QuorumAllConsistencyLevelPolicy(), + + keyspace = new KeyspaceServiceImpl("Keyspace1", new QuorumAllConsistencyLevelPolicy(), connectionManager, FailoverPolicy.ON_FAIL_TRY_ALL_AVAILABLE); } @@ -100,6 +101,68 @@ public void testInsertAndGetAndRemove() throws IllegalArgumentException, NoSuchE } } + @Test + public void testInsertAndGetAndRemoveCounter() throws IllegalArgumentException, NoSuchElementException, + IllegalStateException, HNotFoundException, Exception { + + StringSerializer ss = StringSerializer.get(); + // insert value + ColumnParent cp = new ColumnParent("Counter1"); + //cp.setColumn(bytes("testInsertAndGetAndRemoveCounter")); + + // Insert 3 counters for the same key + keyspace.addCounter("testInsertAndGetAndRemoveCounter_key1", cp, createCounterColumn("A", 5L)); + keyspace.addCounter("testInsertAndGetAndRemoveCounter_key1", cp, createCounterColumn("A", -1L)); + keyspace.addCounter("testInsertAndGetAndRemoveCounter_key1", cp, createCounterColumn("B", 10L)); + // Total for counter A is (5 - 1) = 4. + // Total for counter B is 10. + + ColumnPath cph = new ColumnPath("Counter1"); + cph.setColumn(ss.toByteBuffer("A")); + CounterColumn counter = keyspace.getCounter("testInsertAndGetAndRemoveCounter_key1", cph); + assertNotNull(counter); + assertEquals(4, counter.value); + + cph.setColumn(ss.toByteBuffer("B")); + counter = keyspace.getCounter("testInsertAndGetAndRemoveCounter_key1", cph); + assertNotNull(counter); + assertEquals(10, counter.value); + + // Reuse the ColumnPath associated to Conter B and remove only B. + keyspace.removeCounter("testInsertAndGetAndRemoveCounter_key1", cph); + + // Fetch it and it should not exist + try { + keyspace.getCounter("testInsertAndGetAndRemoveCounter_key1", cph); + } catch (HNotFoundException e) { + // good + } + + // Fetch Counter A again to verify I did not delete it + cph.setColumn(ss.toByteBuffer("A")); + counter = keyspace.getCounter("testInsertAndGetAndRemoveCounter_key1", cph); + assertNotNull(counter); + + // Delete the whole row + cph.column = null; + keyspace.removeCounter("testInsertAndGetAndRemoveCounter_key1", cph); + + // Fetch A again to verify it is not there. + try { + cph.setColumn(ss.toByteBuffer("A")); + counter = keyspace.getCounter("testInsertAndGetAndRemoveCounter_key1", cph); + } catch (HNotFoundException e) { + // good + } + } + + private CounterColumn createCounterColumn(String name, long value) { + CounterColumn cc = new CounterColumn(); + cc.setName(StringSerializer.get().toByteBuffer(name)); + cc.setValue(value); + return cc; + } + /** * Test insertion of a supercolumn using insert */ @@ -110,10 +173,9 @@ public void testInsertSuper() throws IllegalArgumentException, NoSuchElementExce // insert value ColumnParent columnParent = new ColumnParent("Super1"); columnParent.setSuper_column(StringSerializer.get().toByteBuffer("testInsertSuper_super")); - Column column = new Column(StringSerializer.get().toByteBuffer("testInsertSuper_column"), - StringSerializer.get().toByteBuffer("testInsertSuper_value"), connectionManager.createClock()); - - + Column column = new Column(StringSerializer.get().toByteBuffer("testInsertSuper_column")); + column.setValue(StringSerializer.get().toByteBuffer("testInsertSuper_value")); + column.setTimestamp(connectionManager.createClock()); keyspace.insert(StringSerializer.get().toByteBuffer("testInsertSuper_key"), columnParent, column); column.setName(StringSerializer.get().toByteBuffer("testInsertSuper_column2")); @@ -184,8 +246,9 @@ public void testBatchMutate() throws HectorException { ArrayList mutations = new ArrayList(10); for (int j = 0; j < 10; j++) { - Column col = new Column(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j), - StringSerializer.get().toByteBuffer("testBatchMutateColumn_value_" + j), connectionManager.createClock()); + Column col = new Column(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j)); + col.setValue(StringSerializer.get().toByteBuffer("testBatchMutateColumn_value_" + j)); + col.setTimestamp(connectionManager.createClock()); //list.add(col); ColumnOrSuperColumn cosc = new ColumnOrSuperColumn(); cosc.setColumn(col); @@ -223,7 +286,7 @@ public void testBatchMutate() throws HectorException { slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j)); } Mutation mutation = new Mutation(); - Deletion deletion = new Deletion(connectionManager.createClock()); + Deletion deletion = new Deletion().setTimestamp(connectionManager.createClock()); deletion.setPredicate(slicePredicate); mutation.setDeletion(deletion); mutations.add(mutation); @@ -254,8 +317,9 @@ public void testBatchMutateBatchMutation() throws HectorException { for (int i = 0; i < 10; i++) { for (int j = 0; j < 10; j++) { - Column col = new Column(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j), - StringSerializer.get().toByteBuffer("testBatchMutateColumn_value_" + j), connectionManager.createClock()); + Column col = new Column(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j)); + col.setValue(StringSerializer.get().toByteBuffer("testBatchMutateColumn_value_" + j)); + col.setTimestamp(connectionManager.createClock()); batchMutation.addInsertion("testBatchMutateColumn_" + i, columnFamilies, col); } } @@ -281,7 +345,7 @@ public void testBatchMutateBatchMutation() throws HectorException { for (int j = 0; j < 10; j++) { slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j)); } - Deletion deletion = new Deletion(connectionManager.createClock()); + Deletion deletion = new Deletion().setTimestamp(connectionManager.createClock()); deletion.setPredicate(slicePredicate); batchMutation.addDeletion("testBatchMutateColumn_" + i, columnFamilies, deletion); } @@ -318,15 +382,16 @@ public void testBatchUpdateInsertAndDelOnSame() throws HectorException { for (int i = 0; i < 10; i++) { for (int j = 0; j < 10; j++) { - Column col = new Column(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j), - StringSerializer.get().toByteBuffer("testBatchMutateColumn_value_" + j), connectionManager.createClock()); + Column col = new Column(StringSerializer.get().toByteBuffer("testBatchMutateColumn_" + j)); + col.setValue(StringSerializer.get().toByteBuffer("testBatchMutateColumn_value_" + j)); + col.setTimestamp(connectionManager.createClock()); batchMutation.addInsertion("testBatchMutateColumn_" + i, columnFamilies, col); } } SlicePredicate slicePredicate = new SlicePredicate(); slicePredicate.addToColumn_names(StringSerializer.get().toByteBuffer("deleteThroughInserBatch_col")); - Deletion deletion = new Deletion(connectionManager.createClock()); + Deletion deletion = new Deletion().setTimestamp(connectionManager.createClock()); deletion.setPredicate(slicePredicate); batchMutation.addDeletion("deleteThroughInserBatch_key", columnFamilies, deletion); @@ -417,6 +482,55 @@ public void testGetSlice() throws HectorException { } @Test + public void testGetCounterSlice() throws HectorException { + // insert value + ArrayList columnnames = new ArrayList(50); + final StringSerializer ss = StringSerializer.get(); + for (int i = 0; i < 100; i++) { + ColumnParent cp = new ColumnParent("Counter1"); + + keyspace.addCounter("testGetCounterSlice", cp, createCounterColumn("testGetCounterSlice_" + i, i)); + + if (i < 50) { + // I want to query only 50. + columnnames.add(ss.toByteBuffer("testGetCounterSlice_" + i)); + } + } + + // Query 50 counters. From testGetCounterSlice_0 to testGetCounterSlice_49. + ColumnParent clp = new ColumnParent("Counter1"); + + // TODO (patricioe) Slice by range will be in the next snapshot. + //SliceRange sr = new SliceRange(ByteBuffer.wrap(new byte[0]), ByteBuffer.wrap(new byte[49]), false, 150); + SlicePredicate sp = new SlicePredicate(); + //sp.setSlice_range(sr); + sp.setColumn_names(columnnames); + List cols = keyspace.getCounterSlice("testGetCounterSlice", clp, sp); + + assertNotNull(cols); + assertEquals(50, cols.size()); + + Collections.sort(columnnames); + + ArrayList gotlist = new ArrayList(50); + for (int i = 0; i < 50; i++) { + CounterColumn cc = cols.get(i); + gotlist.add(cc.name.duplicate()); + assertEquals(getValueFromName(ss.fromByteBuffer(cc.name.duplicate())), cc.getValue()); + } + assertEquals(columnnames, gotlist); + + // Clean up the data this test wrote + ColumnPath cp = new ColumnPath("Counter1"); + keyspace.removeCounter("testGetCounterSlice", cp); + } + + // extract counter value from names like counter_23. In that case the value is 23. + private long getValueFromName(String counterName) { + return Long.valueOf(counterName.substring(counterName.indexOf("_") +1)); +} + +@Test public void testGetSuperSlice() throws HectorException { // insert value for (int i = 0; i < 100; i++) { @@ -819,7 +933,7 @@ public void testMultigetCount() { assertEquals(5,counts.size()); assertEquals(new Integer(25),counts.entrySet().iterator().next().getValue()); - slicePredicate.setSlice_range(new SliceRange(StringSerializer.get().toByteBuffer(""), + slicePredicate.setSlice_range(new SliceRange(StringSerializer.get().toByteBuffer(""), StringSerializer.get().toByteBuffer(""), false, 5)); counts = keyspace.multigetCount(keys, clp, slicePredicate); diff --git a/core/src/test/java/me/prettyprint/cassandra/service/template/BaseColumnFamilyTemplateTest.java b/core/src/test/java/me/prettyprint/cassandra/service/template/BaseColumnFamilyTemplateTest.java new file mode 100644 index 000000000..d8b426e10 --- /dev/null +++ b/core/src/test/java/me/prettyprint/cassandra/service/template/BaseColumnFamilyTemplateTest.java @@ -0,0 +1,23 @@ +package me.prettyprint.cassandra.service.template; + +import static me.prettyprint.hector.api.factory.HFactory.createKeyspace; +import static me.prettyprint.hector.api.factory.HFactory.getOrCreateCluster; + +import org.junit.Before; + +import me.prettyprint.cassandra.BaseEmbededServerSetupTest; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.hector.api.Cluster; +import me.prettyprint.hector.api.Keyspace; + +public abstract class BaseColumnFamilyTemplateTest extends BaseEmbededServerSetupTest { + + protected Keyspace keyspace; + static final StringSerializer se = StringSerializer.get(); + + @Before + public void setupLocal() { + Cluster cluster = getOrCreateCluster("MyCluster", "127.0.0.1:9170"); + keyspace = createKeyspace("Keyspace1", cluster); + } +} diff --git a/core/src/test/java/me/prettyprint/cassandra/service/template/ColumnFamilyTemplateTest.java b/core/src/test/java/me/prettyprint/cassandra/service/template/ColumnFamilyTemplateTest.java new file mode 100644 index 000000000..f5edb646e --- /dev/null +++ b/core/src/test/java/me/prettyprint/cassandra/service/template/ColumnFamilyTemplateTest.java @@ -0,0 +1,72 @@ +package me.prettyprint.cassandra.service.template; + +import static org.junit.Assert.*; + +import java.util.Arrays; + +import me.prettyprint.cassandra.model.HSlicePredicate; +import me.prettyprint.hector.api.factory.HFactory; + +import org.junit.Test; + +public class ColumnFamilyTemplateTest extends BaseColumnFamilyTemplateTest { + + @Test + public void testCreateSelect() { + ColumnFamilyTemplate template = new ThriftColumnFamilyTemplate(keyspace, "Standard1", se, se, HFactory.createMutator(keyspace, se)); + + ColumnFamilyUpdater updater = template.createUpdater("key1"); + updater.setString("column1","value1"); + template.update(updater); + + template.addColumn("column1", se); + ColumnFamilyResult wrapper = template.queryColumns("key1"); + assertEquals("value1",wrapper.getString("column1")); + + } + + @Test + public void testCreateSelectTemplate() { + ColumnFamilyTemplate template = new ThriftColumnFamilyTemplate(keyspace, "Standard1", se, se, HFactory.createMutator(keyspace, se)); + ColumnFamilyUpdater updater = template.createUpdater("key1"); + updater.setString("column1","value1"); + updater.update(); + template.setCount(10); + String value = template.queryColumns("key1", new ColumnFamilyRowMapper() { + @Override + public String mapRow(ColumnFamilyResult results) { + // TODO Auto-generated method stub + return results.getString("column1"); + } + }); + assertEquals("value1",value); + } + + @Test + public void testQueryMultiget() { + ColumnFamilyTemplate template = new ThriftColumnFamilyTemplate(keyspace, "Standard1", se, se, HFactory.createMutator(keyspace, se)); + ColumnFamilyUpdater updater = template.createUpdater("mg_key1"); + updater.setString("column1","value1"); + updater.addKey("mg_key2"); + updater.setString("column1","value2"); + updater.addKey("mg_key3"); + updater.setString("column1","value3"); + template.update(updater); + + template.addColumn("column1", se); + ColumnFamilyResult wrapper = template.queryColumns(Arrays.asList("mg_key1", "mg_key2", "mg_key3")); + assertEquals("value1",wrapper.getString("column1")); + wrapper.next(); + assertEquals("value2",wrapper.getString("column1")); + wrapper.next(); + assertEquals("value3",wrapper.getString("column1")); + } + + @Test + public void testHasNoResults() { + ColumnFamilyTemplate template = new ThriftColumnFamilyTemplate(keyspace, "Standard1", se, se, HFactory.createMutator(keyspace, se)); + assertFalse(template.queryColumns("noresults").hasResults()); + + } + +} diff --git a/core/src/test/java/me/prettyprint/cassandra/service/template/SuperCfTemplateTest.java b/core/src/test/java/me/prettyprint/cassandra/service/template/SuperCfTemplateTest.java new file mode 100644 index 000000000..94809d7c2 --- /dev/null +++ b/core/src/test/java/me/prettyprint/cassandra/service/template/SuperCfTemplateTest.java @@ -0,0 +1,216 @@ +package me.prettyprint.cassandra.service.template; + +import static org.junit.Assert.*; + +import java.util.Arrays; + +import me.prettyprint.hector.api.beans.HColumn; + +import org.junit.Test; + + +public class SuperCfTemplateTest extends BaseColumnFamilyTemplateTest { + + @Test + public void testSuperCfInsertReadTemplate() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey1","super1"); + sUpdater.setString("sub_col_1", "sub_val_1"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumn("skey1", "super1"); + + assertEquals("sub_val_1",result.getString("super1","sub_col_1")); + + sUpdater.deleteSuperColumn(); + sTemplate.update(sUpdater); + assertEquals("super1",sUpdater.getCurrentSuperColumn()); + result = sTemplate.querySuperColumn("skey1", "super1"); + assertFalse(result.hasResults()); + } + + + @Test + public void testSuperCfMultiSc() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey2","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sUpdater.addSuperColumn("super2"); + sUpdater.setString("sub2_col_1", "sub2_val_1"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumns("skey2", Arrays.asList("super1","super2")); + assertEquals(2,result.getSuperColumns().size()); + /*for (String sName : result.getSuperColumns() ) { + result.getString(sName,"sub1_col_1"); + }*/ + + //assertEquals("sub1_val_1",result.getString("sub1_col_1")); + //assertEquals("sub2_val_1",result.next().getString("sub2_col_1")); + + } + + @Test + public void testQuerySingleSubColumn() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey3","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sTemplate.update(sUpdater); + + HColumn myCol = sTemplate.querySingleSubColumn("skey3", "super1", "sub1_col_1", se); + assertEquals("sub1_val_1", myCol.getValue()); + } + + + @Test + public void testQuerySingleSubColumnEmpty() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey3","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sTemplate.update(sUpdater); + + HColumn myCol = sTemplate.querySingleSubColumn("skey3", "super2", "sub1_col_1", se); + assertNull(myCol); + } + + @Test + public void testSuperCfInsertReadMultiKey() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("s_multi_key1","super1"); + sUpdater.setString("sub_col_1", "sub_val_1"); + sUpdater.addKey("s_multi_key2"); + sUpdater.addSuperColumn("super1"); + sUpdater.setString("sub_col_1", "sub_val_2"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumns(Arrays.asList("s_multi_key1","s_multi_key2"), Arrays.asList("super1")); + assertTrue(result.hasResults()); + assertEquals("sub_val_2",result.getString("super1","sub_col_1")); + assertEquals("sub_val_1",result.next().getString("super1","sub_col_1")); + + } + + @Test + public void testSuperCfInsertReadMultiKeyNoSc() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("s_multi_key1","super1"); + sUpdater.setString("sub_col_1", "sub_val_1"); + sUpdater.addKey("s_multi_key2"); + sUpdater.addSuperColumn("super1"); + sUpdater.setString("sub_col_1", "sub_val_2"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumns(Arrays.asList("s_multi_key1","s_multi_key2")); + assertTrue(result.hasResults()); + assertEquals("sub_val_2",result.getString("super1","sub_col_1")); + assertEquals("sub_val_1",result.next().getString("super1","sub_col_1")); + + } + + @Test + public void testSuperCfKeyOnly() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey1","super1"); + sUpdater.setString("sub_col_1", "sub_val_1"); + sUpdater.addSuperColumn("super2"); + sUpdater.setString("sub_col_1", "sub_val_2"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumns("skey1"); + assertEquals(2, result.getSuperColumns().size()); + assertTrue(result.hasResults()); + result = sTemplate.querySuperColumns("skey1-non-existing-key"); + assertNull(result.getActiveSuperColumn()); + } + + @Test + public void testSuperCfNoResults() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + + assertFalse(sTemplate.querySuperColumns("no_results").hasResults()); + } + + @Test + public void testDeleteSubColumns() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey3","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sUpdater.setString("sub1_col_2", "sub1_val_2"); + sUpdater.setString("sub1_col_3", "sub1_val_3"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumn("skey3","super1"); + assertEquals(3, result.getColumnNames().size()); + + sUpdater.deleteSubColumn("sub1_col_1"); + sTemplate.update(sUpdater); + + result = sTemplate.querySuperColumn("skey3","super1"); + assertEquals(2, result.getColumnNames().size()); + } + + @Test + public void testTemplateLevelDeleteSuper() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey_del_super","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumn("skey_del_super","super1"); + assertEquals(1, result.getColumnNames().size()); + + sTemplate.deleteColumn("skey_del_super", "super1"); + + result = sTemplate.querySuperColumn("skey_del_super","super1"); + assertFalse(result.hasResults()); + assertEquals(0, result.getColumnNames().size()); + } + + @Test + public void testTemplateLevelDeleteRow() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey_row_del","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumn("skey_row_del","super1"); + assertEquals(1, result.getColumnNames().size()); + + sTemplate.deleteRow("skey_row_del"); + + result = sTemplate.querySuperColumns("skey_row_del"); + assertFalse(result.hasResults()); + assertEquals(0, result.getSuperColumns().size()); + } + + @Test + public void testTemplateLevelDeleteMiss() { + SuperCfTemplate sTemplate = + new ThriftSuperCfTemplate(keyspace, "Super1", se, se, se); + SuperCfUpdater sUpdater = sTemplate.createUpdater("skey_row_del_miss","super1"); + sUpdater.setString("sub1_col_1", "sub1_val_1"); + sTemplate.update(sUpdater); + + SuperCfResult result = sTemplate.querySuperColumn("skey_row_del_miss","super1"); + assertEquals(1, result.getColumnNames().size()); + + sTemplate.deleteRow("skey_row_miss_foo"); + sTemplate.deleteColumn("skey_row_del", "foo"); + + result = sTemplate.querySuperColumns("skey_row_del_miss"); + assertTrue(result.hasResults()); + assertEquals(1, result.getSuperColumns().size()); + } + +} diff --git a/core/src/test/java/me/prettyprint/cassandra/utils/TimeUUIDUtilsTest.java b/core/src/test/java/me/prettyprint/cassandra/utils/TimeUUIDUtilsTest.java index 6529f565d..f3f8e0545 100644 --- a/core/src/test/java/me/prettyprint/cassandra/utils/TimeUUIDUtilsTest.java +++ b/core/src/test/java/me/prettyprint/cassandra/utils/TimeUUIDUtilsTest.java @@ -3,13 +3,17 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; +import java.util.Date; import java.util.UUID; +import me.prettyprint.cassandra.BaseEmbededServerSetupTest; import me.prettyprint.cassandra.service.clock.MicrosecondsClockResolution; import me.prettyprint.cassandra.service.clock.MicrosecondsSyncClockResolution; import me.prettyprint.hector.api.ClockResolution; import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import com.eaio.uuid.UUIDGen; @@ -20,6 +24,8 @@ * */ public class TimeUUIDUtilsTest { + + private static Logger log = LoggerFactory.getLogger(TimeUUIDUtilsTest.class); /** * This test must be placed FIRST. Please don't change the order. @@ -53,7 +59,7 @@ public void testTimeUUIDAsByteArray() { // Used the previously generated UUID, convert to array and back to UUID. Then compare their times. long timeInUUID = TimeUUIDUtils.getTimeFromUUID(TimeUUIDUtils.asByteArray(uuid)); - assertEquals(uuid.timestamp(), timeInUUID); + assertEquals((uuid.timestamp() - 0x01b21dd213814000L) / 10000, timeInUUID); } @Test @@ -62,7 +68,40 @@ public void testTimeUUIDAsByteBuffer() { UUID actualUuid = TimeUUIDUtils.uuid(TimeUUIDUtils.asByteBuffer(expectedUuid)); assertEquals(expectedUuid, actualUuid); } + + @Test + public void testTimestampConsistency() { + final long originalTime = System.currentTimeMillis(); + + log.info("Original Time: " + originalTime); + log.info("----"); + + final UUID u1 = TimeUUIDUtils.getTimeUUID(originalTime); + + log.info("Java UUID: " + u1); + log.info("Java UUID timestamp: " + u1.timestamp()); + log.info("Date: "+ new Date(u1.timestamp())); + + log.info("----"); + final com.eaio.uuid.UUID u = new com.eaio.uuid.UUID(originalTime, 0); + log.info("eaio UUID: " + u); + log.info("eaio UUID timestamp: " + u.getTime()); + log.info("Date: "+ new Date(u.getTime())); + log.info("----"); + final long actual1 = TimeUUIDUtils.getTimeFromUUID(TimeUUIDUtils.asByteArray(u1)); + log.info("Java UUID to bytes to time: " + actual1); + log.info("Java UUID to bytes time to Date: " + new Date(actual1)); + + log.info("----"); + + final long actual2 = TimeUUIDUtils.getTimeFromUUID(u1); + log.info("Java UUID to time: " + actual2); + log.info("Java UUID to time to Date: " + new Date(actual2)); + + assertEquals(originalTime, actual1); + assertEquals(originalTime, actual2); + } } diff --git a/core/src/test/java/me/prettyprint/hector/api/ApiV2SystemTest.java b/core/src/test/java/me/prettyprint/hector/api/ApiV2SystemTest.java index 432e5a403..a6fc7b744 100644 --- a/core/src/test/java/me/prettyprint/hector/api/ApiV2SystemTest.java +++ b/core/src/test/java/me/prettyprint/hector/api/ApiV2SystemTest.java @@ -3,6 +3,8 @@ import static me.prettyprint.hector.api.factory.HFactory.createColumn; import static me.prettyprint.hector.api.factory.HFactory.createColumnQuery; import static me.prettyprint.hector.api.factory.HFactory.createCountQuery; +import static me.prettyprint.hector.api.factory.HFactory.createCounterColumn; +import static me.prettyprint.hector.api.factory.HFactory.createCounterColumnQuery; import static me.prettyprint.hector.api.factory.HFactory.createKeyspace; import static me.prettyprint.hector.api.factory.HFactory.createMultigetSliceQuery; import static me.prettyprint.hector.api.factory.HFactory.createMultigetSubSliceQuery; @@ -12,6 +14,7 @@ import static me.prettyprint.hector.api.factory.HFactory.createRangeSubSlicesQuery; import static me.prettyprint.hector.api.factory.HFactory.createRangeSuperSlicesQuery; import static me.prettyprint.hector.api.factory.HFactory.createSliceQuery; +import static me.prettyprint.hector.api.factory.HFactory.createCounterSliceQuery; import static me.prettyprint.hector.api.factory.HFactory.createSubColumnQuery; import static me.prettyprint.hector.api.factory.HFactory.createSubCountQuery; import static me.prettyprint.hector.api.factory.HFactory.createSubSliceQuery; @@ -33,7 +36,9 @@ import me.prettyprint.cassandra.BaseEmbededServerSetupTest; import me.prettyprint.cassandra.serializers.StringSerializer; import me.prettyprint.hector.api.beans.ColumnSlice; +import me.prettyprint.hector.api.beans.CounterSlice; import me.prettyprint.hector.api.beans.HColumn; +import me.prettyprint.hector.api.beans.HCounterColumn; import me.prettyprint.hector.api.beans.HSuperColumn; import me.prettyprint.hector.api.beans.OrderedRows; import me.prettyprint.hector.api.beans.OrderedSuperRows; @@ -46,6 +51,7 @@ import me.prettyprint.hector.api.mutation.Mutator; import me.prettyprint.hector.api.query.ColumnQuery; import me.prettyprint.hector.api.query.CountQuery; +import me.prettyprint.hector.api.query.CounterQuery; import me.prettyprint.hector.api.query.MultigetSliceQuery; import me.prettyprint.hector.api.query.MultigetSubSliceQuery; import me.prettyprint.hector.api.query.MultigetSuperSliceQuery; @@ -53,6 +59,7 @@ import me.prettyprint.hector.api.query.RangeSlicesQuery; import me.prettyprint.hector.api.query.RangeSubSlicesQuery; import me.prettyprint.hector.api.query.RangeSuperSlicesQuery; +import me.prettyprint.hector.api.query.SliceCounterQuery; import me.prettyprint.hector.api.query.SliceQuery; import me.prettyprint.hector.api.query.SubColumnQuery; import me.prettyprint.hector.api.query.SubCountQuery; @@ -77,7 +84,7 @@ public class ApiV2SystemTest extends BaseEmbededServerSetupTest { @Before public void setupCase() { - cluster = getOrCreateCluster("MyCluster", "127.0.0.1:9170"); + cluster = getOrCreateCluster("Test Cluster", "127.0.0.1:9170"); ko = createKeyspace(KEYSPACE, cluster); } @@ -87,6 +94,64 @@ public void teardownCase() { cluster = null; } + @Test + public void testInsertGetRemoveCounter() { + String cf = "Counter1"; + Mutator m = createMutator(ko, se); + MutationResult mr = m.insertCounter("testInsertGetRemoveCounter", cf, + createCounterColumn("testInsertGetRemoveCounter_name", 25)); + + log.debug("insert execution time: {}", mr.getExecutionTimeMicro()); + + // get value + CounterQuery q = createCounterColumnQuery(ko, se, se); + q.setColumnFamily(cf).setName("testInsertGetRemoveCounter_name"); + QueryResult> r = q.setKey("testInsertGetRemoveCounter") + .execute(); + assertNotNull(r); + + HCounterColumn c = r.get(); + assertNotNull(c); + Long value = c.getValue(); + assertEquals(25, value.longValue()); + String name = c.getName(); + assertEquals("testInsertGetRemoveCounter_name", name); + assertEquals(q, r.getQuery()); + + // remove value + m = createMutator(ko, se); + MutationResult mr2 = m.deleteCounter("testInsertGetRemoveCounter", cf, "testInsertGetRemoveCounter_name", se); + + // get already removed value + CounterQuery q2 = createCounterColumnQuery(ko, se, se); + q2.setName("testInsertGetRemoveCounter_name").setColumnFamily(cf); + QueryResult> r2 = q2.setKey("testInsertGetRemoveCounter") + .execute(); + assertNotNull(r2); + assertNull("Value should have been deleted", r2.get()); + } + + @Test + public void testIncrementDecrementCounter() { + String cf = "Counter1"; + createMutator(ko, se).incrementCounter("testIncrementDecrementCounter", cf, "testIncrementDecrementCounter_name", 7); + createMutator(ko, se).decrementCounter("testIncrementDecrementCounter", cf, "testIncrementDecrementCounter_name", 2); + + // The total in the counter is 5. (7 - 2) + + // get value + CounterQuery q = createCounterColumnQuery(ko, se, se); + q.setColumnFamily(cf).setName("testIncrementDecrementCounter_name"); + QueryResult> r = q.setKey("testIncrementDecrementCounter") + .execute(); + assertNotNull(r); + + HCounterColumn c = r.get(); + assertNotNull(c); + Long value = c.getValue(); + assertEquals(5, value.longValue()); + } + @Test public void testInsertGetRemove() { String cf = "Standard1"; @@ -100,7 +165,6 @@ public void testInsertGetRemove() { // Check the mutation result metadata // assertEquals("127.0.0.1:9170", mr.getHostUsed()); - assertTrue("Time should be > 0", mr.getExecutionTimeMicro() > 0); log.debug("insert execution time: {}", mr.getExecutionTimeMicro()); @@ -118,13 +182,13 @@ public void testInsertGetRemove() { String name = c.getName(); assertEquals("testInsertGetRemove", name); assertEquals(q, r.getQuery()); - assertTrue("Time should be > 0", r.getExecutionTimeMicro() > 0); + // remove value m = createMutator(ko, se); MutationResult mr2 = m.delete("testInsertGetRemove", cf, "testInsertGetRemove", se); - assertTrue("Time should be > 0", mr2.getExecutionTimeMicro() > 0); + // get already removed value ColumnQuery q2 = createColumnQuery(ko, se, se, se); @@ -148,7 +212,7 @@ public void testBatchInsertGetRemove() { se, se)); } m.execute(); - + // get value ColumnQuery q = createColumnQuery(ko, se, se, se); q.setName("testInsertGetRemove").setColumnFamily(cf); @@ -379,6 +443,49 @@ public void testSliceQuery() { deleteColumns(cleanup); } + @Test + public void testCounterSliceQuery() { + String cf = "Counter1"; + + //TestCleanupDescriptor cleanup = insertColumns(cf, 1, "testSliceQuery", 4, "testSliceQuery"); + Mutator mutator = createMutator(ko, se); + for (int i = 0; i < 10; i++) { + mutator.addCounter("testCounterSliceQuery_key", cf, createCounterColumn("" + i, i)); + } + mutator.execute(); + + // get value + SliceCounterQuery q = createCounterSliceQuery(ko, se, se); + q.setColumnFamily(cf); + q.setKey("testCounterSliceQuery_key"); + + // try with column name first + q.setColumnNames("4", "5", "6"); + QueryResult> r = q.execute(); + + assertNotNull(r); + + CounterSlice slice = r.get(); + + assertNotNull(slice); + + assertEquals(3, slice.getColumns().size()); + + // Test slice.getColumnByName + assertEquals(4, slice.getColumnByName("4").getValue().longValue()); + assertEquals(5, slice.getColumnByName("5").getValue().longValue()); + assertEquals(6, slice.getColumnByName("6").getValue().longValue()); + + // Test slice.getColumns + List> columns = slice.getColumns(); + assertNotNull(columns); + assertEquals(3, columns.size()); + + // Cleanup + mutator.deleteCounter("testCounterSliceQuery_key", cf, null, se); + mutator.execute(); + } + @Test public void testSuperSliceQuery() { String cf = "Super1"; @@ -929,9 +1036,9 @@ private TestCleanupDescriptor insertColumns(String cf, int rowCount, /** * A class describing what kind of cleanup is required at the end of the test. * Just some bookeeping, that's all. - * + * * @author Ran Tavory - * + * */ private static class TestCleanupDescriptor { public final String cf; diff --git a/core/src/test/java/me/prettyprint/hector/api/CompositeTest.java b/core/src/test/java/me/prettyprint/hector/api/CompositeTest.java new file mode 100644 index 000000000..72128199a --- /dev/null +++ b/core/src/test/java/me/prettyprint/hector/api/CompositeTest.java @@ -0,0 +1,260 @@ +package me.prettyprint.hector.api; + +import static me.prettyprint.hector.api.ddl.ComparatorType.UUIDTYPE; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import java.math.BigInteger; +import java.nio.ByteBuffer; +import java.util.Arrays; +import java.util.UUID; + +import me.prettyprint.cassandra.serializers.BigIntegerSerializer; +import me.prettyprint.cassandra.serializers.ByteBufferSerializer; +import me.prettyprint.cassandra.serializers.DynamicCompositeSerializer; +import me.prettyprint.cassandra.serializers.StringSerializer; +import me.prettyprint.cassandra.serializers.UUIDSerializer; +import me.prettyprint.cassandra.utils.TimeUUIDUtils; +import me.prettyprint.hector.api.beans.Composite; +import me.prettyprint.hector.api.beans.DynamicComposite; + +import org.apache.cassandra.db.marshal.UTF8Type; +import org.apache.cassandra.utils.ByteBufferUtil; +import org.apache.cassandra.utils.UUIDGen; +import org.junit.Test; + +public class CompositeTest { + + @Test + public void testDynamicSerialization() throws Exception { + + // test correct serialization sizes for strings + DynamicComposite c = new DynamicComposite(); + c.add("String1"); + ByteBuffer b = c.serialize(); + assertEquals(b.remaining(), 12); + + c.add("String2"); + b = c.serialize(); + assertEquals(b.remaining(), 24); + + // test deserialization of strings + c = new DynamicComposite(); + c.deserialize(b); + assertEquals(2, c.size()); + Object o = c.get(0); + assertEquals("String1", o); + o = c.get(1); + assertEquals("String2", o); + + // test serialization and deserialization of longs + c = new DynamicComposite(); + c.add(new Long(10)); + b = c.serialize(); + c = new DynamicComposite(); + c.deserialize(b); + o = c.get(0); + assertTrue(o instanceof Long); + + // test serialization and deserialization of random UUIDS + c = new DynamicComposite(); + c.add(UUID.randomUUID()); + b = c.serialize(); + c = DynamicComposite.fromByteBuffer(b); + o = c.get(0); + assertTrue(o instanceof UUID); + assertEquals(UUIDTYPE.getTypeName(), c.getComponent(0).getComparator()); + + // test serialization and deserialization of time-based UUIDS + c = new DynamicComposite(); + c.add(TimeUUIDUtils.getUniqueTimeUUIDinMillis()); + b = c.serialize(); + c = DynamicComposite.fromByteBuffer(b); + o = c.get(0); + assertTrue(o instanceof UUID); + assertEquals(UUIDTYPE.getTypeName(), c.getComponent(0).getComparator()); + + // test compatibility with Cassandra unit tests + b = createDynamicCompositeKey("Hello", + TimeUUIDUtils.getUniqueTimeUUIDinMillis(), 10, false); + c = new DynamicComposite(); + c.deserialize(b.slice()); + o = c.get(0); + assertTrue(o instanceof ByteBuffer); + assertEquals("Hello", c.get(0, StringSerializer.get())); + + o = c.get(1); + assertEquals(UUID.class, o.getClass()); + + o = c.get(2); + assertEquals(BigInteger.class, o.getClass()); + assertEquals(BigInteger.valueOf(10), o); + + // test using supplied deserializer rather than auto-mapped + c = new DynamicComposite(); + c.deserialize(b.slice()); + assertTrue(c.get(0, ByteBufferSerializer.get()) instanceof ByteBuffer); + assertTrue(c.get(1, ByteBufferSerializer.get()) instanceof ByteBuffer); + assertTrue(c.get(2, ByteBufferSerializer.get()) instanceof ByteBuffer); + + // test setting a deserializer for specific components + c = new DynamicComposite(); + c.setSerializersByPosition(StringSerializer.get(), null, + ByteBufferSerializer.get()); + c.deserialize(b.slice()); + assertTrue(c.get(0) instanceof String); + assertTrue(c.get(1) instanceof UUID); + assertTrue(c.get(2) instanceof ByteBuffer); + + b = DynamicComposite.toByteBuffer(1, "string", + TimeUUIDUtils.getUniqueTimeUUIDinMillis()); + c = DynamicComposite.fromByteBuffer(b); + assertTrue(c.get(0) instanceof BigInteger); + assertTrue(c.get(1) instanceof String); + assertTrue(c.get(2) instanceof UUID); + + b = DynamicComposite.toByteBuffer((long) 1, "string", + TimeUUIDUtils.getUniqueTimeUUIDinMillis()); + c = DynamicComposite.fromByteBuffer(b); + assertTrue(c.get(0) instanceof Long); + assertTrue(c.get(1) instanceof String); + assertTrue(c.get(2) instanceof UUID); + + b = DynamicComposite.toByteBuffer((byte) 1, "string", UUID.randomUUID()); + c = DynamicComposite.fromByteBuffer(b); + assertTrue(c.get(0) instanceof BigInteger); + assertTrue(c.get(1) instanceof String); + assertTrue(c.get(2) instanceof UUID); + + b = DynamicComposite.toByteBuffer(Arrays.asList(Arrays.asList(0, 1, 2), 3, + 4, 5, Arrays.asList(6, 7, 8))); + c = DynamicComposite.fromByteBuffer(b); + for (int i = 0; i < 9; i++) { + o = c.get(i); + assertTrue(o instanceof BigInteger); + assertEquals(i, ((BigInteger) o).intValue()); + } + + b = DynamicComposite.toByteBuffer("foo"); + c = DynamicComposite.fromByteBuffer(b); + b = c.getComponent(0).getBytes(); + UTF8Type.instance.validate(b); + } + + @Test + public void testNullValueSerialization() throws Exception { + + // test correct serialization with null values and user specified + // serialization + DynamicComposite c = new DynamicComposite(); + c.addComponent(null, StringSerializer.get()); + + DynamicCompositeSerializer serializer = new DynamicCompositeSerializer(); + + ByteBuffer buff = serializer.toByteBuffer(c); + + DynamicComposite result = serializer.fromByteBuffer(buff); + + assertNull(result.get(0)); + } + + @Test + public void testStaticSerialization() throws Exception { + + ByteBuffer b = createCompositeKey("Hello", + TimeUUIDUtils.getUniqueTimeUUIDinMillis(), 10, false); + Composite c = new Composite(); + c.setSerializersByPosition(StringSerializer.get(), UUIDSerializer.get(), + BigIntegerSerializer.get()); + c.deserialize(b.slice()); + assertTrue(c.get(0) instanceof String); + assertTrue(c.get(1) instanceof UUID); + assertTrue(c.get(2) instanceof BigInteger); + } + + // from the Casssandra DynamicCompositeTypeTest unit test + private ByteBuffer createDynamicCompositeKey(String s, UUID uuid, int i, + boolean lastIsOne) { + ByteBuffer bytes = ByteBufferUtil.bytes(s); + int totalSize = 0; + if (s != null) { + totalSize += 2 + 2 + bytes.remaining() + 1; + if (uuid != null) { + totalSize += 2 + 2 + 16 + 1; + if (i != -1) { + totalSize += 2 + "IntegerType".length() + 2 + 1 + 1; + } + } + } + + ByteBuffer bb = ByteBuffer.allocate(totalSize); + + if (s != null) { + bb.putShort((short) (0x8000 | 'b')); + bb.putShort((short) bytes.remaining()); + bb.put(bytes); + bb.put((uuid == null) && lastIsOne ? (byte) 1 : (byte) 0); + if (uuid != null) { + bb.putShort((short) (0x8000 | 't')); + bb.putShort((short) 16); + bb.put(UUIDGen.decompose(uuid)); + bb.put((i == -1) && lastIsOne ? (byte) 1 : (byte) 0); + if (i != -1) { + bb.putShort((short) "IntegerType".length()); + bb.put(ByteBufferUtil.bytes("IntegerType")); + // We are putting a byte only because our test use ints that + // fit in a byte *and* IntegerType.fromString() will + // return something compatible (i.e, putting a full int here + // would break 'fromStringTest') + bb.putShort((short) 1); + bb.put((byte) i); + bb.put(lastIsOne ? (byte) 1 : (byte) 0); + } + } + } + bb.rewind(); + return bb; + } + + // from the Casssandra CompositeTypeTest unit test + static ByteBuffer createCompositeKey(String s, UUID uuid, int i, + boolean lastIsOne) { + ByteBuffer bytes = ByteBufferUtil.bytes(s); + int totalSize = 0; + if (s != null) { + totalSize += 2 + bytes.remaining() + 1; + if (uuid != null) { + totalSize += 2 + 16 + 1; + if (i != -1) { + totalSize += 2 + 1 + 1; + } + } + } + + ByteBuffer bb = ByteBuffer.allocate(totalSize); + + if (s != null) { + bb.putShort((short) bytes.remaining()); + bb.put(bytes); + bb.put((uuid == null) && lastIsOne ? (byte) 1 : (byte) 0); + if (uuid != null) { + bb.putShort((short) 16); + bb.put(UUIDGen.decompose(uuid)); + bb.put((i == -1) && lastIsOne ? (byte) 1 : (byte) 0); + if (i != -1) { + // We are putting a byte only because our test use ints that fit in a + // byte *and* IntegerType.fromString() will + // return something compatible (i.e, putting a full int here would + // break 'fromStringTest') + bb.putShort((short) 1); + bb.put((byte) i); + bb.put(lastIsOne ? (byte) 1 : (byte) 0); + } + } + } + bb.rewind(); + return bb; + } + +} diff --git a/core/src/test/resources/cassandra-auth.yaml b/core/src/test/resources/cassandra-auth.yaml index 3457db548..19fb059aa 100644 --- a/core/src/test/resources/cassandra-auth.yaml +++ b/core/src/test/resources/cassandra-auth.yaml @@ -373,43 +373,4 @@ index_interval: 128 # NOTE: this keyspace definition is for demonstration purposes only. # Cassandra will not load these definitions during startup. See # http://wiki.apache.org/cassandra/FAQ#no_keyspaces for an explanation. -keyspaces: - - name: Keyspace1 - replica_placement_strategy: org.apache.cassandra.locator.RackUnawareStrategy - replication_factor: 1 - column_families: - - name: Standard1 - compare_with: BytesType - - - name: Standard2 - compare_with: UTF8Type - read_repair_chance: 0.1 - keys_cached: 100 - gc_grace_seconds: 0 - - - name: StandardByUUID1 - compare_with: TimeUUIDType - - - - name: Super1 - column_type: Super - compare_with: BytesType - compare_subcolumns_with: BytesType - - - name: Super2 - column_type: Super - compare_subcolumns_with: UTF8Type - rows_cached: 10000 - keys_cached: 50 - comment: 'A column family with supercolumns, whose column and subcolumn names are UTF8 strings' - - - name: Super3 - column_type: Super - compare_with: LongType - comment: 'A column family with supercolumns, whose column names are Longs (8 bytes)' - - - name: Indexed1 - column_metadata: - - name: birthyear - validator_class: LongType - index_type: KEYS + diff --git a/core/src/test/resources/cassandra.yaml b/core/src/test/resources/cassandra.yaml index 279381f1a..1b14a1072 100644 --- a/core/src/test/resources/cassandra.yaml +++ b/core/src/test/resources/cassandra.yaml @@ -371,36 +371,4 @@ index_interval: 128 # NOTE: this keyspace definition is for demonstration purposes only. # Cassandra will not load these definitions during startup. See # http://wiki.apache.org/cassandra/FAQ#no_keyspaces for an explanation. -keyspaces: - - name: Keyspace1 - replica_placement_strategy: org.apache.cassandra.locator.RackUnawareStrategy - replication_factor: 1 - column_families: - - name: Standard1 - compare_with: BytesType - memtable_throughput_in_mb: 24 - key_cache_save_period_in_seconds: 0 - keys_cached: 0 - - - name: StandardByUUID1 - compare_with: TimeUUIDType - memtable_throughput_in_mb: 24 - key_cache_save_period_in_seconds: 0 - keys_cached: 0 - - - name: Super1 - column_type: Super - compare_with: BytesType - compare_subcolumns_with: BytesType - memtable_throughput_in_mb: 24 - key_cache_save_period_in_seconds: 0 - keys_cached: 0 - - - name: Indexed1 - memtable_throughput_in_mb: 24 - key_cache_save_period_in_seconds: 0 - keys_cached: 0 - column_metadata: - - name: birthyear - validator_class: LongType - index_type: KEYS + diff --git a/object-mapper/.gitignore b/object-mapper/.gitignore new file mode 100644 index 000000000..034f39c0b --- /dev/null +++ b/object-mapper/.gitignore @@ -0,0 +1,15 @@ +.idea +.DS_Store +hector.iml +releases +target +tmp +bin +.classpath +.project +.settings +out +*.ipr +*.iws +*.iml +.springBeans diff --git a/object-mapper/pom.xml b/object-mapper/pom.xml index 52d634838..e83530783 100644 --- a/object-mapper/pom.xml +++ b/object-mapper/pom.xml @@ -8,7 +8,7 @@ hector-object-mapper hector-object-mapper - 1.0-05-SNAPSHOT + 1.1-01-SNAPSHOT @@ -29,17 +29,18 @@ - + org.slf4j slf4j-api + 1.6.1 me.prettyprint hector-core - 0.7.0-27-SNAPSHOT + 0.8.0-1-SNAPSHOT org.slf4j @@ -65,6 +66,16 @@ provided + + org.apache.cassandra + cassandra-thrift + + + servlet-api + javax.servlet + + + log4j log4j @@ -77,12 +88,6 @@ 4.8.1 test - - org.mockito - mockito-all - 1.8.2 - test - org.slf4j slf4j-log4j12 @@ -94,8 +99,7 @@ - - + org.apache.cassandra cassandra-javautils test @@ -105,11 +109,12 @@ org.springframework spring-context 3.0.5.RELEASE - - - org.springframework - spring-orm - 3.0.5.RELEASE + + + spring-aop + org.springframework + + org.springframework @@ -118,28 +123,18 @@ test - org.apache.openjpa - openjpa - 2.0.1 - - - org.apache.openjpa - openjpa-kernel - 2.0.1 - - - org.apache.openjpa - openjpa-persistence-jdbc - 2.0.1 - test + org.apache.openjpa + openjpa-persistence + 2.1.0 + jar + compile + + + openjpa-kernel + org.apache.openjpa + + - - org.apache.openjpa - openjpa-persistence-jdbc - 2.0.1 - test-jar - test - diff --git a/object-mapper/src/main/java/me/prettyprint/hom/CFMappingDef.java b/object-mapper/src/main/java/me/prettyprint/hom/CFMappingDef.java index 40a74e677..8c45e474f 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/CFMappingDef.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/CFMappingDef.java @@ -19,10 +19,10 @@ /** * Holder for the mapping between a Class annotated with {@link Entity} and the * Cassandra column family name. - * + * * @author Todd Burruss - * - * @param + * + * @param */ public class CFMappingDef { private Class realClass; @@ -54,7 +54,7 @@ public CFMappingDef(Class clazz) { /** * Setup mapping with defaults for the given class. Does not parse all * annotations. - * + * * @param realClass */ public void setDefaults(Class realClass) { @@ -107,7 +107,7 @@ public void addPropertyDefinition(PropertyMappingDefinition propDef) { public String getColFamName() { return colFamName; } - + public String getEffectiveColFamName() { if ( null != colFamName ) { return colFamName; diff --git a/object-mapper/src/main/java/me/prettyprint/hom/CassandraPersistenceProvider.java b/object-mapper/src/main/java/me/prettyprint/hom/CassandraPersistenceProvider.java deleted file mode 100644 index 4e8306d9e..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/CassandraPersistenceProvider.java +++ /dev/null @@ -1,53 +0,0 @@ -package me.prettyprint.hom; - -import java.util.Map; - -import javax.persistence.EntityManagerFactory; -import javax.persistence.spi.PersistenceProvider; -import javax.persistence.spi.PersistenceUnitInfo; -import javax.persistence.spi.ProviderUtil; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -public class CassandraPersistenceProvider implements PersistenceProvider { - private static Logger log = LoggerFactory.getLogger(CassandraPersistenceProvider.class); - - private Map defProperties; - - public CassandraPersistenceProvider() { - - } - - public CassandraPersistenceProvider(Map map) { - this.defProperties = map; - } - - @Override - public EntityManagerFactory createContainerEntityManagerFactory( - PersistenceUnitInfo info, Map map) { - if ( log.isDebugEnabled() ) { - log.debug("creating EntityManagerFactory {} with properties {} ", "null", map); - } - // TODO Auto-generated method stub - return null; - } - - @Override - public EntityManagerFactory createEntityManagerFactory(String emName, Map map) { - if ( log.isDebugEnabled() ) { - log.debug("creating EntityManagerFactory {} with properties {} ", emName, map); - } - if ( map == null || map.isEmpty()) { - return new EntityManagerFactoryImpl(defProperties); - } - return new EntityManagerFactoryImpl(map); - } - - @Override - public ProviderUtil getProviderUtil() { - // TODO Auto-generated method stub - return null; - } - -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/ClassCacheMgr.java b/object-mapper/src/main/java/me/prettyprint/hom/ClassCacheMgr.java index f1402293d..8b6d5d7f9 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/ClassCacheMgr.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/ClassCacheMgr.java @@ -30,7 +30,7 @@ /** * Manage parsing and caching of class meta-data. - * + * * @author */ public class ClassCacheMgr { @@ -46,9 +46,9 @@ public class ClassCacheMgr { * Examine class hierarchy using {@link CFMappingDef} objects to discover the * given class' "base inheritance" class. A base inheritance class is * determined by {@link CFMappingDef#isBaseInheritanceClass()} - * + * * @param - * + * * @param cfMapDef * @return returns the base in the ColumnFamily mapping hierarchy */ @@ -66,7 +66,7 @@ public CFMappingDef findBaseClassViaMappings(CFMappingDef cfMa /** * Retrieve class mapping meta-data by Class object. - * + * * @param * @param clazz * @param throwException @@ -88,7 +88,7 @@ public CFMappingDef getCfMapDef(Class clazz, boolean throwException) { /** * Retrieve class mapping meta-data by ColumnFamily name. - * + * * @param * @param colFamName * @param throwException @@ -111,9 +111,9 @@ public CFMappingDef getCfMapDef(String colFamName, boolean throwException /** * For each class that should be managed, this method must be called to parse * its annotations and derive its meta-data. - * + * * @param - * + * * @param clazz * @return CFMapping describing the initialized class. */ @@ -349,7 +349,7 @@ private void generateColumnSliceIfNeeded(CFMappingDef cfMapDef) { /** * Find method annotated with the given annotation. - * + * * @param clazz * @param anno * @return returns Method if found, null otherwise diff --git a/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerConfigurator.java b/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerConfigurator.java index c63797207..772f1cacb 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerConfigurator.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerConfigurator.java @@ -9,7 +9,7 @@ /** * Config wrapper around the properties map required in the JPA * specification - * + * * @author zznate * */ @@ -20,30 +20,30 @@ public class EntityManagerConfigurator { public static final String CLUSTER_NAME_PROP = PROP_PREFIX + "clusterName"; public static final String KEYSPACE_PROP = PROP_PREFIX + "keyspace"; public static final String HOST_LIST_PROP = PROP_PREFIX + "hostList"; - + private final String classpathPrefix; private final String clusterName; private final String keyspace; private CassandraHostConfigurator cassandraHostConfigurator; - - + + /** * Construct an EntityManagerConfigurator to extract the propeties related * to entity management * @param properties */ public EntityManagerConfigurator(Map properties) { - this(properties, null); + this(properties, null); } - + /** - * Same as single argument version, but allows for (nullable) + * Same as single argument version, but allows for (nullable) * {@link CassandraHostConfigurator} to be provided explicitly - * + * * @param properties * @param cassandraHostConfigurator */ - public EntityManagerConfigurator(Map properties, + public EntityManagerConfigurator(Map properties, CassandraHostConfigurator cassandraHostConfigurator) { classpathPrefix = getPropertyGently(properties, CLASSPATH_PREFIX_PROP,true); clusterName = getPropertyGently(properties, CLUSTER_NAME_PROP,true); @@ -58,8 +58,8 @@ public EntityManagerConfigurator(Map properties, } this.cassandraHostConfigurator = cassandraHostConfigurator; } - - + + public static String getPropertyGently(Map props, String key, boolean throwError) { if ( props.get(key) != null ) { return props.get(key).toString(); @@ -94,7 +94,7 @@ public String toString() { .append(KEYSPACE_PROP).append(":") .append(keyspace).toString(); } - - - + + + } diff --git a/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerFactoryImpl.java b/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerFactoryImpl.java deleted file mode 100644 index 10d1674a0..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/EntityManagerFactoryImpl.java +++ /dev/null @@ -1,127 +0,0 @@ -package me.prettyprint.hom; - -import java.util.Map; - -import javax.persistence.Cache; -import javax.persistence.EntityManager; -import javax.persistence.EntityManagerFactory; -import javax.persistence.PersistenceUnitUtil; -import javax.persistence.criteria.CriteriaBuilder; -import javax.persistence.metamodel.Metamodel; - -import org.apache.commons.lang.StringUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import me.prettyprint.cassandra.service.CassandraHostConfigurator; -import me.prettyprint.hector.api.Cluster; -import me.prettyprint.hector.api.exceptions.HectorException; -import me.prettyprint.hector.api.factory.HFactory; - -public class EntityManagerFactoryImpl implements EntityManagerFactory { - - private Logger log = LoggerFactory.getLogger(EntityManagerFactoryImpl.class); - - private EntityManagerConfigurator entityManagerConfigurator; - private Cluster cluster; - - public EntityManagerFactoryImpl() { - - } - - public EntityManagerFactoryImpl(Map properties) { - this(new EntityManagerConfigurator(properties)); - - // cassandraHostConfigurator properties - // Steps: - // 1. initialize Hector - // 2. aquire keyspace - // 3. build the cache mgr (internal to EntityManager) - // 4. build the objectMapper (internal to EntityManger) - } - - public EntityManagerFactoryImpl(EntityManagerConfigurator entityManagerConfigurator) { - this.entityManagerConfigurator = entityManagerConfigurator; - this.cluster = HFactory.getOrCreateCluster(entityManagerConfigurator.getClusterName(), - entityManagerConfigurator.getCassandraHostConfigurator()); - - } - - @Override - public void close() { - cluster.getConnectionManager().shutdown(); - } - - @Override - public EntityManager createEntityManager() { - log.debug("building EMF"); - return buildEntityManager(entityManagerConfigurator.getKeyspace()); - } - - private EntityManagerImpl buildEntityManager(String keyspace) { - EntityManagerImpl entityManager = new EntityManagerImpl(HFactory. - createKeyspace(keyspace, cluster), - entityManagerConfigurator.getClasspathPrefix()); - log.debug("Built entityManager {}", entityManager); - return entityManager; - } - - /** - * Looks for the {@link EntityManagerConfigurator#KEYSPACE_PROP} using the - * keyspace already present on the internal EntityManagerConfigurator if already found - * @param map - * @return - */ - @SuppressWarnings("unchecked") - @Override - public EntityManager createEntityManager(Map map) { - log.debug("building EMF with props"); - String keyspaceStr = EntityManagerConfigurator.getPropertyGently(map, EntityManagerConfigurator.KEYSPACE_PROP, false); - if ( StringUtils.isBlank(keyspaceStr) ) { - keyspaceStr = entityManagerConfigurator.getKeyspace(); - } - return buildEntityManager(keyspaceStr); - } - - @Override - public boolean isOpen() { - try { - cluster.describeClusterName(); - return true; - } catch (HectorException he) { - log.debug("isOpen failed to connecto to cluster: {}",he.getMessage()); - } - return false; - } - - @Override - public Cache getCache() { - // TODO Auto-generated method stub - return null; - } - - @Override - public CriteriaBuilder getCriteriaBuilder() { - // TODO Auto-generated method stub - return null; - } - - @Override - public Metamodel getMetamodel() { - // TODO Auto-generated method stub - return null; - } - - @Override - public PersistenceUnitUtil getPersistenceUnitUtil() { - // TODO Auto-generated method stub - return null; - } - - @Override - public Map getProperties() { - // TODO Auto-generated method stub - return null; - } - -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/HectorObjectMapper.java b/object-mapper/src/main/java/me/prettyprint/hom/HectorObjectMapper.java index c413607d9..f06b35bae 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/HectorObjectMapper.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/HectorObjectMapper.java @@ -48,10 +48,10 @@ *

    * As mentioned above all column names must be Strings - doesn't * really make sense to have other types when mapping to object properties. - * + * * @param * Type of object mapping to cassandra row - * + * * @author Todd Burruss */ public class HectorObjectMapper { @@ -70,9 +70,9 @@ public HectorObjectMapper(ClassCacheMgr cacheMgr) { /** * Retrieve columns from cassandra keyspace and column family, instantiate a * new object of required type, and then map them to the object's properties. - * - * @param - * + * + * @param + * * @param keyspace * @param colFamName * @param pkObj @@ -183,7 +183,7 @@ private byte[] callMethodAndConvertToCassandraType(Object obj, Method meth, Conv * {@link HectorExtraProperties}. If so call * {@link HectorExtraProperties#addExtraProperty(String, String)}, on the * object. - * + * * @param id * ID (row key) of the object we are retrieving from Cassandra * @param clazz @@ -191,7 +191,7 @@ private byte[] callMethodAndConvertToCassandraType(Object obj, Method meth, Conv * @param slice * column slice from Hector of type * ColumnSlice - * + * * @return instantiated object if success, null if slice is empty, * RuntimeException otherwise */ @@ -239,7 +239,7 @@ else if (null != cfMapDef.getDiscColumn() && colName.equals(cfMapDef.getDiscColu /** * Create Set of HColumns for the given Object. The Object must be annotated * with {@link Column} on the desired fields. - * + * * @param obj * @return */ @@ -254,7 +254,7 @@ private Collection> createColumnSet(Object obj) { /** * Creates a Map of property names as key and HColumns as value. See # - * + * * @param obj * @return */ diff --git a/object-mapper/src/main/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyImpl.java b/object-mapper/src/main/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyImpl.java index 50c163403..e8c228185 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyImpl.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyImpl.java @@ -7,12 +7,12 @@ /** * Keys in Cassandra cannot inherently be multi-field, so a strategy must be * employed to concatenate fields together. - * + * *

    * This strategy uses a delimiter to segment the fields. By default the * delimiter is 2 pipes, ||, but can be overriden by calling * {@link #setDelimiter(byte[])}. - * + * * @author B. Todd Burruss */ public class KeyConcatenationDelimiterStrategyImpl implements KeyConcatenationStrategy { @@ -54,7 +54,7 @@ public List split(byte[] colFamKey) { break; } } - + if (delimiterFound) { int segSize = bb.position() - segStart-delimiter.length; segmentList.add(copyFromMark(bb, segStart, segSize)); @@ -63,8 +63,8 @@ public List split(byte[] colFamKey) { bb.mark(); } } - - + + segmentList.add(copyFromMark(bb, segStart, bb.capacity()-segStart)); return segmentList; diff --git a/object-mapper/src/main/java/me/prettyprint/hom/KeyDefinition.java b/object-mapper/src/main/java/me/prettyprint/hom/KeyDefinition.java index b988bd903..3052e9c92 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/KeyDefinition.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/KeyDefinition.java @@ -11,7 +11,7 @@ /** * Defines properties used for representing and constructing a cassandra key and * mapping to/from POJO property(ies). - * + * * @author B. Todd Burruss */ public class KeyDefinition { diff --git a/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnnotationScanner.java b/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnnotationScanner.java index 9e86d8135..c910a20a0 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnnotationScanner.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnnotationScanner.java @@ -13,7 +13,7 @@ /** * Scan for classes annotated with an annotation. The scan starts in the given * package root so it doesn't need to scan through the entire package structure. - * + * * @author Todd Burruss */ public class AnnotationScanner { diff --git a/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyAddHandler.java b/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyAddHandler.java index 734c73742..992f36fa1 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyAddHandler.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyAddHandler.java @@ -12,7 +12,7 @@ * Annotation for marking the method used to add "anonymous" properties to the * POJO. Anonymous properties are Columns in Cassandra that do not map directly * to the POJO. See {@link HectorObjectMapper} for details. - * + * * @author Todd Burruss */ @Retention(RetentionPolicy.RUNTIME) diff --git a/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyCollectionGetter.java b/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyCollectionGetter.java index 3d3131b52..59c5a72f0 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyCollectionGetter.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/annotations/AnonymousPropertyCollectionGetter.java @@ -13,7 +13,7 @@ * Annotation for marking the method used to get "anonymous" properties from the * POJO. Anonymous properties are Columns in Cassandra that do not map directly * to the POJO. See {@link HectorObjectMapper} for details. - * + * * @author Todd Burruss */ @Retention(RetentionPolicy.RUNTIME) diff --git a/object-mapper/src/main/java/me/prettyprint/hom/annotations/Column.java b/object-mapper/src/main/java/me/prettyprint/hom/annotations/Column.java index 8e2c9e134..18ae33eeb 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/annotations/Column.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/annotations/Column.java @@ -11,7 +11,7 @@ /** * Annotation for specifying which POJO properties should be mapped to Cassandra * columns. Must specify "name" as the column name in Cassandra. - * + * * @author */ @Retention(RetentionPolicy.RUNTIME) @@ -20,7 +20,7 @@ /** * The Cassandra column name. - * + * * @return name of column */ String name(); @@ -28,7 +28,7 @@ /** * The optional converter to use when converting POJO property value to/from * byte[]. If not specified, {@link me.prettyprint.hom.converters.DefaultConverter} is used. - * + * * @return Class of converter */ Class converter() default me.prettyprint.hom.converters.DefaultConverter.class; diff --git a/object-mapper/src/main/java/me/prettyprint/hom/annotations/Id.java b/object-mapper/src/main/java/me/prettyprint/hom/annotations/Id.java index db1181bb2..a3078a3fe 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/annotations/Id.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/annotations/Id.java @@ -10,7 +10,7 @@ * Annotation marking the ID property in the POJO. Marking the ID property is * required so the object mapper can get/set the ID (Cassandra row key) in the * POJO. - * + * * @author Todd Burruss */ @Retention(RetentionPolicy.RUNTIME) @@ -19,7 +19,7 @@ /** * The optional converter to use when converting POJO property value to/from * byte[]. If not specified, {@link me.prettyprint.hom.converters.DefaultConverter} is used. - * + * * @return Class of converter */ Class converter() default me.prettyprint.hom.converters.DefaultConverter.class; diff --git a/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParser.java b/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParser.java index e42486a1e..ae9487e8f 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParser.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParser.java @@ -12,7 +12,7 @@ /** * Parse, validate, and set defaults if needed for Inheritance functionality. - * + * * @author bburruss */ public class ColumnParser implements ColumnParserValidator { diff --git a/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParserValidator.java b/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParserValidator.java index 14521bf69..9a19e095f 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParserValidator.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/cache/ColumnParserValidator.java @@ -12,5 +12,5 @@ public interface ColumnParserValidator { void parse(Field field, Annotation anno, PropertyDescriptor pd, CFMappingDef cfMapDef); // void validateAndSetDefaults(ClassCacheMgr cacheMgr, CFMappingDef cfMapDef); - + } diff --git a/object-mapper/src/main/java/me/prettyprint/hom/cache/IdClassParserValidator.java b/object-mapper/src/main/java/me/prettyprint/hom/cache/IdClassParserValidator.java index 65d1bc7a5..0246913fa 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/cache/IdClassParserValidator.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/cache/IdClassParserValidator.java @@ -28,11 +28,11 @@ public void parse(ClassCacheMgr cacheMgr, Annotation anno, CFMappingDef c @Override public void validateAndSetDefaults(ClassCacheMgr cacheMgr, CFMappingDef cfMapDef) { KeyDefinition keyDef = cfMapDef.getKeyDef(); - + if ( null == keyDef.getPkClazz() ) { return; } - + Map pdMap; try { pdMap = cacheMgr.getFieldPropertyDescriptorMap(keyDef.getPkClazz()); @@ -40,12 +40,12 @@ public void validateAndSetDefaults(ClassCacheMgr cacheMgr, CFMappingDef c throw new HectorObjectMapperException("exception while introspecting class, " + keyDef.getPkClazz().getName(), e); } - + if ( keyDef.getIdPropertyMap().size() != pdMap.size() ) { throw new HectorObjectMapperException("Each field in the primary key class, " + keyDef.getPkClazz().getName() + ", must have a corresponding property in the entity, " + cfMapDef.getRealClass().getName() + ", annotated with @" + Id.class.getSimpleName() ); } - + for ( String idFieldName : pdMap.keySet() ) { if ( !keyDef.getIdPropertyMap().keySet().contains(idFieldName)) { throw new HectorObjectMapperException("Each field in the primary key class, " + keyDef.getPkClazz().getName() diff --git a/object-mapper/src/main/java/me/prettyprint/hom/cache/InheritanceParserValidator.java b/object-mapper/src/main/java/me/prettyprint/hom/cache/InheritanceParserValidator.java index 1cea2c8f7..fd48b2e62 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/cache/InheritanceParserValidator.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/cache/InheritanceParserValidator.java @@ -11,11 +11,10 @@ import me.prettyprint.hom.CFMappingDef; import me.prettyprint.hom.ClassCacheMgr; -import org.apache.openjpa.util.UnsupportedException; /** * Parse, validate, and set defaults if needed for Inheritance functionality. - * + * * @author bburruss */ public class InheritanceParserValidator implements ParserValidator { @@ -38,7 +37,7 @@ private void parseInheritanceAnnotation(Inheritance anno, CFMappingDef cf if (InheritanceType.SINGLE_TABLE == anno.strategy()) { cfMapDef.setInheritanceType(InheritanceType.SINGLE_TABLE); } else { - throw new UnsupportedException("Hector object mapper only supports " + throw new RuntimeException("Hector object mapper only supports " + InheritanceType.SINGLE_TABLE + " inheritance at the moment"); } } diff --git a/object-mapper/src/main/java/me/prettyprint/hom/cache/ParserValidator.java b/object-mapper/src/main/java/me/prettyprint/hom/cache/ParserValidator.java index fb0aacebb..5897cb557 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/cache/ParserValidator.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/cache/ParserValidator.java @@ -10,5 +10,5 @@ public interface ParserValidator { void parse(ClassCacheMgr cacheMgr, Annotation anno, CFMappingDef cfMapDef); void validateAndSetDefaults(ClassCacheMgr cacheMgr, CFMappingDef cfMapDef); - + } diff --git a/object-mapper/src/main/java/me/prettyprint/hom/cache/TableParserValidator.java b/object-mapper/src/main/java/me/prettyprint/hom/cache/TableParserValidator.java index 212ac67d4..ad0a5bf1a 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/cache/TableParserValidator.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/cache/TableParserValidator.java @@ -10,7 +10,7 @@ /** * Parse, validate, and set defaults if needed for Inheritance functionality. - * + * * @author bburruss */ public class TableParserValidator implements ParserValidator { diff --git a/object-mapper/src/main/java/me/prettyprint/hom/converters/Converter.java b/object-mapper/src/main/java/me/prettyprint/hom/converters/Converter.java index 5efd4d46e..f837978b7 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/converters/Converter.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/converters/Converter.java @@ -4,7 +4,7 @@ /** * Interface defining a custom object mapper conversion. For instance, from an * enum to a string. - * + * * @author Todd Burruss */ public interface Converter { diff --git a/object-mapper/src/main/java/me/prettyprint/hom/converters/DefaultConverter.java b/object-mapper/src/main/java/me/prettyprint/hom/converters/DefaultConverter.java index 050ef82bd..ba09b835f 100644 --- a/object-mapper/src/main/java/me/prettyprint/hom/converters/DefaultConverter.java +++ b/object-mapper/src/main/java/me/prettyprint/hom/converters/DefaultConverter.java @@ -8,7 +8,7 @@ /** * Default converter used when none specified in {@link Column} annotation. Uses * Java reflection to determine the Java type to use. - * + * * @author Todd Burruss */ public class DefaultConverter implements Converter { diff --git a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStore.java b/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStore.java deleted file mode 100644 index 28c8fbe65..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStore.java +++ /dev/null @@ -1,131 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import java.util.ArrayList; -import java.util.Collection; -import java.util.Iterator; -import java.util.Map; - -import org.apache.openjpa.kernel.OpenJPAStateManager; -import org.apache.openjpa.meta.ClassMetaData; -import org.apache.openjpa.meta.FieldMetaData; -import org.apache.openjpa.meta.JavaTypes; -import org.apache.openjpa.util.OpenJPAId; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.jdbc.core.ColumnMapRowMapper; - -import me.prettyprint.cassandra.model.HColumnImpl; -import me.prettyprint.cassandra.model.MutatorImpl; -import me.prettyprint.cassandra.serializers.BytesArraySerializer; -import me.prettyprint.cassandra.serializers.LongSerializer; -import me.prettyprint.cassandra.serializers.StringSerializer; -import me.prettyprint.cassandra.utils.StringUtils; -import me.prettyprint.hector.api.Cluster; -import me.prettyprint.hector.api.Keyspace; -import me.prettyprint.hector.api.Serializer; -import me.prettyprint.hector.api.beans.ColumnSlice; -import me.prettyprint.hector.api.beans.HColumn; -import me.prettyprint.hector.api.factory.HFactory; -import me.prettyprint.hector.api.mutation.Mutator; -import me.prettyprint.hector.api.query.QueryResult; -import me.prettyprint.hector.api.query.SliceQuery; -import me.prettyprint.hom.EntityManagerConfigurator; -import me.prettyprint.hom.openjpa.EntityFacade.ColumnMeta; - - -/** - * Holds the {@link Cluster} and {@link Keyspace} references needed - * for accessing Cassandra - * - * @author zznate - */ -public class CassandraStore { - - private static final Logger log = LoggerFactory.getLogger(CassandraStore.class); - - private final Cluster cluster; - private final CassandraStoreConfiguration conf; - private Keyspace keyspace; - private MappingUtils mappingUtils; - - public CassandraStore(CassandraStoreConfiguration conf) { - this.conf = conf; - this.cluster = HFactory.getCluster(conf.getValue(EntityManagerConfigurator.CLUSTER_NAME_PROP) - .getOriginalValue()); - // TODO needs passthrough of other configuration - mappingUtils = new MappingUtils(); - } - - public CassandraStore open() { - this.keyspace = HFactory.createKeyspace(conf.getValue(EntityManagerConfigurator.KEYSPACE_PROP) - .getOriginalValue(), cluster); - return this; - } - - public boolean getObject(OpenJPAStateManager stateManager) { - - ClassMetaData metaData = stateManager.getMetaData(); - EntityFacade entityFacade = new EntityFacade(metaData); - Object idObj = stateManager.getId(); - - SliceQuery sliceQuery = mappingUtils.buildSliceQuery(idObj, entityFacade, keyspace); - - //stateManager.storeObject(0, idObj); - - QueryResult> result = sliceQuery.execute(); - - for (Map.Entry> entry : entityFacade.getColumnMeta().entrySet()) { - HColumn column = result.get().getColumnByName(entry.getKey()); - if ( column != null ) - stateManager.storeObject(entry.getValue().fieldId, entry.getValue().serializer.fromBytes(column.getValue())); - } - return true; - } - - public Mutator storeObject(Mutator mutator, OpenJPAStateManager stateManager, Object idObj) { - if ( mutator == null ) - mutator = new MutatorImpl(keyspace, BytesArraySerializer.get()); - if ( log.isDebugEnabled() ) { - log.debug("Adding mutation (insertion) for class {}", stateManager.getManagedInstance().getClass().getName()); - } - ClassMetaData metaData = stateManager.getMetaData(); - EntityFacade entityFacade = new EntityFacade(metaData); - Object field; - for (Map.Entry> entry : entityFacade.getColumnMeta().entrySet()) { - field = stateManager.fetch(entry.getValue().fieldId); - if ( field != null ) { - mutator.addInsertion(mappingUtils.getKeyBytes(idObj), entityFacade.getColumnFamilyName(), - new HColumnImpl(entry.getKey(), field, - keyspace.createClock(), StringSerializer.get(), - entry.getValue().serializer)); - } - - } - return mutator; - } - - public Mutator removeObject(Mutator mutator, OpenJPAStateManager stateManager, Object idObj) { - if ( mutator == null ) - mutator = new MutatorImpl(keyspace, BytesArraySerializer.get()); - if ( log.isDebugEnabled() ) { - log.debug("Adding mutation (deletion) for class {}", stateManager.getManagedInstance().getClass().getName()); - } - ClassMetaData metaData = stateManager.getMetaData(); - EntityFacade entityFacade = new EntityFacade(metaData); - mutator.addDeletion(mappingUtils.getKeyBytes(idObj), entityFacade.getColumnFamilyName(),null,StringSerializer.get()); - return mutator; - } - - - public Cluster getCluster() { - return cluster; - } - - public Keyspace getKeyspace() { - return keyspace; - } - - - - -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStoreConfiguration.java b/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStoreConfiguration.java deleted file mode 100644 index b062fb2cf..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStoreConfiguration.java +++ /dev/null @@ -1,24 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import org.apache.openjpa.conf.OpenJPAConfigurationImpl; -import org.apache.openjpa.lib.conf.ProductDerivations; - -public class CassandraStoreConfiguration extends OpenJPAConfigurationImpl { - - public CassandraStoreConfiguration() { - super(false, false); - - // override the default and the current value of lock manager plugin - // from our superclass to use the single-jvm lock manager - lockManagerPlugin.setDefault("version"); - lockManagerPlugin.setString("version"); - addString("me.prettyprint.hom.classpathPrefix"); - addString("me.prettyprint.hom.keyspace"); - addString("me.prettyprint.hom.clusterName"); - addString("me.prettyprint.hom.hostList"); - - ProductDerivations.beforeConfigurationLoad(this); - loadGlobals(); - - } -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStoreManager.java b/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStoreManager.java deleted file mode 100644 index 732b0d11a..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/CassandraStoreManager.java +++ /dev/null @@ -1,237 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import java.util.ArrayList; -import java.util.BitSet; -import java.util.Collection; -import java.util.Iterator; -import java.util.List; - -import me.prettyprint.cassandra.serializers.LongSerializer; -import me.prettyprint.cassandra.serializers.StringSerializer; -import me.prettyprint.cassandra.service.CassandraHostConfigurator; -import me.prettyprint.hector.api.Cluster; -import me.prettyprint.hector.api.Keyspace; -import me.prettyprint.hector.api.factory.HFactory; -import me.prettyprint.hector.api.mutation.Mutator; - -import org.apache.openjpa.abstractstore.AbstractStoreManager; -import org.apache.openjpa.conf.OpenJPAConfiguration; -import org.apache.openjpa.kernel.FetchConfiguration; -import org.apache.openjpa.kernel.OpenJPAStateManager; -import org.apache.openjpa.kernel.PCState; -import org.apache.openjpa.lib.rop.ListResultObjectProvider; -import org.apache.openjpa.lib.rop.ResultObjectProvider; -import org.apache.openjpa.meta.ClassMetaData; -import org.apache.openjpa.meta.FieldMetaData; -import org.apache.openjpa.util.OpenJPAId; -import org.apache.openjpa.util.StoreException; -import org.apache.openjpa.xmlstore.ObjectData; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -public class CassandraStoreManager extends AbstractStoreManager { - - private static Logger log = LoggerFactory.getLogger(CassandraStoreManager.class); - - private CassandraStore cassandraStore; - - @Override - public ResultObjectProvider executeExtent(ClassMetaData cMetaData, boolean useSubClasses, - FetchConfiguration fetchConfiguration) { - // cMetaData is essentially the query construct - // the ResultObjectProvider runs the query - if ( log.isDebugEnabled() ) { - log.debug("in executeExtent with ClassMetaData {}: useSubClasses: {} and fetchConfiguration: {}", - new Object[]{cMetaData, useSubClasses, fetchConfiguration}); - } - - // ask the store for all ObjectDatas for the given type; this - // actually gives us all instances of the base class of the type - // CFMetaData - // ObjectData[] datas = _store.getData(meta); - // get the Class - // Class candidate = meta.getDescribedType(); - - // create a list of the corresponding persistent objects that - // match the type and subclasses criteria - /* - List pcs = new ArrayList(datas.length); - for (int i = 0; i < datas.length; i++) { - // does this instance belong in the extent? - Class c = datas[i].getMetaData().getDescribedType(); - if (c != candidate && (!subclasses - || !candidate.isAssignableFrom(c))) - continue; - - // look up the pc instance for the data, passing in the data - // as well so that we can take advantage of the fact that we've - // already looked it up. note that in the store manager's - // initialize(), load(), etc methods we check for this data - // being passed through and save ourselves a trip to the store - // if it is present; this is particularly important in systems - // where a trip to the store can be expensive. - pcs.add(ctx.find(datas[i].getId(), fetch, null, datas[i], 0)); - } - return new ListResultObjectProvider(pcs); - */ - return null; - } - - @SuppressWarnings("unchecked") - @Override - protected Collection flush(Collection pNew, Collection pNewUpdated, Collection pNewFlushedDeleted, - Collection pDirty, Collection pDeleted) { - /* defn. of above arguments - * --------------------------- - * pNew - Objects that should be added to the store, and that have not previously been flushed. - pNewUpdated - New objects that have been modified since they were initially flushed. These were in persistentNew in an earlier flush invocation. - pNewFlushedDeleted - New objects that have been deleted since they were initially flushed. These were in persistentNew in an earlier flush invocation. - pDirty - Objects that were loaded from the data store and have since been modified. - pDeleted - Objects that were loaded from the data store and have since been deleted. These may have been in a previous flush invocation's persistentDirty list. - */ - //_updates = new ArrayList(pNew.size() + pDirty.size()); - //_deletes = new ArrayList(pDeleted.size()); - cassandraStore.open(); - OpenJPAConfiguration conf = ctx.getConfiguration(); - Mutator mutator = null; - for (Iterator itr = pNew.iterator(); itr.hasNext();) { - // create new object data for instance - OpenJPAStateManager sm = (OpenJPAStateManager) itr.next(); - Object oid = sm.getObjectId(); - mutator = cassandraStore.storeObject(mutator, sm, oid); - } - // TODO combine pNewUpdated and pDirty and use sm.getDirty for bitmask of field - // - //for (int i = 0; i < fmds.length; i++) - //if (fields.get(i)) - // sm.store(i, toLoadable(sm, fmds[i], _data[i], fetch)); - // - // combine pNewFlushDelete and pDeleted into the mutator - if ( !pDeleted.isEmpty() || !pNewFlushedDeleted.isEmpty() ) { - List deletes = new ArrayList(pDeleted.size() + pNewFlushedDeleted.size()); - deletes.addAll(pDeleted); - deletes.addAll(pNewFlushedDeleted); - for (Iterator itr = deletes.iterator(); itr.hasNext();) { - // create new object data for instance - OpenJPAStateManager sm = (OpenJPAStateManager) itr.next(); - Object oid = sm.getObjectId(); - mutator = cassandraStore.removeObject(mutator, sm, oid); - } - } - mutator.execute(); - - - return null; - } - - @Override - public boolean initialize(OpenJPAStateManager stateManager, PCState pcState, - FetchConfiguration fetchConfiguration, Object obj) { - log.debug("In initialize operation..."); - cassandraStore.open(); - stateManager.initialize(stateManager.getMetaData().getDescribedType(), pcState); - cassandraStore.getObject(stateManager); - // TODO need to add the not-found case - return true; - } - - @Override - public boolean load(OpenJPAStateManager stateManager, BitSet arg1, - FetchConfiguration arg2, int arg3, Object arg4) { - log.debug("In load operation..."); - cassandraStore.getObject(stateManager); - // TODO implement the loading of properties marked as lazy (is this correct?) - // this is a misnomer. load is called to fill in additional information not retrieved from - // initialize call above - return false; - } - - @Override - public boolean exists(OpenJPAStateManager arg0, Object arg1) { - // TODO - // hit cassandra to see if the object exists - log.debug("In CSM.exists()"); - return false; - } - - - @Override - public boolean isCached(List arg0, BitSet arg1) { - log.debug("In CSM.isCached()"); - // TODO Auto-generated method stub - return false; - } - - - - @Override - public int compareVersion(OpenJPAStateManager state, Object v1, Object v2) { - log.debug("in CSM.compareVersion"); - return super.compareVersion(state, v1, v2); - } - - @Override - public boolean syncVersion(OpenJPAStateManager sm, Object edata) { - // TODO Auto-generated method stub - log.debug("in CSM.syncVersion"); - return super.syncVersion(sm, edata); - } - - @Override - protected void open() { - OpenJPAConfiguration conf = ctx.getConfiguration(); - // TODO - // encapsulate into CassandraStore or similar (should take CassandraStoreConfig as an argg - //cluster = HFactory.getCluster(conf.getValue("me.prettyprint.hom.clusterName").getOriginalValue()); - //keyspace = HFactory.createKeyspace(conf.getValue("me.prettyprint.hom.keyspace").getOriginalValue(), cluster); - - cassandraStore = new CassandraStore((CassandraStoreConfiguration)conf); - log.debug("in CSM.open()"); - } - - protected Collection getUnsupportedOptions() { - Collection c = super.getUnsupportedOptions(); - - // remove options we do support but the abstract store doesn't - //c.remove(OpenJPAConfiguration.OPTION_ID_DATASTORE); - //c.remove(OpenJPAConfiguration.OPTION_OPTIMISTIC); - - // and add some that we don't support but the abstract store does - // TODO take these out one by one - c.add(OpenJPAConfiguration.OPTION_EMBEDDED_RELATION); - c.add(OpenJPAConfiguration.OPTION_EMBEDDED_COLLECTION_RELATION); - c.add(OpenJPAConfiguration.OPTION_EMBEDDED_MAP_RELATION); - //c.add(OpenJPAConfiguration.OPTION_OPTIMISTIC); - return c; - } - - @Override - protected OpenJPAConfiguration newConfiguration() { - - CassandraStoreConfiguration conf = new CassandraStoreConfiguration(); - - if ( log.isDebugEnabled() ) { - log.debug("In newConfiguration with conf: {}", conf.toProperties(true)); - } - - return conf; - } - - - -} - -/* - * NOTES - - OpenJPAStateManager holds the 'state' of an Entity instance - - ClassMetaData holds the details of the persistable Class - - Consider using ClassCacheMgr as a MetaDataFactory - http://openjpa.apache.org/builds/2.0.1/apache-openjpa-2.0.1/docs/manual/ref_guide_meta.html - - Consider rolling a custom FetchPlan (or FetchConfiguration?) for CL - - Returns the OpenJPAId.getType() returns Entity class: - log.debug("OID class name: {}",((OpenJPAId)idObj).getType().getName()); - - */ diff --git a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/EntityFacade.java b/object-mapper/src/main/java/me/prettyprint/hom/openjpa/EntityFacade.java deleted file mode 100644 index dea9cecfe..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/EntityFacade.java +++ /dev/null @@ -1,101 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import java.io.Serializable; -import java.util.Arrays; -import java.util.HashMap; -import java.util.Map; - -import javax.persistence.Table; - -import org.apache.openjpa.meta.ClassMetaData; -import org.apache.openjpa.meta.FieldMetaData; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import me.prettyprint.hector.api.Serializer; - -public class EntityFacade implements Serializable { - private static final Logger log = LoggerFactory.getLogger(EntityFacade.class); - - private static final long serialVersionUID = 4777260639119126462L; - - private final String columnFamilyName; - private final Class clazz; - private final Serializer keySerializer; - private final Map> columnMetas; - - public EntityFacade(ClassMetaData classMetaData) { - clazz = classMetaData.getDescribedType(); - this.columnFamilyName = clazz.getAnnotation(Table.class) != null ? clazz.getAnnotation(Table.class).name() : clazz.getSimpleName(); - if ( log.isDebugEnabled() ) { - log.debug("PK field name: {} and typeCode: {}", - classMetaData.getPrimaryKeyFields()[0].getType(), - classMetaData.getPrimaryKeyFields()[0].getObjectIdFieldTypeCode()); - } - this.keySerializer = MappingUtils.getSerializer(classMetaData.getPrimaryKeyFields()[0]); - columnMetas = new HashMap>(); - FieldMetaData[] fmds = classMetaData.getFields(); - for (int i = 0; i < fmds.length; i++) { - if ( fmds[i].getManagement() == FieldMetaData.MANAGE_NONE || fmds[i].isPrimaryKey()) - continue; - if ( log.isDebugEnabled()) { - log.debug("field name {} typeCode {} associationType: {} declaredType: {} embeddedMetaData: {}", - new Object[]{fmds[i].getName(), - fmds[i].getTypeCode(), - fmds[i].getAssociationType(), - fmds[i].getDeclaredType().getName(), - fmds[i].getElement().getDeclaredTypeMetaData() - }); - } - // TODO if fmds[i].getAssociationType() > 0 .. we found an attached entity and need to find it's entityFacade - columnMetas.put(fmds[i].getName(), new ColumnMeta(fmds[i].getIndex(), MappingUtils.getSerializer(fmds[i].getTypeCode()))); - } - } - - public String[] getColumnNames() { - return columnMetas.keySet().toArray(new String[]{}); - } - - public String getColumnFamilyName() { - return columnFamilyName; - } - - public Class getClazz() { - return clazz; - } - - public Serializer getKeySerializer() { - return keySerializer; - } - - public int getFieldId(String columnName) { - return columnMetas.get(columnName).fieldId; - } - - public Serializer getSerializer(String columnName) { - return columnMetas.get(columnName).serializer; - } - - public Map> getColumnMeta() { - return columnMetas; - } - - class ColumnMeta { - int fieldId; - Serializer serializer; - - ColumnMeta(int fieldId, Serializer serializer) { - this.fieldId = fieldId; - this.serializer = serializer; - } - } - - @Override - public String toString() { - return String.format("EntityFacade[class: %s, columnFamily: %s, columnNames: %s]", - clazz.getName(), - columnFamilyName, Arrays.toString(getColumnNames())); - } - - -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/MappingUtils.java b/object-mapper/src/main/java/me/prettyprint/hom/openjpa/MappingUtils.java deleted file mode 100644 index e5038e999..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/openjpa/MappingUtils.java +++ /dev/null @@ -1,194 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.UUID; - -import me.prettyprint.cassandra.model.MutatorImpl; -import me.prettyprint.cassandra.model.thrift.ThriftSliceQuery; -import me.prettyprint.cassandra.serializers.ByteBufferSerializer; -import me.prettyprint.cassandra.serializers.BytesArraySerializer; -import me.prettyprint.cassandra.serializers.DateSerializer; -import me.prettyprint.cassandra.serializers.DoubleSerializer; -import me.prettyprint.cassandra.serializers.IntegerSerializer; -import me.prettyprint.cassandra.serializers.LongSerializer; -import me.prettyprint.cassandra.serializers.ObjectSerializer; -import me.prettyprint.cassandra.serializers.SerializerTypeInferer; -import me.prettyprint.cassandra.serializers.StringSerializer; -import me.prettyprint.cassandra.serializers.UUIDSerializer; -import me.prettyprint.hector.api.Keyspace; -import me.prettyprint.hector.api.Serializer; -import me.prettyprint.hector.api.factory.HFactory; -import me.prettyprint.hector.api.mutation.Mutator; -import me.prettyprint.hector.api.query.SliceQuery; - -import org.apache.openjpa.kernel.OpenJPAStateManager; -import org.apache.openjpa.meta.ClassMetaData; -import org.apache.openjpa.meta.FieldMetaData; -import org.apache.openjpa.meta.JavaTypes; -import org.apache.openjpa.util.LongId; -import org.apache.openjpa.util.OpenJPAId; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * Utility class for bridging Hector/OpenJPA functionality - * - * @author zznate - * - */ -public class MappingUtils { - private static final Logger log = LoggerFactory.getLogger(MappingUtils.class); - - private static final Map> typeSerializerMap = new HashMap>(); - private static final Map, Serializer> classSerializerMap = new HashMap, Serializer>(); - - // TODO need to figure out UUID - static { - typeSerializerMap.put(JavaTypes.STRING, StringSerializer.get()); - typeSerializerMap.put(JavaTypes.INT, IntegerSerializer.get()); - typeSerializerMap.put(JavaTypes.INT_OBJ, IntegerSerializer.get()); - typeSerializerMap.put(JavaTypes.DATE, DateSerializer.get()); - typeSerializerMap.put(JavaTypes.LONG, LongSerializer.get()); - typeSerializerMap.put(JavaTypes.LONG_OBJ, LongSerializer.get()); - typeSerializerMap.put(JavaTypes.DOUBLE, DoubleSerializer.get()); - typeSerializerMap.put(JavaTypes.DOUBLE_OBJ, DoubleSerializer.get()); - - classSerializerMap.put(UUID.class, UUIDSerializer.get()); - classSerializerMap.put(byte[].class, BytesArraySerializer.get()); - classSerializerMap.put(ByteBuffer.class, ByteBufferSerializer.get()); - - } - - public static Serializer getSerializer(int javaType) { - return typeSerializerMap.get(javaType) != null ? typeSerializerMap.get(javaType) : ObjectSerializer.get(); - } - - public static Serializer getSerializer(FieldMetaData fieldMetaData) { - Serializer serializer = getSerializer(fieldMetaData.getTypeCode()); - if ( serializer instanceof ObjectSerializer ) { - Class clazz = fieldMetaData.getType(); - if ( classSerializerMap.get(clazz) != null ) { - serializer = classSerializerMap.get(clazz); - } - } - return serializer; - } - - public SliceQuery buildSliceQuery(Object idObj, EntityFacade entityFacade, Keyspace keyspace) { - SliceQuery query = new ThriftSliceQuery(keyspace, BytesArraySerializer.get(), StringSerializer.get(), BytesArraySerializer.get()); - query.setColumnNames(entityFacade.getColumnNames()); - query.setKey(getKeyBytes(idObj)); - query.setColumnFamily(entityFacade.getColumnFamilyName()); - return query; - } - - /** - * Get the byte[] representing this Id object - * @param idObj - * @return - */ - public byte[] getKeyBytes(Object idObj) { - Serializer serializer = getSerializer(idObj); - if ( idObj instanceof OpenJPAId ) - return serializer.toBytes(((OpenJPAId)idObj).getIdObject()); - return serializer.toBytes(idObj); - } - - /** - * Retrieve the {@link Serializer} implementation for the id Object. - * @see {@link SerializerTypeInferer} for specifics. - * @param idObj - */ - public Serializer getSerializer(Object idObj) { - Serializer serializer; - if ( idObj instanceof OpenJPAId ) { - serializer = SerializerTypeInferer.getSerializer(((OpenJPAId)idObj).getIdObject()); - } else { - serializer = SerializerTypeInferer.getSerializer(idObj); - } - return serializer; - } - - /** - * Return a list of field names for which there are a 1-1 mapping - * to Column names - * - * @param metaData - * @return - */ - List buildColumnList(ClassMetaData metaData) { - FieldMetaData[] fmds = metaData.getFields(); - List cols = new ArrayList(fmds.length); - for (int i = 0; i < fmds.length; i++) { - if (fmds[i].getManagement() != fmds[i].MANAGE_PERSISTENT || fmds[i].isPrimaryKey()) - continue; - - log.debug("fmd.name: {}",fmds[i].getName()); - cols.add(fmds[i].getName()); - } - return cols; - } - - /** - * Map the column names to their respective serializers - * TODO cache this - * @param metaData - * @return - */ - Map buildColumnSerializerMap(ClassMetaData metaData) { - Map serMap = new HashMap(); - - FieldMetaData[] fmds = metaData.getFields(); - for (int i = 0; i < fmds.length; i++) { - if (fmds[i].getManagement() != fmds[i].MANAGE_PERSISTENT || fmds[i].isPrimaryKey()) - continue; - switch (fmds[i].getTypeCode()) { - case JavaTypes.STRING: - serMap.put(fmds[i].getName(), StringSerializer.get()); - break; - case JavaTypes.INT: - serMap.put(fmds[i].getName(), IntegerSerializer.get()); - break; - case JavaTypes.INT_OBJ: - serMap.put(fmds[i].getName(), IntegerSerializer.get()); - break; - case JavaTypes.DATE: - serMap.put(fmds[i].getName(), DateSerializer.get()); - break; - case JavaTypes.LONG: - serMap.put(fmds[i].getName(), LongSerializer.get()); - break; - case JavaTypes.LONG_OBJ: - serMap.put(fmds[i].getName(), LongSerializer.get()); - break; - case JavaTypes.DOUBLE: - serMap.put(fmds[i].getName(), DoubleSerializer.get()); - break; - case JavaTypes.DOUBLE_OBJ: - serMap.put(fmds[i].getName(), DoubleSerializer.get()); - break; - default: - serMap.put(fmds[i].getName(), ObjectSerializer.get()); - break; - } - } - return serMap; - } - - - - public Mutator addMutation(Mutator mutator, Object idObj, OpenJPAStateManager stateManager, Keyspace keyspace) { - ClassMetaData metaData = stateManager.getMetaData(); - Map serMap = buildColumnSerializerMap(metaData); - for (Map.Entry entry : serMap.entrySet()) { - mutator.addInsertion(getKeyBytes(idObj), "TestBeanColumnFamily", - HFactory.createColumn(entry.getKey(), stateManager.fetch(metaData.getField(entry.getKey()).getIndex()), - StringSerializer.get(), entry.getValue())); - } - return mutator; - } -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/service/EntitySchemaManager.java b/object-mapper/src/main/java/me/prettyprint/hom/service/EntitySchemaManager.java deleted file mode 100644 index 36566befc..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/service/EntitySchemaManager.java +++ /dev/null @@ -1,63 +0,0 @@ -package me.prettyprint.hom.service; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import me.prettyprint.cassandra.model.BasicColumnFamilyDefinition; -import me.prettyprint.cassandra.service.ThriftCfDef; -import me.prettyprint.hector.api.Cluster; -import me.prettyprint.hector.api.Keyspace; -import me.prettyprint.hector.api.ddl.ColumnFamilyDefinition; -import me.prettyprint.hector.api.exceptions.HInvalidRequestException; -import me.prettyprint.hom.openjpa.EntityFacade; -import me.prettyprint.hom.service.EntitySchemaStatus.SchemaResult; - -/** - * A service to manage the schema of JPA Entities - * - * @author zznate - */ -public class EntitySchemaManager { - - private static Logger log = LoggerFactory.getLogger(EntitySchemaManager.class); - - private Cluster cluster; - private Keyspace keyspace; - - public EntitySchemaManager(Cluster cluster, Keyspace keyspace) { - this.cluster = cluster; - this.keyspace = keyspace; - } - - /** - * Creates the schema if it does not already exist. If it does exist, - * we catch the HInvalidRequestException and return an {@link EntitySchemaStatus} - * with {@link SchemaResult#NOT_MODIFIED} - * @param entityFacade - * @return {@link EntitySchemaStatus} with {@link SchemaResult#CREATED} if successful - */ - public EntitySchemaStatus createSchema(EntityFacade entityFacade) { - if ( log.isDebugEnabled() ) { - log.debug("EntitySchemaManager creating schema for: {}", entityFacade); - } - BasicColumnFamilyDefinition columnFamilyDefinition = new BasicColumnFamilyDefinition(); - columnFamilyDefinition.setKeyspaceName(keyspace.getKeyspaceName()); - columnFamilyDefinition.setName(entityFacade.getColumnFamilyName()); - // TODO determine how to pull in other CF creation settings... custom annotations? - ColumnFamilyDefinition cfDef = new ThriftCfDef(columnFamilyDefinition); - EntitySchemaStatus entitySchemaStatus; - try { - cluster.addColumnFamily(cfDef); - entitySchemaStatus = new EntitySchemaStatus(SchemaResult.CREATED); - } catch (HInvalidRequestException ire) { - if ( log.isDebugEnabled() ) { - log.debug("Caught HInvalidRequestException on createSchema. Details {}", ire.getMessage()); - } - entitySchemaStatus = new EntitySchemaStatus(SchemaResult.NOT_MODIFIED); - } - return entitySchemaStatus; - - } - - -} diff --git a/object-mapper/src/main/java/me/prettyprint/hom/service/EntitySchemaStatus.java b/object-mapper/src/main/java/me/prettyprint/hom/service/EntitySchemaStatus.java deleted file mode 100644 index 6b4a958a0..000000000 --- a/object-mapper/src/main/java/me/prettyprint/hom/service/EntitySchemaStatus.java +++ /dev/null @@ -1,21 +0,0 @@ -package me.prettyprint.hom.service; - -public class EntitySchemaStatus { - - private final SchemaResult schemaResult; - - public EntitySchemaStatus(SchemaResult result) { - this.schemaResult = result; - } - - - public SchemaResult getSchemaResult() { - return schemaResult; - } - - public enum SchemaResult { - CREATED, - UPDATED, - NOT_MODIFIED - } -} diff --git a/object-mapper/src/test/java/com/mycompany/MySerial.java b/object-mapper/src/test/java/com/mycompany/MySerial.java index f45ecdb18..42fa59114 100644 --- a/object-mapper/src/test/java/com/mycompany/MySerial.java +++ b/object-mapper/src/test/java/com/mycompany/MySerial.java @@ -6,13 +6,13 @@ public class MySerial implements Serializable { private int prop1; private long prop2; - + public MySerial(int prop1, long prop2) { super(); this.prop1 = prop1; this.prop2 = prop2; } - + public int getProp1() { return prop1; } diff --git a/object-mapper/src/test/java/com/mycompany/furniture/FurnitureTest.java b/object-mapper/src/test/java/com/mycompany/furniture/FurnitureTest.java index 485298526..18cee5942 100644 --- a/object-mapper/src/test/java/com/mycompany/furniture/FurnitureTest.java +++ b/object-mapper/src/test/java/com/mycompany/furniture/FurnitureTest.java @@ -15,12 +15,12 @@ import org.apache.cassandra.db.marshal.BytesType; import org.apache.cassandra.thrift.CfDef; import org.apache.thrift.transport.TTransportException; +import org.junit.Before; import org.junit.BeforeClass; import org.junit.Test; public class FurnitureTest extends CassandraTestBase { - static Keyspace keyspace; static EntityManagerImpl entityMgr; @Test @@ -71,18 +71,9 @@ public void testFurniture() { // -------------- - @BeforeClass - public static void setup() throws TTransportException, SecurityException, IllegalArgumentException, + @Before + public void setup() throws TTransportException, SecurityException, IllegalArgumentException, IOException, InterruptedException, NoSuchMethodException, IllegalAccessException, InvocationTargetException { - startCassandraInstance("target/cassandra-data"); - ArrayList cfDefList = new ArrayList(2); - cfDefList.add(new CfDef("TestKeyspace", "Furniture").setComparator_type(BytesType.class.getSimpleName()) - .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); - - Cluster cluster = HFactory.getOrCreateCluster("TestPool", "localhost:9170"); - createKeyspace(cluster, "TestKeyspace", "org.apache.cassandra.locator.SimpleStrategy", 1, cfDefList); - keyspace = HFactory.createKeyspace("TestKeyspace", cluster); - entityMgr = new EntityManagerImpl(keyspace, "com.mycompany.furniture"); } diff --git a/object-mapper/src/test/java/me/prettyprint/hom/CassandraTestBase.java b/object-mapper/src/test/java/me/prettyprint/hom/CassandraTestBase.java index 4e1c4a3d3..d89c7fb33 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/CassandraTestBase.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/CassandraTestBase.java @@ -8,10 +8,12 @@ import me.prettyprint.cassandra.service.ThriftCfDef; import me.prettyprint.cassandra.service.ThriftKsDef; +import me.prettyprint.cassandra.testutils.EmbeddedServerHelper; import me.prettyprint.hector.api.Cluster; import me.prettyprint.hector.api.Keyspace; import me.prettyprint.hector.api.factory.HFactory; +import org.apache.cassandra.config.ConfigurationException; import org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner; import org.apache.cassandra.db.marshal.BytesType; import org.apache.cassandra.io.util.FileUtils; @@ -25,6 +27,7 @@ public class CassandraTestBase { protected static boolean cassandraStarted = false; protected static Keyspace keyspace; protected static Cluster cluster; + protected static EmbeddedServerHelper embedded; public static void startCassandraInstance(String pathToDataDir) throws TTransportException, IOException, InterruptedException, SecurityException, IllegalArgumentException, NoSuchMethodException, @@ -43,32 +46,21 @@ public static void startCassandraInstance(String pathToDataDir) throws TTranspor // eat } - CassandraServiceDataCleaner cleaner = new CassandraServiceDataCleaner(); - cleaner.prepare(); - EmbeddedCassandraService cassandra = new EmbeddedCassandraService(); + embedded = new EmbeddedServerHelper(); try { - cassandra.init(); - } - catch (TTransportException e) { - System.out.println( "Could not initialize Cassandra"); - e.printStackTrace(); - throw e; + embedded.setup(); + } catch (ConfigurationException ce) { + throw new RuntimeException(ce); } cassandraStarted = true; - - Thread t = new Thread(cassandra); - t.setName(cassandra.getClass().getSimpleName()); - t.setDaemon(true); - t.start(); - - System.out.println( "Successfully started Cassandra"); } public static void createKeyspace(Cluster cluster, String name, String strategy, int replicationFactor, List cfDefList) { try { - KsDef ksDef = new KsDef(name, strategy, replicationFactor, cfDefList); + KsDef ksDef = new KsDef(name, strategy, cfDefList); + ksDef.setReplication_factor(replicationFactor); cluster.addKeyspace(new ThriftKsDef(ksDef)); return; } @@ -94,20 +86,22 @@ public static void setupKeyspace() throws TTransportException, InterruptedException, NoSuchMethodException, IllegalAccessException, InvocationTargetException { startCassandraInstance("tmp/var/lib/cassandra"); - + ArrayList cfDefList = new ArrayList(2); cfDefList.add(new CfDef("TestKeyspace", "TestBeanColumnFamily").setComparator_type(BytesType.class.getSimpleName()) .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); cfDefList.add(new CfDef("TestKeyspace", "CustomIdColumnFamily").setComparator_type(BytesType.class.getSimpleName()) .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); cfDefList.add(new CfDef("TestKeyspace", "SimpleTestBeanColumnFamily").setComparator_type(BytesType.class.getSimpleName()) - .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); + .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); cfDefList.add(new CfDef("TestKeyspace", "SimpleRelationshipBeanColumnFamily").setComparator_type(BytesType.class.getSimpleName()) - .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); + .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); cfDefList.add(new CfDef("TestKeyspace", "NoAnonymousColumnFamily").setComparator_type(BytesType.class.getSimpleName()) .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); cfDefList.add(new CfDef("TestKeyspace", "ComplexColumnFamily").setComparator_type(BytesType.class.getSimpleName()) .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); + cfDefList.add(new CfDef("TestKeyspace", "Furniture").setComparator_type(BytesType.class.getSimpleName()) + .setKey_cache_size(0).setRow_cache_size(0).setGc_grace_seconds(86400)); cluster = HFactory.getOrCreateCluster("TestPool", "localhost:9161"); createKeyspace(cluster, "TestKeyspace", "org.apache.cassandra.locator.SimpleStrategy", 1, cfDefList); keyspace = HFactory.createKeyspace("TestKeyspace", cluster); diff --git a/object-mapper/src/test/java/me/prettyprint/hom/ClassCacheMgrTest.java b/object-mapper/src/test/java/me/prettyprint/hom/ClassCacheMgrTest.java index dc7a2f68c..e8801d592 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/ClassCacheMgrTest.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/ClassCacheMgrTest.java @@ -211,12 +211,12 @@ public void testParsingInheritedEntityWithoutAnonymousAddHandler() { public void testParsingComplexEntity() { ClassCacheMgr cacheMgr = new ClassCacheMgr(); CFMappingDef cfMapDef = cacheMgr.initializeCacheForClass(MyComplexEntity.class); - + KeyDefinition keyDef = cfMapDef.getKeyDef(); assertEquals( MyCompositePK.class, keyDef.getPkClazz() ); assertEquals( 2, keyDef.getIdPropertyMap().size() ); assertEquals( keyDef.getIdPropertyMap().size(), keyDef.getPropertyDescriptorMap().size() ); - + } @Ignore( "Cannot enable until method annotations are supported by ClassCacheMgr") diff --git a/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerFactoryTest.java b/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerFactoryTest.java deleted file mode 100644 index f20026ece..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerFactoryTest.java +++ /dev/null @@ -1,47 +0,0 @@ -package me.prettyprint.hom; - -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertNotNull; -import static org.junit.Assert.assertTrue; - -import java.util.HashMap; -import java.util.Map; - -import javax.persistence.EntityManager; -import javax.persistence.EntityManagerFactory; - -import org.junit.Before; -import org.junit.Ignore; -import org.junit.Test; - -@Ignore -public class EntityManagerFactoryTest extends CassandraTestBase { - - private EntityManagerFactory entityManagerFactory; - - - @Test - public void testCreateEntityManager() { - EntityManager entityManager = entityManagerFactory.createEntityManager(); - assertNotNull(entityManager); - assertTrue(entityManagerFactory.isOpen()); - } - - @Test - public void testCloseEntityManagerFactory() { - assertTrue(entityManagerFactory.isOpen()); - entityManagerFactory.close(); - assertFalse(entityManagerFactory.isOpen()); - - } - - @Before - public void initFactory() { - Map properties = new HashMap(); - properties.put(EntityManagerConfigurator.CLASSPATH_PREFIX_PROP, "me.prettyprint.hom.beans"); - properties.put(EntityManagerConfigurator.CLUSTER_NAME_PROP, "TestPool"); - properties.put(EntityManagerConfigurator.KEYSPACE_PROP, "TestKeyspace"); - EntityManagerConfigurator emc = new EntityManagerConfigurator(properties); - entityManagerFactory = new EntityManagerFactoryImpl(properties); - } -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerTest.java b/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerTest.java index 4bc6063b2..64b762103 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerTest.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/EntityManagerTest.java @@ -86,9 +86,9 @@ public void testPersistAndFindComplexType() { entity1.setStrProp2("str-prop-two"); em.persist(entity1); - + MyComplexEntity entity2 = em.find(MyComplexEntity.class, pkKey); - + assertEquals( entity1.getIntProp1(), entity2.getIntProp1() ); assertEquals( entity1.getStrProp1(), entity2.getStrProp1() ); assertEquals( entity1.getStrProp2(), entity2.getStrProp2() ); @@ -106,9 +106,9 @@ public void testMissingColumnsForPojoProps() { entity1.setStrProp2("str-prop-two"); em.persist(entity1); - + MyComplexEntity entity2 = em.find(MyComplexEntity.class, pkKey); - + assertEquals( entity1.getIntProp1(), entity2.getIntProp1() ); assertEquals( entity1.getStrProp1(), entity2.getStrProp1() ); assertEquals( entity1.getStrProp2(), entity2.getStrProp2() ); diff --git a/object-mapper/src/test/java/me/prettyprint/hom/HectorObjectMapperTest.java b/object-mapper/src/test/java/me/prettyprint/hom/HectorObjectMapperTest.java index 609db3ecf..7ec58f098 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/HectorObjectMapperTest.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/HectorObjectMapperTest.java @@ -144,17 +144,17 @@ public void testCreateInstanceCustomIdType() { assertEquals(id, obj.getId()); assertEquals(longProp1, obj.getLongProp1()); } - + @Test public void testIsSerializable() { assertTrue(HectorObjectMapper.isSerializable(UUID.class)); } - + @Test public void testIsNotSerializable() { assertFalse(HectorObjectMapper.isSerializable(HectorObjectMapper.class)); } - + // -------------------- @Before diff --git a/object-mapper/src/test/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyTest.java b/object-mapper/src/test/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyTest.java index 8af7820ce..179ef782c 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyTest.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/KeyConcatenationDelimiterStrategyTest.java @@ -6,7 +6,6 @@ import java.util.LinkedList; import java.util.List; -import org.apache.openjpa.persistence.kernel.common.apps.ByteArray; import org.junit.Test; public class KeyConcatenationDelimiterStrategyTest { diff --git a/object-mapper/src/test/java/me/prettyprint/hom/SpringContainerInitalizationTest.java b/object-mapper/src/test/java/me/prettyprint/hom/SpringContainerInitalizationTest.java deleted file mode 100644 index 97faaee6d..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/SpringContainerInitalizationTest.java +++ /dev/null @@ -1,60 +0,0 @@ -package me.prettyprint.hom; - -import static org.junit.Assert.*; - -import java.util.Properties; - -import javax.persistence.EntityManager; -import javax.persistence.EntityManagerFactory; -import javax.persistence.Persistence; -import javax.persistence.PersistenceUnit; - -import me.prettyprint.hom.beans.MyTestBean; -import me.prettyprint.hom.beans.SimpleTestBean; - -import org.junit.Ignore; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.springframework.test.context.ContextConfiguration; -import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; - -@RunWith(SpringJUnit4ClassRunner.class) -@ContextConfiguration(locations="/spring-jpa-container-context.xml") -public class SpringContainerInitalizationTest extends CassandraTestBase { - - @PersistenceUnit - private EntityManagerFactory entityManagerFactory; - - @Test - @Ignore - public void testLookupEntityManagerFactory() { - - // does not use open-jpa - entityManagerFactory = Persistence.createEntityManagerFactory("myprovider", buildProps()); - //entityManagerFactory = Persistence.createEntityManagerFactory("myprovider"); - - EntityManager entityManager = entityManagerFactory.createEntityManager(); - - assertTrue(entityManagerFactory.isOpen()); - } - - @Test - public void testStandaloneOpenJpaProviderConfig() { - //PersistenceProviderImpl ppi = new PersistenceProviderImpl(); - entityManagerFactory = Persistence.createEntityManagerFactory("openjpa"); - EntityManager entityManager = entityManagerFactory.createEntityManager(); - //assertTrue(entityManager.contains(new SimpleTestBean())); - - //EntityManager entityManager = entityManagerFactory.createEntityManager(); - - } - - private Properties buildProps() { - Properties props = new Properties(); - props.put(EntityManagerConfigurator.CLUSTER_NAME_PROP, "TestPool"); - props.put(EntityManagerConfigurator.CLASSPATH_PREFIX_PROP, "me.prettyprint.hom.beans"); - props.put(EntityManagerConfigurator.HOST_LIST_PROP, "localhost:9161"); - props.put(EntityManagerConfigurator.KEYSPACE_PROP, "TestKeyspace"); - return props; - } -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyComplexEntityMissingIdField.java b/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyComplexEntityMissingIdField.java index 9de281c73..65a066152 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyComplexEntityMissingIdField.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyComplexEntityMissingIdField.java @@ -10,10 +10,10 @@ @IdClass( MyCompositePK.class ) @Table(name="ComplexColumnFamily") public class MyComplexEntityMissingIdField { - + @Id private String strProp1; - + @Column( name ="strProp2") private String strProp2; diff --git a/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyCompositePK.java b/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyCompositePK.java index d1b580c17..a178a342d 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyCompositePK.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/badbeans/MyCompositePK.java @@ -5,7 +5,7 @@ /** * Used with @IdClass. Properties of this class must match @Id properties * defined by the entity. - * + * * @author B. Todd Burruss */ public class MyCompositePK implements Serializable { @@ -15,7 +15,7 @@ public class MyCompositePK implements Serializable { public MyCompositePK() { } - + public MyCompositePK(int intProp1, String strProp1) { this.intProp1 = intProp1; this.strProp1 = strProp1; diff --git a/object-mapper/src/test/java/me/prettyprint/hom/beans/ColorEmbedded.java b/object-mapper/src/test/java/me/prettyprint/hom/beans/ColorEmbedded.java index 4d182fef1..4ba895a98 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/beans/ColorEmbedded.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/beans/ColorEmbedded.java @@ -7,7 +7,7 @@ public class ColorEmbedded implements Serializable { private Colors color; - + public Colors getColor() { return color; } @@ -15,5 +15,5 @@ public Colors getColor() { public void setColor(Colors color) { this.color = color; } - + } diff --git a/object-mapper/src/test/java/me/prettyprint/hom/beans/MyComplexEntity.java b/object-mapper/src/test/java/me/prettyprint/hom/beans/MyComplexEntity.java index 27f08c684..372b22f9e 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/beans/MyComplexEntity.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/beans/MyComplexEntity.java @@ -13,13 +13,13 @@ public class MyComplexEntity { @Id private int intProp1; - + @Id private String strProp1; - + @Column( name ="strProp2") private String strProp2; - + @Column( name ="strProp3") private String strProp3; diff --git a/object-mapper/src/test/java/me/prettyprint/hom/beans/MyCompositePK.java b/object-mapper/src/test/java/me/prettyprint/hom/beans/MyCompositePK.java index 4b4259e4c..3af01bd23 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/beans/MyCompositePK.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/beans/MyCompositePK.java @@ -5,7 +5,7 @@ /** * Used with @IdClass. Properties of this class must match @Id properties * defined by the entity. - * + * * @author B. Todd Burruss */ public class MyCompositePK implements Serializable { @@ -15,7 +15,7 @@ public class MyCompositePK implements Serializable { public MyCompositePK() { } - + public MyCompositePK(int intProp1, String strProp1) { this.intProp1 = intProp1; this.strProp1 = strProp1; diff --git a/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleRelationshipBean.java b/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleRelationshipBean.java index 3b9ab8ca0..a43f0bb5b 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleRelationshipBean.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleRelationshipBean.java @@ -13,11 +13,11 @@ @Entity @Table(name="SimpleRelationshipBeanColumnFamily") public class SimpleRelationshipBean { - + private UUID baseId; - + private String myType; - + private Set simpleTestBeans; @Id diff --git a/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleTestBean.java b/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleTestBean.java index 35e4682ea..a14c35a25 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleTestBean.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/beans/SimpleTestBean.java @@ -10,32 +10,32 @@ @Entity @Table(name="SimpleTestBeanColumnFamily") public class SimpleTestBean implements Serializable { - + private long id; private String name; - - public SimpleTestBean() { + + public SimpleTestBean() { } - + public SimpleTestBean(long id, String name) { this.id = id; this.name = name; } - + @Id public long getId() { return id; } - + public void setId(long id) { this.id = id; } - + @Column(name="name") public String getName() { return name; } - + public void setName(String name) { this.name = name; } @@ -49,7 +49,7 @@ public String toString() { .append(getId()) .toString(); } - - + + } diff --git a/object-mapper/src/test/java/me/prettyprint/hom/cache/IdClassParserValidatorTest.java b/object-mapper/src/test/java/me/prettyprint/hom/cache/IdClassParserValidatorTest.java index 7e34c276f..c4075f0de 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/cache/IdClassParserValidatorTest.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/cache/IdClassParserValidatorTest.java @@ -19,19 +19,19 @@ public void testNotSerializable() { IdClassParserValidator parVal = new IdClassParserValidator(); CFMappingDef cfMapDef = new CFMappingDef(MyTestBean.class); - + IdClass anno = new IdClass() { @Override public Class annotationType() { return IdClass.class; } - + @Override public Class value() { return MyCompositePK.class; } }; - + parVal.parse(cacheMgr, anno, cfMapDef); } } diff --git a/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF1.java b/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF1.java index 55901d21a..be73fad78 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF1.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF1.java @@ -7,7 +7,7 @@ /** * Entity for testing duplication class <-> column family mapping. - * + * * @author Todd Burruss */ @Entity diff --git a/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF2.java b/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF2.java index 57292804f..6b3e2c068 100644 --- a/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF2.java +++ b/object-mapper/src/test/java/me/prettyprint/hom/dupebean/MyDupeCF2.java @@ -7,7 +7,7 @@ /** * Entity for testing duplication class <-> column family mapping. - * + * * @author Todd Burruss */ @Entity diff --git a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/EntityFacadeTest.java b/object-mapper/src/test/java/me/prettyprint/hom/openjpa/EntityFacadeTest.java deleted file mode 100644 index 4f61030b7..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/EntityFacadeTest.java +++ /dev/null @@ -1,49 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import static org.junit.Assert.*; - -import javax.persistence.EntityManagerFactory; -import javax.persistence.Persistence; - -import me.prettyprint.cassandra.serializers.LongSerializer; -import me.prettyprint.cassandra.serializers.StringSerializer; -import me.prettyprint.cassandra.serializers.UUIDSerializer; -import me.prettyprint.hector.api.query.SliceQuery; -import me.prettyprint.hom.beans.SimpleRelationshipBean; -import me.prettyprint.hom.beans.SimpleTestBean; - -import org.apache.openjpa.persistence.JPAFacadeHelper; -import org.junit.BeforeClass; -import org.junit.Ignore; -import org.junit.Test; - -public class EntityFacadeTest { - - private static EntityManagerFactory entityManagerFactory; - - @Test - public void testEntityFacadeValues() { - EntityFacade entityFacade = new EntityFacade(JPAFacadeHelper.getMetaData(entityManagerFactory, SimpleTestBean.class)); - assertEquals("SimpleTestBeanColumnFamily", entityFacade.getColumnFamilyName()); - assertEquals(LongSerializer.get(), entityFacade.getKeySerializer()); - assertEquals(StringSerializer.get(), entityFacade.getSerializer("name")); - } - - @Test - public void testOneToManyFacadeValues() { - EntityFacade entityFacade = new EntityFacade(JPAFacadeHelper.getMetaData(entityManagerFactory, SimpleRelationshipBean.class)); - assertEquals(UUIDSerializer.get(), entityFacade.getKeySerializer()); - } - - @Ignore - public void testBuildSliceQuery() { - EntityFacade entityFacade = new EntityFacade(JPAFacadeHelper.getMetaData(entityManagerFactory, SimpleTestBean.class)); - // ColumnFamily wrapper would work really well here - } - - @BeforeClass - public static void setup() { - entityManagerFactory = Persistence.createEntityManagerFactory("openjpa"); - } - -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/FullContainerTest.java b/object-mapper/src/test/java/me/prettyprint/hom/openjpa/FullContainerTest.java deleted file mode 100644 index 6f54b731d..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/FullContainerTest.java +++ /dev/null @@ -1,63 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import java.util.Map; - -import javax.persistence.EntityManager; -import javax.persistence.EntityTransaction; - -import me.prettyprint.hom.beans.MyTestBean; -import me.prettyprint.hom.beans.SimpleTestBean; - -import org.apache.openjpa.persistence.OpenJPAEntityManager; -import org.apache.openjpa.persistence.OpenJPAEntityManagerFactorySPI; -import org.apache.openjpa.persistence.test.AbstractPersistenceTestCase; -import org.junit.Ignore; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -@Ignore -public class FullContainerTest extends AbstractPersistenceTestCase { - private static Logger log = LoggerFactory.getLogger(FullContainerTest.class); - @Ignore - public void ptestCreateEntityManager() { - OpenJPAEntityManagerFactorySPI emf = createNamedEMF("openjpa"); - try { - EntityManager em = emf.createEntityManager(); - - EntityTransaction t = em.getTransaction(); - assertNotNull(t); - t.begin(); - t.setRollbackOnly(); - t.rollback(); - - // openjpa-facade test - assertTrue(em instanceof OpenJPAEntityManager); - OpenJPAEntityManager ojem = (OpenJPAEntityManager) em; - ojem.getFetchPlan().setMaxFetchDepth(1); - assertEquals(1, ojem.getFetchPlan().getMaxFetchDepth()); - em.close(); - } finally { - closeEMF(emf); - } - } - - public void testPersistEntity() { - OpenJPAEntityManagerFactorySPI emf = createNamedEMF("openjpa"); - - - try { - EntityManager em = emf.createEntityManager(); - SimpleTestBean stb = new SimpleTestBean(); - stb.setId(1); - stb.setName("my name"); - em.persist(stb); - - - } catch (Exception e) { - e.printStackTrace(); - - } finally { - closeEMF(emf); - } - } -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/ManagedEntityTestBase.java b/object-mapper/src/test/java/me/prettyprint/hom/openjpa/ManagedEntityTestBase.java deleted file mode 100644 index 81acb587d..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/ManagedEntityTestBase.java +++ /dev/null @@ -1,18 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import javax.persistence.EntityManagerFactory; -import javax.persistence.Persistence; - -import org.junit.BeforeClass; - -import me.prettyprint.hom.CassandraTestBase; - -public abstract class ManagedEntityTestBase extends CassandraTestBase { - - protected static EntityManagerFactory entityManagerFactory; - - @BeforeClass - public static void setup() { - entityManagerFactory = Persistence.createEntityManagerFactory("openjpa"); - } -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/MappingUtilsTest.java b/object-mapper/src/test/java/me/prettyprint/hom/openjpa/MappingUtilsTest.java deleted file mode 100644 index 84144d570..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/MappingUtilsTest.java +++ /dev/null @@ -1,76 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import static org.junit.Assert.*; - -import java.util.List; - -import javax.persistence.EntityManagerFactory; -import javax.persistence.Persistence; - -import me.prettyprint.cassandra.model.ExecutingKeyspace; -import me.prettyprint.cassandra.model.thrift.ThriftSliceQuery; -import me.prettyprint.cassandra.serializers.LongSerializer; -import me.prettyprint.hector.api.Keyspace; -import me.prettyprint.hector.api.Serializer; -import me.prettyprint.hector.api.mutation.Mutator; -import me.prettyprint.hector.api.query.SliceQuery; -import me.prettyprint.hom.beans.SimpleTestBean; -import me.prettyprint.hom.openjpa.MappingUtils; - -import org.apache.openjpa.kernel.Broker; -import org.apache.openjpa.meta.ClassMetaData; -import org.apache.openjpa.persistence.JPAFacadeHelper; -import org.apache.openjpa.util.LongId; -import org.junit.BeforeClass; -import org.junit.Ignore; -import org.junit.Test; -import org.mockito.Mock; -import org.mockito.Mockito; - -public class MappingUtilsTest { - - private MappingUtils mappingUtils; - private static EntityManagerFactory entityManagerFactory; - private Keyspace keyspace = Mockito.mock(ExecutingKeyspace.class); - - @Test - public void testConvertClassToSlice() { - mappingUtils = new MappingUtils(); - - List cols = mappingUtils.buildColumnList(JPAFacadeHelper.getMetaData(entityManagerFactory, SimpleTestBean.class)); - - assertEquals("name",cols.get(0)); - } - - @Test - public void testGetSerializer() { - mappingUtils = new MappingUtils(); - - Serializer serializer = mappingUtils.getSerializer(new LongId(SimpleTestBean.class, 1L)); - - assertTrue(serializer instanceof LongSerializer); - } - - @Test - public void testBuildSliceQuery() { - mappingUtils = new MappingUtils(); - LongId id = new LongId(SimpleTestBean.class, 1L); - EntityFacade entityFacade = new EntityFacade(JPAFacadeHelper.getMetaData(entityManagerFactory, SimpleTestBean.class)); - SliceQuery sliceQuery = mappingUtils.buildSliceQuery(id, entityFacade, keyspace); - assertTrue(((ThriftSliceQuery)sliceQuery).getColumnNames().contains("name")); - assertEquals(1,((ThriftSliceQuery)sliceQuery).getColumnNames().size()); - } - - @Test - public void testBuildMutation() { - mappingUtils = new MappingUtils(); - LongId id = new LongId(SimpleTestBean.class, 1L); - Broker broker = JPAFacadeHelper.toBroker(entityManagerFactory.createEntityManager()); - //Mutator mutator = mappingUtils.buildMutator(id, broker., keyspace); - } - - @BeforeClass - public static void setup() { - entityManagerFactory = Persistence.createEntityManagerFactory("openjpa"); - } -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/OneToManyRelationshipTest.java b/object-mapper/src/test/java/me/prettyprint/hom/openjpa/OneToManyRelationshipTest.java deleted file mode 100644 index f0cb03985..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/OneToManyRelationshipTest.java +++ /dev/null @@ -1,45 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import static org.junit.Assert.*; - -import java.util.UUID; - -import javax.persistence.EntityManager; - -import me.prettyprint.cassandra.utils.TimeUUIDUtils; -import me.prettyprint.hom.beans.SimpleRelationshipBean; -import me.prettyprint.hom.beans.SimpleTestBean; -import me.prettyprint.hom.service.EntitySchemaManager; -import me.prettyprint.hom.service.EntitySchemaStatus; - -import org.apache.openjpa.persistence.JPAFacadeHelper; -import org.junit.Before; -import org.junit.Test; - - -public class OneToManyRelationshipTest extends ManagedEntityTestBase { - - @Test - public void testManyToOneSaveAndLoad() { - EntityManager em = entityManagerFactory.createEntityManager(); - em.getTransaction().begin(); - - SimpleRelationshipBean srb = new SimpleRelationshipBean(); - UUID uuid = TimeUUIDUtils.getUniqueTimeUUIDinMillis(); - srb.setBaseId(uuid); - srb.setMyType("my_type"); - srb.addSimpleTestBean(new SimpleTestBean(5L, "my owned bean")); - - em.persist(srb); - - em.getTransaction().commit(); - em.close(); - - EntityManager em2 = entityManagerFactory.createEntityManager(); - SimpleRelationshipBean srb2 = em2.find(SimpleRelationshipBean.class, uuid); - assertEquals(1,srb.getSimpleTestBeans().size()); - assertEquals("my owned bean", srb.getSimpleTestBeans().iterator().next().getName()); - assertEquals(uuid,srb.getBaseId()); - - } -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/StandaloneEntityManagerFactoryTest.java b/object-mapper/src/test/java/me/prettyprint/hom/openjpa/StandaloneEntityManagerFactoryTest.java deleted file mode 100644 index 0befcad26..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/openjpa/StandaloneEntityManagerFactoryTest.java +++ /dev/null @@ -1,40 +0,0 @@ -package me.prettyprint.hom.openjpa; - -import static org.junit.Assert.*; - -import javax.persistence.EntityManager; -import javax.persistence.EntityManagerFactory; -import javax.persistence.Persistence; - -import me.prettyprint.hom.CassandraTestBase; -import me.prettyprint.hom.beans.SimpleTestBean; - -import org.junit.BeforeClass; -import org.junit.Test; - -public class StandaloneEntityManagerFactoryTest extends ManagedEntityTestBase { - - @Test - public void testBuildEntityManagerFactory() { - - EntityManager em = entityManagerFactory.createEntityManager(); - em.getTransaction().begin(); - em.persist(new SimpleTestBean(1, "foo")); - em.getTransaction().commit(); - em.close(); - - //em.getTransaction().begin(); - - EntityManager em2 = entityManagerFactory.createEntityManager(); - SimpleTestBean stb = em2.find(SimpleTestBean.class, 1); - //em.getTransaction().commit(); - assertEquals("foo",stb.getName()); - - - em2.getTransaction().begin(); - em2.remove(stb); - em2.getTransaction().commit(); - em2.close(); - } - -} diff --git a/object-mapper/src/test/java/me/prettyprint/hom/service/EntitySchemaManagerTest.java b/object-mapper/src/test/java/me/prettyprint/hom/service/EntitySchemaManagerTest.java deleted file mode 100644 index 6b08414d6..000000000 --- a/object-mapper/src/test/java/me/prettyprint/hom/service/EntitySchemaManagerTest.java +++ /dev/null @@ -1,67 +0,0 @@ -package me.prettyprint.hom.service; - -import static org.junit.Assert.assertEquals; - -import javax.persistence.Column; -import javax.persistence.Entity; -import javax.persistence.EntityManagerFactory; -import javax.persistence.Id; -import javax.persistence.Persistence; - -import me.prettyprint.hom.CassandraTestBase; -import me.prettyprint.hom.beans.SimpleRelationshipBean; -import me.prettyprint.hom.openjpa.EntityFacade; -import me.prettyprint.hom.service.EntitySchemaStatus.SchemaResult; - -import org.apache.openjpa.persistence.JPAFacadeHelper; -import org.junit.BeforeClass; -import org.junit.Test; - -public class EntitySchemaManagerTest extends CassandraTestBase { - - private static EntitySchemaManager entitySchemaManager; - private static EntityManagerFactory entityManagerFactory; - private static EntityFacade entityFacade; - - @BeforeClass - public static void setup() { - entitySchemaManager = new EntitySchemaManager(cluster, keyspace); - entityManagerFactory = Persistence.createEntityManagerFactory("openjpa"); - } - - @Test - public void testCreateSchemaFromEntity() { - entityFacade = new EntityFacade(JPAFacadeHelper.getMetaData(entityManagerFactory, ReallySimpleBean.class)); - EntitySchemaStatus ecs = entitySchemaManager.createSchema(entityFacade); - assertEquals(SchemaResult.CREATED, ecs.getSchemaResult()); - } - - @Test - public void testCreateSchemaFailExisting() { - entityFacade = new EntityFacade(JPAFacadeHelper.getMetaData(entityManagerFactory, ReallySimpleBean.class)); - EntitySchemaStatus ecs = entitySchemaManager.createSchema(entityFacade); - assertEquals(SchemaResult.NOT_MODIFIED, ecs.getSchemaResult()); - } - - @Entity - class ReallySimpleBean { - int id; - String val; - @Id - public int getId() { - return id; - } - public void setId(int id) { - this.id = id; - } - @Column - public String getVal() { - return val; - } - public void setVal(String val) { - this.val = val; - } - - - } -} diff --git a/object-mapper/src/test/resources/META-INF/persistence.xml b/object-mapper/src/test/resources/META-INF/persistence.xml deleted file mode 100644 index 444de2b0e..000000000 --- a/object-mapper/src/test/resources/META-INF/persistence.xml +++ /dev/null @@ -1,43 +0,0 @@ - - - me.prettyprint.hom.CassandraPersistenceProvider - - - - - - - - - - org.apache.openjpa.persistence.PersistenceProviderImpl - - me.prettyprint.hom.beans.SimpleTestBean - me.prettyprint.hom.beans.SimpleRelationshipBean - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/object-mapper/src/test/resources/cassandra.yaml b/object-mapper/src/test/resources/cassandra.yaml index 1bf3adc8f..521d8edd9 100644 --- a/object-mapper/src/test/resources/cassandra.yaml +++ b/object-mapper/src/test/resources/cassandra.yaml @@ -1,9 +1,9 @@ -# Cassandra storage config YAML +# Cassandra storage config YAML -#NOTE !!!!!!!! NOTE +#NOTE !!!!!!!! NOTE # See http://wiki.apache.org/cassandra/StorageConfiguration for # full explanations of configuration directives -#NOTE !!!!!!!! NOTE +#NOTE !!!!!!!! NOTE # The name of the cluster. This is mainly used to prevent machines in # one logical cluster from joining another. @@ -58,10 +58,10 @@ commitlog_directory: target/cassandra-data/commitlog # saved caches saved_caches_directory: target/cassandra-data/saved_caches -# Size to allow commitlog to grow to before creating a new segment +# Size to allow commitlog to grow to before creating a new segment commitlog_rotation_threshold_in_mb: 128 -# commitlog_sync may be either "periodic" or "batch." +# commitlog_sync may be either "periodic" or "batch." # When in batch mode, Cassandra won't ack writes until the commit log # has been fsynced to disk. It will wait up to # CommitLogSyncBatchWindowInMS milliseconds for other writes, before @@ -73,12 +73,17 @@ commitlog_sync: periodic # milliseconds. commitlog_sync_period_in_ms: 10000 -# Addresses of hosts that are deemed contact points. -# Cassandra nodes use this list of hosts to find each other and learn -# the topology of the ring. You must change this if you are running -# multiple nodes! -seeds: - - 127.0.0.1 +# any class that implements the SeedProvider interface and has a constructor that takes a Map of +# parameters will do. +seed_provider: + # Addresses of hosts that are deemed contact points. + # Cassandra nodes use this list of hosts to find each other and learn + # the topology of the ring. You must change this if you are running + # multiple nodes! + - class_name: org.apache.cassandra.locator.SimpleSeedProvider + parameters: + # seeds is actually a comma-delimited list of addresses. + - seeds: "127.0.0.1" # Access mode. mmapped i/o is substantially faster, but only practical on # a 64bit machine (which notably does not include EC2 "small" instances) @@ -105,7 +110,7 @@ concurrent_writes: 8 # By default this will be set to the amount of data directories defined. #memtable_flush_writers: 1 -# Buffer size to use when performing contiguous column slices. +# Buffer size to use when performing contiguous column slices. # Increase this to the size of the column slices you typically perform sliced_buffer_size_in_kb: 64 @@ -115,7 +120,7 @@ storage_port: 7001 # Address to bind to and tell other Cassandra nodes to connect to. You # _must_ change this if you want multiple nodes to be able to # communicate! -# +# # Leaving it blank leaves it up to InetAddress.getLocalHost(). This # will always do the Right Thing *if* the node is properly configured # (hostname, name resolution, etc), and the Right Thing is to use the @@ -127,7 +132,7 @@ listen_address: localhost # The address to bind the Thrift RPC service to -- clients connect # here. Unlike ListenAddress above, you *can* specify 0.0.0.0 here if # you want Thrift to listen on all interfaces. -# +# # Leaving this blank has the same effect it does for ListenAddress, # (i.e. it will be based on the configured hostname of the node). rpc_address: localhost @@ -178,7 +183,7 @@ column_index_size_in_kb: 64 # will be logged specifying the row key. in_memory_compaction_limit_in_mb: 64 -# Time to wait for a reply from other nodes before failing the command +# Time to wait for a reply from other nodes before failing the command rpc_timeout_in_ms: 10000 # phi value that must be reached for a host to be marked down. @@ -208,7 +213,7 @@ endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch dynamic_snitch: true # controls how often to perform the more expensive part of host score # calculation -dynamic_snitch_update_interval_in_ms: 100 +dynamic_snitch_update_interval_in_ms: 100 # controls how often to reset all host scores, allowing a bad host to # possibly recover dynamic_snitch_reset_interval_in_ms: 600000 @@ -238,7 +243,7 @@ request_scheduler: org.apache.cassandra.scheduler.NoScheduler # NoScheduler - Has no options # RoundRobin # - throttle_limit -- The throttle_limit is the number of in-flight -# requests per client. Requests beyond +# requests per client. Requests beyond # that limit are queued up until # running requests can complete. # The value of 80 here is twice the number of @@ -266,7 +271,7 @@ request_scheduler: org.apache.cassandra.scheduler.NoScheduler # the index is at the cost of space. index_interval: 128 -# A ColumnFamily is the Cassandra concept closest to a relational table. +# A ColumnFamily is the Cassandra concept closest to a relational table. # # Keyspaces are separate groups of ColumnFamilies. Except in very # unusual circumstances you will have one Keyspace per application. @@ -277,8 +282,8 @@ index_interval: 128 # - replica_placement_strategy: the class that determines how replicas # are distributed among nodes. Contains both the class as well as # configuration information. Must extend AbstractReplicationStrategy. -# Out of the box, Cassandra provides -# * org.apache.cassandra.locator.SimpleStrategy +# Out of the box, Cassandra provides +# * org.apache.cassandra.locator.SimpleStrategy # * org.apache.cassandra.locator.NetworkTopologyStrategy # * org.apache.cassandra.locator.OldNetworkTopologyStrategy # @@ -286,7 +291,7 @@ index_interval: 128 # replica at the node whose token is closest to the key (as determined # by the Partitioner), and additional replicas on subsequent nodes # along the ring in increasing Token order. -# +# # With NetworkTopologyStrategy, # for each datacenter, you can specify how many replicas you want # on a per-keyspace basis. Replicas are placed on different racks @@ -299,8 +304,8 @@ index_interval: 128 # DC1 : 3 # DC2 : 2 # DC3 : 1 -# -# OldNetworkToplogyStrategy [formerly RackAwareStrategy] +# +# OldNetworkToplogyStrategy [formerly RackAwareStrategy] # places one replica in each of two datacenters, and the third on a # different rack in in the first. Additional datacenters are not # guaranteed to get a replica. Additional replicas after three are placed @@ -318,7 +323,7 @@ index_interval: 128 # and IntegerType (a generic variable-length integer type). # You can also specify the fully-qualified class name to a class of # your choice extending org.apache.cassandra.db.marshal.AbstractType. -# +# # ColumnFamily optional parameters: # - keys_cached: specifies the number of keys per sstable whose # locations we keep in memory in "mostly LRU" order. (JUST the key @@ -330,7 +335,7 @@ index_interval: 128 # or ColumnFamilies with high write:read ratios. Specify a fraction # (value less than 1) or an absolute number of rows to cache. # Defaults to 0. (i.e. row caching is off by default) -# - comment: used to attach additional human-readable information about +# - comment: used to attach additional human-readable information about # the column family to its definition. # - read_repair_chance: specifies the probability with which read # repairs should be invoked on non-quorum reads. must be between 0 @@ -352,7 +357,7 @@ index_interval: 128 # row caches. The row caches can be saved periodically and if one # exists on startup it will be loaded. # - key_cache_save_period_in_seconds: number of seconds between saving -# key caches. The key caches can be saved periodically and if one +# key caches. The key caches can be saved periodically and if one # exists on startup it will be loaded. # - memtable_flush_after_mins: The maximum time to leave a dirty table # unflushed. This should be large enough that it won't cause a flush @@ -366,52 +371,4 @@ index_interval: 128 # NOTE: this keyspace definition is for demonstration purposes only. # Cassandra will not load these definitions during startup. See # http://wiki.apache.org/cassandra/FAQ#no_keyspaces for an explanation. -keyspaces: - - name: Keyspace1 - replica_placement_strategy: org.apache.cassandra.locator.SimpleStrategy - replication_factor: 1 - column_families: - - name: Standard1 - compare_with: BytesType - keys_cached: 10000 - rows_cached: 1000 - row_cache_save_period_in_seconds: 0 - key_cache_save_period_in_seconds: 3600 - memtable_flush_after_mins: 59 - memtable_throughput_in_mb: 255 - memtable_operations_in_millions: 0.29 - - - name: Standard2 - compare_with: UTF8Type - read_repair_chance: 0.1 - keys_cached: 100 - gc_grace_seconds: 0 - min_compaction_threshold: 5 - max_compaction_threshold: 31 - - - name: StandardByUUID1 - compare_with: TimeUUIDType - - - name: Super1 - column_type: Super - compare_with: BytesType - compare_subcolumns_with: BytesType - - - name: Super2 - column_type: Super - compare_subcolumns_with: UTF8Type - rows_cached: 10000 - keys_cached: 50 - comment: 'A column family with supercolumns, whose column and subcolumn names are UTF8 strings' - - - name: Super3 - column_type: Super - compare_with: LongType - comment: 'A column family with supercolumns, whose column names are Longs (8 bytes)' - - - name: Indexed1 - default_validation_class: LongType - column_metadata: - - name: birthdate - validator_class: LongType - index_type: KEYS + diff --git a/object-mapper/src/test/resources/spring-jpa-container-context.xml b/object-mapper/src/test/resources/spring-jpa-container-context.xml deleted file mode 100644 index 46ba03deb..000000000 --- a/object-mapper/src/test/resources/spring-jpa-container-context.xml +++ /dev/null @@ -1,34 +0,0 @@ - - - - - - - - - - - - \ No newline at end of file diff --git a/pom.xml b/pom.xml index cb8b50d72..5ef0052ad 100644 --- a/pom.xml +++ b/pom.xml @@ -88,7 +88,12 @@ org.apache.cassandra cassandra-all - 0.7.0 + 0.8.0-rc1 + + + org.apache.cassandra + cassandra-thrift + 0.8.0-rc1 org.apache.cassandra @@ -96,9 +101,9 @@ 0.7.0 - org.apache.cassandra.deps + org.apache.thrift libthrift - 0.5.0 + 0.6.1 com.github.stephenc.high-scale-lib @@ -137,8 +142,8 @@ mockito-all 1.8.2 test - - + + @@ -321,5 +326,12 @@ 3.0.0.RELEASE 1.6.1 + + + apache-staging + apache-staging + https://repository.apache.org/content/repositories/orgapachecassandra-114 + +