Discussion:
ANN: The second hbase-0.96-1 release candidate is available for download
(too old to reply)
Stack
2013-11-28 03:57:42 UTC
Permalink
The second hbase-0.96.1 release candidate is available for download:

http://people.apache.org/~stack/hbase-0.96.1RC1/

The maven artifacts are available here:

https://repository.apache.org/content/repositories/orgapachehbase-016/

The first RC was sunk by:

HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream

We broke hive building against HBase.

137 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes.

Shall we release this as hbase-0.96.1? Please vote by December 3rd.

Thanks,
Your HBase Team

1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
127 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr) causes
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in StoreScanner
HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to go OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom implementation
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key also
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits sporadically
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is more
than default 30 seconds(or any configured). -> Regionserver node time is
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation completion.
After split/compaction operation using UI, the page is not automatically
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection to
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom implementation
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose permissions
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is not
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin up
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region is
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server OOM
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when re-enabling
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to server
HBASE-9593 Region server left in online servers list forever if it went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command line
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
Jonathan Hsieh
2013-12-04 16:10:31 UTC
Permalink
-1

I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).

https://github.com/jmhsieh/hbase/tree/increval

5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1

Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).

Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr) causes
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in StoreScanner
HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to go
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom implementation
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key also
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits sporadically
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is more
than default 30 seconds(or any configured). -> Regionserver node time is
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation completion.
After split/compaction operation using UI, the page is not automatically
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection to
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom implementation
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose permissions
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 by
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is not
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin up
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region is
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server OOM
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when re-enabling
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command line
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org
Jean-Marc Spaggiari
2013-12-04 16:13:18 UTC
Permalink
Hi Jon,

Is your test running well on 0.96.0?

JM
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to go
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node time
is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation completion.
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection
to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96
by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin
up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region
is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
Jonathan Hsieh
2013-12-04 16:16:04 UTC
Permalink
I don't know -- I'll test this against 0.96.0 and report.

Jon.
Post by Jean-Marc Spaggiari
Hi Jon,
Is your test running well on 0.96.0?
JM
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2
with
Post by Jonathan Hsieh
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I
had
Post by Jonathan Hsieh
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
Post by Jonathan Hsieh
Post by Stack
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits too
fast
Post by Jonathan Hsieh
Post by Stack
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to
go
Post by Jonathan Hsieh
Post by Stack
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node time
is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
Post by Jonathan Hsieh
Post by Stack
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate
to
Post by Jonathan Hsieh
Post by Stack
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency
collection
Post by Jonathan Hsieh
to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even
if
Post by Jonathan Hsieh
Post by Stack
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances
without
Post by Jonathan Hsieh
Post by Stack
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on
0.96
Post by Jonathan Hsieh
by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
Post by Jonathan Hsieh
Post by Stack
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin
up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from
0.94
Post by Jonathan Hsieh
Post by Stack
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region
is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings
to
Post by Jonathan Hsieh
Post by Stack
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another
env
Post by Jonathan Hsieh
Post by Stack
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Jonathan Hsieh
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it
went
Post by Jonathan Hsieh
Post by Stack
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
Post by Jonathan Hsieh
Post by Stack
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper()
may
Post by Jonathan Hsieh
Post by Stack
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
Post by Jonathan Hsieh
Post by Stack
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
Post by Jonathan Hsieh
Post by Stack
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org
Jonathan Hsieh
2013-12-04 16:16:31 UTC
Permalink
I've filed an issue for this and will post more results there. (against
0.96.0 etc).

https://issues.apache.org/jira/browse/HBASE-10079
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr) causes
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to go
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom implementation
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key also
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits sporadically
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is more
than default 30 seconds(or any configured). -> Regionserver node time is
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation completion.
After split/compaction operation using UI, the page is not automatically
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection
to
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom implementation
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96
by
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is not
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin up
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region is
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when re-enabling
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org
lars hofhansl
2013-12-04 16:58:06 UTC
Permalink
Interesting that at least the concurrent flush part wasn't caught by the unit tests (TestAtomicOperation specifically).



________________________________
From: Jonathan Hsieh <jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org>
To: "***@hbase.apache.org" <dev-50Pas4EWwPEyzMRdD/***@public.gmane.org>
Sent: Wednesday, December 4, 2013 8:16 AM
Subject: Re: ANN: The second hbase-0.96-1 release candidate is available for download


I've filed an issue for this and will post more results there.  (against
0.96.0 etc).

https://issues.apache.org/jira/browse/HBASE-10079
-1
I have a new test I tried on 0.96.1rc1. (polishing to share).  Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here)..  I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94.  Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
  hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster).  Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection.  5 runs.  count = 250000. on multiple runs.
  Correctness verified.  1638 inc/s throughput.
Run 2: flushes table with incrementing row.  count = 246875 !=250000.
  correctness failed.  1517 inc/s throughput.
Run 3: kill of rs hosting incremented row.  count = 243750 != 250000.
Correctness failed.  1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row.  246878.!= 250000.
  Correctness failed. 1395 inc/s (including recovery)
  http://people.apache.org/~stack/hbase-0.96.1RC1/
  https://repository.apache.org/content/repositories/orgapachehbase-016/
  HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes.
Shall we release this as hbase-0.96.1?  Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
  http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes.  See below for the full list.
Shall we release this as hbase-0.96.1?  Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr) causes
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to go
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom implementation
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key also
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits sporadically
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is more
than default 30 seconds(or any configured). -> Regionserver node time is
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation completion.
After split/compaction operation using UI, the page is not automatically
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection
to
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom implementation
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96
by
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is not
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin up
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region is
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when re-enabling
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// ***@cloudera.com
Jonathan Hsieh
2013-12-04 17:25:18 UTC
Permalink
That test doesn't do a flush during the increments so doesn't trigger the
bug.

Jon.
Post by lars hofhansl
Interesting that at least the concurrent flush part wasn't caught by the
unit tests (TestAtomicOperation specifically).
________________________________
Sent: Wednesday, December 4, 2013 8:16 AM
Subject: Re: ANN: The second hbase-0.96-1 release candidate is available for download
I've filed an issue for this and will post more results there. (against
0.96.0 etc).
https://issues.apache.org/jira/browse/HBASE-10079
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2
with
Post by Jonathan Hsieh
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I
had
Post by Jonathan Hsieh
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Jonathan Hsieh
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
HBASE-10026 HBaseAdmin#createTable could fail if region splits too
fast
Post by Jonathan Hsieh
Post by Stack
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to
go
Post by Jonathan Hsieh
Post by Stack
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Jonathan Hsieh
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Jonathan Hsieh
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Jonathan Hsieh
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Jonathan Hsieh
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node time
is
Post by Jonathan Hsieh
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
Post by Jonathan Hsieh
Post by Stack
After split/compaction operation using UI, the page is not
automatically
Post by Jonathan Hsieh
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate
to
Post by Jonathan Hsieh
Post by Stack
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection
to
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Jonathan Hsieh
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96
by
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
Post by Jonathan Hsieh
Post by Stack
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Jonathan Hsieh
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
Post by Jonathan Hsieh
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin
up
Post by Jonathan Hsieh
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region
is
Post by Jonathan Hsieh
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another
env
Post by Jonathan Hsieh
Post by Stack
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Jonathan Hsieh
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Jonathan Hsieh
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it
went
Post by Jonathan Hsieh
Post by Stack
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
Post by Jonathan Hsieh
Post by Stack
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
Post by Jonathan Hsieh
Post by Stack
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
Post by Jonathan Hsieh
Post by Stack
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org
lars hofhansl
2013-12-04 20:25:05 UTC
Permalink
Meh. You're right. I only added that for the multi row transactions I did (also a test in TestAtomicOperation).

So we should add this for the Increment test as well.



________________________________
From: Jonathan Hsieh <jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org>
To: "dev-50Pas4EWwPEyzMRdD/***@public.gmane.org" <dev-***@public.gmane.orgache.org>; lars hofhansl <larsh-1oDqGaOF3Lkdnm+***@public.gmane.org>
Sent: Wednesday, December 4, 2013 9:25 AM
Subject: Re: ANN: The second hbase-0.96-1 release candidate is available for download


That test doesn't do a flush during the increments so doesn't trigger the
bug.

Jon.
Post by lars hofhansl
Interesting that at least the concurrent flush part wasn't caught by the
unit tests (TestAtomicOperation specifically).
________________________________
Sent: Wednesday, December 4, 2013 8:16 AM
Subject: Re: ANN: The second hbase-0.96-1 release candidate is available for download
I've filed an issue for this and will post more results there.  (against
0.96.0 etc).
https://issues.apache.org/jira/browse/HBASE-10079
-1
I have a new test I tried on 0.96.1rc1. (polishing to share).  Run #2
with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I
had
verified there but it is broken here)..  I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94.  Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
  hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster).  Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection.  5 runs.  count = 250000. on multiple runs.
  Correctness verified.  1638 inc/s throughput.
Run 2: flushes table with incrementing row.  count = 246875 !=250000.
  correctness failed.  1517 inc/s throughput.
Run 3: kill of rs hosting incremented row.  count = 243750 != 250000.
Correctness failed.  1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row.  246878.!= 250000.
  Correctness failed. 1395 inc/s (including recovery)
  http://people.apache.org/~stack/hbase-0.96.1RC1/
  https://repository.apache.org/content/repositories/orgapachehbase-016/
  HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance improvement
and bug fixes.
Shall we release this as hbase-0.96.1?  Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
  http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes.  See below for the full list.
Shall we release this as hbase-0.96.1?  Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
HBASE-10026 HBaseAdmin#createTable could fail if region splits too
fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to
go
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
than default 30 seconds(or any configured). -> Regionserver node time
is
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
After split/compaction operation using UI, the page is not
automatically
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate
to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection
to
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96
by
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin
up
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region
is
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another
env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it
went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon-psgPW5cihnJWk0Htik3J/***@public.gmane.org
Stack
2013-12-04 22:16:06 UTC
Permalink
Good one Jon.

RC is sunk. Will put up a new one when this is fixed.

Thanks,
St.Ack
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to go
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node time
is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation completion.
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate to
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency collection
to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96
by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if replaying
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin
up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region
is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it went
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a region
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
Stack
2013-12-06 21:43:43 UTC
Permalink
Jon fixed the Increment issue. Nicolas may have fixed the slow down issue
-- we'll know after a few runs this weekend.

I'll put up a new RC Monday regardless.

St.Ack
Post by Stack
Good one Jon.
RC is sunk. Will put up a new one when this is fixed.
Thanks,
St.Ack
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
Post by Stack
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits too
fast
Post by Stack
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to
go
Post by Stack
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node time
is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
Post by Stack
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate
to
Post by Stack
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency
collection to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances without
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on
0.96 by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
Post by Stack
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin
up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from 0.94
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a region
is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region Server
OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings to
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another
env
Post by Stack
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it
went
Post by Stack
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
Post by Stack
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
Post by Stack
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
Post by Stack
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
Stack
2013-12-10 00:09:18 UTC
Permalink
Are we good to go on RC2? Just waiting on HBase-10101? Anything else?
Thanks,
St.Ack
Post by Stack
Jon fixed the Increment issue. Nicolas may have fixed the slow down issue
-- we'll know after a few runs this weekend.
I'll put up a new RC Monday regardless.
St.Ack
Post by Stack
Good one Jon.
RC is sunk. Will put up a new one when this is fixed.
Thanks,
St.Ack
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or #4
(kill -9) are a regression against 0.94. Here's a link to the testcode
(its got all kinds of caveats at the moment and will likely get turn into
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various number
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
Post by Stack
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits too
fast
Post by Stack
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers to
go
Post by Stack
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node
time is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
Post by Stack
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't replicate
to
Post by Stack
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple clients
HBASE-10007 PerformanceEvaluation: Add sampling and latency
collection to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even
if
Post by Stack
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances
without
Post by Stack
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in locateRegion
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on
0.96 by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
Post by Stack
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region is
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not
spin up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from
0.94
Post by Stack
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a
region is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a tool
HBASE-9737 Corrupt HFile cause resource leak leading to Region
Server OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings
to
Post by Stack
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each reseek
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another
env
Post by Stack
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of LiteralByteString
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed to
server
HBASE-9593 Region server left in online servers list forever if it
went
Post by Stack
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the protobuffed
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
Post by Stack
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper()
may
Post by Stack
not
be working correctly
HBASE-7525 A canary monitoring program specifically for regionserver
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
Post by Stack
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from command
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
Post by Stack
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better to
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
Jimmy Xiang
2013-12-10 00:13:28 UTC
Permalink
HBASE-10101 is in. Thanks.
Post by Stack
Are we good to go on RC2? Just waiting on HBase-10101? Anything else?
Thanks,
St.Ack
Post by Stack
Jon fixed the Increment issue. Nicolas may have fixed the slow down
issue
Post by Stack
-- we'll know after a few runs this weekend.
I'll put up a new RC Monday regardless.
St.Ack
Post by Stack
Good one Jon.
RC is sunk. Will put up a new one when this is fixed.
Thanks,
St.Ack
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had
verified there but it is broken here).. I'm not sure if #3 (kill) or
#4
Post by Stack
Post by Stack
Post by Jonathan Hsieh
(kill -9) are a regression against 0.94. Here's a link to the
testcode
Post by Stack
Post by Stack
Post by Jonathan Hsieh
(its got all kinds of caveats at the moment and will likely get turn
into
Post by Stack
Post by Stack
Post by Jonathan Hsieh
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various
number
Post by Stack
Post by Stack
Post by Jonathan Hsieh
of client threads (IncrementBlaster). Verify we have a count of 250000
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple runs.
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
Post by Stack
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December 3rd.
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December
3rd.
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits too
fast
Post by Stack
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers
to
Post by Stack
Post by Stack
Post by Jonathan Hsieh
go
Post by Stack
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp key
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always
returns
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew is
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node
time is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
Post by Stack
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't
replicate
Post by Stack
Post by Stack
Post by Jonathan Hsieh
to
Post by Stack
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple
clients
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-10007 PerformanceEvaluation: Add sampling and latency
collection to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even
if
Post by Stack
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an exception
HBASE-10001 Add a coprocessor to help testing the performances
without
Post by Stack
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in HConnectionManager
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit
tests
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in
locateRegion
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on
0.96 by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
Post by Stack
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region
is
Post by Stack
Post by Stack
Post by Jonathan Hsieh
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners
are
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not
spin up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from
0.94
Post by Stack
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a
region is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog file
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a
tool
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9737 Corrupt HFile cause resource leak leading to Region
Server OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max settings
to
Post by Stack
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each
reseek
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another
env
Post by Stack
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from is
disabled
HBASE-9867 Save on array copies with a subclass of
LiteralByteString
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed
to
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
server
HBASE-9593 Region server left in online servers list forever if it
went
Post by Stack
down after registering to master and before creating ephemeral node
HBASE-9862 manage error per server and per region in the
protobuffed
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
client
HBASE-9861 Location does not have to be refreshed on regionTooBusy
HBASE-9659 some integration tests can no longer be run using maven
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
Post by Stack
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in
IntegrationTestBulkLoad
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9848 Make regionserver Canary exit with different exit codes
HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper()
may
Post by Stack
not
be working correctly
HBASE-7525 A canary monitoring program specifically for
regionserver
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
Post by Stack
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with TableNotEnabledException
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from
command
Post by Stack
Post by Stack
Post by Jonathan Hsieh
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
Post by Stack
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be better
to
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
Stack
2013-12-10 00:14:42 UTC
Permalink
Excellent. Starting build. Thanks lads,
St.Ack
Post by Jimmy Xiang
HBASE-10101 is in. Thanks.
Post by Stack
Are we good to go on RC2? Just waiting on HBase-10101? Anything else?
Thanks,
St.Ack
Post by Stack
Jon fixed the Increment issue. Nicolas may have fixed the slow down
issue
Post by Stack
-- we'll know after a few runs this weekend.
I'll put up a new RC Monday regardless.
St.Ack
Post by Stack
Good one Jon.
RC is sunk. Will put up a new one when this is fixed.
Thanks,
St.Ack
Post by Jonathan Hsieh
-1
I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with
flushes is a correctness regression (HBASE-6195 fixed this in 94,
and I
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
had
verified there but it is broken here).. I'm not sure if #3 (kill) or
#4
Post by Stack
Post by Stack
Post by Jonathan Hsieh
(kill -9) are a regression against 0.94. Here's a link to the
testcode
Post by Stack
Post by Stack
Post by Jonathan Hsieh
(its got all kinds of caveats at the moment and will likely get turn
into
Post by Stack
Post by Stack
Post by Jonathan Hsieh
integration test).
https://github.com/jmhsieh/hbase/tree/increval
5 nodes
hadoop 2.1.0 (a pre cdh5b1 hdfs).
hbase 0.96.1 rc1
Test: 250000 increments on a single row an single col with various
number
Post by Stack
Post by Stack
Post by Jonathan Hsieh
of client threads (IncrementBlaster). Verify we have a count of
250000
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
after the run (IncrementVerifier).
Run 1: No fault injection. 5 runs. count = 250000. on multiple
runs.
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Correctness verified. 1638 inc/s throughput.
Run 2: flushes table with incrementing row. count = 246875 !=250000.
correctness failed. 1517 inc/s throughput.
Run 3: kill of rs hosting incremented row. count = 243750 != 250000.
Correctness failed. 1451 inc/s throughput.
Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000.
Correctness failed. 1395 inc/s (including recovery)
Post by Stack
The second hbase-0.96.1 release candidate is available for
http://people.apache.org/~stack/hbase-0.96.1RC1/
https://repository.apache.org/content/repositories/orgapachehbase-016/
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-10053 ProtobufUtil#toMutationNoData interface change causes
downstream
We broke hive building against HBase.
137 issues [1] have been fixed since 0.96.0, mostly performance
improvement
Post by Stack
and bug fixes.
Shall we release this as hbase-0.96.1? Please vote by December
3rd.
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
The first hbase-0.96.1 release candidate is available for
http://people.apache.org/~stack/hbase-0.96.1RC0/
https://repository.apache.org/content/repositories/orgapachehbase-012/
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
127 issues [1] have been fixed since 0.96.0, mostly performance
improvement
and bug fixes. See below for the full list.
Shall we release this as hbase-0.96.1? Please vote by December
3rd.
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
Thanks,
Your HBase Team
1. http://goo.gl/ZgYBW2
HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr)
causes
Post by Stack
OOM
HBASE-10015 Replace intrinsic locking with explicit locks in
StoreScanner
Post by Stack
HBASE-10026 HBaseAdmin#createTable could fail if region splits
too
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
fast
Post by Stack
HBASE-10024 Add an interface to create put with immutable arrays
HBASE-8465 Auto-drop rollback snapshot for snapshot restore
HBASE-10011 Fix some findbugs in the client
HBASE-9865 Reused WALEdits in replication may cause RegionServers
to
Post by Stack
Post by Stack
Post by Jonathan Hsieh
go
Post by Stack
OOM
HBASE-9890 MR jobs are not working if started by a delegated user
HBASE-9975 Not starting ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink.
HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck
via
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
cmdline
-D option
HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore
HBASE-9956 Remove keyLength cache from KeyValue
HBASE-9165 HBASE-8084 Improvements to addDependencyJars
HBASE-8084 Improvements to addDependencyJars
HBASE-9138 getHaseIntegrationTestingUtility() is misspelled
HBASE-9970 HBase BulkLoad, table is creating with the timestamp
key
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
also
Post by Stack
as a column to the table.
HBASE-9952 Snapshot restore may fail due to NullPointerException
HBASE-9906 Restore snapshot fails to restore the meta edits
sporadically
Post by Stack
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"
HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always
returns
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
false
HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram
HBASE-9872 ModifyTable does not modify the attributes of a newly
modified/changed ColumnDescriptor
HBASE-8559 increase unit-test coverage of package
org.apache.hadoop.hbase.coprocessor
HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client
HBASE-9902 Region Server is starting normally even if clock skew
is
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
more
Post by Stack
than default 30 seconds(or any configured). -> Regionserver node
time is
Post by Stack
greater than master node time
HBASE-9849 [REST] Forbidden schema delete in read only mode
HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter
HBASE-9850 Issues with UI for table compact/split operation
completion.
Post by Stack
After split/compaction operation using UI, the page is not
automatically
Post by Stack
redirecting back using IE8/Firefox.
HBASE-4654 [replication] Add a check to make sure we don't
replicate
Post by Stack
Post by Stack
Post by Jonathan Hsieh
to
Post by Stack
ourselves
HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under
hbase-hadoop1-compat
HBASE-9976 Don't create duplicated TableName objects
HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options
are
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
included
HBASE-5945 Reduce buffer copies in IPC server response path
HBASE-9322 Contention on the SoftValueSortedMap with multiple
clients
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-10007 PerformanceEvaluation: Add sampling and latency
collection to
Post by Stack
randomRead test
HBASE-9989 Add a test on get in TestClientNoCluster
HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore
even
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
if
Post by Stack
there is nothing to rollback.
HBASE-10013 Enhance the message about ZK when there is an
exception
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-10001 Add a coprocessor to help testing the performances
without
Post by Stack
taking into account the i/o
HBASE-10012 Hide ServerName constructor
HBASE-9961 [WINDOWS] Multicast should bind to local address
HBASE-9893 Incorrect assert condition in OrderedBytes decoding
HBASE-9998 Fix javadoc warnings induced by commits
HBASE-9959 Remove some array copy - server side
HBASE-9988 DOn't use HRI#getEncodedName in the client
HBASE-9995 Not stopping ReplicationSink when using custom
implementation
Post by Stack
for the ReplicationSink
HBASE-9924 Avoid potential filename conflict in region_mover.rb
HBASE-9982 TestClientNoCluster should use random numbers
HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose
permissions
Post by Stack
after upgrade to 0.96.x from 0.94.x or 0.92.x
HBASE-9987 Remove some synchronisation points in
HConnectionManager
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9985 Add HBase shell command to start/stop tracing
HBASE-9983 Lower the memory footprint of HRegionLocation
HBASE-9869 Optimize HConnectionManager#getCachedLocation
HBASE-9954 Incorporate HTTPS support for HBase
HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit
tests
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9112 Custom TableInputFormat in initTableMapperJob throws
ClassNoFoundException on TableMapper
HBASE-9947 Add CM action for online compression algorithm change
HBASE-9958 Remove some array copy, change lock scope in
locateRegion
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on
0.96 by
Post by Stack
HBASE-9867
HBASE-9810 Global memstore size will be calculated wrongly if
replaying
Post by Stack
recovered edits throws exception
HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client
behaviors
HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong
format
Post by Stack
HBASE-9757 Reenable fast region move in SlowDeterministicMonkey
HBASE-9710 Use the region name, not the encoded name, when region
is
Post by Stack
Post by Stack
Post by Jonathan Hsieh
not
Post by Stack
on current server
HBASE-9926 Scanner doesn't check if a region is available
HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners
are
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
missed after master recovery
HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not
spin up
Post by Stack
max threads even when not needed
HBASE-9895 0.96 Import utility can't import an exported file from
0.94
Post by Stack
HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows
HBASE-9885 Avoid some Result creation in protobuf conversions
HBASE-9563 Autorestart doesn't work if zkcleaner fails
HBASE-9792 Region states should update last assignments when a
region is
Post by Stack
opened.
HBASE-9813 Log splitting doesn't prevent RS creating new hlog
file
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9900 Fix unintended byte[].toString in AccessController
HBASE-9909 TestHFilePerformance should not be a unit test, but a
tool
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9737 Corrupt HFile cause resource leak leading to Region
Server OOM
Post by Stack
HBASE-9751 Excessive readpoints checks in StoreFileScanner
HBASE-9716 LoadTestTool should provide default min and max
settings
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
to
Post by Stack
the
data generator
HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally
HBASE-9807 block encoder unnecessarily copies the key for each
reseek
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9753 Excessive readpoint checks in MemstoreScanner
HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use
another
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
env
Post by Stack
var for prefix
HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky
HBASE-9747 PrefixFilter with OR condition gives wrong results
HBASE-9727 HBase Rest Server - DELETE scanner operation is a
no-op
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9221 Provide interface for getting a User in the client
HBASE-9886 Optimize ServerName#compareTo
HBASE-9837 HBASE-9080 'Retain assignment should be used when
re-enabling
Post by Stack
table(s)'
HBASE-9901 Add a toString in HTable, fix a log in
AssignmentManager
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9855 evictBlocksByHfileName improvement for bucket cache
HBASE-9836 Intermittent
TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
failure
HBASE-9859 Canary Shouldn't go off if the table being read from
is
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
disabled
HBASE-9867 Save on array copies with a subclass of
LiteralByteString
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9868 Remove some array copy, especially around protobuf
HBASE-9871 PB issue : Increment attributes are not getting passed
to
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
server
HBASE-9593 Region server left in online servers list forever if
it
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
went
Post by Stack
down after registering to master and before creating ephemeral
node
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9862 manage error per server and per region in the
protobuffed
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
client
HBASE-9861 Location does not have to be refreshed on
regionTooBusy
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9659 some integration tests can no longer be run using
maven
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a
secure
Post by Stack
enviroment
HBASE-9843 Various fixes in client code
HBASE-9750 Add retries around Action server stop/start
HBASE-9759 Prevent random number collision in
IntegrationTestBulkLoad
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9848 Make regionserver Canary exit with different exit
codes
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9765
TestImportTsv.testBulkOutputWithTsvImporterTextMapper()
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
may
Post by Stack
not
be working correctly
HBASE-7525 A canary monitoring program specifically for
regionserver
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9748 HBASE-9696
HBASE-9827 Intermittent
TestLogRollingNoCluster#testContendedLogRolling
Post by Stack
failure
HBASE-9785 Fix heap size reporting in HRegion
HBASE-9821 Scanner id could collide
HBASE-9694 HBaseTestingUtility.createTable infinite loop
HBASE-9772 Normalize new client default values
HBASE-9768 Two issues in AsyncProcess
HBASE-9784 Switch to Hadoop-2.2
HBASE-9793 Offline a region before it's closed could cause double
assignment
HBASE-9798 Include dependency hamcrest-core
HBASE-9796 npe in RegionServerCallable
HBASE-9776 Test Load And Verify Fails with
TableNotEnabledException
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
HBASE-9788 [WINDOWS] Update rest server class name
HBASE-9755 Cannot run classes in hbase-server tests jar from
command
Post by Stack
Post by Stack
Post by Jonathan Hsieh
line
Post by Stack
HBASE-9791 MR initializes scanner twice
HBASE-9773 Master aborted when hbck asked the master to assign a
region
Post by Stack
that was already online
HBASE-9720
TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit
Post by Stack
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
occasionally times out
HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS
HBASE-9570 With AccessDeniedException, HBase shell would be
better
Post by Stack
to
Post by Stack
Post by Stack
Post by Jonathan Hsieh
Post by Stack
just
display the error message to be user friendly
HBASE-9758 Log missing rows and their regions if ITBLL fails
HBASE-9692 TestOrderedBytes doesn't display the arrays correctly
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
Continue reading on narkive:
Loading...