[SPARK-23831][SQL] Add org.apache.derby to IsolatedClientLoader
authorYuming Wang <yumwang@ebay.com>
Fri, 13 Jul 2018 21:07:52 +0000 (14:07 -0700)
committerXiao Li <gatorsmile@gmail.com>
Fri, 13 Jul 2018 21:07:52 +0000 (14:07 -0700)
## What changes were proposed in this pull request?

Add `org.apache.derby` to `IsolatedClientLoader`, otherwise it may throw an exception:
```scala
...
[info] Cause: java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$12439ab23, see the next exception for details.
[info] at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
[info] at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
[info] at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
[info] at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
[info] at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
[info] at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
...
```

## How was this patch tested?

unit tests and manual tests

Author: Yuming Wang <yumwang@ebay.com>

Closes #20944 from wangyum/SPARK-23831.

sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala

index 2f34f69..6a90c44 100644 (file)
@@ -182,6 +182,7 @@ private[hive] class IsolatedClientLoader(
     name.startsWith("org.slf4j") ||
     name.startsWith("org.apache.log4j") || // log4j1.x
     name.startsWith("org.apache.logging.log4j") || // log4j2
+    name.startsWith("org.apache.derby.") ||
     name.startsWith("org.apache.spark.") ||
     (sharesHadoopClasses && isHadoopClass) ||
     name.startsWith("scala.") ||
index 0a522b6..1de258f 100644 (file)
@@ -113,4 +113,10 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
     catalog.createDatabase(newDb("dbWithNullDesc").copy(description = null), ignoreIfExists = false)
     assert(catalog.getDatabase("dbWithNullDesc").description == "")
   }
+
+  test("SPARK-23831: Add org.apache.derby to IsolatedClientLoader") {
+    val client1 = HiveUtils.newClientForMetadata(new SparkConf, new Configuration)
+    val client2 = HiveUtils.newClientForMetadata(new SparkConf, new Configuration)
+    assert(!client1.equals(client2))
+  }
 }