kylin.git
3 weeks agoKYLIN-5251 fix kylin run on hadoop3 main
Mukvin [Fri, 2 Sep 2022 08:39:40 +0000 (16:39 +0800)] 
KYLIN-5251 fix kylin run on hadoop3

3 weeks agoKYLIN-5250 work round for no hack aggregation group
Mukvin [Fri, 2 Sep 2022 06:58:01 +0000 (14:58 +0800)] 
KYLIN-5250 work round for no hack aggregation group

3 weeks agoKYLIN-5250 work round for no hack aggregation group
Mukvin [Fri, 8 Jul 2022 07:48:18 +0000 (15:48 +0800)] 
KYLIN-5250 work round for no hack aggregation group

4 weeks ago[maven-release-plugin] prepare for next development iteration
hit-lacus [Sun, 28 Aug 2022 08:07:48 +0000 (08:07 +0000)] 
[maven-release-plugin] prepare for next development iteration

4 weeks ago[maven-release-plugin] prepare release kylin-4.0.2 kylin-4.0.2
hit-lacus [Sun, 28 Aug 2022 08:07:34 +0000 (08:07 +0000)] 
[maven-release-plugin] prepare release kylin-4.0.2

4 weeks agoFix check-env.sh v4.0.2-release
XiaoxiangYu [Fri, 26 Aug 2022 05:40:32 +0000 (13:40 +0800)] 
Fix check-env.sh

4 weeks agominor, cleanup useless code
Shaofeng Shi [Thu, 11 Aug 2022 10:11:04 +0000 (18:11 +0800)] 
minor, cleanup useless code

4 weeks agoKYLIN-5225 add pre-commit property to spark3 profile
Shaofeng Shi [Thu, 11 Aug 2022 02:29:10 +0000 (10:29 +0800)] 
KYLIN-5225 add pre-commit property to spark3 profile

4 weeks agofix: docker/dockerfile/cluster/base/Dockerfile to reduce vulnerabilities
snyk-bot [Thu, 11 Aug 2022 01:13:05 +0000 (01:13 +0000)] 
fix: docker/dockerfile/cluster/base/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-CENTOS7-NSS-2078613
- https://snyk.io/vuln/SNYK-CENTOS7-NSS-2308137
- https://snyk.io/vuln/SNYK-CENTOS7-NSSSYSINIT-2080438
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2308139
- https://snyk.io/vuln/SNYK-CENTOS7-NSSUTIL-2078546

fix: docker/dockerfile/cluster/client/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-CENTOS7-NSS-2078613
- https://snyk.io/vuln/SNYK-CENTOS7-NSS-2308137
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2078658
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2308139
- https://snyk.io/vuln/SNYK-CENTOS7-NSSUTIL-2078546

fix: docker/dockerfile/cluster/hbase/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-CENTOS7-NSSSYSINIT-2080438
- https://snyk.io/vuln/SNYK-CENTOS7-NSSSYSINIT-2308141
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2078658
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2308139
- https://snyk.io/vuln/SNYK-CENTOS7-NSSUTIL-2078546

fix: docker/dockerfile/cluster/kerberos/Dockerfile to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-CENTOS7-NSSSYSINIT-2080438
- https://snyk.io/vuln/SNYK-CENTOS7-NSSSYSINIT-2308141
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2078658
- https://snyk.io/vuln/SNYK-CENTOS7-NSSTOOLS-2308139
- https://snyk.io/vuln/SNYK-CENTOS7-NSSUTIL-2078546

4 weeks agoKYLIN-5225 update haddop version and make spark3 as the default profile
Shaofeng Shi [Wed, 10 Aug 2022 03:31:41 +0000 (11:31 +0800)] 
KYLIN-5225 update haddop version and make spark3 as the default profile

6 weeks agoKYLIN-5225 update download-spark.sh to spark 3.1.3
Shaofeng Shi [Wed, 10 Aug 2022 10:14:13 +0000 (18:14 +0800)] 
KYLIN-5225 update download-spark.sh to spark 3.1.3

6 weeks agoKYLIN-5214 upgrade spring-core and spring-security
dependabot[bot] [Wed, 25 May 2022 23:11:31 +0000 (23:11 +0000)] 
KYLIN-5214 upgrade spring-core and spring-security

3 months agoKylin 5190, fix kylin.sh error when directory starts with s3a:// (#1895)
WANG Hui [Thu, 23 Jun 2022 02:45:58 +0000 (10:45 +0800)] 
Kylin 5190, fix kylin.sh error when directory starts with s3a:// (#1895)

* KYLIN-5190, fix kylin.sh error when directory starts with s3a://, replace s3a:// with s3://

3 months agoKylin 5195, update spark from 3.1.1 to 3.1.3 (#1894)
WANG Hui [Thu, 23 Jun 2022 02:43:08 +0000 (10:43 +0800)] 
Kylin 5195, update spark from 3.1.1 to 3.1.3 (#1894)

* update spark from 3.1.1 to 3.1.3 in kylin4

3 months agoMinor, using default log4j properties for spark driver and exectutor (#1891)
Tengting Xu [Tue, 21 Jun 2022 09:50:36 +0000 (17:50 +0800)] 
Minor, using default log4j properties for spark driver and exectutor (#1891)

* using default log4j properties for spark driver and executor

* Incompatible to hadoop 2.x env

Revert "KYLIN-3610 bump the version of curator and zookeeper"

This reverts commit 8cde6dda

* minor, adapt default properties

3 months agoKYLIN-5187, Problem fix,
WANGHui2022 [Tue, 21 Jun 2022 06:23:37 +0000 (14:23 +0800)] 
KYLIN-5187, Problem fix,
    1. Fix exception to get more detailed message for other FileSystem
    2. Fix de.thetaphi:forbiddenapis:2.3:check for forbidden api, add default charset UTF-8

3 months agoKYLIN-5187 Support soft affinity and local cache feature
Zhichao Zhang [Thu, 16 Sep 2021 01:03:53 +0000 (09:03 +0800)] 
KYLIN-5187 Support soft affinity and local cache feature

    1. Implement LocalDataCacheManager
    2. base xiaoxiang's PR
    3. Implement CacheFileScanRDD
    4. Implement AbstractCacheFileSystem
    5. Optimize performance
    6. Support soft affinity for hdfs
    7. Support ByteBuffer to read data, and avoid to read data one byte by one byte
    8. Support to cache small files in memory : ByteBufferPageStore extends PageStore to support cache data in memory
    9. Pre-init KylinCacheFileSystem to fix s3a issue
   10. Upgrade alluxio client verion to 2.7.4

3 months agominor, fix sonar reported bugs (#1875)
Shaofeng Shi [Thu, 16 Jun 2022 03:03:48 +0000 (11:03 +0800)] 
minor, fix sonar reported bugs (#1875)

* minor, fix sonar reported bugs

* m

3 months agoKYLIN-5184, fix cuboid statistic feature (#1877)
Tengting Xu [Thu, 16 Jun 2022 02:58:50 +0000 (10:58 +0800)] 
KYLIN-5184, fix cuboid statistic feature (#1877)

* KYLIN-5184, fix cuboid statistic feature

* minor, adapt for cuboids without base cuboid

* minor, adapt for cuboids with original logic

* minor, adapt for cuboids with original logic

* minor, adapt for cuboids with original logic

3 months agoKYLIN-5191 fix Can not see executors on spardar page on AWS EMR 6
WANGHui2022 [Wed, 15 Jun 2022 13:18:24 +0000 (21:18 +0800)] 
KYLIN-5191 fix Can not see executors on spardar page on AWS EMR 6

3 months agoKYLIN-5188, fix email notification after cubing and merging job
Mukvin [Wed, 1 Jun 2022 04:01:08 +0000 (12:01 +0800)] 
KYLIN-5188, fix email notification after cubing and merging job

3 months agoKylin 5193 fix login page redirection (#1887)
Tengting Xu [Thu, 16 Jun 2022 02:02:20 +0000 (10:02 +0800)] 
Kylin 5193 fix login page redirection (#1887)

* KYLIN-5193, fix login page redirection

* minor, add refresh cube list button then not need to refresh whole page to update cube list

3 months agoKYLIN-5194, fix deleted user can login when auth cache is valid
Mukvin [Wed, 15 Jun 2022 09:37:55 +0000 (17:37 +0800)] 
KYLIN-5194, fix deleted user can login when auth cache is valid

3 months agoKYLIN-3610 bump the version of curator and zookeeper
mango [Fri, 10 Jun 2022 10:35:18 +0000 (18:35 +0800)] 
KYLIN-3610 bump the version of curator and zookeeper

* chore: bump the version of curator to 5.2.1

Signed-off-by: mango <xu.weiKyrie@foxmail.com>
* bump the version of zookeeper from 3.4.13 to 3.6.3

Signed-off-by: mango <xu.weiKyrie@foxmail.com>
4 months agoKYLIN-5185, fix build docker with mysql
Mukvin [Fri, 20 May 2022 06:12:08 +0000 (14:12 +0800)] 
KYLIN-5185, fix build docker with mysql

4 months agoKYLIN-5183 Customize or get query id for devops
yhj [Sat, 14 May 2022 08:24:25 +0000 (16:24 +0800)] 
KYLIN-5183 Customize or get query id for devops

4 months agominor, change hive.client from `cli` to `spark_catalog`
Mukvin [Fri, 13 May 2022 08:42:30 +0000 (16:42 +0800)] 
minor, change hive.client from `cli` to `spark_catalog`

4 months agoKYLIN-5182, support to ignore only one null count in check duplicate key
Mukvin [Fri, 6 May 2022 10:37:29 +0000 (18:37 +0800)] 
KYLIN-5182, support to ignore only one null count in check duplicate key

4 months agominor, update the development url in README.md
Shaofeng Shi [Sun, 15 May 2022 09:07:45 +0000 (17:07 +0800)] 
minor, update the development url in README.md

4 months agominor, remove sonarcloud link as it stops java8 support
Shaofeng Shi [Sun, 15 May 2022 04:09:17 +0000 (12:09 +0800)] 
minor, remove sonarcloud link as it stops java8 support

4 months agominor, update pull request template
Shaofeng Shi [Sun, 15 May 2022 02:20:34 +0000 (10:20 +0800)] 
minor, update pull request template

4 months agoremove useless codes
Mukvin [Sat, 7 May 2022 08:54:14 +0000 (16:54 +0800)] 
remove useless codes

4 months agominor, fix invalid config
Mukvin [Fri, 13 May 2022 07:38:17 +0000 (15:38 +0800)] 
minor, fix invalid config

4 months agoKYLIN-5111 Refine query metrics
XiaoxiangYu [Wed, 20 Apr 2022 00:52:11 +0000 (08:52 +0800)] 
KYLIN-5111 Refine query metrics

4 months agoAdd sample jobv2 (#1858)
Tengting Xu [Thu, 5 May 2022 07:44:55 +0000 (15:44 +0800)] 
Add sample jobv2 (#1858)

* add sample job

* # adapt for sampling job

* # minor fix

* # minor fix

* minor,adapt for sampling job in frontend

* # minor fix for sampling job

* minor,adapt for sampling table info in frontend

* minor,fix job list related object

* minor,change job request cubename to objectname

Co-authored-by: fanfanAlice <18611532909@163.com>
Co-authored-by: huangchunyan <qingyanxiaonian@163.com>
4 months agoKYLIN-5179, fix StorageCleanupJob remove invalid segement
Mukvin [Wed, 27 Apr 2022 12:07:29 +0000 (20:07 +0800)] 
KYLIN-5179, fix StorageCleanupJob remove invalid segement

4 months agoKYLIN-5178, upgrade tomcat to 8.5.78 (#1863)
Tengting Xu [Fri, 29 Apr 2022 02:31:51 +0000 (10:31 +0800)] 
KYLIN-5178, upgrade tomcat to 8.5.78 (#1863)

* KYLIN-5178, upgrade tomcat to 8.5.78

* minor fix, comment the JasperListener which already removed from tomcat 8.0

5 months ago[KYLIN-5159] upgrade some common libs due to CVEs (#1814)
PJ Fanning [Wed, 6 Apr 2022 09:23:57 +0000 (11:23 +0200)] 
[KYLIN-5159] upgrade some common libs due to CVEs (#1814)

* [KYLIN-5159] upgrade some common libs due to CVEs

* upgrade xerces and jetty

6 months agoKYLIN-5121 fix key not found: numOutputRows on s3
yaqian.zhang [Thu, 10 Mar 2022 02:03:08 +0000 (10:03 +0800)] 
KYLIN-5121 fix key not found: numOutputRows on s3

6 months agoKYLIN-5160 Fix driver extra classpath
XiaoxiangYu [Tue, 15 Feb 2022 10:06:27 +0000 (18:06 +0800)] 
KYLIN-5160 Fix driver extra classpath

6 months agoKYLIN-5166 Use SQL hint to specify the cube that the user wants to query
yaqian.zhang [Mon, 7 Mar 2022 03:21:43 +0000 (11:21 +0800)] 
KYLIN-5166 Use SQL hint to specify the cube that the user wants to query

6 months agoadd api to support for mdx
Mukvin [Sat, 29 Jan 2022 10:12:55 +0000 (18:12 +0800)] 
add api to support for mdx

7 months agoKYLIN-5162 Fix non select sql pushdown
yaqian.zhang [Thu, 24 Feb 2022 07:00:14 +0000 (15:00 +0800)] 
KYLIN-5162 Fix non select sql pushdown

7 months agofix
yaqian.zhang [Tue, 8 Feb 2022 03:27:34 +0000 (11:27 +0800)] 
fix

7 months agoAttach username and password to README.md
Yao Zhou [Wed, 26 Jan 2022 12:11:01 +0000 (20:11 +0800)] 
Attach  username and password to README.md

8 months agoadd license description
hujiahua [Sat, 13 Nov 2021 01:20:08 +0000 (09:20 +0800)] 
add license description

8 months ago[KYLIN-5121] Make JobMetricsUtils.collectMetrics be working again
hujiahua [Thu, 11 Nov 2021 06:39:03 +0000 (14:39 +0800)] 
[KYLIN-5121] Make JobMetricsUtils.collectMetrics be working again

8 months ago[KYLIN-5125] fix up unit test
hujiahua [Mon, 15 Nov 2021 09:55:05 +0000 (17:55 +0800)] 
[KYLIN-5125] fix up unit test

8 months ago[KYLIN-5125] `MetadataCleanupJob.isJobComplete` should use `NSparkCubingJob` because...
hujiahua [Mon, 15 Nov 2021 08:41:25 +0000 (16:41 +0800)] 
[KYLIN-5125] `MetadataCleanupJob.isJobComplete` should use `NSparkCubingJob` because kylin4 use spark engine to build cube (No more MR engine)

8 months ago[KYLIN-5152] support boolean expression on where clause
hujiahua [Mon, 10 Jan 2022 12:22:18 +0000 (20:22 +0800)] 
[KYLIN-5152] support boolean expression on where clause

8 months ago[maven-release-plugin] prepare for next development iteration
nichunen [Wed, 29 Dec 2021 10:35:31 +0000 (10:35 +0000)] 
[maven-release-plugin] prepare for next development iteration

8 months ago[maven-release-plugin] prepare release kylin-4.0.1
nichunen [Wed, 29 Dec 2021 10:33:00 +0000 (10:33 +0000)] 
[maven-release-plugin] prepare release kylin-4.0.1

8 months agoModify shaded-guava version for release kylin4.0.1
nichunen [Mon, 27 Dec 2021 06:12:48 +0000 (06:12 +0000)] 
Modify shaded-guava version for release kylin4.0.1

9 months agotest fix
yaqian.zhang [Tue, 7 Dec 2021 08:59:42 +0000 (16:59 +0800)] 
test fix

9 months agominor fix for KYLIN-5071
Mukvin [Thu, 2 Dec 2021 09:04:55 +0000 (17:04 +0800)] 
minor fix for KYLIN-5071

10 months agoKYLIN-5069 Fix can't get location (#1776)
Yaqian Zhang [Fri, 19 Nov 2021 12:38:53 +0000 (20:38 +0800)] 
KYLIN-5069 Fix can't get location (#1776)

10 months agoKYLIN-5069 Fix wrong file name (#1775)
Yaqian Zhang [Fri, 19 Nov 2021 07:05:03 +0000 (15:05 +0800)] 
KYLIN-5069 Fix wrong file name (#1775)

10 months agoKYLIN-5069 Fix the error of key not found
yaqian.zhang [Thu, 18 Nov 2021 05:50:40 +0000 (13:50 +0800)] 
KYLIN-5069 Fix the error of key not found

10 months agoKYLIN-5069 Fix
yaqian.zhang [Wed, 17 Nov 2021 03:16:00 +0000 (11:16 +0800)] 
KYLIN-5069 Fix

10 months agoAvoid redundant error information
yaqian.zhang [Mon, 15 Nov 2021 08:17:37 +0000 (16:17 +0800)] 
Avoid redundant error information

10 months agoKYLIN-5099 Decrease the default value of 'kylin.cube.cubeplanner.expansion-threshold'
yaqian.zhang [Mon, 15 Nov 2021 03:02:09 +0000 (11:02 +0800)] 
KYLIN-5099 Decrease the default value of 'kylin.cube.cubeplanner.expansion-threshold'

10 months agofix
yaqian.zhang [Wed, 10 Nov 2021 02:21:04 +0000 (10:21 +0800)] 
fix

10 months agoKYLIN-5090 The error message was not printed in class JobWoker
yaqian.zhang [Tue, 9 Nov 2021 09:38:21 +0000 (17:38 +0800)] 
KYLIN-5090 The error message was not printed in class JobWoker

10 months agoKYLIN-5071 Still estimate the rowcount of baseCuboid when baseCuboid is not built
yaqian.zhang [Tue, 9 Nov 2021 11:10:45 +0000 (19:10 +0800)] 
KYLIN-5071 Still estimate the rowcount of baseCuboid when baseCuboid is not built

10 months agoKYLIN-5906 fix cdh6
yaqian.zhang [Mon, 8 Nov 2021 05:58:57 +0000 (13:58 +0800)] 
KYLIN-5906 fix cdh6

10 months agoKYLIN-5117 Support percentile function after aggregate sub query (#1760)
MattHu [Mon, 8 Nov 2021 07:17:57 +0000 (15:17 +0800)] 
KYLIN-5117 Support percentile function after aggregate sub query (#1760)

* Support percentile function after aggregate sub query

* add test

10 months agoKYLIN-4864 Fix .travis.yml syntax (#1700)
Martin Grigorov [Mon, 2 Aug 2021 08:47:48 +0000 (16:47 +0800)] 
KYLIN-4864 Fix .travis.yml syntax (#1700)

* KYLIN-4864 Fix .travis.yml syntax

The commented line about `# mvn sonar:sonar -e -Dsonar.host.url=https://sonarcloud.io ...` has broken the YAML and because of this TravisCI didn't execute any jobs.
I wasn't able to find a way to preserve the commented line and fix the syntax, so I removed the line. Later it could be get from Git history to re-enable Sonar.

While here I've improved few more things:
- install OpenJDK and unzip for both AMD64 and ARM64. This way it uses latest JDK 8 (1.8.0_292) for both CPU architectures
- fix more YAML warnings

* KYLIN-4864 Use Graviton2 ARM64 node

There is some problem with resolving the local address on the arm64 node (non-graviton2) - https://travis-ci.community/t/sudo-unable-to-resolve-host/8841

10 months agoModify branch name for travis ci
yaqian.zhang [Thu, 4 Nov 2021 01:58:53 +0000 (09:58 +0800)] 
Modify branch name for travis ci

10 months ago[KYLIN-4864] Enable running Kylin tests on ARM64 platform (#1558)
Liu sheng [Tue, 27 Jul 2021 02:38:10 +0000 (10:38 +0800)] 
[KYLIN-4864] Enable running Kylin tests on ARM64 platform (#1558)

* [KYLIN-4864] Enable running Kylin tests on ARM64 platform

This change modify the Travis CI configurations to enable the Kylin testing on ARM64 server.

* Use openjdk11 for ARM64 test

10 months agoKYLIN-5111 Record the time spent for each stage of query in kylin4's log
yaqian.zhang [Mon, 1 Nov 2021 02:55:37 +0000 (10:55 +0800)] 
KYLIN-5111 Record the time spent for each stage of query in kylin4's log

10 months agoKYLIN-5112, Use 'return' instead of 'exit' in script prepare_hadoop_dependency.sh
Mukvin [Tue, 2 Nov 2021 02:33:14 +0000 (10:33 +0800)] 
KYLIN-5112, Use 'return' instead of 'exit' in script prepare_hadoop_dependency.sh

10 months agoKYLIN-5069 Refactor hive and hadoop dependency of kylin4
yaqian.zhang [Thu, 14 Oct 2021 03:37:41 +0000 (11:37 +0800)] 
KYLIN-5069 Refactor hive and hadoop dependency of kylin4

12 months agoKYLIN-5067 Skip snapshot build when dimension table's kind is LOOKUP
XiaoxiangYu [Sat, 18 Sep 2021 07:02:10 +0000 (15:02 +0800)] 
KYLIN-5067 Skip snapshot build when dimension table's kind is LOOKUP

12 months agoKYLIN-5086, fix query pushdown with agg func without alias get an empty appearance
Mukvin [Thu, 9 Sep 2021 02:33:23 +0000 (10:33 +0800)] 
KYLIN-5086, fix query pushdown with agg func without alias get an empty appearance

12 months agoKYLIN-5076 Fix spark tracking url missing on spark standalone mode
yaqian.zhang [Fri, 10 Sep 2021 10:19:18 +0000 (18:19 +0800)] 
KYLIN-5076 Fix spark tracking url missing on spark standalone mode

12 months agoKYLIN-5082,exactly match for percentile
leocoder [Sat, 4 Sep 2021 02:19:03 +0000 (10:19 +0800)] 
KYLIN-5082,exactly match for percentile

12 months agoKYLIN-5083, check hdfs service usability for kylin4 on docker
Mukvin [Wed, 8 Sep 2021 07:27:42 +0000 (15:27 +0800)] 
KYLIN-5083, check hdfs service usability for kylin4 on docker

12 months agoKYLIN-5051, minor fix and add quick start about docker in README.md
Mukvin [Thu, 2 Sep 2021 03:56:10 +0000 (11:56 +0800)] 
KYLIN-5051, minor fix and add quick start about docker in README.md

12 months agoKYLIN-5051, create and test docker image for kylin 4.0
Mukvin [Wed, 1 Sep 2021 09:11:49 +0000 (17:11 +0800)] 
KYLIN-5051, create and test docker image for kylin 4.0

13 months agoAdd option to remove stale files under job_tmp for StorageCleanupJob (#1732)
Xiaoxiang Yu [Wed, 18 Aug 2021 04:52:13 +0000 (12:52 +0800)] 
Add option to remove stale files under job_tmp for StorageCleanupJob (#1732)

* Add option to remove stale files under job_tmp for StorageCleanupJob

* fix code style

* fix cleanupJobTmp

* fix sample.sh

* fix kylin.engine.build-base-cuboid-enabled conflict with cube planner

* minor fix

* Fix optimize job

* fix typo

Co-authored-by: yaqian.zhang <598593183@qq.com>
13 months agoKYLIN-5027 Fix query for the cube dose not build basecuboid
yaqian.zhang [Mon, 16 Aug 2021 08:54:37 +0000 (16:54 +0800)] 
KYLIN-5027 Fix query for the cube dose not build basecuboid

13 months agoRefine Kylin Config (#1726)
Xiaoxiang Yu [Mon, 16 Aug 2021 00:40:58 +0000 (08:40 +0800)] 
Refine Kylin Config (#1726)

* Add deprecated notice for some maven modules

* remove duplicated log

* refine kylin config

* refine sample cube desc

* Fix config entry rename

* add switch for sparder canary tool

13 months agoKYLIN-5011 HotFix for scatter skew data in dict-encoding step
yaqian.zhang [Thu, 12 Aug 2021 08:36:53 +0000 (16:36 +0800)] 
KYLIN-5011 HotFix for scatter skew data in dict-encoding step

13 months agoKYLIN-5057 CubeBuildJob in Kylin4.0 run failed when open Spark3.1 AQE
Zhichao Zhang [Wed, 11 Aug 2021 14:28:28 +0000 (22:28 +0800)] 
KYLIN-5057 CubeBuildJob in Kylin4.0 run failed when open Spark3.1 AQE

Root cause:
With KylinJoinSelection strategy, if AQE is true, this strategy will be applied before LogicalQueryStageStrategy,
and then it will be applied JoinSelection strategy again, leading to change the 'Build Type' of BroadcastHashJoin in some cases.

13 months agoJob duration may become negative after the job resumed from error or paused status
yaqian.zhang [Wed, 11 Aug 2021 09:16:16 +0000 (17:16 +0800)] 
Job duration may become negative after the job resumed from error or paused status

13 months agocommit
Zhichao Zhang [Wed, 11 Aug 2021 04:28:12 +0000 (12:28 +0800)] 
commit

13 months agoKYLIN-4903 cache parent datasource to accelerate next layer's cuboid building
Zhichao Zhang [Wed, 11 Aug 2021 04:04:00 +0000 (12:04 +0800)] 
KYLIN-4903 cache parent datasource to accelerate next layer's cuboid building

Fix bug

13 months agoRefine PR template
XiaoxiangYu [Wed, 11 Aug 2021 02:59:41 +0000 (10:59 +0800)] 
Refine PR template

13 months agoKYLIN-5059 Fix error when using different HDFS cluster in cube building (#1721)
zhengshengjun [Wed, 11 Aug 2021 01:53:15 +0000 (09:53 +0800)] 
KYLIN-5059 Fix error when using different HDFS cluster in cube building (#1721)

13 months agoKYLIN-5019 Avoid building global dictionary from all data of fact table every time...
zhengshengjun [Wed, 11 Aug 2021 01:53:06 +0000 (09:53 +0800)] 
KYLIN-5019 Avoid building global dictionary from all data of fact table every time (#1720)

13 months agoKYLIN-4903 cache parent datasource to accelerate next layer's cuboid building (#1722)
zhengshengjun [Wed, 11 Aug 2021 01:52:51 +0000 (09:52 +0800)] 
KYLIN-4903 cache parent datasource to accelerate next layer's cuboid building (#1722)

13 months agoKYLIN-5011 Detect and scatter skewed data in dict encoding step
zhengshengjun [Tue, 10 Aug 2021 09:11:33 +0000 (17:11 +0800)] 
KYLIN-5011 Detect and scatter skewed data in dict encoding step

13 months agoHOTFIX fix slf4j jar version conflict
Zhichao Zhang [Tue, 10 Aug 2021 03:57:59 +0000 (11:57 +0800)] 
HOTFIX fix slf4j jar version conflict

13 months agoHOTFIX for spark3
Zhichao Zhang [Sat, 7 Aug 2021 08:51:13 +0000 (16:51 +0800)] 
HOTFIX for spark3

13 months agoKYLIN-4766 Cleanup segment storage after job be discarded (#1716)
Yaqian Zhang [Sat, 7 Aug 2021 05:20:55 +0000 (13:20 +0800)] 
KYLIN-4766 Cleanup segment storage after job be discarded (#1716)

* KYLIN-4766 Cleanup segment storage after job be discarded

* fix

13 months agoKYLIN-5052 Clean up unused and out-of-date source code/config (#1702)
Xiaoxiang Yu [Mon, 2 Aug 2021 12:05:56 +0000 (20:05 +0800)] 
KYLIN-5052 Clean up unused and out-of-date source code/config (#1702)

* KYLIN-5052 Clean up unused and out-of-date source code/config

- remove unused maven module, include core-dictionary and datasource-sdk
- remove out-of-date UTs
- all source code are not deleted but commented, should deleted them later

* KYLIN-5052 refine unsupported Rest API

* Fix code style issue

13 months agoKYLIN-5027 Fix query result when the cube does not build baseCuboid
yaqian.zhang [Wed, 28 Jul 2021 09:33:57 +0000 (17:33 +0800)] 
KYLIN-5027 Fix query result when the cube does not build baseCuboid

14 months agoFix IT (#1699)
Yaqian Zhang [Tue, 27 Jul 2021 09:18:04 +0000 (17:18 +0800)] 
Fix IT (#1699)

* Fix IT

* fix

* fix

* fix exactlyMatchCuboidMultiSegmentTest()

14 months agoKYLIN-5040 Add rule for exactly match cuboid
yaqian.zhang [Wed, 21 Jul 2021 02:48:51 +0000 (10:48 +0800)] 
KYLIN-5040 Add rule for exactly match cuboid

14 months agoKYLIN-5023 Some fix for spark standalone
yaqian.zhang [Tue, 13 Jul 2021 06:26:30 +0000 (14:26 +0800)] 
KYLIN-5023 Some fix for spark standalone