nifi.git
9 months agoNIFI-4841 Fixing NPE when reverting local changes involving remote group ports. This...
Bryan Bende [Tue, 6 Feb 2018 22:43:59 +0000 (17:43 -0500)] 
NIFI-4841 Fixing NPE when reverting local changes involving remote group ports. This closes #2454.

Signed-off-by: Mark Payne <markap14@hotmail.com>
9 months agoNIFI-4853 - Fixed PutMongoRecord handling of nested records
Matthew Burgess [Wed, 7 Feb 2018 23:15:35 +0000 (18:15 -0500)] 
NIFI-4853 - Fixed PutMongoRecord handling of nested records

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2457.

9 months agoNIFI-4830: Fixed logic errors in BLOB/CLOB processing in JdbcCommon
Matthew Burgess [Thu, 8 Feb 2018 19:02:58 +0000 (14:02 -0500)] 
NIFI-4830: Fixed logic errors in BLOB/CLOB processing in JdbcCommon

This closes #2459.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
9 months agoNIFI-4858: Expose Request Timeout from HTTP Context Map and use that to set as the...
Mark Payne [Thu, 8 Feb 2018 20:02:06 +0000 (15:02 -0500)] 
NIFI-4858: Expose Request Timeout from HTTP Context Map and use that to set as the timeout of the 'AsyncContext' in HandleHttpRequest. Otherwise, the request will never timeout (which is OK because the HttpContextMap will handle this). However, Jetty behind the scenes is adding a task to Scheduled Executor for each request with a delay of whatever the timeout is set to. Since it's currently set to Long.MAX_VALUE, that task will never be run and as a result the ExecutorService's queue will grow indefinitely, eventually exhausting the JVM Heap

This closes #2460.

Signed-off-by: Bryan Bende <bbende@apache.org>
9 months agoNIFI-4699 Use a filter in PostHTTP to pull flowfiles from queue whose URL is the...
Mike Moser [Tue, 9 Jan 2018 15:56:54 +0000 (15:56 +0000)] 
NIFI-4699 Use a filter in PostHTTP to pull flowfiles from queue whose URL is the same

This closes #2412.

9 months agoNIFI-4080: Restored customValidate in ValidateCSV for schema property with no EL
Matthew Burgess [Tue, 6 Feb 2018 16:47:33 +0000 (11:47 -0500)] 
NIFI-4080: Restored customValidate in ValidateCSV for schema property with no EL

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2226.

9 months agoenable EL on FF level
Marco Gaido [Mon, 29 Jan 2018 15:20:30 +0000 (16:20 +0100)] 
enable EL on FF level

9 months agoNIFI-4080: Added EL support to fields in ValidateCSV
Matthew Burgess [Thu, 26 Oct 2017 12:27:27 +0000 (08:27 -0400)] 
NIFI-4080: Added EL support to fields in ValidateCSV

9 months agoNIFI-4846: AvroTypeUtil to support more input types for logical decimal conversion
Koji Kawamura [Tue, 6 Feb 2018 07:36:15 +0000 (16:36 +0900)] 
NIFI-4846: AvroTypeUtil to support more input types for logical decimal conversion

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2451

9 months agoNIFI-4837:
Matt Gilman [Wed, 7 Feb 2018 18:52:02 +0000 (13:52 -0500)] 
NIFI-4837:
- When Jetty initializes fails, performing a shutdown sequence to ensure all allocated resources are released.

This closes #2455.

Signed-off-by: Bryan Bende <bbende@apache.org>
9 months agoNIFI-4828: Fix MergeContent to process all ready bins
Koji Kawamura [Tue, 30 Jan 2018 06:21:00 +0000 (15:21 +0900)] 
NIFI-4828: Fix MergeContent to process all ready bins

Before this fix, MergeContent only processed the first bin even if there
were multiple bins.

There were two unit tests marked with Ignore those had been
failing because of this.

This closes #2444.

Signed-off-by: Mark Payne <markap14@hotmail.com>
9 months agoFix for unit tests that are causing build failures in certain environments
Mark Payne [Wed, 7 Feb 2018 19:13:06 +0000 (14:13 -0500)] 
Fix for unit tests that are causing build failures in certain environments

9 months agoNIFI-4844: Adjust BigDecimal scale to the target Avro schema
Koji Kawamura [Tue, 6 Feb 2018 04:52:48 +0000 (13:52 +0900)] 
NIFI-4844: Adjust BigDecimal scale to the target Avro schema

- Applied the same scale adjustment not only to BigDecimal inputs, but
also to Double values.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2450

9 months agoNIFI-4848: Update HttpComponents version
Koji Kawamura [Wed, 7 Feb 2018 06:42:54 +0000 (15:42 +0900)] 
NIFI-4848: Update HttpComponents version

- httpclient 4.5.3 -> 4.5.5
- httpcore 4.4.4 -> 4.4.9
  - ThreadSafe annotation is removed since 4.4.5, HTTPCLIENT-1743.
    Removed the annotation from DebugFlow processor.
- httpasyncclient 4.1.2 -> 4.1.3
- This closes #2453

9 months agoNIFI-4818: Fix transit URL parsing at Hive2JDBC and KafkaTopic for ReportLineageToAtlas
Koji Kawamura [Thu, 25 Jan 2018 04:57:01 +0000 (13:57 +0900)] 
NIFI-4818: Fix transit URL parsing at Hive2JDBC and KafkaTopic for ReportLineageToAtlas

- Hive2JDBC: Handle connection parameters and multiple host entries
correctly
- KafkaTopic: Handle multiple host entries correctly
- Avoid potential "IllegalStateException: Duplicate key" exception
when NiFiAtlasHook analyzes existing NiFiFlowPath input/output entries
- This closes #2435

9 months agoNIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConnectionFactory...
Mark Payne [Wed, 31 Jan 2018 16:50:42 +0000 (11:50 -0500)] 
NIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConnectionFactory per concurrent task instead of sharing one across the entire processor.

This closes #2445.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
9 months agoNIFI-4395 GenerateTableFetch can't fetch column type by state after instance reboot
Deon Huang [Mon, 29 Jan 2018 15:29:21 +0000 (23:29 +0800)] 
NIFI-4395 GenerateTableFetch can't fetch column type by state after instance reboot

Add multiple states recover

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2442

9 months agoNIFI-4826 - Fixed azure.blobname in ListAzureBlobStorage
zenfenan [Mon, 29 Jan 2018 09:52:32 +0000 (15:22 +0530)] 
NIFI-4826 - Fixed azure.blobname in ListAzureBlobStorage

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2440.

9 months agoNIFI-4410: Improved error handling/logging in PutElasticsearchHttp processors
Matthew Burgess [Tue, 26 Sep 2017 02:25:29 +0000 (22:25 -0400)] 
NIFI-4410: Improved error handling/logging in PutElasticsearchHttp processors

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2175.

9 months agoCorrected Docker tag back to standard format
dchaffey [Sun, 28 Jan 2018 18:23:58 +0000 (18:23 +0000)] 
Corrected Docker tag back to standard format
Modified Docker startup scripts to accept Environment variables if available, or use the usual defaults if not.
Updated readme to reflect changes in allowing environment variables.
Updated readme to correctly reflect the default ports that NiFi will run on.

This closes #2439

Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
9 months agoNIFI-4747 - Removed directory existence check in GetHDFS
Pierre Villard [Tue, 9 Jan 2018 16:45:51 +0000 (17:45 +0100)] 
NIFI-4747 - Removed directory existence check in GetHDFS

This closes #2391

Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
9 months agoNIFI-4823 Made pretty printing configurable in GetMongo.
Mike Thomsen [Mon, 29 Jan 2018 11:44:14 +0000 (06:44 -0500)] 
NIFI-4823 Made pretty printing configurable in GetMongo.

This closes #2441

Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
9 months agoNIFI-4686 MockFlowFile now includes penalized status when copying. This closes #2438.
jlwt90 [Sun, 28 Jan 2018 02:37:50 +0000 (11:37 +0900)] 
NIFI-4686 MockFlowFile now includes penalized status when copying. This closes #2438.

Signed-off-by: Mark Payne <markap14@hotmail.com>
9 months agoNIFI-4786 Allow Expression Evaluation to Kinesis/Firehose Stream Name
dorian.bugeja [Wed, 17 Jan 2018 10:22:39 +0000 (11:22 +0100)] 
NIFI-4786 Allow Expression Evaluation to Kinesis/Firehose Stream Name

Signed-off-by: James Wing <jvwing@gmail.com>
This closes #2409.

9 months agoNIFI-4801 Fixes Swagger spec for uploadTemplate. This closes #2428
Kevin Doran [Tue, 23 Jan 2018 14:59:58 +0000 (09:59 -0500)] 
NIFI-4801 Fixes Swagger spec for uploadTemplate. This closes #2428

9 months agoNIFI-4790 - support HTTPS Proxy in InvokeHTTP
Marco Gaido [Fri, 19 Jan 2018 10:02:56 +0000 (11:02 +0100)] 
NIFI-4790 - support HTTPS Proxy in InvokeHTTP

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2426.

9 months agoNIFI-4773: Moved DB Fetch processors' connection code from setup to onTrigger
Matthew Burgess [Mon, 22 Jan 2018 18:42:35 +0000 (13:42 -0500)] 
NIFI-4773: Moved DB Fetch processors' connection code from setup to onTrigger

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2422.

9 months agoNIFI-4803: Ensure that we set the FileCountAttribute of BinManager to the proper...
Mark Payne [Mon, 22 Jan 2018 19:50:19 +0000 (14:50 -0500)] 
NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to the proper value, regardless of the merge strategy

This closes #2423.

Signed-off-by: Bryan Bende <bbende@apache.org>
9 months agoNIFI-4424 Added functionality to allow NiFi to run in "embedded" mode for eventual...
Peter G. Horvath [Sun, 5 Nov 2017 11:25:15 +0000 (12:25 +0100)] 
NIFI-4424 Added functionality to allow NiFi to run in "embedded" mode for eventual integration test access.

This closes #2251.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
9 months agoNIFI-4799:
Matt Gilman [Mon, 22 Jan 2018 17:09:15 +0000 (12:09 -0500)] 
NIFI-4799:
- Ensure variable registry update request is marked completed.
This closes #2421.

Signed-off-by: Mark Payne <markap14@hotmail.com>
9 months agoNIFI-4744 Detect incorrect authorizers config
Kevin Doran [Fri, 19 Jan 2018 16:50:36 +0000 (11:50 -0500)] 
NIFI-4744 Detect incorrect authorizers config

Adds stricter checks in AuthorizerFactoryBean for unique ids within
a given type of provider and requires unique providers in composite
and composite-configurable user group providers. Failed validation
checks cause startup to fail. Adds test cases for these new rules.

This closes #2419.

Signed-off-by: Bryan Bende <bbende@apache.org>
10 months agoNIFI-4791 Made external links open in new windows and fixed some broken links
Andrew Lim [Thu, 18 Jan 2018 21:36:54 +0000 (16:36 -0500)] 
NIFI-4791 Made external links open in new windows and fixed some broken links

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2413.

10 months agoNIFI-4795:
sbouchex [Thu, 18 Jan 2018 22:49:55 +0000 (23:49 +0100)] 
NIFI-4795:
Fixed allowed values
This closes #2414

10 months agoNIFI-4761 Fixing test case regression.
Matt Gilman [Fri, 19 Jan 2018 17:42:47 +0000 (12:42 -0500)] 
NIFI-4761 Fixing test case regression.

This closes #2418.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
10 months agoNIFI-1604 - PutSolrContentStream - Evaluate stream path against flow file
Pierre Villard [Wed, 16 Aug 2017 15:12:16 +0000 (17:12 +0200)] 
NIFI-1604 - PutSolrContentStream - Evaluate stream path against flow file

This closes #2092.

Signed-off-by: Bryan Bende <bbende@apache.org>
10 months agoNIFI-4766:
Matt Gilman [Wed, 17 Jan 2018 20:55:40 +0000 (15:55 -0500)] 
NIFI-4766:
- Documenting nifi.web.proxy.host property.
- Updating proxy confiugration.
NIFI-4618:
- Documenting nifi.web.proxy.context.path property.
- Addressing PR feedback.
- This closes #2410

10 months agoNIFI-4761
Andy LoPresto [Wed, 17 Jan 2018 19:36:00 +0000 (11:36 -0800)] 
NIFI-4761
Added HostHeaderHandler regression unit tests before adding new functionality.
Added logic for parsing nifi.web.proxy.host property.
Added default property in nifi.properties and pom.xml.
Added logic for IPv6 and custom default hostnames.
Improved error messaging.
Added HostHeaderHandler unit tests.
Disabled HostHeaderSanitizationCustomizer in HTTP mode.
Fixed HTML escaping in error message.
Improved error message.
Added failing unit test for parsing custom hostnames.
Fixed custom hostname parsing.
Fixed unit tests.
Added TODO for IPv6 custom hostname parsing and unit test.
Added IPv6 custom hostname parsing and unit tests.
Fixed checkstyle issues.
Removed empty element in host list when no value defined.
Improved error message formatting.
Added unit tests.
Removed HostHeaderSanitizationCustomizer.
Removed InvalidPropertiesFormatException from NiFiProperties.
Removed InvalidPropertiesFormatException from HostHeaderHandler.
This closes #2415

10 months agoNIFI-4770 - ListAzureBlobStorage now properly writes azure.container flowfile attribute
zenfenaan [Fri, 12 Jan 2018 17:15:13 +0000 (22:45 +0530)] 
NIFI-4770 - ListAzureBlobStorage now properly writes azure.container flowfile attribute

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2402.

10 months agoNIFI-4759 - Fixed a bug that left a hard-coded reference to _id in as the update...
Mike Thomsen [Wed, 10 Jan 2018 13:35:09 +0000 (08:35 -0500)] 
NIFI-4759 - Fixed a bug that left a hard-coded reference to _id in as the update key for MongoDB upserts.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2401.

10 months agoNIFI-4748 - Add endpoint override to Kinesis processors
Joey Frazee [Thu, 11 Jan 2018 21:00:42 +0000 (14:00 -0700)] 
NIFI-4748 - Add endpoint override to Kinesis processors

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2399.

10 months agoNIFI-4784: Fixed runStatus allowedValues. This closes #2407
sbouchex [Tue, 16 Jan 2018 21:23:51 +0000 (22:23 +0100)] 
NIFI-4784: Fixed runStatus allowedValues. This closes #2407

10 months agoNIFI-4781: Updated ClientAuthenticationMethod as optional field response from OpenID...
Senthilannaswamy [Tue, 10 Oct 2017 07:35:48 +0000 (13:05 +0530)] 
NIFI-4781: Updated ClientAuthenticationMethod as optional field response from OpenID provider. This closes #2213

10 months agoNIFI-4756: Updated PublishKafkaRecord processors to include attributes generated...
Mark Payne [Wed, 10 Jan 2018 14:04:52 +0000 (09:04 -0500)] 
NIFI-4756: Updated PublishKafkaRecord processors to include attributes generated from schema write strategy into the message headers when appropriate

This closes #2396.

Signed-off-by: Bryan Bende <bbende@apache.org>
10 months agoNIFI-4782: Allow the value of a Required Property to be moved when changing version...
Mark Payne [Tue, 16 Jan 2018 17:36:52 +0000 (12:36 -0500)] 
NIFI-4782: Allow the value of a Required Property to be moved when changing version of a flow or reverting a flow

This closes #2406.

Signed-off-by: Bryan Bende <bbende@apache.org>
10 months agoNIFI-4428: This closes #2310. Updated POM versions to current master branch
Matthew Burgess [Tue, 16 Jan 2018 14:18:08 +0000 (09:18 -0500)] 
NIFI-4428: This closes #2310. Updated POM versions to current master branch

Signed-off-by: joewitt <joewitt@apache.org>
10 months agoNIFI-4428: Replaced JSON input with RecordReader
Matthew Burgess [Wed, 22 Nov 2017 16:36:30 +0000 (11:36 -0500)] 
NIFI-4428: Replaced JSON input with RecordReader
using per-record flowfiles
NIFI-4428: Added initial L&N, fixed dependency hierarchy

NIFI-4428: Exposed additional ZK properties

NIFI-4428: More work to reduce dependencies in Druid artifacts

NIFI-4428: Removed inaccessible attribute

NIFI-4428: Fixed typo in property description

NIFI-4428: Updates to Druid NAR L&N

NIFI-4428: Updated LICENSE to refer to ICU License for icu4j

10 months agoNIFI-4428: - Implement PutDruid Processor and Controller
vvaks [Wed, 27 Sep 2017 17:23:23 +0000 (13:23 -0400)] 
NIFI-4428: - Implement PutDruid Processor and Controller

update

added provenance report

added parameters for batch control

WIP

10 months agoNIFI-4763: Ignore differences in components' Bundle Version when comparing a local...
Mark Payne [Wed, 10 Jan 2018 15:01:27 +0000 (10:01 -0500)] 
NIFI-4763: Ignore differences in components' Bundle Version when comparing a local flow to a flow in the registry

This closes #2393.

Signed-off-by: Bryan Bende <bbende@apache.org>
10 months agoNIFI-4769: Use FlowFile for EL at Fetch and PutAzureBlobStorage
Koji Kawamura [Fri, 12 Jan 2018 02:03:05 +0000 (11:03 +0900)] 
NIFI-4769: Use FlowFile for EL at Fetch and PutAzureBlobStorage

This commit add back the existing capability for those Processors to use
incoming FlowFile attributes to compute account name and account key,
which had been removed by NIFI-4004.

Also, the same capability is added for SAS token.

This closes #2400.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
10 months agoNIFI-4768: Add exclusion filters to S2SProvenanceReportingTask
Matthew Burgess [Thu, 11 Jan 2018 20:00:03 +0000 (15:00 -0500)] 
NIFI-4768: Add exclusion filters to S2SProvenanceReportingTask

NIFI-4768: Updated exclusion logic per review comments

This closes #2397.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
10 months agoNIFI-4751 updated docker version
joewitt [Fri, 12 Jan 2018 20:19:15 +0000 (15:19 -0500)] 
NIFI-4751 updated docker version

10 months agoNIFI-4751 changed to next minor release version snapshot
joewitt [Fri, 12 Jan 2018 20:15:32 +0000 (15:15 -0500)] 
NIFI-4751 changed to next minor release version snapshot

10 months agoMerge branch 'NIFI-4751-RC1'
joewitt [Fri, 12 Jan 2018 20:00:32 +0000 (15:00 -0500)] 
Merge branch 'NIFI-4751-RC1'

10 months agoNIFI-4767 - Fixed issues with RecordPath using the wrong field name for arrays and...
Mark Payne [Thu, 11 Jan 2018 20:04:14 +0000 (15:04 -0500)] 
NIFI-4767 - Fixed issues with RecordPath using the wrong field name for arrays and maps. Also addressed issue where Avro Reader was returning a Record object when it should return a Map

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2398.

10 months agoNIFI-4441: patch avro maps in union types. This closes #2207.
Patrice Freydiere [Wed, 11 Oct 2017 20:17:15 +0000 (22:17 +0200)] 
NIFI-4441: patch avro maps in union types. This closes #2207.

Signed-off-by: Mark Payne <markap14@hotmail.com>
10 months agoNIFI-4764: Add tooltips to status bar icons
Marco Gaido [Wed, 10 Jan 2018 16:00:55 +0000 (17:00 +0100)] 
NIFI-4764: Add tooltips to status bar icons
add tooltip to process groups
cleanup
This closes #2394

10 months agoNIFI-3648 removed message copying when not in debug mode. This closes #1637.
Mike Moser [Thu, 30 Mar 2017 14:34:26 +0000 (14:34 +0000)] 
NIFI-3648 removed message copying when not in debug mode. This closes #1637.

Signed-off-by: Mark Payne <markap14@hotmail.com>
10 months agoNIFI-4754:
Matt Gilman [Wed, 10 Jan 2018 20:30:05 +0000 (15:30 -0500)] 
NIFI-4754:
- Only showing the import link in the new Process Group dialog when initiating from the toolbar.
This closes #2395.

Signed-off-by: Mark Payne <markap14@hotmail.com>
10 months agoNIFI-4752: Addressed issue with some event types having potentially the wrong FlowFil...
Mark Payne [Tue, 9 Jan 2018 18:34:41 +0000 (13:34 -0500)] 
NIFI-4752: Addressed issue with some event types having potentially the wrong FlowFile UUID listed (could have child UUID when it's supposed to have parent flowfile UUID). In testing fix, also found an issue with Search threads not being daemon and Re-Index threads not propertly being shutdown so addressed those as well.

This closes #2390.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
10 months agoNIFI-4751-RC1 prepare for next development iteration NIFI-4751-RC1
joewitt [Tue, 9 Jan 2018 06:39:49 +0000 (23:39 -0700)] 
NIFI-4751-RC1 prepare for next development iteration

10 months agoNIFI-4751-RC1 prepare release nifi-1.5.0-RC1 nifi-1.5.0-RC1 rel/nifi-1.5.0
joewitt [Tue, 9 Jan 2018 06:39:32 +0000 (23:39 -0700)] 
NIFI-4751-RC1 prepare release nifi-1.5.0-RC1

10 months agoNIFI-4515 - This closes #2224. Enabled EL on Kerberos properties for Kafka 0.10 ...
Pierre Villard [Mon, 23 Oct 2017 12:34:18 +0000 (14:34 +0200)] 
NIFI-4515 - This closes #2224. Enabled EL on Kerberos properties for Kafka 0.10 & 0.11 & 1.0 processors

Signed-off-by: joewitt <joewitt@apache.org>
10 months agoNIFI-4697 This closes #2381. clarified some PostHTTP documentation
Mike Moser [Fri, 5 Jan 2018 21:13:05 +0000 (21:13 +0000)] 
NIFI-4697 This closes #2381. clarified some PostHTTP documentation

Signed-off-by: joewitt <joewitt@apache.org>
10 months agoNIFI-4708 This closes #2389. Fixes encrypt-config log4j configuration
Kevin Doran [Tue, 9 Jan 2018 03:28:08 +0000 (22:28 -0500)] 
NIFI-4708 This closes #2389. Fixes encrypt-config log4j configuration

Signed-off-by: joewitt <joewitt@apache.org>
10 months agoNIFI-4751 correcting extranous NOTICE entries for MIT License and fixed the cassandra...
joewitt [Tue, 9 Jan 2018 04:30:11 +0000 (21:30 -0700)] 
NIFI-4751 correcting extranous NOTICE entries for MIT License and fixed the cassandra nar LandN

10 months agoNIFI-4708 This closes #2388. Correcting logic for determining decryption key in NiFiR...
Bryan Bende [Tue, 9 Jan 2018 02:44:04 +0000 (21:44 -0500)] 
NIFI-4708 This closes #2388. Correcting logic for determining decryption key in NiFiRegistryMode.groovy

Signed-off-by: joewitt <joewitt@apache.org>
10 months agoNIFI-4749: This closes #2386. Pass the record reader's schema along to the ResultSetR...
Mark Payne [Mon, 8 Jan 2018 21:18:34 +0000 (16:18 -0500)] 
NIFI-4749: This closes #2386. Pass the record reader's schema along to the ResultSetRecordSet so that it is able to resolve the schema for Record fields

Signed-off-by: joewitt <joewitt@apache.org>
10 months agoNIFI-4727 Added CountText processor and unit test.
Andy LoPresto [Tue, 2 Jan 2018 19:47:33 +0000 (14:47 -0500)] 
NIFI-4727 Added CountText processor and unit test.

This closes #2371.

Signed-off-by: Kevin Doran <kdoran.apache@gmail.org>
10 months agoNIFI-4750 Ensuring preDestruction is called on authorizer and appropriate policy...
Bryan Bende [Mon, 8 Jan 2018 21:21:03 +0000 (16:21 -0500)] 
NIFI-4750 Ensuring preDestruction is called on authorizer and appropriate policy/user-group providers. This closes #2387

10 months agoNIFI-4436:
Matt Gilman [Mon, 8 Jan 2018 19:49:26 +0000 (14:49 -0500)] 
NIFI-4436:
- Only enforcing the unique name check when the registry has a different identifier.

This closes #2385.

Signed-off-by: Bryan Bende <bbende@apache.org>
10 months agoNIFI-4679 Added new content in User Guide related to Registry integration and replace...
Andrew Lim [Sat, 6 Jan 2018 05:32:55 +0000 (00:32 -0500)] 
NIFI-4679 Added new content in User Guide related to Registry integration and replaced screenshots and edited text for recent UI changes. This closes #2382

10 months agoNIFI-4739:
Matt Gilman [Thu, 4 Jan 2018 20:54:19 +0000 (15:54 -0500)] 
NIFI-4739:
- Ensuring the logout action is using a relative link.

This closes #2374

10 months agoNIFI-4708 Add Registry support to encrypt-config.
Kevin Doran [Sat, 30 Dec 2017 13:54:18 +0000 (08:54 -0500)] 
NIFI-4708 Add Registry support to encrypt-config.
Adds support for NiFI Registry config files to the encrypt-config tool
in NiFi Toolkit.
Also adds decryption capability to encrypt-config tool.

This closes #2376.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
10 months agoNIFI-4436: Ensure that on save, we assign a Versioned Component Identifier to inner...
Mark Payne [Thu, 4 Jan 2018 21:09:02 +0000 (16:09 -0500)] 
NIFI-4436: Ensure that on save, we assign a Versioned Component Identifier to inner process groups that are tracking to remote flows, if they don't have one. This would occur, for instance, if a Process Group was imported into an existing group (or copied/moved into it) and then the existing group was saved.

NIFI-4436: Fixed a bug that caused a flow not to successfully change version if a connection is added to an existing component and that component is running at time of version change

NIFI-4436: Fixed bug with ordering of controller services being enabled and disabled

NIFI-4436: Fixed bug that prevented local input and output ports from being stopped and started as needed

NIFI-4436: Fixed bugs around referencing controller services that are at a higher level than the versioned flow

NIFI-4436: Ensure that we clear components from FlowController's cache when removed and that they are added to cache when created.

NIFI-4436: Fixed error message coming back if component is invalid when trying to be restarted/re-enabled

NIFI-4436: Addressed issue with children of a removed process group not being considered 'affected components' and as a result not being stopped/disabled/restarted/re-enabled

This closes #2219.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Thu, 4 Jan 2018 01:57:37 +0000 (20:57 -0500)] 
NIFI-4436:
- Minor tweak to when the max height for the options list is calculated per PR comments.
- Resolving logic issue in two phase commit when updating variable registry.
- Fixing variable visibility.

10 months agoNIFI-4436: Fixed bug that caused a Process Group to be 'dirty' if a processor that...
Mark Payne [Wed, 3 Jan 2018 20:29:39 +0000 (15:29 -0500)] 
NIFI-4436: Fixed bug that caused a Process Group to be 'dirty' if a processor that was referencing a non-existent controller service is updated to reference an externally available controller service

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Wed, 3 Jan 2018 22:16:57 +0000 (17:16 -0500)] 
NIFI-4436:
- Addressing PR feedback.
- Addressing two phase commit logic issue when changing the flow version.

10 months agoNIFI-4436: Fixed bug that caused in Process Groups' names not to be fixed when revert...
Mark Payne [Wed, 3 Jan 2018 14:45:57 +0000 (09:45 -0500)] 
NIFI-4436: Fixed bug that caused in Process Groups' names not to be fixed when reverting changes

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Wed, 3 Jan 2018 15:46:04 +0000 (10:46 -0500)] 
NIFI-4436:
- Code clean up.
- Backing out incomplete fix for variable registry two phase commit to address in separate JIRA.

10 months agoNIFI-4436: Bug fixes
Mark Payne [Tue, 2 Jan 2018 18:09:09 +0000 (13:09 -0500)] 
NIFI-4436: Bug fixes

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Tue, 2 Jan 2018 15:26:52 +0000 (10:26 -0500)] 
NIFI-4436:
- Bumping registry version to 0.1.0.
- Addressing PR comments.

10 months agoNIFI-4436: Fixed bug that causes a deadlock when changing version of a PG. Before...
Mark Payne [Sat, 30 Dec 2017 19:16:26 +0000 (14:16 -0500)] 
NIFI-4436: Fixed bug that causes a deadlock when changing version of a PG. Before this patch, an update would obtain a write lock and then recurse downward through the child groups, obtaining write locks to update variable registries. At the same time, if a Processor is obtaining a Controller Service, it will obtain a Read Lock on the Process Group and then recurse upward through the ancestors, obtaining Read Lock. If the timing is right, we can have a group obtain a read lock, then try to obtain its parent's Read Lock. At the same time, an update to the group could hold the Write Lock on the Process Group and attempt to obtain a Write Lock on child (where the Processor lives), resulting in a deadlock.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Fri, 15 Dec 2017 14:49:13 +0000 (09:49 -0500)] 
NIFI-4436:
- Minor UX tweaks.
- Updating imports due to refactoring of Registry models.
- Fixing checkstyle issues.

10 months agoNIFI-4436: Bug fixes
Mark Payne [Wed, 13 Dec 2017 18:57:59 +0000 (13:57 -0500)] 
NIFI-4436: Bug fixes

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Tue, 12 Dec 2017 21:39:05 +0000 (16:39 -0500)] 
NIFI-4436:
- Updating buckets permissions based on new model.
- Adding check to ensure that flow name is non null before checking the length.
- Adding versioned flow state to the Process Group tab in the Summary table.
- Fixing issue with navigating to Controller Services from the local changes dialog.

10 months agoNIFI-4436: Bug fixes; ensure correct Exception types are thrown
Mark Payne [Mon, 11 Dec 2017 20:36:56 +0000 (15:36 -0500)] 
NIFI-4436: Bug fixes; ensure correct Exception types are thrown

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Mon, 11 Dec 2017 20:14:44 +0000 (15:14 -0500)] 
NIFI-4436:
- Code clean up.
- Improved error handling.
- Minor UX improvements.
- Always showing Process Group state to complement the aggregation counts.
- Adding the Process Group state to the top status bar.

10 months agoNIFI-4436: Removed isCurrent, isModified from VersionControlInformation and associate...
Mark Payne [Tue, 5 Dec 2017 21:18:16 +0000 (16:18 -0500)] 
NIFI-4436: Removed isCurrent, isModified from VersionControlInformation and associated DTO. Bug fixes & code refactoring

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Wed, 6 Dec 2017 15:47:28 +0000 (10:47 -0500)] 
NIFI-4436:
- Code clean up.
- Improved error handling.
- Minor UX improvements.
- Adding message to indicate that variables do not support sensitive values.
- Preventing a user from changing the flow version to the current version.
- Only presenting buckets a user has appropriate permissions to.
- Adding basic auditing to the version control actions.

10 months agoNIFI-4436: Bug fix to ensure that RPG's ports are not removed until after connections...
Mark Payne [Fri, 1 Dec 2017 21:31:22 +0000 (16:31 -0500)] 
NIFI-4436: Bug fix to ensure that RPG's ports are not removed until after connections are established to the ports; ensure that if a registry's name is changed that it is updated immediately in VersionControlInformation objects

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Mon, 4 Dec 2017 21:11:11 +0000 (16:11 -0500)] 
NIFI-4436:
- Updating front end to use version control state/status.
- Fixing copy/paste issue during revert local changes.
- Code clean up in the breadcrumbs.
- Update VersionsResource authorization and two phase commit object usage.

10 months agoNIFI-4436: More intelligently flag a ProcessGroup to indicate whether or not it has...
Mark Payne [Tue, 28 Nov 2017 17:33:00 +0000 (12:33 -0500)] 
NIFI-4436: More intelligently flag a ProcessGroup to indicate whether or not it has any local modifications compared to Versioned Flow - Bug fixes - Updated to include status of a Versioned Process Group to include VersionedFlowState and explanation

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Tue, 28 Nov 2017 21:22:11 +0000 (16:22 -0500)] 
NIFI-4436:
- Addressing miscellaneous minor UX issues.
- Updating comments UX for all components.
- Updating the styling of PG and RPG to be more consistent.
- Adding the icons for nested versioned process groups.
- Calculating the number/states of nested versioned process groups.

10 months agoNIFI-4436, NIFI-4461: When copying and pasting an RPG, ensure that we copy Batch...
Mark Payne [Wed, 22 Nov 2017 14:55:30 +0000 (09:55 -0500)] 
NIFI-4436, NIFI-4461: When copying and pasting an RPG, ensure that we copy Batch Settings for each Port. Bug fixes. Now works in clustered mode.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Mon, 27 Nov 2017 21:10:39 +0000 (16:10 -0500)] 
NIFI-4436:
- Fixing default border radius.
- Code clean up.
- Ensuring component visibility is updated after updating/reverting.
- Fixing sort on component name in local changes dialog
NIFI-4526:
- Added front end controls for updating RPG target URL.

10 months agoNIFI-4436: Bug fixes - Checkpoint before allowing multiple Process Groups with same...
Mark Payne [Fri, 17 Nov 2017 16:02:33 +0000 (11:02 -0500)] 
NIFI-4436: Bug fixes - Checkpoint before allowing multiple Process Groups with same Versioned Component ID and same parent - Ensure that if flow update is cancelled while processors are being stopped/services disabled that we stop waiting for that to occur. Also ensure that if we fail to update flow that we re-enable/restart the processors and services - Updated verbiage to use a ConciseEvolvingDifferentDescriptor when getting local modifications for a versioned flow - Do not allow outer process group to be saved to flow registry or have local modifications reverted if it has a descendant process group that is under version control and is dirty. Fixed bug where ComponentDifferenceDTO was populated with wrong component id and group id

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Thu, 16 Nov 2017 19:41:41 +0000 (14:41 -0500)] 
NIFI-4436:
- Clearing bucket/flow/versions when changing the selected registry/bucket.
- Using the versioned flow to get the group name when importing.
- Adding menu items for viewing local changes.
- Showing local changes during revert request.

10 months agoNIFI-4436: Integrate with actual Flow Registry via REST Client - Store Bucket Name...
Mark Payne [Sat, 4 Nov 2017 18:19:49 +0000 (14:19 -0400)] 
NIFI-4436: Integrate with actual Flow Registry via REST Client - Store Bucket Name, Flow Name, Flow Description for VersionControlInformation - Added endpoint for determining local modifications to a process group - Updated authorizations required for version control endpoints - Add state and percent complete fields ot VersionedFlowUpdateRequestDTO - If a variable exists in a parent process group, do not include it in imported/updated process group when interacting with flow registry - Code cleanup, documentation; bug fixes - Ensure that we are passing NiFiUser to the flow registry client when appropriate - Updated to work against new version of flow registry client; deleted file-based flow registry client

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
10 months agoNIFI-4436:
Matt Gilman [Fri, 10 Nov 2017 20:58:59 +0000 (15:58 -0500)] 
NIFI-4436:
- Adding the version number to the start version control, commit, and change version dialog.
- Showing a loading item in the combo's while querying for the registries, buckets, and flows.
- Adding tooltips to display version control information on the canvas.
- Adding progress bar dialogs for changing version and reverting local changes.
- Updating canvas and breadcrumb according to the version control state.
- Updating to use registry name, bucket name, and flow name where appropriate.

10 months agoNIFI-4436: Integrate with actual Flow Registry via REST Client - Store Bucket Name...
Mark Payne [Sat, 4 Nov 2017 18:19:49 +0000 (14:19 -0400)] 
NIFI-4436: Integrate with actual Flow Registry via REST Client - Store Bucket Name, Flow Name, Flow Description for VersionControlInformation - Added endpoint for determining local modifications to a process group - Updated authorizations required for version control endpoints - Add state and percent complete fields ot VersionedFlowUpdateRequestDTO

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>