Improved metadata concatenation

Assignee

Pavel Salamon

Reporter

Sprint

Description

New components often need to propagate concatenation of several different metadata. The concatenation algorithm should be in shared utility method.

Currently, CrossJoin and SalesforceBulkWriter use the concatenation method, but the result is not robust enough (fixlen metadata are not handled well, delimiters are not normalized etc.).

The result of new concatenation:

  • delimited metadata with field delimiter set (delimiter taken from first input metadata)

  • result record properties same as record properties from first input metadata

  • result field properties same as input fields (except size and delimiter)

  • no size attribute on any field

  • no explicit delimiter on any field

This change is backwards incompatible because CrossJoin already propagates using the original algorithm (copy first input metada, then add a copy of all fields from following input metadata). However, the new algorithm only produces different metadata in situations where the original algorithm's result is unusable in practice, so it should be ok.

Steps to reproduce

None

Activity

Show:

Pavel Salamon August 2, 2016 at 4:02 PM
Edited

CrossJoin and SalesforceBulkWriter use the new algorithm.

If all input metadata are fixlen, result is fixlen and field sizes are kept.

Fixed

Details

Priority

Fix versions

QA Testing

JUnit test

QA Test Identification

Components

Created August 2, 2016 at 1:06 PM
Updated September 12, 2023 at 9:46 AM
Resolved August 2, 2016 at 4:02 PM