Overview:
Introducing an enhanced Batch Processing capability to Xponent with a new Connector and Listener. This new feature streamlines data integration, enabling faster time to market with batch data. Users can quickly ingest data into their workflows, making it immediately actionable within their customer journeys.
Batch Listener enables the seamless integration of data from batch sources into graphs, which provides enhanced data accessibility. When combined with the Batch Listener and Connector, the following capabilities can be leveraged:
- Connect to AWS S3 and efficiently trigger graphs with a defined frequency.
- Process files in various File Formats, including Parquet, Delimited (CSV, TSV, etc.), and JSON, ensuring compatibility with diverse data sources.
- Effectively handle encrypted and/or compressed files, ensuring data security and integrity throughout the processing pipeline.
- Configure the interval at which the source location is monitored for new data, allowing for precise and timely updates. The minimum interval for monitoring is set at 10 minutes, enabling efficient data synchronization.
With Batch Listener and Connector, you can harness the power of batch data sources and integrate them into your graphs, empowering you to unlock valuable insights and drive data-informed decision-making.
Bug Fixes
KIT-10507 – Fixed the Project Version creation issue, which fails if Project does not contain Kafka connector(s).