Derive More Benefit from Your Data with Digital Q.DataStreamer
May 16, 2022
Digital transformation results in accumulation of large amounts of data that can help to take more effective managerial decisions and improve business processes. To turn data into a driver of new business opportunities and an important company asset, it is essential to automate their collection, processing and transformation into data arrays that can be used to solve business tasks.
Digital Q.DataStreamer by Diasoft is designed to collect, process and download data into a data warehouse or other ETL systems and ensure integration of different IT systems. Automation of complex processes for transformation of data from multiple data streams helps to save both time and efforts and keep control over all received data. The solution is part of the technological platform Digital Q.DataFlows.
Digital Q.DataStreamer supports both streaming data processing and batch data processing, depending on a specific business task or peculiarities of data sources and recipients. It allows using different data integration options: from simple replication to complex preparation and transformation of data with the help of the drag-and-drop interface.
Digital Q.DataStreamer components:
- Data Stream Designer (Designer of ETL Processes) – A low-code tool for quick and simple design of ETL processes and setup of parameters and events.
- Tools for execution of ETL processes.
- Monitoring tools designed for visual monitoring of ETL processes and detection of possible errors.
- Scheduler used to set up schedules of ETL processes.
In Digital Q.DataStreamer, ETL processes can be started automatically in accordance with the pre-set schedule or they can be initiated with the help of HTTP requests via the REST API, from the product’s UI, upon an Apache Kafka event
Digital Q.DataStreamer is compatible with different types of database: relational databases (PostgreSQL, Oracle, MS SQL, etc.), specialized analytical databases (ClickHouse), and document oriented databases (ElasticSearch).