Ingesting Daily Transaction Data
Mission Statement
Our client required a process to ingest daily transaction data from files sent by a partner company. Although the task was straightforward, it was repetitive, needed to be done every day, and was susceptible to human error.
Tools Used
Alteryx Server
Alteryx Designer
Impact
Efficiency: The automation eliminates the need for manual data extraction, saving time and reducing errors.
Data Quality: The structured workflow reduces the likelihood of inconsistencies in the retrieved data.
Detailed Solution
We developed an automated solution using Alteryx to handle the daily processing of CSV files. The client received four different types of CSV files, each with a slightly different format. To address this, we created four distinct workflows and macros, each tailored to process a specific file type.
The process begins with the Directory tool in Alteryx, which reads all the files in the folder and routes them to the correct macro based on their file type. Each macro then prepares the data, reformatting it to be compatible with the client’s system before loading it into their database.
To avoid reprocessing the same files, a final step was added where a run command script archives all the files after processing. This ensures that only new files from the current day are processed in future runs.
The entire process is now scheduled on the client’s Alteryx Server, making it fully automated, saving the client time each day.