Automating BLS Labor Data Integration with FME and ArcGIS Online
Background
The Office of Performance Management supports data-driven decision-making by maintaining public-facing datasets and analytical tools. Many external datasets provide valuable context for understanding economic, demographic, and operational trends, but manually downloading and updating these datasets can be time-consuming and introduce opportunities for errors.
The Bureau of Labor Statistics (BLS) publishes regularly updated labor market data that can provide valuable insight into employment trends and economic conditions. However, incorporating this information into internal and public-facing applications required a repeatable process for extracting, transforming, and publishing updated data.
Project Overview
This project developed an automated Extract, Transform, Load (ETL) workflow using Safe Software’s Feature Manipulation Engine (FME) to retrieve BLS data and publish updated results to ArcGIS Online.
The goal was to create a monthly process that could operate with minimal manual intervention while ensuring that users always had access to the most current available information.
Challenges
Before automation, updating external datasets required several manual steps:
Downloading updated files from the BLS website
Updating locally stored files
Updating hosted feature layers in ArcGIS Online
Confirming that applications and dashboards were using the latest information
This approach required staff time and created the potential for inconsistent updates or outdated information being displayed to users.
Solution
An FME workflow was developed to automate the complete data pipeline.
The workflow performs the following functions:
Data Extraction
Connects to BLS data sources and retrieves the latest available labor statistics.
Eliminates the need for manual downloads and file preparation.
Data Transformation
Processes the incoming data into a standardized format.
Performs necessary field mapping, formatting, and data preparation to ensure compatibility with ArcGIS Online hosted layers.
GIS Integration
Publishes updated data directly to ArcGIS Online.
Maintains the connection between the source data and public-facing GIS resources.
Workflow Automation
Configures the process to run automatically on a monthly schedule.
Reduces manual maintenance requirements while improving consistency and reliability.
Technical Implementation
The solution utilized:
Safe Software FME for ETL workflow development
BLS data services as the authoritative source
ArcGIS Online hosted feature layers for data publication
Scheduled automation tools to execute recurring updates
The workflow was designed with repeatability and maintainability in mind, allowing future modifications as data sources, reporting requirements, or publication needs change.
Results
The automated pipeline improved the process for maintaining economic data resources by:
Reducing manual data update requirements
Improving data consistency and reliability
Creating a repeatable process for integrating external datasets
Allowing staff to focus more time on analysis rather than routine data maintenance
The project demonstrated how GIS automation and ETL technologies can be combined to create sustainable data workflows that support both internal analysis and public transparency.
Skills Demonstrated
ETL workflow development
FME automation
Data integration
ArcGIS Online administration
GIS data publishing
Workflow scheduling
Data quality management
Public-sector data management