Data Warehouse Architect

Core Objective:

Design and implement a robust, scalable, and high-performance data warehouse using the Microsoft Fabric environment. This architecture will serve as the backbone for near real-time analytics and reporting for airport curbside management solutions.

Responsibilities:

1. Data Warehouse Design

• Platform Expertise:

• Use Microsoft Fabric (hosted on Azure) to design a relational data warehouse.

• Leverage Azure tools for ETL processes, ensuring smooth integration of structured and unstructured data.

• Design Constraints:

• Create a system that supports real-time data ingestion and querying with refresh intervals of 1-10 minutes.

• Optimize performance for up to 20 simultaneous users querying the system.

2. Data Integration

• Data Sources:

• Integrate 14 APIs providing data from:

• Visual analytics (vehicle count and categorization by cameras).

• LPR cameras (license plate recognition with entry/exit timestamps).

• Traffic systems like Skidata (garage access, vehicle tracking).

• Employee, Uber/Lyft, and VIP vehicle data.

• Valet and reservation systems.

• Public data, such as flight schedules and weather patterns.

• ETL Development:

• Design extract, transform, and load (ETL) processes to handle high-frequency data inputs.

• Ensure the ability to process terabytes of data annually with future scalability in mind

3. Reporting and Visualization

• Prepare the warehouse to feed Power BI dashboards for real-time reporting on:

• Monthly and daily traffic patterns (vehicle types, categories like taxi, limo, personal).

• Occupancy and curbside activity levels.

• Historical trends for predictive analysis (to be expanded in Phase 2).

• Ensure that dashboards refresh dynamically based on real-time updates.

4. Performance and Scalability

• Design for high availability and redundancy, ensuring no data loss or significant latency.

• Implement indexing, partitioning, and caching strategies to handle queries on datasets spanning weeks or months while maintaining quick response times.

5. Security and Compliance

• Ensure compliance with security protocols, especially around sensitive data like license plates.

• Design access controls for multi-user environments, ensuring data integrity.

6. Future Readiness

• Design with a data lake integration strategy for Phase 2, allowing unstructured data and advanced analytics.

• Lay the foundation for AI-powered insights and correlations between data sources (e.g., flight patterns affecting curbside activity).

Technical Expertise

1. Microsoft Ecosystem:

• Deep knowledge of Microsoft Fabric tools and Azure cloud services.

• Proficient in SQL Server and relational database design.

• Familiarity with Power BI for report integration.

2. ETL and Data Modeling:

• Strong experience in ETL tools and processes for large-scale, high-frequency data ingestion.

• Expertise in designing efficient, scalable data models tailored to real-time operations.

3. Data Analytics:

• Experience with visual analytics tools and API-driven data ingestion.

• Ability to integrate diverse data types (structured/unstructured) into a unified system.

Experience

• 5-10+ years in data architecture and database design, preferably with large-scale projects (e.g., airports, logistics, or high-traffic systems).

• Demonstrated ability to manage real-time systems with high performance and multi-user access.

• Familiarity with visual analytics platforms (e.g., Eagle Eye) and real-time categorization systems.

Soft Skills

• Strong communication skills to collaborate with technical and non-technical teams.

• Ability to document processes and articulate designs for stakeholder understanding.

• Problem-solving mindset to adapt solutions for high-complexity environments.

Engagement Details

• Project Timeline: Estimated 4–6 months for the design and deployment phase, with possible extensions for optimization and ongoing support.

• Location: Open to local or global resources, but time efficiency and availability must be prioritized.

• Scope of Work: Focused on designing and delivering the data warehouse architecture (not API development).