Skip to content

Instantly share code, notes, and snippets.

@Oxalisviolacea
Oxalisviolacea / highwing_challenge.md
Last active September 26, 2022 18:42
Highwing Data Engineering Code Challenge

Introduction

Thank you for the opportunity to complete this challenge! I recently implemented a similar system to ingest our carrier's policies, contacts, and enrollments, so my solution is based on things I learned during and after that process. My code snippets are pretty short and given more time I'd love to show you more error handling, logging, and testing. This document includes my solution, next steps, and answers to the assessment questions.

Assumptions:

  • Every broker sends policy, carrier, and client data
  • Every broker sends all files at the same time
  • Every broker sends their data in force
  • Every broker includes the carrier and client foreign key in the policies file

Ingestion Process

Graphql

What is it
According to their website, it's a query language for APIs and a runtime for fulfilling those queries with your existing data.

How can it be beneficial over RESTful CRUD endpoints
I watched this video and read this article

  • Write queries using an object structure, opposed to a text string.
  • Graphql has one endpoint, opposed to many.
  • It has schema (integer, string, etc.), opposed to JSON.
  • There is no versioning, you just add new fields.

Career Journal

Module 0

  1. When you've worked towards a goal in the past, what systems or tools have been helpful for you in accomplishing that goal? How could you adapt those same systems/tools to use while at Turing?**
  • Teaching others the material helps solidify concepts
    • Adaptation: respond to comments in the slack channel when classmates ask for help
  • Pre and post learning
    • Adaptation: read the notes before class with questions ready; if they're not answered, follow-up after class
  • Creating a study space that's free of distractions
  • Adaptation: keep my phone and laptop on 'do not disturb' while I'm in class