Incoming New Grad Software Engineer
Created a mechanism for the Job Ingestion team to categorize source failures and decrease the time needed for engineers to fix these failures. Added a runtime configuration to a Scrapy spider to get scrape details for sources. Wrote a Python script to run a spider and retrieve scrape details for sources. Created a Hadoop project using Scala and Spark SQL to categorize source failures. Analyzed results for accuracy to verify the categories and to see how this project can be expanded in the future to help fix failed sources.